Two agents on two different providers turn a paper into a reviewed dataset.
A curator decides what lands.
Reading a shedding study takes a curator hours: find the paper, pull the measurements
out of tables and figures, reconcile the units, and check every number against the
source. The agents do the first pass of that work and hand a curator a set of
proposals to accept or reject — never a finished dataset to rubber-stamp.
The pipeline
How a study becomes a dataset
Numbered because it genuinely is a sequence — each step consumes the last one's output.
01
Fetch and freeze
From a PubMed identifier, the agent tries PMC first (XML, then HTML, then a
browser for JavaScript-heavy pages), falls back to the publisher's site, and
finally to metadata alone. Whatever it retrieves is written to an
evidence bundle and never fetched again, so every later step
reads exactly the same source.
Open-access sources only. The browser step handles scripted pages; it is not a
way around a paywall.
02
Extract
A preprocessing pass resolves the awkward parts — notably turning the
paper's dates into days since the reference event — and a second pass
emits the dataset against the schema as structured output. Digitized figure
measurements are read in from CSV alongside the text and tables.
Runs on OpenAI.
03
Review
A structural pass checks the dataset against the canonical schema with no model
involved at all. A paper-grounded pass then reads the frozen evidence and reports
what disagrees with it, sorted into three kinds: the data is wrong, the schema has
no way to express what the paper says, or something about the modeling deserves a
human's judgement.
Runs on Anthropic's Claude — a different provider from extraction, on purpose.
04
A curator decides
Every finding is a proposal. The curator accepts, edits, or rejects each one, and
accepted fixes are applied and the dataset re-reviewed. The review pass is tuned to
raise anything worth a second look rather than to stay quiet, which only works
because a person is the gate.
The phase-2 architecture. Extraction and review run today, along
with the schema and controlled vocabulary they read and the curated YAML they
produce. Literature discovery, the web API, and the benchmark and usability loops
around them are the target, not the current state.
Optional branch
When the paper needs a word the schema doesn't have
Sometimes a study reports a specimen or attribute the schema cannot express. The review
agent proposes the addition rather than discarding the value or forcing it into the
wrong field. If the curator accepts, the schema adopts it and the study is
re-extracted from the frozen evidence bundle under the new schema
— no refetching, so the only thing that changed is the schema.
The agents
Two agents, two providers
OpenAI
Extraction
Retrieves the paper, freezes the evidence, and produces the dataset: a
preprocessing pass with date arithmetic, then structured extraction validated
against the schema before anything is written.
Claude
Review
Checks the dataset against its source and the canonical schema, and returns
findings with mechanical patches where the fix is unambiguous, plus proposed
schema changes where it is not. Its structural checks run without an API key.
A third agent, literature discovery, is planned but not built. Studies are chosen by the
team today.
Guardrails
What keeps it honest
Every claim carries its quote
Findings must quote the frozen evidence verbatim. If the quote cannot be located
in that text, the finding is dropped rather than downgraded —
an invented quote is worse than a missed problem.
The reviewer is not the author
Extraction and review run on different providers. A model checking its own output
shares its own blind spots; this way agreement means something.
Nothing applies itself
There is no confidence threshold above which a change lands unseen. Every finding
is a proposal, and a curator accepts, edits, or rejects it.
The evidence does not move
Because the source is frozen at fetch time, a review months later reads exactly
what the extraction read, and a re-extraction changes only what the schema change
intended.
Where it stands
What the pipeline has produced
107
AI-curated studies
39
Hand-built benchmark
146
Studies in total
Curated studies over time, from the data repository's commit history. The
catalogue sat at 39 hand-built studies when the first
agent-extracted batch landed in Feb 2026; the shaded
stretch is everything since. Figure as of 2026-09-18 — the totals
above are live.
The 39 studies our biocurators extracted by hand from May
2024 are what the pipeline is measured against — the agents were built to
reproduce that standard, not to relax it. Every study curated since June 2025 has gone
through the agents and a human reviewer.
None of this is running yet. It is here so you can tell it apart from what is.
A discovery agent
The missing first step: given a topic and inclusion criteria, search the
literature and propose the studies worth extracting, instead of the team
choosing them by hand.
Machine-readable metadata
JSON-LD alongside the YAML, using established vocabularies, so datasets can be
discovered and consumed without scraping this site.
A web API
Query the catalogue over HTTP rather than downloading it, for pipelines that
want a slice rather than the whole repository.
AI-powered evidence synthesis
The same freeze, extract, and quote-grounded review, applied wherever the
numbers worth having are locked in published papers.
Get involved
Work with us
The agents themselves are internal. What they produce is not: every dataset, the
schema each one is validated against, and the Python package that reads them are
public, and the data is released under CC BY 4.0.
So the most useful thing you can send us is an extraction that is wrong.
A curator can correct the dataset, and the correction shapes how the agents are
prompted for the next study — which is the part of this you can affect without
access to the pipeline. A study you think belongs in here is the next most useful.
