For the complete documentation index, see llms.txt. This page is also available as Markdown.

Extract data from a batch of PDFs

Outcome: a CSV with one row per paper and a citation for every cell, built from a folder of PDFs.

What you need

Key with files:write, documents:write, extractions:write; the PDFs; a field list.

Approach

Upload → ingest → define schema once → run → export. The schema is reusable; write it carefully and reuse it across projects.

Walkthrough

  1. POST /v1/files for each PDF (bounded concurrency; respect rate-limits)

  2. POST /v1/documents per file; wait for succeeded

  3. POST /v1/extraction-schemas — descriptions matter more than names (extraction-schema)

  4. POST /v1/extractions with all document_ids

  5. Poll or subscribe; read partial rows as they land

  6. POST /v1/extractions/{id}/export → CSV

Handling the hard parts

  • Unparseable PDFs. Scanned-without-OCR and encrypted files fail at ingest. Collect them and report; do not silently drop.

  • Partial failure. Per-row error means the run succeeds with holes. Reconcile explicitly.

  • Nulls are the honest answer. A null value means the extractor did not find it. Do not backfill with a guess.

  • Verification. Keep the per-cell citation in your output so a human can spot-check any row.

Cost

Extraction costs 2 credits per source at standard (5 at high_quality), independent of field count. Document ingest costs 1 credit per 10 pages.

For 5,000 papers averaging 12 pages: ingest ≈ 10,000 credits, extraction = 10,000 credits, exports free — about 20,000 credits, or roughly $200 at top-up rates. Runs are capped at 500 sources, so this is 10 runs; they queue against your concurrency limit (rate-limits).

Production checklist

Concurrency cap · resume from partial state · store extr_ IDs for audit · check content-licensing before redistributing extracted values

Related

extractions · extraction-schema · files · documents

Last updated