Document extraction API for RAG, automation & workflows

Why document extraction is the bottleneck in modern apps

Every team building a document-heavy product hits the same wall in week one: the documents are PDFs, slide decks, and scanned images, and none of those formats are designed for code to read. A PDF tells your screen where each letter should appear; it does not say "this is a heading" or "this is a table." A PowerPoint file is a zip of XML files where text scatters across shapes, tables, and a separate notes pane. A scanned image is just pixels until you OCR it. Your application can't do anything useful until that mess becomes structured data.

Most teams start by gluing together three or four open-source libraries — one for PDFs, one for PPTX, one for OCR, one for table detection. It works on the demo file. It breaks on the second customer's documents. By month three the extraction layer has become its own product, full-time maintained by an engineer who'd rather be building the actual application.

A document extraction API skips that path. You hand it any document and get back the same shape of structured output — headings, paragraphs, tables, images, all typed and addressable. This guide covers what to look for in a modern extraction API, the three main use cases it should support (RAG, automation, document workflows), and the production patterns you'll want from day one.

What "good" document extraction looks like

Before evaluating any API, it helps to know what good output looks like. A useful extraction result has all of these:

• Real structure — separate pieces for headings, paragraphs, lists, tables, images, math, and diagrams. Not one giant block of text.
• The same shape every time — every document should return output in a predictable format, so your code doesn't break on the next file.
• Both JSON and Markdown from one call — JSON for code that needs to filter and inspect, Markdown for LLMs and chunkers that read it natively.
• Same schema across formats — your code shouldn't fork on whether the input was a PDF, PPTX, or HEIC photo from someone's phone.
• Tables stay as tables — real 2D data with separate headers and rows, ready to drop into pandas or a database.
• Works on scanned documents too — your code shouldn't have to ask "is this a real PDF or a scan?" before calling the API.
• Control over how long your data is kept — important if the documents contain personal, medical, or confidential information.

How OCRQueen does it

OCRQueen is an API that takes a document and returns both structured JSON and Markdown. The JSON shape is fixed: every piece of content comes back as one of a known set of types (heading, paragraph, list, table, image, chart, diagram). Tables stay as real 2D data — rows and columns, not text that looks like a table. The same schema applies whether the input is a PDF, a slide deck, or a photo someone took with their phone.

Two things that matter once you go live:

1. Prepaid billing. You add money to a wallet up front. Spending stops when the wallet is empty. There's no surprise invoice at the end of the month — important for batch ingestion or automation jobs where one runaway script can otherwise produce a five-figure bill. See /pricing.
2. You decide how long files are kept. For each request you can say "delete the source file after 1 hour" or "delete the result the moment you've sent it back to me." Useful for HIPAA, GDPR, finance, and anything else where data shouldn't sit around. Full details at /docs/data-retention.

Get a working setup in five minutes

Install the Python library:

pip install ocrqueen

Grab an API key from /dashboard/keys, then run this:

import os
from ocrqueen import OCRQueen

client = OCRQueen(api_key=os.environ["OCRQUEEN_API_KEY"])

# Open any document and send it off — PDF, PPTX, image, all the same
with open("annual-report.pdf", "rb") as f:
    job = client.extract.create(file=f)

# Wait for the extraction to finish
result = client.jobs.wait(job)

# You get both outputs from the same call — no second request.
document_json = result.result               # structured JSON
markdown      = result.result["markdown"]   # Markdown — easy for LLMs

print(markdown[:600])

What the output actually looks like

Each extraction returns a document object with two main parts: pages (the actual content) and extraction (info about the job — page count, mode used, how long it took). Each page contains an ordered list of blocks, and each block has a type:

{
  "source": { "page_count": 12, "mime_type": "application/pdf" },
  "extraction": { "profile": "standard", "duration_ms": 3140 },
  "markdown": "# Annual Report Q1\n\n## Summary\n\nStrong revenue growth...",
  "pages": [
    {
      "page": 1,
      "blocks": [
        { "type": "heading", "level": 1, "text": "Annual Report Q1" },
        { "type": "heading", "level": 2, "text": "Summary" },
        { "type": "paragraph", "text": "Strong revenue growth across all regions." },
        {
          "type": "table",
          "headers": ["Region", "Revenue", "Growth"],
          "rows": [
            ["North America", "$2.45M", "18%"],
            ["Europe",        "$1.78M", "12%"],
            ["Asia Pacific",  "$2.12M", "24%"]
          ]
        },
        {
          "type": "chart",
          "chart_type": "bar",
          "title": "Revenue by region",
          "categories": ["NA", "EU", "APAC"],
          "series": [{ "name": "Q1", "values": [2450000, 1780000, 2120000] }]
        }
      ]
    }
  ]
}

Don't wait for the result — use webhooks instead

Calling client.jobs.wait() works great for small scripts. But in a real application — especially automation or batch ingestion — you don't want your server sitting around waiting for each extraction to finish. The better pattern: submit the job, return right away, and let OCRQueen call you back when it's done. That callback is a webhook.

job = client.extract.create(
    file=pdf_bytes,
    options={
        "callback_url": "https://your-app.com/ocrqueen-webhook",
        "extraction_profile": "advanced",
    },
)
# Save job.id alongside your document record and move on.
# The webhook delivers the extraction when it's ready.

To make sure the incoming webhook is really from OCRQueen (and not someone faking it), we sign every delivery. The signature comes in the OCRQueen-Signature header — an HMAC-SHA256 of the raw request body using your endpoint's signing secret. Verify it before trusting the data:

from ocrqueen import verify_webhook

@app.post("/ocrqueen-webhook")
def handle_webhook(request):
    body = request.body  # raw bytes — do NOT re-serialize
    signature = request.headers.get("OCRQueen-Signature", "")
    if not verify_webhook(body, signature, secret=os.environ["WEBHOOK_SECRET"]):
        return Response(status_code=400)
    payload = json.loads(body)
    process_extraction(payload)
    return Response(status_code=204)

Safe retries with idempotency keys

Production code retries when things go wrong. The problem: a retry can accidentally trigger a second extraction — which means a second bill. The fix is an idempotency key: a label you attach to your request so that retrying with the same label returns the original job instead of running it again.

import hashlib

# Use a hash of the file as the key — same file means same key,
# means same result, every time.
content_hash = hashlib.sha256(pdf_bytes).hexdigest()
idempotency_key = f"doc-extract-{content_hash}"

job = client.extract.create(
    file=pdf_bytes,
    idempotency_key=idempotency_key,
)

Handling sensitive documents

For medical records, financial files, ID documents, contracts, or anything else with personal data, you probably don't want the file or its extracted content sitting on any server longer than it has to. Two retention settings handle this. Set both to zero and use a webhook to receive the result — the file is deleted right after extraction, and the result is deleted as soon as we send it to you:

job = client.extract.create(
    file=pdf_bytes,
    options={
        "retain_hours": 0,           # delete source file immediately
        "result_retain_hours": 0,    # delete result after webhook delivery
        "callback_url": "https://your-app.com/ocrqueen-webhook",
    },
)

If you need to delete a specific job on demand later (for example, a "right to be forgotten" request), there's a one-call purge:

client.jobs.purge(job.id)
# Source file is deleted. Result is deleted. Only a small record
# remains for billing purposes — no content, no document data.

Three common use cases

Use case 1: RAG — feeding documents into an AI assistant

RAG ("Retrieval-Augmented Generation") is a fancy name for a simple pattern: chop up your documents, store them, and look up the right piece when a user asks a question. The trick is chopping at sensible boundaries — headings work well. Markdown makes this easy because headings stay as headings, so a header-aware splitter produces semantically coherent chunks:

from langchain.text_splitter import MarkdownHeaderTextSplitter

def ingest_for_rag(pdf_bytes, vector_store):
    result = client.jobs.wait(client.extract.create(file=pdf_bytes)).result
    splitter = MarkdownHeaderTextSplitter(headers_to_split_on=[
        ("#", "h1"), ("##", "h2"), ("###", "h3"),
    ])
    chunks = splitter.split_text(result["markdown"])
    vector_store.add_documents(chunks)

Use case 2: Automation — extracting invoice and form data

For batch automation — invoice processing, form intake, contract review — use webhooks and idempotency keys based on file content. The structured JSON gives you typed tables and headings you can route directly into your business logic, no fuzzy regex required:

import pandas as pd

def parse_invoice_tables(extraction_result):
    for page in extraction_result["pages"]:
        for block in page["blocks"]:
            if block["type"] == "table" and "Total" in (block.get("headers") or []):
                yield pd.DataFrame(block["rows"], columns=block["headers"])

# Submit with webhook + idempotency for safe batch processing:
job = client.extract.create(
    file=invoice_bytes,
    idempotency_key=f"invoice-{hashlib.sha256(invoice_bytes).hexdigest()}",
    options={"callback_url": "https://your-app.com/invoice-webhook"},
)

Use case 3: Workflows — document tooling with mixed inputs

Real document workflows aren't only PDFs. Legal teams handle contracts in PDF and Word, sales teams send PowerPoint decks, customer success captures screenshots, field reps email iPhone photos. One API call handles all of them with the same code and the same output shape:

def extract_any(file_path: str):
    with open(file_path, "rb") as f:
        job = client.extract.create(file=f)
    return client.jobs.wait(job).result

# All of these work the same way:
extract_any("contract.pdf")
extract_any("training-deck.pptx")
extract_any("whiteboard-photo.heic")
extract_any("receipt-screenshot.png")

What to know before going to production

Frequently asked questions

What is a document extraction API?

A document extraction API takes a file (PDF, PowerPoint, image, scan) and returns its content as structured data — typically JSON with typed blocks for headings, paragraphs, tables, images, and other elements. It replaces the patchwork of open-source libraries most teams glue together when they need to read documents from code.