Versioning JSON Schemas for Email Extraction Without Breaking Your Pipeline

The first schema you send to POST /v1/parse is never the last one. A new sender appears, a field turns out to be fabricated more often than it is real, downstream code needs a key it did not need last month. Every one of those changes rewrites the contract between the email, the extraction, and your database, and if you change the contract without tracking it, you end up with a table where half the rows mean one thing and half mean another.

This guide covers how to version that contract yourself. MailFrame validates each extraction against the schema you send on the request and returns the typed result in the same HTTP response. Registered schemas do carry a server-side version counter that increments each time you update one, but neither the parse response nor the parse history ties a result to that counter or to the exact schema document that produced it. Parse history does record which registered schema a parse used, so a version label baked into the schema name survives there, but the label itself is still yours to choose. Which schema version you sent, what it meant, and how you migrate between versions are all decisions your application owns, and the label that records them has to be saved by your code next to each stored result. That is a feature, not a gap. It keeps the history in the one place you can query it.

If you have not yet drawn the optional/required line for your schema, read optional vs. required fields in JSON Schema email extraction first. Most versioning pain starts with a required array that was too ambitious on day one.

What a schema version is, concretely

A version is a specific JSON Schema document, frozen, with a label you control. Three things have to be true for versioning to be useful:

1. The document is immutable once labelled. receipt.v2 means exactly one JSON Schema. If you edit it, that is receipt.v3, even if the edit was one character.
2. Every stored result carries the label. A row that says schema_version: "receipt.v2" can be interpreted correctly forever. A row without it cannot.
3. The label is yours. It lives in your repository and your database, not in the parse response. The response tells you whether the extraction satisfied the schema you sent; it does not tell you which of your versions that was. You already know, because you sent it.

The simplest way to satisfy all three is to keep each schema version as a file in your codebase and send it inline on every request. The request is a little larger, but the version and the code that consumes it are reviewed, tagged, and deployed together, which is exactly the coupling you want.

schemas/
  receipt.v1.json
  receipt.v2.json
  receipt.v3.json

Only one of those files is the current default. The older ones stay in the tree because historical fixtures (more on those below) are tested against them, and because a re-extraction job may deliberately run an old version to reproduce an old row.

The API docs also describe registering a custom schema by name and sending that name as schema_id instead of the inline document. A name such as invoice_v1 is one way to carry your version label into the request, and it keeps request bodies small. Treat the name as your version label all the same. Updating a registered schema in place increments its server-side version counter, but that counter lives on the schema record, not on the parse results it produced: the response does not carry it, and the server does not keep a snapshot of which document a given result was validated against. If invoice_v1 is edited on Tuesday, rows parsed on Monday and rows parsed on Wednesday are indistinguishable unless your own code recorded a label for each. The server’s counter is useful for confirming an update landed; it is not a substitute for the label stored next to each row.

Backward-compatible changes

A change is backward-compatible when every result that was valid under the old schema is still valid under the new one, and code written against either version can read rows produced by the other. Both halves matter. The schema half is mechanical: an old stored row, re-validated against the new document, still passes. The consumer half depends on how your code was written. A consumer that reads only the keys it knows and ignores the rest will keep working when a new key appears; a consumer that rejects unexpected keys, or deserialises into a strict struct, will not. “Compatible” below assumes tolerant consumers. If yours are strict, even the additions in this list need a consumer deploy before the parse path changes. For extraction schemas, compatible changes mostly mean:

• Adding an optional property. Old rows do not have the key, old valid rows still validate, and tolerant old consumers ignore it. New consumers read it defensively, because it is absent on every row produced before the change and may be absent on new ones too. If the schema sets additionalProperties: false, the key has to be declared before the extractor is allowed to return it, so this is still a schema change, just a safe one.
• Loosening a constraint. Raising a maxLength, widening an enum, allowing null where it was forbidden. Every old valid result is still valid.
• Improving a description. Descriptions guide extraction and change nothing structurally. They can noticeably change what gets extracted, though, so they still get a new version label and a fixture run. Do not treat them as free.

Here is receipt.v1 and a compatible receipt.v2 that adds two optional properties, tax_cents and payment_method. The required array is unchanged. tax_cents is also nullable, so the extractor can say “the email had no tax line” explicitly on new rows; on old rows the key is simply missing, and code that reads both has to handle both cases:

{
  "title": "receipt.v1",
  "type": "object",
  "required": ["merchant", "total_cents", "currency"],
  "properties": {
    "merchant": { "type": "string", "minLength": 1 },
    "total_cents": { "type": "integer", "minimum": 0 },
    "currency": { "type": "string", "pattern": "^[a-z]{3}$" }
  },
  "additionalProperties": false
}

{
  "title": "receipt.v2",
  "type": "object",
  "required": ["merchant", "total_cents", "currency"],
  "properties": {
    "merchant": { "type": "string", "minLength": 1 },
    "total_cents": { "type": "integer", "minimum": 0 },
    "currency": { "type": "string", "pattern": "^[a-z]{3}$" },
    "tax_cents": { "type": ["integer", "null"], "minimum": 0 },
    "payment_method": { "type": "string", "enum": ["card", "bank_transfer", "wallet", "other"] }
  },
  "additionalProperties": false
}

The title is a plain JSON Schema annotation. It has no effect on validation and the API does not interpret it as a version. It is there so a schema document that gets copied into a log, a ticket, or a fixture still says which version it is.

Breaking changes

A breaking change is anything that makes an old valid result invalid, or changes the meaning of an existing key:

• Adding a property to required, even a nullable one. Making tax_cents required in receipt.v2 would feel safe because null is allowed, but every v1 row has no tax_cents key at all, so every v1 row is invalid under v2. The new rows are fine; the history is not. If you want a key that is guaranteed present going forward, that is a breaking change with a dual-read step, or a backfill of the old rows under the new label.
• Removing a property, or removing it from required. Code that read it unconditionally breaks.
• Renaming a property. total to total_cents is a removal plus an addition.
• Changing a type. "type": "number" to "type": "integer", or a string date to an object.
• Tightening a constraint. Adding a pattern, narrowing an enum, lowering a maximum. Emails that parsed cleanly last week now produce validation errors.
• Changing semantics without changing shape. amount_cents used to mean the pre-tax subtotal and now means the grand total. The schema diff is a one-word description change. The data corruption is permanent if you do not bump the version.

That last one is the reason to version on meaning, not just on structure. A diff tool will not catch it. A human reading the changelog will.

Breaking changes get a new major label and a migration plan with three parts:

1. Dual-read. Consumers are updated to handle both receipt.v2 rows and receipt.v3 rows, keyed on the stored schema_version. This ships first and is the only part that touches existing data.
2. Cut over. The parse path starts sending receipt.v3. From this point new rows carry the new label.
3. Optionally backfill. If old rows need the new shape, and your application still holds the raw email for them, re-run that MIME through /v1/parse with receipt.v3 and write the result as a new row with the new label. A backfill is only possible for rows whose source email is still within your retention window, which is one input into deciding that window. It is also why your deduplication key should include the schema version: a deliberate re-extraction under a new version is not a duplicate of the old row.

Do not edit old rows in place to look like the new version. A row labelled receipt.v2 whose contents were massaged into the v3 shape is a row nobody can trust.

Store the version with every result

The code that calls the API is where the version label and the result meet, so it is where the label gets attached. The excerpt below is framing code, not a drop-in module: rawMime is the email your application already holds, receiptV2 is the parsed contents of schemas/receipt.v2.json, and holdForReview and saveReceipt stand in for your own review queue and storage. It must run server-side: it reads your API key from the MAILFRAME_API_KEY environment variable and fails fast if it is missing, and a secret bundled into browser JavaScript is exposed to anyone who loads the page.

// rawMime, receiptV2, holdForReview, saveReceipt: provided by your application.
// Server-side only. MAILFRAME_API_KEY is a secret; never bundle it into browser code.
const SCHEMA_VERSION = "receipt.v2";

const apiKey = process.env.MAILFRAME_API_KEY;
if (!apiKey) throw new Error("MAILFRAME_API_KEY is not set");

const res = await fetch("https://api.mailframe.ai/v1/parse", {
  method: "POST",
  headers: {
    Authorization: "Bearer " + apiKey,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ schema: receiptV2, raw_mime: rawMime }),
});

if (!res.ok) {
  throw new Error(`MailFrame error ${res.status}: ${await res.text()}`);
}

const { id, data, status, validation_errors = [] } = await res.json();

if (status !== "completed" || validation_errors.length > 0) {
  // Keep the errors verbatim. They are the evidence you will need when
  // deciding whether the next schema version should loosen this constraint.
  await holdForReview({ parseId: id, schemaVersion: SCHEMA_VERSION, rawMime, status, validation_errors });
} else {
  await saveReceipt({ parseId: id, schemaVersion: SCHEMA_VERSION, data });
}

Two things are deliberate here. The version is a constant at the top of the file, so a deploy that changes it is a one-line diff that a reviewer cannot miss. And the version is stored on both branches, success and review, so a rejected email can be re-run later under the version that rejected it or under a newer one, and you can tell which.

An illustrative response for a plain-text receipt email sent with receipt.v2 (field names follow the current API shape; actual values depend on the email and the extraction run, so do not treat this as a guaranteed output):

{
  "id": "parse_7c21e4",
  "status": "completed",
  "validation_errors": [],
  "data": {
    "merchant": "Acme Corp",
    "total_cents": 4250,
    "currency": "usd",
    "tax_cents": null,
    "payment_method": "card"
  }
}

Nothing in that response says receipt.v2. Your code does. Note tax_cents: null: under v2 that is a valid, explicit “no tax line found.” A v1 row has no tax_cents key at all. A consumer that spans both versions has to treat missing and null as the same thing or deliberately distinguish them, and should decide which in writing.

Keep validation_errors as review material

The temptation when a parse comes back with a non-empty validation_errors array is to log a one-line warning and move on. Resist it. Those errors are the single best signal for what the next schema version should change, and they are only useful if you keep them attached to the raw email and the version that produced them.

Store, per rejected parse:

• the parse id from the response
• the schema version label you sent
• the validation_errors array, unmodified
• a durable pointer to the source email, or the raw MIME itself if your retention and access policy allows you to hold it
• whatever partial data came back, if any

Then review them in batches, grouped by version and by error. Patterns tend to fall into a few buckets:

The point of the table is that validation errors are not failures to be suppressed. They are the mechanism by which the schema learns what the emails actually look like. A pipeline that discards them has no feedback loop, and its schema stops improving the week it ships.

One caution that the optional/required guide makes at length: a result with an empty validation_errors array is structurally valid, not necessarily correct. Keep your semantic checks (tax not exceeding total, identifiers present in the source) as a separate review branch, and keep those findings too. They feed the next version just as much as the structural ones do.

Test every version against historical fixtures

A fixture is a raw email you have already parsed, saved alongside the result you accepted for it. A fixture set is the regression suite for your schema. Before any new version goes live, it runs against every fixture and you read the diff.

Build the set from the shapes you see in real traffic, not from the traffic itself:

• Keep full MIME, not a cleaned-up body. The schema change you are testing may depend on a header or a part that a stripped-down copy no longer has.
• Cover every sender and every sparse case. Include the receipts with no tax line, the invoices with no due date, the one notification from the sender who formats dates differently. Those are the emails that break on a required change.
• Redact before it becomes a fixture, and keep structure. Replace names, addresses, account numbers, and card digits with plausible stand-ins of the same shape. A fixture whose $42.50 became $0.00 tests nothing, but a fixture that still carries a customer’s real address is a data leak waiting for a repository clone. Real customer emails do not belong in CI or in a fixture directory; whether and how long you retain the originals, and who can read them, is governed by your own retention and access-control policy, not by the fixture suite.
• Record the version that produced the expected result. A fixture is a triple: redacted email, schema version, expected result. When you add receipt.v3, you add a third expected result for each fixture, not overwrite the second.

Then the test itself is small. For each fixture, send the fixture MIME and the candidate schema to /v1/parse, and compare:

1. Did status change? A fixture that completed under v2 and does not complete under v3 is either an intended breaking change or a bug, and you should know which before cut-over.
2. Did validation_errors change? New errors on old fixtures mean the new version is stricter than the data. That may be what you wanted. Confirm it.
3. Did data change? Diff field by field. Added keys are expected. Changed values on keys that did not change in the schema are the interesting ones, and usually trace back to a description edit.

Because extraction is not perfectly deterministic, a value-level diff may show small variations that have nothing to do with the schema change. Treat a single flipped value on one fixture as something to look at, not as an automatic failure, and treat the same key flipping across many fixtures as a real signal. Keeping the fixture set broad is what makes that distinction readable.

Run the suite in CI on any change under schemas/. The cost is one /v1/parse call per fixture per run, which is small next to the cost of discovering a semantic regression in production rows that already have the new label.

Checklist for a new schema version

• The new document has a new label, even for a one-line change, and the old document is unchanged in the tree.
• The change is classified as compatible or breaking, in writing, including any change in meaning that the diff does not show.
• Breaking changes ship consumer dual-read before the parse path cuts over.
• Every stored result, accepted or held for review, carries the version label that produced it.
• Held-for-review rows keep the validation_errors array verbatim, with the parse id and a pointer to the source email.
• Consumers that span versions are known to tolerate added keys, or have been deployed before the parse path changes.
• The deduplication key includes the schema version, so a deliberate re-extraction under a new version is not treated as a duplicate.
• The fixture suite has run against the new version and the status, validation_errors, and data diffs have been read by a person.
• A fixture exists for every sender and every sparse case the last batch of validation errors surfaced.

Next steps

• The API reference covers the full POST /v1/parse request and response shape, including schema, schema_id, status, and validation_errors.
• Optional vs. required fields is the companion to this guide: most compatible changes are the result of drawing that line more carefully.
• Preventing duplicate records explains why the schema version belongs in your deduplication key.
• Ready to run a schema of your own against real email? Apply for early access.