Optional vs. Required Fields in JSON Schema Email Extraction

The required array is the most consequential line in an email extraction schema. It decides what happens when an email simply does not contain a piece of data: does the parse fail validation, does the field come back absent, or does a plausible-looking value appear where none existed in the source?

This guide covers how to draw the optional/required line for POST /v1/parse, how to make “not present” representable instead of forcing a guess, and why a result that passes every schema constraint can still be wrong. It assumes you already know the basics of building a JSON Schema email parser; if not, start there and come back.

What required actually says

In JSON Schema, required is a structural assertion: the listed keys must be present in the object. It says nothing about whether the source email contained the information. When you send a schema to MailFrame, extraction targets that shape and the result is validated against it before it is returned. A required key whose type does not allow null is therefore a promise that every email you send has that data. A required key whose type does allow null promises only that the key will be present in every result; the value itself may still be unknown. The rest of this guide uses that second combination deliberately, so keep the distinction in mind.

The first kind of promise is easy to break. Consider a receipt schema where every required key is non-null:

{
  "type": "object",
  "required": ["merchant", "total_cents", "currency", "order_number", "tax_cents"],
  "properties": {
    "merchant": { "type": "string" },
    "total_cents": { "type": "integer" },
    "currency": { "type": "string" },
    "order_number": { "type": "string" },
    "tax_cents": { "type": "integer" }
  }
}

Plenty of real receipts never mention tax, and many have no order number at all. Every one of those emails now faces a dilemma: either the parse reports validation errors because the key is missing, or the extractor produces something to satisfy the constraint. Neither outcome is what you wanted. The first throws away emails you could have processed; the second is a fabricated value with a valid type.

Over-requiring invites fabrication

A required integer field with no corresponding data in the email has two possible outcomes: the key is left out and the parse reports a validation error, or an integer gets invented to satisfy the constraint. The same goes for a required format: date string with no date in the message, or a required enum with no matching phrase. Strict typing does not prevent fabrication. It just guarantees that whatever gets fabricated will look clean.

This is the core trade-off. Schema constraints narrow the space of acceptable outputs. If the true answer, “this email does not say,” is outside that space, you have made the honest answer impossible and left only wrong answers available.

The fix is to make absence representable:

• Drop the key from required. The field can then be omitted when the email has nothing to say. Your application checks for the key before using it.
• Require the key but allow null. Keep the key in required and use "type": ["integer", "null"]. The key must then be present in every result, and null is the explicit way to say “unknown.” This is the only combination that gives downstream code a stable key to rely on.

One point that is easy to get backwards: making a property nullable does not, by itself, make it always present. An optional property, nullable or not, is allowed to be absent. So an optional nullable field may come back as a value, as null, or not at all, and your code has to handle all three. If you want a key you can always read, it has to be in required.

Here is the same receipt schema with the line drawn deliberately:

{
  "type": "object",
  "required": ["merchant", "total_cents", "currency", "order_number", "tax_cents"],
  "properties": {
    "merchant": { "type": "string", "minLength": 1 },
    "total_cents": { "type": "integer", "minimum": 0 },
    "currency": { "type": "string", "pattern": "^[a-z]{3}$" },
    "order_number": { "type": ["string", "null"] },
    "tax_cents": { "type": ["integer", "null"], "minimum": 0 },
    "shipping_cents": { "type": ["integer", "null"], "minimum": 0 }
  },
  "additionalProperties": false
}

Three fields must carry a real value, and each one is something a receipt cannot be a receipt without. Two more, order_number and tax_cents, are required keys that may legitimately be null, so downstream code can always read them. The last, shipping_cents, is fully optional and may be a value, null, or missing. The additionalProperties: false line is a validation rule: a result that carries keys you did not ask for fails validation instead of being returned as clean data.

A rule of thumb for drawing the line

Ask one question per field: if this value were missing, would I refuse to process the email?

Be suspicious of your own first draft. Teams tend to require everything their database column list contains, then discover that a third of their emails lack one of those columns. Fields that are commonly over-required in practice:

• Tax, shipping, and discount amounts on receipts
• Tracking numbers on order confirmations (often sent in a later email)
• Invoice due dates (frequently “upon receipt” or absent)
• Customer names (many notifications only carry an address)
• Secondary identifiers like PO numbers or reference codes

If you are using a pre-built schema from the schema library, look at its field table first. Those schemas already distinguish fields a given email type always carries from fields it sometimes carries.

Schema constraints are not semantic correctness

Passing validation means the output has the right shape. It does not mean the output is true.

Take this extraction for a $42.50 receipt:

{
  "merchant": "Acme Corp",
  "total_cents": 4250,
  "currency": "usd",
  "order_number": null,
  "tax_cents": 4250,
  "shipping_cents": null
}

It passes the schema above. It is also wrong: the total has been duplicated into tax_cents. No type, minimum, or pattern constraint can catch that, because the schema has no way to express “tax must not exceed the total” or “tax must appear in the email.” JSON Schema validates structure, not meaning.

Treat the schema as the first filter, not the last. A few checks that belong in your code, after the response comes back:

• Cross-field consistency. Neither tax nor shipping should exceed the total (a shipping-only charge can legitimately equal it), and on a receipt that lists goods, a tax equal to the total is almost certainly a duplicated value. A refund amount should not exceed the original charge. Line items should sum to roughly the total.
• Source presence. For fields you care about most, confirm the value or a recognizable form of it actually appears in the raw email before trusting it. A reference_id that does not occur anywhere in the MIME body is a red flag.
• Range sanity. A total_cents of 1 or 100,000,000 on a consumer receipt deserves a second look even though both are valid non-negative integers.
• Null and absent are results, not errors. When a nullable field is null, or an optional key is missing entirely, record that the email did not say rather than backfilling a default that looks like data.

Each of these is a line or two of ordinary application code, and together they catch the class of errors a schema structurally cannot.

Calling the API with an inline schema

POST /v1/parse accepts the raw email as raw_mime alongside either a schema_id for a schema you have registered or a pre-built one, or an inline schema document. The result comes back synchronously in the same HTTP response. Here is the receipt schema from above sent inline:

curl -X POST https://api.mailframe.ai/v1/parse \
  -H "Authorization: Bearer $MAILFRAME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "schema": {
      "type": "object",
      "required": ["merchant", "total_cents", "currency", "order_number", "tax_cents"],
      "properties": {
        "merchant": { "type": "string", "minLength": 1 },
        "total_cents": { "type": "integer", "minimum": 0 },
        "currency": { "type": "string", "pattern": "^[a-z]{3}$" },
        "order_number": { "type": ["string", "null"] },
        "tax_cents": { "type": ["integer", "null"], "minimum": 0 },
        "shipping_cents": { "type": ["integer", "null"], "minimum": 0 }
      },
      "additionalProperties": false
    },
    "raw_mime": "From: receipts@example.com\r\nTo: ops@example.com\r\nSubject: Your Receipt\r\n\r\nThanks for shopping at Acme Corp. Your total was $42.50."
  }'

The sample email mentions a merchant and a total but no order number, tax, or shipping. An illustrative response for it might look like this (field names follow the current API shape; actual values depend on the email and the extraction run, so do not treat this as a guaranteed output):

{
  "id": "parse_3e9b7d",
  "status": "completed",
  "validation_errors": [],
  "data": {
    "merchant": "Acme Corp",
    "total_cents": 4250,
    "currency": "usd",
    "order_number": null,
    "tax_cents": null
  }
}

The two required nullable keys are present and null, and the optional shipping_cents key is simply absent, which the schema also permits. Neither is invented. The schema made that honest answer possible; it does not force it, which is why the checks in the previous section still matter.

Once a schema stabilizes, register it and send its name as schema_id instead of repeating the document on every call. The API docs cover the schema endpoints and the full request and response shape.

Handling the response

Branch on the response rather than reading data unconditionally. The excerpt below is framing code, not a drop-in module: schema and rawMime are the schema document and raw email your application already holds, and holdForReview, passesSemanticChecks, and saveReceipt stand in for whatever review queue, consistency checks, and storage you use. This code must run server-side: it reads your API key from the environment, and a secret bundled into browser JavaScript is exposed to anyone who loads the page.

// schema, rawMime, holdForReview, passesSemanticChecks, saveReceipt: provided by your application.
// Server-side only. MAILFRAME_API_KEY is a secret; never bundle it into browser code.
const res = await fetch("https://api.mailframe.ai/v1/parse", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.MAILFRAME_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ schema, raw_mime: rawMime }),
});

if (!res.ok) {
  throw new Error(`MailFrame error ${res.status}: ${await res.text()}`);
}

const { data, status, validation_errors = [] } = await res.json();

if (status !== "completed" || validation_errors.length > 0) {
  // Structural failure: a required key was missing, a type did not match,
  // or an unexpected key appeared under additionalProperties: false.
  await holdForReview({ rawMime, status, validation_errors });
} else if (!passesSemanticChecks(data)) {
  // Structurally valid but suspicious: tax or shipping > total, tax equal to
  // total on a receipt with goods, id not in source, etc.
  // Remember optional keys may be absent, so check with `in` or `?? null`
  // rather than assuming every property exists.
  await holdForReview({ rawMime, data, reason: "semantic" });
} else {
  await saveReceipt(data);
}

Notice that the semantic branch exists at all. Validation errors tell you about shape. Your own checks tell you about truth. Keeping them as separate branches makes it obvious, when you read the review log later, which kind of problem you are looking at.

Checklist

Before you register a schema for production traffic:

• Every non-null key in required is one you would refuse to process the email without.
• Every nullable key in required is there because downstream code needs a stable shape, not because the email always carries that value.
• Every field that is sometimes absent in real emails is either optional or required-and-nullable.
• Every key your code reads unconditionally is in required (nullable if it may be unknown); optional keys are read defensively because they may be missing.
• additionalProperties is false where unexpected keys should fail closed, so a result carrying keys you did not ask for fails validation instead of passing through. This is not mandatory for every integration; leave it off if extra keys are harmless to you.
• Required fields carry tight constraints (minLength, minimum, pattern, enum).
• Application code checks cross-field consistency and source presence after validation.
• null or a missing optional key is stored as “the email did not say,” not replaced with a default.
• You have sent a handful of real, varied emails through /v1/parse and read the output by hand, including the ones you expected to be sparse.

A schema that admits “this email does not say” removes the pressure to fabricate. It does not, on its own, make every record trustworthy; that comes from pairing it with the semantic checks above and reading real output before you rely on it.