Skip to content

0005 — Citation verification design and the fail-closed 422 policy

Status

Accepted (v0.1.0)

Context

This system's whole value proposition is that answers are grounded in retrieved context, not free-standing model output. That guarantee is worthless if it's only aspirational — a hallucinated citation (a claim tagged as if it were sourced, pointing at a chunk that was never actually in the context Claude received) or an uncited claim must never reach the caller looking identical to a genuinely verified one.

Claude is prompted (generation/prompt.py) to return a StructuredAnswer (generation/ schemas.py) whose claims each carry [DocID:ChunkID]-style citations, but a prompt is a request, not an enforcement mechanism — nothing about asking the model nicely to cite correctly guarantees it always will.

Decision

Verify every claim's citations against the actual retrieved context after generation, and fail closed: an answer that doesn't fully verify is never served as a normal 200 response.

  • generation/citation_parser.py::is_well_formed_tag defines the shape rule for a DocID/ChunkID half: non-empty, no colon, no whitespace.
  • generation/citation_verifier.py::verify_citations(answer, context_chunk_ids) walks every claim in order and returns the first failure, with one of three specific reasons:
  • missing_citation — a claim has zero citations at all.
  • malformed_citation — a citation's shape doesn't pass is_well_formed_tag.
  • hallucinated_citation — the citation is well-formed but its chunk_id was never part of context_chunk_ids (the reranked chunks actually sent to Claude for this query). An answer with zero claims (Claude declining to answer) is vacuously verified — there's nothing to check, and nothing unverified is served either.
  • api/routes/query.py::query is the only caller of verify_citations, and treats it as the sole gate on whether to serve the answer: on verified=False, it raises HTTPException(422, detail={"reason": ..., "claim_index": ...}) instead of returning the StructuredAnswer — the answer is discarded, not "served with a warning" or "served with citations stripped." On a genuine 200, and only then, a query_log row is written (never on 422, and never on the 403 auth-mismatch path, which is rejected upstream in api/deps.py before this handler runs at all) — this is also what eval/cost_per_query.py's cost accounting relies on: only real, verified, served answers count as billable/loggable queries.

Why fail-closed, not fail-open with a flag

An alternative design would return the answer anyway, annotated with a verified: false flag, and let the caller decide whether to trust it. This was rejected: a caller integrating against this API (or a downstream UI rendering the answer) has to actively remember to check that flag on every single response, forever, or a hallucinated/uncited claim silently reaches an end user indistinguishable from a verified one. Failing closed (422, no answer body to accidentally trust) makes the unsafe case impossible to render by accident, at the cost of the caller sometimes getting no answer instead of an answer they must double-check.

Consequences

  • A well-behaved, correctly-citing model response and a citation-verification bug in this code are both invisible to the caller as anything other than "either 200 with a fully-cited answer, or 422" — there's no partial-credit response shape that leaks a half-trusted answer.
  • Cost accounting (query_log, eval/cost_per_query.py) only reflects genuinely served answers; a citation-verification failure costs Claude API tokens (the call still happened) but is never counted as a "successful query" in the cost/query metric — a deliberate choice, not an oversight, since a 422 was never actually delivered as a usable answer to the caller.
  • tests/unit/test_citation_verifier.py and tests/integration/test_api_ingest_query_e2e.py (which mocks an uncited response to prove the live 422 path, not just the unit-level function) both exercise this; the fail-closed policy has coverage at both layers, not just the pure function.
  • If a future version needs partial/best-effort answers, that requires a new, explicit response shape and API contract change — this design deliberately does not offer a softer default.