When AI Citations Do Not Support the Answer

A citation can show that a document was retrieved without proving that the document supports the sentence beside it. Verify the claim against the cited passage, including qualifiers, dates and exceptions. A working link is only the first check.

Split the answer into claims

Consider this illustrative sentence: “All contractors can claim travel costs immediately.” It contains at least three claims: who qualifies, what expense is covered and when reimbursement applies.

A cited policy saying “Employees may request approval before travel” does not support that sentence. The words are related, but the scope and timing differ.

Use a claim ledger:

Claim	Source passage	Verdict	Action
Contractors qualify	No explicit support	Unsupported	Remove or ask
Travel may be reimbursed	Conditional policy	Partial	Preserve condition
Payment is immediate	No timing specified	Unsupported	State unknown

The ledger is an original review method, not a claim that a citation system performs these checks automatically.

Inspect where citations come from

Find out whether the application attaches citations from retrieved passages, model-generated reference IDs or tool events. Mixed citation channels can introduce presentation problems; an Open WebUI upstream discussion reports confusion between RAG and tool citations.

That report is evidence of a user difficulty, not proof that the same bug affects your installed release.

Capture the retrieved passage and the model's actual input when your application supports it. If the correct evidence was absent, the first failure is retrieval. If it was present but the answer overstated it, focus on generation and validation.

Check page numbering and versions

PDF index 8 and printed page 8 may refer to different pages because of a cover or preface. Store both when possible. Also check whether the source link opens the same document version that was indexed.

A citation can be technically correct for an old version and misleading for the current policy. Include a stable document identifier and revision date in the evidence record.

Test unsupported questions

Ask a question whose answer is deliberately absent from the fixture. A reliable document workflow should distinguish “not found” from “false” and should not attach a nearby citation merely to look complete.

Then ask a question whose answer spans two passages. Verify that both are retrieved and that the conclusion follows from them. This is harder than matching a single keyword.

Set a release standard

For each factual sentence that matters, require a supporting passage or an explicit uncertainty label. Review the highest-impact claims first; a large number of citation badges does not establish quality.

The existing AI claim-checking guide supplies the broader process. Use prompt regression testing to keep unsupported-question cases from returning after an update.

Related troubleshooting

• RAG Cannot Read a Scanned PDF: What to Check
• Why RAG Still Mentions a Removed Document

This guide draws on the linked documentation. Examples are illustrative unless explicitly identified as measured results.