Skip to content
BENIBIX Learn

Faithfulness (AI)

Also known as Answer Faithfulness, Groundedness, Faithfulness Score

Whether an AI answer only says what its source text backs up. It matters most when the AI answers from your files.

Think of it like

It is like a book report that only says what is in the book. Nothing is made up.

Example

Lucy's help bot answers from the company handbook. A judge model finds that 46 of 50 claims in its answers are in the handbook, so faithfulness is 92%.

Why it matters

It catches made-up claims in apps that answer from your own files. Those claims sound just as sure as true ones.

Definition

Faithfulness checks whether an AI's answer sticks to the source text it was given. The source can be a document, search results or a set of notes. A faithful answer makes only claims that the source backs up. To test it, the answer is split into claims, and each claim is checked against the source. The score is the share of claims that are backed. An answer can be true in the real world and still fail, if the source never said it.
Each claim needs a line back to the source. Three of four are backed, so the score is 75%.

Common mistake

Only testing if the answer is right. A lucky guess from the model's memory passes, even when the source said nothing.

Key takeaways

  1. 1
    Claim by claim. Split the answer and check each part.
  2. 2
    Source is the judge. Backed by the text it was given.
  3. 3
    Not the same as true. True but unbacked still fails.

Compare