Skip to content
BENIBIX Learn

Ragas

Also known as Ragas Metrics, Ragas Evaluation, RAG Assessment

A free Python tool. It scores AI apps that answer from your own documents.

Think of it like

It is like marking an open-book test. Did they find the right pages, and did they copy them right?

Example

Jack's help bot answers from 500 product manuals. Ragas shows context recall at 0.62, so he fixes how the bot finds pages before he touches the prompt.

Why it matters

A wrong answer can come from bad search or bad writing. Scoring each half shows which one to fix.

Definition

Ragas is a free, open-source Python tool for grading AI apps, best known for apps that answer from your own files. These apps first fetch some text, then write an answer from it. Ragas scores both halves, mostly with a judge model, from 0 to 1. Context precision and context recall check if the right text was fetched. Faithfulness checks if the answer sticks to that text. Response relevancy checks if it answers the question.

Example code

bash
pip install ragas
ragas quickstart rag_eval
Install Ragas, then make a starter test for an app that answers from files.
Each metric checks one link in the chain, from question to text to answer.

Common mistake

Only scoring the final answer. A good answer from the wrong pages hides a search problem that will bite later.

Key takeaways

  1. 1
    Two halves. Fetching and answering, scored apart.
  2. 2
    0 to 1 scores. Higher is better on every metric.
  3. 3
    Makes test sets. Questions built from your own files.

Compare