📏DeepEval
Also known as DeepEval Framework, deepeval test run
A free Python tool that tests AI answers like unit tests. Each must pass a score.
Think of it like
It is like a spelling test for a robot's answers. Each answer must reach a mark, or the test fails.
Example
Emma adds 30 DeepEval tests to her support bot's code. They run each night with her other tests, and one failing faithfulness check stops a bad release.
Why it matters
It puts AI checks in the same place as the code tests. A bad prompt change then fails the build, like a bug would.
Definition
Example code
python
from deepeval import assert_test
from deepeval.metrics import AnswerRelevancyMetric
from deepeval.test_case import LLMTestCase
def test_refund():
case = LLMTestCase(
input="What if these shoes don't fit?",
actual_output="You have 30 days to get a full refund.",
)
assert_test(case, [AnswerRelevancyMetric(threshold=0.7)])Common mistake
Key takeaways
- 1Like pytest. Test files and one command.
- 2Scored metrics. Each scores 0 to 1 against a line.
- 3Judge models. Most metrics ask another model.