📏Promptfoo
Also known as Promptfoo CLI, promptfoo eval, promptfooconfig.yaml
A free tool that tests your prompts on many AI models at once.
Think of it like
It is like a spelling test for a robot. You can give it again and again.
Example
Lucy changes her support bot's prompt. She runs promptfoo eval, and 3 of her 40 test cases now fail, so she fixes them before anyone sees.
Why it matters
A prompt change that fixes one answer can break five others. Running the same tests every time catches that.
Definition
Example code
yaml
prompts:
- 'Answer in one word. What is the capital of {{country}}?'
providers:
- anthropic:messages:claude-haiku-5-5
tests:
- vars:
country: France
assert:
- type: icontains
value: ParisCommon mistake
Key takeaways
- 1Tests in a file. Prompts, models and cases in one YAML file.
- 2Every model at once. Compare answers side by side.
- 3Red teaming too. It attacks your app on purpose.