📏Needle in a Haystack Test
Also known as Needle in a Haystack, NIAH, Needle-in-a-Haystack Benchmark
A test that hides one fact in a long text and asks the model to find it.
Think of it like
It is like hiding one odd sentence in a long book. Then you ask if the reader spotted it.
Example
Lucy hides the line 'The secret code is 7429' in a 300-page report. The model finds it near the start but misses it in the middle.
Why it matters
Models now take very long inputs. This test checks if they can really use every part of them.
Definition
Common mistake
Key takeaways
- 1Hide one fact. Then ask about it.
- 2Move it around. Start, middle and end.
- 3Grow the pile. Try longer and longer texts.