💬LLM Streaming
Also known as Streaming Response, Token Streaming
The reply is sent bit by bit as it is written, not all at the end.
Think of it like
Think of it like you watch a friend write a note word by word. You do not wait for the end.
Example
Without streaming, Emma's chat app shows a spinner for 8 seconds, then the full reply. With streaming, words start in half a second.
Why it matters
People leave when an app looks stuck. Streaming makes slow answers feel fast.
Definition
Common mistake
Key takeaways
- 1Piece by piece. As tokens are written.
- 2First words fast. Under a second.
- 3Same total time. It only feels faster.