☁️Serverless GPU
Also known as Serverless Inference, Serverless GPU Inference, Scale-to-Zero GPU
GPUs that switch on when a request comes. They drop to zero when no one asks.
Think of it like
It is like a taxi, not a car of your own. You pay only for the ride, not for the parked hours.
Example
Lucy's photo app gets most of its traffic in the evening. On serverless GPUs it runs 10 GPUs at peak and none at night, and she pays only for the busy time.
Why it matters
A GPU left on all day is costly. Paying only for busy seconds can cut the bill a lot when traffic is uneven.
Definition
Common mistake
Key takeaways
- 1On when called. GPUs start for each wave of requests.
- 2Zero when idle. No requests, no bill.
- 3Cold starts. The first request may wait.