Skip to content
BENIBIX Learn

Serverless GPU

Also known as Serverless Inference, Serverless GPU Inference, Scale-to-Zero GPU

GPUs that switch on when a request comes. They drop to zero when no one asks.

Think of it like

It is like a taxi, not a car of your own. You pay only for the ride, not for the parked hours.

Example

Lucy's photo app gets most of its traffic in the evening. On serverless GPUs it runs 10 GPUs at peak and none at night, and she pays only for the busy time.

Why it matters

A GPU left on all day is costly. Paying only for busy seconds can cut the bill a lot when traffic is uneven.

Definition

Serverless GPU is a way to run your own model on rented GPUs that switch on only when requests arrive. You send your model or a container, and the service starts GPUs when someone calls it. When no one calls, it drops back to zero and you pay nothing. The catch is a cold start, a wait of seconds while a GPU wakes and loads the model. It suits traffic that comes and goes.
The right panel is serverless GPU. GPUs switch on for requests and drop to zero when it is quiet.

Common mistake

Ignoring the cold start. The first request after a quiet spell can wait while a GPU wakes up.

Key takeaways

  1. 1
    On when called. GPUs start for each wave of requests.
  2. 2
    Zero when idle. No requests, no bill.
  3. 3
    Cold starts. The first request may wait.

Compare