# Serverless GPU

> GPUs that switch on when a request comes. They drop to zero when no one asks.

**Field:** Artificial Intelligence  
**Category:** Cloud AI Platforms  
**Also known as:** Serverless Inference, Serverless GPU Inference, Scale-to-Zero GPU

## Think of it like

It is like a taxi, not a car of your own. You pay only for the ride, not for the parked hours.

## Definition

Serverless GPU is a way to run your own model on rented GPUs that switch on only when requests arrive. You send your model or a container, and the service starts GPUs when someone calls it. When no one calls, it drops back to zero and you pay nothing. The catch is a cold start, a wait of seconds while a GPU wakes and loads the model. It suits traffic that comes and goes.

## Example

Lucy's photo app gets most of its traffic in the evening. On serverless GPUs it runs 10 GPUs at peak and none at night, and she pays only for the busy time.

## Why it matters

A GPU left on all day is costly. Paying only for busy seconds can cut the bill a lot when traffic is uneven.

## Common mistake

Ignoring the cold start. The first request after a quiet spell can wait while a GPU wakes up.

## Three principles

- **On when called.** GPUs start for each wave of requests.
- **Zero when idle.** No requests, no bill.
- **Cold starts.** The first request may wait.

## Related terms

- [GPU Cloud](https://learn.benibix.com/term/ai/gpu-cloud)
- [Model as a Service (MaaS)](https://learn.benibix.com/term/ai/model-as-a-service)
- [Serverless Computing](https://learn.benibix.com/term/infrastructure/serverless-computing)
- [Model Serving](https://learn.benibix.com/term/ai/model-serving)
- [Runpod](https://learn.benibix.com/term/ai/runpod)

## Told apart from

- [Serverless GPU vs Model as a Service](https://learn.benibix.com/compare/serverless-gpu-vs-model-as-a-service): People think a serverless GPU is the same as model as a service, and it is not. With model as a service, you call a model the provider runs and pay per token. With a serverless GPU, you bring your own model, and the service runs it on GPUs that switch on when called. You pay for the busy seconds. One rents a model. The other rents the chips.

![Two panels. Left: a rented machine with GPUs runs all day, with a clock and a bill that grows by the hour, from a GPU cloud such as CoreWeave or Runpod. Right: requests arrive, GPUs switch on to answer them, then drop to zero when no one asks, and the bill counts only the busy seconds.](https://learn.benibix.com/diagrams/ai7-gpu-compute.svg)

---

Source: https://learn.benibix.com/term/ai/serverless-gpu
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
