Skip to content
BENIBIX Learn

Model Fallback

Development🧬 AI EngineeringLLM Gateways

Also known as LLM Fallback, Fallback Model, Model Failover

Sending a call to a backup model when the first one fails.

Think of it like

Think of it like a spare tyre. When one tyre goes flat, you put on the spare and keep driving.

Example

Lucy's writing app uses a big model first. When that firm's service goes down, calls go to a smaller model from a different firm. Users barely notice.

Why it matters

Model providers do go down. A fallback keeps your app working through someone else's outage.

Definition

Model fallback is a rule that sends a call to a backup model when the first choice fails. A call can fail when the model maker is down, a rate limit is hit, the call takes too long or the reply is broken. Then the app tries the next model on its list, and does not show an error. The backup may be a smaller model, or come from a different firm. The user gets an answer, maybe a bit different, not an error.
The first model fails, so the same call goes to a backup model, which answers.

Common mistake

Never testing the backup. A fallback model that was never tried with your prompts may answer in the wrong format.

Key takeaways

  1. 1
    A list of models. In order.
  2. 2
    Try the next. When one fails.
  3. 3
    Test the backup. Before you need it.

Compare