Skip to content
BENIBIX Learn

VLA vs VLM

People often think a vision-language-action model is just a vision language model in a robot. Both look at pictures and read words. A vision language model answers in words, such as 'the cup is on the left'. An action model outputs moves, such as how far to turn each joint to pick the cup up. Most are built on a vision language model, then trained on clips of robots at work.

AI🦾 Robotics & Vision

Vision-Language-Action Model (VLA)

An AI model that sees, reads what you ask, and tells a robot how to move.

A vision-language-action model is an AI model that drives a robot. It looks through the robot's cameras, reads an order in words, and outputs the next moves. It is often built on a vision language model, so it already knows what a cup or a sink is. It is then trained on many clips of robots doing tasks, to turn that into arm and hand moves. One model can then try many tasks from plain words, with no code for each one. It is one of the main ways robots are getting more general.

Example
Emma types 'put the red cup in the sink' to a robot in her lab. Its VLA model finds the red cup in the camera view and moves the arm to pick it up and drop it in.
Why it matters
One model that follows plain words can do many jobs. It brings what chat models know about the world into a robot's hands.
Full entry
AI🎨 Multimodal

Vision Language Model (VLM)

An AI that can look at a photo. Then it tells you about it in words.

A vision language model is an AI model that can take in pictures as well as text. It answers in text. First, an image encoder turns the picture into tokens, small bits the model can read like words. The model then reads those tokens along with your question. So it can describe a photo, read a chart, or answer questions about a screen. Most big chat models today can do this.

Example
Emma sends a photo of a broken bike chain to a chat app and asks what is wrong. The model sees that the chain has come off the back gear and tells her how to put it back.
Why it matters
Much of the world is pictures, charts and screens, not text. A model that can see can help with all of them.
Full entry

In one line

Vision-Language-Action Model (VLA)
An AI model that sees, reads what you ask, and tells a robot how to move.
Vision Language Model (VLM)
An AI that can look at a photo. Then it tells you about it in words.