VLA vs VLM
People often think a vision-language-action model is just a vision language model in a robot. Both look at pictures and read words. A vision language model answers in words, such as 'the cup is on the left'. An action model outputs moves, such as how far to turn each joint to pick the cup up. Most are built on a vision language model, then trained on clips of robots at work.
Vision-Language-Action Model (VLA)
An AI model that sees, reads what you ask, and tells a robot how to move.
A vision-language-action model is an AI model that drives a robot. It looks through the robot's cameras, reads an order in words, and outputs the next moves. It is often built on a vision language model, so it already knows what a cup or a sink is. It is then trained on many clips of robots doing tasks, to turn that into arm and hand moves. One model can then try many tasks from plain words, with no code for each one. It is one of the main ways robots are getting more general.
Vision Language Model (VLM)
An AI that can look at a photo. Then it tells you about it in words.
A vision language model is an AI model that can take in pictures as well as text. It answers in text. First, an image encoder turns the picture into tokens, small bits the model can read like words. The model then reads those tokens along with your question. So it can describe a photo, read a chart, or answer questions about a screen. Most big chat models today can do this.
In one line
- Vision-Language-Action Model (VLA)
- An AI model that sees, reads what you ask, and tells a robot how to move.
- Vision Language Model (VLM)
- An AI that can look at a photo. Then it tells you about it in words.