Skip to content
BENIBIX Learn

Serialization and Protobuf

Level 2 · Data formats

How a program turns its data into text or bytes to save or send, and back. How JSON Schema checks the shape, and why Protobuf is smaller.

Course contentsLesson 4 of 11

After this lesson you can

  • Explain serialization in one sentence
  • Read a small JSON Schema
  • Say when Protobuf beats JSON

12 mincovers 3 glossary entries

Worth reading first: JSON, XML and CSV

How data leaves a program and comes back

Serialization turns the data in a program into text or bytes you can save or send. The other side reads it and builds the data back. That step is called deserialization. Emma's name and age take 24 bytes as JSON text, and just 8 bytes in Protobuf. Each saved game and each web request does this.

Inside a program, data is spread out in memory in a way only that program knows. A file or a network can only carry a line of bytes. So this step puts the data in a line, in a format both sides know.

JSON is the most common choice. People can read it, and each language can use it. In JavaScript, JSON.stringify does the packing and JSON.parse does the unpacking. Python has json.dumps and json.loads for the same two jobs.

Both sides must agree on the shape, not just the format. If one side sends age as text and the other wants a number, things break. JSON Schema is a way to write that shape down. It is itself written in JSON, and it lists each field, its type and which fields are required.

A tool can check any JSON against a schema and point to each mistake before the data is used. Code editors use schemas too, to warn you as you type a config file. The current version of JSON Schema is called 2020-12.

Protocol Buffers, or Protobuf, is a format from Google that packs data into compact bytes. You describe the data in a .proto file, giving each field a number. A tool then writes code for your language that packs and unpacks it. The bytes are small and fast to read, but people cannot read them.

Two mistakes are common. One is reading data from a stranger with no check. In some formats, that can even run an attacker's code. The other is changing a Protobuf field's number once it is in use. Then old messages land in the wrong field.

json
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "type": "integer", "minimum": 0 }
  },
  "required": ["name"]
}
A JSON Schema: name must be text, age a whole number of 0 or more, and name is required.
protobuf
message Person {
  string name = 1;
  int32 age = 2;
}
The same shape as a Protobuf message. The numbers 1 and 2 must never change once in use.
Emma's record leaves memory as 24 bytes of text or 8 bytes of Protobuf. The other program builds it back.

Worked example

Jack catches bad data early

Jack's shop gets 5,000 orders a day as JSON. Once a week, one arrives with the price as text, and the total comes out wrong.

He adds a JSON Schema that says price must be a number. Now the bad order is turned away at the door, with a message naming the field.

The words for this lesson3

Check yourself

What does serialization do?

Finished this lesson?

Marking it complete also marks its 3 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.