Schema Evolution vs Schema Drift
People think schema drift and schema evolution are the same change. Evolution is planned: a team adds a field on purpose and checks that readers still work. Drift is a surprise: a source changes its shape and nobody downstream was told. The first is a habit to build. The second is a problem to catch.
Schema Evolution
Changing the shape of data over time. Code that reads it must keep working.
Schema evolution is changing the shape of data over time without breaking what reads it. A schema is the list of fields and their types. Adding a new field is usually safe, because old rows can leave it empty. Renaming or deleting a field can break every report that uses it. Good systems check each change before it goes live.
Schema Drift
When the columns of new data change and no one says so.
Schema drift is when the shape of incoming data changes without warning. A column is added, renamed or dropped, or its type changes. The source team often makes the change for a good reason and does not know who reads the data. Downstream, a pipeline may crash. Worse, it may keep running and fill a column with blanks.
In one line
- Schema Evolution
- Changing the shape of data over time. Code that reads it must keep working.
- Schema Drift
- When the columns of new data change and no one says so.