Table Catalog vs Data Catalog
People think a table catalog and a data catalog are the same thing, and they are not. A table catalog is for query engines. It tells Spark or Trino where each lake table lives right now. A data catalog is for people. It is a search page that says what a table means and who owns it. Some products do a bit of both.
Table Catalog
A list that tells tools where each lake table is right now.
A table catalog is the service that lets query engines find the tables in a data lake. It keeps the list of table names. For each one, it points at the file that describes the table right now. When a write changes a table, that pointer is swapped in one step. So every engine, from Spark to Trino, sees the same version. Hive Metastore, Unity Catalog and the Iceberg REST catalog are all examples.
Data Catalog
A search box for all of a firm's data. Each table gets a page.
A data catalog is a searchable list of all the data a company has. Each table gets a page with its description, owner, columns and tags. It shows where the data came from and who uses it. People search it to find the right table before they write a query. It fills itself by scanning warehouses and tools, and people add the meaning.
In one line
- Table Catalog
- A list that tells tools where each lake table is right now.
- Data Catalog
- A search box for all of a firm's data. Each table gets a page.