Skip to content
BENIBIX Learn

Apache Parquet

Also known as Parquet, Parquet File

A file type that keeps data by column. Most data lakes use it.

Think of it like

It is like a tool box with a drawer for each kind of tool. You open just the drawer you need.

Example

A team turns a 10 GB CSV of orders into Parquet. The file shrinks to about 2 GB, and their queries run many times faster.

Why it matters

Almost every data tool reads and writes it. Picking it for lake files is rarely wrong.

Definition

Apache Parquet is an open file format that stores data by column. A file is split into row groups, and each group stores its columns apart. Each column chunk keeps small notes, such as its smallest and largest value. Readers use those notes to skip chunks they do not need. It is the most common file format in data lakes.

Example code

python
import pandas as pd
df = pd.read_parquet("sales.parquet", columns=["month", "total"])
Read just two columns. The rest of the file is never loaded.
Same table, two layouts. A sum of totals reads one block in the column layout.

Common mistake

Writing thousands of tiny Parquet files. Each file has a cost to open, so a few large files read faster.

Key takeaways

  1. 1
    By column. Each column is stored on its own.
  2. 2
    Skips chunks. Min and max notes let readers skip.
  3. 3
    Everywhere. Nearly every data tool reads it.

Compare