Skip to content
BENIBIX Learn

Polars

Data⚙️ ProcessingDataFrames

Also known as Polars DataFrame, py-polars

A fast tool for tables. It plans the whole job first.

Think of it like

It is like pandas with a bigger motor. It does the same job, but all parts of the chip work at once.

Example

A daily job in pandas takes 20 minutes. Rewritten in Polars, it takes under 2.

Why it matters

It makes one laptop handle data that once needed a cluster. That saves money and time.

Definition

Polars is a free DataFrame library written in Rust, with a Python API. It stores data by column and uses all the cores of the computer at once. Its lazy mode plans the whole query before it runs, so it can skip work and read less. It can also stream data that is bigger than memory. It is often many times faster than pandas.

Example code

python
import polars as pl

out = (pl.scan_parquet("sales.parquet")
         .group_by("city").agg(pl.col("total").sum())
         .collect())
This scan is lazy. Nothing is read until the last step. Only two columns are read.

Common mistake

Calling collect() after every step. That throws away the plan that makes lazy mode fast.

Key takeaways

  1. 1
    Rust and columns. Fast at the core.
  2. 2
    Every core. Work runs in parallel.
  3. 3
    Lazy plans. See the whole query, then run.

Compare