# Apache Arrow

> A shared way to hold tables in memory. Tools can swap them fast.

**Field:** Data Engineering  
**Category:** Storage & Warehousing  
**Also known as:** Apache Arrow Format, PyArrow, Arrow Columnar Format

## Think of it like

It is like shops that all use the same boxes. Goods move from shop to shop without being unpacked.

## Definition

Apache Arrow is an open standard for holding table data in memory, column by column. It works the same in Python, Java, Rust and more. When two tools both use Arrow, they can pass data at little or no cost. There is no slow step to convert it first. Parquet is for storing data on disk, and Arrow is the layout in memory.

## Example

A Polars script hands a table to DuckDB. Both speak Arrow, so the 10 million rows move across without being converted.

## Why it matters

Converting data between tools can take longer than the work itself. A shared layout removes that cost.

## Common mistake

Thinking Arrow is a file format like Parquet. Arrow is for memory. Parquet is for disk.

## Three principles

- **In memory.** Not mainly a file on disk.
- **Columnar.** Each column kept together.
- **Shared.** Many languages, one layout.

## Related terms

- [Columnar Storage](https://learn.benibix.com/term/data/columnar-storage)
- [Apache Parquet](https://learn.benibix.com/term/data/apache-parquet)
- [Polars](https://learn.benibix.com/term/data/polars)
- [DuckDB](https://learn.benibix.com/term/data/duckdb)
- [pandas](https://learn.benibix.com/term/data/pandas)
- [DataFrame](https://learn.benibix.com/term/data/dataframe)

## Told apart from

- [Apache Arrow vs Apache Parquet](https://learn.benibix.com/compare/apache-arrow-vs-apache-parquet): People think Apache Arrow is a file format like Parquet, and it is not. Parquet is how table data is kept on disk, packed small. Arrow is how table data is laid out in memory while tools work on it. They work well as a pair. A tool reads Parquet from disk into Arrow in memory.

![The same three-row table stored two ways. In row storage each row's id, city and total sit side by side on disk. In column storage all the ids sit together, then all the cities, then all the totals, so a query for totals reads one block.](https://learn.benibix.com/diagrams/row-vs-column-storage.svg)

---

Source: https://learn.benibix.com/term/data/apache-arrow
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-16
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
