Skip to content
BENIBIX Learn

Great Expectations (GX)

Also known as GX, GX Core, Great Expectations Library

A free Python tool that tests data. It comes with lots of checks.

Think of it like

It is like a list of promises your data makes. Then someone checks each promise.

Example

A team runs GX on each new file from a partner. A file with a quantity below zero is sent back before it gets to the warehouse.

Why it matters

It gives Python pipelines one shared way to write and report data tests. The results also show what the data should look like.

Definition

Great Expectations is a free, open source Python library for testing data. Each check is called an expectation, such as "values in this column are not null". It ships with dozens of ready-made ones. You run a set of expectations against a batch of data and get a pass or fail for each. It can also write the results as a web page for people to read. A paid cloud version adds a shared workspace.

Example code

python
import great_expectations as gx

context = gx.get_context()
source = context.data_sources.add_pandas("pandas")
asset = source.add_dataframe_asset(name="orders")
batch_def = asset.add_batch_definition_whole_dataframe("all")
batch = batch_def.get_batch(batch_parameters={"dataframe": df})

check = gx.expectations.ExpectColumnValuesToBeBetween(
    column="quantity", min_value=1, max_value=100)
print(batch.validate(check).success)
Point GX at a DataFrame, then test that every quantity is between 1 and 100.
GX lives at the gate on top. Each rule there is one expectation.

Common mistake

Letting it scan the data and keeping every rule it suggests. Many are too strict and fail on normal days.

Key takeaways

  1. 1
    Expectations. Each check is a named promise.
  2. 2
    Ready-made library. Dozens of common checks built in.
  3. 3
    Readable results. Reports people can open and read.