Skip to content
BENIBIX Learn

Data Profiling

Also known as Data Profile, Column Profiling

Counting what is really in a table: blanks, values and ranges. It comes before checks.

Think of it like

It is like a check-up at the doctor. First you get the facts, then you decide.

Example

Jack profiles a new customer table of 10,000 rows. 312 emails are blank, and one fake email shows up 240 times. He writes two quality checks from what he found.

Why it matters

Checks written without a profile guess at the data. A profile shows the real problems worth a check.

Definition

Data profiling means looking at data to learn what is really in it. A profile counts the rows, the empty cells and the different values in each column. It shows the smallest, the largest and the most common values. You do it before you trust a new table or write checks for it. A profile describes the data. It does not judge it.

Example code

sql
SELECT COUNT(*), COUNT(email), COUNT(DISTINCT email)
FROM customers;
-- 10000 | 9688 | 9410
Rows, filled emails and different emails, in one query. So 312 emails are blank.
The left half is a profile. It counts what is inside one column and finds the fake email.

Common mistake

Trusting an old profile forever. Data changes, so run it again when the source does.

Key takeaways

  1. 1
    Measure first. Count before you judge.
  2. 2
    Every column. Blanks, values, ranges and top values.
  3. 3
    Feeds the checks. Each surprise becomes a quality rule.

Compare