Skip to content
BENIBIX Learn

Data Uniqueness

Also known as Uniqueness (Data Quality), Duplicate Records

Whether each thing is in the data just once.

Think of it like

Each child in class has one name tag. If two tags say Emma, someone gets counted twice.

Example

A retried load adds the same 500 orders a second time. A uniqueness check on order_id catches it before the report runs.

Why it matters

Copies make every count and total too big. Sales look higher than they were.

Definition

Data uniqueness is whether each real thing, such as an order or a customer, appears only once. Every order should have one row, and every customer one record. Copies creep in when a load runs twice or two systems hold the same person. A check groups rows by their key and looks for any key that appears more than once. It is one of the easiest ones to test.

Example code

sql
SELECT order_id, COUNT(*) AS copies
FROM orders
GROUP BY order_id
HAVING COUNT(*) > 1;
Any row this returns is an order that appears more than once.
Uniqueness is the fifth tile. The same order sits in the table twice.

Common mistake

Testing a made-up key that is new on every row. Test the real key, such as the order number.

Key takeaways

  1. 1
    Once per thing. One row for each real item.
  2. 2
    Check the key. Group by the business key.
  3. 3
    Easy to test. One query finds every copy.