⚙️Data Skew
Also known as Partition Skew, Skewed Data, Key Skew
When one part of a split job gets far more rows than the rest, so all wait.
Think of it like
It is like five friends sharing chores, but one gets all the dishes. Everyone waits for that one friend.
Example
A job groups sales by country on 50 machines. 80% of the sales come from one country, so one machine does most of the work while 49 wait.
Why it matters
It is a common reason a big job runs for hours instead of minutes. Adding machines does not help.
Definition
Common mistake
Key takeaways
- 1Uneven parts. One key holds most of the rows.
- 2Slowest part wins. The job ends when the last part ends.
- 3Split the busy key. Add a random suffix to spread it out.