One option can have a higher success rate in each subgroup and a lower rate overall. The arithmetic is consistent: the options may face very different mixtures of easy and difficult cases. Before interpreting an aggregate comparison, inspect the counts and ask what question the combined rate answers.
An original fictional dataset
Imagine two document-processing workflows. The cases and outcomes below are invented; they are not Ithy performance measurements or results from a real experiment. Download the four-row CSV.
| Case group | Workflow A | A success rate | Workflow B | B success rate |
|---|---|---|---|---|
| Easy | 90 successes / 100 cases | 90% | 19 / 20 | 95% |
| Difficult | 2 / 10 | 20% | 30 / 100 | 30% |
| Combined | 92 / 110 | 83.64% | 49 / 120 | 40.83% |
B has the higher rate in each group. A has the higher combined rate because most of A's cases are easy, while most of B's are difficult. The combined rate is weighted by each workflow's own case mix, not a simple average of its two percentages.
Expose the weights
A: (100/110 × 90%) + (10/110 × 20%) = 83.64%
B: (20/120 × 95%) + (100/120 × 30%) = 40.83%
As a separate descriptive calculation, give easy and difficult cases equal weight for both options:
A at a 50:50 mixture: (90% + 20%) / 2 = 55%
B at a 50:50 mixture: (95% + 30%) / 2 = 62.5%
That answers “what do these observed subgroup rates give under this chosen common mixture?” It does not establish the outcome of assigning new cases to either workflow. The 50:50 choice is an illustrative standard, not a discovery that the real population has that mixture.
Why the reversal is called a paradox
E. H. Simpson's 1951 paper concerns interpreting interaction in contingency tables. The phenomenon illustrates how combining categories can change an association. Our numbers are original examples, not a reproduction of that paper's table.
There is no arithmetic contradiction between the subgroup and combined results. They summarize different mixtures. The error comes from treating them as answers to the same question without checking how cases entered the groups.
Subgroup adjustment is not automatically a causal answer
The table does not reveal how workflows were assigned, how difficulty was defined, whether other variables differ or how uncertainty should be estimated. Simply controlling for every available column can also answer the wrong question if the variable is an effect of the process being studied rather than a relevant pre-existing difference.
For an operational decision, specify the target case mix and investigate the assignment process. A suitable experiment or justified causal design requires more than these four rows. Do not use this example to claim either workflow causes better outcomes.
A checklist for an aggregate comparison
- Write the numerator and denominator behind each rate.
- Check that “success” has the same definition and observation period.
- Inspect relevant subgroup counts and how the case mix differs.
- State whether the goal is description, prediction or a causal decision.
- If standardizing, disclose the common weights and why they fit that goal.
- Keep uncertainty, missing cases and selection effects visible.
The correlation and denominators guide applies a related check to support-ticket data. The interval guide explains another common problem: using uncertainty about a mean to describe individual outcomes.