Categorical Traps: Missing Categories and Broken Sorting
You group by customer tier. "Gold" customers don't appear in the results because there are zero this month.
Or your report shows Bronze → Gold → Platinum → Silver. Wait, what?
The problem: Categories inferred from data miss empty groups and sort alphabetically, not logically.
The fix: Define categorical dtype upfront with proper order.
The rule:
Define categories upfront with CategoricalDtype Use observed=False in groupby to include empty categories Use .reindex() to show zeros in pivot tables Set ordered=True for logical sorting
Karina Data Scientist
Categorical Traps: Missing Categories and Broken Sorting
You group by customer tier. "Gold" customers don't appear in the results because there are zero this month.
Or your report shows Bronze → Gold → Platinum → Silver. Wait, what?
The problem: Categories inferred from data miss empty groups and sort alphabetically, not logically.
The fix: Define categorical dtype upfront with proper order.
The rule:
Define categories upfront with CategoricalDtype
Use observed=False in groupby to include empty categories
Use .reindex() to show zeros in pivot tables
Set ordered=True for logical sorting
1 week ago | [YT] | 22