stratified
Stratified Splitting is a data partitioning technique that preserves the class distribution of the original dataset in every split — ensuring that if 5% of the full dataset is fraudulent, both the training set and test set contain approximately 5% fraud cases, preventing the dangerous scenario where a random split accidentally concentrates all rare examples in one partition and leaves the other with none, which would make evaluation unreliable or training ineffective.
What Is Stratified Splitting?
- Definition: A splitting strategy that samples from each class proportionally when dividing data into train/test (or K folds) — guaranteeing that the class distribution in each partition mirrors the original dataset.
- The Problem: With imbalanced data (99% negative, 1% positive), a random 80/20 split might produce a test set with 0 positive examples — making accuracy, precision, and recall impossible to evaluate. Even with balanced data, random splits can create misleading class distributions.
- When It's Critical: Any classification task where class proportions matter — which is essentially every classification task.
Random vs Stratified Split
| Scenario | Random Split (Test Set) | Stratified Split (Test Set) |
|---|---|---|
| Original: 95% Neg, 5% Pos | Could be 100% Neg, 0% Pos ⚠️ | ~95% Neg, ~5% Pos ✓ |
| Original: 50% Cat, 50% Dog | Could be 60% Cat, 40% Dog | ~50% Cat, ~50% Dog ✓ |
| Original: 80%A, 15%B, 5%C | Could lose all C examples | ~80%A, ~15%B, ~5%C ✓ |
Stratified K-Fold Cross-Validation
| Fold | Class A (Majority) | Class B (Minority) | Proportion Preserved? |
|---|---|---|---|
| Fold 1 (Test) | 190 | 10 | 95%/5% ✓ |
| Fold 2 (Test) | 190 | 10 | 95%/5% ✓ |
| Fold 3 (Test) | 190 | 10 | 95%/5% ✓ |
| Fold 4 (Test) | 190 | 10 | 95%/5% ✓ |
| Fold 5 (Test) | 190 | 10 | 95%/5% ✓ |
Python Implementation
from sklearn.model_selection import (
train_test_split, StratifiedKFold, StratifiedShuffleSplit
)
# Stratified train/test split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, stratify=y, random_state=42
)
# Stratified K-Fold
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
for train_idx, test_idx in skf.split(X, y):
X_train, X_test = X[train_idx], X[test_idx]When Stratification Matters Most
| Scenario | Risk Without Stratification | Impact |
|---|---|---|
| Rare disease detection (0.1% positive) | Test set might have 0 positive cases | Cannot evaluate recall at all |
| Multi-class with rare classes | Minority class absent from some folds | Cross-validation scores unreliable |
| Small datasets (<500 examples) | Class proportions easily skewed by randomness | Misleading train/test performance gap |
| Highly imbalanced (>20:1 ratio) | Random split virtually guaranteed to misrepresent minority | Unstable evaluation metrics |
Stratified Splitting is the essential data partitioning technique for classification tasks — guaranteeing that class proportions are preserved in every train/test split and cross-validation fold, preventing the evaluation failures and training biases that random splitting causes when class distributions are imbalanced or datasets are small.