Testing RNG Uniformity: Visual Checks, Statistical Tests, and Sample Sizes

Uniformity ensures each outcome is equally likely—critical for fairness and valid simulations. Here’s a practical toolkit to catch bias.

Visual Checks

• Histograms: Compare observed frequencies to expected counts across buckets. Look for smoothness and symmetry.
• QQ plots: Compare quantiles of your sample to the target distribution (uniform[0,1] or normal after transform).
• Runs plots: Plot successive outputs to spot clumping or periodicity.

Statistical Tests

• Chi‑square goodness‑of‑fit: Discrete buckets; compare observed vs expected counts. Needs adequate expected counts (≥5 per bin).
• Kolmogorov–Smirnov (KS): Continuous case; compares empirical CDF to the target CDF.
• Anderson–Darling: Like KS but more sensitive to tails.

Practical Procedure

1. Generate large samples (e.g., 100k values) from your RNG.
2. For integers, pick ranges that align with your use case; for floats, transform to [0,1).
3. Bin and run chi‑square (discrete) or KS/AD (continuous); compute p‑values.
4. Repeat with multiple seeds and check for consistent failures.

Common Pitfalls

• Too few samples: small datasets won’t reveal subtle bias.
• Poor binning: uneven bins distort chi‑square results.
• Re‑using a failing seed to “prove” bias; verify across many seeds.

FAQs

How many samples do I need? Thousands for coarse checks; hundreds of thousands for subtle issues. Aim for ≥5 expected counts per bin in chi‑square.