genStats: Standardizing Experimentation Analysis at Scale

CarGurus developed genStats, an internal Python framework standardizing A/B test analysis across product domains by automating Jupyter notebook generation, centralizing statistical logic, and packaging domain-specific metrics into reusable metric families. Analysts input test metadata via CLI to generate ready-to-run notebooks with pre-built SQL queries and statistical calculations, backed by Poetry-managed environments and automated report generation. Impact includes reducing setup time from hours to minutes, consistent lift calculations, and improved analyst onboarding.