---
title: Ronny Kohavi — who he is | RGM® Glossary
url: https://realgrowthmatters.com/glossary/ronny-kohavi/
updated: 2026-06-10
source_html: https://realgrowthmatters.com/glossary/ronny-kohavi/
---

# Ronny Kohavi

proper noun

Most of your winning tests are false alarms — Kohavi built the machinery that catches them.

Name
:   Ronny Kohavi

Built
:   Microsoft's ExP platform

Key work
:   Trustworthy Online Controlled Experiments (2020)

Earlier
:   Amazon data mining, Airbnb VP

## Forms & parts of speech

Kohavi · proper noun

Scientist; shorthand for rigorous testing.

"Apply the **Kohavi** standard — would this result survive an A/A test?"

## Who he is, in plain terms

Ronny Kohavi is the computer scientist who scaled online experimentation from a tactic to an institution — director of data mining and personalization at Amazon, then founder of Microsoft's Experimentation Platform (ExP), which grew to run tens of thousands of experiments a year across Bing and Office, then VP at Airbnb. *Trustworthy Online Controlled Experiments* (2020, with Diane Tang and Ya Xu) is the field's standard text — known simply as "the experimentation book."

## The key ideas

The Overall Evaluation Criterion (OEC) — one agreed metric combination that defines success before any test runs; Twyman's law, his favorite warning — any figure that looks interesting or unusual is usually wrong, so investigate stunning wins before celebrating them; A/A tests and sample-ratio-mismatch checks to validate the instrument itself; small percentage changes are the honest norm (Bing's famous title-spacing win was rare); and institutional memory — most ideas fail, so a platform that records everything beats a hero analyst.

## Why he still matters

Every growth team that ships by experiment inherits his guardrails, usually without knowing their source. The uncomfortable statistics he publicized — that the majority of well-designed experiments at sophisticated companies fail to move their target metric — is the strongest argument for testing at all: intuition at Microsoft and Amazon scale was wrong most of the time, and yours probably is too.

**Worked example.** A team celebrates a 14% conversion lift from a checkout test. The Kohavi review finds a sample-ratio mismatch — the variant quietly broke the page for older browsers, removing their (lower-converting) traffic from the bucket. Twyman's law strikes again. The fix ships, the rerun shows an honest 1.8% lift, and the team adds automatic SRM alarms so the next false miracle gets caught by machinery instead of luck.

**Failure modes to watch.** Declaring victory on peeking and stopping tests early; celebrating implausible lifts without checking sample-ratio mismatch; and running experiments with no pre-agreed OEC so every result can be argued into a win.

## Synonyms & antonyms

### Synonyms

Ronny KohaviKohavi

## Origin & history

Born in Israel; Stanford PhD in machine learning (1995) under a lineage that produced the MLC++ library and the C4.5-era decision-tree work. The experimentation chapter began at Amazon (2003) and became Microsoft's ExP platform in 2006.

Etymology: [source](https://en.wikipedia.org/wiki/Ron_Kohavi).

## Usage trends

Search interest for this term over the last five years:

[View interest-over-time on Google Trends →](https://trends.google.com/trends/explore?q=ronny%20kohavi&date=today%205-y)

## Common questions

Who is Ronny Kohavi?
:   Computer scientist who built Microsoft's experimentation platform, led experimentation at Airbnb, and co-wrote the standard text on online controlled experiments.

What is an OEC?
:   The Overall Evaluation Criterion — the pre-agreed metric (or weighted combination) that defines an experiment's success.

What is Twyman's law?
:   Any statistic that looks interesting or unusual is most likely wrong — investigate surprising results before acting on them.

## Related tools & calculators

- tool[Experiment planner](/tools/experiment-planner/)
- tool[Funnel drop-off analyzer](/tools/funnel-drop-off-analyzer/)

## Resources & people to follow

- book*Trustworthy Online Controlled Experiments* — Kohavi, Tang & Xu
- referenceExP-platform.com — his papers
- referenceWikipedia — Ron Kohavi

Curated, non-competitor resources verified per term.

## Related training

- module[CRO & experimentation](/training/cro-experimentation/)

## Disciplines

Areas of marketing where ronny kohavi is a core concern:

[Growth strategy](/training/growth-marketing-foundations/)[Analytics](/training/marketing-analytics/)

## Read next

## Related terms

[A/B test](/glossary/ab-test/)[Statistical significance](/glossary/statistical-significance/)[Conversion rate](/glossary/conversion-rate/)[Cognitive bias](/glossary/cognitive-bias/)[North Star metric](/glossary/north-star-metric/)

## Sources

1. trends[Google Trends — "ronny kohavi"](https://trends.google.com/trends/explore?q=ronny%20kohavi&date=today%205-y)