Natural Intelligence

Meta / self-correction · Principle 14

Measure What You Want More Of

Whatever a system tracks and rewards is what it will get more of, which means choosing a metric is choosing a destination, whether or not that destination was the intended one.

Try this

Explain it to a child

Imagine a teacher says, "Whoever raises their hand the most gets a gold star." Pretty soon, some kids might start raising their hands even when they don't actually have a good answer, just to get the star. The teacher wanted kids who were thinking hard and participating, but the gold star ended up just measuring "who raises their hand," which isn't quite the same thing anymore. This happens all over the place, not just in classrooms, whenever you reward a number instead of the real thing you actually care about, people (and even some animals, like baby cuckoo birds tricking other birds into feeding them!) find clever ways to get the number up without actually doing the real thing. That's why it's smart to check every so often: is this number still telling us what we actually wanted to know?

What it means

Systems tend to optimize for whatever they measure, regardless of whether that measurement genuinely reflects what the system's designers actually care about. GDP, test scores, engagement metrics, all became targets that shaped real-world behavior, sometimes in directions nobody actually wanted. This principle exists to test whether a system's chosen metrics genuinely reflect its real goals, or have quietly become goals in their own right.

Where this comes from

Open what you want. Nothing below is needed to use the principle; it is here because a claim without its working is an assertion.

In nature

Brood-parasitic cuckoos offer a striking natural example of exactly the failure mode Goodhart's Law describes, playing out through evolution rather than economics. Host bird parents never directly assess "is this actually my genetic offspring", that would be far too costly to determine in the moment. Instead, they evolved a proxy for it: feed whatever chick begs loudest and most intensely, since in an honest system, begging intensity reliably correlates with genuine hunger and genuine offspring. Cuckoo chicks exploit precisely this proxy-target gap. Research shows a single cuckoo chick's rapid begging call can be so intense it stimulates host parents to feed at the same rate they would for an entire brood of their own chicks, described in the research literature as a "supernormal stimulus," a signal that doesn't just mimic the natural cue but exceeds and overrides it. Multiple cuckoo lineages have independently evolved to closely mimic the specific begging calls of their particular host species, and some species can even adjust the pitch or duration of their calls to different hosts. The parents' measurement system (loud begging = feed this one) was never actually measuring "this is my chick", it was measuring a proxy, and evolution found and exploited the gap between the proxy and the real target, exactly the way Goodhart's Law predicts happens whenever a measure becomes the actual target of optimization, whether by a cuckoo or a call-center employee.

What the science says

Goodhart's Law, formulated by economist Charles Goodhart in 1975 in the context of monetary policy and later generalized by anthropologist Marilyn Strathern into its now-famous form, "when a measure becomes a target, it ceases to be a good measure", describes a structural problem directly relevant to this principle's own claim. A closely related formulation, Campbell's Law (Donald Campbell, sociologist, 1976/1979), states that the more a quantitative social indicator is used for decision-making, the more it becomes subject to corruption and distortion of the very process it was meant to track.

This isn't a rare edge case; documented real-world examples span government, business, and everyday operations: U.S. Postal Service contractors reportedly told drivers to scan still-undelivered packages as "delivered" just before a deadline to hit on-time metrics; airlines paid over $80 million in fines for falsifying on-time mail delivery data; call center workers rewarded for call volume have been documented ending calls prematurely, technically hitting the metric while degrading the actual service the metric was meant to represent. The core mechanism is consistent across all these cases: a metric is originally chosen as a proxy for something harder to measure directly, but once people are rewarded or evaluated based on the proxy itself, they optimize the proxy, not the underlying reality it was standing in for.

Ancient wisdom

Confucius, in the Analects (13.3), developed an explicit philosophical doctrine, zhengming, "the rectification of names", arguing that when the words and measures a society uses no longer accurately correspond to reality, the entire system built on top of them fails, cascading outward from language into action. Asked what he would prioritize first in governing a state, Confucius answered "rectifying names," explaining the mechanism directly: if names (terms, designations, measures) are not correct, speech will not proceed smoothly; if speech fails, affairs will not be successfully accomplished; if affairs fail, social ritual and order will not flourish; and if that order collapses, "punishments and rewards will not be appropriate", meaning the entire chain of governance, from language to consequence, breaks down starting from an initial mismatch between a name (or measure) and what it's actually supposed to represent.

This maps with striking precision onto the modern problem Goodhart's and Campbell's Laws describe: a "ruler" who no longer actually governs like a ruler, or a metric that no longer actually measures what it was named to measure, produces the same structural failure, a growing gap between label and reality that eventually corrupts everything built on top of the mismatch. Confucius's solution wasn't abandoning measurement or naming altogether; it was insisting that names and measures be continuously checked against what they actually correspond to in practice, and corrected, "rectified", the moment that correspondence breaks down, rather than left to drift while everyone continues acting as if the label still means what it originally meant.

History

Bhutan's Gross National Happiness (GNH) index offers a rare real-world case of a government deliberately choosing to measure something other than the default economic metric, precisely because it recognized what GDP was, and wasn't, capturing. The concept was introduced in 1972 by Bhutan's fourth king, Jigme Singye Wangchuck, who declared GNH "more important than Gross Domestic Product," explicitly rooted in Bhutan's Buddhist heritage. Where GDP measures only monetary economic output, a metric that famously counts a car crash or oil spill as a net positive, since repair and cleanup both add to production, GNH is built from 33 indicators across nine domains, including psychological wellbeing, health, education, and ecological resilience, deliberately designed to track what the government actually wanted more of, not merely what was easiest to count in currency. The often-repeated observation, attributed to Robert Kennedy, that GDP measures "everything except that which makes life worthwhile," captures the exact problem GNH was built to correct.

The honest complication worth including: GNH itself became a contested political football during Bhutan's transition to democracy in 2008 and 2012, with genuine debate over whether the framework should prioritize domestic wellbeing improvements or serve as an international model, a reminder that even a well-designed alternative metric doesn't escape the political and definitional struggles the original metric was meant to solve. GNH still operates alongside GDP in Bhutan's own policy-making, not as its full replacement, and the 2011 UN resolution Bhutan sponsored, calling for wellbeing-inclusive development globally, has influenced other countries (New Zealand, Scotland, Finland) without displacing GDP as the dominant global metric anywhere. Choosing a better metric, in other words, was a necessary step, but not by itself a sufficient one, the surrounding political and institutional will to actually act on what the better metric reveals mattered just as much.

In practice

Individual scale

People who set a specific, measurable personal goal (steps walked, calories tracked, pages written) commonly report the metric itself starting to distort the underlying behavior over time, hitting a step count by pacing in place rather than genuinely being more active, for instance, illustrating that Goodhart's Law applies at the individual level, not only to institutions.

Organizational scale

Companies that shift performance evaluation toward a single, easily-gamed metric (raw sales volume, lines of code written, tickets closed) frequently report employees optimizing specifically for that number in ways that technically satisfy it while undermining the actual goal it was meant to represent (aggressive, low-quality sales; bloated, poorly-written code; tickets closed without genuinely resolving the customer's problem), a well-documented pattern across many industries, not a hypothetical risk.

Policy/civic scale

Bhutan's own experience with GNH, already noted in History, illustrates a genuine attempt to design around this exact problem at national scale: rather than a single number, GNH uses 33 separate indicators across nine domains specifically to make the metric harder to game by optimizing any one number in isolation, since improving one domain (say, economic output) at the direct expense of another (ecological or psychological wellbeing) shows up clearly as a trade-off rather than disappearing into a single aggregated score.

What argues against it

This principle's own positive claim contains its most serious complication built directly inside it: Goodhart's Law and Campbell's Law both describe exactly what happens once "measure what you want more of" is taken too literally, the moment a metric becomes the explicit target people are evaluated or rewarded against, people (and, as the cuckoo example shows, evolutionary pressure itself) optimize the metric rather than the underlying reality it was meant to represent. Real, well-documented cases span government and industry: postal workers reportedly scanning still-undelivered packages as delivered just before a deadline; airlines paying over $80 million in fines for falsified on-time delivery records; call center employees ending calls prematurely to hit call-volume targets while degrading actual service quality.

This means the principle can't simply be "pick a metric for what you want and reward people for hitting it", that's precisely the setup Goodhart's Law shows backfiring. The research on mitigating this (rather than eliminating measurement altogether, which isn't a real option for any system that needs to know whether it's succeeding) converges on a few consistent approaches: regularly revisiting whether the metric still actually reflects the real goal rather than assuming it always will; using multiple, deliberately diverse metrics that can catch each other's blind spots (Bhutan's 33-indicator approach is a direct example); and treating any metric as provisional and open to correction rather than a fixed, permanent target, which connects this principle directly back to Principle 15's core claim that every system needs a working mechanism to notice and correct its own drift, including drift in what it's choosing to measure in the first place.

Where that leaves us

Across Bhutan's Gross National Happiness framework, Confucius's zhengming doctrine, cuckoo brood parasitism, and Goodhart's and Campbell's Laws, a single structural tension runs through this entire principle: choosing what to measure is one of the most powerful levers a system has, because whatever gets measured and rewarded shapes real behavior, but the moment a measure becomes a fixed target rather than a genuine indicator, the system starts optimizing the measurement itself, and the connection between the measure and the reality it was meant to represent quietly breaks.

This principle's own honest complication is not a minor footnote, it's arguably the central finding. "Measure what you want more of" is true and useful advice at the moment a metric is first chosen thoughtfully (as Bhutan did, deliberately correcting for what GDP was missing), but it decays into exactly the failure Confucius warned against, a name detached from the reality it names, the longer that metric goes unexamined and the more power gets attached to hitting it. Diverse, multiple metrics that check each other; regular, honest revisiting of whether the metric still fits; and treating measurement as a living, correctable process rather than a fixed target all emerge, across every domain examined here, as the actual defense against this decay, not choosing a perfect metric once and trusting it indefinitely, which no metric can survive intact.

Open questions
  • Bhutan's 33-indicator GNH approach tries to resist gaming through multiplicity rather than a single number. Is there a general principle for how many, and which kinds, of diverse metrics genuinely resist Goodhart's Law, or does any sufficiently important combined metric eventually get gamed as a whole, the way a single one does?
  • The cuckoo example shows that even evolution, an unplanned, purely selection-driven process, finds and exploits the gap between a proxy measure and the real target it stands for. Does this suggest gaming a metric is close to an inevitable outcome for any sufficiently high-stakes measurement system, given enough time and pressure, regardless of how carefully the metric is initially designed?
  • Confucius's zhengming was aimed at correcting language and social roles, not quantitative metrics specifically. Does his proposed remedy (continuously checking correspondence between name and reality, and correcting when they diverge) actually transfer cleanly to modern quantitative measurement, or does numerical measurement have its own distinct failure modes that a 2,500-year-old philosophical framework doesn't fully anticipate?
  • This principle and Principle 15 (self-correction) are deeply intertwined, a metric needs correction, and correction needs a way to detect when the metric itself has gone wrong. Is there a risk of infinite regress here (what metric tells you your metric-correction process itself is working), or is there a natural place this chain of correction can reasonably stop?
Evidence and references

Draft. Several citations here are secondhand and are being checked against the primary sources. Where the underlying science is contested, the dispute is described rather than settled.

History

  • Bhutan's Gross National Happiness, Wikipedia, "Gross National Happiness"; Springer Nature, "National progress, sustainability and higher goals: the case of Bhutan's Gross National Happiness"; International Relations Review, "Measuring What Matters: Bhutan and the Politics of Happiness"; DevelopmentAid, "Gross National Happiness: an alternative way to measure progress?"

Modern Science

  • Goodhart's Law and Campbell's Law, Goodhart, C., 1975, generalized by Strathern, M., 1997; Campbell, D., "Assessing the impact of planned social change," Evaluation and Program Planning, 1979; CNA Corporation, "Goodhart's Law: Recognizing & Mitigating Manipulation in Measures for Analysis," 2022 (USPS/Amazon and airline examples); Psych Safety, "Goodhart's Law, Campbell's Law, and the Cobra Effect"

Nature

  • Cuckoo brood parasitism and supernormal begging stimuli, Davies, N. et al., "Nestling cuckoos, Cuculus canorus, exploit hosts with begging calls that mimic a brood," Proceedings of the Royal Society of London B, 1998; Current Zoology, "Imperfect mimicry of host begging calls by a brood parasitic cuckoo"; PMC, "Supernormal Stimulus Begging Calls of Brood-Parasitic Nestlings Depress the Parental Care in an Uncommon Host"

Ancient Wisdom

  • Confucius and zhengming (rectification of names), Analects 13.3; Grokipedia, "Rectification of names"; Wikipedia, "Rectification of names"; Riegel, J., cited via Warp, Weft, and Way

Related

This principle is a draft. If something here is wrong, or a source does not say what we say it says, tell us; that is the fastest way it gets better.

All 15 principles