The Shelf Got Worse, the Library Got Better

Imagine your school library moves the "Books You Might Like" shelf to the middle of the hallway and makes it big and colorful. Way more kids walk past it and see it. But fewer of the kids who see it actually pick up a book from that shelf.
So that shelf got worse at its one job.
But here's the weird part: the library overall did better. Kids stayed longer, wandered into other sections, and checked out more books total — just not from that shelf.
Now: do you keep the new shelf, or move it back?
Two stories that both sound true
- Story A: it's broken. The shelf annoys kids, they walk away from it, and they happen to bump into other stuff. The good numbers are luck, not a good design.
- Story B: it's fine, maybe good. The shelf isn't the point — kids reading more is the point. The shelf nudges kids deeper into the library instead of parking them in one spot. Mission accomplished, just not the way we expected.
The same numbers support both. That's what makes it hard.
How they broke the tie
They stopped staring at the big total and asked three sharper questions:
- Is the extra reading happening specifically among kids who saw the shelf and skipped it? If it's happening to everybody, then the shelf isn't what caused it — something else is.
- Are kids reading more, or just reading elsewhere? Same amount of reading moved to a different room isn't a win, it's a shuffle.
- Where are they ending up? If they drift to sections they actually wanted, great. If they drift to the section where the library charges a rental fee, that's making money off frustration, not off doing a better job.
The answers mostly pointed to Story B — but not perfectly. So they kept the new shelf and set a long-term alarm: if kids start coming to the library less often over the next few months, revisit the decision.
The key concepts, named
| Concept | What it means |
|---|---|
| Local metric vs. guardrail metric | The local metric is "did people click this one thing." The guardrail is "is the whole system healthy." When they disagree, the guardrail is usually the better guide — because that's the actual goal, and the local metric was only ever a stand-in for it. |
| A proxy is not the goal | Shelf clicks were never the point. Kids reading was the point. |
| Sign vs. mechanism | "The number went up" tells you the direction. It doesn't tell you why. Shipping on direction alone means betting the reason is a good one without checking. |
| Additive vs. reallocated | Did you create value, or just move it around? A total that looks bigger can be the same pie, re-sliced. |
| Fungible attention | On a browse-y platform (or a library), attention flows between places, so one spot losing isn't automatically a loss. But you have to verify that flow in your data, not assume it because the business "feels" exploration-shaped. |
| Lagging guardrail | Some damage shows up months later — people quietly stop coming. So "shipped" shouldn't mean "case closed." |
The one-sentence takeaway: when the small metric and the big metric fight, the big metric usually wins — but only after you've figured out why it won.
SQL Growth