The $28 Billion Habit of Not Checking Your Inputs

By Your Health Magazine Contributor

Your Health Magazine Contributor

. https://YourHealthMagazine.net

More Health News & Research Articles

The $28 Billion Habit of Not Checking Your Inputs

Every organization audits its results. Quarterly numbers get reviewed three times before anyone outside finance sees them. Campaign performance gets picked apart line by line. A bad hire gets a post-mortem, while a bad launch gets two of them. However, almost nobody audits what went in at the start.

Science has put a price on that habit: A 2015 analysis in PLOS Biology by Leonard Freedman, Iain Cockburn and Timothy Simcoe estimated that the United States spends about $28 billion a year on preclinical research that cannot be reproduced. That figure is a model rather than a measurement, built on the conservative assumption that half of preclinical work fails to replicate, and the authors are open that the real total could sit well above or below it. 

The breakdown matters more than the total. The largest single cause sat outside the reasoning altogether: biological reagents and reference materials, the physical material researchers ordered, received and assumed was what the label said. That category accounted for 36% of the irreproducible spend. Study design, the part scientists are trained to interrogate hardest, came second at 28%.

A Number Worth Sitting With

The pattern shows up wherever anyone has looked for it. In 2012, C. Glenn Begley and Lee M. Ellis reported in Nature on a decade of internal work at Amgen, where a team had tried to reproduce 53 preclinical cancer papers the company considered landmarks. The findings held up in six of them, or 11%. 

The papers were picked for novelty rather than at random and Amgen never identified them, so the number describes one company’s experience instead of a measured failure rate for the field. A 2016 Nature survey of 1,576 self-selected researchers pointed the same way: 52% said science had a significant reproducibility problem, and more than 70% reported having failed to reproduce another scientist’s result.

These were not careless people. They held doctorates, worked in funded labs, followed published protocols and submitted to peer review. The defect entered upstream, in a vial or a cell line or a reference standard, and everything built on top of it inherited a flaw that no amount of downstream rigor could detect. You cannot reason your way out of a bad ingredient. You can only find it earlier or find it later.

Why Careful People Accept Uncertain Inputs

Verification runs into three forces in almost every organization.

Checking looks like an accusation

Asking a supplier, a colleague or an analyst to prove their number reads as doubt about their competence. Most professional cultures reward smooth handoffs over awkward ones, so the question goes unasked and the trust gets extended on the strength of the relationship rather than the evidence.

Verification produces nothing to show

A batch that passes inspection generates no story. A forecast whose source assumption turns out to be sound generates no meeting. The work is invisible when it succeeds and only becomes visible as the disaster it quietly prevented, which nobody can see either. Budgets and calendars are built around visible output, and verification has none.

The bill arrives too late to trace

Freedman’s $28 billion arrives as thousands of small invoices, two or three years behind the decisions that produced them, by which point the line between an unchecked input and a wasted quarter is almost impossible to draw. Consequences that arrive late and untraceable never teach anyone anything.

What Research Suppliers Had to Start Documenting

The life sciences responded to this with paperwork, which sounds like the least interesting fix available and turned out to be the most useful one. A Certificate of Analysis is a lot-specific record of what was measured on the material that shipped, not a claim about what the product generally is. 

At research grade, the core of it is two numbers: identity confirmed by mass spectrometry and purity quantified by high performance liquid chromatography, tied to a lot number and a date. Other tests exist, including water content, residual solvents and net content of the active compound, and they are standard on pharmaceutical-grade material but usually optional extras further down the market. 

Most research certificates are issued in-house by the seller, which is the weak point, and some suppliers of research-grade compounds now publish independent third-party purity results by lot in the catalog listing rather than holding them for buyers who think to ask. Strip away the chemistry and the underlying idea transfers to any input in any business. An input deserves trust when three conditions hold:

• Someone measured it, and that somebody was not the party with an interest in the result
• The record states what was measured and, by omission, what was not
• The record is tied to the specific batch, version or period you received, not to a general standard the seller aspires to

Most business inputs fail all three and get used anyway.

The Same Gap in Ordinary Work

Once you start looking for unverified inputs outside a laboratory, they are everywhere:

• A market research dataset whose sample size, recruiting method and fielding dates are not stated anywhere in the deck built on top of it
• Competitor pricing pulled from a spreadsheet that was last updated 18 months ago by someone who has since left
• A five-year revenue model resting on a growth rate that nobody in the room can source
• Candidate credentials that three interviewers discussed and no one confirmed
• A vendor certification that expired two years ago and still sits in the procurement file
• An AI-generated summary of a 40-page contract that no human opened

Each of these is cheap to check at the moment it enters and expensive to check after 6 months of work has been layered on top. The cost curve is the same one the reproducibility literature describes. Verification is a fixed cost paid early or a compounding cost paid late.

4 Questions Before You Trust an Input

The practical version of this fits on an index card. Before an input becomes the foundation for anything, ask:

1. Who produced this measurement, and did they have a stake in the answer? Self-reported numbers are not worthless, but they belong in a different category than independently produced ones, and the categories should not be mixed without saying so.
2. What exactly was measured, and what was left out? A purity figure of 99% means nothing without knowing what the remaining 1% is and whether anyone looked. The same holds for a satisfaction score, an uptime figure or a pipeline number.
3. Does this record apply to what I received? A test result from a different batch, a different quarter or a different software version is a reference point, not evidence about the thing in front of you.
4. If this were wrong, when would I find out? This is the question that sets how much checking is worth doing. An input whose failure surfaces in a week needs less scrutiny than one whose failure surfaces in two years, after the decision is irreversible.

The Work Nobody Applauds

There is no recognition in this. No one is promoted for the contaminated batch they caught in week one, because the project that did not fail leaves no evidence behind. The reward for verification is a quarter that proceeds normally, which is indistinguishable from luck.

That is what makes it a matter of character rather than process. Any organization can write a verification step into a checklist. Whether anyone performs it honestly when the deadline is tight, the supplier is a friend and the number is probably fine depends on people who have decided that being right matters more than looking fast.

The Amgen team found six papers out of 53 that held up. Nobody published the list, and why the rest failed was never fully sorted out. What the wider literature does say is where to look first, and the answer is rarely the reasoning. It is the material, the datasets and the assumptions that arrived before anyone started thinking hard.