Why the Claims Outrun the Evidence

Somewhere in the biomedical literature there is a compound with a couple of hundred papers behind it — the count depends on how you search — almost all reporting benefit, and not one controlled trial in a human being. Claims about research peptides outrun the evidence because the route a finding travels, from an animal room to a sentence somebody reads, contains no step at which anyone is obliged to say what the finding was seen in. Each stage is performed competently by people doing ordinary work, and each discards a qualifier the previous stage thought too obvious to repeat. By the end, a result obtained in a few young rodents with a surgically produced injury has become a statement about healing: present tense, no species, no subject at all.

The interesting question is not whether that happens. The interesting question is how it happens without anybody deciding that it should. The convenient answer — that somebody at the end of the chain is lying — is not wrong so much as insufficient. It explains the worst examples and none of the ordinary ones, and it cannot account for the awkward fact that the gap is already wide open in the scientific literature itself, long before anything is for sale.

The easy explanation, and why it is not enough

There is overreach at the commercial end of this, and it is not the subject of this piece. Strip out every dubious claim ever written to move a product and the gap between what is said about research peptides and what is known narrows; it does not close. The literature would still hold hundreds of papers reporting benefit in animals, the clinical column would still be empty, and a reader in good faith with nothing to sell would still come away with an impression of established effect that the evidence does not support.

The phenomenon sits upstream of commerce, then, and is worth naming as what it is: a structural property of how biomedical knowledge is produced, filtered and passed along. The distortion is not injected at any one point. It accumulates, in small and individually defensible increments, at every point.

What happens between the mouse and the person

Start with the base rate, because it is worse than most people assume and it is not a secret. In 2006 a review in JAMA took seventy-six animal studies cited heavily enough to count as influential and asked what had become of each. Roughly a third were borne out by later randomised trials in humans. About a fifth were contradicted. The largest group, nearly half, had never been tested in people at all 5. The following year a systematic review in the BMJ came at it from the other direction, comparing treatment effects directly for interventions where both animal experiments and human trials existed. Agreement was inconsistent: for around half the animal work pointed where the clinical work later went, and for the rest it did not 2.

Those are not the numbers of a scandal. They are the numbers of a hard problem, and roughly what the design of the experiments should lead you to expect. The reasons are unglamorous and specific.

None of these is a flaw. They are what makes an animal experiment interpretable: control the variation, standardise the injury, shorten the timescale, and a signal appears that would otherwise be buried. But each simplification is a step away from the situation anybody actually cares about, and the effect you measure belongs to the simplified one. Translation is not a formality at the end. It is a second, harder experiment, and mostly it has not been done.

The trouble starts before the translation does

There is a more uncomfortable layer underneath. The translation gap assumes the animal findings themselves are solid, and that assumption has taken a battering. When an industry team set out to reproduce fifty-three landmark preclinical cancer studies — papers a field had built on — they confirmed the findings in six 3. An economic analysis published in 2015 put the share of preclinical research that cannot be reproduced above one half, and estimated the United States spend on irreproducible preclinical work at roughly twenty-eight billion dollars a year 4. Money at that scale is not an accident in the system; it is a description of how the system runs.

The mechanisms are well understood and again require nobody to behave badly. Journals publish positive results far more readily than negative ones, so experiments that found nothing sit in drawers and the published record is a biased sample of the work done. Preclinical sample sizes are often small, which does not merely make findings uncertain — it makes the surviving positive ones systematically overstated, since with few animals only a large apparent effect clears the threshold at all. And one dataset can be analysed several defensible ways: which timepoint is primary, which animals were excluded, which outcome gets reported.

The classic statement of where this leads was made in 2005, arguing from the structure of research rather than any particular misdeed. Given small studies, modest effect sizes, many relationships tested, latitude in design and analysis, and a crowded field chasing the same question, the probability that a published claim is true can fall below a coin toss — so that for a good deal of published research, being false is the expected outcome rather than an aberration 1. That argument is two decades old and has been absorbed rather than refuted. Preclinical peptide work swims in the same water.

The catalogue is a record of what stalled

Everything so far applies to biomedical research in general. Here is the part specific to peptides, and it is the single most important idea in this piece.

Consider what happens to a peptide that does have convincing human data. It becomes a medicine: a licence, a manufacturer, a label listing what it is approved for, a prescriber standing between it and the person taking it. Then it leaves. A company that spent nine figures proving a molecule works in people does not go on to sell it as a laboratory reagent. Success removes a compound from the category entirely.

Now consider a peptide with a run of encouraging animal papers and nothing beyond them. It cannot be sold as a medicine, because it has not earned that description and no regulator will call it one. What it can do is exist in a market that does not require the evidence — supplied for laboratory research, described in the language of the papers, bought by people who have read them. It has not been rejected. It has run out of road, and the only road still open is the one with no gate on it.

Put those two movements together and the selection is inverted. Research-chemical catalogues do not happen to contain unproven compounds; they can contain little else, because proof is the very thing that takes a compound out of them. Judging peptide science by such a list is like judging cinema by the films that never found a distributor. You are looking, by construction, at what did not make it through — and why any particular one did not is invisible from where you stand.

Stalled is not the same as failed

This is where the argument has to be made carefully, because it is easily turned into a smear it does not support. A compound that stopped short of human trials has not been shown to be useless. It has been shown to be unfunded, which is a different fact about a different thing.

The commonest reason is ownership. A clinical programme is an eight-figure undertaking before anyone counts the years, and the money is advanced against an exclusive period in which to recover it. A short peptide whose sequence was published openly decades ago cannot be protected, so there is no proposition to put to an investor: spend the money, prove it works, then watch anyone with a synthesiser make the identical molecule next week. The trial does not happen — and it would not happen if the compound were superb. Efficacy has no bearing on that calculation at all. The same fate meets compounds orphaned by a reorganisation, or by an indication too small to interest capital.

The honest version runs the other way too. Some compounds stalled because somebody looked harder and the effect thinned out, or because early toxicity work went badly, and that news often never reaches the literature — negative results are the least published category there is. So absence of human evidence is genuinely uninformative. It fits an unpatentable molecule that works, and it fits one that quietly did not survive a more careful experiment, and from outside the two look identical. The silence is mostly about money and institutions. It is not a verdict either way.

How the model system disappears from the sentence

The last stage is linguistic, and it does the most damage for the least effort. Consider the phrase "shown to promote healing". It is a perfectly accurate description of a rat study. It is also, to any ordinary reader, a claim about people. Both readings live in the same five words, because the passive voice has quietly removed the only thing that would separate them: shown by whom, in what, under what conditions. The construction that makes a sentence sound rigorous is the one that deletes what you would need to check it.

Then compression takes over. The sequence below is a composite rather than a quotation from any particular paper, but its shape will be familiar to anyone who has followed a finding from its source to its final resting place.

1. The results section: treatment accelerated healing of a surgically divided tendon in young male rats, relative to saline, over the period observed.
2. The paper's own abstract, compressing for space: the peptide accelerated tendon healing in rats.
3. A review gathering many such papers into one sentence: the peptide has been shown to accelerate tendon healing.
4. A general-audience article summarising the review: the peptide supports tendon repair.
5. A page written for someone deciding what to buy: supports tendon and ligament repair.

Read that ladder rung by rung and there is no step at which somebody lied. Every line is a fair summary of the one above it. The species vanishes at the third rung, which is ordinary practice in review writing. The comparison and the timescale go at the fourth, because general readers do not want them. By the fifth the sentence describes a property of the molecule rather than an observation made under conditions — and a property seems like the sort of thing that ought to hold for you as much as for a rat. The falsehood is nowhere in particular. It emerged from a series of reasonable abbreviations, which is a harder problem than dishonesty: nobody to hold responsible, no single edit that fixes it.

The one question worth asking

The temptation, having followed all this, is to swing to the far end and treat every claim about every peptide as noise. That is the same mistake wearing different clothes. Blanket disbelief is exactly as unreasoning as blanket belief: both settle the matter without reading anything, and both are wrong about the same compounds. Some of these molecules do useful things, and a few will turn out to matter.

What survives is a single question, small enough to actually use. In what was this shown? Not whether the claim sounds plausible, not whether the source seems respectable, not how many papers there are — just the model system. A cell line in a dish, a rodent with an induced injury, a larger animal, a handful of people with no control group, or a randomised trial in patients resembling whoever the claim is aimed at. These are wildly different kinds of statement, and the words used to report them are frequently identical.

Two things then become visible at once. The answer nearly always exists and is easy to get: the paper says so, usually in its title, and the abstract is public. And the answer has nearly always been left out of the claim you were shown — which is the genuinely informative part. When a source has the model system and drops it, the omission tells you something; when a source volunteers it unprompted, that is generally the source worth reading. Which is why the rest of this site keeps stopping to say which animal, how many, and for how long. Not pedantry. It is the only part of the sentence that was ever load-bearing.