Reading Published Results

What does that case study actually state?

A published result can only be interpreted if the page says what the number is a proportion of, what it was measured against, over what period, alongside what else, and who produced it. Paste up to five case studies and this reads every numeric claim against those five conditions, one row per claim.

Free and instant, nothing stored, no address asked for. It runs the same instrument that read 6,815 case studies from 369 firms for the study behind it. Four conditions can be read off a page. The fifth, the comparison, needs the firm-wide read.

Carlos

Carlos

Marketing science agent

Carlos
Every result comes from the instrument, not from me

Paste a case study on the right, or give me the link here. Either works, and they reach the same answer.

Read the pages

Paste up to five case studies

Three sites at a time. Nothing is stored, and no address is asked for.

We honour robots.txt. A page a site asks crawlers to leave alone is reported as skipped rather than read.

Common questions

Common questions

Reading a marketing case study

What makes a marketing case study credible?

A number a reader can interpret. That takes five things stated on the page or behind it: what the number is a proportion of, what it was measured against, over what period, alongside what other activity, and who produced it. Four of the five cost a sentence. Only the comparison asks for real work.

Why does this only read four of the five conditions?

Because the fifth cannot be read off a page. Whether a result was measured against a holdout, matched markets or nothing at all is a property of how the work was designed, not of the write-up describing it. A page-level tool that claimed to score it would be guessing. The firm-wide read answers it across every claim on a site.

Does a missing condition mean the number is false?

No, and the study behind this tool is careful about it. It observes what pages state, not whether any figure is true. The finding is that disclosure is a convention nobody chose rather than a competence gap: among firms whose own sites describe running controlled experimental designs, 76.6% of their published claims still sit on a page that states no method.

Can I check an agency I am considering hiring?

Yes. Paste their published case studies and read what the pages state. That is the client side of this tool and it needs nothing from you. What it will not do is produce a firm-level score from a handful of pages, because three pages a person chose is not a sample and a number built on one would be exactly the sort of claim this firm exists to object to.

Why does the firm-wide read need a work email?

It reads every claim on a domain and returns a scored card. Handing that to anyone who types a competitor's URL would make this a tool for grading other people's firms, which is not what it is for. An address on the same domain, confirmed by clicking, is the cheapest proof of ownership that holds, and it is also how the card is delivered.

Where do the population figures come from?

A cross-sectional study of 6,815 published case studies from 369 firms that place media on a client's behalf. The paper, the instrument and every failure found in it are published in full and ungated.

The study this is computed from: Funny Numbers. The conditions, before a campaign runs: the design-stage checklist.