Rank them on the quality of the evidence, not the size of the numbers. A case study is worth reading only when it names the starting position, the stage of the product, the advertising target that applied at that stage, the time period, and what the agency got wrong on the way.
The short version
- A percentage with no starting point is decoration. Growth from a broken listing is easy and tells you nothing.
- Ask for the launch target and the maturity target. One number for both means one playbook for every client.
- Return on investment must include ad spend, or you are reading a revenue chart with a nicer title.
- Score the evidence and the operator separately. Strong proof plus a stretched team still ends badly.
- Ask for the engagement that did not work. The answer sorts candidates faster than any other question.
The mistake that wastes a shortlist
Most buyers read case studies for the outcome. They scan for the biggest multiple, shortlist the three biggest, and end up comparing marketing departments rather than operators. The cost of that mistake is a full quarter, because you only discover the mismatch once the work starts.
Read for the setup instead. Every honest study answers four questions before it reports a result: where the account started, what stage the product was in, what was spent, and over how long. When any of those is missing, the number floats free of the conditions that produced it.
The scorecard
Score each candidate out of 100 using the weights below. Anything under 60 comes off the list regardless of how impressive the headline was.
| Criterion | Weight | A score of 5 looks like | A score of 1 looks like |
|---|---|---|---|
| Starting position | 20 | Baseline revenue, rating, and conversion rate stated | "Increased sales by 340 percent" |
| Product stage named | 20 | Launch, growth, or mature, with the target for that stage | Stage never mentioned |
| Costs inside the return | 15 | Ad spend, fees, and landed cost included | Revenue growth reported as return |
| Time period | 15 | Dated window, at least two quarters | No dates anywhere |
| Attribution method | 10 | Explains what was measured and how | Claims credit for the whole account |
| A named failure | 10 | A specific engagement that went wrong | Everything worked |
| Reference call offered | 10 | A current client you may speak to unaccompanied | References "available on request" |
Multiply each score by its weight, add them, divide by five. The exercise takes twenty minutes per candidate and it will change your ranking, usually by moving the loudest name down.
The two advertising numbers
The single question that separates operators from resellers of dashboards is this one. Ask for the advertising cost of sales target at launch and the target at maturity for the same product.
Those numbers should be far apart. At launch you are buying rank and data, so an aggressive target is correct and a low one means you are underfunding the entry. At maturity you are defending share profitably, so the same target would be waste. An agency quoting a single house number across every client is applying a template to your catalog.
Follow up by asking what event moves a product from the first number to the second. A real answer names conditions, usually a stable rating, a proven conversion rate, and a customer acquisition cost that has settled.
Building the return on investment yourself
Take their reported figure and rebuild it. Revenue attributable to the work, minus ad spend, minus Amazon fees, minus landed cost, minus the management fee. Then divide by the management fee. That is the number that decides whether the engagement pays.
Most published case studies do not survive this rebuild, which is not always dishonesty. It is usually that the study was written by someone who was never asked to include the fee.
What most agencies will not tell you
Case study libraries are survivorship exhibits. Nobody publishes the account that stalled, the launch that ran out of inventory in week six, or the client who left after two quarters. That is true of our library too, and it is why I would rather hand you a scoring method than a deck.
The second thing rarely said out loud is that a good agency inherits a good product more often than it rescues a bad one. A large share of any impressive result belongs to the product, the price, and the timing. When you read a study, try to separate what the operator did from what the market handed them. Ask directly which parts of the outcome the agency believes it caused, and treat a candid split as a strong signal.
Ours: the majority of brands we take on are profitable within their first year. That is a benchmark you can hold any agency to, including us, and it is more useful than a single spectacular chart.
Related answers
- KPIs an Amazon agency should report weekly
- How does Amazon ads performance pricing work
- What does a good Amazon account audit include
- Best Amazon agencies for US and Canada sellers
- Hiring an Amazon agency: the complete guide
Run the scorecard on us before you run it on anyone else. Start with a free written audit at Flapen.
