Compare both on the same five numbers: contribution margin per unit, blended customer acquisition cost, organic share of sales, inventory sell-through, and time to fix a broken listing. Revenue and ACoS alone will hide a team that is buying growth. Set the reporting cadence before either option starts.
The short version
- Revenue is not a KPI, it is a side effect. Any team can buy revenue with your money.
- Organic share of sales is the honest growth signal. It rises only when the listing, the rank and the reviews improve.
- Response time belongs on the scorecard. How fast a broken listing gets fixed is worth more than a point of ACoS.
- Count the traffic channels being worked. There are five available and most sellers run two.
- Same metrics, same cadence, both options. A comparison where each side reports different numbers is not a comparison.
Why revenue comparisons mislead
The mechanism is simple. On Amazon, advertising can manufacture sales at almost any volume you are willing to fund. So a team measured on revenue has a guaranteed way to look successful for two quarters: spend more. Nothing about the underlying business improves, and the cost surfaces later as thin margin and a catalog that cannot survive a spend cut.
That is why the comparison set below is built from numbers that resist being bought.
| KPI | What it tells you | How to read it | Cadence |
|---|---|---|---|
| Contribution margin per unit | Whether the growth is worth having | After ad cost, Amazon fees, freight and returns. Not gross margin | Monthly |
| Blended customer acquisition cost | The real price of a new customer | Total spend across every channel divided by new orders, not just ad-attributed ones | Monthly |
| Organic share of sales | Whether rank is improving | Rising share at flat or falling spend is the strongest signal in the account | Monthly |
| Inventory sell-through | Whether demand is real and cash is moving | Units sold against units held, per product | Weekly |
| Time to fix a broken listing | Operational competence | Hours from suppression or content loss to restoration | Per incident |
| Traffic channels active | Coverage against available demand | Organic, paid, promotions, influencer and creator, off-channel | Quarterly |
That last row is the benchmark I would hold both options to. Five channels exist: organic, paid, promotions, influencer and creator, and off-channel. Most sellers run two, usually organic and paid, then wonder why growth stalls when the category gets more competitive. An in-house hire will typically be strong in one channel, whichever one they came from. Ask an agency which of the five they actually operate rather than which they list on a slide.
Failure modes, ranked by what they cost
1. Measuring ACoS instead of margin
The most expensive mistake on this list. ACoS looks like a profitability metric and is not one. It ignores fees, freight, returns and the organic sales your ads influenced. A team can hit an excellent ACoS on a product that loses money on every unit shipped. Put contribution margin at the top of the report and ACoS underneath it, in that order.
2. Judging an in-house hire against an agency's first month
A new employee spends a quarter learning your catalog. An agency arrives with a method and shows early structural wins, because the first fixes in any neglected account are the same ones. Compare at month six, not month one, or you will fire a good hire and keep a fast start.
3. Reporting that arrives only when asked
If you have to request the weekly number, the number is being managed rather than reported. Fix the cadence in writing at the start. We send a written update every week and hold a live review every two weeks, and clients have Slack access continuously between them. Set the same expectation for an internal hire.
4. Attributing everything to the last click
Off-channel traffic, influencer content and promotions frequently show up as organic or brand-name search later. Judged on last-click alone, the channels that build the brand look like waste, so they get cut, and then organic share quietly falls two quarters later.
5. No agreed definition of a broken listing
Suppressed images, lost A+ content, a hijacked buy box and a variation that split are all outages. Without a written definition and a response window, nobody owns them, and they get discovered by a sales dip rather than by monitoring.
What most agencies will not tell you
The KPI most agencies volunteer is the one they control most easily. Ad-attributed sales and ACoS come straight from the ad console, they move quickly, and they look like progress. The numbers that matter to your business, contribution margin and blended acquisition cost, require your cost data, which means the agency has to ask for it. Notice who asks.
The other thing worth naming: an in-house team has the same incentive problem in a different shape. Their job exists because the work exists, so recommending fewer campaigns, fewer SKUs or a discontinued product is a recommendation against their own headcount. Neither structure is honest by default. Written metrics and a fixed cadence are what make either one honest.
Related answers
- What to use for Amazon PPC: agency or internal team
- KPIs an Amazon agency should report weekly
- Amazon agency vs in-house team pros and cons
- In-house Amazon team cost breakdown
- Amazon agency pricing and economics: the complete guide
If you want your current numbers read against this set before you decide, ask Flapen for the free audit.
