What to Ask in Your AI Agent Vendor Demo (and Before Approving the Pilot)

The deck always opens with the same slide: "Customers see 70% deflection." It never includes the setup fee, the seat count where the math turns positive, or the source of the 70%. Four questions reset that conversation before procurement signs anything.

We sit in on AI agent demos every week — Sierra, Decagon, Ada, Glean, Moveworks, Intercom Fin, the whole field. The pattern is consistent. Vendors pitch a gross compression percentage drawn from one customer case study, divided by the most favorable definition of "deflection," with no mention of the $35K implementation engagement, the 24-month minimum, or the seat count below which the agent loses money on day one. A CFO walks out impressed, the pilot gets approved, and twelve months later the savings line in the board deck is half what was promised — or negative once setup is amortized.

The fix isn't to skip the demo. The fix is to ask four questions that move the conversation from marketing math to finance math.

Question 1: "What's your sourced compression on my exact stack — not your theoretical max?"

Every vendor has a hero number. Sierra cites case studies in the 60-70% range. Decagon's marketing leans on Klarna's 75%. Ada and Intercom Fin land in the same band. None of those numbers should land in your pro forma untouched.

The reason is that vendor-published percentages are gross, not net, and they're typically pulled from a single deployment with optimal conditions: a high-volume support queue, a narrow domain, an already-clean knowledge base. They tell you what the agent did once. They don't tell you what it does on your stack.

At SeatCompress we apply explicit discount factors to every claim before it lands in our catalog:

• Case study evidence: 0.7x discount. A Klarna-style 75% becomes 0.75 × 0.7 = 0.525 before any cap.
• Analyst (Gartner, Forrester, IDC): 0.85x. Already conservative.
• Vendor marketing / blog claims: 0.5x. Cut in half by default.
• Trade press: 0.6x.

Then a category cap clamps the result. Vertical replacement agents — the ones that actually displace a specific tool's seats, like Decagon on Zendesk — cap at 0.65. Horizontal AI assistants like a Mistral Le Chat Team or a generic Claude Team cap at 0.15 because they don't remove seats; they sit alongside them. (More on why we built this gate in why we source every compression percentage.)

So the question to the vendor is: "Show me a sourced compression number against my actual stack of tools, at my seat count, with the discount factor and source type marked." If the answer is "we see 70% across our book," push: which book, what definition, what's the analyst source, what does that become when applied to our current Zendesk seat count, not the average customer's.

If they can't decompose the number, treat the deck claim as vendor marketing and apply the 0.5x yourself. A "70% deflection" pitched without a citation becomes 35% in your model, and on a vertical replacement agent that gets clamped to 65% only if you have a case-study source backing the underlying claim.

Question 2: "What's your setup cost — itemized, not bundled into the subscription?"

The second question kills more pilots than the first. Vendors quote monthly subscription cleanly and burrow setup into a separate SOW that lands two weeks later. The setup line item is where the year-1 math goes negative.

Public anchors we use in our viability engine:

• Sierra: $35,000 setup + $6,000/mo subscription. Year one all-in: $107,000.
• Decagon: $25,000 setup + $5,000/mo. Year one all-in: $85,000.
• Glean: $50,000 setup + roughly $60/user/mo enterprise. Year one for a 500-seat deployment: $410,000.
• Moveworks: $33,000/mo with negotiated setup; no public floor. Year one all-in: ~$400K minimum.

The flat-fee floor we apply when a vendor refuses to disclose setup is $15,000, or one month of subscription, whichever is higher. Per-user-priced agents (Cursor Business, ThoughtSpot Spotter, Otter Business) typically have no setup overhead — that's the structural difference. If a vendor tries to position a flat-fee agent as having "no implementation cost," they're either rolling it into Q1 invoice or planning to surprise you with it later.

Make them put it on the slide. "What's the line-item setup, and is it amortizable across the multi-year term, or front-loaded in year one?" Front-loaded is normal; pretending it doesn't exist is not.

Question 3: "At what seat count do I cross the unlock threshold on my dominant tool?"

This is the question vendors least want to answer, because the honest answer for many enterprise AI agents is: "your current seat count, or higher."

Here's the structure. A flat-fee agent like Sierra costs $72,000/yr in subscription plus $35,000 setup, amortized = $107,000 in year one. To break even, the agent has to compress enough Zendesk seats (at, say, $115/mo enterprise) to clear $107K of gross savings. At Sierra's catalog-derived 60% compression on Zendesk, the math says you need roughly 130 active Zendesk agents before the deploy is profitable in year one. Below that, you're paying for the agent to learn on your dime.

Decagon's threshold is lower (~95 agents) because the subscription is cheaper and the case-study-backed compression on Zendesk lands at 0.65. Moveworks' threshold is much higher because $33K/mo subscription needs roughly 400+ seats of compressed Slack/Teams/ServiceNow/Freshservice value before it clears.

This is also why the segment matters: a 5,000-employee enterprise running 200+ Zendesk agents crosses Sierra's unlock cleanly. A 1,500-employee company running 40 support agents does not. The agent that's a no-brainer for one buyer is a year-one loss for the other, and the vendor's case studies almost never disclose the customer's seat count.

Ask: "At my current active seat count on the target tool, do I clear breakeven in year one? Year two? What's the seat count at which I cross?" If the vendor doesn't have this math ready, build it yourself: divide the year-one all-in agent cost by (your per-seat tool cost × the discounted compression %). The result is the seats you need.

Our calculator does this per-tool against your actual stack — paste in your Zendesk seats and Sierra populates with the unlock math.

Question 4: "Show me the analyst report or case study — not the marketing deck."

The last question is procedural. Every claim in the demo should be traceable to a primary source. The vendor's own blog post claiming "up to 95% deflection" isn't a source — it's marketing reflected back at itself. A Gartner Magic Quadrant note, a Forrester TEI study, a published customer case study with the customer's name and methodology, an analyst earnings transcript citing operational metrics — those are sources.

The discount factor we apply tells you why this matters:

• An analyst report (0.85x discount) is the most reliable signal. A Forrester-cited 60% compression on Confluence by Glean becomes 51% in the pro forma — barely degraded.
• A case study with a named customer and disclosed methodology (0.7x) is second-best. Klarna's 75% on Decagon becomes 52.5% pre-cap.
• Vendor marketing (0.5x) is the weakest. "Up to 95%" becomes 47.5% — and then the vertical_replacement cap clamps it to 65% only if there's a case study underneath supporting the magnitude.

So ask for the source document, read it yourself, and apply the right discount. This isn't pedantry — it's the difference between a pro forma that survives a board challenge and one that doesn't.

Also worth asking: "Has the source been updated in the last twelve months?" AI agent capability moves quarterly. A 2024 case study cited in a 2026 demo deck is older than the model the agent runs on today, and the comparison vendor's product has likely shipped its own AI in the interim — reducing the delta the case study claimed.

Worked example: 12,000-employee SaaS company evaluating Sierra

Concrete numbers. A 12,000-person SaaS company running:

• Zendesk: 220 active agents at $115/mo enterprise rate — $303,600/yr
• Intercom Fin already in production on tier 2 tickets
• Confluence and Notion sprawling across engineering and product

Sierra demos at 70% deflection citing two case studies. The buying committee includes the VP of Support (excited), the VP of Engineering (skeptical), and the CFO (signing). Sierra quotes $35K setup + $6K/mo subscription. The sales rep mentions a 24-month minimum.

Run the four questions:

1. Sourced compression on Zendesk. Sierra's published case studies give 60-70% on the surface. Apply the 0.7 case-study discount: 0.49. Apply the 0.65 vertical_replacement cap: not binding. So the defensible compression for the pro forma is 49%, not 70%.
2. Setup itemized. $35,000 SOW, paid up front, non-refundable if pilot doesn't extend. Year-one all-in: $107,000.
3. Unlock threshold. Sierra at 49% compression on 220 Zendesk agents at $115/mo = 220 × 0.49 × $115 × 12 = $148,764 gross annual savings. Year-one net: $148,764 − $107,000 = +$41,764. Profitable, but only by ~28% over year-one cost — squarely in the "break-even" band where execution risk dominates.
4. Sources. One of the case studies is a Sierra-published blog post (vendor_marketing — 0.5x). The other is a podcast appearance with a named customer (case_study — 0.7x). Mixed quality; lean on the case_study number.

The CFO's read: pilot is viable but thin. Negotiate setup down to $20K (publicly precedented for first-year pilots) and the year-one net jumps to $56,764 against $92,000 all-in — a 62% margin over cost. Refuse the 24-month minimum, run a 6-month pilot tied to a deflection KPI, and the worst-case downside is $36K spent (6 × $6K) plus $20K setup = $56K against $74,382 of pro-rated gross savings = still net positive.

What you don't do: sign the 24-month commit at $107K year-one based on 70% deflection. That's the path where Q3 board review shows a $30K shortfall against plan and the AI initiative gets blamed.

For the broader frame on per-resolution-priced support agents and where they outperform per-seat flat fees, see per-resolution agents and the volume crossover and the deeper math on Intercom Fin and Sierra in Intercom Fin vs Zendesk seat replacement math.

What the CFO does Monday morning

Three concrete moves before approving any AI agent pilot this quarter:

1. Mandate a "sources slide" in every vendor demo. Every compression % the vendor cites must have a source type marked: case study, analyst, vendor marketing, trade press. No source → treat as judgment (apply your own discount, default 0.5). This single change kills 30% of inflated-claim risk before procurement even sees the SOW.

2. Require itemized setup in the LOI. Setup cost, services hours, knowledge-base integration cost, escape clauses if pilot deflection falls below threshold. If the vendor refuses, apply the $15K floor and assume it's higher. Sierra, Decagon, Glean, and Moveworks all have meaningful setup; agents that claim "zero setup" are either per-user-priced (legitimate, no setup) or hiding it (not legitimate).

3. Compute your own unlock threshold before the second call. Year-one all-in cost ÷ (active seats on target tool × per-seat cost × discounted compression %) = the seat count where the math turns positive. If you're under it, the pilot is a bet on growth, not a savings program. Price it accordingly. If you're well over, you have leverage — use it to negotiate setup down and the term short.

The agents are real. The compression is real. The 70% in the deck isn't the 70% you'll see. Four questions surface the gap before you sign. That's the difference between an AI initiative that lands in the board deck as a win and one that lands as an unfavorable variance line nobody wants to own.