Why We Killed Our Own Pricing Scraper

On July 1, 2026, a cron we had shipped two months earlier proposed 113 changes to vendor prices in our catalog. Roughly 90 percent of them were wrong. The next day we turned it off.

This is the story of a piece of automation that did exactly what we designed it to do, produced mostly garbage anyway, and taught us where the line sits between a machine job and a human job in a CFO tool. The short version: machines are good at noticing that a page changed. They are bad at reading a marketing page and extracting the truth. We kept the first half and fired the second.

What we built

Vendor prices drift. Zendesk moves a tier, Slack renames a plan, Microsoft reshuffles a bundle, and a catalog that anchors a CFO's renegotiation math goes stale. So in May 2026 we built a monthly job to catch it.

The design was reasonable on paper. Once a month the cron fetched 143 vendor pricing pages. It took a sha256 hash of each page and compared it against the last run, so unchanged pages cost nothing: no model call, no diff, skip. For the pages that had actually changed, it ran the new HTML through a language model with a forced tool call, pulled out the tier ladder (plan name, price per seat, standard-tier flag), and compared that against the price already in our catalog. Any gap became a pending diff: a proposed change queued for a human to approve or reject before anything touched the live catalog.

Note the guardrail that was already there. Nothing auto-published. Every proposed change waited for review. That guardrail is the only reason this story is about wasted review time and not about a corrupted catalog.

What broke

The change-detection half worked. sha256 is sha256; it told us precisely which of the 143 pages were different from last month, and it never lied about that.

The extraction half is where it fell apart. The July 1 run produced 113 diffs, and when we sat down to triage them, roughly 90 percent were noise. Three failure modes, over and over.

Free-tier detection artifacts. A vendor adds a "Free" column to its pricing table for a self-serve motion, and the extractor reads the $0 as a price change on the paid tier, or picks the free plan as the new standard anchor. The plan a CFO actually buys did not move a dollar.

Standard-tier re-flags. The model re-decides which tier is the "standard" one from month to month based on page layout, and a cosmetic reshuffle of the pricing grid reads as a reclassification. No price changed. The label moved.

Flat-price misparses. This is the one that convinced us. The extractor could not reliably tell a per-seat price from a flat monthly fee, and marketing pages almost never make it easy.

Three examples, all from the July run, all checked against the real catalog value:

• Chili Piper lists a plan around "$3,500 per month, up to 30 seats included." The extractor read that as $3,500 per seat. Chili Piper's real per-seat rate in our catalog is $22.50. The proposed number was about 155 times too high.
• Warp sold its Teams plan at $15 per user per month when the July run fired (it has since repriced to $18). The extractor proposed $180, which is exactly $15 times 12: it read the annual figure as the monthly one. A 12 times error from a single misread column.
• QuickBooks Online is priced flat per company, not per seat. At the time of the run the ladder was Simple Start $38, Essentials $75, Plus $115, Advanced $275 a month, no metering (QuickBooks has since raised Essentials, Plus, and Advanced to $85, $140, and $340). The extractor grabbed the $115 Plus tier and filed it as $115 per seat. It is a real number on the page. It is not a per-seat price, and QuickBooks does not have one.

None of these are model failures in the "the AI is dumb" sense. They are failures of the task. A vendor pricing page is a sales asset. It is laid out to make a plan look attractive, not to be parsed into a normalized per-seat number. "$3,500/month, up to 30 seats" is unambiguous to a human buyer and a coin flip to an extractor that has to decide whether 3,500 is the seat price, the plan price, or the annual commitment.

The decision

We discontinued the cron on July 2, 2026. We did not delete it. The route, the library, the database tables, and the tests are still there, dormant, so it can be run by hand or revived if we ever fix the extraction. But it no longer fires on a schedule, because a job that generates 113 review items to surface maybe a dozen real changes is not saving anyone time. It moves the work from "find the changes" to "find the changes hidden inside the false changes," which is worse.

What replaced it is deliberately boring: quarterly manual verification. Every few months a human opens the vendor's live pricing page, reads the price with their own eyes, quotes the exact price text verbatim into the record, saves the source URL next to it, and updates the catalog. Slower. Fully sourced. No misparses, because a person who can see "$3,500/month, up to 30 seats" does not file it as a per-seat rate.

We ran that manual pass the same week. It live-fetched each vendor page, quoted the price text verbatim, saved the URL, and updated the catalog. It rejected all 113 machine diffs and caught the handful of genuine changes the machine had buried, with none of the fabrications the machine had invented.

The actual lesson

Here is the split we should have seen at design time.

Change detection is a machine job. Hashing 143 pages and reporting which ones differ is deterministic, cheap, and correct. A person should never do that by hand.

Price extraction from a marketing page is a human job. It requires reading intent off a document engineered to obscure the normalized number you want. A model can draft a guess, but every guess needs a human to confirm it against the literal page text, and once a human has to confirm every guess, the automation has not removed the work. It has added a layer of plausible-looking wrong answers on top of it.

This is the same trust contract that runs through the rest of the product. We source every compression percentage to a public claim or tag it honestly as an estimate, and we run every vendor claim through a discounting algorithm before it reaches a CFO's screen, precisely because an unsourced number that looks authoritative is more dangerous than an obvious gap. A pricing scraper that emits $3,500 per seat for a $22.50 tool is the same failure wearing a different hat: a confident number with no human behind it.

Change detection still runs automatically elsewhere in our stack. The way we catch new AI agent launches leans on exactly this principle: hash-and-diff and feed cursors do the noticing, and a human does the deciding. That system works because we drew the machine and human line in the right place. The pricing scraper failed because we drew it one step too far into the machine's side.

What a CFO should take from this

If a SaaS management tool tells you it "automatically keeps vendor pricing current," ask the follow-up: current from what source, verified by whom. If the answer is "our system scrapes vendor pages and updates automatically," you are looking at a catalog that can silently file a flat fee as a per-seat rate and hand you a renegotiation number that is 155 times too high. The failure will not announce itself. It will look like a normal row.

The version we trust is the slower one. A price in our catalog traces to a human who read the vendor's page, quoted the number, and saved the link. When a vendor's rep pushes back in a renewal call, that is the number that holds, because there is a person and a URL behind it, not a hash and a hope.

Try the free calculator, 15 seconds, no signup. The prices it runs on were verified by a human, and we would rather tell you that than tell you they were scraped.