Why every QA comparison table is a marketing lie — and how we fixed ours
Third iteration of the comparison table — this one frames each row as a customer question, not a feature dump. ROI calculator landed and 89 users ran the numbers in week one. We also deleted two homepage sections. Honest metrics inside.
Tomas Mertin
4 min read
The previous comparison table had 11 rows and zero useful answers. We rewrote it from the customer's question, not from our feature list.
Shipped
Comparison table v3. Third iteration. Previous versions read like a feature list dump. This one frames each row as a customer question ('Can it generate tests from a PR diff?') and shows binary answers across QAWave, Applitools, Mabl, and in-house SDET. 11 rows, no marketing fluff.
ROI calculator. Lives between the Problem section and HowItWorks. Three inputs: engineer count, hours/week on tests, flaky rate. Outputs annual waste in EUR and payback period for our $35k sprint. Conservative formulas — overshooting ROI claims is the fastest way to lose technical buyers.
Killed two sections. The 'Trusted by' logo row (we don't have public logos yet — fake social proof is worse than no social proof) and a generic 'How AI works' explainer (everyone reading our site already knows). Site is shorter and converts better.
Numbers
Unique visitors: 1,378. Discovery calls booked: 4. ROI calculator interactions: 89 (44% of visitors who reached the section). Median session: 3m 42s — up from 2m 31s in W19.
Learned
Pricing transparency is a wedge. Three of four discovery calls this week mentioned the $35k fixed price as the reason they booked. 'You're the only QA vendor that puts a number on the homepage.' Most competitors hide pricing behind a sales call — that's the gap.
Eval harness for the Copywriter agent missed a regression. Two banned words slipped through to staging before Brand Guardian caught them in the pre-publish check. Root cause: the eval dataset hadn't been updated when we expanded the banlist. Updated the workflow so banlist changes auto-extend the eval suite.
Next week (W21)
Hero rewrite (NetworkGraph is testing badly — replacing with a CI terminal visualization). 3-agent content review pass. Push hard on the first design partner conversation — verbal yes is in, need the LOI.