Five Questions a CFO Asks Before Signing Off on an AI Vendor
Sales decks answer "what can it do." They're built to. The questions that actually separate a durable AI vendor from a well-funded demo rarely come up unless someone in the room has signed a check before and had to defend it later.
Here are the five we ask on every engagement, before we look at a single slide.
1. What happens when the model is wrong?
Not "how accurate is it" — every vendor has a favorable accuracy number. The real question is what the system does with the error: does it flag low-confidence output for a human, silently ship it, or have no concept of confidence at all? A vendor who hasn't thought about this hasn't shipped anything real.
2. Whose data made the demo work?
A demo trained or tuned on the vendor's cleanest reference dataset tells you nothing about your data, your document formats, or your edge cases. Ask to see it run, live, on your ugliest real-world input. Watch what they say next.
3. What's the cost curve at 10x volume?
Per-seat pricing on a pilot of eight users is not a business model, it's an introductory offer. Get the unit economics at the volume you'd actually run in production, including inference costs the vendor doesn't put in the deck.
4. Who else is calling this "agentic"?
"Agentic" has become the most agent-washed word in enterprise software. Ask specifically what decisions the system makes without a human step, and what happens when it makes a bad one. Most answers reveal a well-designed workflow tool wearing an agent's marketing.
5. What does the exit look like?
Data portability, model lock-in, and what you keep if you cancel — these questions get vague answers from vendors who haven't built for churn, which tells you something about how they expect the relationship to end.
None of these questions require an engineering background. They require the habit, from thirty years of finance, of asking what happens when things don't go as planned.
If you can't get straight answers to these five before the contract is signed, you won't get them after. That's usually where our Capability Verification engagements start — after the fact, when it's expensive to ask.