AI Best Practices & Procedures
A working reference for operators, boards, and acquirers evaluating AI investments — built from the same framework DirectSourceAI applies in every paid engagement. Updated as the evidence changes.
Why most AI projects fail, in one paragraph.
MIT's NANDA initiative studied enterprise generative AI deployments and found that roughly 95% delivered no measurable profit-and-loss impact, despite $30–40 billion in combined investment. The study's central finding is the one most companies miss: the gap was attributed to implementation approach, not model quality or regulation. In plain terms, these were business failures wearing a technology costume. That single fact reframes the entire evaluation problem — it means a CFO's questions matter more than a developer's answers.
Before you approve any AI spend, verify six things.
Write down the specific financial outcome expected, in dollars, before any money moves. If it cannot be written down, it does not exist as a business case.
Confirm the tool will be built into an existing workflow, not layered on top of one. Bolt-on tools are the pattern behind most abandoned pilots.
Ask the vendor to demonstrate autonomous multi-step task completion, dynamic tool selection, persistent memory across sessions, and recovery from a failed step. Gartner estimates only a small fraction of vendors marketing "agentic AI" can do all four.
Determine whether the tool relies on data or workflows unique to your company, or whether a competitor could replicate the setup with the same off-the-shelf model.
Model the cost at three times projected usage before approving a usage-based contract. Usage pricing has ended more AI budgets than poor performance has.
Name, in writing, who is accountable when the system produces a wrong or harmful output, and what the response procedure is.
Terms worth knowing before your next vendor call.
Marketing an existing chatbot or rules-based automation tool as an autonomous AI agent without the underlying capability to justify the term.
A system that can plan, select tools, take multi-step action toward a goal, and adapt when a step fails — without a human directing each individual action.
The common state in which an AI pilot never scales to production and never gets formally cancelled, quietly consuming budget with no accountable owner.
A product that adds a simple interface around a foundation model API with no proprietary data or workflow advantage — easily replicated by a competitor.
Common questions on AI project evaluation.
Roughly 95% of enterprise generative AI pilots show no measurable P&L impact, primarily due to implementation approach rather than the underlying technology.
Rebranding conventional automation as an autonomous AI agent without genuine multi-step autonomy, tool selection, memory, or failure recovery.
Apply the six-point framework above: business case, workflow fit, vendor authenticity, data position, true cost curve, and governance.
Want this applied to your specific project?
This page is the public version of the framework. The AI Reality Check is the private, written version — applied to one specific project, with a verdict at the end.
See the AI Reality Check