Before You Buy an AI Tool, Ask These 6 Questions

The other day I told a business owner I work in AI. His first question was, “What tool are you selling?” That question is the whole problem. Leaders have started thinking of AI as a tool when it is a system, a workflow, an operating layer that makes the work better. If one tool could fix everything, one giant tool would already exist. SaaS happened because each product solved a specific problem the others did not.

So when a leader says “we need an AI tool,” what I usually hear is “we want a quick fix for a problem we think is important.” The sharper question is whether the answer is even a tool, or an agentic workflow, a forecasting model, a data-capture layer, a process redesign, or no AI at all. Buying the wrong one of those is exactly the AI confusion tax I keep seeing companies pay.

The trap: five disconnected tools#

The common mistake is buying several AI tools when the business needs one workflow it can extend. A tool that cannot integrate becomes another island. A tool that extends into adjacent work becomes leverage. Ask what workflow you are improving and what existing system AI should extend, and which kind of intelligence the problem actually needs, before you ask what to buy. Collecting disconnected tools is how you end up a tourist in your own stack, which I cover in AI tool tourism.

Know which vendor you are talking to#

There are three kinds of AI vendors right now. Tool vendors build and implement one specific tool. Agentic vendors understand automation, but many hear “workflow” and immediately reach for an agent without grasping the whole business problem. The third kind understands the pain first and will tell you whether you need a tool, an agent, a model, a data layer, a redesign, or nothing. That third kind is the most valuable. A one-hour call with them can save more than the cost of any tool, because they stop you from buying into the wrong problem.

The 6-question buying scorecard#

If these six do not have clear answers, you are not buying an AI capability. You are buying uncertainty with a better demo.

1. What data are you working with?#

AI is Y = f(X). The X has to exist before anything useful comes out as Y. A good vendor asks about the mess: scattered across systems, inconsistent but partially tracked, or a field you never captured. If the data does not exist at all, the answer is not an AI tool. It is instrumentation, capture, and process discipline.

2. What real output are you creating?#

Name the concrete output and the decision it feeds. A forecast that no one acts on is not output. This is the same trap behind a lot of failed AI projects: a technically fine model that changes no decision.

3. What governance model do you have?#

Hallucination, explainability, auditability, escalation, and human review depend on how the solution is built and how good the data is. A tool can be fine for a POC without these. Once you scale it, missing governance becomes missing control.

4. Which ROI pillar does it improve?#

ROI is more measurable than people claim when you break it into pillars: time, money, efficiency, customer satisfaction, product, and strategic position. If a vendor cannot say which pillar the tool moves, the purchase is hard to defend. And ROI tracks the scale of the problem. If the problem is cheap, solving it will not produce a 5x return.

5. What is the upgrade strategy?#

This is not software updates. It is how the process and your mental model improve over time in a VUCA world. A strategy describes a path from A to B that is both followable and trackable. If it is neither, it is not a strategy.

6. Who owns the workflow, data, decision, and outcome?#

Without ownership you can buy a tool, run a POC, and still have no accountable system for making the result useful. SAP HANA is the classic case: many companies bought it and never got the ROI, because the data ownership, treatment, and integration were never settled. The label changes with AI, the failure mode does not.

Red flags in the demo#

Two patterns should slow you down. First, a demo that only works on the vendor’s perfectly prepared data. Ask what happens when your data is messy, incomplete, or missing a field. Second, a product that is a thin wrapper over an LLM. If your own team can build the same thing with a prompt and a light codebase, it is probably not worth buying. A tool earns its price when it brings a real domain layer: business customization, ROI logic, integrations, and governance built around the work. This is the same reason GenAI is a plier, not every job’s tool.

When to build instead of buy#

If you can build it inside your own codebase in one to three months, do not buy it. Build when the problem sits inside your capability: clear business problem, enough structure, a tech or contract team. A small internal build is often faster, cheaper, and better aligned than an off-the-shelf product, because it is shaped around your real workflow and data. Do not build what is outside the box, like a foundation model from scratch. And whoever you put on the evaluation matters as much as the tool, which is its own decision in who should explore AI tools.

The right people in the early room are business, technology, operations, the work owners, and the data owners. The data owner especially, because only they can tell you the data is messy and whether the tool will still work. I have seen this decide whether a project ever produces value, like the demand-planning work in this forecasting case, where the win came from aligning inputs across functions, not from the model alone.

If this is how you want to think before your next AI purchase, I write AI Cross-Current for operations leaders, or you can book a clarity call.