When not to use AI: a decision framework

Every client asks: 'Should we use AI for this?' The answer isn't always yes. AI is powerful, but it's not magic. Sometimes a simple rule-based system is better.

Here's the framework we use to decide when AI is the right tool.

The three questions

Before reaching for AI, we ask:

• Is the problem well-defined? (Can you specify inputs and outputs clearly?)
• Is the failure mode tolerable? (What happens when the system is wrong?)
• Is there a simpler solution? (Can rules, heuristics, or traditional ML solve it?)

If the answer to any of these is unfavorable, AI might not be the right choice.

When AI is the right choice

AI works best for:

• Unstructured data (text, images, audio) where patterns are hard to encode as rules
• Tasks that benefit from learning (systems that improve with more data)
• Problems with high dimensionality (too many features for humans to reason about)
• Tasks where approximate answers are acceptable (recommendations, summarization)

Examples: classifying support tickets, generating product descriptions, detecting anomalies in time series data.

When AI is the wrong choice

Avoid AI when:

• The problem is deterministic (clear rules, no ambiguity)
• Errors are unacceptable (financial calculations, medical dosages, legal decisions)
• You need explainability (regulatory requirements, user trust)
• The solution needs to be deterministic (same input → same output, always)

Examples: calculating taxes, processing payments, enforcing compliance rules.

If you can write it as a SQL query or a series of if-statements, don't use AI. You're adding complexity and cost for no benefit.

The cost of AI

AI isn't free. The costs are:

• Inference costs (API calls or GPU time)
• Latency (LLMs are slower than rules)
• Complexity (harder to debug, harder to test)
• Unpredictability (non-deterministic outputs)
• Maintenance (models drift, need retraining)

These costs are worth it when AI provides capabilities you can't get otherwise. They're not worth it when a simpler solution works.

Case study: the rule that beat the model

A client wanted to classify support tickets into categories (billing, technical, account). They assumed they needed an LLM.

We started with a simple keyword-based classifier:

• If ticket contains 'invoice' or 'payment' → billing
• If ticket contains 'bug' or 'error' → technical
• If ticket contains 'password' or 'login' → account

It achieved 87% accuracy. The LLM achieved 91% accuracy. But the LLM cost $0.03 per ticket, while the rule-based system cost $0.0001 per ticket.

For 10,000 tickets per month, that's $300/month for the LLM vs $1/month for rules. The 4% accuracy gain wasn't worth 300x the cost.

The hybrid approach

Often the best solution is a hybrid:

• Use rules for the easy cases (80% of traffic)
• Use AI for the hard cases (20% of traffic)
• Escalate ambiguous cases to humans

This gives you the speed and cost of rules for most cases, with the power of AI for edge cases.

The decision matrix

We use this matrix to decide:

• High volume + low complexity → rules
• High volume + high complexity → hybrid (rules + AI)
• Low volume + high complexity → AI
• High stakes + any complexity → rules + human review

The takeaway

AI is a tool, not a solution. Use it when it provides capabilities you can't get otherwise. Don't use it when simpler solutions work.

The best engineers know when not to use AI.