Congratulations. Your AI is popular. Can you afford it?

Great news.

People are using your AI.

Not just the project team.

Not just the executives who watched the demo.

Actual employees.

They like it.

They’re asking questions.

They’re coming back.

They’re telling other people about it.

Usage is growing every week.

Congratulations.

This is exactly what you wanted.

There is just one small problem.

Have you looked at the bill?

Welcome back to Production.

The Pilot Was Cheap

Of course it was.

Twenty people were using it.

Maybe fifty.

They asked a few questions each day.

The project team knew the use cases.

Testing was controlled.

Everyone was being relatively careful.

Then you released it.

Now Finance discovered it.

Sales discovered it.

Operations discovered it.

Marketing discovered it.

Jeannine definitely discovered it.

And Jeannine told three other people.

Those people told their teams.

Suddenly your successful AI pilot has something every project team dreams about:

Users.

Lots of them.

Unfortunately, users are expensive.

This Isn’t Just About Tokens

Whenever AI cost comes up, the conversation usually starts with tokens.

How much does the model cost?
How many input tokens?
How many output tokens?
Which model should we use?

Those are legitimate questions.

But production AI doesn’t exist by itself.

Remember our production architecture?

User → AI → Search → Documents → APIs → Data platforms → Databases → Business systems

Every question can create work throughout that chain.

The model is only one line item.

One Question Isn’t Always One Query

Jeannine asks: “Which customers had declining sales last quarter?”

Simple question.

Maybe the AI needs to:

Determine what “sales” means.
Identify the appropriate data source.
Retrieve metadata.
Search documentation.
Query the semantic model.
Call an API.
Run a database query.
Retrieve customer information.
Send context to the model.
Generate the answer.
Log the interaction.
Capture telemetry.
Store conversation history.

Maybe none of those individual operations are particularly expensive.

But Jeannine isn’t finished.

“Why did Acme decline?”

Then: “Compare that to last year.”

Then: “What about their service revenue?”

Then: “Show me their top ten products.”

Then: “Which sales rep owns them?”

Then: “Summarize this so I can send it to my manager.”

One business question became six AI requests.

And each request may generate multiple downstream operations.

That’s the part the demo rarely shows.

AI Changes the Cost of Curiosity

Before AI, getting an answer had friction.

Open Power BI.
Find the report.
Set the filters.
Export the data.
Open Excel.
Maybe ask an analyst.
Maybe write some SQL.
Maybe decide the question wasn’t worth the effort.

Friction limited demand.

Not intentionally.

But it did.

AI removes that friction.

Now asking another question costs the user almost nothing.

So they do.

That’s one of AI’s greatest benefits.

It’s also one of its most important economic changes.

When you make questions cheaper for humans, you can make answering them more expensive for systems.

That’s a production architecture problem.

The Sherpa’s Notebook

I’ve seen this same pattern long before generative AI.

Give people easier access to data and they use more data.

That’s usually the point.

But infrastructure that worked perfectly when a handful of technical users ran carefully designed queries can behave very differently when hundreds of people can generate requests without knowing – or caring – what happens behind the interface.

And they shouldn’t have to care.

Jeannine should not need to know that her perfectly reasonable question triggered:

Three semantic searches.
Two API calls.
Four SQL queries.
A warehouse resume.
A model invocation.
And 37,000 tokens.

She asked a business question.

The architecture owns the bill.

The $0.03 Question

This is where AI economics can become deceptive.

Suppose an interaction costs three cents.

Three cents sounds like nothing.

It truly is nothing.

Until you have:

1,000 users.

Asking 20 questions per day.

Over 250 working days.

That’s 5,000,000 interactions.

At three cents each:

$150,000.

Still potentially an excellent investment.

That’s not the point.

The point is that: $0.03 doesn’t sound like an architectural decision.

5,000,000 interactions does.

And our three cents may not include everything happening behind the model.

The Hidden Bill

Your AI bill may include:

Model inference.
Embedding generation.
Vector search.
API calls.
Database compute.
Snowflake warehouses.
Fabric capacity.
Power BI queries.
Storage.
Network transfer.
Logging.
Telemetry.
Monitoring.
Conversation history.
Security scanning.
Content filtering.
Document processing.
Data pipelines.

And increasingly:

Other AI calls made by AI.

Agent calls model.

Agent calls tool.

Tool invokes another service.

Another agent gets involved.

Another model evaluates the result.

The original user still sees: “Here you go.”

Architecture sees the receipt.

AI Can Wake Up Expensive Things

This one deserves special attention.

Some platforms charge based on compute being active.

Your traditional workload may be predictable.

An ETL process runs once an hour.

Reports refresh on schedule.

Users generate a known pattern of queries.

Then AI arrives.

Now queries happen whenever somebody asks a question.

8:03.

8:07.

8:11.

8:14.

8:19.

8:26.

A compute resource that used to wake up once an hour may suddenly stay active because AI keeps touching it.

That can completely change the economics.

The AI request itself may be cheap.

What it keeps awake may not be.

“Just Use the Best Model”

During a demo?

Sure.

Use the smartest model you have.

You’re trying to prove the idea works.

Production is different.

Does “What time does the Atlanta office close?” need your most expensive reasoning model?

Probably not.

Does “Analyze these three contracts and identify conflicting liability clauses” need something more capable?

Quite possibly.

Those requests don’t necessarily belong on the same model.

Production systems may need routing.

Simple tasks → smaller/cheaper model.

Complex reasoning → more capable model.

Known answers → maybe no model at all.

Cached answer → definitely don’t solve it again.

AI architecture isn’t simply send everything to the biggest model and hope Finance doesn’t notice.

The Most Efficient AI Call May Be No AI Call

This sounds strange in an AI series.

But if someone asks “What are our support hours?” and the answer is deterministic, frequently requested, and already available…

do we need a sophisticated reasoning process every time?

I wouldn’t think so.

Likewise, if 400 employees ask essentially the same question after a policy announcement, repeatedly rebuilding the answer may not make sense.

Caching matters.

Precomputation matters.

Traditional application architecture still matters.

AI didn’t repeal computer science.

Context Has a Cost Too

Agents Make This Even More Interesting

Our newly hired production user from the last article has been busy.

A user asks, “Find the problem and fix it.”

The agent reasons.

Calls a tool.

Examines the result.

Calls another tool.

Asks another model.

Retries something.

Checks the result.

Calls another API.

Validates the answer.

Writes the change.

Creates a ticket.

Sends the user a summary.

Very impressive.

Also, How much did that cost?

Agentic systems can turn one user request into a variable amount of work.

That makes cost predictability harder.

We don’t just need to know: How many users do we have?

We may need to know: How much work does a request cause?

Retries Are Expensive Too

Production systems fail.

APIs time out.

Databases get busy.

Models return unexpected responses.

Agents retry.

That’s normal engineering.

But poorly controlled retries can become expensive very quickly.

Especially when:

Retry 1 calls the model again.

Retry 2 repeats the search.

Retry 3 reruns the database query.

Retry 4 calls another agent.

And nobody notices because the user’s answer eventually succeeds.

Your reliability strategy can quietly become your cost strategy.

Cost Is an Observability Problem

If Finance asks: “Why did AI spending increase 40% this month?”

can you answer?

Not: “More people used it.”

I mean actually answer.

Which application?
Which users or departments?
Which use cases?
Which models?
Which tools?
Which data sources?
Which agents?
Which workflows?
How many requests?
How many retries?
What was the cost per successful task?
What changed?

Without that visibility, your AI budget is just another mysterious cloud invoice.

Measure Business Work, Not Just Tokens

Tokens matter.

But they’re infrastructure telemetry.

The business doesn’t care that an interaction consumed 14,382 tokens.

The business cares that the AI:

Resolved a support issue.
Prepared a customer summary.
Found a contract risk.
Saved an analyst thirty minutes.
Created a sales opportunity.
Prevented an error.

If an AI workflow costs $2 and saves an employee an hour, nobody should panic because another model could have done it for $1.60.

Likewise, a two-cent interaction that provides no value is still wasted money.

Cheap isn’t the same as valuable.

The metric we eventually care about isn’t simply: Cost per token.

It’s closer to: Cost per useful outcome.

Don’t Optimize Success Away

There is a danger here.

Once somebody notices AI spending increasing, the immediate reaction may be:

We need to reduce usage.

Maybe.

But increasing cost can be evidence of increasing adoption.

If 500 employees are using a system that saves each of them meaningful time, a larger bill might be fantastic news.

The goal isn’t make AI as cheap as possible.

The goal is make the economics understandable and sustainable.

Those are very different objectives.

Put Guardrails Around Economics Too

We already think about security guardrails.

AI needs economic guardrails as well.

Budgets.
Usage thresholds.
Rate limits.
Model routing.
Caching.
Context limits.
Query controls.
Agent step limits.
Retry limits.
Alerts.
Cost attribution.
Maybe even approval for particularly expensive operations.

Not because we want users afraid to ask questions.

Because production systems need boundaries.

Jeannine Shouldn’t Have to Care

This part matters.

I don’t want Jeannine staring at a token meter before asking:

“Why did Acme’s sales decline?”

That’s terrible user experience.

Users should ask useful business questions.

The architecture should decide how to answer them efficiently.

Maybe that’s a cached result.

Maybe it’s Power BI.

Maybe it’s SQL.

Maybe it’s search.

Maybe it’s a small model.

Maybe it’s the expensive model.

Maybe it requires an agent.

The user shouldn’t have to understand the economics of every component behind the answer.

That’s our job.

The Sherpa’s Lesson

A successful AI system creates demand.

That’s good.

But demand changes architecture.

The pilot asks: Can AI answer this question?

Production eventually asks: Can AI answer this question 5 million times?

And: What else happens every time it does?

Don’t just monitor the model bill.

Monitor the entire supply chain behind the answer.

Understand which workloads AI creates.

Understand which resources it keeps running.

Understand how agents multiply work.

Understand what different use cases actually cost.

And connect that cost to business value.

Because the goal isn’t cheap AI.

It’s sustainable AI.

So when somebody walks into the meeting excitedly announcing: “AI usage doubled again this month!”

you should absolutely celebrate.

Then ask: “Great. What did it cost us – and what did we get for it?”