Eventually, your AI is going to be wrong.
Not necessarily because it hallucinated.
Not necessarily because the model failed.
It might have used stale data.
It might have retrieved the wrong document.
It might have followed an outdated business rule.
It might have encountered conflicting definitions.
It might have correctly interpreted information that was simply wrong.
Or somebody might have asked a perfectly reasonable question that nobody anticipated.
Whatever the reason, eventually someone is going to look at an AI-generated answer and say:
That’s just wrong.
Then comes the much more interesting question:
Whose problem is it?
Welcome to Production.
Everybody Owns a Piece
Imagine an internal AI assistant that answers questions about customers.
Nothing particularly exotic.
The AI platform is managed by IT.
Data Engineering built the pipelines.
The Data team manages the warehouse.
Security controls access.
Governance maintains policies.
The business owns the customer process.
A vendor provides the model.
Everything has an owner.
Wonderful.
Now the AI tells someone that a customer is inactive.
The customer is very much active.
Who owns that?
- IT?
- The Data team?
- The business?
- Governance?
- The AI team?
- One or more of the many vendors?
Suddenly the beautifully organized ownership diagram becomes considerably less helpful.
Because everyone owns a component.
Nobody necessarily owns the answer.
The Sherpa’s Notebook
I’ve seen versions of this problem long before AI entered the picture.
A report contains a number that someone at management level believes is wrong.
The investigation begins.
The report developer says: “That’s what the query returns.”
The database team says: “That’s what’s stored in the table.”
The application team says: “That’s what the source system sent us.”
The business says: “Well, that’s not what it means.”
And there it is.
Every technical component can be operating exactly as designed while the final answer is still wrong.
AI doesn’t eliminate that problem.
It adds another layer to it.
Now we can say: “That’s what the AI said.”
Which may be the least useful explanation of all.
“The AI Was Wrong” Isn’t a Root Cause
This phrase is going to cause organizations a lot of trouble:
The AI gave the wrong answer.
Why?
There are a lot of possibilities.
Maybe the model fabricated information that wasn’t supported by anything.
That’s an AI behavior problem.
Maybe it retrieved an obsolete policy document.
That’s potentially a retrieval, metadata, or document-management problem.
Maybe the source database contained an incorrect customer status.
That’s a data quality problem.
Maybe two departments define “active customer” differently.
That’s a business-definition problem.
Maybe the AI wasn’t supposed to see the information it used.
That’s a security problem.
Maybe the information was correct yesterday but today’s pipeline failed.
That’s an operational problem.
Maybe the AI gave a perfectly accurate answer to a poorly defined question.
That’s everybody’s favorite kind of problem.
These require very different fixes.
Blaming “the AI” tells us almost nothing.
The Answer Has a Supply Chain
We already understand this concept with data.
A number on a dashboard has lineage.
It originated somewhere.
It was transformed.
It moved through systems.
Business logic was applied.
Someone defined what it meant.
Someone decided who could access it.
Someone presented it to the user.
AI answers have a lineage too.
Or at least they should.
An answer might depend on:
User Question → Prompt/Instructions → Retrieved Context → Enterprise Data → Business Rules → Model → Response
If the final response is wrong, we need to be able to work backward.
- What information did the AI use?
- Where did that information originate?
- How current was it?
- Which business definitions applied?
- Which instructions were given to the model?
- What permissions affected what it could retrieve?
- Which model produced the response?
- What changed?
Without that visibility, troubleshooting production AI becomes:
“Can you ask it again and see if it does it?”
That’s not an incident-response strategy.
That’s troubleshooting by slot machine.
The Business Cannot Outsource Meaning
There’s another uncomfortable part of AI ownership.
- IT can own the platform.
- Data Engineering can own the pipelines.
- Security can own access controls.
- Governance can own the process around definitions.
But none of those groups should independently decide what active customer means.
Or recognized revenue.
Or a qualified lead.
Or a high-risk account.
Or whatever business concept the AI is being asked to explain.
Those definitions belong to the business.
This becomes especially important when an AI system starts answering questions directly instead of simply presenting rows and columns.
The AI is effectively communicating the organization’s understanding of its own information.
Someone has to be accountable for that meaning.
And “IT” isn’t a business definition.
The Vendor Doesn’t Own Your Answer Either
This is another tempting escape hatch.
“We’re using Vendor X’s model.”
Wonderful.
The model vendor may be responsible for the behavior and reliability of the model itself.
They are not responsible for your definition of an active customer.
They don’t know that Finance stopped using a particular field three years ago.
They don’t know that the policy document in SharePoint was superseded but never removed.
They don’t know that Status = 0 means something completely different in two of your applications.
They don’t know which of your reports the CFO actually considers authoritative.
That’s your organization’s context.
The vendor provides technology.
You provide truth.
So Who Should Own the Answer?
Probably not one person.
But that doesn’t mean that no one should be accountable.
Production AI needs ownership at several levels.
Platform ownership answers: Who keeps the AI service running?
Data ownership answers: Who is responsible for the quality and availability of the underlying data?
Business ownership answers: Who determines what the information actually means?
Security ownership answers: Who determines what the AI and its users are allowed to access?
AI product ownership answers: Who is accountable for how the overall system behaves for its users?
Those responsibilities can belong to different people.
They probably should.
But when something goes wrong, everyone should already know where the investigation begins.
You don’t want to have to design that process during the incident.
What Happens When Jeannine Finds It?
Jeannine asks the AI a question.
The answer looks wrong.
What should she do?
That sounds trivial until you realize how many AI implementations haven’t answered it.
- Is there a thumbs-down button?
- Does that feedback go anywhere?
- Can she report an incorrect answer?
- Can someone see what sources were used?
- Can the response be reproduced?
- Can someone determine which data was available at that moment?
- Can the business owner review the disputed definition?
- Can the problem be classified as model, data, security, retrieval, business logic, or something else?
- Most importantly: Does Jeannine know that questioning the answer is expected?
Because trust does not mean teaching users that AI is correct.
Trust means giving users a system that behaves predictably and a clear path when they believe it doesn’t.
Production AI Needs Incident Response
We’ve spent decades building operational processes around applications and data platforms.
Servers fail.
Queries slow down.
Pipelines break.
Storage fills.
Deployments go badly.
We monitor those things.
We alert on them.
We troubleshoot them.
We assign ownership.
AI introduces another category of production incident:
The system is technically healthy, but the answer is just flat wrong.
Everything might be green.
The API responded in 1.2 seconds.
The model returned successfully.
The pipeline completed.
The database is online.
Authentication worked.
And the user still received bad information.
Traditional uptime monitoring won’t catch that.
Which means production AI needs operational processes for answer quality, not merely system availability.
Don’t Wait for the Wrong Answer
Before production, pick a scenario.
The AI tells an executive something important that is demonstrably wrong.
Now walk through what happens next.
- Who gets contacted?
- Who can inspect the response?
- Who can trace the supporting information?
- Who determines whether the source data was correct?
- Who owns the business definition?
- Who decides whether the issue requires changing data, retrieval logic, instructions, permissions, or the application itself?
- Who communicates back to the user?
If the answer to several of those questions is:
“We’ll figure that out when it happens.”
You’ve identified some production requirements.
The Sherpa’s Lesson
AI ownership isn’t about deciding who owns the model.
That’s the easy part.
The harder question is:
Who owns the actual outcome?
Because an AI answer is rarely produced by one thing.
It’s the result of models, data, metadata, business definitions, permissions, retrieval, application logic, and human questions all interacting.
When the answer is right, nobody cares which component deserves the credit.
When the answer is wrong, everyone suddenly cares which component deserves the blame.
Don’t build your ownership model around blame.
Build it around accountability.
Make the answer traceable.
Make the business definitions owned.
Make feedback actionable.
Make escalation obvious.
Make it possible to determine what actually happened.
Because eventually your AI will be wrong.
The important question isn’t whether that happens.
It’s whether your organization knows what to do next.
Leave a Reply
You must be logged in to post a comment.