When AI Can Give You an Answer, What Makes It Worth Trusting?

We're entering a world where almost anyone can produce a convincing answer to a complicated question.

A report can be analysed in seconds. A contract can be challenged. A strategy can be reviewed. A professional recommendation can be questioned by someone who has very little experience in the subject.

That's an extraordinary development.

It makes expertise more accessible and gives people opportunities to question information they might previously have accepted without challenge.

But it introduces another problem.

The ability to generate an answer is not the same as the ability to judge whether that answer is correct.

And I believe that distinction is becoming commercially important.

When the customer brings their own expert

A family member involved in a large architectural practice in Greece recently described a situation that caught my attention.

The practice produces professional reports based on technical expertise, research and judgement.

Increasingly, clients are putting those reports through AI and asking it to identify mistakes or weaknesses.

They then return with an AI-generated critique and request explanations or corrections.

Some of those questions are useful. A genuine omission or inconsistency should be examined regardless of whether a person or an AI system identified it.

But other findings may be based on incomplete context, misunderstood technical requirements or assumptions that don't apply to the particular project.

The architects must then investigate the claims and explain the reasoning.

In some cases, that means spending considerable additional time educating clients about why an apparently authoritative AI response isn't necessarily correct.

The practice is introducing an additional fee to address the time spent managing these AI-generated reviews.

The example raises an interesting question.

If AI makes it almost free to generate a professional-looking opinion, what happens to the value of the expertise required to assess it?

AI has changed the cost of questioning

Historically, obtaining a second professional opinion often required time, access and money.

Now a client can generate an alternative interpretation almost instantly.

That can be beneficial.

It can encourage scrutiny, expose weak reasoning and help people ask better questions.

But it can also create a flood of assertions that somebody else must evaluate.

A technically competent professional may now spend more time responding to questions that are confidently expressed but poorly grounded.

The cost of producing the question has fallen.

The cost of establishing a trustworthy answer hasn't necessarily fallen with it.

AI may make information abundant while increasing the importance of knowing what deserves to be believed.

That's not an argument against AI.

It's an argument for understanding the new responsibilities that come with using it.

A convincing answer can still be wrong

This isn't simply a theoretical concern.

In its 2024 Generative Artificial Intelligence Profile, the US National Institute of Standards and Technology identifies a problem it calls confabulation: AI systems generating inaccurate or false information and presenting it with apparent confidence.

NIST warns that this becomes particularly important when people rely on AI outputs to make consequential decisions.

It also recognises that fabricated explanations or citations can make incorrect answers appear more credible.

That is precisely the danger illustrated by the architectural example.

The client may be right to question the report.

The AI may have identified a genuine issue.

But the confidence of the critique cannot establish its validity.

Someone must examine the evidence, understand the context and determine what the finding actually means.

Source: NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, 2024.

What developing Sentio has taught me

I've encountered a related challenge through the development of Sentio, an emerging AI platform we've been building to help turn founder intelligence into organisational intelligence.

The ambition is to make important business context, relationships, decisions and experience more useful to the people who need them.

But working on AI systems that interpret organisational information has reinforced how easily a plausible explanation can go beyond what the evidence actually supports.

A person may report something they believe happened.

Another person may describe the same event differently.

A meeting summary may omit important context.

A statement about a relationship may be interpreted as an established fact when it was only one person's perception.

An AI system can then connect those pieces into a convincing explanation.

The problem is that the explanation may be wrong.

We've encountered these kinds of interpretation and attribution problems during development and evaluation.

They have reinforced an important principle for me.

Interpretation must not quietly become certainty.

A trustworthy system needs to distinguish what is known, what someone reported, what has been inferred and what remains unresolved.

Sometimes the right response isn't another answer.

It's a question.

Context changes meaning

Consider a business conversation.

A customer says they're unhappy with how a project is progressing.

What does that mean?

Is the work behind schedule?

Were expectations poorly established?

Has something changed in the customer's business?

Is there a history of earlier misunderstandings?

Does the customer believe a promise was made that the team doesn't remember making?

A transcript may preserve the words.

But the words alone may not explain the situation.

History, relationships, responsibilities and circumstances matter.

AI can help retrieve and connect relevant information. But it must be careful not to invent what it cannot establish.

That's why I believe trustworthy AI needs to be designed around clarification and correction, not simply confident answers.

And it needs to recognise that some things cannot be determined from the available information.

The human role is changing, not disappearing

There's a tendency to frame AI as a competition between human and machine intelligence.

I think that's often the wrong comparison.

The more useful question is how they should work together.

AI can analyse large volumes of information, identify possible inconsistencies, generate alternatives and help people examine unfamiliar subjects.

People bring contextual knowledge, professional responsibility, relationships, values and judgement about consequences.

Neither contribution is automatically reliable.

People make mistakes. Experts disagree. Organisations carry biases and incomplete knowledge.

AI can help expose those weaknesses.

But it can also introduce new ones.

The goal should be a process in which evidence can be examined, assumptions questioned, interpretations corrected and responsibility clearly assigned.

A professional shouldn't dismiss an AI-generated concern simply because it came from AI.

Equally, a customer shouldn't assume the concern is valid simply because AI expressed it confidently.

Both need a way to establish what the evidence supports.

Trust may become a service customers pay for

I suspect we're going to see changes in the economics of professional services.

If producing a draft report, analysis, design or recommendation becomes easier, customers may become less willing to pay traditional prices for production alone.

But they may place greater value on verification, explanation, professional assurance and accountable judgement.

The architectural example suggests one possible version of that change.

A firm may increasingly need to distinguish between producing its professional work and reviewing additional AI-generated challenges submitted by clients.

That additional work has a cost.

It also has potential value if it helps clients understand the evidence and make better decisions.

However, firms will need to demonstrate what the customer receives. Simply adding an AI fee without a clear service or responsibility behind it is unlikely to build trust.

The commercial opportunity isn't to charge people for confusion.

It's to help them reach sound conclusions in an environment where generating information is increasingly easy.

What should a CEO be asking?

As AI becomes more deeply embedded in business operations, I think leaders need to examine how their organisations establish trust.

Who verifies AI-generated information before it reaches a customer?

Can employees distinguish between source evidence and an AI interpretation?

What happens when a customer challenges a recommendation?

Who has authority to approve, correct or reject an AI-generated conclusion?

Can the business explain how an important decision was reached?

And perhaps most importantly, does AI make people better at exercising judgement, or simply more dependent on answers they don't fully understand?

These are not just technical questions.

They are questions about leadership, culture, customer relationships and commercial responsibility.

The value of knowing when not to answer

One of the principles we've adopted while developing Sentio is simple:

Think broadly. Claim narrowly.

Explore possibilities, but don't present an interpretation as a fact unless the evidence supports it.

Express uncertainty honestly.

Ask questions when understanding is incomplete.

Allow people to correct the system.

And keep responsibility for consequential decisions with the people authorised to make them.

That may sound less impressive than an AI system that confidently answers everything.

But I believe it's a more useful foundation for trust.

Because a business doesn't become more capable simply by receiving more answers.

It becomes more capable when its people can understand those answers, question them and act responsibly.

What will your customers trust tomorrow?

AI is changing how easily people can produce professional-looking work.

It's also changing how customers evaluate the expertise they purchase.

That creates both a threat and an opportunity.

Some traditional services may become less distinctive.

But the ability to establish what is reliable, explain complex issues and take responsibility for outcomes may become increasingly important.

The question for business leaders is not whether AI can produce a convincing answer.

It often can.

The question is whether your organisation can establish why that answer deserves to be trusted — and who is prepared to stand behind it.

In the new economics of value, that may become one of the most important things customers pay for.

Joe Papadatos is the founder of Two Icons. Sentio is an emerging AI platform within the Two Icons family, focused on organisational intelligence and human capability.

Grow Better.