AI features in production: what has to be true before customers rely on the answers

We tested a cheaper AI model for a system that reads business documents. On eight real documents, the cheaper model gave its own answers slightly higher confidence scores than the larger model it would have replaced. Compared field by field with the larger model’s output, it returned shorter lists of items in 7 of the 8 documents (15 instead of 19 in one of them), and in 3 documents it left out a key condition the larger model had captured.

On a dashboard, the switch looked obvious: about six seconds faster and 320 fewer tokens on every call (tokens are the units AI providers bill by), under 2% of the tokens a call uses. What stopped it was a habit, not a hunch: before any model change, both options run on the same real documents and the results are compared field by field. Since then, the reference has become stricter: a set of documents whose key facts a person has checked against the originals, so a comparison shows not only that two models disagree, but which one is right.

This guide is for companies whose product or core process already uses an AI model, or is about to, to read documents such as invoices, orders, emails or support requests, extract information from them, or make decisions. It covers five conditions that have to hold before customers rely on the answers, with the measurements behind them, and three checks you can run this month. If you are still deciding whether a prototype built with AI tools is ready for real users, start with what a vibecoded prototype still needs; if the question is a tool your team built for itself, see AI-built internal tools: keep, replace or rebuild?.

1. Confidence is the model’s opinion of itself

A confidence score tells you how sure a model is. It says nothing reliable about whether the answer is right for your documents, and a model can be most confident exactly where it is wrong. The cheaper model in the example was the more confident of the two.

The only measure that settles it is a reference set:

• Real examples. Start with 20 documents or requests your product actually handles, including the awkward ones: a scanned page, an unusual layout, an amount written in an unexpected way.
• The correct answer, written by a person. Someone who knows the material writes down what the model should return for each example, field by field, checked against the original.
• A score for every change. Every change of model, prompt or setting runs against the set before release, and the result is compared field by field with the version in use today.
• A set that grows. Every wrong answer a customer reports becomes a new example, so the same mistake is caught next time.

Twenty examples will not prove that a feature is right. They are enough to catch a change that is badly wrong: here, eight documents were enough.

The same habit settles cost questions

Measuring on your own documents also answers questions that are otherwise guesses. In the same system, letting the model reason for longer on one type of document filled in 62% of the key fields instead of 50%, for about 12% more output tokens. On another type of document, the same setting produced four times as many output tokens, took four times as long and filled no more fields. A filled field still has to match the reference to count as right, but a setting that fills nothing new is easy to rule out.

Both results are specific to that system and those documents, and that is the point: the right setting depends on your material, and only measurements on your own documents show it. Without them, a team either pays for quality it does not get or saves money on quality it needed.

2. The bill is decided by calls per task

The price per token is the vendor’s decision. The number of model calls behind one task is yours, and it usually matters more. In the same system, one complete analysis of a five-document file takes 12 model calls, about 201,000 input tokens and just over two minutes. Multiply that by every file, every customer and every month, and the way the feature is designed weighs on the bill at least as much as the price list does, and it is the part you control.

Five design rules keep that number under control:

1. Read clean data directly. When information arrives in structured fields, the system reads the fields. The model only sees genuine free text.
2. Match the model to the stakes. A small, inexpensive model handles routine steps; a larger one handles the steps where a mistake is expensive, and the reference set shows where that line is.
3. Answer an identical request once. A request that has already been answered is served from the earlier result.
4. Give every customer a budget. Each customer’s usage is counted against a fixed limit that the system enforces, and a call that has to be retried counts once.
5. Smooth out bursts. A large import is spread out, so one customer cannot use a month’s budget in an hour.

The result is a known AI cost per customer, so pricing becomes a decision rather than a guess. When a better or cheaper model appears, switching is a change to one step, tested against the reference, not a rewrite.

3. Customers keep working while the model runs

A feature that asks a model for an answer while the user waits works perfectly in a demo with one short document. With real documents and real users, the model is slower, requests time out, and people click again. Every extra click can start the same paid work again.

The production version of the same feature works differently:

• The request is recorded and acknowledged straight away, and the screen shows where the work stands: waiting, running, done or failed.
• A second click on the same document finds the job already running and shows its progress, instead of starting the same paid work again.
• The work runs in the background and survives a server restart, so nothing is lost halfway.
• Each step starts when the previous one finishes: the document arrives, its text is read, then the information is extracted.
• Every call to the model has a time limit and is retried only after temporary failures, such as a timeout, a rate limit or a provider error, never after a request the provider rejected.

For the business, a slow AI provider slows down one feature instead of the whole application, and more users mean a longer queue rather than more errors.

4. When a document gives orders

If customers upload files that a model reads, a sentence inside a file can work as an instruction to the model: ignore the previous rules, send this data, approve this request. This is called prompt injection, and it matters as soon as the model can act, not only answer. It is first on the OWASP list of security risks for applications built on language models.

Two safeguards carry most of the weight. The model gets no more rights than the user it works for, so a document uploaded by one customer can never reach another customer’s data through the model. And any action with consequences, such as sending, paying, deleting or changing a record, waits for a person to confirm it. In our AI solutions and automation projects, we decide, step by step, which decisions a model may take alone and which keep a person in the loop. Microsoft’s security team went a step further on 19 October 2026: an agent should start without standing access and receive only the permissions one task needs, for as long as it needs them.

5. Every answer can be traced

Months later, a customer questions a result. To answer, you need to know which version of the instructions the model was given, which model ran, how many tokens it used and how long it took.

That means two records. Every version of a prompt is kept, with who changed it and why, so a result can be traced to the instructions that produced it. And every call to a model is logged with the model, the usage and the duration. The same records show what a change did: whether the new prompt filled more fields on the reference set, and what it cost.

Three checks for this month

1. Twenty examples. Take 20 real examples, have someone who knows the material write down the correct answer for each, run them through your current setup and count how many it gets right.
2. Calls per task. Count the model calls behind one ordinary request to your most-used AI feature. That one number tells you more about next year’s bill than any price list.
3. The largest document, twice. Upload the largest real document you have, time it, then click the processing button twice in quick succession and see what happens.

The second and third checks take under an hour. The first takes longer the first time, and the set you build stays useful for every future change. A result you did not expect is the most useful thing a check can produce, because it shows where to start.

Tracking AI against set indicators is still rare. In PwC’s survey of 366 large companies in Central and Eastern Europe, presented on 8 October 2026, only 2% of the Romanian companies had an AI strategy approved by management, with objectives and indicators to track it, against 17% across the region.

Questions buyers ask

Should we switch when a new model comes out?

Run it against your reference set first. Switch when it scores at least as well on your examples, at a cost you accept. If the design keeps the model choice in one place, the test takes an afternoon and the switch is a small change.

How many examples do we need to start?

Twenty is enough to catch a change that is badly wrong. Grow the set with every mistake found in use, and with every new document type the product accepts.

Who writes the correct answers?

The people who know the material: the colleagues who read these invoices, orders or requests today. Engineers build the scoring; the reference itself is business knowledge, and it is one of the most valuable things the product owns.

Is this only for large systems?

The checks above work on any AI feature, including one built quickly with AI tools. The larger the share of the business that rests on the answers, the more the reference, the budget and the records are worth.

If your product already relies on AI output

Use the contact form to tell us what your AI feature does, what it reads, which decision rests on its answer, and how you check it today. We will reply with the first check we would run. Where the feature is part of a larger system, our custom software development projects cover the rest of what it needs to run in production.