What Does Your Best Expert Know That Your AI Doesn’t?

Knowledge and expertise

“What does the policy say?” is a familiar question for enterprise AI.

“Why did we hit our sales target but make less profit?” is a harder one.

Documents and retrieval-augmented generation (RAG) help with the first: retrieve the relevant passages, connect them to the question and prepare an answer. For the second, we usually turn to someone who understands how the business works. They start asking questions, comparing metrics and testing assumptions. A few minutes into the conversation, they might say: “Hold on. Check this first.”

What do they know that our AI system doesn’t?

An expert spends years studying relationships within a process. They understand which parameters affect each other, where a delay separates an action from its outcome, and under what conditions a familiar rule stops working.

Some of that knowledge is documented. But even if we describe it all in detail, we still have to apply it.

For example, we can document a rule: more delivery demand requires more couriers. But how many? At which sites, and during which hours? What happens if we also increase the minimum order value? How will that affect conversion, late deliveries, costs and customers’ future behavior?

Answering these questions requires examining combinations of conditions and comparing the consequences of different decisions.

These relationships can involve many variables, nonlinear effects and thresholds. A small increase in demand may have little impact in one situation and cause queues to grow rapidly in another. An experienced specialist knows how to account for such effects. How can we give AI that capability?

Modeling the process

Our starting point is to formalize the process itself.

Some relationships are already expressed in equations, including physical and chemical ones. Others can be estimated from historical data using statistical models. Simulation and expert rules can also help.

The result is a set of models that define the observed measures, the parameters we can control and the conditions we cannot influence.

The next step is to establish the outcome we want and the means available to achieve it.

Example: last-mile delivery

Consider a familiar example: the last mile of delivery.

Suppose we have a forecast of delivery requests by site and hour. We also have historical orders, the minimum order value in effect at each point in time, late-delivery rates and logistics costs for each hour.

Which metrics should we use to assess the process?

Revenue per courier per hour, the proportion of costs attributable to couriers and similar measures describe the economics. But we also need to consider counterbalancing measures: late deliveries, the rate at which visits turn into orders, and long-term customer loyalty.

What can we control?

The number of couriers, order batching rules, minimum order value, target delivery time, hourly pay and other parameters.

These parameters are interconnected. We can use the data to build models that predict how the system behaves under different operating settings, then use an optimizer to search those settings.

Choosing exactly what to optimize deserves a separate discussion. Queue length per courier? Late deliveries? Deliveries per courier per hour?

Reducing the number of couriers may lower delivery costs. But beyond a certain workload, queues will start growing. Raising the minimum order value changes delivery economics and may also affect conversion.

How do we find a solution among all these possibilities? Before searching for the best solution, we need to agree on what “best” means. Or at least what is acceptable.

What outcome do we need? What service level is mandatory? What trade-offs are acceptable in exchange for lower costs? Over what time horizon should we assess the effect?

That is a business decision. The models help evaluate its consequences.

At this point, we have several models that estimate the state of the process under different operating parameters. An optimizer uses those models to select feasible combinations of parameters according to the chosen objective.

This gives us a computational system that already captures a substantial part of the domain expertise: relationships, constraints and ways of assessing outcomes.

Agents and optimization

Now we can connect AI agents to it.

The agent has access to model descriptions, metric definitions, permitted actions and instructions for working with the optimizer. A user describes the task in plain language:

“Reduce delivery costs without increasing the current rate of late deliveries.”

The agent clarifies ambiguities, translates the request into a formal problem, configures the optimizer and runs the calculation. It then checks the result against the constraints, examines the proposed changes and explains the reasoning behind them.

The user can continue:

“What if the number of couriers is fixed?”

Or:

“Why are you suggesting more capacity at this site when it has fewer orders?”

The system returns to the models and calculations. Users can discuss the recommendation and explore alternatives.

We tested this idea in a different setting: a continuous manufacturing process. We used historical data, process models and an optimizer, adding an agent interface for defining tasks and discussing the results. The experiment gave us a concrete setting in which to examine what such a system requires: how to formalize the user’s intent, which checks are needed and what the calculations actually allow us to conclude.

A conversation with the metric

The most interesting possibility, though, is that the metric itself becomes something people can have a conversation with.

We often try to simplify a metric so that it is easier to explain to employees. Yet assessing a complex process may require a complex measure that accounts for service quality, resource use and other factors. Some changes also have delayed effects.

This approach allows the metric to remain complex. Its meaning, calculation method and underlying trade-offs must be defined and made available to the agents.

Management can discuss the overall outcome. Logistics teams can explore capacity and routing. Marketing can examine the consequences of changing demand. Teams at each site can explore the actions available to them.

Each department can ask questions from its own perspective while staying connected to the bigger picture:

“What drove the change in this metric?”

“How would our decision affect it?”

“Why does an improvement in our area create problems elsewhere?”

The answers may be unexpected. That makes it essential to trace their basis: which data was used, which assumptions were made, what was calculated and how sensitive the conclusion is to those assumptions.

Coordinating the process

A longer process may span several stages, each with its own models and optimizers.

In e-commerce, promotions affect demand, demand affects inventory and warehouse workload, and order fulfillment affects delivery and customer support. The effects of decisions extend across the chain.

Agents can help coordinate these stages by exchanging forecasts and constraints, clarifying available resources, running new calculations and comparing options against a shared objective.

This requires consistent data definitions and a coordination mechanism. Having agents exchange messages does not, by itself, improve a process. The architecture needs to account for the consequences of decisions beyond an individual stage and support improvements in the overall outcome.

Monitoring and adaptation

One more question remains: what happens when the process itself changes?

In manufacturing, raw material properties, equipment or measurement systems may change. In delivery, changes may involve demand patterns, courier availability, road conditions, service rules or the wider market.

A good recommendation can gradually lose its relevance. Monitoring therefore needs to be built into the system from the start.

We need to compare predicted and actual results, detect discrepancies and investigate possible explanations. Have conditions changed? Is there a data error? Has the model stopped capturing an important relationship? Was the decision implemented differently from what was assumed?

The answer determines how the system should adapt.

Sometimes, changing the optimizer’s settings is enough. Sometimes, resources need to be reallocated or processing rules revised. In other cases, a model needs recalibration, a new model must be built or more data needs to be collected.

In this setting, self-adaptation is a sequence of changes that can be checked: detect a deviation, investigate it, propose an adjustment, test it and evaluate the outcome. How independently the system can act depends on the task.

Building the system

Agents can then manage a complete cycle around the process models: problem formulation, calculation, verification, explanation, monitoring and adaptation.

This can be built incrementally. Start with one process and existing models. Define the metrics, constraints and available actions. Connect the computational tools, describe them for the agent, and set up logging and checks. Then add related process stages and feedback.

The speed comes from reducing manual work between problem formulation, calculations, scenario comparison and explanation. Trust rests on the ability to reproduce the calculation and check the basis of the recommendation.

Each department gains access to a system it can question about its own part of the process and its connections to the rest.

The conversation with the metric changes. Alongside “What is the number?”, people can ask: “Why?”, “Under what conditions?” and “What can we change?”