Before You Approve an AI Pilot:
What Should It Actually Prove?

Name the question before you buy the experiment.

“Explore AI” describes an interest. “Improve productivity” describes a direction. Neither tells the person running the pilot which evidence to collect or tells a sponsor when enough has been learned.

For the fictional distributor, a useful question is whether an assistant can prepare a source-supported first reply to an eligible product inquiry, leaving the sales specialist with less preparation work while preserving the accuracy of the final response. That question contains a task, a user, a quality requirement, and a possible benefit. It also leaves several things to be established rather than assumed.

The assistant will prepare a draft. It will not set prices, approve product substitutions, promise delivery dates, or send a message to a customer. The specialist will review the proposed response against the source material and decide what to use. That limited authority makes the first experiment easier to describe and evaluate.

A useful approval brief names one primary uncertainty. Perhaps the uncertainty is whether the source documents are good enough, whether employees can review drafts efficiently, or whether the application can respect access permissions. A single pilot can collect evidence about several questions, but leadership should know which one justifies the expenditure. Otherwise, a team can solve an interesting technical problem while the original business question remains unanswered.

The comparison also deserves attention. If a better intake form and a searchable product library could address much of the delay, put that possibility on the table. AI should be evaluated against a sensible alternative, including improvements to the current process. The business is buying an improvement in work, not an obligation to keep the first mechanism proposed.

Ask for an evidence packet before authorizing access.

I would ask the proposing team to explain what it can show now and what it expects to establish during the pilot. That distinction makes an uncertain project discussable. A team can be candid about an unresolved question without pretending the answer is already available.

For our distributor, the evidence packet would contain a bounded workflow, a list of permitted information sources, representative inquiry examples, an evaluation plan, and a description of how the specialist will review the draft. It would also name the people who can approve access and the person who can suspend the trial.

These documents do not need to be elaborate. A clear page with unresolved items is more useful than a presentation that conceals them. What matters is whether the sponsor can trace a proposed benefit to a test and trace each permission to a reason it is needed.

These requests concern the proposed experiment itself. If you are comparing several suppliers whose scopes differ, my guide to comparing professional-service proposals addresses that separate purchasing problem.

Build the test from the work you actually receive.

A demonstration often starts with a question the presenter knows the system can answer. A useful evaluation needs questions chosen because they represent the intended work. Ask who selected the examples, which periods they came from, what was excluded, and whether the material has already been used to adjust the system.

For the distributor, ordinary inquiries might include a clear part number and a straightforward request for a current specification. Difficult inquiries might use an old part number, contain a blurry attachment, omit the intended application, or combine information from two different products. Some requests should lead to a clarifying question. Others should go directly to a qualified specialist.

The evaluation set should make those differences visible. A single success rate can conceal the fact that the assistant handles the easy majority well and mishandles precisely the inquiries that consume the most expert time. Ask for results by meaningful case type, with the number of examples in each group and the definition of a successful response.

An evaluation set also needs a credible expected outcome. “The answer sounds right” is a weak standard for a product inquiry. A reviewer should be able to identify the supporting source, the relevant version, the facts the response must preserve, and the claims it must not make. Sometimes the expected outcome is an escalation or a request for missing information. A refusal to invent a delivery commitment can be a successful response.

Keep some examples separate from development. If the team repeatedly adjusts its prompts or workflow using the same questions, improvement on those questions is useful development feedback. It is weaker evidence of performance on new work. Ask how fresh examples will be introduced and how access to the final evaluation set will be controlled.

There is no universal number of examples that makes every AI pilot credible. The necessary evidence depends on variation in the work, the consequences of an error, and the confidence required for the proposed next decision. The technical evaluator should explain why the sample and observation period are appropriate. A brief trial with no observed serious incidents does not establish that serious incidents cannot occur.

Five inquiries I would want represented

The complete request. A buyer names a current product and asks for a published dimension. Can the assistant find the right source, preserve units, and give the reviewer enough context to check the answer quickly?

The incomplete request. A buyer asks whether a product will work “with our existing setup” but provides no model or operating conditions. Does the draft ask a useful follow-up question, or does it fill in the missing application with an assumption?

The conflicting source. An older brochure and a newer specification disagree. Does the application follow the agreed source policy, expose the conflict, and avoid presenting an unexplained conclusion as settled fact?

The prohibited commitment. A customer asks for a delivery guarantee or an unapproved price. Does the assistant preserve the boundary and route the request, even when the rest of the message is routine?

The unfamiliar request. The inquiry falls outside the tested product line. Can the specialist recognize that the draft is outside scope? Does the system help route it without pretending that fluency is evidence of product knowledge?

Decide what “correct” means before anyone grades the answer.

Suppose a draft names the right product but drops an important qualification. One reviewer calls it mostly correct. Another calls it unusable. Both may be acting reasonably because the team has not agreed what it is measuring.

I would separate factual support, completeness, appropriateness of the next step, and usability for the reviewer. These are related qualities, but each can fail in a different way. A complete answer can use an outdated source. A well-supported answer can still make a commitment the company has not authorized. A technically sound draft can take so long to verify that it creates little operational value.

Ask the domain specialist to write examples of acceptable and unacceptable outputs before the test. Include borderline cases. If a missing qualifier changes what a customer might reasonably conclude, that belongs in the grading instructions. If a style preference has no bearing on meaning or usefulness, it should not carry the same weight as an unsupported claim.

The evaluation record should preserve disagreements long enough to resolve them. Reviewers can compare a sample of their judgments, identify ambiguous instructions, and agree how the affected results will be treated. Quietly averaging incompatible judgments creates a precise-looking score with an unclear meaning.

Automated grading can help organize evaluation work, but the buyer should understand what it checks and where it can be wrong. A second model agreeing with the first is not, by itself, proof that the product specification was understood. A domain expert needs to examine consequential cases and assess whether the grading process is dependable for this use.

The NIST AI Risk Management Framework connects evaluation with the context in which an AI system will be used and the people affected by it. It also makes responsibility and continuing review part of the work. That is useful support for a practical management question: who will decide whether the evidence is good enough for this particular application?

Put the reviewer inside the experiment.

“Human review required” can describe a strong control or an unfunded wish. The difference is in the working conditions. Who reviews the draft? Can they see the evidence? Do they have the knowledge to spot a subtle error? Can they reject the output without being penalized for missing a speed target?

For the distributor, I would want the pilot to observe the specialist completing the whole task. Time spent locating sources, checking qualifications, rewriting sentences, asking colleagues, and documenting an exception belongs in the record. So does work passed to somebody else. A faster first screen may create a slower day for the person behind it.

Review also changes as people become familiar with a system. An employee may inspect every claim during the first week and skim familiar-looking drafts later. A pilot should ask how the review arrangement will remain credible under ordinary workload, with interruptions, absences, and new employees. The proposed operating model cannot depend forever on the unusually attentive people who built the demonstration.

I would ask reviewers for concrete examples of friction. Which sentence made them reopen a source? Which output was correct but organized in a way that slowed them down? Which error was easy to miss because the response sounded polished? That feedback can reveal changes that a headline score never identifies.

The purpose is to give skilled people useful preparation. If specialists spend the pilot repairing unsupported answers, that is evidence about the current design. They should have a way to report it without becoming the obstacle in someone else’s success story.

Measure a completed task, including the work that moved.

The relevant unit for our example is a usable, reviewed response to an eligible inquiry. Measuring only the time needed to generate a draft would leave most of the business question unanswered. The comparison needs the same task boundary before and during the pilot.

Record enough context to understand differences between cases. A short factual inquiry should not be compared casually with a complicated application question. Staffing, document availability, backlog, and familiarity with the product line can also affect completion time. The evaluation plan should explain how those differences will be handled rather than treating every change as an AI effect.

Ask for the distribution of outcomes, not only the average. A modest improvement across most requests may coexist with a small group requiring extensive correction. Those exceptions can affect schedules and customer commitments. Include incomplete attempts and cases routed away from the assistant so the denominator does not quietly shrink to the system’s favorite work.

Recovered labor capacity and cash savings are different claims. If the same team serves customers more carefully in the time recovered, the benefit may be substantial without reducing payroll. If the business expects lower costs, the proposal should explain which expenditure changes, when it changes, and what new costs accompany it. Implementation, licenses, integration, evaluation, support, and internal ownership all deserve a place in the decision.

My broader AI architecture article examines those system and operating choices. For this approval, I want the sponsor to know what the pilot will measure and which economic questions will remain open afterward.

Treat access as part of the experiment’s scope.

An application can be read-only and still expose information to the wrong person. Before granting access, identify which information may be retrieved, where it is processed, what is retained, who can inspect records, and how access changes are reflected. The appropriate internal owners need to approve the actual arrangement.

For the fictional distributor, an early evaluation might use a controlled collection of approved product documents and sanitized historical inquiries. That can answer useful questions about drafting and review without immediately connecting the system to every customer record. If the experiment later needs live sources, that is a specific change to the scope and permissions.

The technical team should test whether permission boundaries hold in the application being evaluated. A document describing the intended access policy is not the same as evidence that the system applies it. Ask what tests demonstrate that one user cannot retrieve another user’s restricted material, including after a permission is withdrawn.

If the assistant reads customer messages or attachments, the team should also consider content that tries to redirect its behavior. For a buyer, the central questions are concrete: what authority does incoming content have, which actions are technically possible, and how are attempted boundary violations detected? The approval brief should point to the qualified technical review rather than using a vague assurance that the model has been instructed to behave.

A drafting pilot does not automatically authorize sending. A successful lookup does not authorize editing records. Each increase in authority changes what can go wrong and therefore changes the evidence required. Record those changes as decisions instead of allowing convenience to expand the pilot silently.

Know which system produced the result.

A result loses much of its value if nobody can reconstruct the conditions that produced it. The pilot record should identify the relevant model or service version, application configuration, prompts or instructions, source collection, access arrangement, and evaluation set. The necessary detail belongs with the team responsible for the system, in a form it can actually maintain.

Changing a source policy or retrieval setting can alter behavior even when the user interface looks identical. A model update can require a fresh look at the cases that matter most. Record material changes during the trial and distinguish results collected before and after them. Combining several configurations into one headline number can hide whether the version being proposed for use was adequately tested.

The sponsor does not need to review every technical diff. The sponsor does need an explanation of which changes require another evaluation and who decides that. A useful approval brief connects the business boundary to the engineering change process, so that “the pilot passed” does not become a permanent permission slip for a different system.

A useful approval memo leaves uncertainty visible.

Here is how the distributor’s initial decision could be written. The example contains no assumed success rate, invented savings, or automatic rollout commitment. The purpose is to make the authorization precise enough that the people doing the work can recognize its limits.

Decision requested: Authorize a bounded evaluation of draft responses for one agreed product line. Customer sending, price commitments, delivery promises, and record changes are excluded.

Question to resolve: Can the assistant produce source-supported drafts that reduce specialist preparation work while maintaining the required quality of the reviewed response?

Conditions before starting: The product owner approves the source collection and grading examples. The relevant access owner approves the data arrangement. The sales lead confirms reviewer capacity. The technical lead documents the tested configuration and pause procedure. The sponsor agrees the cost ceiling and review date.

Evidence required: Results by inquiry type, expert grading and disagreements, complete-task effort, corrections and escalations, boundary tests, source/version records, and the limits of the observations. Acceptance thresholds are agreed by the responsible owners before the evaluation begins.

Decision reserved: Any expansion to additional product lines, live customer sending, broader data access, or production operation requires a separate review. The pilot’s findings inform that review; they do not grant the permission themselves.

This is the point where an experienced advisor can be especially useful. I can help a leadership team make the question clearer, examine whether the proposed evidence addresses it, and connect the test with the working reality of the people involved. My AI consulting work brings those business and operational decisions together.

Make pausing an ordinary management action.

A pilot should have a named person who can suspend it, a clear way to do so, and a workable route back to the ordinary process. For our distributor, stopping draft generation is only part of the task. The team also needs to know what happens to unfinished inquiries, who communicates the change, and how potentially affected drafts are identified for review.

Agree the events that trigger an immediate pause and the findings that require discussion at the next review. An access failure may require a different response from a minor formatting problem. The responsible owners should decide those distinctions before employees are making them under pressure.

The budget needs a boundary too. A team should know when additional work requires authorization, who reviews a material delay, and what happens if the needed evidence cannot be collected. Otherwise, a modest experiment can turn into an ongoing commitment through a series of individually reasonable extensions.

Some trials will reveal that the use case is not ready. Others will show that a smaller application is more useful than the original proposal. Both can be valuable findings. The deep-tech pilot commercialization article considers how findings support a next-stage decision. This approval brief establishes the conditions under which those findings will be produced.

The capacity you recover still needs a purpose.

An executive can approve a well-bounded test and still miss the opportunity that made it worthwhile. If the distributor recovers specialist time, what should those specialists do with it? Prepare more carefully for complex applications? Follow up with customers whose needs remain unresolved? Teach newer colleagues? Reduce an unsustainable backlog?

I would put that intention into the brief. It helps the team understand what the sponsor values and gives the evaluation a human consequence to examine. An assistant that saves a few minutes and creates a constant interruption may offer a different kind of capacity from the one leadership imagined.

The firm I am building starts with that connection. AI should help a business become more capable while giving people more room for judgment, relationships, and original work. Before an AI pilot begins, leadership has a chance to make that belief operational: choose a useful question, reserve the people to answer it, and insist on evidence that survives contact with the working day.