How much of a routine task depends on something that was never written down?

Suppose a team tests an approved AI tool by asking it to turn a procedure into a checklist. The draft looks sensible until a colleague notices a missing step. Experienced people usually check with someone before proceeding, although the procedure never says so.

Another colleague raises a different question. That check used to be required. Is it still necessary?

The team now has something useful to investigate. There may be missing guidance, an outdated habit or a decision that nobody has clearly recorded. The AI response has exposed a question. It has not answered it.

This is one way a small, carefully bounded trial can help an organisation understand its knowledge. Better sources support better work. Using those sources in an approved task can reveal where they need attention. Both can develop together, provided the trial begins with clear safeguards.

Enough foundation for the task

Before testing, choose a low-risk task with a result that can be checked before anyone relies on it. Preparing a draft checklist from an approved procedure is a more manageable starting point than allowing a tool to direct work without review.

Identify the minimum source material needed. Confirm which version is current and whether it is suitable for the task. Someone needs responsibility for resolving questions about its content. If the task depends on a rule that is already disputed, settle that point or narrow the task before proceeding.

Agree which tool is permitted, what information may be entered and which sources it may access. Existing access to a document does not automatically mean its contents are approved for use in any AI service. Keep confidential or personal information outside the trial unless that specific use is authorised.

Name the person who will review the result and give them a clear way to raise uncertainty. Decide what happens when the source cannot support an answer. Leaving a question unresolved is an acceptable trial result.

These arrangements allow learning within a defined boundary. They do not require the team to repair every record in the organisation before starting.

Work out what went wrong

When an answer disappoints, it is tempting to conclude that the organisation has poor information. That may be part of the explanation. It needs checking.

Start with the source the tool actually received. Was the relevant passage included? If the tool searches documents, did it retrieve the right version and section? Information can exist in an approved source and still fail to reach the model. That points to a problem in how the material was supplied or found.

Then read the source without the AI response in front of you. Could a competent colleague answer the question from it?

If an essential step is missing, there is a source gap to investigate. If two current documents give different instructions, the responsible person needs to resolve the conflict. If staff rely on experience to interpret a sentence, some of that context may need to be written down.

Next, examine the task itself. A request to summarise a procedure may produce a reasonable overview while leaving out exceptions that a working checklist needs. Clarifying the purpose and what must be included may resolve the problem without changing the procedure.

There can also be a model error. The source may be clear, available and relevant, yet the response invents a requirement or leaves out an explicitly requested step. Correcting the document would not address that failure. The team may need to change how it uses the tool, strengthen the check or keep that part of the task with a person.

Sometimes more than one cause is involved. Record what the evidence supports and what remains uncertain. A polished second answer, by itself, does not establish why the first one failed.

Bring the question back to the work

Return to the missing check in our hypothetical procedure. The useful conversation is with the people responsible for that work.

What purpose does the check serve? Who has authority to confirm whether it remains necessary?

An experienced colleague’s explanation is valuable evidence. It still needs to be reconciled with the approved process. A local workaround should not become an organisational instruction simply because it makes an AI draft look more complete.

Once the responsible person confirms the requirement, update the maintained source or the workflow as appropriate. Make the correction available where people normally look. Saving it only in one person’s prompt leaves the next colleague with the same uncertainty.

Keep a small feedback record

For the next approved trial, keep one short record alongside the work. Include:

  • The task, instructions, approved tool and source version used, with the date and model version or settings where available.

  • What the response did, compared with what was expected.

  • The likely cause, the evidence for it and any uncertainty.

  • The agreed correction and the person responsible.

  • What happened when the task was tested again.

In the checklist example, the record could show that an approval step was absent from the source, that its owner confirmed it was still required, and that the procedure was updated. A retest would use the same instructions with the revised source and check whether the required step now appears correctly. Another person should be able to follow the revised guidance too.

A successful retest is evidence that the change helped in that case. Repeat the check on a few relevant examples before treating the correction as dependable.

Keep the record focused. Its value is the connection between a problem observed, a correction made and evidence that the correction helped.

Over a few tasks, it may reveal recurring difficulties worth addressing together. For now, start with one question the trial exposed and follow it back to the work.

The next person who opens that procedure should have a little less to guess.