Don't build a pipe on a lie

Every clinic I know has a pipe somewhere that runs on a lie. A list that says a patient is overdue when she isn't. A medication list nobody has reconciled in years. A phone number that rings someone else.

When a person reads that list, they catch some of the errors. They know the patient. They squint at the date. When a machine reads it, it doesn't squint. It acts, fast and confident, on every line.

You don't want to create a new pipe that's based on a lie or an error.

Automation is a multiplier

Software doesn't make information better. It makes it travel. Good data, automated, reaches more people sooner. Bad data, automated, reaches more people sooner too, and now it has the authority of a system behind it.

So the first question for any automation is not "which model should we use?" It's "is the thing we're about to pipe actually true?"

The order I built it in

I didn't go straight to the language model. I was worried about exactly this. So I built from the bottom up, boring layers first.

• Check the source by hand. Pull the data with plain queries and compare a sample against the chart, line by line. If the list is wrong a third of the time, stop. Fix the list.
• Write the actions down. The system gets a fixed menu of things it is allowed to do: send this reminder, book this kind of visit, flag this for a nurse. If an action isn't on the list, it can't happen.
• Put an auditor on it. Every action is logged with what it read, what it decided, and why. If I can't replay it, it didn't happen.
• Then, and only then, the model. A language model is good at reading messy text. That is its job here. Reading, not deciding what medicine to give.

The gate

Every action has its own bar for how sure the system has to be. Checking whether someone's insurance is active can run on a lower bar. Paging a doctor at night needs a much higher one. Below the bar, the system does one thing: it stops and asks a person.

Two rules sit on top of that.

• The system can't lower its own bar. Only a person can change a threshold, and the change is signed and recorded.
• "I don't know" is a normal answer. A pause that hands the question to a human is the system working, not failing.

This is slower to build than a demo. It is also the only version I'd trust with my patients. (The line I won't let AI cross.)

If you're building one

• Measure the source before you automate it.
• Start with rules you can read, not a model you can't.
• Log everything you'd want to see in a review.
• Make "stop and ask" the easiest path, not the embarrassing one.

The model is the last mile. The pipe is the whole road. Fix the road first. Period.