1-2-1: Month six is the real test

I found out this week that my own AI team has been making recommendations off numbers that stopped being true in February.

Nothing was broken. Everything was running. That was the problem.

1 Principle

Systems don't fail at launch. They fail around month six, when the novelty ends and it turns out nobody owns them.

Launch has everything going for it. A date, an audience, someone's name against it. Month six has none of that. And the failure is silent: the thing keeps producing output at exactly the rate it always did. What decays is whether the output is still right, and your confidence in it outlives its accuracy by a long way.

2 Things Top of Mind

#1 My own system went stale and said nothing

I run my content analysis against a set of performance baselines. Median engagement, what counts as an outlier, which formats earn their slot. I built them in February, they worked, I moved on. This week one of my agents went looking and found the file hadn't been recomputed since. Seven months of recommendations, all confidently measured against an audience that had moved on without me. The median impressions had gone up by half. Nothing errored. It just got less true every fortnight.

This week: Pick one thing your systems rely on to make a judgement. A price list, a set of targets, a tone guide, a customer segment. Then work out the date it was last true. The timestamp on the file tells you when someone touched it, which is a different question.

#2 The rollout that still runs on last year's facts

You rolled something out in spring. Adoption was good, the team liked it, you moved on to the next thing. It's still running. The instructions behind it still name a service you stopped offering. It still routes questions to someone who left in July. Every output comes back fast, well formatted, and slightly wrong in a way nobody has the standing to raise. It works well enough that no one will ever escalate it. Which is exactly why it'll still be wrong in March.

This week: Put one person's name against one workflow. Give them fifteen minutes a month to read the output and say whether it's still true. That's the whole job. An unowned system will run wrong for as long as you let it.

1 Question

What in your business is still running confidently on facts that stopped being true?