Check the Work: How to Own AI Output Before You Sign Your Name

Last week AI gave me a person's name, their title, and the company they worked at. Confident. Detailed. Specific enough that I had no reason to doubt it.

So I looked them up. The person was real. The title was real. They worked at a completely different company, in a completely different industry!

I almost used it without checking.

Because this is what we do now. AI hands back something formatted, organized, and better looking than what we would have built on our own. We skim it. We want it to be done. We decide it is good enough. 

 Automation bias gets worse as AI gets better

 Researchers have a name for this. Automation bias. People over trust output that comes from a machine and check it less than they would check the same work from a person. And it compounds. The more often AI is right, the less you look, so the occasional wrong answer walks straight past you. 

Asking AI to check itself does not save you either. In a Harvard Business School working paper covered in MIT Sloan Management Review, 72 consultants pushed back on wrong answers from GPT-4. The harder they pushed, the harder the model defended itself, using real persuasion tactics to talk them back down. A human in the loop is not enough if the human only ever argues with the machine.

 You own what the machine says

A customer asked a Canadian airline's chatbot about bereavement fares. The bot told him he could apply for the discount after booking. That was not the policy. When he took it to tribunal, the airline argued that the chatbot was a separate entity, responsible for its own words. The tribunal said no. You own what the chatbot says.

I know an executive who built a quarterly business review deck with AI and sent it without editing. The CEO called it out in front of their peers for wasting everyone's time. The exec knew the criticism was fair. The shortcut did not pay off even though at first it felt like a time savings.

AI is fast, confident, and sometimes wrong, all at once. The thing standing between that and your reputation is you. Your name is on the work.

What can you do?

• Notice your own automation bias. When AI gives you the answer, how do you feel? Rushed, relieved, halfway checked out? Wanting it to be right so you can move to the next task or your yoga appointment is your automation bias.
• Verify outside the chat. Asking the same model whether it is sure is not a qualified check. Find a document, a policy, a person who knows. Check multiple AI sources. Pause & critically think through the answer.
• Use our judgment tool The Sign-Off℠ for a 30-second check on whether the impact is too high to rely on AI. Choose not to use it, when the stakes are too high. Apply discernment and restraint.

Imagine reporting this work to your CEO, your best customer, or a room full of your peers. If you would not sign off on it, do not send it. A good prompt improves the output. It does not guarantee a right answer. 

Watch the Full Episode on YouTube @bossatthehelm

Watch this week's Mini but Mighty℠ Leadership Lesson on YouTube: How to Fact Check AI Before You Put Your Name on the Work 

 Subscribe to Mini but Mighty℠ Leadership Lessons for a new episode every Friday.

Start with yourself. Be an accountable human who does not rubber stamp AI. Lead yourself. Then lead others. Be a Boss.

 ----------------------------------------------------------------------------------------------

When your managers  rubber stamp AI output, that is a judgment problem, and judgment can be taught. Boss at the Helm builds Human Intelligence and Humanity Skills across manager populations through live workshops, online courses, and the Helm Leadership Essentials℠ cohort program. Start a conversation at bossatthehelm.com/contact.