Averages lie…especially when evaluating AI accuracy.

Is an 80% model accuracy good? Here’s how you know:

Ask your team these two questions:
– When does the model perform well?
– When does the model perform poorly?

Here’s what you may find:
– Accurate on weekdays, but not weekends.
– Accurate for some customers, but not for the others.
– Accurate at peak times, but not at non-peak times.
– Accurate with 6-months of data, but not with 3-months of data.
– Accurate when users keep data clean, but not when system hygiene is poor.

The goal isn’t 100% accuracy.

The goal is the “right accuracy” under the “right conditions.”

Experienced AI teams do this well.

Articles for Startup Product Leaders
Get 1 concise, actionable tip each week

Subscribe

Don’t take my word for it…

“Your posts are consistently hitting the nail on the head. Appreciate the experience.”

“Please keep sharing your experiences – a lot of your recent posts had great nuggets of value”

“You will be hard-pressed to find a smarter, more caring, empathetic executive.”

“Josh is a stellar Product professional. Out of all the books on his desk, I expected one to be authored by him.”