Barry O'Reilly on Judgement, Clarity, and Artificial Organizations
Product State Q&A
Barry O’Reilly is the the best-selling author of Artificial Organizations, Unlearn, and Lean Enterprise. He’s the Co-founder & Chief Innovation Officer at Nobody Studios, Founder of Outlier Venture Partners, and Founder of ExecCamp.
Website / LinkedIn
EC: As AI systems get better at generating answers, what does good human judgment look like? And where are leaders over-delegating it?
BO: AI has made answers cheap. That’s the problem.
Product leaders used to compete on access to information such as customer data, experiments, insights. Now everyone has that. The constraint has shifted. It’s no longer what do we know? It’s what do we decide with confidence and how fast?
In the work I’ve done across companies like American Airlines, Slack and Progyny, the leaders pulling ahead aren’t the ones using more AI. They’re the ones using it to strengthen judgment under pressure.
Good judgment today looks like:
Knowing what not to trust in AI output
Framing the right problem before asking for answers
Making calls with incomplete data—faster, not later
Owning decisions, not outsourcing them
Less decision reversal, and more on to the next limiting factor of your product
Where leaders get into trouble is subtle: They don’t hand over decisions, and hand over thinking. They accept AI-generated narratives without pressure testing them. That’s where things break.
We’ve already seen it. Deloitte had to refund an Australian government contract because AI-generated outputs included fabricated citations. The issue wasn’t the model, it was leaders treating output as truth.
The leaders getting this right are doing three behaviors differently:
Using AI to challenge—not confirm—their thinking
They ask: ‘What would prove this wrong?’Separating synthesis from decision
AI synthesizes. Humans decide. That boundary is non-negotiable.Rehearsing decisions before high-stakes moments
They pressure test options privately before committing publicly.
At Nobody Studios — the venture studio I cofounded where we’re building 100 AI-native companies — we see this daily. The founders who win don’t ask for better answers. They ask better questions.
If you’re a product leader, start here:
Before using AI, write down the decision you’re trying to make
Ask AI for counterarguments, not summaries
Force yourself to make the call without asking ‘one more question’
Because the risk isn’t that AI replaces your judgment.
It’s that you slowly stop using it.
EC: What’s a ‘successful behavior’ leaders cling to that isn’t working as it used to?
BO: Speed used to be the constraint. Now it’s judgment.
And yet most product leaders are still optimized for the last era:
Shipping faster
Running more experiments
Increasing output e.g. look how many agents I have running today!
That behavior worked. It got them promoted.
Now it’s becoming a liability.
I call this ‘productivity flex’—high output that looks like progress but degrades decision quality.
In one study I referenced in the book by MIT showed ~95% of AI pilots fail to deliver measurable business value. Not because the tech doesn’t work but because leaders bolt it onto old workflows.
The most dangerous legacy behavior?
Defaulting to activity instead of clarity.
More dashboards. More experiments. More features. Less understanding.
The companies that break out of this trap redesign how decisions happen—not how work gets done.
Take Amazon.
By 2013, they were deploying software every seven seconds—not because they worked harder, but because they built systems that accelerated decision cycles, not just execution.
Or look at Progyny, another case study in Artificial Organizations.
Their CEO didn’t use AI to cut headcount. He used it to eliminate cognitive load—so leaders could focus on better decisions, not more work. That single shift changed adoption entirely.
The pattern is consistent:
Legacy leaders optimize for output
AI-native leaders optimize for decision velocity × decision advantage
If you’re leading product today, challenge this:
Are you shipping more — or deciding better?
Are experiments increasing — or are insights compounding?
Are you measuring activity—or outcomes?
A simple test:
If your team stopped using AI tomorrow, would your decisions get worse?
If not, you don’t have an advantage. You have noise.
EC: What’s a clear signal that a team is optimizing for activity — and what should a leader do the moment they see it?
BO: This is the easiest failure mode to spot and the hardest to correct.
The signal is simple: Decisions aren’t getting easier.
You’ll see it in:
Meetings that end with ‘we need more data’
Experiments that never get killed
Teams debating the same issues repeatedly
Beautifully structured outputs with no clear action
In other words: More work, same uncertainty.
That’s not a tooling issue. It’s a judgment system failure.
As the book makes clear, the biggest breakdown in organizations isn’t data, it’s synthesis.
Teams don’t struggle to collect information. They struggle to decide what matters.
Steve Elliott, former Head of Jira Align (which he sold to Atlassian as AgileCraft, where I was an advisor), now CEO of Dotwork nailed this:
“Executives don’t trust systems, so they rebuild decisions manually in spreadsheets. Not because they love spreadsheets, because they’re the only place thinking actually happens.”
That’s the tell.
The fix isn’t more tools. It’s redesigning the decision loop:
Capture → Synthesise → Decide → Act
And tightening it relentlessly.
At Nobody Studios, we force this through one brutal rule: If a decision hasn’t improved in 30 days, kill the initiative. That changes behavior fast.
The moment you see activity over learning:
Stop the meeting and force the decision question
’What are we actually deciding right now?’Limit options to three
More options = Delayed accountabilityIntroduce a kill metric
If this fails, what signal tells us — and when?Track decision cycle time
Not output. Not usage. Time-to-decision.
Because learning isn’t about doing more. It’s about deciding better, faster and moving on.
One Final Thought
AI raises the bar for all leaders, product leaders especially.
The leaders who win won’t be the ones with the most tools. They’ll be the ones who redesign how decisions happen; first for themselves, then for their teams, then for the organization.
That’s what an ‘Artificial Organization’ actually is. And it starts with one uncomfortable shift: Stop optimizing for activity. Start optimizing for judgment.
“The risk isn’t that AI replaces your judgment. It’s that you slowly stop using it.”
— Barry O’Reilly
