Video clip source: A conversation with OpenAI's CPO Kevin Weil, Anthropic's CPO Mike Krieger, and Sarah Guo
Why I’m Sharing This Post
Over the last few weeks, I’ve been sharing clips that illustrate how agents are the next big step in how AI will revolutionize knowledge work:
They Said It Was Impossible. Then The World's Top CEO Did It 5X Better In 1 Year With AI.
The AI Agent Paradox: When Everyone Has an AI Voice, Who Will Actually Be Heard?
AI-First Billionaire CEO: Building Your Own AI Dream Team Is Your Greatest Career Move
World's Top AI CEO: Smart Knowledge Workers Will Soon Have 2,000 AI Minds Working Alongside Them
This series is amazing because it illustrates real-world, transformative case studies, and then it unpacks their implications.
While the reception to the series has been very positive, several people have said they are overwhelmed by the prospect of urgently:
Reimagining how they do their work.
Figuring out the tools, prompting, and programming of how to use agents today.
Setting everything up so that agents can run semi-autonomously.
The beauty of this clip and my breakdown of it is that it provides a simplified breakdown of a larger framework that you can immediately start playing without programming knowledge.
This breakdown, along with my playbook that the paid subscribers get give you an immediate taste of what you can do with agents.
With that context set, let’s jump in…
Quick Overview
In this 90-second clip, OpenAI’s Chief Product Officer and Co-Founder of Instagram, Kevin Weil, gives us a rare glimpse inside how OpenAI and its most advanced customers use OpenAI.
Let’s break down his points one by one:
The Power of AI Specialization: The Future Isn't One Super-AI, It's Many AIs Working Together
Best Is Not Always Better: Why You Don’t Always Want The Most Capable Model To Perform A Task
AI Validates AI: The Counterintuitive Path to Reliability
The Collaboration Blueprint: Human Teams as Models for AI
#1. The Power of AI Specialization: The Future Isn't One Super-AI, It's Many AIs Working Together
“You end up putting together workflows and orchestration between models… So you use them for what they're good at.”
—Kevin Weil
What This Means
Think of AI models like specialists on a medical team. Just as you wouldn't want a heart surgeon to handle every aspect of your healthcare, you shouldn't rely on a single AI model for everything. The future belongs to those who can create "teams" of AI models working together, each handling what they do best.
This is counterintuitive because most discussions about AI focus on finding the "best" model, when in reality, the magic happens when you combine different models strategically. To better understand why, I recommend reading my article on the Wisdom Of Crowds.
How To Apply This
Beginner (manual)
Start by mapping out your current workflows and identify distinct tasks
For each task, list the key requirements (speed, accuracy, input, output, creativity, training data and examples, personality, etc.)
Begin experimenting with different AI tools for different parts of your workflow
Use ChatGPT o1-preview for reasoning
Use Claude for detailed analysis
Use MidJourney for visuals
Document which tools work best for which tasks
Manually iterate on improving the quality of the outputs for each task
Understand how the tasks chain together (output of one becomes the input of another) into simple workflows
Intermediate (no-code automation)
Create more and more complex workflows
Look for no-code automation tools (like Zapier or Make) to connect these models via APIs
#2. Best Is Not Always Better: Why You Don’t Always Want The Most Capable Model To Perform A Task
"Sometimes when you ask me a question, you don't want me to wait 60 seconds. I should just give you an answer."
—Kevin Weil
What This Means
There's a counterintuitive truth hidden here:
Sometimes the "smartest" AI isn't the best choice.
The most capable models often come with two drawbacks:
They cost more
They take longer to give you an answer (in the case of ChatGPT’s o1 model).
Just as you don't need a PhD to answer "What's 2+2?", not every AI task requires deep, slow, and expensive reasoning. This insight challenges the common assumption that we should always use the most advanced AI model available.
If you’re just paying $20/month to manually use ChatGPT or Claude, this insight may not matter because your cost is fixed. But, when you switch to customized agentic workflows, you’ll switch to using APIs, which means you’ll pay based on usage. Thus, as you create more complex workflows with multiple iteration loops, the costs of operating agents can skyrocket. Furthermore, “thinking time” delays can add up. So using cheaper, faster models customized for specific tasks becomes important.
How to Apply This
Audit your current AI usage and identify where speed is more critical than depth
Create a decision tree for routing questions to different AI models based on complexity
Set up simple A/B tests comparing fast vs. deep models for common tasks
Measure both response time and accuracy to find the sweet spot
Create clear guidelines for when to use quick versus deep-thinking models
#3. AI Validates AI: The Counterintuitive Path to Reliability
"They can hallucinate... But you can finetune models to be very precise about the kinds of inputs and outputs that they expect and have these models start working in concert together and models that are checking the outputs of other models realizing when something doesn't make sense, asking it to try again."
—Kevin Weil
What It Means
Instead of seeing AI hallucination as a fatal flaw for security applications, it can be managed by having AI models cross-check each other. It's like having multiple security guards who each specialize in different threats and validate each other's observations.
How to Apply This
Identify critical processes where accuracy and reliability are essential
For each of these processes, implement a secondary "validator" model to check the first model's output
Create clear error-handling procedures when models disagree
#4. The Collaboration Blueprint: Human Teams as Models for AI
"Going back to reasoning about how we work as humans. How do we do complex things as humans? You have different people who often have different skill sets, and they work together to accomplish a hard task."
What It Means
This observation reveals something remarkable: you already have deep intuitive knowledge about how to build effective AI systems. Throughout your career, you've developed a sophisticated understanding of how teams work. How to:
Spot talent
Structure collaborations
Orchestrate complex projects.
This hard-won wisdom about human teams turns out to be a powerful blueprint for designing AI systems.
Let’s go a little deeper. Think about how you've learned about teams by being part of them or leading them:
You know when to bring in a detail-oriented reviewer versus a big-picture strategist.
You understand the dance between quick, initial responses and deep, thorough analysis.
You've learned when to trust autonomous work and when to add oversight.
These same instincts can guide you in orchestrating AI systems.
The Hidden Patterns of Effective Human Teams You Can Apply To AI
Successful human teams exhibit several key patterns that translate directly to AI system design:
Clear Role Definition
Human Teams: Each member knows their core responsibilities
AI Application: Assign specific tasks to models best suited for them
Example: Using a fast model for initial screening, a thorough model for deep analysis
Efficient Handoffs
Human Teams: Smooth transfer of work between team members
AI Application: Clean data formatting between models
Example: One model summarizing content for another to analyze
Quality Control Systems
Human Teams: Peer review and approval processes
AI Application: Validator models checking output quality
Example: Having a specialized model verify factual accuracy
Adaptive Workflows
Human Teams: Adjusting processes based on feedback
AI Application: Dynamic routing between models based on task complexity
Example: Adding specialized models for edge cases
How to Apply This
Study your most effective human teams and document their interaction patterns
Map these interaction patterns to potential AI workflow designs
Start with simple two-model collaborations that mirror basic human partnerships
Gradually add more specialized AI models as needed, maintaining clear "roles"
Implement regular "performance reviews" of your AI workflows just as you would with human teams
Conclusion: You Can Participate In The AI Agent Revolution Even If You’re Not A Techy
The future of AI excellence isn't about building a single superintelligent system—it's about orchestrating specialized models that work together, much like how the most successful human organizations operate. As we've seen through Kevin Weil's insights, this organizational approach to AI isn't just theoretical—it's already transforming how leading companies design and implement their AI strategies.
For those who felt overwhelmed at the start of this article by the prospect of completely reimagining their work processes, there's an empowering truth:
You don't need to revolutionize everything overnight.
Just as the most effective human organizations evolve through incremental improvements and thoughtful collaboration, your journey with AI agents can begin with small, strategic steps that compound over time.
Whether you're starting with manual workflows or advancing to no-code automation, the principles remain the same: start small, focus on specialization, verify results, and learn from human organizational wisdom.
As AI tools continue to evolve and become more user-friendly, those who begin experimenting today—even with basic implementations—will find themselves uniquely positioned to harness more sophisticated capabilities tomorrow. The question isn't whether AI agents will transform knowledge work; it's whether you'll be ready to orchestrate your own AI dream team when that transformation accelerates.
For now, take that first step. Map out one workflow. Experiment with two complementary AI models. Let them cross-check each other's work. The future of work, at least for the next few years, isn't about replacing human intelligence—it's about amplifying it through thoughtfully designed AI collaborations.
In the end, remember this…
The magic doesn’t just lie in better individual models. It lies in intelligence that emerges from orchestrated collaboration between diverse models.
Paid Subscriber Bonus: The AI Team Validation Playbook
Once you break down a workflow into its smallest steps, you might have dozens, if not hundreds, of steps. And, if those steps don’t produce consistently reliable results, you may find yourself constantly jumping into your workflow to give the same advice over and over. As a result, you’ll have less time to work on the parts of your workflow that you enjoy the most and where you provide the most value. It can feel like you’re managing a team of low-level interns. In the end, you might conclude that working with agents is not worth your time.
Enter “The AI Team Validation Playbook"—templates for ensuring your AI outputs are accurate, reliable, and trustworthy.
This resource will help you stop wondering if your AI is hallucinating and start knowing your outputs are rock-solid.
