AGI Is Coming Faster Than I Realized

Author’s Note: This post is free because I think it’s a super important topic that is overlooked. If you want to support this newsletter so I can put more time into creating better-quality articles on AI augmentation, please consider becoming a paid subscriber.

---

The rapid advancement of AI is about to change the world as we know it. The more research I do, the more my conviction increases.

To navigate this transformative era, it's crucial to understand two fundamental aspects of AI:

1. Its incredible potential

2. Its accelerating timeline

Why This Understanding Of AI Matters

Your grasp of AI's potential and timeline directly shapes your strategies for the future. If AI were progressing slowly with limited impact, it might not warrant much attention. However, if it were evolving at breakneck speed with transformative power, it would demand a radical shift in your approach to life and work.

The AI Scaling Laws and Moore's Law

The importance of understanding timelines and capabilities is why I’ve dedicated so much time to understanding:

• The AI Scaling Laws

• The timeline predictions of top AI experts

• Moore’s Law

These principles shed light on the significance of the recent launch of ChatGPT o1-preview, OpenaAI’s most advanced model. This model announcement is a bigger deal than people think.

ChatGPT o1-preview: A Game Changer

Traditional AI Scaling: The Old Way

Traditionally, AI got smarter by:

• Using more computing power

• Increasing model size (parameters)

• Training on more data

This scaling happened during the training phase, before the AI was ready to use.

The O1-Preview Breakthrough: Scaling During Use

ChatGPT O1-Preview introduces a game-changing approach:

• Scaling happens during the inference phase (when you're actually using the AI)

• The AI "thinks" before answering, like a human taking time to ponder

How It Works

1. You ask a question

2. Instead of answering immediately, the AI spends time reasoning (10-100 seconds)

3. This extra "thinking time" leads to more insightful answers

The New Scaling Law: More Compute = Smarter AI

As computers get faster and more efficient:

• AI can "think" longer for the same cost

• Example: What's now 10 minutes of AI thinking could become 10 years in the future

Potential Impact

• Tasks that take humans days could be done by AI in seconds

• More compute → Longer reasoning time → Smarter, more capable AI

• The path to AI agents that go out in the world and do things becomes more feasible.

The Big Picture Implications

• Compute is a critical resource to both scaling laws. This is why hundreds of billions of dollars are being spent on more and more advanced chips. According to Jensen Huang, the CEO of the largest chip company in the world (see video below), because of three exponentials (Moore’s Law, Training Scaling Law, Inference Scaling Law), the effective improvement rate of compute is 100,000x per 10-year period. To put the power of compute in context, Sam Altman, CEO of OpenAI, says, “What are the limitations of GPT? “There are many questions as to whether it exists, but I will confidently say ‘no.’” We are confident that there are no limits to the GPT model and that if sufficient computational resources are invested, it will not be difficult to build AGI that surpasses humans.”
  

• AGI may come faster than we think. As my conviction on AI’s timeline and capabilities evolves, I’m spending more and more of my time using and thinking about AI. And, I’m throwing away my long-term strategies, so I can think from AI-first principles and rapidly adapt in a “hyper-change” world.

Other AI Experts Add Context

UPENN Researcher Ethan Mollick gives a solid overview in Scaling: The State of Play in AI:

When the o1-preview and o1-mini models from OpenAI were revealed last week, they took a fundamentally different approach to scaling… o1-preview achieves really amazing performance in narrow areas by using a new form of scaling that happens AFTER a model is trained. It turns out that inference compute - the amount of computer power spent “thinking” about a problem, also has a scaling law all its own. This “thinking” process is essentially the model performing multiple internal reasoning steps before producing an output, which can lead to more accurate responses…

Unlike your computer, which can process in the background, LLMs can only “think” when they are producing words and tokens. We have long known that one of the most effective ways to improve the accuracy of a model is through having it follow a chain of thought (prompting it, for example: first, look up the data, then consider your options, then pick the best choice, finally write up the results) because it forces the AI to “think” in steps. What OpenAI did was get the o1 models to go through just this sort of “thinking” process, producing hidden thinking tokens before giving a final answer. In doing so they revealed another scaling law - the longer a model “thinks,” the better its answer is.

Just like the scaling law for training, this seems to have no limit, but also like the scaling law for training, it is exponential, so to continue to improve outputs, you need to let the AI “think” for ever longer periods of time. It makes the fictional computer in The Hitchhikers Guide to the Galaxy, which needed 7.5 million years to figure out the ultimate answer to the ultimate question, feel more prophetic than a science fiction joke. We are in the early days of the “thinking” scaling law, but it shows a lot of promise for the future.

Jensen Huang, founder and CEO of the largest chip company in the world, Nvidia, explains the significance of AI having more time to think:

Source: T-Mobile

Jensen Huang explains that the rate of progress in AI is Moore’s Law Squared. That is an improvement rate of 100,000x per 10-year period:

Source: Salesforce Dreamforce conference

Ben Thompson, my favorite tech analyst, gives his take on o1-preview’s significance:

Source: Ben Thompson

An o1-preview lead researcher shares how results scale with thinking time:

Source: Twitter

Source: Twitter

The new o1-preview lead researcher makes a pretty shocking admission about AI consciousness:

Source: Twitter