Why The Next AI Breakthroughs Will Be In Reasoning, Not Scaling

Intro (0s)

• A conversation about achieving Artificial General Intelligence (AGI) took place around a year ago, with one argument suggesting that AI will eventually become capable of designing better chips than humans, eliminating a bottleneck to greater intelligence (5s).
• The idea is that this development will put us on a pathway to AGI in a way that wasn't possible before (15s).
• In a previous episode, the topic of discussion was what to do with two more orders of magnitude, but since then, Sam has expressed a desire to go to four orders of magnitude (25s).
• Currently, AI models are rapidly improving, with capabilities emerging that weren't possible a month ago (37s).
• This rapid progress is considered a significant moment in history (43s).
• The hosts of the podcast are Gary, Jared Harge, and Diana, and they are affiliated with Y Combinator, which has funded companies worth over $600 billion and hundreds of companies every year (58s).
• Recently, Sam Altman wrote an article that is relevant to the topic of discussion (1m14s).

The intelligence age (1m15s)

• A wild essay predicted that Artificial General Intelligence (AGI) and Artificial Superintelligence (Asi) are coming within thousands of days, with an estimated timeframe of 4 to 15 years (1m22s).
• The essay's ideas are similar to those discussed by Sam Altman, the founder of Open AI, in 2015, which at the time sounded like the ideas of a "crazy person" but now seem plausible (1m51s).
• In 2015, Sam Altman believed that AGI would be better at doing science than humans and would accelerate the rate of scientific progress in every field (2m57s).
• One of the motivations behind Open AI was to create an AI that could accelerate scientific progress, and this idea is still connected to the current work on advanced reasoning capabilities (3m10s).
• The development of advanced reasoning capabilities is crucial for AI to be able to do science and accelerate technological progress (3m32s).
• The paper on O1, a model developed by Open AI, highlights its capabilities and potential for the future, including its ability to perform well in chip design (3m49s).
• The ability of AI to design chips better than humans could potentially eliminate one of its bottlenecks for getting greater intelligence (4m1s).
• The current progress in AI development, including the capabilities of O1, suggests that we are on the pathway to achieving AGI (4m10s).

YC o1 hackathon (4m18s)

• Diode Computer is a company that builds AI designers for circuit design, and their previous product could handle PCB design, which involves four major steps: system design, component selection, schematic design, and layout and routing (4m37s).
• The company's previous product could automate schematic design and, to some extent, routing, but it was not able to handle system design and component selection (5m34s).
• The company has now demonstrated a new product, called 01, which can automate system design and component selection, allowing it to read data sheets and select the right components for a specific project (5m50s).
• The 01 product can take high-level constraints, such as building a wearable heart rate monitor, and match the specific components needed, including a microcontroller, accelerometer, and heart rate monitor sensor (6m7s).
• The product can then output a system diagram and generate code in a language called Arile, which can be used to build a PCB (6m57s).
• The output of the 01 product can be used to generate a layout for the board, which can then be fine-tuned and used to create a fully working printed circuit board (7m12s).
• The company's system can also call an auto-router on the specific board, allowing for the creation of a fully working PCB (7m41s).
• The 01 product goes beyond the traditional EDA (Electronic Design Automation) process, which involves design, simulation, and bug verification, by automating the entire process from system design to component selection and layout (7m58s).
• A paper used different models for different tasks and workflows, such as 40 mini for PDF extraction and 01 for reasoning, to select the correct components for a circuit board before placing it on the board, which is a common pattern in building interesting products with AI (8m26s).
• The process of selecting components, such as servos, motors, and sensors, requires a lot of thinking and is a hard task for humans, making it a suitable application for AI models like 01 (9m12s).
• Diode, a company, tried to use GPT-40 for component selection but failed, and then successfully used 01 for the same task, demonstrating a step-function capability unlock (9m25s).
• A hackathon organized by Diana featured actual YC-funded startups building features for their products using 01, showcasing how the model can unlock capabilities for real companies (9m52s).
• Camper, a company, uses 01 to create cat designs with natural language input, allowing users to design complex systems like air foils optimized for specific conditions without requiring extensive mechanical engineering knowledge (10m34s).
• Camper's system can run multiple simulations simultaneously and solve partial differential equations, making it a co-pilot for solid works and allowing users to design complex systems with ease (11m13s).
• The system can even write and solve equations, such as the Navier-Stokes equations, to solve air foil design problems, demonstrating the capabilities of 01 in reasoning and problem-solving (11m39s).
• Sam wants to increase the magnitude by four orders to reach a trillion-dollar spend, which is considered a significant and ambitious goal (12m0s).

4 orders of magnitude (12m9s)

• Abstracting complex concepts, such as understanding the nature of physics, could be possible if scaling laws hold, making it plausible to tackle difficult engineering challenges like room temperature fusion, weather prediction, and complex physical phenomena (12m21s).
• These complex physical phenomena are hard to solve and typically require PhDs, but advancements in AI, particularly with Chain of Thought and reasoning, could lead to breakthroughs in these areas (12m57s).
• The concept of providing feedback not just on outputs, but on the steps to get there, is a key idea in teaching models how to think, allowing for fine-tuning of various steps to ensure the model thinks as desired (13m21s).
• This approach is similar to the AGI conversations, which focus on teaching models to think better, rather than just producing correct answers, and is made possible by the scaling laws, which provide more surface area for throwing compute at the problem (13m47s).
• The scaling laws enable iterative improvement of results by spending more money and time, similar to what might be expected from a human scientific organization, and potentially even more consistently (14m10s).
• The architecture of the AI model is inspired by previous work, including the beginning of OpenI, and has been developed over many years (14m34s).

The architecture of o1 (14m42s)

• The AI model that won video game competitions, specifically DOTA, was a breakthrough in the tech industry, showcasing the power of reinforcement learning techniques, which were also inspired by Alpha Go and Alpha Zero (14m42s).
• DOTA is a complex game that requires resources and planning, and the AI model's success was due to its ability to learn from playing against itself a million times, using Q-learning as the fundamental algorithm behind reinforcement learning (14m42s).
• The connection between DOTA and GPT-type models lies in incorporating reinforcement learning into the generative model, which requires a large amount of factually correct data and a reward function to reason about the output (15m57s).
• The training process likely involved interesting techniques, including the use of secret data sources, such as math and science problems, to improve the model's performance (16m20s).
• There are two research directions being explored in parallel: scaling up the underlying language model (LM) and using reinforcement learning to unlock the model's potential in the real world (17m16s).
• The 01 model, which uses reinforcement learning, is a significant step forward, and its full version is expected to be a huge improvement over the 01 preview, with 02 and 03 models not far behind (17m35s).
• The 01 model is still opaque, and its development required creating a new dataset to train the chain of thoughts, which was a costly endeavor (18m0s).
• Large language models (LLMs) can be improved by breaking down tasks into steps and using evaluation sets, as discovered by Jake Heler for case text, which is also applicable to other tasks (18m29s).
• The prescription for improving LLMs involves two parts: breaking down tasks into steps and using evaluation sets, with the latter being crucial even if the model can break down tasks on its own (18m52s).
• Some companies have achieved 100% success by following Jake Heler's recommendations, which include having a large evaluation set and carefully testing every step of the reasoning pipeline (19m28s).
• The key to future AI breakthroughs may lie in reasoning, not scaling, with a focus on creating large evaluation sets and proprietary data that is not readily available online (19m42s).
• The value of a company's AI model (Moe) may ultimately lie in its ability to access and utilize proprietary data that is not publicly available, which can be achieved through enterprise sales and partnerships (20m25s).
• Startups may benefit from targeting customers who are willing to pay for high accuracy and perfection, such as those in industries that require precise and specialized knowledge (21m41s).
• Companies like Camper may be a good example of this approach, as they focus on providing high-quality and specialized products that require precise and accurate information (21m51s).
• The next AI breakthroughs may come from companies that are willing to do the hard work of collecting and utilizing proprietary data, rather than relying solely on publicly available information (21m36s).

Getting that final 10-15% of accuracy (21m52s)

• There is a growing interest in text-to-CAD design, particularly among hobbyists and those who want to quickly prototype and test their ideas, but also among professionals who require high accuracy and precision, such as those designing airplane parts (21m53s).
• The strongest technical teams will have the option to go all the way and cater to customers who demand 100% accuracy and are willing to pay a premium for it (22m20s).
• The use of AI in design and prototyping may not commoditize technology and make it less important to have a strong technical team, but rather the opposite, as the value will likely be captured by the strongest technical teams who can build on top of existing technology and achieve the final 10% of accuracy (22m49s).
• The key differentiators for companies using AI in design and prototyping will be the prompts, evaluations, UI layer, and integrations, as simply having good prompts is not enough for a company to adopt the technology (23m2s).
• Distribution, branding, and difficulty in switching will also be important factors in the success of companies using AI in design and prototyping (23m25s).
• The classic moats of software still apply, and companies that can establish a strong brand and make it difficult for customers to switch will have an advantage (23m45s).
• Evaluations will still be crucial in the world of AI, as founders will need to figure out how to build the best product on top of the technology (23m56s).
• Gigl, a company that was funded for a different idea, has pivoted to helping companies fine-tune open-source models to achieve equivalent performance (25m17s).
• Open AI was initially the primary focus, but it was found that such businesses are not great due to decreasing model costs and increasing performance of open-source models, making fine-tuning less necessary (25m30s).
• Companies like AO pivoted to finding vertical applications for their AI expertise, such as AI customer support, which is a competitive space but allows for squeezing out a comparative edge (26m1s).
• AI customer support deals with many edge cases and squishy problems, making it challenging, but intensely technical teams can still find ways to gain an edge (26m34s).
• Despite the potential, hardly any adoption of AI customer support has happened yet, and the space remains wide open (26m52s).
• Rules-based systems work well for simple cases, and there's a lack of trust in AI's ability to solve complex problems, which contributes to the slow adoption (27m10s).
• Companies like Zepto have started to adopt AI customer support, with Zepto automating 30,000 tickets per day and having over 1,000 people working on those tickets previously (27m50s).
• The automation of customer support jobs, although potentially replacing human jobs, can also free people from rote and unfulfilling work, allowing them to pursue more meaningful careers (28m20s).
• A previous implementation of a model had a 70% error rate, but after using a technique described by Jake Keer, the error rate was reduced to 5%, a significant improvement of an order of magnitude (29m29s).
• This improvement is particularly notable for complex, time-consuming, and expensive problems that were previously unsolvable, with the model now achieving 85% accuracy, up from 0% (30m9s).
• The model in question is 01 Preview, a new technique that is still being developed and refined, with the company trying to protect its advantage by hiding the actual model and using a fake one to give the impression of breaking down problems into steps (30m38s).
• The next step for 01 is expected to be the addition of interpretability and direct ability, allowing users to see the steps and edit them, which would be a significant unlock for the model's capabilities (30m56s).
• Currently, 01 can output a chain of thought, but it cannot be edited, and the ability to edit each step would take the model to the next level of fine-tuning (31m26s).
• The current state of models like 01 is the worst they will ever be, with rapid improvements being made week to week, and new capabilities emerging that were not possible just a month ago (31m39s).
• The improvement in models like 01 is expected to have a significant impact on various companies and ideas, with some benefiting greatly from the uplift, while others may not be as affected (31m56s).
• The opposite kinds of ideas that may not benefit as much from 01 are not explicitly stated, but it is implied that companies and ideas that rely on complex, time-consuming, and expensive problems may be less affected by the model's improvements (32m5s).

The companies/ideas that should pivot because of o1 (32m6s)

• Companies building AI coding agents or AI program engineers may need to reassess their strategies due to the advancements in o1, which is outperforming in solving programming problems (32m31s).
• The Chain of Thought infrastructure, which some teams have heavily invested in, may not be a significant leap forward with o1, as it is already directable (32m45s).
• The opaque nature of the Chain of Thought and the difficulty in altering its path once it starts are current challenges for users and systems (33m2s).
• New model capabilities can unlock new startup ideas, and the recent advancements in AI models have made phone calling-related startups successful (33m29s).
• The o1 series of models may enable new startup ideas that can improve the physical world, particularly in fields like mechanical engineering, electrical engineering, chemical engineering, and bioengineering (33m54s).
• These advancements could lead to real-world abundance and improvements in people's lives, rather than just minor conveniences (34m29s).
• There may be a sense of urgency to develop and apply these technologies, as there is a fear of AI in society, and it is up to the developers to create positive change (34m38s).

Outro (34m44s)

• The goal is for a technologist to help bring about an age of abundance sooner rather than later (34m46s).
• Achieving this goal could lead to abundance winning out over fear (34m52s).
• This concludes the current episode of "The Light Cone" with the next episode to follow (35m1s).