Stanford ECON295/CS323 I 2024 I The Age of AI, Eric Schmidt

AI in the Short Term

• The discussion focuses on the future of Artificial Intelligence (AI), particularly in the short term, defined as the next one to two years.
• The concept of a "million token context window" is introduced, which allows AI models to process and understand a vast amount of information, potentially up to 10 million words.

AI Agents and Text-to-Action

• The discussion then shifts to AI agents, which are defined as AI systems that can perform tasks, interact with the web, and potentially act on behalf of users.
• The importance of "text to action" is highlighted, which involves AI systems taking text input and translating it into real-world actions.
• The speaker predicts that the combination of large context windows, AI agents, and text-to-action capabilities will have a significant impact on the world, potentially exceeding the influence of social media.
• The speaker uses the example of a potential ban on TikTok to illustrate how AI agents could be used to create copies of existing applications.
• The speaker emphasizes the potential of AI agents to learn and adapt, drawing a comparison to human brains and their ability to process and retain information.
• The speaker describes a future where users can give commands in natural language to create programs, similar to how Python is used.
• This would allow anyone to have their own personal programmer, eliminating the need for expensive and limited human programmers.
• The speaker believes this will happen within the next year or two.

AI Development and Resources

• The speaker discusses the rapid development of AI models, with a significant gap emerging between the most advanced models and others.
• The speaker notes that the development of these advanced models requires massive investments, potentially reaching hundreds of billions of dollars.
• The speaker highlights the importance of energy resources for AI development, emphasizing the need for collaboration with Canada due to its abundant hydropower.
• The speaker discusses the potential for Nvidia to dominate the AI chip market, given the high demand for processing power.
• The speaker acknowledges the competition from companies like Intel and AMD, but notes that Nvidia currently holds a monopoly.
• The speaker mentions the challenges of switching between different AI chip architectures, highlighting the need for compatibility and interoperability.

AI Development and Competition

• The speaker expresses concern about Google's focus on work-life balance potentially hindering its competitiveness in the AI race, contrasting it with the dedication of startups.
• The speaker believes that founders are essential for driving innovation and success in industries with network effects, where time is a critical factor.
• The speaker highlights the importance of a strong work ethic and the need for companies to be willing to push their employees hard, citing examples from Google, Tesla, and TSMC.
• The speaker criticizes Microsoft's decision to partner with OpenAI, believing it was a risky move that could have jeopardized their AI leadership.
• The speaker emphasizes the importance of national security and the competition between the US and China in the field of AI.
• The speaker believes that the US is currently ahead of China in AI development, citing the US's chip advantage and the government's efforts to maintain this lead.
• The speaker mentions the US government's ban on Nvidia chips being exported to China and the potential impact on China's AI development.
• The speaker expresses confidence in the US government's commitment to AI development, citing the Chips Act and the Biden administration's AI Act.
• The speaker believes that the US and China are likely to engage in a long-term battle for knowledge supremacy.

AI Safety and Regulation

• The speaker discusses the challenge of detecting danger in AI systems that have learned something harmful but cannot explain what they have learned.
• The speaker mentions a threshold of 10 to the 26 flops for reporting AI development to the government, a rule similar to the EU's 10 to the 25 flops threshold.
• The speaker believes that the distinction between these thresholds will become irrelevant due to the development of federated training, which allows AI models to be trained on distributed data.

AI and Warfare

• The speaker discusses their involvement in the Ukraine war, specifically their work on developing AI-powered drones to counter Russian tanks.
• The speaker explains that their goal is to use AI to create more cost-effective and powerful robotic warfare systems, aiming to deter land invasions.
• The speaker acknowledges that this work makes them a licensed arms dealer, a path they do not recommend for others.
• The speaker expresses concern about the potential for Russia to gain a significant advantage in the war, potentially leading to Ukraine losing territory and ultimately the entire country.
• The speaker criticizes Marjorie Taylor Greene for blocking funding that could help Ukraine.

Understanding AI

• The speaker mentions an article they co-authored with Henry Kissinger and Dan Hleir about the evolving nature of knowledge.
• The speaker discusses the increasing complexity of AI models, making it difficult to understand their inner workings. He draws a parallel to teenagers, where we understand their existence but not their thoughts.
• The speaker suggests that we will eventually learn to understand the boundaries and limitations of AI systems, even if we cannot fully comprehend their internal mechanisms.
• He proposes the concept of "adversarial AI," where companies will specialize in breaking AI systems to identify vulnerabilities and understand their knowledge.
• The speaker believes that this approach will be crucial for developing the next generation of AI systems.
• He addresses the issue of AI hallucinations and suggests that advancements in technology will reduce these occurrences.
• The speaker emphasizes the importance of testing and validation to ensure AI systems perform as intended.
• He mentions "Chain of Thought reasoning" as a potential solution for improving AI performance and making it more transparent.

AI Development and Investment

• The speaker attributes the rapid progress in AI to factors like increased computing power, data availability, and significant investments.
• He defines AI as "Learning Systems" and highlights the emergence of new algorithms beyond Transformers.
• There is a belief in the market that the invention of intelligence has infinite return.
• The speaker believes that the industry will go through a huge investment bubble and then sort itself out.
• The speaker mentions a company called Mrol in France that has produced a second version of its model.
• The speaker believes that the capital costs of developing AI models will fundamentally change how software is built.
• The speaker believes that software programmers' productivity will at least double.
• The speaker mentions a company called Augment that targets large software programming teams.

Impact of AI on Society

• The speaker believes that the combination of context window expansion, agents, and text-to-action will have unimaginable impacts.
• The speaker believes that the context window allows for solving the problem of recency in AI models.
• The speaker gives an example of an AI agent model used in chemistry.
• The speaker believes that text-to-action will allow everyone to have their own programmer.
• The speaker believes that incumbents like Google are vulnerable to attacks from AI-powered competitors.
• The speaker discusses the issue of AI influencing public opinion and misinformation, particularly during elections.
• The speaker believes that social media companies are not well-organized enough to police misinformation.
• The speaker believes that misinformation is a significant threat to democracy and that the current system is not equipped to address it.
• He proposes public key authentication as a potential solution to combat misinformation, citing the example of digitally signed communications from public figures.
• The speaker argues that algorithms are biased towards outrage and sensational content, as this generates more revenue for companies.
• He advocates for a form of "equal time rule" on platforms like TikTok, which he considers to be more akin to television than social media.
• The speaker expresses concern about the lack of access to data centers for universities, hindering research and development in AI.
• He believes that the labor market impact of AI will be similar to previous technological waves, with low-skill jobs being replaced while high-skill jobs will adapt and evolve.
• The speaker suggests that computer science education should adapt to the age of AI by incorporating collaborative learning and the use of AI tools.
• The speaker discusses the different ways of doing gradient descent, including the use of Transformers, which are a systematic way of multiplying matrices.
• The speaker believes that India is the "big swing state" in terms of AI development, as many top AI professionals come from India to the US.
• The speaker expresses concern about the state of AI development in Europe, particularly in Germany, due to restrictions imposed by the EU.
• The speaker believes that learning to code is still important, even with the advancement of AI models, as understanding how these systems work is crucial.
• The speaker discusses the limitations of distributed training for AI models, citing the speed of memory to CPU or GPU as a major bottleneck.
• The speaker mentions that Nvidia is working on combining memory, CPU, and GPU functions into a single chip, which could potentially address the limitations of distributed training.
• The speaker believes that the use of copyrighted material for training AI models is a complex issue, drawing a comparison to the music licensing lawsuits of the 1960s.
• The speaker believes that the use of AI will lead to a similar royalty agreement as the one used for music, where a stipulated percentage of revenue is paid to the creators of the AI models.
• The speaker believes that the trend of large companies dominating the AI space will continue, similar to the trend in other industries.
• The speaker believes that the large capital required to build data centers is a major factor in the dominance of large companies in the AI space.
• The speaker mentions that Reed Hastings and Mustafa Suleyman, founders of Inflection AI, decided to partner with Microsoft because they couldn't raise the necessary capital to compete independently.
• The speaker believes that countries without the resources to develop their own AI models will need to partner with other countries or companies.

AI and Education

• The speaker emphasizes the importance of rapid prototyping in the development of AI-based products and services.
• The speaker suggests that students use AI tools to quickly prototype their ideas and stay ahead of competitors.
• The speaker encourages students to use large language models (LLMs) for their assignments, but emphasizes the importance of full disclosure when using these tools.

AI Regulation and Implications

• The discussion focuses on the implications of AI, specifically GPT, for businesses and regulation.
• The speaker addresses the challenge of creating incentives for global cooperation on AI regulation, acknowledging the potential for both benefits and drawbacks.
• The speaker highlights the "coopertition" concept, where regulation can both help and hinder competition, citing examples of companies seeking standards while also engaging in a "race to the bottom" on safety concerns.
• The speaker discusses the potential impact of AI on jobs, suggesting that high-skill jobs requiring human judgment will likely remain safe, while tasks requiring less judgment may be more susceptible to automation.
• The speaker notes that AI tools, such as coding assistants, are currently most helpful for moderately skilled coders, not beginners or experts, highlighting the need for some basic understanding to effectively utilize such tools.
• The speaker raises the question of whether this pattern of requiring some basic understanding will always hold true for AI applications.
• The speaker introduces the concept of applying the levels of autonomy in self-driving cars (Level 0-5) to other tasks in the economy, suggesting that this framework could be used to analyze the potential impact of AI on various industries.
• The speaker discusses the evolution of tasks from human-only to human-machine collaboration to fully autonomous machine execution.
• The speaker uses chess as an example, highlighting how human-machine collaboration was once superior to purely machine-based chess playing, but now machines like AlphaZero have surpassed this combination.
• The speaker proposes that the "middle zone" where humans and machines collaborate is beneficial for both productivity and shared prosperity.
• The speaker discusses the example of chip manufacturing, which is highly automated and requires minimal human intervention.
• The speaker notes that while chip manufacturing output is increasing, employment in the sector is not growing significantly.

AI Agents and Context Windows

• The speaker mentions that there is a growing trend towards AI agents and text-based AI models, which are expected to become more prevalent in the coming year.
• The speaker highlights the potential of AI agents to revolutionize tasks, as they can perform actions in a more dynamic and context-aware manner than traditional AI models.
• The speaker mentions that Andrew Ng has been advocating for the importance of AI agents, particularly in the context of 2024.
• The speaker discusses the importance of context windows in AI models, citing Eric Horvitz's taxonomy of fine-tuning, larger context windows, and retrieval augmented generation (RAG).
• The speaker highlights the significant progress made in increasing the size of context windows, allowing models to process large amounts of information, such as entire books.

AI Investment and Future Potential

• The speaker attributes the current surge in investment in AI to the abundance of "low-hanging fruit" in the field, leading to rapid advancements and a positive feedback loop.
• The speaker compares the current AI landscape to a period of increasing returns, where more resources lead to greater value and further advancements.
• The speaker emphasizes the importance of AI literacy for non-technical stakeholders, including policymakers and the general public, to ensure informed decision-making and understanding of the technology.
• The speaker discusses the importance of understanding the implications of AI, particularly in the areas of politics, economics, and labor markets.
• They argue that while technical contributions are important, the bigger bottleneck lies in understanding the business and economic implications of AI.
• The speaker uses the example of electricity as a general-purpose technology to illustrate the concept of complementary innovations.
• They explain that while the initial introduction of electricity into factories did not significantly improve productivity, it eventually led to the development of new factory designs and organizational structures.
• The speaker highlights the importance of organizational and human capital complementarities in realizing the full potential of general-purpose technologies.
• They cite the work of Paul David at Stanford, who studied the transition from steam-powered to electric factories, and found that it took about 30 years for factories to adopt a fundamentally different design that utilized distributed power and unit drive systems.
• The speaker emphasizes that understanding the implications of AI requires thinking beyond the technology itself and considering its impact on various aspects of society, including politics, economics, and labor markets.
• The speaker argues that while AI is a valuable technology, its full potential will not be realized until there are significant organizational and business model innovations.
• He draws a parallel to the steam engine and electricity, where it took decades for people to fully understand and utilize the transformative potential of these technologies.
• The speaker believes that AI will require a similar shift in thinking and organizational restructuring to achieve its full productivity gains.
• He acknowledges that while some AI tools, like ChatGPT, have been adopted quickly, he suspects that even larger productivity gains are possible once complementary innovations are developed.
• The speaker emphasizes that technology alone is not enough to drive productivity and that factors like regulation, cultural resistance, and safety concerns can significantly slow down the adoption of AI.
• He cites the example of improved radiology systems that were not widely adopted due to cultural resistance.
• The speaker also highlights the importance of investing in human capital and organizational capital to fully leverage the potential of AI.
• He suggests that there is a significant opportunity for business schools and economics departments to rethink their areas of study in light of the advancements in AI.
• The speaker acknowledges the need for increased funding for AI research and development, particularly in universities.
• He mentions that there are efforts underway to secure larger funding for AI research, but it remains uncertain whether these efforts will be successful.

AI Research and Development

• The speaker discusses the importance of both large language models and new algorithms in the field of AI.
• Jeff Hinton, a prominent figure in deep learning, emphasizes the significance of innovative algorithms, suggesting that universities have a competitive advantage in this area.
• The speaker highlights the role of patience and long-term research in universities, citing the example of fusion research.
• The speaker acknowledges the importance of scaling laws, which include increased compute, data, and algorithmic improvements, in the development of AI systems.
• The speaker expresses uncertainty about the proximity of achieving Artificial General Intelligence (AGI), noting that the definition of AGI is not clear-cut.
• The speaker mentions Peter Norvig's article arguing that AGI is already present in current AI systems, particularly large language models.
• The speaker points out the Moravec paradox, which suggests that tasks that are easy for humans, such as buttoning a shirt, are often difficult for machines, while tasks that are challenging for humans, such as solving complex mathematical problems, are often easier for machines.
• The speaker is referring to a technology officer of OpenAI.
• The speaker also mentions that the technology officer was briefly the CEO of OpenAI.
• The speaker invites the audience to ask questions to the technology officer.