How To Build The Future: Sam Altman

Coming up (0s)

• The goal was to pursue Artificial General Intelligence (AGI) from the beginning, despite it being considered an impossible task at the time (0s).
• Initially, saying AGI was the goal was not allowed in the field, and this led to criticism (1s).
• The initial pursuit of AGI was done with limited resources compared to other organizations like DeepMind (13s).
• To compensate for limited resources, the strategy was to focus on a single approach and concentrate on it (22s).
• Having a high level of conviction in one bet is valuable, and this is why startups are exciting (29s).
• The world still does not fully understand the potential of AGI and related technologies, making it a promising area for innovation (33s).
• Startups have opportunities in this area because the world is not yet fully aware of its potential (35s).

Intro: Is this the best time to start a tech company? (43s)

• The current time is considered the best time yet to start a technology company, with the potential for even better times in the future, as each successive major technological revolution enables more capabilities than before (1m12s).
• Big companies have an edge when things are moving slowly, but upstarts have an advantage during dynamic periods of technological revolution, such as the current age of intelligence (1m28s).
• The development of Artificial Super Intelligence (ASI) is estimated to be thousands of days away, with the potential for significant progress in the next three to nine years, leading to a system capable of doing many things (1m54s).
• The rate of progress in the field of intelligence has been compounding over the last three years, and if this rate continues or increases, significant advancements can be expected (2m15s).
• The potential applications of abundant energy, which may be unlocked in the near future, include fixing the climate, establishing a space colony, and discovering new physics, among others (3m13s).
• The discovery of abundant energy could lead to an age of abundance, where physical work is unlocked with the help of robotics, language, and intelligence, leading to significant advancements in various fields (4m5s).
• The spirit of techno-optimism and encouraging founders to think bigger is a unique aspect of Y Combinator (YC), which is essential in a world that often discourages ambitious ideas (3m39s).
• The potential for near-limitless intelligence and abundant energy is considered exciting and worth discussing semi-seriously, with many possibilities that cannot be imagined yet (3m23s).
• The two key inputs to achieving everything else that is desired are truly abundant intelligence and truly abundant energy, which would enable the creation of better ideas and their implementation in the physical world (4m32s).
• Having abundant energy would also allow for the widespread use of AI, which requires significant amounts of energy (4m57s).
• The simultaneous development of abundant intelligence and energy is a significant unlock, and it is unclear whether this is a natural effect of increasing technological progress or a coincidence (5m2s).
• In an age of abundance, robots could potentially manufacture anything, leading to material progress for everyone, not just the wealthy (5m19s).
• The availability of unlimited energy would have a direct impact on quality of life, and driving down the cost of energy while increasing its abundance is crucial (5m55s).
• Even if a major nuclear breakthrough is not achieved, solar power combined with storage is on a good enough trajectory to provide a sufficient amount of energy (5m43s).
• Eventually, all problems in physics will be solved, and abundant energy will be achieved, it is just a matter of when (6m3s).
• The concept of abundant energy is relative, and what feels like abundant energy today will feel insufficient to future generations (6m18s).
• The universe is vast, with a lot of matter, and there is still much to be explored and utilized (6m23s).

How Sam got into YC (6m27s)

• Sam Altman got into Y Combinator (YC) as a Stanford freshman in 2005, despite Paul Graham suggesting he wait until the next batch, and Altman insisted he was a sophomore and coming anyway (6m38s).
• Altman doesn't identify as a formidable person, but rather someone who questions why things have to be a certain way and is willing to do things from first principles (7m38s).
• He found YC to be a collection of "weird people" who were just doing their own thing, which resonated with him, and he believes that people can often just do stuff or try stuff a surprising amount of the time (8m6s).
• Altman thinks that having a peer group of people who believe in doing things and taking action is crucial, and he advises young people to find such a group as early as possible (9m2s).
• He notes that being around inspiring peers is extremely valuable, and he was surprised by how much more potent this was with a room full of founders at YC compared to his time at Stanford (9m43s).
• Altman believes that peer groups can be very powerful in shaping one's ambitions and actions, and he quotes Carl Jung, saying that the world will ask you who you are, and if you don't know, it will tell you (10m17s).
• He emphasizes the importance of being intentional about who you want to be and who you want to be around as early as possible, and notes that no one is immune to peer pressure, so it's essential to pick good peers (10m38s).

The early days of YC Research (10m53s)

• YC Research was established to bring experimentation to Y Combinator, with a focus on funding various research efforts, including AI, which was gaining popularity around 2014-2016 (10m53s).
• The idea for YC Research was inspired by the history of Xerox PARC and Bell Labs, and the desire to create good research labs in Silicon Valley again (11m47s).
• The goal of YC Research was to allocate capital to smart people and projects, with the understanding that some would succeed and others would fail (12m4s).
• AI was having a "mini moment" around 2014-2016, with the publication of the book "Superintelligence" and impressive results from DeepMind, but it was still unclear if AI would be successful (12m14s).
• The establishment of YC Research was also influenced by the success of ImageNet, a deep learning model that could identify objects in images, including distinguishing between hot dogs and other objects (12m38s).
• The founder of YC Research had been an AI enthusiast for a long time and saw the establishment of the research lab as an opportunity to try to make a significant contribution to the field (12m33s).

Getting the first OpenAI team together (12m49s)

• The initial people involved in YC Research and OpenAI included Greg Brockman, who was known from the early Stripe days, and Ilia, who was discovered through a YouTube video and impressed with his intelligence and presence (12m53s).
• The conversation with Greg Brockman started with a discussion about AI ideas and the goal of starting a lab, which led to the decision to go after Artificial General Intelligence (AGI) from the very beginning (13m36s).
• This goal was considered ambitious and possibly crazy at the time, but it attracted the attention of young and talented people who were willing to take a risk (13m51s).
• The group was formed over the course of nine months, with people meeting one by one and in different configurations, and it started to take shape (14m53s).
• One of the favorite memories was the first meeting at Greg's apartment on January 3rd, 2016, where around 10 people gathered to discuss the next steps after announcing the project in December 2015 (15m11s).
• The initial goals for the effort were to figure out how to do unsupervised learning, solve reinforcement learning (RL), and never get more than 120 people, although the last goal was not achieved (16m57s).
• The original ideas, including the use of deep learning and the development of a big unsupervised model, were incredibly right and have led to the current system, despite many detours and twists along the way (16m9s).
• The early people, particularly Ilia, were impressed by their ability to stick to the big picture and adapt to changes, and the project has evolved into something very crazy and improbable (15m58s).

Why scaling was considered heretical (17m13s)

• The idea of scaling was considered heretical, with many people finding it offensive, as they believed that making neural networks bigger wouldn't necessarily make them better (17m16s).
• The core beliefs at the time were that deep learning works and gets better with scale, which were somewhat heretical beliefs (17m33s).
• Although people already knew that making neural networks bigger made them better, there was a strong belief that this wouldn't continue to be the case, with many eminent leaders in the field expressing this view (17m50s).
• The criticism was not just about being wrong, but also about the potential negative consequences of pursuing this idea, such as perpetuating an AI winter (18m29s).
• Despite the criticism, the results kept showing that scaling up neural networks made them better, which eventually led to a focus on scaling as a key area of research (18m37s).
• The idea of scaling was seen as an emergent phenomenon that was important and fundamental, even if the details were not fully understood (19m0s).
• The approach was to concentrate on scaling and push it as far as possible, rather than trying to outsmart themselves by pursuing multiple approaches at once (19m29s).
• The team at Open AI believed in the power of scale and its emergent properties, and were willing to push it to its limits, even if it meant going against conventional wisdom (19m51s).
• The team had a talented group of researchers who were motivated to work on scaling, and were able to access the necessary compute resources to pursue their research (20m29s).
• The criticism from some in the industry was that Open AI was wasting resources by focusing too much on scaling, and that this would lead to an AI winter (20m54s).
• However, the team believed in the value of conviction and focus on a single bet, rather than spreading resources across multiple bets (21m22s).

Conviction can be powerful (21m42s)

• When faced with pushback that doesn't make sense, it's essential to proceed anyway, as there are no "adults in the room" with all the answers, and one must iterate quickly to find their way (21m46s).
• Having conviction is crucial, but it's equally important to adapt and be truth-seeking when proven wrong, as being high conviction and wrong without adapting is not effective (22m37s).
• Conviction is necessary when operating without data, but it's essential to be willing to change course when data becomes available, as holding on to conviction past the moment of data can be detrimental (23m8s).
• Prioritization is an exercise that increases the likelihood of success, as making a choice and focusing on it is crucial when faced with limited options (23m37s).
• The development of language models was not a straightforward process, and the journey involved many assumptions, mistakes, and lessons learned along the way (24m0s).
• One of the strengths of the team was their ability to be wrong, learn from their mistakes, and adapt quickly, which was essential for making progress in scientific bets and product development (24m27s).
• The team initially worked on various projects, including robots, agents, and video games, before discovering the potential of language models, which was not immediately apparent (24m52s).
• A key insight around positive or negative sentiment in language models was discovered by Alec Radford, who noticed a single neuron flipping between positive and negative sentiment while generating Amazon reviews (25m28s).
• This insight led to the development of the GPT series, which was a significant breakthrough in unsupervised learning at the time (26m10s).

Commercializing GPT-4 (26m15s)

• Jake Heller from CaseText, a Y Combinator (YC) alum, was one of the first to commercialize GPT-4 in a significant way, building huge test cases around it and eventually selling his company for $650 million after gaining access to GPT-3, 3.5, and 4 (26m16s).
• Heller described getting GPT-4 as a major breakthrough, as GPT-3.5 would still hallucinate too much to be used in a legal setting, but GPT-4 could be made to do exactly what he wanted if he chopped the prompts down into a workflow (26m25s).
• Initially, founders were not interested in building businesses on GPT-3, but with GPT-3.5, startups began to show interest, and with GPT-4, there was a surge in demand, with users wanting to buy the product (27m15s).
• The development team was impressed with GPT-4's capabilities, running tests and playing around with it, but they were aware that the real test was putting it in customer hands (27m54s).
• The team was excited about GPT-4, but there was anxiety until the product was released to users, as the real test of a product's success is user adoption (28m41s).

What drew Sam to create Loopt (28m53s)

• Sam Altman started a company called Loopt at 19, which was a geolocation service that allowed users to find their friends, about 15 years before Apple created a similar feature (28m54s).
• The idea for Loopt was conceived when mobile phones were just starting to emerge, about 3 years before the iPhone was released (29m25s).
• At that time, it was clear that carrying computers in pockets was going to be a significant development (29m30s).
• Altman was drawn to the idea of utilizing mobile phones, which were initially just used for making calls and sending texts (29m39s).
• He remembers getting his first phone with internet access, which had a slow, text-based browser that allowed him to check his email (29m49s).
• Altman became hooked on the idea of mobile phones as portable computers after getting his first phone with internet access in high school (30m7s).
• He envisioned the potential of mobile phones beyond their initial limitations, such as the dial pad, and saw them as a promising technology (30m17s).

Learning from platform shifts (30m24s)

• The idea of technology and its future was present from a young age, with the first computer being an LC2, a significant difference from the modern smartphones of today (30m35s).
• The experience of going through a platform shift, such as the launch of the iPhone and the App Store, was valuable in understanding the messiness of the beginning and the impact of small actions on the direction it takes (31m10s).
• This experience was a result of being part of the launch of the iPhone and the App Store, which provided insight into the process of creating a platform shift (31m8s).
• The experience of managing people, doing Enterprise sales, and other aspects of running a company were useful lessons learned from the first startup, despite it not being successful (31m51s).
• The rate of experience and education gained from the first startup was incredible, with the idea that one's 20s are an apprenticeship for future work, as quoted by PG (32m9s).
• The difficulty of finding product-market fit and escape velocity was a significant challenge, but the experience was still valuable for learning and growth (32m22s).
• Doing a startup has a high rate of generalized learning, making it a great way to gain experience and knowledge, especially at a young age (32m32s).
• The current generation of 18-20 year olds are deciding to forgo traditional education to pursue opportunities in the rapidly changing tech landscape, in order to not miss the wave of innovation (33m5s).

Tech incumbents are unaware of what is happening with AI (33m15s)

• Many successful company founders, even those of billion-dollar companies, are not aware of the current developments in AI, which is surprising and creates opportunities for startups (33m16s).
• The lack of awareness about AI among established companies is reminiscent of when Facebook almost missed the shift to mobile because they were focused on web software and had to acquire companies like Instagram, Snapchat, and WhatsApp to adapt (33m42s).
• Platform shifts are often driven by young people with no prior knowledge, who are able to approach problems with a fresh perspective and build new technologies (33m58s).
• The current state of the world, where many are still unaware of the potential of AI, creates an exciting opportunity for startups to innovate and make a significant impact (33m29s).
• The YC Founders are taking advantage of this opportunity by quickly building new and innovative technologies, often with a better understanding of the current landscape than more established companies (33m36s).

Sam's recommended startup path (34m8s)

• Founders like Elon, Bezos, and others started their journey with a different project before moving on to their current ventures, and it's worth considering whether there's a recommended path for those who want to work on hard tech projects, such as solving the money problem first or running towards the project directly (34m10s).
• Having the ability to invest one's own money in a project can be beneficial, as seen in the case of Open AI, where early checks were written, and it would have been hard to get someone else to do it at the beginning (34m55s).
• Investing in projects can provide valuable lessons and support, but it's also possible to feel like time is being wasted if the project doesn't work out, as was the case with Loop (35m28s).
• If given the chance to go back in time, it's hard to say what would be done differently, but it's possible that a better project could have been chosen instead of Loop, and AI was always the main goal (35m41s).
• The history of people building technology to improve lives is long, and it's worth appreciating the work of those who came before, even if they're not well-known, as their contributions have had a significant impact (36m10s).
• It's nice to be able to add to the progress made by others and to be grateful for the work of those who have contributed to the development of technology (36m50s).

Reflecting on the OpenAI drama (36m56s)

• OpenAI has had a great year, but not without drama, and the company is good at creating drama, with the past year being a speedrun of a medium-sized or big tech company's growth, which normally takes a decade, but OpenAI has achieved in less than two years (36m56s).
• As the company scales, it goes through management teams at some rate, and the people who are good at the zero-to-one phase are not necessarily good at the one-to-ten or ten-to-100 phase (37m35s).
• OpenAI has made plenty of mistakes, done a few things right, and changed what they're going to be, which comes with a lot of change, but the goal is to keep making the best decisions at every stage (37m50s).
• The company is heading towards a period of more calm after a dynamic period, but there will be other periods in the future where things are very dynamic again (38m18s).
• OpenAI's quality and pace are beyond world-class compared to established software players, and for the first time, they feel like they know what to do to build an AGI, which will still take a huge amount of work (38m30s).
• The company knows what to go for and what to optimize for on the product side, and with clarity, they can move fast if they're willing to focus on a few things and try to do them very well (39m12s).
• OpenAI's research path, infrastructure path, and product path are fairly clear, which allows them to orient around that and move fast (39m24s).
• The degree to which everybody can get aligned and pointed at the same thing is a significant determinant in how fast the company can move (39m51s).

What startups are building with current models (39m58s)

• Recent advancements in technology have led to a rapid progression from level one to level two, and now to level three, with the potential to reach level four, which is the innovator stage, where systems can iteratively improve and create new things, such as an airfoil that can fly, as demonstrated by a CAD Cam startup during a hackathon at YC (40m23s).
• The use of current models in creative ways can lead to a huge amount of innovation, as seen in the demo by camper, which used underlying software for CAD Cam and language as an interface to a large language model, allowing for tool use and code generation (41m10s).
• The combination of language models and code generation can create tools for itself and compose those tools, leading to a rapid acceleration of innovation (41m35s).
• The potential for future advancements is vast, with the possibility of discovering all of physics and solving complex problems, making it an exciting time to be alive (41m57s).
• The concept of AGI has become overloaded, and a new framework has been proposed, with level one being chatbots, level two being reasoners, level three being agents with the ability to perform longer-term tasks, and level four being innovators, with level five being the ability to do so at the scale of a whole company or organization (42m23s).
• The progression to level five may involve multiple agents that self-correct and work together, similar to an organization, and may lead to the creation of companies with billions of dollars in revenue and fewer than 100 employees (43m40s).
• The potential for startups to make a significant impact with limited resources, such as one person and 10,000 GPUs, is vast, making it a great time to be a startup founder (44m11s).

Outro: Advice for early founders + final thoughts (44m16s)

• Advice for early founders is to bet on the current tech trend, as the models are going to get significantly better quickly, and this will greatly differ from what could be done without it (44m17s).
• Startups have an advantage with speed, focus, and conviction, allowing them to react quickly to the fast-moving technology, which is their number one edge (44m50s).
• It is recommended to build something with AI and take advantage of the ability to see a new thing and build something that day, rather than putting it into a quarterly planning cycle (45m3s).
• When working with new technology platforms, it's easy to think that the laws of business don't apply, but it's essential to remember that building a moat, a competitive edge, or a better product is still necessary (45m27s).
• Embracing new technology quickly can lead to short-term explosions of growth, but it's crucial to remember to build something of value and not fall for the idea that the technology alone is enough (45m44s).
• Building a business is still the ultimate goal, and while technology can make it faster and better, the rules still apply (46m4s).
• There is excitement about the potential for Artificial General Intelligence (AGI) in 2025, but also a personal excitement for having a kid, which is expected to be a life-changing experience (46m13s).
• The goal is to build a better world for future generations, and the current technological advancements make it a fun and hopeful time (46m34s).