TikTok Ban, Data Centers in Space… and What's a Vector? | E2073

Alex kicks off the show. (0s)

• Welcome to the show, which is being hosted by Alex on a Friday, and is a special episode featuring the founders of two interesting private market companies in the world (0s).
• The show is sponsored by Lemon, which offers 15% off the first four weeks of developer time for hiring pre-vetted remote developers (19s).
• Northwest Registered Agent is also a sponsor, providing services for starting a business, including forming an LLC, trademarks, domains, and custom websites, all of which can be completed in 10 clicks and 10 minutes (25s).
• Another sponsor is Vanta, which offers compliance and security solutions for startups, making it easier for companies to get a SOC 2 report, and is offering $1,000 off for a limited time to twist listeners (43s).
• Alex has a couple of news items to share before the main discussion, to keep viewers up-to-date on current events going into the weekend (10s).

Supreme Court ruling on TikTok and key market numbers (58s)

• The Supreme Court of the United States has upheld a law that would force a divesture or ban of TikTok in the US, with the decision being made on January 19th (1m1s).
• The ban is expected to cause chaos, especially with the inauguration taking place the following week (1m23s).
• Bench, a former accounting startup, had $65.4 million in liabilities when it failed, with much of the money owed to the National Bank of Canada, as well as employees, investors, and executives (1m33s).
• Crypto wallet Phantom has raised $150 million at a $3 billion valuation, with the company claiming 15 million monthly active users and more active traders and trading revenue than wallets from MetaMask and Coinbase Wallet combined (2m19s).
• The recent funding round for Phantom is attributed to a shift in sentiment towards crypto, with people becoming more interested in crypto on-ramps, exchanges, and wallets after the recent election and Bitcoin reaching $100,000 per token (2m46s).
• Insight Partners has raised $12.5 billion for new funds, including a flagship fund and a dedicated buyout co-invest fund, which will likely be used for a multi-strategy approach (3m12s).
• The $12.5 billion raised by Insight Partners is notable in the context of venture capital concentration, where the number of funds raising money is decreasing, and larger funds are performing better (3m33s).

Interview introduction with Weaviate and Lumen Orbit CEOs (3m49s)

• A billion-dollar raise from Insight Partners has been mentioned, which seems fitting with the current news cycle (3m50s).
• Two interviews will be conducted: one with the CEO and co-founder of Weaviate, and the other with the co-founder and CEO of Lumen Orbit (3m59s).
• The reason for choosing these two companies, Weaviate and Lumen Orbit, will be explained (4m5s).

Deep dive into vector databases with Weaviate's Bob van Luijt. (4m6s)

• Weaviate is a startup that has been added to the Twist 500 list of the most important private market companies in the world, and it's a company that highlights innovation in the market today, particularly in the field of AI (4m8s).
• To understand how many AI apps are built today, it's essential to understand vectors and vector databases, which are a critical part of the modern AI stack (4m42s).
• Vectors are a mathematical representation of data that provides both magnitude and direction, essentially showing how far away and in what direction (6m28s).
• Vector embeddings are the process of assigning vector values to words and or sentences, taking data and assigning those vectors to individual pieces of unstructured or structured data (6m40s).
• The reason vector embeddings are interesting is that they allow us to make sense of unstructured data, such as language, images, or audio, by organizing them in space and assigning vector embeddings (6m58s).
• By doing so, we can work with unstructured data using distance calculations, making it valuable for various applications (7m23s).
• Vector embeddings enable us to see the proximity of different data points, such as concepts like wolf, dog, and cat, and understand their relationships based on their distance (7m43s).
• The concept of vector embeddings is not new, but it has become more relevant with the advancement of machine learning, which allows us to train and work with these vectors (8m9s).
• Weaviate's CEO and co-founder, Bob van Luijt, is working on vector databases, which are essential for building AI apps, and his company has become a household name in the world of technology (4m31s).
• Bob van Luijt explains that vector databases are critical for working with unstructured data, and his company is innovating in this space with an open-source model (5m8s).

Lemon. TWiST listeners get 15% off your first 4 weeks of developer time (8m23s)

• Finding great developers can be challenging, especially for startups trying to scale and raise money, leading to slow product velocity (8m25s).
• Lemon is a platform that offers thousands of on-demand developers who are experienced, results-oriented, and charge competitive rates (8m41s).
• Lemon handles the process of finding and vetting developers, making it easier for startups to integrate them into their teams (8m59s).
• Startups choose Lemon because they only offer handpicked developers with at least 3 years of experience, and only 1% of candidates are accepted (9m2s).
• If something goes wrong, Lemon will find a replacement developer ASAP, and many Launch Founders have had great experiences with the platform (9m12s).
• TWiST listeners can visit Lemon.TWiST and find their perfect developer or tech team in just 48 hours or less, with a 15% discount on the first 4 weeks (9m23s).
• Researchers have developed a way to measure distances between words in sentences, allowing them to create embeddings that capture the relationships between words (9m57s).
• This concept can be applied to images by analyzing the colors of pixels, allowing researchers to say something about the distance between images (10m30s).
• The idea of vector embeddings is to capture the distances between data points, regardless of the modality, and apply machine learning to index these vectors (10m48s).
• Vector indexing involves applying vector embeddings to a large dataset, allowing machines to understand the relationships between data points (11m5s).
• The process of assigning vector numbers to discrete data points using machine learning models can be complex and may seem like magic to some (11m21s).

Exploring deep learning, vector indexing, and Weaviate's open-source significance (11m35s)

• The concept of vector indexing involves pre-calculated distances between different vectors, but this approach becomes difficult to implement due to linear scaling complexity, making it challenging to calculate distances between every word on the web (13m42s).
• The idea of training a model to predict distances between words emerged as a solution, allowing for faster processing and making it possible to work with large datasets (13m6s).
• This approach, known as cooccurrence, involves training a model to predict the distance between words based on their frequency of appearance together in sentences, with the model improving its predictions as it is trained further (14m23s).
• The use of deep learning enables the prediction of distances between words, making it more efficient than brute force calculation and allowing for faster processing of large datasets (13m54s).
• The concept of vector indexing has its roots in academic research and philosophy, dating back almost 100 years, but it wasn't until the development of deep learning that it became possible to implement this approach effectively (12m39s).
• The GloVe model, developed at Stanford, was an early example of this approach, allowing for the training of a model on large datasets such as Wikipedia and enabling faster processing of word distances (13m22s).
• The prediction element in deep learning allows for the estimation of distances between words, making it possible to work with large datasets and enabling applications such as natural language processing (13m47s).
• Vectors are numerical representations of distance, magnitude, and direction, and they can be used to store and manage large amounts of data, including embeddings, which are numerical values associated with individual bits of data (15m16s).
• Vector indexing is the process of figuring out how far apart different vectors are, and this information is typically stored in a vector database (15m23s).
• The development of vector databases was driven by the need to store and manage large amounts of vector data, particularly in the context of machine learning and AI (15m30s).
• The company ev8 has developed an open-source vector database to meet this need, and this database is designed to handle the unique requirements of vector data (15m33s).
• The use of vector embeddings has become increasingly important in recent years, particularly with the rise of machine learning and AI, and this has created a need for specialized databases that can handle this type of data (16m11s).
• The development of vector databases is part of a larger trend in the database industry, where new data types and use cases drive the creation of new database companies and technologies (16m58s).
• The rise of AI and machine learning has created a large market for vector databases, and companies like ev8 are well-positioned to take advantage of this trend (17m25s).
• The concept of product-market fit is relevant to the development of vector databases, as it describes the situation where a product or technology meets a specific need in the market, and this is what happened with the rise of AI and machine learning (17m43s).
• The quote from Mark Andriessen, "if the market puts two fingers off your nose and pulls them toward you, that's what happened" with AI and vector databases, illustrates the idea of product-market fit (17m54s).

Northwest Registered Agent. For just $39 plus state fees, Northwest will handle your complete business identity. Visit ⁠ today. (18m3s)

• To build a successful business, two essential elements are required: a killer idea and a properly set up company, with Northwest Registered Agent offering an affordable option to form a business for just $39 plus state fees (18m5s).
• The process of forming a business with Northwest Registered Agent is simple, straightforward, and hassle-free, with no hidden fees or upsells, allowing entrepreneurs to focus on building their business (18m17s).
• The business formation process with Northwest Registered Agent can be completed in 10 clicks and 10 minutes, with the company handling all the necessary paperwork quickly and accurately (18m31s).
• Thousands of entrepreneurs trust Northwest Registered Agent due to their affordable, efficient, and stress-free business formation services, with an expert team available for support (18m47s).
• Northwest Registered Agent offers a convenient and valuable service, providing entrepreneurs with peace of mind and allowing them to pursue their business goals without being held back by paperwork (18m52s).
• The concept of product-market fit is mentioned, where customers are eager to purchase a product, and the market is coming to the business, driving demand for more (19m22s).

Discussion on vector databases use cases and retrieval augmented generation (RAG) (19m32s)

• Vector databases unlock several things that make AI applications useful, including retrieval augmented generation (RAG), which allows users to have their own data interplay with a large language model (LLM) without needing to retrain the original model (19m35s).
• RAG enables users to take an LLM and have their own data interact with it, allowing for more personalized and accurate results (19m46s).
• One of the challenges with storing data in a database with factor embeddings is that the more words in a paragraph, the more it centers in the center of vector space, losing its meaning (20m28s).
• The Transformer paper solved this problem by introducing attentional mechanisms that allow the model to consider the context of the words in a paragraph, rather than just their individual meanings (20m56s).
• The Transformer model plays a game of "telephone" with the words in a paragraph, predicting the next token in the string based on the context of the previous words (21m6s).
• Generative AI works by translating individual words into tokens, which are then used to request vector embeddings from a model, allowing for similarity searches and predictions (21m32s).
• The model returns a vector embedding, which is then used to determine the next token in the string, allowing for generative AI applications (21m54s).
• Open AI popularized this technology by creating a chat interface that showcased its capabilities, leading to widespread interest in using vector databases for AI applications (22m27s).
• Vector databases can be used to help people take their structured and unstructured data and get it into a vector database, allowing it to be used in AI contexts (22m59s).
• The original research paper on retrieval-augmented generation proposed making large models smaller by augmenting generated information with retrieved data, using a vector database to retrieve unstructured data and pipe it to the model for output generation (23m50s).
• Two interesting developments are happening: intertwining models and databases, and piping output back into the database, creating an agent (24m24s).
• Vector databases are used for similarity search, hybrid search, and enabling retrieval-augmented generation (RAG) (24m53s).
• Another perspective on vector databases is that they provide a technological innovation and a developer experience, allowing developers to build applications with functionalities like hybrid search and offloading in just a few lines of code (25m33s).
• The developer experience is improved by the ability to build applications quickly using a vector database, a large language model, and a few lines of code, making it easier for developers to create applications and reducing the barrier to entry (25m51s).
• The aperture for what can be built is getting wider as the barrier to entry for developers decreases, allowing for more applications to be created (26m30s).
• The combination of a vector database, a large language model, and a developer's data and API key can create an application, making the developer look like a genius and potentially leading to career advancement (26m18s).
• The functionality of a system is important, but it is also crucial to help developers build applications, especially for those who are new to the field and may not have extensive experience (26m42s).
• There are many genius developers who know how to create complex applications, but there are also people who are just starting out and need help getting started with building applications, such as AR applications (26m51s).
• Efforts are being made to help all developers, not just experienced ones, build applications with ease, including providing simple ways to get started with just a few lines of code (27m1s).
• New AI infrastructure companies are working to support all developers in building applications, not just a select few (27m9s).

Weaviate's business model, revenue streams, and growth (27m17s)

• Weaviate's business model is based on an open-source piece of software with services attached, which comes in two main varieties: a serverless managed instance and Enterprise Partnerships with major cloud providers (27m26s).
• The Enterprise Partnerships are with the big three cloud providers: AWS, GCP, and Azure, allowing customers to spin up Weaviate Vector databases on their existing cloud infrastructure (27m39s).
• Another element of the business model is BYOC, which stands for "bring your own cloud", offering a wide variety of deployment options due to the open-source model (28m0s).
• The open-source model allows for flexibility in deployment options, which is beneficial for nailing product-market fit (28m8s).

Vanta. TWiST listeners automate your SOC2 and get $1,000 off (28m12s)

• Vanta is an all-in-one compliance solution that helps startups get audit-ready and build a strong security foundation quickly and painlessly (28m41s).
• Vanta automates manual security tasks, streamlines audits, and connects users with trusted VCs, auditors, and a marketplace for essentials like pen testing (28m53s).
• Over 8,000 companies, including Y Combinator, Techstars, and Launch startups, trust Vanta to simplify compliance, and new users can get $1,000 off at vanta.com/twist (29m14s).
• When it comes to revenue mix, traditional infrastructure companies and enterprises pay the most, while startups are more quantitative with smaller bills (29m39s).
• Vanta's growth is driven by both smaller developers and larger corporate customers, with enterprises contributing significantly to revenue growth (29m37s).
• The company started monetizing halfway through 2023 with its Surus offering and quickly reached its first million dollars in revenue (30m31s).
• The first enterprise request came in, but the company initially lacked enterprise sellers, prompting the building of a whole team to address this need (31m7s).
• The company has seen significant growth, with enterprises going live and applications moving into production, leading to the expansion of its enterprise sales team (31m56s).
• Vanta's story is a classic example of an open-source infrastructure project gaining adoption, scaling up, and eventually attracting enterprise sellers (32m15s).
• The company has crossed 100 people and is proud of its progress, with its growth and adoption being recognized by other companies in the industry (32m47s).

The future of AI, agentic architectures, and enterprise adoption (32m52s)

• The concept of "agentic AI" has been widely discussed, but the term "agent" has multiple meanings, making it unclear what it specifically refers to (32m52s).
• An agent is defined as something that does something with data, rather than just presenting it, and can be given prompts to perform specific tasks (33m24s).
• A simple example of an agent is a generative feedback loop that translates data into a specific language, such as American English (33m34s).
• Agents are considered a real thing, but also a marketing term, as they are used to explain complex concepts to the world and consolidate language (34m2s).
• The development of agents has progressed from Vector search to RAG (Retrieve, Augment, Generate) and now to agentic architectures, which enable new use cases to emerge (34m23s).
• The majority of customers started with Vector search, and now people are building new things with agents, which is interesting because it validates the existence of Vector databases (34m35s).
• Vector databases are essential for agentic AI, and without them, agentic AI would not be possible (35m45s).
• Vector databases make it easier to work with agents and are built specifically for that purpose, which is why people adopt and use them (36m2s).
• The majority of developers prefer to use tooling built for specific use cases, rather than relying on old databases like Oracle, which can do everything but may not be the best fit for their needs (36m27s).
• This preference is often overlooked by hardcore developers who forget that not everyone has the same level of expertise and may need help building great software for their business or company (36m58s).
• The developer experience plays a crucial role in addressing this need and cannot be underestimated (37m18s).
• The AI industry is expected to undergo a paradigm shift this year, driven by agentic architectures that enable models to have an "opinion" on data and turn "chicken into chicken salad" (37m47s).
• This shift is expected to solve the long-standing problem of Master Data Management, which has been a major issue in data management since the beginning (37m51s).
• The solution to this problem will not only address existing issues but also open the door to new businesses, products, startups, and ideas based on this new paradigm (39m31s).
• The impact of this shift will be significant, as it will enable companies to make sense of their data and answer complex questions, such as how many products were sold globally, which is currently a challenge due to the messiness of the data (39m7s).
• A new development is expected to solve the long-standing problem of bad data and open the door to new products and solutions in various sectors, including enterprises and startups (39m49s).
• This breakthrough presents an enormous new opportunity for people looking for ideas to build on and is considered a great time to start working on new projects (40m9s).
• The current time is likened to the early days of mobile app development, with AI now being at a similar starting point, making it an ideal time to begin building (40m22s).
• The year 2025 is expected to be particularly busy due to these developments, marking the start of a new paradigm (40m35s).
• The conversation also led to a review of notes about what a vector is, a topic that was revisited during the discussion (40m42s).

Lumen Orbit's vision for space-based data centers (40m50s)

• Lumen Orbit is a company that aims to put digital infrastructure, specifically data centers, in space instead of on the ground, leveraging lower launch costs and greater launch capacity (40m59s).
• The need for more compute power to support AI development has been a recurring topic, with AI being a highly compute-intensive technology that requires significant energy (41m20s).
• Data centers currently consume a substantial amount of power, with some US states using 10-15% of their total power consumption for data centers, and one state using as much as 26% (41m48s).
• The energy demands for data centers are expected to increase exponentially over the next few years, requiring new solutions to power the necessary compute (42m5s).
• Potential solutions to meet the increasing energy demands include fusion, building more nuclear reactors, and harnessing solar energy from space (42m23s).
• Lumen Orbit is exploring the idea of using solar panels in space to power data centers, providing a potential alternative to traditional energy sources (42m42s).
• Philip Johnston, co-founder and CEO of Lumen Orbit, is working on making this vision a reality, with the goal of supporting the development of AI and other compute-intensive technologies (42m50s).

Philip Johnston on the technical aspects of Lumen Orbit (42m54s)

• Lumin Orbit is working on building large data centers in space to take advantage of the abundant energy, passive cooling, and scalability available in space (43m36s).
• The idea for this project came from initially looking at space-based solar power, which could become more feasible with low launch costs, and the forecast that half of all terrestrial electricity consumption will go into data processing by the 2020s (43m46s).
• The project aims to use data and energy in space instead of transferring it back to Earth, which would result in a 95% efficiency loss using microwaves (44m10s).
• The data center in space is envisioned as a 4-kilometer per side square block of solar panels, comprising thousands of cells (44m33s).
• The main risk remaining for the project is the need for launch costs to come down significantly, with the potential for a 10x, 100x, or even 1,000x reduction in the next five years (45m3s).
• The success of the project depends on the development of competing heavy launch vehicles, such as Starship, to drive down launch costs (45m43s).
• Passive cooling in space is possible due to the cold temperatures, but it's not easy to get rid of heat in space, and data centers consume a lot of power and produce tons of heat (45m57s).
• Developing a large, low-cost, low-mass deployable radiator is a core part of the technology being developed, which is not an easy task due to the lack of atmosphere in space, requiring a large black body radiator to radiate heat into deep space (46m5s).
• The radiator needs to be kept at around 20 degrees Celsius higher than the surrounding temperature to radiate a significant amount of heat, approximately 800 watts per square meter (46m26s).
• To put this into perspective, a typical household light is around 20 watts, and one square meter of solar panel in space generates around 200 watts, while one square meter of radiator dissipates around 800 watts (46m50s).
• As a result, a 4 km by 4 km square of solar panels would be needed to power a 5-gigawatt data center, and a 1 km by 1 km square of radiator would be required to dissipate the heat (47m11s).
• There is a risk of solar panels being damaged by space debris and micro-asteroids, but this can be mitigated by flying in very low or very high orbits (47m37s).
• Flying in very low orbits, below 400 km, is relatively clean and free of debris, but requires more propulsion energy, while flying in very high orbits, around 1200 km, is also clean but requires more shielding from radiation (47m54s).
• As the satellite gets larger, the amount of shielding needed as a percentage of the mass of the satellite decreases, making it feasible to fly in higher orbits (48m52s).
• For smaller satellites, flying in very low orbits is preferred, but as the technology scales up, flying in higher orbits becomes a more viable option (49m7s).

Satellite demonstrators, technology challenges, and VC interest (49m12s)

• The first satellite demonstrator is expected to launch as early as May this year, featuring a 1-kilowatt, 50-kg satellite with a state-of-the-art terrestrial Nvidia chip, about 100 times more powerful than the typical radiation-hardened chips used in space (49m13s).
• The satellite will have three ports of connectivity, including a terminal to connect to the Iridium network, an antenna to connect to customer satellites, and an antenna to connect to ground stations (50m21s).
• The second satellite, Li 2, is scheduled to launch in mid-2026 and will have an optical terminal, possibly two, allowing for connections to customer satellites and directly into the Starlink network (50m43s).
• The optical terminal uses laser technology, enabling satellite customers to connect directly into Starlink (51m5s).
• There is a growing demand for computing in orbit, particularly from military satellites and Earth observation constellations, which will be the initial customers for the satellite-based data center (51m44s).
• As launch costs decrease over the next five years, the service is expected to transition into a commercial offering that can move most data centers to space from Earth (52m4s).
• The venture thesis behind this project is based on the intersection of three trends: huge demand for energy, huge demand for compute, and launch costs decreasing by 100x (52m43s).
• The total addressable market (TAM) for this project is estimated to be around $101 trillion, making it an attractive opportunity for investors (53m8s).

Addressing chip obsolescence and cost advantages in space (53m26s)

• Chips in space have a longer lifetime compared to those on Earth, with a four-year life expectancy, but they can be run for five or six years in space due to the zero marginal electricity cost, making it more economical than on Earth (53m58s).
• The cost of electricity is a significant factor in the overall cost footprint of a data center, and launching a data center into space can save a lot of money in the long run (54m53s).
• Running a 40-megawatt data center on Earth for four years would cost around $140 million in electricity costs, whereas launching it into space would cost around $10 million for the launch and $5 million for solar panels, with the cost of chips, radiators, and cooling systems being the same (55m28s).
• The real advantage of space-based data centers is the ability to scale up quickly, as launching multiple modules can provide a significant increase in power capacity in a short amount of time, unlike on Earth where building a large energy project can take decades (55m57s).
• Nvidia's upcoming Blackwell line of chips is expected to be significantly better than the current H100 chips, which could make older chips obsolete, but the longer lifetime of chips in space can mitigate this issue (53m29s).
• The economics of space-based data centers are favorable due to the zero marginal electricity cost, making it possible to run chips for longer periods and achieve significant cost savings over time (54m33s).

Deployment strategies and competition in the space data center market (56m21s)

• Modular data center designs are being explored, where instead of having all compute resources in one spot, they would be distributed along a spine, allowing for easier attachment of modules with solar panels and radiators on each side (56m33s).
• The concept of modular data centers can be scaled up by connecting multiple units, potentially creating a large 16x16 grid (56m31s).
• Deploying data centers in space involves unpacking and assembling the modules, a process that is relatively solved, with experience from deploying large solar panels and radiators on satellites and the International Space Station (57m25s).
• The deployment of large solar panels and radiators in space is a solved problem, with experience from NASA's Luna path founder Mission, which involved deploying very large solar panels and radiators (57m27s).
• Future data centers in space will use roll-out solar panels, which are thin, flexible, and can be rolled out to cover large areas (57m52s).
• Robotics in space and space construction are advancing, with humanoid robots potentially managing data centers in space within five years (58m6s).
• Humanoid robots are better suited for space construction than humans, as they are more resilient and require less maintenance (58m26s).
• The development of space data centers is dependent on the success of launch systems like Starship and New Glen, which need to fly frequently to enable the industrialization of low to higher orbits (59m38s).
• Potential stumbling blocks for the development of space data centers include global conflicts, such as a large-scale war with China, which could disrupt the progress of space technology (59m45s).
• The physics behind space data centers and launch systems like Starship has been proven, and the focus is now on scaling up the technology (59m57s).

Strategies for staying ahead of competition and future plans for Lumen Orbit (1h0m12s)

• To stay ahead of competition, Lumen Orbit has a strong team with experts from SpaceX and MIT, making it difficult for others to replicate, and they are also ahead in terms of capital investment, which is necessary for their space-based data center project (1h1m1s).
• The company's strategy involves being the first to launch and establish a strong presence in the market, making it harder for competitors to catch up, and they plan to partner with big hyperscalers like Microsoft, Meta, Google, and Oracle, who do not have their own space arms (1h0m58s).
• Lumen Orbit's project involves launching a solar-powered data center in space, which will be a significant achievement and a major milestone in the industry, and they plan to make a lot of noise about the launch to raise awareness and excitement (1h2m1s).
• The company's goal is to make space-based data centers a reality, and they believe that this will be a game-changer for the industry, with the potential to revolutionize the way data is stored and processed (1h2m10s).
• Lumen Orbit's team is confident that they are ahead of the competition and that their project will be successful, and they are excited to share their progress and achievements with the public (1h1m16s).
• The company's launch is expected to happen later in the year, and they plan to share updates and news about the project as it progresses (1h1m30s).