Research Forum Keynote: Research in the Era of AI

AI Disruption and Research Infrastructure

• The speaker compares the current disruption in computer science, particularly in AI, to the disruption in biology in the 1700s when the discovery of cell division challenged the prevailing theory of cell crystallization.
• The speaker highlights the transformative impact of large language models like GPT-4 and the need for new research infrastructure, such as large datasets and GPU compute, to support AI research.

Responsible AI

• Microsoft has evolved from having a single organization for responsible AI to deeply integrating responsible AI and AI societal impact thinking into every engineering group across the company.
• The speaker emphasizes the importance of responsible AI and the need to understand the potential harms and risks of AI technologies, as well as their broader opportunities and societal impact.

Open-Source and Small Language Model Development

• Microsoft Research AI has made significant progress in open-source and small language model development, represented by the FI models, which focus on specific reasoning and problem-solving strategies.
• AutoGen, an open-source platform, enables multiple AI systems to collaborate as independent agents, enhancing problem-solving capabilities.

Training Models for Specific Domains

• Research on training models for specific domains, such as Orca and Orca 2, has shed light on the nature of training data.
• The interplay between specialization and generalization in AI models is being explored, with questions about the necessity of intense specialization in domains like healthcare.
• PromptBase and MedPrompt demonstrate the effectiveness of suitably prompted general-purpose language models in outperforming specialized models in certain tasks.
• Specialization may lead to the loss of some cognitive function, as shown in a study where specialized training caused a large language model to forget information about Harry Potter.

Generative AI in Various Research Areas

• Generative AI has had a broad impact across various research areas within Microsoft, including program analysis, verification, and socio-technical systems.
• Microsoft Research has established a new lab called AI for Science, focusing on applying large language models to scientific domains such as materials discovery, climate science, and drug discovery.
• Collaborations with Pacific Northwest National Laboratories and the Global Health Drug Discovery Initiative have yielded promising results in discovering new electrolyte substances and potential drug treatments.

Future of AI and Collaboration

• The future of AI holds great promise in extending beyond language and medical images to 3D structure learning and physical system predictions.
• The speaker emphasizes the importance of collaboration, openness, and a clear-eyed approach to mitigating potential risks associated with emerging AI technologies.