How GitHub Next took Copilot Workspace from concept to code

Introduction and Overview

• The session is about GitHub Next's journey in taking Copilot Workspace from concept to code, with a focus on the early prototyping and iteration of the project, a demo of Copilot Workspace, and learnings from the technical preview (7s).
• Chris Readington, from the developer relations team, introduces himself and explains that he works with the developer community to understand how they use GitHub and how to improve the developer experience (25s).
• Cole, from the GitHub Next team, introduces himself as the lead designer and front-end engineer who has been working on Copilot Workspace for the last year, and explains that he loves building things for people who build things (50s).
• The session aims to give attendees a behind-the-scenes look at how GitHub Next builds software, dispelling the myth that their process is too different from others, and providing a sneak peek at Copilot Workspace (1m36s).
• The three main takeaways from the session are: how GitHub Next built Copilot Workspace, a demo of Copilot Workspace with tips and tricks, and learnings from the technical preview that made it successful (1m51s).
• The session will also cover how GitHub Next has been experimenting with AI technologies and has built up a wealth of experience over the years, being at the forefront of this field (3m38s).
• The team that built GitHub Copilot and prototyped it initially is GitHub Next, and they have been continuing to innovate in the AI space, with a focus on making the experience better for day-to-day developers and businesses (3m20s).
• The session will be interactive, with the presenters encouraging audience participation and feedback throughout (2m38s).
• The context for the session is set by explaining that a year in AI is a long time, and that GitHub has been continuing to innovate in the AI space since the release of the technical preview of GitHub Copilot three years ago (2m45s).
• GitHub Next is a small team of about 20 researchers and engineers who investigate the future of software development by building prototypes, doing internal testing, and shipping technical previews to learn from users (3m53s).
• The team's goal is to reduce the complexity of building software using AI, making it easier for developers to go from task to code (4m40s).
• To achieve this, the team follows guiding principles for delightful developer experiences, including reducing activation energy, making forward progress obvious, allowing developers to steer, and making AI output easy to verify (4m59s).

Guiding Principles for Developer Experience

• To achieve this, the team follows guiding principles for delightful developer experiences, including reducing activation energy, making forward progress obvious, allowing developers to steer, and making AI output easy to verify (4m59s).
• Reducing activation energy means making getting started as easy as possible by reducing the complexity of the task (5m14s).
• Making forward progress obvious involves creating intuitive tools that guide users towards success (5m35s).
• Allowing developers to steer means keeping them in the loop and in control, as AI is a tool for developers, not a replacement (5m57s).
• Making AI output easy to verify involves providing the right information and tooling to ensure confidence in the answers and code generated (6m11s).

Early Prototyping and the Evolution of Copilot Workspace

• During an "exploration season" in the summer of 2023, the GitHub Next team explored various ideas, including a natural language compiler called Speckling (6m54s).
• Speckling allows developers to write entire programs in natural language, but the non-deterministic nature of large language models made for an unstable compiler (7m16s).
• This led to a new idea called Extract, which aims to address the instability of the Speckling compiler (7m59s).
• Extract, Apply or EA is a tool that can extract a natural language specification from an existing codebase, allowing users to edit the spec and then apply those edits back onto the code using AI (8m2s).
• The initial UI for EA was created by Don Sim, one of the researchers, and served as the seed for what would eventually grow into Copilot Workspace (8m44s).
• The initial EA prototype evolved significantly over time, with changes including a new name, updated design language to match the rest of GitHub, and the integration of a terminal and a VS Code extension (9m22s).
• The team conducted user studies to learn how people use Copilot Workspace and how to make it better, and also used the tool to build itself (9m57s).
• The biggest evolution from EA to Copilot Workspace was revisiting the idea of task-to-code, recognizing that developers typically need to plan and iterate when working on a task (10m13s).
• The task-to-code workflow in Copilot Workspace involves planning, using knowledge of the existing codebase, understanding the desired state, and iterating to get to a solution (10m57s).
• The evolution of the workflow can be seen from spec laying to extract, edit, apply, and eventually to the task-to-code workflow currently in Copilot Workspace (11m5s).

Demo of Copilot Workspace: Task-to-Code Workflow

• A demo of the task-to-code workflow in action was given using a project called Mona Space, which involves adding a new font to a set of fonts that can be used in a terminal (11m12s).
• The GitHub Next team is working on a real production site, making changes to it, and using a real repository that the team also uses, with a GitHub issue created to add a Rust example to the font playground, clearly stating the acceptance criteria and intent (12m17s).
• The team uses the "Open in Workspace" button to get into the flow of working with Workspace, where they can see the issue on the left and a brainstorming tab on the right, with different modes, including a spec and a new brainstorming mode (12m47s).
• The spec mode involves stating the current state of the repository, what the code currently does, and the desired state, while the brainstorming mode allows for a more exploratory approach to figure out the goal (13m9s).
• The team evaluates the current state of the repository, determines the end goal, and steps to the next stage of the process, which is the plan, where Co-Pilot Workspace generates a plan of action (13m41s).
• The plan is like a recipe, outlining the steps to implement the change, such as opening the code sample file, adding a Rust code sample, and installing the package, making it easy to verify and understand if the AI has done what was intended (14m25s).
• The team can implement the selected files and run commands within the plan section, such as executing all commands to run the necessary actions, like installing a new package (15m27s).
• The team has successfully added a Rust sample to the code sample file and imported the necessary package, with Co-Pilot Workspace generating the necessary code and allowing the team to execute the commands to complete the task (15m18s).
• Behind the scenes of GitHub Next, creating a code space is happening, which includes compute under the covers that powers things that can be done in the terminal, such as running an app environment, building, testing, and running, etc. (15m40s)
• The code space allows users to install packages, run commands, and open a terminal just like in a local environment, providing an on-demand developer environment that can be spun up with everything configured. (15m43s)
• There is a nice integration between Workspace and CodeSpace, allowing users to easily access the app environment by clicking a button and getting forwarded to the port. (16m50s)
• The code in the environment syncs across to the live preview, enabling a quick iterative loop of trying changes, testing them, and checking them, similar to the inner loop of development locally. (17m43s)
• Users can spin up a CodeSpace to use extensions, step-through debugging, and other advanced tools that may not be available in the user interface, and changes made in the CodeSpace will be in sync with the app preview and CoPilot Workspace. (18m10s)
• GitHub issues are a great entry point into CoPilot Workspace, often containing helpful details like acceptance criteria and links to documentation, but they are not the only entry point, as users can also open pull requests and ad hoc tasks without creating an issue. (18m50s)

Entry Points and Contextualization

• GitHub issues are a great entry point into CoPilot Workspace, often containing helpful details like acceptance criteria and links to documentation, but they are not the only entry point, as users can also open pull requests and ad hoc tasks without creating an issue. (18m50s)
• GitHub's Copilot Workspace allows users to enter task descriptions directly in the interface, open template repositories, and bootstrap new ones using natural language, with entry points deeply integrated into GitHub.com (19m25s).
• Users can also open Copilot Workspace from anywhere using query parameters, specifying the repository and task description in the query, and even use a Raycast extension to start a new task with a few keystrokes (19m46s).
• The more context provided to Copilot Workspace, the better the results, and users can share their side projects and explore integration possibilities (20m28s).
• Good issue descriptions with context, such as acceptance criteria, file and package names, and well-scoped projects, help Copilot Workspace pick the most relevant files and generate better code (20m47s).
• Copilot Workspace can crawl URLs and use them as context when generating code, and users can see the list of referenced files and URLs (21m11s).
• The brainstorming feature is an evolution of the spec, filling in details left out of the task description by understanding the current behavior of the repository and summarizing the desired state (21m38s).
• Brainstorming can suggest questions to get users started, and users can ask their own questions and add answers as context to the task (22m5s).

Features and Workflow within Copilot Workspace

• The planning feature is like a recipe, telling users which files to change, delete, or run terminal commands, and users can edit any line of the plan (22m44s).
• Once users have a plan, they can move on to writing code, making manual edits if desired, and accessing the code environment (23m8s).
• GitHub Code Spaces provides a full toolbox for building, testing, and running code, and allows users to focus on specific files with the Target button and Co-Pilot Workspace (23m38s).
• Co-Pilot Workspace enables users to work with files using natural language or code, and provides features like auto-completion and code suggestions (24m21s).
• The Co-Pilot extension can be used within Co-Pilot Workspace, allowing users to leverage the power of code spaces behind the scenes (24m26s).
• Users can edit existing code and use natural language to run commands, such as linting or formatting files, without needing to know the specific commands (25m7s).
• Co-Pilot Workspace provides a build, test, and run workflow, with suggested stages based on the repository context, to streamline the development process (25m40s).
• The platform allows users to iterate on their code and adjust their plan as needed, with features like steerability and the ability to switch between different states (26m24s).
• Co-Pilot Workspace enables users to brainstorm, plan, and code in a flexible and iterative way, using natural language to make changes and adjustments (26m37s).
• GitHub's Copilot Workspace allows users to experiment with ideas and iterate on them using natural language, with features like undo and redo buttons, enabling users to bounce between different ideas and refine their work (27m9s).
• The platform also allows users to create a pull request and generate a pull request description, linking it back to the issue and adding a sharing URL (27m50s).

Tips and Tricks for Using Copilot Workspace

• Tips and tricks for using Copilot Workspace include decomposing tasks into smaller ones, being specific and providing context, iteratively reviewing and refining work, and using the platform's natural language revision features (28m28s).
• Additionally, users can use the platform's brainstorming feature to help define tasks and ask questions, and can build, test, and run code entirely within Copilot Workspace (28m55s).

Learning from the Technical Preview and Community Feedback

• The GitHub Next team values user feedback and has made changes to Copilot Workspace based on user feedback, including simplifying the feedback loop and creating a thought partner (29m48s).
• To simplify the feedback loop, the team elevated natural language and made it easy to see the diff with the last iteration, allowing users to choose between viewing the latest changes, uncommitted changes, or all changes (30m11s).
• To make Copilot Workspace a better thought partner, the team built a brainstorming feature that suggests questions and ideas and allows users to ask their own questions (30m42s).
• The GitHub Next team has made significant changes to the technical preview of Copilot Workspace, which can be found in the change log, and JC writes the change logs every week, providing detailed information on the updates made. (30m54s)
• The team has learned valuable insights from running the technical preview, including the importance of connecting directly with the community through a central Discord Channel where users can share feedback and ask questions. (31m30s)
• The team responds to questions and addresses feedback in the Discord Channel, which has had a direct impact on the direction of Copilot Workspace. (31m46s)
• The team encourages community members to write blog posts sharing their experiences with Copilot Workspace. (31m59s)
• Clearly communicating changes through a weekly change log is crucial in generating excitement and building momentum around the project, while also ensuring that the team delivers actual user impact every week. (32m9s)
• "Dog fooding" the app, or using the product being built, is essential in understanding the user experience and catching bugs before users do, with JC being a huge advocate for this practice. (32m31s)

Key Takeaways and Future Directions

• The key takeaways from the project include the importance of design principles, such as reducing activation energy and making AI output easy to verify, which have remained consistent throughout the development of Copilot Workspace. (33m7s)
• Copilot Workspace is an AI-native development environment designed to help with everyday tasks, focusing on solving problems rather than writing lines of code. (33m45s)
• The tool allows users to collaborate with the AI to solve tasks, such as fixing accessibility issues, by feeding in prompts and working through a flow of starting with a task or issue. (34m21s)
• GitHub Next took Copilot Workspace from concept to code by having a clear plan and recipe to follow, allowing for manual iteration and iteration using natural language, and ultimately raising a pull request (34m32s).
• The community has been at the heart of everything GitHub Next has been doing with Copilot Workspace, with the team interacting daily with the community and solving friction points as they arise (34m46s).
• The team encourages users to get involved and provide feedback, whether positive or negative, to help improve the tool and plan future sessions (35m8s).
• Using one's own tools to build and test is crucial in avoiding edge cases that might come up for users, as demonstrated by JC, the Chief Dog Fooding Officer (35m9s).
• Thinking from the user's perspective is essential, even down to the detail of a change log, to ensure that the tool is user-friendly and meets their needs (35m24s).
• The team has shipped Vision capabilities in Copilot Workspace, allowing users to create architecture diagrams and user interfaces with the tool's assistance (35m31s).
• The session has only touched the tip of the iceberg, and users can explore further by signing up for the waitlist at G.I cw-signup or joining the Discord Channel (36m3s).
• The team invites users to provide feedback and share their experiences with Copilot Workspace, and encourages them to scan the QR code for tangible examples and tips (36m37s).
• The feedback will help plan future sessions and conferences, and the team thanks users for their participation and looks forward to seeing them at the GitHub Next booth (37m0s).