Open Source Spotlight: Building Copilot Extensions with Nick Taylor

Introduction and Guest Speaker

• Open Source Friday is a show that highlights amazing projects on GitHub and their creators, hosted by Andrea Griffiths, also known as Alak, Columia Dev.
• The show welcomes viewers from around the world and introduces the guest, Nick Taylor, who will be discussing GitHub Copilot extensions.
• Nick Taylor is a passionate advocate for open source, TypeScript, and serverless computing, and has created a Copilot extension template using Node.js and the wholeo framework.
• The template provides everything needed to create a GitHub Copilot extension, and Nick will guide viewers through the process of creating their own extension.
• Nick previously did a stream with Gregor and Francis from GitHub, discussing the SDKs, and the link to that stream will be shared during the transmission.
• Nick Taylor is from Canada and has had a long and amazing career, with a notable talk at Netlify Compost, which will be linked to during the show.
• The show will cover setting up the developer environment, creating a GitHub extension, and testing the extension live.

GitHub Copilot and Extensions Overview

• The discussion is about GitHub Copilot extensions, with the guest having been part of the technical preview and having access to Copilot Chat early on, which led to building a proof of concept for a GitHub Copilot extension for OpenSauce, their previous workplace (6m22s).
• To get early access to GitHub features, one can join the GitHub Next Discord and ask for access, or sign up on GitHub.com for previews (6m2s).
• GitHub Copilot is an AI-powered tool that assists users in their editor, available in Visual Studio Code, Visual Studio, and on GitHub.com, and can be used in various environments, including GitHub code spaces (7m43s).
• Copilot Chat is a feature that allows users to interact with an AI chat tool, similar to ChatGPT, and is available in all products that support Copilot (8m27s).
• The guest built a proof of concept for a GitHub Copilot extension for OpenSauce, which will be shown later in the discussion (6m41s).
• The guest's interest in Copilot extensions was piqued after getting early access, leading them to invest time in building their own extension (6m58s).
• Copilot is a feature baked into various IDEs, including Visual Studio Code, Visual Studio, GitHub.com, and GitHub Codespaces, allowing users to ask questions about their codebase or project, and it can be extended with Copilot extensions to enhance its capabilities (8m36s).
• Copilot extensions enable users to tap into any web-based services, integrating APIs and other tools to provide more functionality, such as domain-specific knowledge or project insights (9m18s).
• Companies like Sentry have developed Copilot chat extensions, allowing users to create new issues or access other services directly from their IDE (9m41s).
• The Extension Marketplace offers a wide range of Copilot extensions, including those from Docker, Perplexity, Stack Overflow, and Launch Darkly, among others (10m12s).
• These extensions can augment users' knowledge base and provide access to tools they already use, eliminating the need to leave their IDE (10m32s).

OpenSauce Copilot Extension

• An open-source extension was created for Open Sauce, which allows users to ask questions about their projects, including non-code specific queries, and provides insights similar to GitHub Insights Plus+ (10m42s).
• The Open Sauce extension can answer questions like "What happens if a key contributor leaves the project?" (referred to as the "Lottery Factor") or "Who has worked on Rust recently?" (11m47s).
• The extension aims to provide more comprehensive project insights, going beyond what Copilot itself can offer (11m44s).

Benefits and Use Cases of Copilot Extensions

• Co-pilot is good at code refactoring and answering specific questions, but it may not be able to answer more general or abstract questions, such as explaining a binary search tree (12m38s).
• The purpose of creating a Co-pilot extension is to allow users to ask questions about their repositories directly in the chat, without having to leave their IDE (12m46s).
• When using a Co-pilot extension, users have access to all the answers and prompts from Co-pilot chat, including those from other extensions, providing more context and information (13m6s).
• This access to context allows users to ask follow-up questions and get more detailed responses, making the extension more useful and powerful (13m29s).
• The extension created for Open Sauce was designed to help users access the knowledge base of the project, providing more in-depth details about the project beyond the code (14m8s).
• The extension was a proof of concept and did not go live, but it demonstrated the potential of Co-pilot extensions to augment user capabilities and provide more information (15m24s).

Best Practices and Considerations for Using AI Tools

• The use of Co-pilot chat and AI tools can be beneficial for users, including senior developers, as it provides a tool for rubber ducking and exploring ideas, but it should not be taken as truth without questioning (16m6s).
• Embracing these tools can help users to work more efficiently and effectively, and can provide a new way of interacting with code and projects (15m53s).
• Rubber ducking is a concept where developers talk to an inanimate object, like a rubber duck, to get their thoughts flowing and communicate better, and this concept can be applied to using AI tools like Co-Pilot Chat, which helps to get thoughts flowing even if it doesn't always provide the right answers (16m20s).
• When using AI tools, it's essential to be as specific as possible, as they can try to build something but may not always understand the exact requirements (17m0s).
• New developers should be cautious when using AI tools and not assume that the output is 100% correct, as AI is still non-deterministic and can make mistakes (17m33s).
• It's crucial to question the output of AI tools and not simply copy-paste the code, but rather use it as a starting point and try it out (17m58s).
• The process of using AI tools is similar to pair programming, where ideas are bounced off each other, and not everything suggested will be used in the final code (18m11s).
• Senior developers can use AI tools to their advantage by augmenting the way they share knowledge with their teams (18m27s).
• Co-Pilot uses AI to check for mistakes, and users can ask Co-Pilot questions, but it may not always be able to answer (18m54s).

Using and Setting Up a Copilot Extension

• To use a Co-Pilot extension, users must start their query with "at" and choose the extension, as otherwise, Co-Pilot will be used instead (19m34s).
• The Co-Pilot Preview SDK provides examples of extensions, such as the Blackbeard Agent, which can be used to answer specific questions (19m51s).
• A live demo of the Open Sauce proof of concept Co-Pilot extension template is shown, which can answer questions about a repository, such as who has worked on security issues recently in Remix Run Remix (20m10s).
• The extension is running off the computer and might be slower than if it was deployed, but it can provide information that Co-Pilot on its own can't answer by leveraging services in Open Sauce (20m59s).
• The extension can be used on several properties, such as GitHub and VS Code, and is available on multiple places with one Co-Pilot extension (22m25s).

Setting Up the Development Environment

• To get started with the first Co-Pilot extension, a GitHub repository template can be used, and a new repository can be created by clicking "Use this template" and following the prompts (23m7s).
• The repository can be cloned using the GitHub CLI, and the project can be opened in VS Code to start development (23m55s).
• The project is now set up for development, and the next steps can be walked through to learn more about building Co-Pilot extensions (24m36s).
• A GitHub Co-Pilot extension project is set up by installing dependencies with npm install, and then changing the port from 3000 to 4000 to avoid conflicts with other projects (24m41s).
• The project is started with npm run dev, which begins the Hono dev server, and it can be accessed at localhost:4000, displaying the welcome message for the Co-Pilot extension template (25m11s).
• To create a GitHub Co-Pilot extension, a GitHub app is required, which needs a public URL, so port forwarding is used to make the local project publicly accessible (25m41s).
• The port forwarding feature in VS Code is used to create a public URL for the project, but by default, the port forwarding is set to private, so it needs to be changed to public (25m52s).
• After making the port forwarding public, the project can be accessed at the provided URL, and it displays the welcome message for the Co-Pilot extension template (26m43s).

Creating a GitHub App for the Extension

• To set up a GitHub app, the GitHub settings need to be accessed, and then the developer settings, where a new GitHub app can be created, and the public URL of the project needs to be provided (27m31s).
• The GitHub app requires a homepage URL, which can be the same as the public URL of the project, and a callback URL, which is currently required but might not be in the future (27m53s).
• The Co-Pilot extension template can be used to create a GitHub Co-Pilot extension, but it's also possible to create a Co-Pilot extension only for VS Code, which is a different approach (27m8s).
• To set up a GitHub app for a Copilot extension, it's necessary to create a new app and give it read-only access to Copilot chat, following the principle of least privilege, which means only giving the necessary permissions (29m9s).
• The app needs to be configured with a private key and the Copilot section in GitHub apps needs to be enabled, setting the skill set to "agent" (30m3s).
• The app's URL needs to be added, which will be used when asking questions to the Copilot extension template (30m18s).
• The setup process is documented in the extension template repository, based on a blog post by the creator, and also ported to the instructions by M Rodriguez (31m0s).
• After setting up the app, the user needs to install it, which is also documented in the repository (31m49s).
• Once installed, the app can be used in Copilot, and the user can ask questions to the extension (32m1s).
• The authorization flow for the app is a one-time process, and the user needs to allow it to access their GitHub account (32m33s).

Understanding AI Agents and Templates

• The GitHub Copilot page is visited to demonstrate the user experience, which is expected to be improved in the future (32m53s).
• An AI agent is defined as a software entity that performs tasks on behalf of users by autonomously making decisions and taking actions to achieve specific goals (33m20s).
• A template is used to quickly set up a GitHub Copilot extension, allowing for rapid development and experimentation (34m10s).
• The template is not intended for deployment and should be modified to suit individual needs (34m16s).

Demo and Code Review

• GitHub Copilot chat and extensions can be set up and used in Visual Studio Code (VS Code) (34m24s).
• The demo showcases the process of cloning a template, starting a local web server, setting up port forwarding, and configuring a GitHub app to use as a GitHub Copilot extension (35m19s).
• The code examples are reviewed, and the use of Hono is explained, although it is not a requirement, and other frameworks like Express can be used instead (36m4s).
• The current SDK is only available for JavaScript and TypeScript projects, but it is expected to be available for other languages in the future (36m31s).
• The community is encouraged to contribute to the preview SDK project by opening issues to port the project to other languages, which can have a significant impact on the development of Copilot extensions (36m54s).
• Working on projects with millions of users can have a significant impact, and porting a project to another language can be a rare opportunity to make a difference (37m10s).
• The preview SDK is being used to pull out imports, including Hono, which is a framework for handling requests, and is similar to Express (37m36s).

Code Explanation and Handling Requests

• The code is configured to handle post requests to the API, and the GitHub app has permissions set up for this, represented by a slash in the URL (38m0s).
• When a post request is made, GitHub Co-Pilot sends a token in the header, which is used for security purposes, and the verify and parse request function from the preview SDK is used to validate the request (38m40s).
• The verify and parse request function checks if the request is valid and safe to use, and if not, it returns an error (39m1s).
• The token in the header is used to access GitHub, and if it's not present, an error is returned (39m30s).
• The create error event function from the preview SDK is used to show an error in the UI, and Co-Pilot chat will display the error with a UI-related message (39m50s).
• The create AC event function from the preview SDK is used to acknowledge the event and display a spinning animation in Co-Pilot chat, indicating that Co-Pilot is working (41m4s).

Interacting with the GitHub API

• The octokit project from GitHub is used to interact with the GitHub API and retrieve data (41m24s).
• The getUserMessage method retrieves the question asked by the user, along with previous messages for added context, allowing developers to build a more comprehensive Copilot extension (41m32s).
• The prompt function is used to ask a question and receive an answer, similar to interacting with AI services like Copilot Chat, and can be used to build a Copilot extension (43m36s).
• The createTextEvent method writes the response in Copilot Chat, while the messageContent method contains the rest of the message (44m14s).
• To complete the interaction, the createDoneEvent method is used to inform Copilot that the task is finished, and error handling is also available (44m43s).
• The template provided is a starting point for building a Copilot extension, and developers can customize it as needed (44m53s).
• To deploy a Copilot extension, developers can use their preferred deployment method, such as Azure or Netlify, and reference the URL in the GitHub app (45m15s).
• The preview SDK offers additional features and tools for building Copilot extensions, providing a more comprehensive development experience (45m35s).
• The project contains various features, including custom prompts, verify, security features, acknowledge sending, create confirmation events, and more, which can be explored for further information (45m42s).
• The create confirmation event allows users to confirm actions, and Co-pilot will generate a yes or no prompt to ensure the user wants to proceed (46m1s).
• The create references event enables users to natively show issues or PRs in Co-pilot, with a slightly different UI (46m16s).
• The create references event can be used to display information in Co-pilot, such as issues or PRs, and can be implemented using the SDK (46m20s).

Building and Deploying the Extension

• Co-pilot can be used to write a Co-pilot extension or VS Code extensions, and it can also be used to help with writing code (48m44s).
• The SDK provides features like create references event, which can be used to display information in Co-pilot, and can be implemented using the SDK (46m20s).
• The create AC event is used to display a spinning thing while the code is running, and it might be slower in development mode (50m13s).
• The project is still in development mode, and errors may occur due to changes made to the code (50m25s).
• The Copilot extension template allows users to create their own extensions and integrate them with other services, making it easy for anyone to start writing their own extensions (51m32s).

Creating Specific Agents and Extensions

• The template can be used to create extensions that support specific services or APIs, such as PostHog, and enhance the suggestions provided by Copilot (52m32s).
• The trend in AI is moving towards creating specific agents or extensions that are good at one thing, rather than trying to create a single service that does everything (53m20s).
• The Copilot extension template allows users to leverage the capabilities of Copilot and extend its functionality to support specific use cases (53m44s).
• The beauty of the Copilot extension template is that it allows users to create multiple extensions that can be used together to augment the capabilities of Copilot (53m51s).
• The template can be used to create extensions that support specific services or APIs, and can be used to enhance the suggestions provided by Copilot (52m27s).
• Users can create extensions that support services such as PostHog, and use them to enhance the suggestions provided by Copilot (52m43s).
• The Copilot extension template is a way to extend and leverage the capabilities of Copilot, and can be used to create specific agents or extensions that are good at one thing (53m44s).
• The template allows users to create multiple extensions that can be used together to augment the capabilities of Copilot, and can be used to support specific use cases (53m49s).

Q&A and Closing Remarks

• The discussion is wrapping up, and the host is giving viewers an opportunity to ask questions, with one question from Caesar Cesar Cru about using Copilot to review a Domain-Driven Design (DDR) with a specific class from QA. (54m51s)
• DDR is domain-driven, and it could be interesting to get specifics from an issue or use a tool like Girkin, which is used by business people to describe what it does, and integrate it into a Copilot extension. (55m5s)
• To answer Caesar's question, it's possible to use Copilot to review a DDR with a specific class from QA as long as there's a way to implement something from a QA standpoint to validate it, such as an API service. (55m39s)
• The host suggests that people think about pain points at work and how they can leverage Copilot extensions to help with workflows, making non-essential tasks easier and keeping users in flow while working. (56m4s)
• The host thanks Nick for his time and wants to share links discussed during the call, including the extension template, which is a straightforward way for people to contribute to the project and create extensions in different languages. (56m40s)
• Nick is open to new opportunities, including job offers, and viewers can follow him online to learn more about his work and extensions, such as the one he showcased during the call. (57m16s)
• The Open Source Friday streams are available on the Open Source Friday playlist on YouTube, allowing viewers to rewatch and take steps to build their own Copilot extensions using Nick's template and documentation (58m20s).
• Nick's repository has great documentation that leads users straight through the process of building a Copilot extension using NYX template (58m35s).
• Viewers are encouraged to build their own Copilot extensions and share their creations, with a promise of a reward for those who take action (58m46s).
• A GitHub action was created to showcase Open Sauce profiles, which puts an image that links to the user's profile on Open Sauce (59m16s).
• The Copilot extensions mentioned are available in the marketplace, and users can explore services they already use to see if they have a Copilot extension (59m41s).
• Nick will be speaking at Danny Thompson's conference, Commit Your Code 2025, where he will do live coding and talk about the value of Copilot extensions (1h0m20s).
• Nick will also be giving talks at a local conference in Montreal, K Fu, about HTML forms and Copilot extensions (1h0m50s).
• Nick is encouraged for his contributions to the SDK and is appreciated for his presence.
• Viewers are asked to follow Nick on social channels and give his repository a star, especially for the Co-Pilot extension.
• Companies looking for engineers, developers, or content creators are encouraged to consider Nick for a position.
• Viewers who build a Co-Pilot extension template are asked to share it, even if it's not published in the marketplace or for a specific service.
• The host appreciates the support for Open Source and invites viewers to share their projects, potentially featuring them on Open Source Friday.
• The show will be taking a short hiatus for the next couple of weeks but will return in January with exciting announcements from GitHub.
• Viewers are encouraged to subscribe to the YouTube channel to stay updated on upcoming announcements.
• The host gives shoutouts to Cojo, Tally, Ray, Carlos, Tonka, and Pima for their support and presence.