Mercado Libre

• Mercado Libre is the largest online commerce and payments ecosystem in Latin America, operating in 18 countries with hundreds of millions of users.
• Mercado Libre is exploring how to scale the use of GitHub Copilot across the entire team after seeing a 50% improvement in productivity on personal projects.
• Mercado Libre believes that GitHub Copilot has the potential to significantly increase the number of features it can deliver to customers.

Pacífico Seguros

• Pacífico Seguros, a Peruvian insurance company, adopted GitHub Copilot across its software development and business operations.
• The company received considerable budget support from stakeholders to cover licenses for all developers.
• Pacífico Seguros faced challenges in the cultural and organizational change required for Copilot adoption, including resistance from some developers due to concerns about dependency and job security.
• To address these challenges, Pacífico Seguros conducted qualitative surveys, analyzed GitHub metrics, and engaged in face-to-face conversations with developers to understand their concerns and use cases.
• The company implemented a more sustainable adoption model by identifying and diversifying use cases for each developer role.
• Pacífico Seguros uses various documentation methods, including wikis, pages, and a WhatsApp community, to manage their knowledge and facilitate onboarding and role-specific learning.
• They have a three-layer governance model with champions as the support layer, ensuring that adoption is driven by developers who genuinely find value in the tool.
• To encourage developers to collaborate, they provided recognition, dedicated spaces on GitHub, merchandise raffles, and opportunities to define things.
• They implemented a self-service system where developers can request licenses through GitHub actions, ensuring that 100% of satisfaction surveys are completed monthly.
• New developers are required to pass a test to demonstrate a basic understanding of the documentation before being granted a license.
• They matured their documentation practices by implementing documentation as code and centralizing all documentation in a single location.

Measuring Copilot Adoption and Impact

• A framework for measuring the adoption of GitHub Copilot, consisting of four levels: adoption, activity, efficiency, and impact, was introduced.
• Adoption metrics measure developer satisfaction, license activation, and Net Promoter Score (NPS).
• Activity metrics measure how much developers use the tool.
• Efficiency metrics measure developer productivity and are influenced by the time it takes for developers to learn and adopt the tool.
• Impact metrics measure the business impact of Copilot, such as improved code quality, faster deployments, and increased revenue.
• Copilot adoption doesn't guarantee immediate productivity gains. Studies suggest it takes at least 3 months for developers to efficiently adopt Copilot and achieve a 20-22% increase in productivity.
• Measuring Copilot's impact involves a framework that considers adoption, activity, efficiency, and a challenge phase.
• Reducing time to market (TTM) is a key metric, but it's influenced by various factors beyond coding, such as testing, security, and documentation.
• Copilot adoption requires organizations to mature other practices like continuous testing, security automation, and documentation as code to fully realize TTM improvements.
• The goal is not just to increase developer productivity but to leverage the time saved to introduce new features, generate revenue, and expand business opportunities.