Transforming Developer Productivity: How Spotify Scaled w/ Backstage Plugins&Open-Sourced Innovation

Backstage Development Team

• Per is the lead developer experience and has been leading Backstage since its start in 2017. Per joined Spotify in 2016 as an engineering manager. (21s)
• Mike is the tech lead for Backstage at Spotify and has been with the company for five and a half years. Before joining the Backstage team, Mike worked in the premium mission, where he used Backstage daily. (1m40s)

Spotify's Productivity Challenges

• In 2017, Spotify faced productivity challenges due to its vast and complex ecosystem of components and services. The autonomous engineering culture, while positive in many aspects, resulted in fragmented documentation, diverse technologies, and inconsistent practices, making it difficult for engineers to navigate and learn efficiently. (3m0s)
• Spotify engineers faced three main productivity challenges: difficulty finding information, excessive context switching, and a lack of standardization across the engineering organization. (7m41s)

Backstage: A Solution to Productivity Challenges

• To address these challenges, Spotify developed Backstage, an internal developer portal that serves as a single pane of glass for their infrastructure. (10m47s)
• Backstage's success is attributed to two key engineering best practices: infrastructure as code and a plugin architecture that enables distributed code ownership. (13m9s)

Backstage Architecture

• Backstage is a platform with a web-based React frontend and a horizontally scalable backend written in Node.js. The backend interacts with infrastructure components like databases and caches, as well as third-party APIs, primarily using HTTP. (15m22s)
• A plugin in Backstage is a package of Typescript code compiled to Javascript. These plugins can be published to NPM for public use or kept private. They primarily function in either the frontend or backend, enhancing the platform's capabilities. (15m49s)

Backstage's Tool Library

• Backstage offers a "tool library" comprising core services in both the frontend and backend. This collection provides functionalities that simplify plugin development, promoting consistency and encouraging contributions to the platform. (18m49s)
• Backstage offers a database management system where plugin owners receive a configured database connection, abstracting away the configuration and separation of databases. (19m10s)

Backstage's Plugin Integration

• Backstage has moved away from manual plugin integration that required specific code modifications. (20m2s)
• Backstage now utilizes a declarative integration system, allowing plugin and plugin module installation via yarn, simplifying the adoption process. (20m41s)

Backstage's Extensibility

• Backstage's extensibility allows developers to add features and capabilities to their instances. Developers can browse and manage various resources within the platform. (27m31s)
• The speaker uses a hypothetical scenario where their company manages planets. They demonstrate an API that provides information about planets in the solar system. (28m0s)
• The speaker highlights the integration of a new backend and frontend system in Backstage, replacing the standard systems that required manual wiring. This new system offers a declarative integration solution. (27m21s)

Benefits of Backstage's Plugin Architecture

• Backstage's plugin architecture allows teams to work independently on features without coordination, enabling parallel development and distributed ownership of features. (32m43s)
• Backstage's consistent plugin-based foundation makes expertise transferable between plugins, facilitating quicker contributions when individuals switch teams. (33m27s)
• Organizations adopting Backstage have the flexibility to choose or build plugins that align with their specific requirements, enabling them to create a customized developer platform. (34m10s)

Best Practices for Backstage Adoption

• To reduce repetition in systems, code should be moved from adopter instances into plugins and then into the framework. (34m36s)
• Core systems should be built with the extensibility model to ensure its effectiveness and suitability for its intended purpose. (35m34s)
• Nested extensibility, where extensions can themselves be extended, is a powerful concept that allows for maximum flexibility in a system. (35m42s)