Decentralizing Decision Making with Shawna Martell & Dan Fike

InfoQ Dev Conference in Boston

• The InfoQ Dev conference in Boston will feature over 20 senior software practitioners sharing their experiences and practical advice on critical topics such as generative AI, security, and modern web applications (6s).
• The conference will provide opportunities for attendees to connect with peers and speakers at social events, and is brought to the audience by the team behind QCon and the International Software Developer Conference (27s).

Cartek's Navigators Program

• Shawna Martell and Dan Fike from Cartek recently spoke at QCon London about decentralizing decisions and empowering individual contributors through a concept called Navigators (56s).
• The Navigators group was created to address the lack of a clear engineering strategy at Cartek, which made it difficult for engineers to make decisions aligned with the organization's principles (1m37s).
• The engineers at Cartek were struggling to understand what rubric to use when making decisions, and the lack of clear guidance led to arbitrary or random decision-making (1m44s).
• The Navigators are a result of Cartek's engineering strategy, which was created to provide a framework for decision-making and help engineers navigate trade-offs effectively (2m15s).
• The engineering strategy was created in response to requests from engineers throughout the organization who were craving guidance and context for their decision-making (3m16s).
• The strategy was not imposed upon the organization, but rather was created to fulfill the needs of the engineers and provide a clear direction for decision-making (3m32s).
• The goal was to create a cohesive set of principles that could be agreed upon in advance, rather than repeatedly rehashing the same issues without conviction in the answers (3m39s).
• The process began by documenting the existing principles underpinning current decisions, rather than trying to define where the organization wanted to go, in order to establish a baseline for incremental change (3m56s).
• It was important to focus on the current state, rather than an idealized version, to avoid creating unrealistic expectations and to understand what needed to change (4m43s).
• Writing down the current decision-making practices, even if they seemed absurd, helped to identify what was not working and what needed to change (5m4s).
• This process is similar to documenting architectural decisions, where implied and accidental decisions can be uncovered and improved upon (5m27s).
• The idea of documenting current decisions to understand the underlying principles was influenced by Will Larsson, who suggested writing down five architecture decision documents to tease out the underpinning principles (6m12s).

Establishing Decision-Making Principles

• The focus is on identifying principles, rather than just facts, with principles being defined as conditional statements that guide decision-making, such as "when X, then Y" (6m44s).
• Distilling principles from decisions requires careful analysis and sometimes requires going back to the original decisions to understand the underlying reasoning (7m9s).
• Architecture decision records are used to capture the trade-offs considered, the good and bad aspects, and the reasons behind the final decision, providing context to the decision-making process (7m19s).
• Decision records help in understanding the "why" behind the process, including how architecture decisions are made and how systems are built (7m37s).
• A fictional example illustrates the importance of decision records, where two options for implementing a feature are weighed, and the pros and cons of each option are considered, but the decision record does not establish the organization's principles (7m48s).
• Establishing principles is crucial, as it helps in understanding what is important to the organization, such as scalability or maintenance efforts, and steers decisions in the right direction (8m14s).
• Not having a clear understanding of the principles can lead to designing for an end state without considering how to get there, resulting in a lack of direction and unclear roadmaps (9m4s).

The Role of Navigators

• Knowing the starting point is essential to creating a roadmap and navigating towards the destination, which is why the term "Navigators" is used to describe the process of finding the way (9m25s).
• The term "Navigators" is contrasted with "cartographer," as a cartographer builds the map, whereas Navigators help others understand and interpret the map, providing guidance and direction (9m40s).
• Navigators, who are senior and talented individuals in the organization, are involved in the process of producing the strategy, vision, and plan, but their role is to help others understand and interpret the map, not to create it (9m59s).
• The organization has a map that serves as a guide for decision-making, but it can be hard to read and navigate, so they have designated Navigators to help others make decisions in accordance with the map's principles (10m27s).
• The Navigators are a dozen individuals across an organization of about 400 engineers, and they sit in various positions, from deep within the organization to reporting to VPs (11m4s).
• The role of a Navigator is not about their title or position, but rather about having shown good judgment and being empowered to make decisions using the engineering strategy, with the understanding that they will be held accountable to the CTO (11m30s).
• Navigators are not responsible for making all decisions, but rather for ensuring that their team is following the strategy and providing guidance when needed (12m3s).
• Navigators are a resource for their teams when they are struggling to make a decision or need help applying the strategy, and they are also responsible for keeping their finger on the pulse of what's going on to insert themselves when necessary (12m27s).
• To do their job well, Navigators need to have deep context on the area of the product or code they are responsible for, and they need to understand where their team's work intersects with the strategy (12m56s).
• Navigators are not making every decision, but they are probably aware of every decision, and if they let rogue decisions slide, that's an issue (13m27s).
• Ideally, when things happen that are contrary to the strategy, a Navigator should know about them and either fix them or surface them as a shortcoming in the strategy itself (13m37s).
• The Navigators are essentially the largest input to the strategy as it evolves, and they need to surface areas where the strategy is imperfect and needs to be corrected (13m48s).
• A gap in the map of knowledge and strategy is addressed by having a conduit for information to flow both ways, connecting team members to broader strategy decisions and vice versa (14m15s).
• The Navigators, a group of 12 individuals from various parts of the organization, including front-end engineers, back-end engineers, site reliability engineers, and security engineers, serve as this conduit (14m29s).
• The Navigators have a direct relationship with the executive running The Navigators program but exist elsewhere in the hierarchical org chart with indirect paths that eventually roll up to the executive (14m41s).
• Some Navigators don't even roll up to the executive, and they can be seen as root nodes in a shadow org chart of influence that exists parallel to the actual management hierarchical org chart (14m58s).
• Navigators are selected based on their influence over the right set of people in this shadow org chart and their judgment to interpret the strategy for those folks (15m9s).
• The structure is less hierarchical and more about leveraging natural inclinations and relationships that already exist amongst team members (15m28s).

Decentralized Decision-Making

• The organization does not have formal architect roles or titles, but some staff engineers functionally operate as architects on projects or teams where it is warranted (16m7s).
• The decision to not have a formal architect role is intentional, as the organization wants to decentralize decision-making amongst other team members and avoid creating a choke point (16m29s).
• The strategy is written down to allow Navigators and other team members, including junior software engineers, to make decisions and understand if they are doing a good job without needing to consult an architect (16m41s).
• The organization follows a process similar to the architecture advice process, where anyone can make an architecture decision as long as they consult with affected people and those with relevant experience (17m14s).
• At the organization, anyone can make an architecture decision as long as they fulfill the requirement to seek advice from the strategy and Navigators (17m28s).
• Decision-making involves using resources to hold decisions up against a measuring stick and check if they are aligned with the organization's take, and this process helps identify gaps in the engineering strategy (17m43s).
• The organization's strategy serves as a collective source of advice, rather than a single person, and individuals make decisions informed by the strategy (18m47s).
• The strategy outlines criteria for making trade-off decisions, such as performance versus scalability, and helps individuals understand what is important in a given context (19m42s).
• Gaps in the strategy can lead to a lack of clear guidance on certain issues, and in some cases, the organization may not have formed an opinion on a particular matter (19m47s).
• An example of this is the guidance on whether to build a new feature or service in the existing monolith or create something new, where the organization had abstract direction but no specific guidance (20m47s).
• In such cases, individuals are directed to check in with their Navigators, who can provide more specific guidance and help them understand the reasoning behind the lack of universal guidance (21m14s).

Iterative Refinement of Decisions

• The organization's approach to decision-making involves iterating and refining decisions, with the understanding that they may not be perfect initially, but can be improved over time (18m25s).
• The goal is to make decisions that are incrementally less wrong, by holding them up against the strategy and identifying areas for improvement (18m34s).
• Developing principles for decision-making within an organization is crucial, but it's often more effective to try different approaches over time and refine the strategy based on the outcomes of those decisions (21m18s).
• Leaving some wiggle room in decision-making processes allows for flexibility and adaptation to unique situations, rather than trying to find a one-size-fits-all solution (21m43s).
• Navigators play a key role in helping teams figure out their micro-strategies within the context of the larger organizational strategy, by providing guidance and support while allowing teams to make their own decisions (22m1s).

Collaboration among Navigators

• Navigators develop relationships with each other through collective time spent together, which enables them to consult with each other and share insights when faced with difficult decisions (22m43s).
• Although Navigators don't have collective responsibility, their shared title and individual responsibilities encourage collaboration on hard decisions and the sharing of knowledge and expertise (23m23s).
• The Navigator approach is an attempt to bring together smart people to make decisions, but it's distinct from simply putting the best and brightest in a room, as it focuses on developing relationships and sharing knowledge (23m50s).
• The Navigator model is particularly effective for addressing crosscutting concerns, as it leverages the talents of smart people to solve complex problems, rather than having them focus on a single task or project (24m16s).
• The ideal Navigator is a t-shaped individual who has broad knowledge and deep expertise in their area of specialty, and can communicate effectively with others who have similar profiles (24m27s).
• Decentralizing decision-making involves understanding and discussing problems together, even if individuals don't have the exact same problem, to find a collective solution (24m38s).

Addressing Crosscutting Concerns

• Putting all the best and brightest people in a room to work on a microservice can result in the worst possible outcome due to too many people being involved in one thing (24m50s).
• There is an implied mentoring responsibility in teaching others to make decisions, and companies often struggle with getting new, younger engineers to be comfortable making decisions (25m6s).
• The Navigators play a mentoring role in teaching people how to make decisions, and it would be hard to convince that someone who isn't already focused on fostering talent within their team would make a good Navigator (25m31s).
• Navigators are senior leaders positioned within their teams, and their responsibility is to ensure that junior team members understand how to make decisions, negotiate trade-offs, and seek advice (25m50s).

Mentoring and Decision-Making

• To become a Navigator, it's not about applying, but rather a combination of being recognized as a key person in the organization and being nominated by VPs who run a particular organization (26m28s).
• Managers are not considered for the Navigator role because they are more focused on team and execution, whereas Navigators need to bring a deep technical understanding to make better decisions (27m5s).
• The Navigator role requires a different lens than managers, looking at the same set of problems, and including managers in the Navigator role doesn't provide the additional benefits being looked for (27m58s).
• Decentralizing decision-making involves combining two lenses to make both individuals and teams more effective in their roles, with engineering managers optimizing for their team while minimizing disruption to the organization, and directors optimizing for the organization while minimizing disruption to their team (28m8s).
• Strategy Navigators can inject an additional perspective to optimize for the organization, finding where team and organizational goals align, and bringing a strategic element to decisions that may get lost in technical details (28m47s).

Measuring Success

• This approach allows Navigators to contribute unique insights to low-level decisions that shape big organizational outcomes, which may not be expected of all managers or directors (29m15s).
• Measuring success involves evaluating how the Navigator's approach improves decision-making, with an example being a team considering building a new platform service to solve a problem that had manifested in multiple domains (29m26s).
• In this example, the Navigator was able to use the organization's strategy to hold the decision up against the strategy and determine that building a one-size-fits-all solution was not the best approach, avoiding the need to build consensus with multiple groups and stakeholders (30m56s).
• The Navigator's approach empowered them to make a decision quickly and efficiently, without needing to slowly build consensus over time, and allowed them to say that the decision was not aligned with the organization's strategy (31m34s).
• Decentralizing decision-making involves changing the organization's thought process, where a standard strategy is the default answer, and any deviations from it require a good argument, which often falls apart, making the decision process more effective (31m39s).
• In the past, decision-making required convincing people of the right course of action and explaining why they should care, but having a clear decision-maker, such as a Navigator, can smooth over this process (32m16s).

Getting Started with Decentralized Decision-Making

• To get started with decentralizing decision-making, it's essential to have a well-articulated organizational strategy, as a Navigator's program without one won't be effective (32m52s).
• Symptoms of missing a strategy include slow decisions, escalating decisions, and a sense of being lost among team members, often indicated by qualitative signs such as people questioning decisions and feeling uncertain about the right course of action (33m5s).
• The first step in addressing these symptoms is to develop a clear strategy, which involves looking at past decisions, design docs, and identifying non-controversial principles to start with (33m48s).
• To develop a strategy, start by looking at past decisions, writing new design docs, and identifying principles that are not controversial, then gradually build upon them (33m58s).
• As the strategy becomes clearer, people may become upset with the reality of the current decision-making process, but this is an opportunity to make quality of life changes to the strategy and improve the organization (34m17s).
• Writing down the current state of decision-making and measuring it is essential to improving it, as it allows for identifying areas that need change and tracking progress (34m47s).
• To develop a solution, it is essential to first identify the problem, which involves understanding the current engineering culture and operations, taking the time to think about and write down the issues, and diving deeper into the matter (34m55s).

Developing a Strategy

• When reviewing past design decisions, it may be necessary to engage in a form of "archaeology" to uncover the implicit trade-offs made by the organization, as these may not be explicitly stated in design documents (35m36s).
• By examining past decisions, it is possible to identify patterns and trends in the organization's priorities, such as favoring performance over scalability (35m49s).
• To develop a strategy, it is crucial to understand the organization's current state, determine the desired direction, and identify the right people to help make decisions (36m54s).
• Engineering leaders often have a handful of trusted individuals in mind who can provide guidance and advice, and these individuals can serve as "Navigators" to help deploy the strategy across the organization (37m14s).
• Navigators may need to be inserted into teams where they are not currently present, and they will need to gain context to effectively operate in their new roles (37m46s).
• Senior leaders are likely to be aware of the individuals who can serve as Navigators and can play a key role in deploying them effectively across the organization (38m1s).
• The episode is coming to a close, with an invitation for listeners to join again for another episode soon (38m18s).