The Ongoing Challenges of DevSecOps Transformation and Improving Developer Experience

CuCon Conference and Adam Kentosh's Background

• The upcoming conference, CuCon, in San Francisco from November 18 to 22, will feature engineers sharing their experiences on implementing innovations in real-world scenarios, exploring tracks like architectures, engineering productivity, and generative AI in production (10s).
• Adam Kentosh, a field CTO of Digital AI, has over 20 years of experience in technology, starting with large-scale Hadoop clusters and moving into consulting, working with Fortune 20 companies transitioning to cloud containerization and Kubernetes (1m11s).
• Adam worked at Hashi Corp, experiencing the company's growth, and is now at Digital AI, bridging the gap between field teams, product, and marketing teams, ensuring that customer needs are met and messaging resonates with customers (1m52s).

DevSecOps Transformation Challenges

• Despite being discussed for a while, DevSecOps transformation remains an ongoing challenge, with 80% of companies still in mid-transformation, according to the 2023 State of DevOps report (3m9s).
• The reason for this slow transformation is complex, but it can be attributed to technology changes and disruption, including the maturity of cloud, mobile devices becoming the primary way customers interact with applications, and the evolution of DevOps itself (4m1s).
• Adam notes that while DevOps transformation has occurred at the team level, enterprise-wide transformation remains elusive for many customers, and this is a problem that needs to be addressed (3m54s).
• The concept of DevOps has evolved significantly since its inception, from simply having developers and operations teams in the same room to a more integrated and automated approach (2m50s).
• The slow pace of DevOps transformation is surprising, given that companies started adopting it around 2010-2012, and it has been 10-12 years since then (3m28s).
• Initially, teams were given autonomy to choose their tools and create their own DevOps teams, which helped with speed to market and velocity, but as the organization grew, standardization became a challenge (4m20s).
• The challenge is to determine how much standardization is enough, as too much can hinder innovation, while too little can lead to inefficiencies (5m25s).

Addressing Standardization Challenges in DevOps

• There are two options to address this challenge: either build an interface layer on top of different technologies to integrate them, or focus on metrics that can be driven out of the platform, such as team performance and bug rates (5m40s).
• The first approach involves not bothering with standardization and instead focusing on gathering metrics to help teams perform better, using frameworks like Dora to level up teams (6m42s).
• The second approach involves creating a platform engineering team to develop developer-oriented tooling and standard interfaces, reducing toil and increasing productivity (6m53s).
• Both options are viable, but businesses may prefer some level of tool standardization to simplify contracts and negotiations (7m20s).

Balancing Standardization and Innovation in DevOps

• Finding a balance between standardization and innovation is difficult, and the shift to DevOps has also increased cognitive load on developers (7m46s).
• Dora's framework provides a context for leveling up teams and focusing on specific areas of improvement, using metrics such as performance and bug rates (6m24s).
• Recent reports from GitLab and GitHub have shown that developers spend only around 25-32% of their time writing code, with the rest spent on tasks such as improving existing code, administrative tasks, meetings, testing, and securing applications (8m19s).
• The shift left approach puts a lot of strain and stress on developers, who are not only expected to write code but also understand how to test and secure it, release it, and maintain its operations over time (8m42s).

Improving Developer Experience

• As a result, developer experience is becoming increasingly important, with some companies even having a VP of Developer Experience to ensure their developer community is productive, has a good impact, and is satisfied (9m12s).
• To reduce the strain on developers, a product-centric approach is recommended, where product teams have capabilities such as testing, security, and development, and work collaboratively together (10m27s).
• This approach would allow for better collaboration and a more productive product team, as developers have been asking for better collaboration in surveys (10m57s).
• However, the challenge lies in changing the organizational structure to match the team structure, as dedicated testing and security organizations may resist giving up control and blending teams (11m22s).
• Organizational changes are necessary to adopt a product-centric approach, but this can be restrictive and limiting, leading to a situation where developers are expected to own more responsibilities (11m53s).
• To improve developer experience, small nudges can be made by implementing targeted use cases, such as trials, where a product-centric approach is taken, and the team is allowed to be successful, validating their numbers and justifying the working model (12m22s).
• This approach enables the organization to work together better, improve overall performance, and scale the model by creating best practices and showcasing real results and numbers (13m12s).

Pragmatic DevSecOps Transformation

• The industry has figured out how to implement DevSecOps transformation, but 80% of organizations are still mid-transformation, facing the same problems despite methodology changes over the last couple of decades (13m39s).
• Flagship examples of organizations that have done DevSecOps transformation well include Fang or mang companies, which operate at a different level and produce content around progressive development approaches (14m8s).
• However, most organizations, such as those in financial services, insurance, and gaming, do not necessarily care about delivery frequency and instead prioritize compliance, security, and customer experience (14m51s).
• Being pragmatic about what is useful and what will resonate with the organization is essential, and it is crucial to consider what outcomes the organization cares about, such as new features, stability, or security (15m27s).
• Technology has enabled a more reasonable approach to delivery, allowing for better quality, more secure applications, and a balance between delivery frequency and stability (15m59s).

Developer Mindset and Organizational Structure

• Developers enjoy learning new things and are often happy to take on new tasks such as testing and release pipelines, which provides them with opportunities for professional development and new skills (16m33s).
• The engineering mindset is to always look for ways to solve problems, which can sometimes lead to individuals taking on tasks without being asked (17m35s).
• From an individual standpoint, the biggest reason for not learning is the desire to solve problems and take on new challenges (17m47s).
• From an organizational standpoint, the structure may not have evolved to support collaboration and a product-oriented approach, which can hinder the adoption of new practices (17m55s).

Challenges of DevSecOps Adoption in Traditional Organizations

• Companies that are successful in DevSecOps often have a product mindset and were built from the ground up with this approach, which can make it difficult for traditional organizations to adopt this mindset (18m32s).
• Traditional organizations may be asked to fundamentally change their business model, which can be challenging due to organizational structure, politics, and skill levels (18m47s).
• Asking companies to adopt a new approach can be like asking them to act in an unnatural way, which can lead to challenges and difficulties in implementation (19m27s).
• Organizations are trying to evolve by bringing teams together in a meaningful way without necessarily changing their reporting structure or team composition, but this approach may not be conducive to their growth (19m42s).

Prioritizing Outcomes over Velocity in DevSecOps

• The role of platform engineering or DevSecOps is crucial, and velocity should not be the primary focus, but rather the outcomes that organizations want to deliver for their customers (20m9s).
• Taking a deliberate approach to software development is more meaningful than focusing solely on velocity, and organizations should prioritize the stability of their customer environment and application (20m30s).

Software Engineering Intelligence Platform

• Platform engineering has value, but there is equal value in the concept of a software engineering intelligence platform, which involves unifying data from planning, coding, testing, securing, and operating software development (21m2s).
• Historically, organizations have purchased solutions for each area of software development, generating data into separate databases, but this approach has limitations for business intelligence and agility (21m34s).
• A software engineering intelligence platform would allow organizations to link data together, enabling them to answer questions about team performance, risk, and impact on cycle time across the organization (22m32s).
• Unifying data into a meaningful data lake and putting engineering processes in place would set up AI initiatives and provide valuable insights for organizations (22m36s).
• This approach would help organizations reduce the time spent on change advisory board reviews and tier one resolution calls, breeding value for the organization (23m5s).

Leveraging AI and ML for Improved Operations

• Gardner coined the term "software engineering intelligence platform" in March, highlighting the importance of this concept in the industry (21m19s).
• Measuring the current state of an organization and unifying data to answer meaningful questions is crucial for applying artificial intelligence and machine learning to make recommendations (23m26s).
• The goal of many organizations is to leverage AI and ML over the next two years to improve their operations (23m45s).
• To continue the conversation on DevSecOps transformation and improving developer experience, Adam can be reached on LinkedIn (23m54s).
• Adam is open to having conversations and hearing counter perspectives on the topic, which helps him understand different approaches and strategies used by other companies (24m7s).
• Adam has the opportunity to talk to 50 to 100 customers every year, mostly in the large Enterprise space, and values hearing perspectives from outside this space (24m15s).