Applying AI to the SDLC: New Ideas and Gotchas! - Leveraging AI to Improve Software Engineering

19 Sep 2024 (3 months ago)

Current State of AI in Software Engineering

Tracy Ben, a software architect and researcher at The MITRE Corporation, believes that the current state of artificial intelligence (AI) in software engineering is at "the peak of inflated expectations" on the Gartner hype cycle. (5m1s)
Ben believes that it will take 2-5 years to reach the "plateau of productivity" because software engineering is complex and not a linear process. (5m12s)

Ben worked with data scientists and data engineers to understand the different types of AI and machine learning (ML) that exist beyond generative AI. (6m5s)
Generative AI is only one type of AI that can be used to improve software development. Many other types of AI and machine learning algorithms already exist and are being used. (6m46s)

Generative AI can be used for a variety of tasks in software development, including generating code, debugging, documentation, and explaining existing code. (11m57s)
Generative AI can be a useful tool for requirements analysis by helping to uncover sentiments and requirements that may not be immediately obvious. (12m58s)

Developers should store both their prompts and the specific model or service used in version control due to variations in how prompts interact with different models. (13m18s)
It is crucial to ensure diversity in data sets, interviews, and user groups to mitigate potential biases present in AI models. (13m52s)
While generative AI can assist with synthetic test data generation, it is important to be aware of potential issues such as data privacy, data integrity, irrelevant test generation, and a lack of transparency and explainability in generated code. (15m29s)

There are unequal productivity gains for individuals using AI code generation tools, with junior developers experiencing fewer gains than senior developers. (19m30s)
Code churn, which refers to the repeated process of checking in and modifying code, is on track to double in 2024 compared to previous years. (20m22s)
AI-generated code can introduce security vulnerabilities, as demonstrated by an example where SQL injection vulnerabilities were present in code generated by GitHub Copilot. (21m6s)

Organizations should ensure their software development lifecycle (SDLC) is in good shape before incorporating generative AI. If there are existing problems, adding AI might not be beneficial. (25m59s)
Before using AI, organizations should implement continuous integration practices, establish a reliable pipeline for deployment, and ensure environment parity for testing. (27m58s)

When incorporating generative AI, organizations should anticipate changes in workflow, measurements, and metrics. It's crucial to acknowledge that perceived productivity gains might not immediately translate to real gains and that team productivity should be prioritized over individual productivity. (30m58s)
To understand developer productivity when using AI, it is important to consider human elements such as satisfaction and trust, especially with non-deterministic generative AI. (31m24s)
When using AI, especially in an enterprise setting, it is crucial to be aware of data security and privacy concerns, particularly when using third-party subscription services that might require sharing sensitive information like codebases. (33m13s)

Organizations should approach AI implementation strategically by conducting needs assessments, running focused pilots, establishing governance, monitoring feedback, and dedicating resources to stay updated on the rapidly evolving AI landscape. (34m23s)
Software architectural principles, capabilities, and trade-off analysis should still be considered when designing toolchains for AI-assisted software engineering. (37m27s)
It is important to keep humans in the loop when using AI in software engineering and to ensure that all prompts, model numbers, and names are kept in source code. (38m16s)

Data silos are on the rise as different AI tools are being used, which can lead to slower flow and more quality issues. (42m1s)
AI is transitioning from being an independent tool to becoming agents integrated into the software development lifecycle (SDLC), necessitating a consideration of the impact on human team members and the definition of specific roles for AI agents. (44m27s)

Organizations need to understand and address the use of generative AI, including prompt engineering, ethics, and its impact on team performance, trust, and reliability, while prioritizing cybersecurity and establishing clear guidelines for its use. (46m51s)
Qualified software engineers and developers who understand and can leverage software engineering principles will be in a good position to adapt to the integration of AI into the software development lifecycle. (43m37s)