The Coming AI Code Generation Chaos May Already Be Here: Solutions?

This post originally appeared in Forbes.

In addition to solving many problems around fast, easy and accurate software code generation, AI will help create much more new code. That will create problems for everyone managing it. But, those problems don’t have to cause chaos. The good news is that companies that have successfully managed to tame code management chaos and gain a degree of control, standardization and predictability may have an answer for the coming explosion of AI-generated code.

Picture a company with zero software engineers in 2000 that now boasts over 10,000 software developers. Their work supports millions of useful self-service actions initiated by their customers, like browsing product details or requesting a credit line increase at an online bank. The company’s software teams make 10,000 software product code updates daily, use all kinds of services and software tools while trying to balance their performance and usage costs, and train hundreds of machine learning models in real-time.

Enterprises whose software-related challenges look like that are becoming increasingly common. For companies whose business processes run on code, growing efficiently without descending into complete management chaos should be a challenge on their Top 10 list.

To solve the code management problem associated with rapidly expanding tools, practices and standards, it helps to understand how the problem grows.

Small companies typically give considerable freedom to their software teams, who usually end up building monoliths, or unified software applications that are self-contained and independent from other applications. But this freedom has a cost. Program managers often have no way of knowing whether developers apply product updates correctly, or whether they use the best coding practices. As a result, code errors and security vulnerabilities grow — and so do timelines for finding bugs and fixing errors.

To regain control, midsize companies often migrate to break down software monoliths into smaller, better-manageable software ecosystems called microservices. Switching to a microservices architecture also allows them to benefit from cloud efficiencies.

But even having reached the ideal size of microservices, with component conflicts reduced and a good time to market achieved, other problems surface: With many different technologies in the mix, the cognitive load for software developers increases and becomes a barrier.

What is cognitive load? It’s the skill level of an individual software developer needed to make and run an application. More microservice components mean more cognitive load, so the development speed slows down, and costs increase. In addition, the proliferation of microservices complicates governance, compliance and security.

Now, larger companies are looking to cut development costs, implement optimization algorithms, gain control of microservices governance and improve and automate security with compliance built in. Their leaders look into building internal platforms that standardize and streamline the processes that manage workflows, code, pipelines, infrastructure and data.

Possible solutions combine standardization of routine tasks, a curated number of tools that work in concert and an environment where roles regulate access levels, permissions and resource access levels — and doing it all with transparency and visibility for the management.

Internal developer platforms (IDPs) are one such solution that attempts to combine those features. Implementing platforms allows management to push centralized decisions across all teams and help remove biased decisions by teams who have been used to using various tools of their choice chaotically, so a switch to platform thinking also requires a culture shift that not every developer may love.

How can businesses that choose the platform approach improve the chances of adopting it successfully?

• Start With The Build Vs. Buy Evaluation: Although engineers prefer to build something from scratch, a more practical approach that can deliver results faster may be to customize one of the existing platforms available on the market. Start with customizable open platforms. If it needs too much customization, you can consider a more specialized platform to fit your industry and your specific IT environment.

• Go For Easy Wins: The chances are that many development teams have a nagging issue they’ve been struggling with that a platform may be able to help address. For example, tasks that prevent developers from doing work with business impact — like the wait between requesting a new project environment and getting it or having to choose, support and monitor another development tool all on their own.

• Define And Communicate Platform Scope: For example, the best candidates for standardizing the majority of development environments are monitoring, logging, debugging, security, authentication and authorization, connections to back-end systems, failover, load shedding and throttling, common shared libraries, testing infrastructure, and release infrastructure — but you don’t have to implement them all at once. But, do communicate the rules that will still allow developers to choose their own tools outside of the platform.

• Be Ready To Offer Help: Groom or hire outside experts who will be available to train and will be available to answer questions the developers may have.

Implementing an IDP and encouraging platform thinking is a challenge that takes time. Today, the main reason for an approach favoring automation and standardization is the need for companies to keep the marginal cost of developing new products low, and the speed of new product releases fast.

Soon, as codebases swell, even smaller teams may start facing code and tool management problems that larger companies face now. AI code generation tools are already here. Some are already integrated into developer tools, like OpenAI’s Codex and GitHub’s Copilot. Other developers are testing the possibilities with ChatGPT.

What’s already clear is that the volume of code is already set to rise exponentially. My concern is that many companies will not be ready, so they cannot take full advantage of the new possibilities. A shift in thinking that prioritizes keeping the cognitive load manageable offers a promising solution in a world with a lot more code.