Software developers can make huge productivity gains through the adoption of AI tools – sometimes with speed increases of up to 20x. That was the outstanding conclusion of our recent CTO Roundtable, Exploring AI-Driven Productivity in Software Development, hosted by Iain Bishop and Aleksandar Karavasilev, CEO and CTO of Damilah Technology.

Kicking off the event, they presented the findings from a series of controlled experiments they had conducted at Damilah, which demonstrated that time savings of up to 2x are quickly and easily achievable by using AI tools, in particular for tasks such as:

• Quality analysis
• Coding
• Unit testing

Furthermore, when engineers already experienced in using AI performed the set tasks, they were able to operate 4x to 6x faster.

While Iain and Aleksandar highlighted these impressive gains, the discussion revealed even greater possibilities. A start-up founder explained how combining a series of AI tools for different stages of development enabled him to deliver productivity improvements of up to 20x. To do this, he used:

• Perplexity for market research
• Replit and Lovable for building apps
• Tabnine for auto-completion of code

This illustrated how companies willing to fully embrace AI across the development lifecycle can potentially achieve extraordinary results – for example, by enabling them to rapidly put new product ideas in front of users as basic MVPs. As a result, this founder now plans to launch at least three start-ups within a year.

Increasing dynamism

This kind of accelerated development cycle can usher in a whole new era of dynamism in the market, it was concluded—the main points being:

• Start-ups are usually lacking capital.
• AI enables them to develop an application at pace and at low cost, before significant sums of money need to be invested.
• This helps to de-risk new ventures and make it easier to attract funding.

But even companies who take a steadier or more cautious approach to AI-adoption, it was agreed, can still make immediate productivity gains. This could be, for example, by starting with ChatGPT, then progressing to more sophisticated tools such as GitHub Copilot or Cursor.

Overcoming barriers and managing adoption

The conversation also turned to barriers to AI adoption. Several of those present described a certain amount of reluctance or resistance among larger enterprises, often due to concerns relating to company policies and regulatory matters.

However, it was noted that simply banning the use of AI tools can lead to “shadow AI”, where developers, in particular, will find workarounds in a constant pursuit of innovative ways of working.

Several attendees agreed that the solution to this is not to block, but rather to encourage the use of specified AI tools with some strict guardrails in place to ensure certain boundaries are not crossed, particularly with regard to security, privacy and IP protection

Addressing recruitment challenges

Questions were also raised about whether developers should be allowed to use AI during the hiring process, as often this could mask an applicant’s true capabilities when it comes to coding and testing. However, as one participant pointed out, companies should be looking to hire employees who are adept at using AI, as having an AI-proficient workforce will be necessary to maintain competitive advantage in the future.

Another attendee recommended a solution to the hiring problem: they deliberately asked applicants to use an AI tool which they knew would produce a certain bug in its code. They were then able to check whether the applicants had detected the bug, and therefore whether they were capable of testing and fixing code without the use of AI.

Furthermore, as one participant noted from their own experiences, many developers will decline to work for a company that denies them the opportunity to use AI tools to enhance and accelerate their work.

Encouraging adoption and keeping pace

One recommendation to encourage those more hesitant about AI, alongside formal training, was to hold informal “brown bag” sessions. The participants from Damilah talked about their own experiences of this – holding the sessions once a fortnight, and inviting anyone from their own company or their client base to share their experiences and knowledge of AI tools.

The importance of keeping on top of rapid changes in the quality of AI tools also emerged. Several participants pointed out that some, which didn’t perform well a few months ago, are now proving to be highly valuable – software-testing tool CodeRabbit being a good example of this.

Human accountability and measurement challenges

Participants also discussed optimal approaches to handling bugs in AI-generated code – as, they generally agreed, no AI tool is capable of producing perfect code.

In particular, “code bloat” was highlighted as a major issue. In other words, as AI operates so rapidly, it’s easy to generate huge amounts of code – and inevitably the more code that exists, the more bugs there will be.

Because of this, there was a consensus that:

• It’s essential to ensure human involvement at every step of the development process.
• “People must remain accountable for what they produce,” said one attendee.
• Code reviews – both by humans and tools such as CodeRabbit – are particularly important.

Measuring success

The roundtable also discussed the challenges related to measuring the gains achieved by AI. The key points raised were:

• Software development, by its very nature, involves performing a different task for each new project.
• Therefore, the only way to effectively measure productivity gains is to have two teams performing the same tasks in parallel – one with the use of AI tools, and the other without, to act as a control group.
• AI tools, used in the right ways, can clearly unlock major productivity gains – but many businesses do not have the time or resources to conduct such controlled experiments to measure the gains with any accuracy.
• In light of this, the tests conducted by Damilah have proved invaluable.

Sharing knowledge and expertise

Our roundtable participants agreed that keeping up with the lightning pace of change in AI is almost impossible. It therefore requires individuals and companies to work together to share knowledge and experiences for the advantage of all.