How we’re using AI to enhance our software developers’ productivity

And who could blame them? As technologists, we’re all excited about what AI can do for us, and we’re all asking the same question: how can we leverage AI tools to improve productivity and help our teams – and our clients – achieve their objectives?

Fortunately, those developers didn’t stay annoyed for long, as we let them use AI in a second hackathon.

Here’s what we did…

A series of controlled experiments

We wanted to establish some clarity, for ourselves and our clients, regarding the impacts AI can have on software development. We fully appreciate that some companies are just beginning to explore AI-assisted development, while others are already integrating it into their workflows. But wherever you are on this journey, the potential of these tools to deliver real, measurable value is undeniable – and we were keen to understand that in greater detail.

So, we designed a series of controlled experiments. First, we split our engineers into two groups: one using traditional methods (the disgruntled ones); the other using AI tools like GitHub Copilot (the far happier ones). We then gave each team a series of identical tasks to complete, which included developing code and testing it.

Then, in the second hackathon, we expanded the experiment to include a wider range of AI tools, such as Cursor, Codeium and Tabnine, and gave the developers short training sessions on using them effectively.

And the results were striking. While the engineering teams using traditional methods delivered great work in a reasonably good time, the ones using AI completed the tasks far more rapidly – sometimes nearly five times faster – and with a higher degree of quality, finding edge cases more effectively and producing better unit tests.

For quality analysts, the impact was even more pronounced: AI-assisted testing was up to nine times faster and improved the ability to identify edge cases, leading to significantly higher-quality outcomes.

Overall, across all our experiments, we found that the average time taken to complete an entire task was around twice as fast.

What’s more, a team of colleagues conducted a hackathon to test the value of using AI to accelerate and enhance our product discovery and inception processes – with equally astounding results. (find out more here)

Delivering added value for our clients

We are now starting to roll out the usage of AI into live environments with our clients, while leveraging the learning from our experiments.

One of the key benefits we’re finding is that – as well as the significant acceleration in pace – AI removes much of the boring, repetitive or time-consuming work, such as writing unit tests, troubleshooting code and referring to community forums for solutions. This allows our developers to focus on more interesting – and more valuable – activities.

For instance, in a recent live case with one of our clients, two developers struggled for hours to resolve a specific issue. When they finally turned to an AI tool, they fixed it in just 10 minutes, enabling them to move quickly onto a higher-level task.

Another value-driver is that we’re able to share the results of our experiments with our clients. We understand that many of them do not have the time or resources to run hackathons of this kind, due to the day-to-day pressures of their business. But, thanks to our partner-shoring.

approach – where we build fully integrated teams with our clients, all working towards the same goals – the insights we have developed are available for our mutual benefit. This includes not only productivity metrics, but also operational best practices, the ways AI can be integrated into workflows, and identifying the most effective AI tools for specific tasks.

Addressing the risks and challenges of AI

Despite the outstanding success of our experiments, we are also fully aware that there are several risks and challenges that need to be addressed with the adoption of AI.

We believe that security and the protection of intellectual property (IP) should always be a primary concern, and that is no different when it comes to the use of AI. To mitigate this, we’ve implemented strict policies and only use tools with advanced security safeguards. For example, the tools will immediately discard any code snippets or prompts that we use once they have generated a suggestion.

Another important challenge is ensuring that our engineers maintain ownership of their code and there is always a human in the loop at every stage of development. In its present state, AI cannot replace human judgement and expertise. While it nearly always generates great suggestions, it’s still up to our people to validate them, adapt them to our specific needs, and check for errors.

Where next?

As for the future… we’re truly excited. The pace of AI development is astonishing. In just a couple of years, we’ve seen transformational advances – and the next five years promise exponential improvements. As AI tools mature, the opportunities for end-to-end software development based on a series of high-level business requirements are becoming increasingly feasible.

In the meantime, we’re committed to staying ahead of the curve. By continuously exploring new tools and refining our workflows in a constant search for improved quality and efficiency, we’re not just enhancing our own capabilities – and keeping a smile on the face of all our developers – we’re driving valuable innovation that delivers measurable benefits for our clients.

To discuss how our AI-enhanced development – and the experiments we’ve been conducting – can benefit your business, get in touch now.

Aleksandar Karavasilev, CTO at Damilah