Regnology Automates Ticket-to-Code with agentic GenAI on Vertex AI

A significant challenge in software development  faced by many tech driven companies with internal software development teams, is the "Ticket-to-Code Problem”.This issue arises when-reported bugs or requirements are logged as tickets, which then need to be transformed into actionable  code changes. Regnology, a global technology pioneer in regulatory reporting solutions, is no exception. 

The process for ticket to code involves understanding the problem, analyzing the user story, finding the relevant parts of the code that should be modified, and implementing the necessary code changes. This workflow can be time-consuming and prone to errors, particularly in large organizations with complex codebases. According to McKinsey, inefficient software development processes can lead to a significant loss in productivity, with some studies suggesting that up to 50% of developers' time can be wasted on inefficient workflows.

"Our development teams spend a significant amount of time converting customer-reported issues into actionable code changes,"says Antoine Moreau, CIO of Regnology. "The Ticket-to-Code Problem is a critical bottleneck that impacts productivity and efficiency. Using Gen AI, we aim to streamline this process, reducing errors and saving valuable development time."

To tackle this challenge, Regnology developed an innovative solution known as the Ticket-to-Code Writer. This tool automates the conversion of bug tickets into actionable code, significantly streamlining the software development process. Recognizing the potential to further enhance this solution, Regnology worked with Google, leveraging both Google's Vertex AI and their AI expertise. This collaboration scaled up the tool's capabilities, integrating advanced AI technologies to refine and expand its functionality, making it an even more powerful resource for developers, and bringing 60% in time efficiencies for them.

Tackling the Ticket-to-Code Problem with Vertex AI

Ticket-to-Code Writer is an AI-powered tool running in production, designed to streamline the transition from bug tickets to code implementation. This solution is part of Regnology's broader strategy to integrate cutting-edge AI and cloud technologies into its offerings, enhancing the productivity and accuracy of its software development processes.

The Technical Solution: A Deep Dive into the Ticket-to-Code Writer

The Ticket-to-Code Writer features an architecture that incorporates a mixture of AI agents, each specializing in different stages of the ticket-to-code development process. This setup uses the concept of similarity search and other techniques to ensure accuracy and reduce risks like hallucinations, which can occur with large language models (LLMs). Here's how it works:

1. Ticket Parsing & Summarization [1]: The process starts with parsing the user story or bug ticket, extracting key information, including keywords and core requirements, to understand the necessary code changes.
2. Code Search & Localization [3]: This module takes center stage in identifying specific code segments needing modification. Using Google's Vertex AI search, it performs a vector similarity search, leveraging Approximate Nearest Neighbor (ANN) algorithms to scan through potentially thousands of files. This module accesses the latest changes in the repository, which are then chunked, embedded, and stored in GCP buckets, ensuring the system is continuously updated with the most recent code changes.
3. Modification Plan Generation [4]: Once the relevant files are identified, the Modification Plan Generator, powered by the Gemini 1.5 Pro LLM, steps in. This AI agent crafts a detailed technical plan, breaking down the required changes into manageable subtasks. It specifies what code needs to be added, modified, or removed, complete with code snippets and explanations. It also decides if some files fetched by the Code Analysis Module require modifications or not. Users can interact with this step, requesting modifications in simple text, making the process accessible even to those less familiar with technical jargon.
4. Automated Code Generation & Pull Request Creation [5-8]: Upon approval of the plan, this module, leveraging the Gemini 1.5 Pro LLM, automatically generates the necessary code changes and creates a pull request.
   One challenge encountered during this phase involves handling less common programming languages, which the LLM may not be initially trained to manage effectively. To address this, Dynamic Few-Shot Prompt Composition [5] was created. This process involves providing examples of previous tickets and their corresponding code implementations. These examples are stored in GCP buckets, and Vertex AI similarity search is again utilized to locate related similar tickets, then crafting the prompt for the new desired task. This tailored approach enables the LLM to generate accurate code, even in unfamiliar programming languages, by leveraging past examples to inform its output.

The solution includes a post-processing and validation step after each module, ensuring that the output meets required quality and format standards. This process leverages the Vertex AI Gen AI Evaluation Service to systematically evaluate and select the best responses, providing associated quality metrics and explanations. One of the strengths of this architecture is that each module can be separately benchmarked and tested, allowing for the identification of bottlenecks in the overall performance and accuracy.

Resulting  Impact and Next Steps

The Ticket-to-Code Writer offers several significant benefits:

• Increased Efficiency: By automating the transition from user stories to code changes, the tool reduces the time required to resolve bugs and implement new features by approximately 60%, crucial for large projects and teams with extensive codebases.

• Enhanced Onboarding for New Developers: The tool provides a structured approach to understanding and navigating complex codebases, significantly reducing the learning curve for new team members.

• Improved Consistency and Accuracy: The AI-driven approach ensures that code changes align with user story requirements, reducing the likelihood of errors and enhancing overall software quality.

• Better Collaboration and Communication: By providing clear documentation and an automated workflow, the solution facilitates better communication between developers, business analysts, and other stakeholders, ensuring alignment with project goals.

As Regnology and Google continue to refine and expand the Ticket-to-Code Writer's capabilities, they are exploring additional features and enhancements. The goal is to enable individuals to chat with their code, facilitating interactions such as automated reviewing of new code, applying code changes from tickets, and potentially even creating new files when needed.

Regnology plans to work on generating documentation from code and developing a code explainer that can answer questions using code, such as understanding how a particular field value in a table is being generated. This approach aims to create an even more efficient, accurate, and user-friendly solution capable of addressing the ever-evolving challenges of software development.