The research highlights two primary challenges: the lack of training data that accurately reflects the decision-making processes needed in real debugging scenarios and the inability of AI models to fully utilize debugging tools.

Testing demonstrated that even a basic AI agent utilizing existing language models was unable to solve more than half of the debugging tasks in benchmarks, indicating a significant gap in proficiency compared to human engineers.

These findings align with earlier studies that show while AI can generate code for specific tasks, it often introduces bugs and security vulnerabilities, reinforcing its role as an assistant rather than a replacement for human developers.

Despite the narrative that generative AI could replace human developers, the study indicates that these tools primarily enhance productivity by suggesting examples rather than actively debugging.

A recent study from Microsoft Research reveals that current AI coding tools, including GitHub Copilot, struggle with effectively debugging code, underscoring the limitations of AI in software engineering.

Researchers propose that with targeted training approaches, such as developing specialized datasets focused on debugging, AI models could improve their debugging capabilities over time.

To address these challenges, the study introduces 'debug-gym', a novel environment designed to enhance AI debugging skills by allowing models to interact with real-world codebases and utilize tools akin to those used by developers.