Improving code security with AI-based vulnerability scanning

Project overview

• Project goal: Development of an AI-based vulnerability scanner to automatically detect security vulnerabilities across more than 60,000 code repositories

• Team: 10 TNG consultants

• Project start: March 2026

• Key achievements: 60,000+ repositories scanned, up to four times more potential vulnerabilities identified compared to static analysis tools.

The situation

One of our clients had long recognized that powerful LLMs are transforming the search for vulnerabilities in code, creating new opportunities for attacks as well as for proactive IT security.

Traditional static analysis tools, such as CodeQL, identify vulnerabilities based on predefined patterns but reach their limits when it comes to more complex security issues. At the same time, the sheer number of more than 60,000 code repositories makes comprehensive manual reviews practically impossible. Our client therefore wanted to use AI to automatically identify potential vulnerabilities across its entire codebase.

Our target

Our goal was to develop an AI-based vulnerability scanner able to identify security vulnerabilities that are difficult or impossible to detect with traditional deterministic analysis tools.

The solution also needed to integrate seamlessly into the client's existing GitHub infrastructure and established workflows. Another key requirement was cost control: With tens of thousands of repositories, the AI-based analysis needed to deliver high-quality results while remaining economically scalable.

Our approach

Our ten-person TNG team initially developed a simple version of the scanner and integrated the scanning logic into the client's existing GitHub infrastructure. Using GitHub Actions, repositories were automatically made available for scanning and then analyzed. Identified vulnerabilities were then uploaded to GitHub Advanced Security.

The first version was based on a simple agentic approach. This revealed a key limitation: While a single AI agent can identify relevant vulnerabilities, it does not examine large and complex repositories systematically enough.

We therefore further developed the scanner into a structured multi-agent system. The scanning process includes:

• dividing large repositories into smaller units,

• using specialized AI agents to identify relevant files and potential vulnerabilities,

• validating the findings and removing duplicates, and

• filtering out potential false positives.

Scaling the solution to more than 60,000 repositories presented a particular challenge. We therefore evaluated and compared different models and technical frameworks to optimize costs while maintaining the same level of quality.

The result

By now, we have analyzed nearly all of the code on the client's GitHub instance with our AI-based scanner. One particular advantage is its semantic analysis: The scanner can identify more complex and less obvious vulnerabilities that traditional pattern-based analysis tools do not detect. Overall, this enabled us to recognize up to four times as many potential vulnerabilities.

The identified vulnerabilities are displayed directly alongside the relevant code in GitHub. In addition, we developed reporting solutions in business intelligence tools that provide stakeholders with an overall view of the scan results. Development teams have already started addressing these vulnerabilities, and their feedback so far has particularly highlighted the quality of the results.

We also significantly improved the solution's cost efficiency through continuous benchmarking of different models and frameworks. By using OpenCode instead of Claude Code, among other measures, we were able to reduce token consumption by up to 80 percent compared to the original estimation, without having to compromise on quality.

Currently, we continue to develop the scanner to further improve the quality of the analysis and identify even deeper vulnerabilities while further reducing the cost of large-scale scans.