GitHub and Microsoft's AI bug scanner produced 99 verified reports across six projects
GitHub and Microsoft's AI pilot cut 9,791 raw alerts to 99 verified reports across six open source projects, with 55 flaws already fixed — but each project needed two days of human review.

Why it matters
- GitHub and Microsoft's pilot produced 99 verified private security reports across six open source projects, with 55 fixed and 14 already released.
- The system flagged 9,791 possible issues; AI triage and manual review by seven Microsoft FORGE Lab researchers filtered them down.
- Caddy maintainers fixed all 22 reports they received within a week.
- GitHub is extending AI security scans to all 216 alumni projects of its Secure Open Source Fund, whose software is downloaded over 20 billion times a month.
- Each fund project receives USD $10,000 via GitHub Sponsors; more than 300 maintainers have fixed more than 4,000 vulnerabilities.
GitHub and Microsoft have published early results from a pilot that used AI to find and help fix security flaws in open source software: 99 verified private reports across six widely used projects, with 55 already fixed and 14 shipped in a release when the figures were published.
The numbers behind that headline matter more than they first appear. The system flagged 9,791 possible issues in total. Automated filtering, AI triage and manual review by Microsoft researchers cut that pile down to the 99 reports maintainers actually received. The six pilot projects were AutoGPT, Caddy, Express, FastMCP, ImageMagick and Netty.
The pilot, disclosed on November 18, 2026, combined Codename MDASH — Microsoft's multi-model scanning system — with an AI triage process from GitHub Security Lab and hands-on checks by Microsoft researchers. The stated goal was simple: reduce the number of raw alerts hitting maintainers and replace them with private reports containing proof-of-concept material and suggested fixes.
Why it matters: open source maintainers, many of them volunteers or small teams, are drowning in AI-generated vulnerability submissions — duplicates and non-issues included. AI has sharply lowered the cost of finding potential flaws, but not the cost of validating and fixing them. This pilot is one of the first serious attempts to close that gap.
How did the pipeline actually work?
The process ran in three stages. First, Codename MDASH scanned each code base using Microsoft's MAI Cyber model and proposed fixes. Then a triage agent built by GitHub Security Lab removed duplicate findings, tested their validity and narrowed the list to higher-impact issues. Finally, researchers from Microsoft FORGE Lab manually checked what remained.
Only then did reports reach maintainers, via GitHub's private vulnerability reporting system. Every report included a suggested fix, and maintainers kept full control over whether and when to patch.
The design followed two stated principles: send only findings that matter, and never send unverified reports to maintainers. That's a deliberate reversal of the current dynamic, where scanners spray maintainers with unfiltered output and humans absorb the cost.
The speed was notable too. The first fixes shipped roughly a week after the initial scans. Caddy maintainers fixed all 22 reports they received within a week or less.
Takeaway: the innovation isn't finding bugs — it's the filtering layer that turns thousands of raw alerts into a few dozen actionable reports.
What did maintainers say?
Maintainer feedback, quoted by the companies, was positive. ImageMagick — described as maintained by a two-person team — received 22 reports from the pilot, and one maintainer said the format cut assessment and fix time significantly.
"Normally, a security advisory would take us around 30 minutes to 1 hour on average to read, understand, and make a fix. It is a huge difference. Codename MDASH was able to find unknown, obscure security bugs in our project with succinct reports, and it filled in gaps from some prior reports that hadn't been noticed," said Grobmeier.
A second assessment came from FastMCP. "The Codename MDASH scan provided unique insights into the security of our project above and beyond what AI scanners have provided us in the past," said Grobmeier.
A third comment targeted the volume problem directly. "Codename MDASH is very useful for cutting through the noise of vulnerability reports," said Grobmeier.
Takeaway: the value for maintainers came as much from report quality as from bug discovery.
Can this scale?
Here's the pilot's central tension. Seven Microsoft FORGE Lab researchers spent about two days per project checking findings by hand. That level of scrutiny may be difficult to expand broadly without additional resources — and it's one of the project's main lessons.
The pilot's own results make the point starkly: 9,791 raw findings became 99 reports only because humans stood in the middle of the pipeline. While AI can generate large numbers of possible findings cheaply, open source security still relies on careful human validation and on maintainers' own judgment about how to patch their code.
Takeaway: the bottleneck has shifted from finding bugs to verifying them, and the pilot solves that bottleneck with labor that doesn't currently scale.
What happens next?
GitHub is now extending AI security scans to all alumni projects in its Secure Open Source Fund, with findings reviewed before they're reported privately. The fund has supported 216 projects so far, and the participating repositories account for software downloaded more than 20 billion times a month.
Each project in the fund receives USD $10,000 through GitHub Sponsors, alongside security training. More than 300 maintainers in the programme have fixed more than 4,000 vulnerabilities. Broader programme data also shows more than 850 secret leaks prevented or resolved and more than 1,500 vulnerable dependencies updated.
The six MDASH pilot projects were all alumni of that fund. GitHub said the alumni support package will now also include AI security scans and access to Copilot Max.
The stakes reach well beyond these projects themselves. Flaws in open source software propagate into commercial software supply chains far downstream of the original code. When maintainers publish a security advisory and it enters the GitHub Advisory Database, Dependabot can alert downstream repositories that depend on the affected package — meaning every verified fix here ripples outward to thousands of dependent projects.
GitHub framed the ongoing commitment in maintainer-first terms. "Maintainers do the fixing. Our job is to make sure they never do it alone," said Wolf.
The open question is whether the human review layer — the part that made these 99 reports trustworthy — can grow at anything close to the pace of AI-generated findings. If it can't, the industry's alert-noise problem gets worse, not better.
Source: itbrief.com.au
More from Sophie Lindqvist
Show full bio
Staff writer covering regulation, policy and their impact on artificial intelligence and machine learning at AI In Context.
749 articles
Related articles
- GitHub and Microsoft Pilot Turns 9,791 AI Findings Into 55 Fixes
- Four Labs, One Blueprint: How AI Security Agents Converged in 2026
- OpenAI's Daybreak Launches Patch the Planet for Open-Source Security
- OpenAI Launches Aardvark, an Agentic AI Security Researcher
- OpenAI's Codex Security Enters Research Preview