Vulnerability intelligence company VulnCheck recently released its third tracking of Anthropic’s security program, Project Glasswing.analyze. According to figures from Anthropic's own public ledger as of November 2, Glasswing reported a total of 37,966 discoveries in the seven months since it was launched. Only 7,349, or about 19.4%, were made public, and only 217 were marked as fixed.

When Anthropic launched Glasswing on April 7, it said it had found "thousands of high-risk and severity vulnerabilities" and was responsibly exposing them to open source project maintainers. VulnCheck member Patrick Garrity, who wrote the article, began building an open source database in mid-April to track vulnerabilities that can be attributed to Anthropic.

Nearly 12,000 new discoveries were made in two months, and only 15 more were fixed.

VulnCheck compares the ledgers from September 8 to November 2: total discoveries increased from 26,153 to 37,966, which is 11,813 more; fixed increased from 202 to 217, which is only 15 more; withdrawn increased from 245 to 419, and those marked as duplicates increased from 2 to 1,670. During the same period, the ledger listed the true positive rate as rising from 91.4% to 94.2%.

Garrity believes that while there are more duplications and retractions than fixes, the true positive rate is still rising, indicating that the results may not be reflected in the way the ledger is classified. He also tracked 442 CVE vulnerability numbers that can be attributed to Anthropic, accounting for about 0.5% of all 93,450 CVEs this year, only 2 of which are listed as being actually exploited.

More than half of the reports received by curl were judged not to be vulnerabilities

VulnCheck also cited the open source download tool curl. Curl maintainer Daniel Stenberg recently said that the project received a batch of 45 vulnerability reports from Anthropic, and Anthropic self-assessed that one of them was a medium risk and the rest were low risk. Of the 34 disclosed as of the time of the analysis, 19 have been judged not to be vulnerabilities, 6 have been confirmed as low risk, and 9 are still under evaluation.

Garrity concluded that AI is indeed a useful tool for finding vulnerabilities, but the seven-month figure does not match what Glasswing said when it was launched, and the verification work is also passed on to the open source maintainers.

Anthropic has previously acknowledged that human verification is a bottleneck

When Anthropic launched the open source project self-service scanning tool OSS Scanner on October 8,Official articleIt stated that more than 29,000 suspected vulnerabilities had been identified at that time, but only about 6,000 had been manually reviewed, admitting that it was limited by "human verification capabilities"; it had also sent out nearly 5,000 unverified reports in batches at the request of some maintainers.

Anthropic also stated that OSS Scanner's reports are "entirely generated by models and have not been manually reviewed or classified." They may contain errors or invalid content and are left to the discretion of the project maintainers. Chain News previously reported on this plan, see Anthropic uses the strongest model to scan vulnerabilities in open source projects for free. As of press time, Anthropic has not publicly responded to VulnCheck’s analysis.

this article Anthropic vulnerability project discovered 38,000 vulnerabilities in seven months, only 217 were fixed first appeared in Chain News ABMedia。