说明 agent 挖洞的盲区有规律,人机配合仍是差异点。
ai_agent漏洞挖掘议题
It was great to speak with so many of the smartest vulnerability researchers in the world this week at OffensiveCon Tokyo.
One key takeaway from speaking to colleagues and contemporaries in the government contracting side of the industry: The AI agent pipelines, no matter how they’re laid out, which models one uses, or how much they spend on tokens in the process, are all converging on the same bugs. But in A/B tests, where we have some of our best human researchers (*I mean dudes that make and have made for years $4M/year on account of how many A-grade exploits they surface) pull apart the latest versions of iOS/Android/Chrome or whatever, in the way that they used to do before AI, these hugely talented human researchers find bugs that are utterly unique, and far higher quality, and that none of the models are finding. Even when those models/agents are armed with the same data and tools (decompilers, debuggers, etc).
Also, the human researcher only gets paid the lion’s share of their compensation at such a point as you are sure that you are able to actually generate revenue from their discoveries. The precise details of this split between salary and bonus can vary, of course.
Whereas with the AI agents, it’s very easy to find one’s self having spent hundreds of thousands of dollars on tokens, only to have those tokens generate the same bugs that the same model (or other models distilled from that model) already generated for like 15 other people around the world; half of whom have no access to responsible government buyers for said discoveries and so will thus be reporting back to the vendor in exchange for a $5K bounty.