模型选型参考:误报低但漏报高,决定跑扫描用哪个模型前先看这组数据。
llmbenchmarkai_agent
RT @pilvar222: We ran Kimi K3.1 Pro on our cybersecurity benchmark, it's super precise (4.2% false positives), but it doesn't find as many vulnerabilities as other models
- At pass@3, it found 26/32 CVEs (81.3%), tying Opus 5. Still behind DeepSeek V4.1 Flash 1017 (87.5%)
- Only 4.2% of what it reports are false positives (95.8% precision), the 4th lowest rate on the benchmark. DeepSeek is at ~14%, so you get roughly 3x less noise
- Cost: $16.14 per CVE found, 8x cheaper than GPT-6 Astra ($135.23), which has a similar false positive rate
1/3 🧵