⚡Megatron LLM Hub
←返回搜索
Twitter

Kimi K3.1 Pro 在漏洞发现基准上误报率 4.2%,但召回不及 DeepSeek。

模型选型参考:误报低但漏报高,决定跑扫描用哪个模型前先看这组数据。
llmbenchmarkai_agent
RT @pilvar222: We ran Kimi K3.1 Pro on our cybersecurity benchmark, it's super precise (4.2% false positives), but it doesn't find as many vulnerabilities as other models - At pass@3, it found 26/32 CVEs (81.3%), tying Opus 5. Still behind DeepSeek V4.1 Flash 1017 (87.5%) - Only 4.2% of what it reports are false positives (95.8% precision), the 4th lowest rate on the benchmark. DeepSeek is at ~14%, so you get roughly 3x less noise - Cost: $16.14 per CVE found, 8x cheaper than GPT-6 Astra ($135.23), which has a similar false positive rate 1/3 🧵