基准分提升不等于能力提升,做 LLM 安全评测时需警惕分数陷阱
llmai_agent报告
RT @pdiscoveryio: We let GPT-6 Astra directly edit Qwen3.5-9B's weights with no fine-tuning or gradient descent.
SecEval: 32.5% → 51.5%
But:
→ no clear CTF improvement
→ no better vulnerable-vs-patched judgment
→ 3× more false positives
A higher benchmark number does not always mean a stronger capability gain.