⚡Megatron LLM Hub
←Back to search
Twitter

讨论:越狱/未对齐的前沿模型同样能完成奖励黑客式攻击

关系到自托管开源模型的攻击面与能力评估前提。
llmai_agent议题
> LLMs are useful tools in the hands of skilled professionals, but for others, they're often just a Dunning-Kruger accelerator. i don’t think this holds when you are dealing with models that are as capable as abliterated frontier red models. if frontier models could pull off something like the hugging face incident through reward hacking, why assume an abliterated model couldn’t achieve the same level of capability when explicitly prompted by someone with little security expertise?