关系到自托管开源模型的攻击面与能力评估前提。
llmai_agent议题
> LLMs are useful tools in the hands of skilled professionals, but for others, they're often just a Dunning-Kruger accelerator.
i don’t think this holds when you are dealing with models that are as capable as abliterated frontier red models. if frontier models could pull off something like the hugging face incident through reward hacking,
why assume an abliterated model couldn’t achieve the same level of capability when explicitly prompted by someone with little security expertise?