⚡Megatron LLM Hub
←返回搜索
Twitter

为治幻觉给模型加工具调用、联网、子 agent,攻击面同步扩大

工具调用与子 agent 是 agent 攻击面的来源,不只是能力升级。
llmai_agent报告
If you asked GPT-4 a chemistry problem, it would either give you the correct solution from its memory, or it would “hallucinate” and give you the wrong answer. But it would give you an answer. And it would do so without tool-calls, internet search, etc. Depending on the topic, it would give the wrong answer a high percentage of the time. So naturally, the labs put a premium on giving the models the ability to search the web, access tools like scripts and calculators, spawn sub-agents, etc. The frontier labs all but gave up on trying to get the models to memorize and recite facts and data correctly. Altman & co would explain “LLMs are not databases and they’re really bad at being databases. If you ask it who won the World Series in 1929, you’re using it wrong. It’s a substrate to reason over text.” And so the era of tool calls started. Some companies like Perplexity (currently somehow valued at $20B+) got an early start on this, delivering basic SerpAPI+LLM as a service with some basic caching to speed it up. Now that it’s been 2 years since the foundation labs seemingly gave up on training facts into the weights of the models such that the models can recite them quickly and correctly, it seems that this capability is an incredibly auspicious gap that poses a major inconvenience. Earlier today, I randomly wondered to myself “I wonder if the placebo effect is strong enough such that in studies, people given saline instead of insulin were able to somehow “think” their blood sugar down?” So I asked ChatGPT, and I also texted a friend of mine who is a very accomplished pharmaceutical researcher. My friend texted me back almost instantly and even included a reference, and ChatGPT went searching all over the internet for 10 minutes to give me the same answer. It would be much preferred if the model could just recall facts accurately, the way that you and I and my pharmaceutical researcher friend can.