⚡Megatron LLM Hub
←Back to search
Twitter

提议形式化证明:VM 内的 AI Agent 无论执行什么代码都无法逃逸到宿主

自托管 Agent 沙箱的隔离假设,正被形式化验证重新审视。
ai_agent逃逸议题
hear me out, mathematicians entering ai alignment maybe consider this challenge. i think its probably millennium level problem(?) take any complex real world software, like a browser or a kernel. let's pick a kvm based sandbox, which i suspect is somewhat less complex than chromium(?), running an increasingly capable, potentially adversarial AI agent. we want to formally prove one simple property, no matter what code the agent executes inside the VM, it cannot escape and compromise the host. for that we need to prove the entire relevant software stack, kvm, cpu virtualization, kernel, virutal devices that could affect the isolation. (to make things easiers, lets assume hardware wont have weird states) can we mathematically establish whether formally verifying a system of this complexity is even computationally feasible?