⚡Megatron LLM Hub
←Back to search
Twitter

测算:本地跑前沿模型需约 65 万美元机架与每月 2500 美元电费。

本地部署llm成本
There is a lot of talk about Mac Studio, DGX Sparks, etc. These are great rigs for enthusiasts. I’ve used and do use them. But we shouldn’t forget that if you want to run serious frontier workloads on-prem, with models like Kimi K3.1, you’re going to be looking at spending a minimum of about $650K for a competent rack, and adding about $2,500/mo to your power bill (depending on where you’re located) If you are dealing with extremely sensitive matters, this makes sense. For some, this sounds like an astronomical cost. For others, it’s garden variety capex that will be written off against profits.