本地部署llm成本
There is a lot of talk about Mac Studio, DGX Sparks, etc. These are great rigs for enthusiasts. I’ve used and do use them.
But we shouldn’t forget that if you want to run serious frontier workloads on-prem, with models like Kimi K3.1, you’re going to be looking at spending a minimum of about $650K for a competent rack, and adding about $2,500/mo to your power bill (depending on where you’re located)
If you are dealing with extremely sensitive matters, this makes sense.
For some, this sounds like an astronomical cost. For others, it’s garden variety capex that will be written off against profits.