⚡Megatron LLM Hub
←返回搜索
Twitter

讨论 Mac Studio 大内存机型跑大模型的性价比取舍

本地部署硬件
I need you guys to help me work through this conflict that I'm having about Mac Studios. I would love to wait and order the 512 GB variant of the M5 Ultra Studio and I think what they've built is pretty amazing. The problem, I think, that these larger devices have is that if you're getting 512 GB, you are presumably going to run some of the largest available models. The problem with that is the largest models generally have far more active parameters, which also means it's going to run much slower. Almost unusable imo. I'm not trying to use 30tps. So then the question becomes, is it really worth it to run a model with 40B active params? And if not, then why would you need more capacity? Is the 256GB not the sweet spot with a model like Qwen 3.8 Next? Only 6B active parameters is going to run far better on Apple silicon. I'm not convinced the trade off is worth it. Especially when you factor in concurrency. Thoughts?