120B 稀疏模型的本地速度实测,自托管 agent 选型参考
llmai_agent本地部署
RT @Sentdex: Nemotron 3.5 Super VL (120B A12B) is now the leading US open source model. Congrats to the NVIDIA AI team!
I am still doing testing of this model, but I think the ranking would be even stronger if we were just measuring agentic coding performance. This one may very well be a solid choice for a speedy local coding agent model
I'm running many things in parallel, will get proper average and max numbers in time, but just randomly prodding this model for some quick speed checks and over 550 tok/sec. All vanilla settings. Once optimized 600+, maybe even 650+, will be trivial, and so far it does appear to be a token efficient problem solver.
Really exciting to see what looks like a solid release so far from nvidia!