⚡Megatron LLM Hub
←返回搜索
Twitter

讨论小模型表现不佳有多少源于规模、多少源于缺少精细权重优化。

llm模型
This makes me wonder: How much of the underperformance of small models has to due with them being small vs. the fact that the researchers making them got slightly less crappy performance than the last small model, so they called it good enough and put it out How much more hidden performance might small models have hidden behind surgical weight optimizations like this?