⚡Megatron LLM Hub
←返回搜索
Twitter

称 OpenAI 正高价收购小型 HFT 公司,以获取高质量手写代码训练数据

模型训练数据来源转向专业代码,直接影响代码审计与漏洞发现能力边界
llm训练数据
OpenAI is reportedly offering huge money to buy out smaller HFT firms and companies like [redacted], because they realize that they really do need more data that is handcrafted, high quality, high performance code in order to improve coding performance of the models. GitHub slop, vibeslop synthetic self-licking-ice cream-cone data and even what many AI-bros wrongly thought of as high quality open source code (like the Linux kernel ~lol~) is not improving the quality of the code that their models are producing. The reward-hacking behavior towards “just make the code compile to earn your reward” has been proven to be extremely counter-productive. Current SOTA models struggle to maintain coherent awareness of critical performance concepts like cache locality, branch prediction, instruction-level parallelism, memory ordering, false sharing, lock contention, vectorization, NUMA topology and the actual cost of moving data through a modern memory hierarchy. People will tell you that this stuff doesn’t matter, because modern compilers are so good. And in a normal world, I would concur with this sentiment. Normally, it would be true. In today’s world, you don’t need to really panic about a few too many if statements, because compilers will optimize away the cost. But these models, some how, some way, have an uncanny ability to write code that is SO bad, that not even the compilers can fix it. Their understanding of I/O performance is arguably even worse. They routinely generate code that is almost completely oblivious to storage hardware: pathological write amplification, excessive fsync() calls, unnecessary serialization, unbounded logging, tiny synchronous writes, redundant disk operations and utterly incompetent batching and buffering strategies. Under sustained high-performance workloads, these pathologies can go well beyond merely degrading throughput. They can generate enough unnecessary physical I/O to accelerate SSD wear, exhaust write endurance, and contribute to premature drive failure. I’ve recently experienced hardware failure myself when I stupidly trusted GPT-6 to write and run a high throughput data service. Which leads one to wonder… with trillions of tokens’ worth of newly vibecoded, I/O-oblivious software now finding its way into production environments, including the major hyperscale clouds, has anyone actually examined whether this phenomenon is beginning to materially affect storage-device failure rates? And this is to say nothing of their performance at higher levels of abstraction. The conventional wisdom is that these models are much better at Python, JavaScript and the various higher-level frameworks than they are at systems programming. People who spend most of their time in that world will even confidently proclaim that “coding is solved.” But consider the extraordinary irony that OpenAI’s own ChatGPT/Codex desktop applications, built around JavaScript frameworks, have been the subject of repeated, frequent, catastrophic resource-management failures. Most commonly, runaway memory consumption reaching tens of gigabytes, psychotic background I/O loops, and SQLite logging behavior that has generated tens or even hundreds of gigabytes of disk writes during ordinary application use. And again - this is from the application written with the benefit of OpenAI’s own internal models.