Training an AI model from scratch has always been the privilege of the very large. The compute bills are staggering. Training GPT-4 reportedly cost well over $100 million. Even a well-funded startup building a 10-billion-parameter model today is looking at thousands of hours on the most expensive chips available, with rental rates for cutting-edge GPUs running around $5 per hour or more. The financial barrier has kept nearly all serious model-building inside a handful of large tech companies.
Nous Research, a small open-source AI lab formally incorporated in 2022 and backed by roughly $70 million in total funding, has published a method that takes a direct shot at that barrier. Their technique, Token Superposition Training, reduces the time needed to train a model by up to 2.5 times without requiring any new hardware, any changes to the model's design, or any changes to the data.
The idea is simple once explained. Normally, an AI model trains by looking at one word at a time and guessing what comes next. This method instead groups several words together during the first portion of training and feeds them in as a single blended unit. The model absorbs more text per hour of work. Once it has learned a good general understanding of language this way, training switches back to the normal word-by-word approach for the remaining portion. The finished model is identical in every way to one trained traditionally.
The results at scale are concrete. For a 10-billion-parameter model, the standard approach consumed roughly 12,300 GPU-hours. The new method reached a better quality result in about 4,800 GPU-hours. At current market rates for top-tier GPU rentals, which sit around $5 per GPU per hour, that difference is roughly $37 million on a single training run. For smaller models in the 3-billion-parameter range, the saving was around 1.8 times in total compute, which still translates to tens of thousands of dollars per run.
Why does this matter to people outside of AI labs? The answer is that pre-training is the step that creates a general-purpose model, and that general-purpose model is what everything else gets built on top of. Industries with large proprietary datasets, such as insurance, legal, healthcare, and logistics, have strong incentives to train models on their own data from the start rather than just tweaking a general model made by someone else. The problem is that the cost of doing this at any meaningful scale has been prohibitive. A method that cuts that cost by more than half changes the calculation for a meaningful number of organisations.
Nous Research is not a household name, but they are not insignificant either. They started as a volunteer collective on Discord in 2022, formalised into a company the following year, and have since built a line of open-source models used widely across the AI community. Their $50 million Series A was led by Paradigm, a well-known crypto and technology venture firm, and they have consistently published work at the frontier of open-source AI research.
The broader context here is that the AI field is actively hunting for efficiency gains because raw scaling, simply throwing more chips at the problem, is hitting diminishing returns and escalating costs. Research institutions from MIT to major labs have been publishing efficiency techniques covering everything from model compression to better training objectives. Token Superposition Training fits into this wave, but it stands out because it requires almost no changes to existing workflows. It is a modification to the training loop itself, not a redesign of the model or the infrastructure around it.
This is also being released as open research, not as a commercial product. The full method is documented in an academic paper and the approach is simple enough that any team already running model training could adopt it. That is by design. Nous Research's entire identity is built around making serious AI capabilities available outside of the biggest tech companies. The firms that will benefit most are not OpenAI or Google, who already have the resources to burn on long training runs. The firms that benefit are the smaller AI labs, national research institutions, and eventually the companies in traditional industries who are beginning to consider building their own models rather than renting capability from a third party.