Inferstep

23 october 2026 · mission · inferstep lab

Why we are rethinking how intelligence scales

The most capable AI is built with more computing power and more money every year, and a handful of companies hold most of it. Is that the only road? Inferstep is an open lab rethinking how intelligence scales, and we are working toward AI that runs entirely on a device you own.

Two stacks of wheat in a field at the end of a summer day, with a village and hills behind them.
Monet, Stacks of Wheat (End of Summer) (1890–91), Art Institute of Chicago. Public domain.

We founded Inferstep as an open lab to find out whether capable AI has to live in a few very large data centres. We think it does not, and we are working toward one alternative: AI that runs entirely on a device you own, and is good enough for real work.

the problem

Since 2020 the computing power used to train the leading language models has grown about five times a year, by Epoch AI's count. The capital spending of Google's parent Alphabet, Amazon, Meta, Microsoft and Oracle was, in Epoch's words, "nearing half a trillion dollars" in 2025. Those five companies held about 71% of the world's AI computing power at the end of 2025, up from 63% at the start of 2024. Stanford's AI Index counts 93 of the notable AI models of 2025 from companies, and two from universities.

Data centres used about 1.5% of the world's electricity in 2025, yet as much as 20 to 30% of it in the regions where they cluster, the IEA notes. In Ireland they took 23% of all metered electricity. The demand now reaches the devices people buy: in the third quarter of 2026, smartphone shipments fell by 22% in what IDC calls an "AI-led memory crisis", with memory costs up about 350% in a year.

Scale has earned its place: in 2020 researchers at OpenAI showed that a language model's error falls predictably as its size, its data and its computing power grow. A large data centre also shares its machines across many users and answers many requests at once, which makes each answer cheaper. The assumption we question is this one: that the next gains in AI, and the AI people use every day, have to live in a few very large data centres.

the question

Ilya Sutskever, a co-founder of OpenAI, said in November 2025 that a hundred times more scale "would be different, for sure", then asked: "But is the belief that if you just 100x the scale, everything would be transformed? I don't think that's true." He still expects research to need "big computers". In a 2025 survey of 475 people by the Association for the Advancement of Artificial Intelligence, 76% of respondents judged it unlikely or very unlikely that scaling up current approaches will produce general intelligence. They chose to take part, and most were academics.

what we see

Capability that once needed a very large model now arrives in much smaller ones. In 2022 the smallest model to score above 60% on MMLU, a test of knowledge across 57 subjects, was Google's PaLM, with 540 billion parameters. Two years later Microsoft's Phi-3-mini passed the same mark with 3.8 billion, 142 times fewer. A model can be tuned to a single public test, so treat the size of that change with care.

PaLM, 2022 540 billion parameters Phi-3-mini, 2024 3.8 billion
The smallest model to score above 60% on MMLU, in 2022 and in 2024, drawn with areas in proportion to their parameters. Source: Stanford AI Index 2025.

Epoch AI found that the best open models that run on one high-end gaming GPU match the best models of six to twelve months earlier. It adds that small open models are more likely to be tuned for tests, so the real lag "may be somewhat longer". Apple runs a model of about three billion parameters on its devices, and in 2026 it introduced one of 20 billion for its strongest machines, while its most demanding work still goes to data centres. We found no measure of how much computing power all homes hold together, so our case rests on what one device can do.

our aim

We are working toward AI that runs entirely on a device you own: a laptop or a desktop first, and a phone as the models allow. Your words and files stay with you, and no service can change its price or switch it off. We also think a small model can do more than its size suggests when it has room to draft several answers, test them and choose. ATLAS, our coding agent, is built to measure that.

The models we build on were trained by large companies in large data centres. Running them yourself changes who holds your data and what each answer costs you. Who trains the most capable models is a separate problem, and pooled training, below, is one attempt at it.

what it costs

Privacy is the clearest gain: nothing has to leave the device for the model to answer, though a model that looks facts up on the web sends those searches out. The price is the hardware: a capable device is expensive, more so since the memory crisis, and a data centre spreads its machines across many users at once.

On energy the evidence runs against us: for the same model, an answer on your own device takes more energy than one from a busy data centre. A study by Stanford University and Together AI found that a data-centre GPU gave 1.6 to 2.3 times as many correct answers per unit of energy as a laptop chip, and serving 64 requests at once made the data centre 11 to 20 times more efficient again. A 2026 preprint found answers on a phone about three times less energy-efficient than a busy server, and, under its assumptions, five to seven times worse for the climate per token once the making of the phone is counted. Smaller models save energy wherever they run, and the Stanford study gives that as the reason for its savings: "The dominant effect is the reduction in model size, not a hardware efficiency advantage." We will measure the energy of whole tasks on a device you own, compare it with published figures for hosted models, and publish the results.

Open models trail the best closed ones by about four months on Epoch AI's measure, and anyone who downloads one can misuse it; once released, it cannot be withdrawn. ATLAS runs open models that their makers chose to publish, under their licences and use policies. Small models also know less, and they fall well behind on multi-step tasks with tools: on the Tau2 test, Google's phone-sized Gemma 4 E4B scores 42.2%, and its 31-billion-parameter sibling 76.9%. That gap matters most to our aim, and it is the one we work on.

others on these roads

Spreading AI out can mean different things:

Pooled training has moved fastest: in November 2024 Prime Intellect released INTELLECT-1, a model of 10 billion parameters trained across five countries on up to 112 data-centre GPUs, and Nous Research and Pluralis Research work on the same problem. For pooled running, Petals showed from 2022 to 2024 that large models can run across volunteers' machines, and exo runs one model across the devices in a home. We want to work with groups like these. Every one of these roads gains when models become smaller and cheaper to run.

how we start

ATLAS is a coding agent that runs on your own GPU, with the model you choose. For new code files and larger edits, it drafts one or more versions and tests them in a sandbox before it writes the file. Code can be checked, so we can measure whether a model on a device you own, given room to try and to check its work, does a job you can rely on. Release 3.1.6 is open source under the AGPL-3.0 licence, and its known problems are listed on the ATLAS page. Our notes on best-of-K sampling, test-time scaling and what a passing test proves explain the ideas and their limits. We will publish results when the measurements are finished.

what could prove us wrong

If our measurements show that models able to run on a device you own cannot do real work well, we will say so and change course.

how you can help

Yann LeCun, then Meta's chief AI scientist, said in 2025 of the next advances in AI: "It's not going to come from a single entity. It's going to come from the entire research community across the world."

sources

build it with us

ATLAS is open source. Start with an issue marked for newcomers, or pass this page on.

start contributing github ↗