Small Language Models: Why Model Size Matters (for the Climate—and Beyond)

For years, the underlying assumption was simple: the larger and more powerful the model, the better—and the data center footed the bill—whether in terms of energy, cost, or environmental impact. But something is changing, and it’s worth understanding what’s really driving this shift, with the most up-to-date figures in hand.

How much does a single prompt really cost?

Google has published an end-to-end measurement—not an external estimate, but actual production data—of how much it costs to respond to a median text request on Gemini: about 0.24 watt-hours of energy and 0.26 milliliters of water—less than the energy required to keep a television on for a few seconds.

The most interesting figure isn’t the absolute number, but the trend: over twelve months, the energy required for the same type of prompt has dropped by about 33 times, and emissions by 44 times.

How was this possible? Through a combination of techniques that, when used together, are a game-changer: “mixture of experts” architectures that activate only a small fraction of the parameters for each request, numerical compression techniques that do not compromise perceived quality, and custom-designed chips, which are now up to 30 times more efficient than they were just a few years ago.

The paradox no one talks about

Here, however, we need the most up-to-date—and most uncomfortable—stat of all. Google’s 2026 Environmental Report, published on June 30, 2026, shows that the electricity consumed by its data centers rose from 14.4 million MWh in 2020 to 30.8 million in 2024, reaching approximately 42 million MWh in 2025—the largest year-over-year increase in the company’s history (+37%). Total emissions rose by 18% in the last year alone and are now 81% higher than in 2019.

How does this square with the -33x figure mentioned earlier? It’s what economists call the “Jevons paradox”: when a resource becomes more efficient to use, we use more of it, not less. Google’s own report explicitly acknowledges this, stating that the growth of its AI infrastructure is accelerating faster than the power grid can decarbonize. The International Energy Agency, in its April 2026 report, explains this mechanism on a global scale: as simple text-based queries become increasingly affordable, adoption is skyrocketing—active users have tripled, and revenue has quintupled in a year according to major providers—and is shifting toward much more resource-intensive tasks: video generation, multi-step reasoning, and autonomous agents, which can consume hundreds or thousands of times more energy than a simple text prompt.

This is an important clarification for any discussion of “frugal” AI: technical efficiency is a necessary condition, but it alone is not enough to reverse the trajectory of energy consumption. A change in behavior is also needed—using the right model for the right task, rather than the most powerful one available out of habit.

Bigger isn’t (always) better

Mistral AI has published an even more direct study on the relationship between model size and environmental impact, analyzing the entire lifecycle of its flagship model (Large 2): training generates approximately 20,400 metric tons of CO2 equivalent and consumes 281,000 cubic meters of water, while answering a single question produces about 1.14 grams of CO2 and 45 milliliters of water.

A model’s environmental footprint grows almost in direct proportion to its size. In practice: if one model has ten times as many parameters as another, generating the same number of words with that model costs—all else being equal—about ten times as much in terms of CO2 and water consumption.

This isn’t an exact law of physics, but it is the observed trend, and it has a very concrete consequence: using a massive model for tasks that don’t require it—summarizing an email, classifying a text, answering a simple question—isn’t just a philosophical waste; it’s a waste that can be measured in money, energy, and water.

It’s interesting to note that Mistral seems to have taken its own conclusion seriously: throughout 2026, it built an entire family of models of different sizes—from Large 3 down to the small Ministral models with 3, 8, and 14 billion parameters—and explicitly recommended that developers use “Large 3 only when it’s really needed” and the Ministral models for lighter tasks. They have not yet republished a study with updated figures for these new models, but the philosophy of “right-sizing” is now part of their product catalog, not just a research paper.

A Name for This Movement: Frugal AI

This philosophy now has a name and a shared manifesto. The Frugal AI Hub, an initiative that brings together researchers and companies around this theme, has codified four principles that are becoming a benchmark for those designing responsible AI systems: resource efficiency (smaller, faster, and more cost-effective models to train and run), sustainability (reducing the carbon footprint and promoting renewable energy and infrastructure), accessibility and inclusion (enabling even small businesses, startups, and environments with limited connectivity to use AI), and finally, impact and scalability (seriously evaluating return on investment, not just technical ambition).

This is not an isolated initiative: it’s a sign that “frugal” is ceasing to be synonymous with “less capable” and is becoming a design criterion in its own right, on par with accuracy and speed.

The tailor-made suit: not every task requires a genius

If scale matters so much, the natural question becomes: how do you choose the right model for each task? Google, through its developer guidelines, proposes a framework that is becoming a de facto standard in the industry: the “right-sizing” of artificial intelligence.

For very specific and repetitive tasks, a dedicated small model is more than sufficient and can run on the most cost-effective server available. For a broad family of general-purpose but well-defined tasks, a “small” state-of-the-art model is now recommended everywhere: on the device, available for on-demand download, or on a server. Only for tasks that truly require extensive reasoning is a cutting-edge model still needed—firmly in the cloud.

The hybrid approach is already in your browser, even if you haven’t noticed it

This isn’t just theory from an academic paper: it’s already infrastructure in production. Google has integrated into its developer tools (Firebase AI Logic) a system that automatically chooses whether to process a request with a lightweight model directly in the browser—via Gemini Nano integrated into Chrome—or, if the task is too complex or the device isn’t compatible, to seamlessly switch to a more powerful cloud model.

The benefit for app developers is twofold: near-zero latency and, potentially, no API call costs for requests handled locally. We’re still in the early stages—currently, it only works for single-turn text, with a limit of about 6,000 tokens, and only on desktop Chrome—but the direction is clear.

AI assistants don’t have to be universal geniuses

There’s one final piece, perhaps the most conceptual, coming from a research group at NVIDIA in collaboration with the Georgia Institute of Technology. In their paper “Small Language Models Are the Future of Agentic AI,” the authors note that most AI “agents”—systems that perform tasks autonomously, such as making reservations, searching for information, or orchestrating other tools—don’t need to hold a brilliant conversation on any topic. They need to do one specific thing well, repeatedly, and reliably.

For this type of work, the researchers argue, small models are already sufficiently capable, faster, easier to specialize, and up to 10–30 times cheaper to run—and they will become the beating heart of agent-based systems in the near future. The massive model remains the “expert advisor” for complex reasoning, while a swarm of small models handles everything else.

When a Single Developer Takes on the Giants: The DwarfStar4 Case

There is a very concrete—and in some ways surprising—example of how far this philosophy can go when it is driven by a single person rather than a corporate lab. Salvatore Sanfilippo—known in the tech community as “antirez,” creator of Redis, the in-memory data store that has become standard infrastructure for the modern web (used as a database, cache, message broker, and, more recently, a vector search engine for AI applications)—single-handedly built DwarfStar4 (ds4): an inference engine written from scratch in C, designed to run a nearly state-of-the-art open-weight model, DeepSeek V4 Flash, directly on high-end personal hardware (Macs with at least 96 GB of RAM, NVIDIA graphics cards, and home clusters connected in parallel), without any cloud connection.

It’s not technically a “small model”: the weight file is about 76 GB. But the principle is the same one driving this entire movement, applied with a different approach: instead of shrinking the model, Sanfilippo has made the inference process so efficient—thanks to asymmetric quantization and by leveraging the model’s “mixture of experts” structure—that it’s now practical to run locally something that, until recently, would have been conceivable only on a server cluster. The project, distributed for free under the MIT license, surpassed 12,000 stars on GitHub in its first few weeks.

This is proof that the shift toward on-premises AI is not driven solely by the product strategies of Google or Microsoft: it’s also a grassroots movement, driven by those who believe that having a state-of-the-art model on your own computer—without sending data to anyone—should be a real option—not just for hobbyists, but for anyone who values privacy and independence from the cloud.

What does this shift in direction mean, in practice?

Putting these pieces together—the growing but not yet decisive efficiency of the models, the proportionality between scale and impact, the principles of Frugal AI, the “right model for the right task” framework, the hybrid infrastructure already in production, and even individual experiments like DwarfStar4—a coherent picture emerges: artificial intelligence in the near future will not be a single, massive entity residing in a distant data center, but a layered ecosystem, where much of the day-to-day work moves closer to us.

This doesn’t mean data centers will stop growing—training the most advanced models will remain a centralized endeavor for many years to come, and as both Google and the IEA show, the overall energy bill continues to rise anyway. But the demand for AI computing is diverging, and this divergence has very concrete consequences for who builds the hardware, who wins and who loses in the supply chain, and where it makes sense to look today if you’re observing this sector from an investor’s perspective.

In the next article, we’ll dive into the numbers: what the quarterly financial reports from Big Tech, chip manufacturers, and industry outsiders really reveal about this divergence between data centers and the edge—and which lesser-known players might benefit the most.