The rise of small language models (SLMs) and push for offline, localized AI

From flip phones to film cameras, more people are stepping away from the always-online world. Small, local AI models may offer a way to keep the benefits of automation and assistance without going fully back to the cloud.

Woman with wavy hair wearing glasses adjusts them while looking at the camera.

With apologies, Nyasha Green turned off her Meta glasses during our interview. We had been communicating for a few minutes, and she had been sharing very thoughtful insights into small language model (SLM) technology before her glasses briefly got in the way. “I’m actually using my Meta AI glasses right now and I forgot that they’re set to notify me of alerts,” Green explained. “So while I was talking it started notifying me.”

It was a quick fix, and yet, quite accidentally, Green had demonstrated how much AI technology has advanced over the last several years, echoing advances made over the last several decades.

Not so long ago was the time of ENIAC, a 1940s machine that many consider to be the first true electronic computer of the modern age. It weighed 30 tons and reportedly required as much power as a small town to operate. Eight decades later, Green sat on our call and comfortably wore its far superior technological successor on her face. She explained that these Meta glasses were an example of how SLMs can be used in highly specific, efficient ways to complete specialized tasks, compared with large language models (LLMs), which handle an array of complex duties.

“[SLMs] are better equipped at running in smaller technology, like glasses, which is crazy to say. We have smart AI glasses, watches, and things like that.” Green is the Technical Curriculum Developer at Innovating With AI, a LinkedIn Learning instructor, and website development consultant. She’s also very passionate about SLMs and predicts they’ll only become more widely used as people, even those still apprehensive about LLMs, start to catch on.

“I’m a big open source advocate,” Green enthused, “so I’m very excited when anybody puts out anything open source. Llama is an open-source SLM, so it’s just really awesome. You can do so much with it, and you can really make it your own.” She revealed that she was using SLM technology for a personal project. 

“I’m in the process of putting my family tree history into Llama so that I can send this out to my family in the world, and they can know all about our family because I have a huge family. [It’s] very big, and AI is going to help me get that information out about them.”

Nyasha Green, Technical Curriculum Developer at Innovating With AI

As for how Llama factors into the project, Green shared that it allowed her to develop a specialized model to function for a very specific purpose. “The one I’m making for my family tree, that’s going to host a lot of personal details about my family, some of which who are still alive.”

Because this is such an important project featuring such sensitive information, it was important to use an SLM that protected not just her privacy, but that of the family members featured. This SLM would be great for the purpose of preserving that privacy and controlling access to her family tree.

SLMs could prove crucial to AI adoption & advancement

When discussing the advantages of using SLMs, it becomes clear that this comparatively bite-sized technology addresses many of the concerns people have about AI as a whole. As Green explains, “One of the big cons of LLMs or why people are rallying against them, is because of the large data centers that are made to host them. They take up so many resources, and they require [so much power] to run versus an SLM.”

SLMs can be configured to run entirely offline, stored on local machines, and operate in very limited, specific ways. For those who want to use AI but want greater control over their data and a way to use the tech offline, SLMs could be a perfect on-ramp to adopting the technology, though Green acknowledges there is a catch. 

“If you go the open source route, you can self-host so much stuff you have control over locally, but your security in that sense is only as good as you make it. You still have to have some security and privacy knowledge.” She said that you may enjoy greater security with an SLM on a laptop, but if you don’t password-protect the device at all, your data could still be in jeopardy.

But as long as you use common-sense practices for guarding your information, Green suggests that “the chances of this…sensitive information getting out is slim.” 

Analog renaissance and AI: Why SLMs may provide respite for those stepping back from tech immersion

The pros of SLMs make them ideal at a time when some are backing away not just from AI usage, but from all forms of Internet-based technology. Some are wondering if we’re entering a so-called “Analog Renaissance” in response to years of life getting dictated by online engagement and digital communication. Think books over Kindle and “dumb phones” designed solely to make or send calls and text messages. 

Tony Fernandes, the founder and Chief AI Experience Officer of HumanFocused.AI and CEO of UEGroup, admitted that he fits an “unusual” profile for someone you’d expect to take a deliberate step back from modern technology. “I have spent my whole career in the tech industry, and I work with AI.” Even so, Fernandes has decided to revert as much of his life as possible to a simpler, tech-free life.

“I bought a place in the Sierra Nevada mountains where I split wood for heat in the wood-burning stove. Nighttime entertainment is looking at the stars. I also have a place in Portugal where I hang out, with no TV or gadgets. My entertainment is going to live analog music events (traditional instruments only) and singers without using amps or any augmentation.” He also reads regular books; no scrolling allowed.

It’s a head-scratching paradox at first glance, but Ava Sirrah, PhD, a professor at New York University’s Stern School of Business, isn’t too surprised by stories like this. 

Ava Sirrah, PhD

“I’ve been tracking this weariness with perpetual connectivity, and the rejection of AI isn’t always about the intelligence itself.” 

According to survey data from the QuestionPro Research Center, nearly 61% of U.S. adults report feeling occasionally overwhelmed by modern technology, causing 28% to take intentional tech breaks and 41% to deliberately disconnect. Question Pro researchers determined that “the [Analog Renaissance] isn’t about rejecting technology altogether—it’s about recalibrating. People aren’t going backward; they’re trying to create healthier boundaries.”

Pie chart illustrating how often people feel mentally or emotionally overwhelmed by technology or constant connectivity.

Yet, in such an environment, SLMs and localized AI could thrive. These systems are much smaller and more affordable than larger models, and capable of completing a small group of specific tasks, whether that’s assemble a family tree or translate other languages in real time using eyeglasses. 

The parameters of SLMs are much smaller. Llama 3.2, for instance, is available in one and three billion parameter sizes; by comparison, some models, like Kimi, developed by the Beijing-based Moonshot AI, reportedly exceed 1 trillion parameters. Using a much more compact model, being able to train it with very specific information from a few websites, or even train it entirely offline, may let SLMs and similarly localized AI provide some people an experience divorced from the Internet, one where they feel a stronger sense of control and privacy. 

One could experience this type of AI little by little, through humanizing projects like family tree development or with tech you can hold in your hand like eyeglasses, rather than solely through an abstract, endless, and potentially overwhelming digital landscape.

AI is then able to become part of one’s everyday life, not by replacing too much too fast or creating ecological controversy, but by seamlessly elevating the human experience, little by little. 

Analog, SLMs, subcultures, and what comes next

Even if SLMs become a truly massive trend in the years ahead, Green doesn’t see a future in which they outright replace LLMs. “We want SLMs smaller, because we want something more efficient…but that’s going to come at the cost of information.” In other words, even as SLMs “become more common,” she believes they lack the components necessary to completely replace most LLMs.

“You won’t be able to find [or do] everything with an SLM, and that’s by design. They are smaller models for a reason. So I think keeping your expectations in line will help when you’re using an SLM.” 

The conversation around SLMs taking on smaller burdens and achieving reasonable goals comes at a time when some are wondering whether a cultural shift towards analog is here to stay or a temporary breather before the collective goes full speed ahead, echoing the contraction and expansion seen with tech leaps in past decades.

AI strategist and consultant Bob Hutchins, PhD, believes many people are seeking “a middle ground,” and that most people “want to get the cognitive aid that AI can provide, but they do not want to give up their data to a cloud-based system,” or unknown “corporate” entities. 

Said Hutchins, “I envision ‘Analog-Plus’ devices. Off-line SLMs that assist without distracting.” He also predicts that “frictionless digital life will remain the norm” and that we may see the rise of an “analog integrated subculture,” where “the ability to be unquantifiable will be the ultimate luxury.”

Bob Hutchins, PhD

Sirrah offered a similar opinion regarding offline AI amid the ongoing analog-themed pushback. “Removing the ‘Internet’ from the ‘AI’ could bridge the gap for those who currently view LLMs as a threat to intellectual privacy.”

Setting expectations for digital overwhelm

As SLMs become more popular and you look to make use of them, it’s important to consider the pros and cons, what’s gained and what’s lost. With them, you gain considerably more control over your data; when localized, your information is far more private and secure. If you build an SLM dedicated to mapping your specific family tree, you shouldn’t expect it to also list all the ruling members of the British royal family over the last few hundred years. Be mindful, specific, and keep your expectations rounded.

While it’s likely we may see a pivot towards an analog-friendly era, it doesn’t inherently mean that all things AI will be going away. There’s no fixed outcome and many experts predict that we’re more likely to see AI become smaller and more manageable than be replaced by old or non-tech items. As was the case with ENIAC, new tech is almost always bulky, huge, and overwhelming. Don’t think of the LLMs or even SLMs of today as the final product and then mistakenly limit your expectations accordingly; it could slow you down at a time when things start to progress quickly!

Instead, choose to build and use SLMs or LLMs based on what you or your customers need in the here and now, keeping enough of an open mind that you’re able to pivot as AI technology inevitably advances.

Separator

Join free: Weekly AI insights and analysis

Trusted by 200,000+ AI enthusiasts, entrepreneurs and consultants.