Just when you think you’ve seen it all, something new leaps onto the center stage with a fanfare of flügelhorns (once heard, never forgotten). In this case, it’s an alternative way to perform AI inference and anomaly detection on the edge, eliminating the need to use artificial neural networks (ANNs) and instead employing a largely forgotten branch of Russian research from the 1960s that was nearly lost to us in the fog of time.
What we’re talking about is an approach that doesn’t demand special AI acceleration hardware, such as graphics processing units (GPUs) or neural processing units (NPUs). Using only a standard microcontroller unit (MCU), this approach we’re about to discuss can perform inference and anomaly detection 50 times faster while consuming 50 times less energy (I have benchmark numbers to back up these claims).
Have we been doing edge AI the hard way all along? (Source: Me and ChatGPT)
I know, I know… “50 times faster while consuming 50 times less energy than what?” I hear you cry. I’m glad to see you’re paying attention. All will be revealed in the fullness of time. In the meantime, however, may I once again recommend that you cast your orbs over the classic How to Lie with Statistics by Darrell Huff.
Let’s pause for a moment and reflect on what this means. If what I just told you is true (and why would I lie?), then this could be a real game-changer. Let’s consider just one market as a test case—today’s utility companies and their smart meters.
Although people in the utility industry don’t usually call the traditional electromechanical meters from yesteryear “dumb meters” in formal writing, it’s fair to say that those meters weren’t exactly setting records on the industry standard Cognitive Sparkle Index. Those meters could measure only cumulative energy consumption using a rotating aluminum disk. Even worse, they required a human meter reader to visit each premise periodically to record and report usage.
When utility companies talk about smart metering, they generally mean smart metering for electricity, gas, and water. The basic concept is the same in all three cases: a meter that measures consumption and automatically communicates the data to the utility. Modern incarnations of these meters support two-way communication, facilitating remote configuration, monitoring, and updates.
As an aside, electricity smart meters typically don’t use batteries as their primary power source. Instead, they are connected directly to the mains supply and power themselves from the line they’re measuring. That said, many include a small backup battery or supercapacitor to maintain the real-time clock during outages, record outage/restoration events, and preserve critical data.
By comparison, a modern low-power gas or water smart meter might consume only microwatts on average. As a result, a high-quality lithium battery can last 10–20 years, sometimes even longer. Why not use the flow of the fluid in question (gas or water) to generate energy-harvesting top-up power for a rechargeable battery? Ironically, these systems have become so energy-efficient that a meter capable of running for decades on a sealed lithium battery is typically considered a better solution than a more sophisticated self-charging design that might theoretically run forever but introduces additional complexity and additional failure modes. But we digress…
So-called smart meters began appearing in pilot projects in the 1990s, but large-scale deployments didn’t really take off until the mid-2000s to early 2010s. The situation varies significantly from country to country and from utility to utility, but in places like the USA, most traditional meters have been replaced by smart equivalents, and virtually all newly installed meters are smart.
Of course, this all depends on what we mean by “smart.” Let’s use a smart electricity meter as an example. When the smart meter moniker was first coined, “smart” meant something that could measure and report things like energy consumption (kWh), time-of-use consumption (how much power is used at different times of day), voltage levels, current, power factor, outage and restoration events, tamper detection, and power quality metrics.
That’s not really all that smart by modern standards. Today, when someone says, “smart meter,” many people might assume we’re talking about a meter equipped with edge AI that can spot anomalies, detect faults before they occur, identify energy theft, predict maintenance requirements, distinguish between normal and abnormal operating conditions, and make intelligent decisions without relying on cloud connectivity. Would that it were so…
The problem is that most existing, deployed smart meters are built around low-cost 32-bit microcontrollers, such as Arm Cortex-M0/M0+/M3/M4-based devices. These processors are certainly capable of performing limited edge AI inference, but they were never designed to be AI powerhouses. Running sophisticated machine-learning models on such devices often involves tradeoffs in performance, memory usage, latency, and energy consumption.
This is particularly important for existing battery-powered gas and water smart meters. There’s little point in deploying AI on such a meter if the additional processing burden dramatically shortens the battery life. A meter that is expected to operate unattended for 10, 15, or even 20 years is unlikely to receive a warm welcome if it suddenly starts demanding attention every few months.
On the bright side, the past year or so has seen a flurry of announcements featuring cunning new low-power processors that combine conventional CPU cores with NPUs and other AI acceleration hardware. These devices promise to bring increasingly sophisticated edge AI capabilities to deeply embedded systems.
Unfortunately, that doesn’t solve the problem for the hundreds of millions of smart meters already deployed. If you were running a utility company, would you be eager to tell your shareholders that you’ve decided to revisit all those “20-year” smart meters that were installed only a few years ago and replace them with shiny new AI-capable versions? And it’s not as though you would simply swap one processor chip for another. In most cases, you’d be obliged to replace the entire electronics assembly—or perhaps the entire meter. My eyes are watering just thinking about the time, cost, logistics, and resources involved.
By comparison, suppose there was a different way to do AI inference on edge devices that didn’t require power-hungry neural networks. Something that would run much faster on a low-cost 32-bit microcontroller while consuming a fraction of the power. And, most importantly, something that could be remotely uploaded to existing smart meters without requiring any hardware changes.
And so we come to the crux of the biscuit, as it were. I was recently chatting with Noel Hurley, the CEO of Literal Labs. Noel is a man who “knows his onions,” as they say. He started his career as a chip designer at Philips Semiconductors before joining ARM when it was still a relatively small (~40 people) company in Cambridge. He went on to co-found XMOS, later returned to ARM, and now leads Literal Labs as it seeks to shake up long-established assumptions about how AI inference should be performed at the edge.
The story behind Literal Labs begins not in the sun-drenched, cappuccino-fueled, venture-funded exuberance of Silicon Valley, but in the austere academic and research institutions of the Soviet Union of the 1960s. There, mathematician Mikhail Tsetlin was exploring a radically different approach to machine intelligence based on automata, learning, and logic. While researchers elsewhere increasingly focused on neural networks, Tsetlin pursued an alternative path. As the decades passed and neural networks came to dominate the AI landscape, much of Tsetlin’s work faded from view. It could easily have remained a historical curiosity, gathering dust in academic archives and eventually disappearing into the mists of time.
Fortunately for Literal Labs—and potentially for the rest of us—that didn’t happen. The ideas resurfaced through the work of researchers, including Professor Ole-Christoffer Granmo at the University of Agder and Professor Alex Yakovlev at Newcastle University. The result was the development of Tsetlin Machines, first introduced in 2019, and ultimately the creation of Literal Labs, which was spun out of Newcastle University in 2023.
Rather than trying to build ever-larger neural networks and ever more powerful accelerators, Literal Labs has returned to first principles and asked a deceptively simple question: what if we’ve been approaching AI the hard way?
Traditional neural networks are built around vast numbers of numerical weights and matrix multiplications. That’s why modern AI hardware is packed with multiply-accumulate (MAC) units, tensor engines, and specialized accelerators. Literal Labs takes a fundamentally different approach. Its Logic-Based Networks (LBNs) are built from logic rather than arithmetic. Instead of relying on huge collections of weighted numerical operations, these networks are constructed from logical relationships and rules.
The resulting models can perform many of the same inference and anomaly-detection tasks as neural networks while being dramatically more computationally efficient. Since silicon itself is fundamentally built from logic gates, the approach feels intuitively natural—almost as though the hardware and the algorithm were designed for each other.
Of course, a clever AI architecture is only part of the story. If every customer required a team of AI specialists and machine-learning experts before they could make use of the technology, adoption would be somewhat limited. This is where Literal Labs’ ModelMill platform comes into play. Users simply provide their data and let the platform do the heavy lifting. ModelMill analyses the incoming data, assists with preparation and classification, and then automatically trains not one model but potentially hundreds of candidate models. It evaluates the results against the customer’s objectives and presents the most suitable solutions.
Engineers already have enough to worry about without becoming specialists in neural network architecture, hyperparameter tuning, feature engineering, and all the other dark arts that pervade the machine-learning ecosystem. ModelMill automates much of this complexity, freeing users to focus on the problems they are trying to solve rather than the mechanics of training AI models.
The company’s current focus is on time-series and tabular data. Think industrial sensors, machine monitoring, predictive maintenance, utility infrastructure, environmental monitoring, water systems, energy networks, and similar applications. One example Noel mentioned involves predicting inverter failures in large solar installations days before they occur. Another involves water utilities monitoring conditions within sewer networks to predict problems before they become expensive environmental incidents. These are exactly the sorts of real-world industrial problems where edge AI promises enormous benefits, but where traditional neural-network approaches often struggle because of power, cost, and deployment constraints.
One particularly interesting observation Noel made is that successful deployments often lead to retraining. Many people imagine AI as a one-time exercise: gather some data, train a model, deploy it, and then forget about it. In practice, organizations continuously collect additional data, add new sensors, discover new operating conditions, and uncover new failure modes. As a result, they often wish to retrain and refine their models on a regular basis. By automating the creation and deployment of new models, ModelMill makes this process practical rather than painful.
Once a model has been trained, ModelMill generates deployable C/C++ code. This is another aspect I find particularly attractive. Rather than requiring proprietary runtime environments or specialized hardware, the resulting model can be compiled and deployed onto conventional microcontrollers, CPUs, or server platforms. Literal Labs provides the necessary wrappers and APIs so that engineers can integrate the generated model into existing applications with minimal effort. In essence, the AI becomes just another software component that can be incorporated into an existing project.
Of course, all of this would be merely interesting if there were no objective evidence to support the performance claims. Fortunately, there is. This brings us to MLPerf Inference: Tiny, a benchmarking suite developed by MLCommons and widely regarded as one of the industry’s most respected frameworks for evaluating AI inference performance on resource-constrained devices. MLPerf Tiny provides standardized workloads, datasets, measurement methodologies, and evaluation criteria, allowing different approaches to be compared on a genuinely apples-to-apples basis.
One of the benchmark tasks focuses on anomaly detection using the ToyADMOS dataset. This consists of audio recordings containing the sounds produced by various machines and devices. The challenge is to identify anomalous conditions hidden within these recordings—exactly the sort of problem encountered in predictive maintenance and machine-health monitoring applications. Traditionally, the benchmark employs a neural-network-based approach. Literal Labs took a different path. Instead of optimizing the neural network, they replaced it entirely with one of their logic-based models and then executed the benchmark using the same methodology and hardware environment.
The results are certainly eye-catching. Running on a modest Cortex-M7-based microcontroller platform, Literal Labs achieved inference latency approximately 54 times faster than the best qualifying published result while consuming roughly 52 times less energy. Memory requirements were also substantially reduced, while the resulting model still exceeded the benchmark’s required accuracy threshold. Perhaps most remarkable of all, the logic-based model running on a low-cost microcontroller delivered results that would traditionally be expected from far more powerful AI platforms. Even if one approaches such results with healthy engineering skepticism—and one should—they are difficult to ignore.
As longtime readers may know, I have something of a soft spot for FPGAs. During our discussion, my mind immediately wandered toward the possibility of taking the C/C++ output generated by ModelMill, converting it into Verilog, feeding it through a synthesis flow, and deploying the resulting logic directly into FPGA fabric. Noel confirmed that the company has explored hardened implementations of its technology and suggested that another order of magnitude, or even two, in performance might be achievable.
To be clear, this is not the company’s current focus, nor am I suggesting that everyone should rush off and start generating FPGA-based Tsetlin engines tomorrow morning. I’m merely observing that it’s a rather intriguing thought experiment that may prove irresistible to a certain class of engineers (yes, Adam Taylor at Adiuvo, I am indeed talking about you).
We began this column by discussing smart meters, as they provide an easy-to-understand illustration of the edge AI problem. Billions of edge devices already exist in the field. Most were never designed with neural-network-based AI in mind. And replacing them would be enormously expensive. However, smart meters are merely one example. The same arguments apply to industrial monitoring systems, predictive-maintenance platforms, environmental sensors, utility infrastructure, smart buildings, agricultural systems, transportation networks, medical devices, manufacturing equipment, and countless other embedded applications.
If Literal Labs can consistently deliver performance improvements akin to those demonstrated by its benchmark results, then perhaps the most interesting question is not whether logic-based AI can compete with neural networks, but whether neural networks can compete with logic-based AI.
Could it be that we’ve been doing edge AI the hard way all along?
