feature article
Subscribe Now

Two New Ways to Program Your AI

NXP and Eta Compute Roll Out ML-At-The-edge Programming Tools

“Some people have a way with words, and other people, uh, not have way.” – Steve Martin

The scariest part of moving to a new country is learning a new language. You step off the plane and into a new land with illegible signs, strange customs, and unfamiliar culture. Which way do I go? Whom do I ask for help? Where do I even start? 

The strange world of AI/ML is terra incognita for most of us. We’ve heard rumors of its vast richness and unlimited opportunity. It certainly sounds exciting, for those willing to take the plunge. But how do you navigate its unfamiliar territory without a local guide or even a phrasebook to get you started? 

Fortunately, the ML-on-IoT landscape is populated by friendly natives willing to lend a hand. They want you to succeed, and they’re willing to donate their time to make the transition as easy as possible. Maybe, one day, you’ll become a productive member of their society. Which means you might also become a paying customer. 

Say hello (or Grüß Gott, ¡Hola!, Χαίρετε, xin chào, or こんにちは) to TensAI Flow and Glow, two new programming tools from Eta Compute and NXP, respectively. Although they’re as different as Swedish and Swahili, both exist for the same purpose: to get you conversant with machine-learning coding as painlessly as possible. 

Eta Compute, you’ll recall, is the company that produces the ECM3532 microcontroller chip, a two-headed MCU-with-DSP created especially for low-power inference at the edge. The company is convinced that uploading sensor data to a cloud server is the wrong way to do things and has dedicated its hardware and software efforts to overturning that usage model. Chips should be smart enough to collect and analyze sensor data without outside help and without draining your batteries or bank account. 

Making the hardware was tough enough, but making it easy to program may have been even tougher. TensAI Flow is the company’s new software toolchain to ease the burden. It takes high-level models created or collected by TensorFlow Lite, Edge Impulse, or ONNX, links them with its neural network zoo, and converts them to ECM35xx binaries. The software bundle also comes with libraries for common functions like image classification, motion detection, and sound processing, as well as drivers for popular sensors. TensAI Flow handles the nontrivial task of dividing up the workload between the chip’s ARM processor and its DSP companion. Finally, it produces runtime executables for FreeRTOS, the lightweight scheduler. 

TensAI Flow can’t train your neural network for you, but it significantly eases the task of getting that model onto real hardware. That’s a boon for pioneers in this largely unexplored world of ML-on-IoT programming. It provides a quick reality check for new projects, and it’s a quick-turnaround tool for iterating once you get further along. And it’s free. Some parts of the TensAI Flow package are open-sourced, but most is Eta Compute’s original work, freely licensed as long as it’s for Eta Compute hardware. 

Over on the other side of the world, chip behemoth NXP is also polishing up its ML credentials with the release of its Glow compiler for microcontrollers. Glow is a machine-learning compiler originally created at Facebook but now open-sourced and gaining industry support. Like any compiler, it’s language-specific but hardware-agnostic. A generic out-of-the-box Glow compiler will produce acceptable results (like gcc) but nothing to write home about. 

That’s where NXP comes in. The company has optimized its Glow compiler for its own line of i.MX RT microcontrollers, which – just like those at Eta Compute – are based on an ARM Cortex-M paired with a DSP core. (Weirdly, NXP doesn’t use its own CoolFlux DSP, opting instead for Tensilica’s HiFi 4 design.) The compiler accepts PyTorch, ONNX, or TensorFlow Lite input, should you decide to go that route. NXP’s Glow implementation includes CMSIS-NN libraries and other chip-specific tweaks to get the best out of your models. 

It certainly seems to work. In NXP’s benchmarks, the company showed a 2× improvement in performance just by linking in the CMSIS-NN library, and a huge 13× improvement when the compiler targets the on-chip DSP. Let’s hear it for hardware/software optimization. 

As NXP’s Director of Enabling Technologies, Markus Levy, points out, ML code is just middleware — it’s never the entire application, so performance gains mean more headroom for your “real” code. Your ML algorithms may hold your most valuable and hard-won IP, but they’re not what the customer sees. A big jump in compiler efficiency is like getting free CPU cycles to spend elsewhere. 

NXP and Eta Compute are both betting big on high-volume ML hardware, and both know that accessible software is the way to move chips. Maybe their new tools can bridge the language barrier and get more developers talking.

Leave a Reply

featured blogs
May 24, 2024
Could these creepy crawly robo-critters be the first step on a slippery road to a robot uprising coupled with an insect uprising?...
May 23, 2024
We're investing in semiconductor workforce development programs in Latin America, including government and academic partnerships to foster engineering talent.The post Building the Semiconductor Workforce in Latin America appeared first on Chip Design....

featured video

Introducing Altera® Agilex 5 FPGAs and SoCs

Sponsored by Intel

Learn about the Altera Agilex 5 FPGA Family for tomorrow’s edge intelligent applications.

To learn more about Agilex 5 visit: Agilex™ 5 FPGA and SoC FPGA Product Overview

featured paper

Altera® FPGAs and SoCs with FPGA AI Suite and OpenVINO™ Toolkit Drive Embedded/Edge AI/Machine Learning Applications

Sponsored by Intel

Describes the emerging use cases of FPGA-based AI inference in edge and custom AI applications, and software and hardware solutions for edge FPGA AI.

Click here to read more

featured chalk talk

Digi XBee 3 Global Cellular Solutions
Sponsored by Mouser Electronics and Digi
Adding cellular capabilities to your next design can be a complicated, time consuming process. In this episode of Chalk Talk, Amelia Dalton and Alec Jahnke from Digi chat about how Digi XBee Global Cellular Solutions can help you navigate the complexities of adding cellular connectivity to your next design. They investigate how the Digi XBee software can help you monitor and manage your connected devices and how the Digi Xbee 3 cellular ecosystem can help future proof your next design.
Nov 6, 2023
25,806 views