feature article
Subscribe Now

Two New Ways to Program Your AI

NXP and Eta Compute Roll Out ML-At-The-edge Programming Tools

“Some people have a way with words, and other people, uh, not have way.” – Steve Martin

The scariest part of moving to a new country is learning a new language. You step off the plane and into a new land with illegible signs, strange customs, and unfamiliar culture. Which way do I go? Whom do I ask for help? Where do I even start? 

The strange world of AI/ML is terra incognita for most of us. We’ve heard rumors of its vast richness and unlimited opportunity. It certainly sounds exciting, for those willing to take the plunge. But how do you navigate its unfamiliar territory without a local guide or even a phrasebook to get you started? 

Fortunately, the ML-on-IoT landscape is populated by friendly natives willing to lend a hand. They want you to succeed, and they’re willing to donate their time to make the transition as easy as possible. Maybe, one day, you’ll become a productive member of their society. Which means you might also become a paying customer. 

Say hello (or Grüß Gott, ¡Hola!, Χαίρετε, xin chào, or こんにちは) to TensAI Flow and Glow, two new programming tools from Eta Compute and NXP, respectively. Although they’re as different as Swedish and Swahili, both exist for the same purpose: to get you conversant with machine-learning coding as painlessly as possible. 

Eta Compute, you’ll recall, is the company that produces the ECM3532 microcontroller chip, a two-headed MCU-with-DSP created especially for low-power inference at the edge. The company is convinced that uploading sensor data to a cloud server is the wrong way to do things and has dedicated its hardware and software efforts to overturning that usage model. Chips should be smart enough to collect and analyze sensor data without outside help and without draining your batteries or bank account. 

Making the hardware was tough enough, but making it easy to program may have been even tougher. TensAI Flow is the company’s new software toolchain to ease the burden. It takes high-level models created or collected by TensorFlow Lite, Edge Impulse, or ONNX, links them with its neural network zoo, and converts them to ECM35xx binaries. The software bundle also comes with libraries for common functions like image classification, motion detection, and sound processing, as well as drivers for popular sensors. TensAI Flow handles the nontrivial task of dividing up the workload between the chip’s ARM processor and its DSP companion. Finally, it produces runtime executables for FreeRTOS, the lightweight scheduler. 

TensAI Flow can’t train your neural network for you, but it significantly eases the task of getting that model onto real hardware. That’s a boon for pioneers in this largely unexplored world of ML-on-IoT programming. It provides a quick reality check for new projects, and it’s a quick-turnaround tool for iterating once you get further along. And it’s free. Some parts of the TensAI Flow package are open-sourced, but most is Eta Compute’s original work, freely licensed as long as it’s for Eta Compute hardware. 

Over on the other side of the world, chip behemoth NXP is also polishing up its ML credentials with the release of its Glow compiler for microcontrollers. Glow is a machine-learning compiler originally created at Facebook but now open-sourced and gaining industry support. Like any compiler, it’s language-specific but hardware-agnostic. A generic out-of-the-box Glow compiler will produce acceptable results (like gcc) but nothing to write home about. 

That’s where NXP comes in. The company has optimized its Glow compiler for its own line of i.MX RT microcontrollers, which – just like those at Eta Compute – are based on an ARM Cortex-M paired with a DSP core. (Weirdly, NXP doesn’t use its own CoolFlux DSP, opting instead for Tensilica’s HiFi 4 design.) The compiler accepts PyTorch, ONNX, or TensorFlow Lite input, should you decide to go that route. NXP’s Glow implementation includes CMSIS-NN libraries and other chip-specific tweaks to get the best out of your models. 

It certainly seems to work. In NXP’s benchmarks, the company showed a 2× improvement in performance just by linking in the CMSIS-NN library, and a huge 13× improvement when the compiler targets the on-chip DSP. Let’s hear it for hardware/software optimization. 

As NXP’s Director of Enabling Technologies, Markus Levy, points out, ML code is just middleware — it’s never the entire application, so performance gains mean more headroom for your “real” code. Your ML algorithms may hold your most valuable and hard-won IP, but they’re not what the customer sees. A big jump in compiler efficiency is like getting free CPU cycles to spend elsewhere. 

NXP and Eta Compute are both betting big on high-volume ML hardware, and both know that accessible software is the way to move chips. Maybe their new tools can bridge the language barrier and get more developers talking.

Leave a Reply

featured blogs
Nov 30, 2021
Have you ever wondered why Bill is a common nickname for William and Dick is a common nickname for Richard?...
Nov 30, 2021
Explore the history of the chip design process, from the days of Integrated Device Manufacturers (IDMs) to EDA tools and today's era of democratized design. The post Just What Is Democratized Design Anyway? appeared first on From Silicon To Software....
Nov 30, 2021
The demand for smaller electronics devices can be achieved by high-density layers in multi-layer build-up substrates or multi-layered printed circuit boards (PCB). Vias are essential in the design... [[ Click on the title to access the full blog on the Cadence Community site...
Nov 8, 2021
Intel® FPGA Technology Day (IFTD) is a free four-day event that will be hosted virtually across the globe in North America, China, Japan, EMEA, and Asia Pacific from December 6-9, 2021. The theme of IFTD 2021 is 'Accelerating a Smart and Connected World.' This virtual event ...

featured video

Architecture All Access: Modern FPGA Architecture

Sponsored by Intel

In this 20-minute video, Intel Fellow Prakash Iyer takes you on a journey within the architecture of an FPGA, starting with simple logic gates and then moving up through architecture, design, and applications. Along the way, he answers many questions you might have about FPGAs, even if you’ve worked with FPGAs for years.

Click here for more information

featured paper

Building Automation and Control Systems (BACS)

Sponsored by Maxim Integrated (now part of Analog Devices)

Analog Devices' industrial communication products provide building automation engineers with a broad range of Analog IO, Digital IO, Isolation, and communication interfaces that combine low power, robust performance, and improved diagnostics in the smallest possible form factors.

Click to read more

featured chalk talk

Machine-Learning Optimized Chip Design -- Cadence Design Systems

Sponsored by Cadence Design Systems

New applications and technology are driving demand for even more compute and functionality in the devices we use every day. System on chip (SoC) designs are quickly migrating to new process nodes, and rapidly growing in size and complexity. In this episode of Chalk Talk, Amelia Dalton chats with Rod Metcalfe about how machine learning combined with distributed computing offers new capabilities to automate and scale RTL to GDS chip implementation flows, enabling design teams to support more, and increasingly complex, SoC projects.

Click here for more information about Cerebrus Intelligent Chip Explorer