feature article
Subscribe Now

AI at the IoT Endpoint

Quicklogic Fosters Sensing Ecosystem

Computation is entering an era of unprecedented heterogeneous distribution. The diverse demands of IoT applications require everything from heavy-iron, deep-learning data-crunching to ultra-low-latency snap recognition and judgment. Our IoT devices and systems must be simultaneously aware and responsive to their own local context and able to harness the power of massive compute resources for more global issues. A self-driving vehicle can’t afford to send gobs of raw sensor data upstream to the cloud and then wait for an answer on target identification to return before deciding whether to brake or swerve. It needs to decide immediately whether or not there’s a human in the crosswalk, but it can wait awhile before rendering an AI judgment on whether the pedestrian’s attire was fashionable.

In intelligent assistants such as Alexa, Google Assistant, and Siri, it would be utterly impractical (and a major privacy intrusion) for every one of the millions of devices in service to send all the audio back to the cloud for wake word recognition. To prevent the communication and compute problem from becoming untenable, we need to do as much of the AI processing as possible locally, without sending data over the network or waking power-hungry application processors. This brand of processing requires a combination of specialized hardware acceleration with miserly power consumption. It requires hardware that can easily adapt to various configurations of sensors at the IoT edge. It requires BOM-friendly low cost and high volume applicability. 

This is QuickLogic’s wheelhouse.

Life would be easy if every engineering team included data scientists who could design the training regimens working hand in hand with hardware experts who could partition the problem between conventional software, programmable hardware, and specialized neural network configuration. But life is not easy. Most projects don’t have access to the wide range of skills and expertise required to optimally engineer an AI endpoint for their IoT design. To make that happen, we need an ecosystem with plug-and-play hardware, software, and AI components and IP that will allow an average engineering project to take advantage of endpoint AI. This month, QuickLogic and several partners are introducing just such an ecosystem.

QuickLogic, along with SensiML, General Vision, and Nepes Corporation, are introducing the “QuickAI” ecosystem and development HDK, which combines QuickLogic’s EOS S3 SoC FPGAs with NM500 neuromorphic processors and AI IP to allow design teams to add endpoint AI to a wide range of applications. The NM500 neuromorphic processor is built by Nepes using IP licensed from General Vision. It features 576 Neurons while consuming a meager 0.1 Watt of power. General Vision’s NeroMem IP provides a scalable, silicon-trainable, low-power network architecture, which the company says is capable of learning and recalling patterns autonomously without the need for high-powered data-center processors for training. Rounding out the QuickAI platform is SensiML’s analytics toolkit, which is designed to help designers quickly and easily build smart sensor algorithms for IoT edge/endpoint devices.

The General Vision NeroMem provides the backbone of the platform, enabling embedded exact and fuzzy pattern matching and learning using a scalable architecture of Radial Basis Function neurons. The architecture is parallel, guaranteeing a fixed latency for any particular number of neurons and delivering high levels of computation at very low clock frequencies for power efficiency. General Vision provides a tool suite and SDK called “Knowledge Builder” that trains and configures the neurons in the NeuroMem network.

The NM500 implements the General Vision NeuroMem in a small form-factor component, which can be trained in the field to recognize patterns in real time. Multiple NM500s can be chained to provide an arbitrary number of neurons. Nepes also provides software tools to be used in configuring and training the NM500 neurons. SensiML’s Analytics Toolkit is designed to automate the management of training data, optimize the choice of feature extraction algorithms, and automate code generation for the resulting AI solution. The QuickLogic EOS S3 voice- and sensor-processing platform performs audio processing and sensor aggregation. In addition to FPGA fabric, it includes ARM Cortex M4F & FFE cores that can pick up the conventional processing chores on a tiny power budget. 

The QuickAI HDK platform is designed as a demo, evaluation, and development platform for endpoint AI applications. It includes the QuickLogic EOS S3 in a “stamp module” that can also be used in production, two Nepes NM500 neuromorphic processors, two PDM microphones, an NRF51822 Bluetooth low energy (BLE) module, a USB to UART, MX25R3235 flash, a MAG AK9915 3-axis magnetic sensor, and a 70-pin expansion connector. The platform is expandable to add more NM 500s for applications that require more neurons. The goal of the HDK is to reduce development time and time to market for endpoint AI applications that involve motion, acoustic, or image processing. 

If you’re designing industrial applications such as vision-based inspection, QuickAI can enable classification of textures such as foods and surfaces that uses high-speed template learning and matching to adapt to changes in materials or color. The FPGA can capture and aggregate sensor data, perform feature extraction using FFT or MFCC, and pass the reduced information along to the NM500 for processing. The FFE provides an ultra-low power AON function accelerator. The result is an easy-to-design, low-power, high-performance adaptable system that can perform advanced pattern matching at the edge without the need to push data up to the cloud for additional crunching.

Starting life as a smaller player in the FPGA market, QuickLogic has pivoted into numerous high-value niche markets where they could take advantage of the unique features of their programmable logic technology for targeted applications. Because of the low-power performance and low cost of their devices, they have carved out a good business in the mobile market, and the current trend toward moving AI to the endpoint in IoT systems has provided fertile ground for this type of platform to succeed. The collaboration with specialized AI players like General Vision, Nepes, and SensiML creates a robust development platform that should eliminate much of the friction for design teams wanting to take advantage of AI technology at the IoT edge. It will be interesting to watch how and where this technology catches on.

One thought on “AI at the IoT Endpoint”

Leave a Reply

featured blogs
Dec 3, 2021
Hard to believe it's already December and 11/12ths of a year's worth of CFD is behind us. And with the holidays looming, it's uncertain how many more editions of This Week in CFD are... [[ Click on the title to access the full blog on the Cadence Community sit...
Dec 3, 2021
Explore automotive cybersecurity standards, news, and best practices through blog posts from our experts on connected vehicles, automotive SoCs, and more. The post How Do You Stay Ahead of Hackers and Build State-of-the-Art Automotive Cybersecurity? appeared first on From Si...
Dec 3, 2021
Believe it or not, I ran into John (he told me I could call him that) at a small café just a couple of evenings ago as I pen these words....
Nov 8, 2021
Intel® FPGA Technology Day (IFTD) is a free four-day event that will be hosted virtually across the globe in North America, China, Japan, EMEA, and Asia Pacific from December 6-9, 2021. The theme of IFTD 2021 is 'Accelerating a Smart and Connected World.' This virtual event ...

featured video

Architecture All Access: Modern FPGA Architecture

Sponsored by Intel

In this 20-minute video, Intel Fellow Prakash Iyer takes you on a journey within the architecture of an FPGA, starting with simple logic gates and then moving up through architecture, design, and applications. Along the way, he answers many questions you might have about FPGAs, even if you’ve worked with FPGAs for years.

Click here for more information

featured paper

Add Authentication Security to Automotive Endpoints Using the 1-Wire Interface

Sponsored by Analog Devices

By adding a single authentication IC, automotive designers can authenticate a component with only one signal between an ECU and endpoint component. This is particularly important as counterfeit and theft are increasingly problems in automotive applications. This application note describes how to implement the DS28E40 Deep Cover 1-Wire Authenticator in a system to provide authentication for optical cameras, headlamps, EV Batteries, occupancy sensors, and even steering wheels, and more.

Click to read more

featured chalk talk

How Trinamic's Stepper Motor Technologies Improve Your Application

Sponsored by Mouser Electronics and Analog Devices

Stepper motor control has come a long way in the past few years. New techniques can give greater control, smoother operation, greater torque, and better efficiency. In this episode of Chalk Talk, Amelia Dalton chats with Lars Jaskulski about Trinamic stepper solutions and how to take advantage of micro stepping, load measurement, and more.

Click here for more information about Trinamic TMCM-6110 6-Axis Stepper Motor Driver Board