feature article
Subscribe Now

Lattice mVision Stack

Powering Up Low-Power Embedded Vision

We’ve elaborated at length in these pages on the remarkable capabilities FPGAs bring to the world of embedded vision and vision analytics. With enormous numbers of new applications demanding embedded vision capabilities at the endpoint and edge, there’s a huge green field of opportunity out there, just waiting for FPGAs to take over. The ability of FPGAs to accelerate AI tasks on a miserly power budget, perform various video filtering and processing tasks, and adapt to handle basically any interface standard for the myriad sensors, displays, and other related hardware out there is unmatched by any other technology on the market.

But, adoption of FPGAs continues to be limited by the difficulty – both perceived and real – of customizing the devices for each particular application. Historically, FPGA use has been the exclusive purview of engineering teams with expertise in HDL design and FPGA experience. The skill and knowledge required to create Verilog descriptions of custom hardware, simulate and synthesize that code, and complete place and route as well as timing closure – has been a major barrier to entry for the FPGA party for the last three decades. 

FPGA companies are acutely aware of this, of course, and perhaps the most effective weapon they have to combat the problem is the pre-engineered “stack” for specific classes of applications. A stack is an almost-complete sample application – including development boards, reference designs, relevant IP blocks, software, firmware, and whatever else you need to get to your application’s version of “Hello World.” With a stack, you can usually be up and running with something that approximates your project in a matter of hours or even minutes from first powering up your development kit. Then, it’s just a matter of modifying the already-working system to include your own special sauce. While this doesn’t do away with the need for hardware and FPGA expertise altogether, it certainly can dramatically reduce the learning curve.

Lattice Semiconductor has just announced mVision, a new “stack” aimed at machine vision in power-constrained edge and endpoint designs. Lattice has long led the field in ultra-low power FPGAs (think fractions of watts). They have shipped over a billion devices into systems such as smartphones, cameras, and other battery-powered and portable devices. Lattice estimates the machine vision market will grow to over $18B by 2025, and they figure mVision will help their aim to claim a chunk of that loot in the high-volume, low-power end of the spectrum.

mVision supports three different families of Lattice FPGAs – their MIPI-bridging Crosslink and Crosslink-plus families (with up to 6K LUTs, 150 mW power dissipation, and 2.5×2.5 mm packaging), the recently-announced Lattice Nexus Crosslink-NX (built on 28nm FDSOI technology, with 40K LCs, 0.3 W power dissipation, and 6×6 mm packaging) and the larger and more powerful ECP5 family (with 85K LUTs, 1 W power dissipation, and 10×10 mm packaging). This gives us a nice range of capability, power dissipation, and form factor depending on our project’s constraints – all under the same mVision umbrella.

Lattice’s video interface platform (VIP) development boards are clever mix-and-match stacked modules that let you pair your choice of FPGA, camera interface, and IO interfaces. Supported interfaces include MIPI, and LVDS image sensors, MIPI, eDP, DisplayPort, HDMI, GigE Vision, and USB3 IO. Basically, you can connect to most any image sensor and most any display standard with ready-made IP blocks that drop into your design, and the development boards give you the hardware to back it up.

Once you’re connected to your choice of sensors and displays, Lattice has assembled a powerful library of IP blocks to help you tame your video stream. These include image processing functions such as Defect Pixel Correction, HDR, Color Space Conversion, Black-level Correction, 2D Scaler, Gamma Correction, Overlay, AWB, De-Bayer, Video Frame Buffer, Color Filter, and Interpolation; bridging and aggregation functions including CSI-2/DSI D-PHY Receiver, CSI-2/DSI D-PHY Transmitter, Byte to Pixel Converter, Pixel to Byte Converter, FPD-LINK Receiver, and FPD-LINK Transmitter; and connectivity functions including DisplayPort Rx, DisplayPort Tx, GigE, and GigE vision. 

If figuring out how to connect the IP blocks you want to make an application is a little baffling, Lattice helps you get started with a rich set of reference designs that will cover a large gamut of typical applications. These include MIPI DSI/CSI-2 to OpenLDI LVDS Interface Bridge, N to 1 CSI-2/DSI Duplicator, subLVDS to MIPI CSI-2 Image Sensor Bridge, N Input to 1 Output MIPI CSI-2 Camera Aggregator Bridge, 4:1 Image Aggregation, MIPI CSI-2 N to 1 Sideby-side Aggregation, 3D Depth Mapping, and ISP. If what you’re doing resembles one of those applications, your design is mostly done already (don’t tell your boss). 

Lattice’s tool support includes their legacy “Diamond” as well as their newer “Radiant” design tool suites – depending on your FPGA choice. And, if all these reference designs, IP, and tools still seem a little more than your team wants to bite off, Lattice has a range of design services available with experienced engineering talent that is well-versed in this type of design work – from their own Lattice Design Group, as well as third parties including Helion Engineering Vision, Tata Elxsi, and Bitec. These providers can do anything from developing a specialized IP block to implementing entire turnkey solutions.

The notion of the stack is to enable teams with a wide range of engineering skillsets and capabilities to take advantage of the remarkable capabilities that FPGAs bring to the embedded vision market. With the mVision stack, Lattice is largely uncontested at the ultra-low end of the power spectrum. None of the competitors such as Xilinx, Intel, and even Microchip (Microsemi) offer devices with the very-low power consumption of Lattice’s devices (although their FPGAs tend to be much larger and more capable). If your application is heavily power constrained, there don’t appear to be a lot of comparable FPGA alternatives.

It’s been interesting to watch Lattice re-define their mission and strategy after the failed acquisition a couple years ago. What seemed like bad news for the company at the time may have sparked a new wave of energy and innovation that will benefit the market overall. Lattice’s focus on small, inexpensive, low-power programmable logic technology now makes them unique in a market where demand for those qualities is likely to expand rapidly.

One thought on “Lattice mVision Stack”

  1. Hi Kevin — great column — I must admit that I’ve been impressed with the stuff Lattice has been doing over the past couple of years — I also like the “stack” approach, like the mVision stack you talk about here — and I also think their sensAI and Neural Network Compiler (for artificial intelligence (AI) and machine learning (ML) designs) is rather interesting.

Leave a Reply

featured blogs
Dec 1, 2020
If you'€™d asked me at the beginning of 2020 as to the chances of my replicating an 1820 Welsh dresser, I would have said '€œzero,'€ which just goes to show how little I know....
Dec 1, 2020
More package designers these days, with the increasing component counts and more complicated electrical constraints, are shifting to using a front-end schematic capture tool. As with IC and PCB... [[ Click on the title to access the full blog on the Cadence Community site. ]...
Dec 1, 2020
UCLA’s Maxx Tepper gives us a brief overview of the Ocean High-Throughput processor to be used in the upgrade of the real-time event selection system of the CMS experiment at the CERN LHC (Large Hadron Collider). The board incorporates Samtec FireFly'„¢ optical cable ...
Nov 25, 2020
[From the last episode: We looked at what it takes to generate data that can be used to train machine-learning .] We take a break from learning how IoT technology works for one of our occasional posts on how IoT technology is used. In this case, we look at trucking fleet mana...

featured video

Improve SoC-Level Verification Efficiency by Up to 10X

Sponsored by Cadence Design Systems

Chip-level testbench creation, multi-IP and CPU traffic generation, performance bottleneck identification, and data and cache-coherency verification all lack automation. The effort required to complete these tasks is error prone and time consuming. Discover how the Cadence® System VIP tool suite works seamlessly with its simulation, emulation, and prototyping engines to automate chip-level verification and improve efficiency by ten times over existing manual processes.

Click here for more information about System VIP

featured paper

Exploring advancements in industrial and automotive markets with 60-GHz radar

Sponsored by Texas Instruments

The industrial and automotive markets have a tremendous need for innovative sensing technologies to help buildings, cities and automobiles sense the world around them and make more intelligent decisions.

Click here to read the article

Featured Chalk Talk

Series 2 Product Security

Sponsored by Mouser Electronics and Silicon Labs

Side channel attacks such as differential power analysis (DPA) present a serious threat to our embedded designs. If we want to defend our systems from DPA and similar attacks, it is critical that we have a secure boot and root of trust. In this episode of Chalk Talk, Amelia Dalton chats with Gregory Guez from Silicon Labs about DPA, secure debug, and the EFR32 Series 2 Platform.

Click here for more information about Silicon Labs xGM210P Wireless Module Starter Kit