feature article
Subscribe Now

Lattice mVision Stack

Powering Up Low-Power Embedded Vision

We’ve elaborated at length in these pages on the remarkable capabilities FPGAs bring to the world of embedded vision and vision analytics. With enormous numbers of new applications demanding embedded vision capabilities at the endpoint and edge, there’s a huge green field of opportunity out there, just waiting for FPGAs to take over. The ability of FPGAs to accelerate AI tasks on a miserly power budget, perform various video filtering and processing tasks, and adapt to handle basically any interface standard for the myriad sensors, displays, and other related hardware out there is unmatched by any other technology on the market.

But, adoption of FPGAs continues to be limited by the difficulty – both perceived and real – of customizing the devices for each particular application. Historically, FPGA use has been the exclusive purview of engineering teams with expertise in HDL design and FPGA experience. The skill and knowledge required to create Verilog descriptions of custom hardware, simulate and synthesize that code, and complete place and route as well as timing closure – has been a major barrier to entry for the FPGA party for the last three decades. 

FPGA companies are acutely aware of this, of course, and perhaps the most effective weapon they have to combat the problem is the pre-engineered “stack” for specific classes of applications. A stack is an almost-complete sample application – including development boards, reference designs, relevant IP blocks, software, firmware, and whatever else you need to get to your application’s version of “Hello World.” With a stack, you can usually be up and running with something that approximates your project in a matter of hours or even minutes from first powering up your development kit. Then, it’s just a matter of modifying the already-working system to include your own special sauce. While this doesn’t do away with the need for hardware and FPGA expertise altogether, it certainly can dramatically reduce the learning curve.

Lattice Semiconductor has just announced mVision, a new “stack” aimed at machine vision in power-constrained edge and endpoint designs. Lattice has long led the field in ultra-low power FPGAs (think fractions of watts). They have shipped over a billion devices into systems such as smartphones, cameras, and other battery-powered and portable devices. Lattice estimates the machine vision market will grow to over $18B by 2025, and they figure mVision will help their aim to claim a chunk of that loot in the high-volume, low-power end of the spectrum.

mVision supports three different families of Lattice FPGAs – their MIPI-bridging Crosslink and Crosslink-plus families (with up to 6K LUTs, 150 mW power dissipation, and 2.5×2.5 mm packaging), the recently-announced Lattice Nexus Crosslink-NX (built on 28nm FDSOI technology, with 40K LCs, 0.3 W power dissipation, and 6×6 mm packaging) and the larger and more powerful ECP5 family (with 85K LUTs, 1 W power dissipation, and 10×10 mm packaging). This gives us a nice range of capability, power dissipation, and form factor depending on our project’s constraints – all under the same mVision umbrella.

Lattice’s video interface platform (VIP) development boards are clever mix-and-match stacked modules that let you pair your choice of FPGA, camera interface, and IO interfaces. Supported interfaces include MIPI, and LVDS image sensors, MIPI, eDP, DisplayPort, HDMI, GigE Vision, and USB3 IO. Basically, you can connect to most any image sensor and most any display standard with ready-made IP blocks that drop into your design, and the development boards give you the hardware to back it up.

Once you’re connected to your choice of sensors and displays, Lattice has assembled a powerful library of IP blocks to help you tame your video stream. These include image processing functions such as Defect Pixel Correction, HDR, Color Space Conversion, Black-level Correction, 2D Scaler, Gamma Correction, Overlay, AWB, De-Bayer, Video Frame Buffer, Color Filter, and Interpolation; bridging and aggregation functions including CSI-2/DSI D-PHY Receiver, CSI-2/DSI D-PHY Transmitter, Byte to Pixel Converter, Pixel to Byte Converter, FPD-LINK Receiver, and FPD-LINK Transmitter; and connectivity functions including DisplayPort Rx, DisplayPort Tx, GigE, and GigE vision. 

If figuring out how to connect the IP blocks you want to make an application is a little baffling, Lattice helps you get started with a rich set of reference designs that will cover a large gamut of typical applications. These include MIPI DSI/CSI-2 to OpenLDI LVDS Interface Bridge, N to 1 CSI-2/DSI Duplicator, subLVDS to MIPI CSI-2 Image Sensor Bridge, N Input to 1 Output MIPI CSI-2 Camera Aggregator Bridge, 4:1 Image Aggregation, MIPI CSI-2 N to 1 Sideby-side Aggregation, 3D Depth Mapping, and ISP. If what you’re doing resembles one of those applications, your design is mostly done already (don’t tell your boss). 

Lattice’s tool support includes their legacy “Diamond” as well as their newer “Radiant” design tool suites – depending on your FPGA choice. And, if all these reference designs, IP, and tools still seem a little more than your team wants to bite off, Lattice has a range of design services available with experienced engineering talent that is well-versed in this type of design work – from their own Lattice Design Group, as well as third parties including Helion Engineering Vision, Tata Elxsi, and Bitec. These providers can do anything from developing a specialized IP block to implementing entire turnkey solutions.

The notion of the stack is to enable teams with a wide range of engineering skillsets and capabilities to take advantage of the remarkable capabilities that FPGAs bring to the embedded vision market. With the mVision stack, Lattice is largely uncontested at the ultra-low end of the power spectrum. None of the competitors such as Xilinx, Intel, and even Microchip (Microsemi) offer devices with the very-low power consumption of Lattice’s devices (although their FPGAs tend to be much larger and more capable). If your application is heavily power constrained, there don’t appear to be a lot of comparable FPGA alternatives.

It’s been interesting to watch Lattice re-define their mission and strategy after the failed acquisition a couple years ago. What seemed like bad news for the company at the time may have sparked a new wave of energy and innovation that will benefit the market overall. Lattice’s focus on small, inexpensive, low-power programmable logic technology now makes them unique in a market where demand for those qualities is likely to expand rapidly.

One thought on “Lattice mVision Stack”

  1. Hi Kevin — great column — I must admit that I’ve been impressed with the stuff Lattice has been doing over the past couple of years — I also like the “stack” approach, like the mVision stack you talk about here — and I also think their sensAI and Neural Network Compiler (for artificial intelligence (AI) and machine learning (ML) designs) is rather interesting.

Leave a Reply

featured blogs
Feb 28, 2021
Using Cadence ® Specman ® Elite macros lets you extend the e language '”€ i.e. invent your own syntax. Today, every verification environment contains multiple macros. Some are simple '€œsyntax... [[ Click on the title to access the full blog on the Cadence Comm...
Feb 27, 2021
New Edge Rate High Speed Connector Set Is Micro, Rugged Years ago, while hiking the Colorado River Trail in Rocky Mountain National Park with my two sons, the older one found a really nice Swiss Army Knife. By “really nice” I mean it was one of those big knives wi...
Feb 26, 2021
OMG! Three 32-bit processor cores each running at 300 MHz, each with its own floating-point unit (FPU), and each with more memory than you than throw a stick at!...

featured video

Designing your own Processor with ASIP Designer

Sponsored by Synopsys

Designing your own processor is time-consuming and resource intensive, and it used to be limited to a few experts. But Synopsys’ ASIP Designer tool allows you to design your own specialized processor within your deadline and budget. Watch this video to learn more.

Click here for more information

featured paper

How to Fast-Charge Your Supercapacitor

Sponsored by Maxim Integrated

Supercapacitors (or ultracapacitors) are suited for short charge and discharge cycles. They require high currents for fast charge as well as a high voltage with a high number in series as shown in two usage cases: an automatic pallet shuttle and a fail-safe backup system. In these and many other cases, the fast charge is provided by a flexible, high-efficiency, high-voltage, and high-current charger based on a synchronous, step-down, supercapacitor charger controller.

Click here to download the whitepaper

Featured Chalk Talk

Cadence Celsius Thermal Solver

Sponsored by Cadence Design Systems

Electrical-thermal co-simulation can dramatically improve the system design process, allowing thermal design adaptation to be done much earlier. The Cadence Celsius Thermal Solver is a complete electrical-thermal co-simulation solution for the full hierarchy of electronic systems from ICs to physical enclosures. In this episode of Chalk Talk, Amelia Dalton chats with CT Kao of Cadence Design Systems about how the Celsius Thermal Solver can help detect and mitigate thermal issues early in the design process.

More information about Celsius Thermal Solver