feature article
Subscribe Now

IP of Providence

Altera Boosts Video Analytics

The age of intelligent video is upon us.  We’ve all played with the new Kinect devices from Microsoft.  We’ve read about lane departure and collision avoidance systems being integrated into cars.  We’ve heard about technologies like facial recognition being used in security applications.  No longer are we content to stream “dumb” video from place to place.  While the master control room with giant arrays of video feeds may make a compelling image for science fiction, the reality will be more like a giant array of cameras – and a very small number of monitors – showing us only the things that actually deserve our attention.  

In security systems, we now have the capability to deploy large numbers of high-quality cameras at a very reasonable cost.  High Definition (HD) video and WIde Dynamic Range (WDR) techniques give us the best quality images we’ve ever had.  Now, the weak links are the human eyes watching all those hours of uneventful image streams.  What we need are intelligent cameras that can monitor their own video and can bring the humans in only when something interesting happens.  

With as much data as all those cameras generate, we end up with a networking and storage problem as well.  There is no need to transmit terabytes of video of the same empty parking lot through your network and use up valuable storage space to accumulate it on massive servers. Ideally, we’d be able to analyze the data at the source – in the camera – and send along only the parts that matter.  Our networks and storage servers would appreciate that very much.

This is not a new idea, of course.  But intelligent, real-time monitoring of all that data, particularly on HD video streams, is a big processing challenge.  Conventional DSP-based solutions burn up too much power to be practically deployed inside cameras, so people look to devices like FPGAs to bring the computation efficiency up to useful levels.  Just this year we’ve done articles on Lattice’s HD/HDR video development kit and Xilinx’s video development platforms and discussed the debut of the Embedded Vision Alliance.  

Now, Altera is attacking another critical part of the video analytics picture – the hardware IP.  While FPGAs can deliver spectacular computation efficiency (in terms of power), the effort and expertise required to build a competent analytics system using an FPGA are considerable.  Altera is announcing this week that they’ve partnered with video analytics firm Eutecus to offer a line of video analytics IP for Altera FPGAs.  The interesting new technology here comes from Eutecus, of course, but equally intriguing is the business model Altera is using to deploy the technology to their customers.

The products consist of “MVE,” a multi-core video analytics engine that can fit in a Cyclone IV FPGA, plus a software GUI that allows you to set up and configure the analytics rules and events.  The IP takes advantage of Altera’s Nios II soft-core embedded processors as well as the capability to create hardware accelerators in the FPGA fabric.  The MVE consists of Eutecus’s InstantVision Embedded software running on Altera’s Nios II core – combined with “C-MVA” (Cellular Multicore Video Analytics) co-processor IP cores implemented in the Cyclone IV fabric.  Of course, since you’re putting all that IP in an FPGA, you can also do other tasks such as managing video inputs and outputs, streaming, conversion, display, and whatever else your system requires.  

One of the cool parts of the Eutecus system is the ease of programming the events.  The provided GUI lets the developer configure each device for the specific scene being monitored and define the events that should trigger an alert.  The overall geometry of the scene can be specified to help the system understand the perspective effects, and the relative sizes of objects of interest – cars, people, etc – can be configured.  Then, events to be monitored can be specified almost like state transitions.  “If an object of this size starts here – and then moves to this other location – we know that was a car making a left turn.”  “If an object of this size is in this area – we know that is a pedestrian in the roadway.”  “If two objects of this size are in this proximity in this area – we have a potential collision.”  The rules are simple to change and edit.

Development teams wanting to deploy intelligent security systems based on HD cameras could clearly save enormous amounts of development time using this analytics kit.  However, one of the most interesting parts of the collaboration is the method by which Altera is deploying and delivering the IP.  Many of the companies developing these types of systems are smaller systems houses with limited development budgets.  Buying a one-time IP license for a large fee would be a stretch for many of them.  To get around that problem, this IP is being made available directly from Altera, without a big, up-front NRE charge. Instead, the IP is licensed on a royalty basis.

Uh, oh.

I bet you are imagining complicated legal documents, verifiable reporting of manufacturing and shipping volumes, and other nightmare scenarios involving lawyers, accountants, and big mahogany negotiating tables, huh?

It’s actually much cooler and easier than that.  

Altera licenses the IP using their normal encrypted IP distribution mechanism.  However, the IP itself is enabled by a security chip that sits alongside the FPGA.  For each device where you want to deploy the IP, you buy a security chip from Altera, and the device unlocks the license on the FPGA-based IP.  No fancy contracts required.  Pretty nifty, huh?  This IP model actually has a lot of potential for higher-end FPGA IP.  Assuming the overhead of connecting the security chip is small, and assuming that the level of security is adequate (for the commercial purposes of Altera and their partners), it’s a very customer-friendly model that could solve a lot of the traditional issues with complex and expensive IP distribution and deployment.

For now, when you license the IP to use in your FPGA design, you’re locked to a specific version.  However, we can see this category of IP being ripe for easy upgrades and enhancements – which we’d be wanting to deploy in systems already in the field.  For that, Altera would need to come up with an infrastructure and a license model to handle all that updating.  It’s not quite as simple as a mobile phone app.  At least not yet.

As for the Orwellian future that technologies like this could enable – we can only try to make sure that the all-seeing-eyes that we are placing on the world use that knowledge for our benefit.  There are myriad applications where video analytics can dramatically enhance safety and security, and we will be better off for that.

3 thoughts on “IP of Providence”

  1. Hmmm… makes me wonder whether they could house the security chip inside the FPGA package, eliminating the need for a separate outside chip. A “security-enabled” version of the FPGA. Of course, that creates an inventory/OPN challenge… Build to order, since it’s a back-end thing?

  2. I’m less than impressed by an idea that increases board area and manufacturing complexity and BOM costs for no benefit to the end consumer.

    It would be much better if the FPGA’s all had unique embedded crypto keys (OTP programmed at the Altera factory).
    The programming procedure would then be that the programming software at the OEM’s factory would read the public part part of the the crypto key from the FPGA via JTAG and would send it via the internet to Altera (or their distributed ‘cloud’ servers). They would combine it with their key and return it. This new key would be then used to encrypt the bitstream used to program the FPGA configuration device (which is also on the same JTAG chain).
    With broadband and modern PC speeds, this is trivial to do in real time as it is just a key exchange.
    Altera could then simply bill you for the number of units you actually used.
    This system would also prevent people copying your products, as each configuration bitstream would be traceable and uniquely coded to that one chip.

Leave a Reply

featured blogs
Nov 24, 2020
In our last Knowledge Booster Blog , we introduced you to some tips and tricks for the optimal use of the Virtuoso ADE Product Suite . W e are now happy to present you with some further news from our... [[ Click on the title to access the full blog on the Cadence Community s...
Nov 23, 2020
It'€™s been a long time since I performed Karnaugh map minimizations by hand. As a result, on my first pass, I missed a couple of obvious optimizations....
Nov 23, 2020
Readers of the Samtec blog know we are always talking about next-gen speed. Current channels rates are running at 56 Gbps PAM4. However, system designers are starting to look at 112 Gbps PAM4 data rates. Intuition would say that bleeding edge data rates like 112 Gbps PAM4 onl...
Nov 20, 2020
[From the last episode: We looked at neuromorphic machine learning, which is intended to act more like the brain does.] Our last topic to cover on learning (ML) is about training. We talked about supervised learning, which means we'€™re training a model based on a bunch of ...

featured video

Available DesignWare MIPI D-PHY IP for 22-nm Process

Sponsored by Synopsys

This video describes the advantages of Synopsys' MIPI D-PHY IP for 22-nm process, available in RX, TX, bidirectional mode, 2 and 4 lanes, operating at 10 Gbps. The IP is ideal for IoT, automotive, and AI Edge applications.

Click here for more information about DesignWare MIPI IP Solutions

featured paper

Keys to quick success using high-speed data converters

Sponsored by Texas Instruments

Whether you’re designing an aerospace system, test and measurement equipment or automotive lidar AFE, hardware designers using high-speed data converters face tough challenges with high-frequency inputs, outputs, clock rates and digital interface. Issues might include connecting with your field-programmable gate array, being confident that your first design pass will work or determining how to best model the system before building it. In this article, we take a look at each of these challenges.

Click here to download the whitepaper

featured chalk talk

ROHM Gate Drivers

Sponsored by Mouser Electronics and ROHM Semiconductor

Today’s rapid growth of power and motor control applications demands a fresh look at gate driver technology. Recent advances in gate drivers help designers hit new levels of efficiency and performance in their designs. In this episode of Chalk Talk, Amelia Dalton chats with Mitch Van Ochten of ROHM about the latest in isolated and non-isolated gate driver solutions.

Click here for more information about ROHM Semiconductor Automotive Gate Drivers