feature article
Subscribe Now

IP of Providence

Altera Boosts Video Analytics

The age of intelligent video is upon us.  We’ve all played with the new Kinect devices from Microsoft.  We’ve read about lane departure and collision avoidance systems being integrated into cars.  We’ve heard about technologies like facial recognition being used in security applications.  No longer are we content to stream “dumb” video from place to place.  While the master control room with giant arrays of video feeds may make a compelling image for science fiction, the reality will be more like a giant array of cameras – and a very small number of monitors – showing us only the things that actually deserve our attention.  

In security systems, we now have the capability to deploy large numbers of high-quality cameras at a very reasonable cost.  High Definition (HD) video and WIde Dynamic Range (WDR) techniques give us the best quality images we’ve ever had.  Now, the weak links are the human eyes watching all those hours of uneventful image streams.  What we need are intelligent cameras that can monitor their own video and can bring the humans in only when something interesting happens.  

With as much data as all those cameras generate, we end up with a networking and storage problem as well.  There is no need to transmit terabytes of video of the same empty parking lot through your network and use up valuable storage space to accumulate it on massive servers. Ideally, we’d be able to analyze the data at the source – in the camera – and send along only the parts that matter.  Our networks and storage servers would appreciate that very much.

This is not a new idea, of course.  But intelligent, real-time monitoring of all that data, particularly on HD video streams, is a big processing challenge.  Conventional DSP-based solutions burn up too much power to be practically deployed inside cameras, so people look to devices like FPGAs to bring the computation efficiency up to useful levels.  Just this year we’ve done articles on Lattice’s HD/HDR video development kit and Xilinx’s video development platforms and discussed the debut of the Embedded Vision Alliance.  

Now, Altera is attacking another critical part of the video analytics picture – the hardware IP.  While FPGAs can deliver spectacular computation efficiency (in terms of power), the effort and expertise required to build a competent analytics system using an FPGA are considerable.  Altera is announcing this week that they’ve partnered with video analytics firm Eutecus to offer a line of video analytics IP for Altera FPGAs.  The interesting new technology here comes from Eutecus, of course, but equally intriguing is the business model Altera is using to deploy the technology to their customers.

The products consist of “MVE,” a multi-core video analytics engine that can fit in a Cyclone IV FPGA, plus a software GUI that allows you to set up and configure the analytics rules and events.  The IP takes advantage of Altera’s Nios II soft-core embedded processors as well as the capability to create hardware accelerators in the FPGA fabric.  The MVE consists of Eutecus’s InstantVision Embedded software running on Altera’s Nios II core – combined with “C-MVA” (Cellular Multicore Video Analytics) co-processor IP cores implemented in the Cyclone IV fabric.  Of course, since you’re putting all that IP in an FPGA, you can also do other tasks such as managing video inputs and outputs, streaming, conversion, display, and whatever else your system requires.  

One of the cool parts of the Eutecus system is the ease of programming the events.  The provided GUI lets the developer configure each device for the specific scene being monitored and define the events that should trigger an alert.  The overall geometry of the scene can be specified to help the system understand the perspective effects, and the relative sizes of objects of interest – cars, people, etc – can be configured.  Then, events to be monitored can be specified almost like state transitions.  “If an object of this size starts here – and then moves to this other location – we know that was a car making a left turn.”  “If an object of this size is in this area – we know that is a pedestrian in the roadway.”  “If two objects of this size are in this proximity in this area – we have a potential collision.”  The rules are simple to change and edit.

Development teams wanting to deploy intelligent security systems based on HD cameras could clearly save enormous amounts of development time using this analytics kit.  However, one of the most interesting parts of the collaboration is the method by which Altera is deploying and delivering the IP.  Many of the companies developing these types of systems are smaller systems houses with limited development budgets.  Buying a one-time IP license for a large fee would be a stretch for many of them.  To get around that problem, this IP is being made available directly from Altera, without a big, up-front NRE charge. Instead, the IP is licensed on a royalty basis.

Uh, oh.

I bet you are imagining complicated legal documents, verifiable reporting of manufacturing and shipping volumes, and other nightmare scenarios involving lawyers, accountants, and big mahogany negotiating tables, huh?

It’s actually much cooler and easier than that.  

Altera licenses the IP using their normal encrypted IP distribution mechanism.  However, the IP itself is enabled by a security chip that sits alongside the FPGA.  For each device where you want to deploy the IP, you buy a security chip from Altera, and the device unlocks the license on the FPGA-based IP.  No fancy contracts required.  Pretty nifty, huh?  This IP model actually has a lot of potential for higher-end FPGA IP.  Assuming the overhead of connecting the security chip is small, and assuming that the level of security is adequate (for the commercial purposes of Altera and their partners), it’s a very customer-friendly model that could solve a lot of the traditional issues with complex and expensive IP distribution and deployment.

For now, when you license the IP to use in your FPGA design, you’re locked to a specific version.  However, we can see this category of IP being ripe for easy upgrades and enhancements – which we’d be wanting to deploy in systems already in the field.  For that, Altera would need to come up with an infrastructure and a license model to handle all that updating.  It’s not quite as simple as a mobile phone app.  At least not yet.

As for the Orwellian future that technologies like this could enable – we can only try to make sure that the all-seeing-eyes that we are placing on the world use that knowledge for our benefit.  There are myriad applications where video analytics can dramatically enhance safety and security, and we will be better off for that.

3 thoughts on “IP of Providence”

  1. Hmmm… makes me wonder whether they could house the security chip inside the FPGA package, eliminating the need for a separate outside chip. A “security-enabled” version of the FPGA. Of course, that creates an inventory/OPN challenge… Build to order, since it’s a back-end thing?

  2. I’m less than impressed by an idea that increases board area and manufacturing complexity and BOM costs for no benefit to the end consumer.

    It would be much better if the FPGA’s all had unique embedded crypto keys (OTP programmed at the Altera factory).
    The programming procedure would then be that the programming software at the OEM’s factory would read the public part part of the the crypto key from the FPGA via JTAG and would send it via the internet to Altera (or their distributed ‘cloud’ servers). They would combine it with their key and return it. This new key would be then used to encrypt the bitstream used to program the FPGA configuration device (which is also on the same JTAG chain).
    With broadband and modern PC speeds, this is trivial to do in real time as it is just a key exchange.
    Altera could then simply bill you for the number of units you actually used.
    This system would also prevent people copying your products, as each configuration bitstream would be traceable and uniquely coded to that one chip.

Leave a Reply

featured blogs
Aug 17, 2018
If you read my post Who Put the Silicon in Silicon Valley? then you know my conclusion: Let's go with Shockley. He invented the transistor, came here, hired a bunch of young PhDs, and sent them out (by accident, not design) to create the companies, that created the compa...
Aug 16, 2018
All of the little details were squared up when the check-plots came out for "final" review. Those same preliminary files were shared with the fab and assembly units and, of course, the vendors have c...
Aug 15, 2018
VITA 57.4 FMC+ Standard As an ANSI/VITA member, Samtec supports the release of the new ANSI/VITA 57.4-2018 FPGA Mezzanine Card Plus Standard. VITA 57.4, also referred to as FMC+, expands upon the I/O capabilities defined in ANSI/VITA 57.1 FMC by adding two new connectors that...
Aug 14, 2018
I worked at HP in Ft. Collins, Colorado back in the 1970s. It was a heady experience. We were designing and building early, pre-PC desktop computers and we owned the market back then. The division I worked for eventually migrated to 32-bit workstations, chased from the deskto...
Jul 30, 2018
As discussed in part 1 of this blog post, each instance of an Achronix Speedcore eFPGA in your ASIC or SoC design must be configured after the system powers up because Speedcore eFPGAs employ nonvolatile SRAM technology to store its configuration bits. The time required to pr...