feature article
Subscribe Now

IP of Providence

Altera Boosts Video Analytics

The age of intelligent video is upon us.  We’ve all played with the new Kinect devices from Microsoft.  We’ve read about lane departure and collision avoidance systems being integrated into cars.  We’ve heard about technologies like facial recognition being used in security applications.  No longer are we content to stream “dumb” video from place to place.  While the master control room with giant arrays of video feeds may make a compelling image for science fiction, the reality will be more like a giant array of cameras – and a very small number of monitors – showing us only the things that actually deserve our attention.  

In security systems, we now have the capability to deploy large numbers of high-quality cameras at a very reasonable cost.  High Definition (HD) video and WIde Dynamic Range (WDR) techniques give us the best quality images we’ve ever had.  Now, the weak links are the human eyes watching all those hours of uneventful image streams.  What we need are intelligent cameras that can monitor their own video and can bring the humans in only when something interesting happens.  

With as much data as all those cameras generate, we end up with a networking and storage problem as well.  There is no need to transmit terabytes of video of the same empty parking lot through your network and use up valuable storage space to accumulate it on massive servers. Ideally, we’d be able to analyze the data at the source – in the camera – and send along only the parts that matter.  Our networks and storage servers would appreciate that very much.

This is not a new idea, of course.  But intelligent, real-time monitoring of all that data, particularly on HD video streams, is a big processing challenge.  Conventional DSP-based solutions burn up too much power to be practically deployed inside cameras, so people look to devices like FPGAs to bring the computation efficiency up to useful levels.  Just this year we’ve done articles on Lattice’s HD/HDR video development kit and Xilinx’s video development platforms and discussed the debut of the Embedded Vision Alliance.  

Now, Altera is attacking another critical part of the video analytics picture – the hardware IP.  While FPGAs can deliver spectacular computation efficiency (in terms of power), the effort and expertise required to build a competent analytics system using an FPGA are considerable.  Altera is announcing this week that they’ve partnered with video analytics firm Eutecus to offer a line of video analytics IP for Altera FPGAs.  The interesting new technology here comes from Eutecus, of course, but equally intriguing is the business model Altera is using to deploy the technology to their customers.

The products consist of “MVE,” a multi-core video analytics engine that can fit in a Cyclone IV FPGA, plus a software GUI that allows you to set up and configure the analytics rules and events.  The IP takes advantage of Altera’s Nios II soft-core embedded processors as well as the capability to create hardware accelerators in the FPGA fabric.  The MVE consists of Eutecus’s InstantVision Embedded software running on Altera’s Nios II core – combined with “C-MVA” (Cellular Multicore Video Analytics) co-processor IP cores implemented in the Cyclone IV fabric.  Of course, since you’re putting all that IP in an FPGA, you can also do other tasks such as managing video inputs and outputs, streaming, conversion, display, and whatever else your system requires.  

One of the cool parts of the Eutecus system is the ease of programming the events.  The provided GUI lets the developer configure each device for the specific scene being monitored and define the events that should trigger an alert.  The overall geometry of the scene can be specified to help the system understand the perspective effects, and the relative sizes of objects of interest – cars, people, etc – can be configured.  Then, events to be monitored can be specified almost like state transitions.  “If an object of this size starts here – and then moves to this other location – we know that was a car making a left turn.”  “If an object of this size is in this area – we know that is a pedestrian in the roadway.”  “If two objects of this size are in this proximity in this area – we have a potential collision.”  The rules are simple to change and edit.

Development teams wanting to deploy intelligent security systems based on HD cameras could clearly save enormous amounts of development time using this analytics kit.  However, one of the most interesting parts of the collaboration is the method by which Altera is deploying and delivering the IP.  Many of the companies developing these types of systems are smaller systems houses with limited development budgets.  Buying a one-time IP license for a large fee would be a stretch for many of them.  To get around that problem, this IP is being made available directly from Altera, without a big, up-front NRE charge. Instead, the IP is licensed on a royalty basis.

Uh, oh.

I bet you are imagining complicated legal documents, verifiable reporting of manufacturing and shipping volumes, and other nightmare scenarios involving lawyers, accountants, and big mahogany negotiating tables, huh?

It’s actually much cooler and easier than that.  

Altera licenses the IP using their normal encrypted IP distribution mechanism.  However, the IP itself is enabled by a security chip that sits alongside the FPGA.  For each device where you want to deploy the IP, you buy a security chip from Altera, and the device unlocks the license on the FPGA-based IP.  No fancy contracts required.  Pretty nifty, huh?  This IP model actually has a lot of potential for higher-end FPGA IP.  Assuming the overhead of connecting the security chip is small, and assuming that the level of security is adequate (for the commercial purposes of Altera and their partners), it’s a very customer-friendly model that could solve a lot of the traditional issues with complex and expensive IP distribution and deployment.

For now, when you license the IP to use in your FPGA design, you’re locked to a specific version.  However, we can see this category of IP being ripe for easy upgrades and enhancements – which we’d be wanting to deploy in systems already in the field.  For that, Altera would need to come up with an infrastructure and a license model to handle all that updating.  It’s not quite as simple as a mobile phone app.  At least not yet.

As for the Orwellian future that technologies like this could enable – we can only try to make sure that the all-seeing-eyes that we are placing on the world use that knowledge for our benefit.  There are myriad applications where video analytics can dramatically enhance safety and security, and we will be better off for that.

3 thoughts on “IP of Providence”

  1. Hmmm… makes me wonder whether they could house the security chip inside the FPGA package, eliminating the need for a separate outside chip. A “security-enabled” version of the FPGA. Of course, that creates an inventory/OPN challenge… Build to order, since it’s a back-end thing?

  2. I’m less than impressed by an idea that increases board area and manufacturing complexity and BOM costs for no benefit to the end consumer.

    It would be much better if the FPGA’s all had unique embedded crypto keys (OTP programmed at the Altera factory).
    The programming procedure would then be that the programming software at the OEM’s factory would read the public part part of the the crypto key from the FPGA via JTAG and would send it via the internet to Altera (or their distributed ‘cloud’ servers). They would combine it with their key and return it. This new key would be then used to encrypt the bitstream used to program the FPGA configuration device (which is also on the same JTAG chain).
    With broadband and modern PC speeds, this is trivial to do in real time as it is just a key exchange.
    Altera could then simply bill you for the number of units you actually used.
    This system would also prevent people copying your products, as each configuration bitstream would be traceable and uniquely coded to that one chip.

Leave a Reply

featured blogs
Oct 4, 2022
We share 6 key advantages of cloud-based IC hardware design tools, including enhanced scalability, security, and access to AI-enabled EDA tools. The post 6 Reasons to Leverage IC Hardware Development in the Cloud appeared first on From Silicon To Software....
Oct 4, 2022
Anyone designing a data center faces complex thermal management challenges . Yes, there's a large amount of electrical power required, but the other side of that coin is that almost all the power gets turned into heat, putting a tremendous strain on the airflow and cooling sy...
Sep 30, 2022
When I wrote my book 'Bebop to the Boolean Boogie,' it was certainly not my intention to lead 6-year-old boys astray....

featured video

PCIe Gen5 x16 Running on the Achronix VectorPath Accelerator Card

Sponsored by Achronix

In this demo, Achronix engineers show the VectorPath Accelerator Card successfully linking up to a PCIe Gen5 x16 host and write data to and read data from GDDR6 memory. The VectorPath accelerator card featuring the Speedster7t FPGA is one of the first FPGAs that can natively support this interface within its PCIe subsystem. Speedster7t FPGAs offer a revolutionary new architecture that Achronix developed to address the highest performance data acceleration challenges.

Click here for more information about the VectorPath Accelerator Card

featured paper

Algorithm Verification with FPGAs and ASICs

Sponsored by MathWorks

Developing new FPGA and ASIC designs involves implementing new algorithms, which presents challenges for verification for algorithm developers, hardware designers, and verification engineers. This eBook explores different aspects of hardware design verification and how you can use MATLAB and Simulink to reduce development effort and improve the quality of end products.

Click here to read more

featured chalk talk

HARTING's HAN® 1A Connector Series

Sponsored by Mouser Electronics and HARTING

There is a big push in the electronics industry today to make our designs smaller and more modular. One way we can help solve these design challenges is with the choice of connector we select for our designs. In this episode of Chalk Talk, Goda Inokaityte from HARTING and Amelia Dalton examine the role that miniaturized connectivity plays in the future of electronic design. They also how HARTING's Han 1A connectors can help reduce errors in installation, improve serviceability and increase modularity in your next design.

Click here for more information about HARTING Han® 1A Heavy Duty Power Connectors