feature article
Subscribe Now

IP of Providence

Altera Boosts Video Analytics

The age of intelligent video is upon us.  We’ve all played with the new Kinect devices from Microsoft.  We’ve read about lane departure and collision avoidance systems being integrated into cars.  We’ve heard about technologies like facial recognition being used in security applications.  No longer are we content to stream “dumb” video from place to place.  While the master control room with giant arrays of video feeds may make a compelling image for science fiction, the reality will be more like a giant array of cameras – and a very small number of monitors – showing us only the things that actually deserve our attention.  

In security systems, we now have the capability to deploy large numbers of high-quality cameras at a very reasonable cost.  High Definition (HD) video and WIde Dynamic Range (WDR) techniques give us the best quality images we’ve ever had.  Now, the weak links are the human eyes watching all those hours of uneventful image streams.  What we need are intelligent cameras that can monitor their own video and can bring the humans in only when something interesting happens.  

With as much data as all those cameras generate, we end up with a networking and storage problem as well.  There is no need to transmit terabytes of video of the same empty parking lot through your network and use up valuable storage space to accumulate it on massive servers. Ideally, we’d be able to analyze the data at the source – in the camera – and send along only the parts that matter.  Our networks and storage servers would appreciate that very much.

This is not a new idea, of course.  But intelligent, real-time monitoring of all that data, particularly on HD video streams, is a big processing challenge.  Conventional DSP-based solutions burn up too much power to be practically deployed inside cameras, so people look to devices like FPGAs to bring the computation efficiency up to useful levels.  Just this year we’ve done articles on Lattice’s HD/HDR video development kit and Xilinx’s video development platforms and discussed the debut of the Embedded Vision Alliance.  

Now, Altera is attacking another critical part of the video analytics picture – the hardware IP.  While FPGAs can deliver spectacular computation efficiency (in terms of power), the effort and expertise required to build a competent analytics system using an FPGA are considerable.  Altera is announcing this week that they’ve partnered with video analytics firm Eutecus to offer a line of video analytics IP for Altera FPGAs.  The interesting new technology here comes from Eutecus, of course, but equally intriguing is the business model Altera is using to deploy the technology to their customers.

The products consist of “MVE,” a multi-core video analytics engine that can fit in a Cyclone IV FPGA, plus a software GUI that allows you to set up and configure the analytics rules and events.  The IP takes advantage of Altera’s Nios II soft-core embedded processors as well as the capability to create hardware accelerators in the FPGA fabric.  The MVE consists of Eutecus’s InstantVision Embedded software running on Altera’s Nios II core – combined with “C-MVA” (Cellular Multicore Video Analytics) co-processor IP cores implemented in the Cyclone IV fabric.  Of course, since you’re putting all that IP in an FPGA, you can also do other tasks such as managing video inputs and outputs, streaming, conversion, display, and whatever else your system requires.  

One of the cool parts of the Eutecus system is the ease of programming the events.  The provided GUI lets the developer configure each device for the specific scene being monitored and define the events that should trigger an alert.  The overall geometry of the scene can be specified to help the system understand the perspective effects, and the relative sizes of objects of interest – cars, people, etc – can be configured.  Then, events to be monitored can be specified almost like state transitions.  “If an object of this size starts here – and then moves to this other location – we know that was a car making a left turn.”  “If an object of this size is in this area – we know that is a pedestrian in the roadway.”  “If two objects of this size are in this proximity in this area – we have a potential collision.”  The rules are simple to change and edit.

Development teams wanting to deploy intelligent security systems based on HD cameras could clearly save enormous amounts of development time using this analytics kit.  However, one of the most interesting parts of the collaboration is the method by which Altera is deploying and delivering the IP.  Many of the companies developing these types of systems are smaller systems houses with limited development budgets.  Buying a one-time IP license for a large fee would be a stretch for many of them.  To get around that problem, this IP is being made available directly from Altera, without a big, up-front NRE charge. Instead, the IP is licensed on a royalty basis.

Uh, oh.

I bet you are imagining complicated legal documents, verifiable reporting of manufacturing and shipping volumes, and other nightmare scenarios involving lawyers, accountants, and big mahogany negotiating tables, huh?

It’s actually much cooler and easier than that.  

Altera licenses the IP using their normal encrypted IP distribution mechanism.  However, the IP itself is enabled by a security chip that sits alongside the FPGA.  For each device where you want to deploy the IP, you buy a security chip from Altera, and the device unlocks the license on the FPGA-based IP.  No fancy contracts required.  Pretty nifty, huh?  This IP model actually has a lot of potential for higher-end FPGA IP.  Assuming the overhead of connecting the security chip is small, and assuming that the level of security is adequate (for the commercial purposes of Altera and their partners), it’s a very customer-friendly model that could solve a lot of the traditional issues with complex and expensive IP distribution and deployment.

For now, when you license the IP to use in your FPGA design, you’re locked to a specific version.  However, we can see this category of IP being ripe for easy upgrades and enhancements – which we’d be wanting to deploy in systems already in the field.  For that, Altera would need to come up with an infrastructure and a license model to handle all that updating.  It’s not quite as simple as a mobile phone app.  At least not yet.

As for the Orwellian future that technologies like this could enable – we can only try to make sure that the all-seeing-eyes that we are placing on the world use that knowledge for our benefit.  There are myriad applications where video analytics can dramatically enhance safety and security, and we will be better off for that.

3 thoughts on “IP of Providence”

  1. Hmmm… makes me wonder whether they could house the security chip inside the FPGA package, eliminating the need for a separate outside chip. A “security-enabled” version of the FPGA. Of course, that creates an inventory/OPN challenge… Build to order, since it’s a back-end thing?

  2. I’m less than impressed by an idea that increases board area and manufacturing complexity and BOM costs for no benefit to the end consumer.

    It would be much better if the FPGA’s all had unique embedded crypto keys (OTP programmed at the Altera factory).
    The programming procedure would then be that the programming software at the OEM’s factory would read the public part part of the the crypto key from the FPGA via JTAG and would send it via the internet to Altera (or their distributed ‘cloud’ servers). They would combine it with their key and return it. This new key would be then used to encrypt the bitstream used to program the FPGA configuration device (which is also on the same JTAG chain).
    With broadband and modern PC speeds, this is trivial to do in real time as it is just a key exchange.
    Altera could then simply bill you for the number of units you actually used.
    This system would also prevent people copying your products, as each configuration bitstream would be traceable and uniquely coded to that one chip.

Leave a Reply

featured blogs
Nov 30, 2023
No one wants to waste unnecessary time in the model creation phase when using a modeling software. Rather than expect users to spend time trawling for published data and tediously model equipment items one by one from scratch, modeling software tends to include pre-configured...
Nov 27, 2023
See how we're harnessing generative AI throughout our suite of EDA tools with Synopsys.AI Copilot, the world's first GenAI capability for chip design.The post Meet Synopsys.ai Copilot, Industry's First GenAI Capability for Chip Design appeared first on Chip Design....
Nov 6, 2023
Suffice it to say that everyone and everything in these images was shot in-camera underwater, and that the results truly are haunting....

featured video

Dramatically Improve PPA and Productivity with Generative AI

Sponsored by Cadence Design Systems

Discover how you can quickly optimize flows for many blocks concurrently and use that knowledge for your next design. The Cadence Cerebrus Intelligent Chip Explorer is a revolutionary, AI-driven, automated approach to chip design flow optimization. Block engineers specify the design goals, and generative AI features within Cadence Cerebrus Explorer will intelligently optimize the design to meet the power, performance, and area (PPA) goals in a completely automated way.

Click here for more information

featured paper

3D-IC Design Challenges and Requirements

Sponsored by Cadence Design Systems

While there is great interest in 3D-IC technology, it is still in its early phases. Standard definitions are lacking, the supply chain ecosystem is in flux, and design, analysis, verification, and test challenges need to be resolved. Read this paper to learn about design challenges, ecosystem requirements, and needed solutions. While various types of multi-die packages have been available for many years, this paper focuses on 3D integration and packaging of multiple stacked dies.

Click to read more

featured chalk talk

GaN Solutions Featuring EcoGaN™ and Nano Pulse Control
In this episode of Chalk Talk, Amelia Dalton and Kengo Ohmori from ROHM Semiconductor examine the details and benefits of ROHM Semiconductor’s new lineup of EcoGaN™ Power Stage ICs that can reduce the component count by 99% and the power loss of your next design by 55%. They also investigate ROHM’s Ultra-High-Speed Control IC Technology called Nano Pulse Control that maximizes the performance of GaN devices.
Oct 9, 2023
6,438 views