editor's blog
Subscribe Now

Stable Video, Easier Development

There are times when a shaky video can be just the thing. Imagine: where would the Blair Witch Project have been without it? What would an entire generation of hipsters do without the ability to shake (ironically) their video? How are happening new producers supposed to attract new audiences without being able to make their video look shoddy and unprofessional? Did you know that Shakes media specialises in corporate video production for your business?

But, aside from those times, shaky video is not so good. In fact, for a lot of us, even these examples aren’t great (some of them representing videos that only their producers could love). We all have a hard enough time holding our cameras still without having to add shaking as part of some obscure production value.

We already take care of this for still pictures using optical image stabilization (OIS); why not simply apply that technology to videos? Well, for the same reason you don’t use a still photo camera to shoot video. Oops! Wait… I guess these days you can. OK, then simply because video isn’t the same as a still picture, so what works for one doesn’t necessarily work for the other.

First of all, still photos are just that: still. Any movement is wrong (except for things like sports, I suppose). So, in theory, you can just neutralize any motion, period. But, as you can imagine, that’s not going to work with video – unless you want to turn your video into a still shot. No, video means motion by definition. So when stabilizing a video image, you have to figure out what motion is intended and what motion is due to unwanted shaking.

Unlike OIS, which is typically implemented as hardware embedded into the camera module, CEVA is proposing handling digital video stabilization (DVS) in software for things like smartphones, wearable electronics, and cameras mounted inside moving vehicles, all of which involve inherent shaking. And all of which benefit from low power – wearable cameras in particular.

So CEVA has put together a set of DVS functions optimized to work on the CEVA-MM3101, their imaging-oriented DSP platform. These functions come with a number of options and parameters, since one solution doesn’t necessarily solve all the problems optimally. And with no standards out there, they see this as an opportunity for their customers to differentiate their camera solutions.

Other reasons for using a programmable solution instead of a hardware version are the fact that the DSP hardware can be reused for other functions – or can combine DVS with other functions like Super-Resolution.

Their solution provides correction along four axes: the x/y/z directions and then one angular direction: roll (which they call rotation about the Z axis, so I guess the convention is that forward is along the Z axis). They also provide correction for the “Jello effect”: this is distortion that occurs due to a shutter that’s rolling while the image or camera is moving, causing something of a relativistic leaning effect. And they can scale to handle 4K Ultra HD on a single core and adapt to various lighting conditions.

But power is also critical: they say that existing solutions use around 1 W of power; they’re touting less than 35 mW for 1080p30 video when implemented on a 28-nm process.

This new set of libraries could be integrated into an application using their new Application Development Kit (ADK), which they announced at the same time. The ADK is a framework for easing application development and optimization.

One noted feature is called SmartFrame. This allows a developer to operate on an entire video frame while the underlying framework takes care of logistical details. In particular, it can tile up the frame and apply “tunneling” to multiple algorithms, which they refer to as “kernels.”

This tunneling combines a pipeline architecture with the ability to chain multiple kernels together for back-to-back execution. Without tunneling, each kernel would be called by a program, and execution would return to the calling function after the kernel completed so that the program could then call the next kernel.

Instead, the framework allows the first kernel to work on one tile and then hand that tile directly off to the next kernel in the chain while the first kernel starts work on the second tile. And so forth for additional tiles and kernels. This minimizes the amount of data copying needed, and control doesn’t return to the calling program until the entire frame has been processed by all of the kernels.

The ADK also makes it possible to call DSP offloads from CPU programs, something we saw with CEVA’s AMF announcement.

You can find out more about CEVA’s DVS capabilities in their DVS announcement, and there’s more about their ADK in their ADK announcement.

Leave a Reply

featured blogs
Sep 27, 2021
Conduct full-lifecycle IC design reliability verification with PrimeSim Reliability Analysis, a chip design tool ensuring safety in SoC and IC applications. The post A Pioneering Approach to Safety: PrimeSim Reliability Takes Center Stage appeared first on From Silicon To So...
Sep 27, 2021
The latest release of the Tempus Timing Signoff Solution, 21.1, contains a lot of new capabilities. There are far too many to cover in a single blog post'”the customer presentation on Tempus is 90... [[ Click on the title to access the full blog on the Cadence Community sit...
Sep 24, 2021
Wi-Fi, NB-IoT, Bluetooth, LoRaWAN... This webinar will help you to choose the appropriate connectivity protocol for your IoT application....
Sep 23, 2021
The Global Environment Facility Small Grants Programme (GEF SGP), implemented by the United Nations Development Programme, is collaborating with the InnovateFPGA contest. Showcase your  skills with Intel Edge-Centric FPGAs and help develop technical solutions that reduce env...

featured video

Product Update: Complete DesignWare 400G/800G Ethernet IP

Sponsored by Synopsys

In this video product experts describe how designers can maximize the performance of their high-performance computing, AI and networking SoCs with Synopsys' complete DesignWare Ethernet 400G/800G IP solution, including MAC, PCS and PHY.

Click here for more information

featured paper

Authenticate Automotive Endpoints for Genuine Parts

Sponsored by Maxim Integrated (now part of Analog Devices)

Learn how to implement the DS28E40 Deep Cover 1-Wire Authenticator in a system to provide authentication for endpoints such as optical cameras, headlamps, EV Batteries, occupancy sensors, steering wheels, and a myriad of other automotive applications.

Click to read more

featured chalk talk

NEUTRIK Fiber Optic Solutions

Sponsored by Mouser Electronics and Neutrik

The advantages and benefits of fiber optics are a mile long…but how can you design with them? How can you clean them? How do you repair them? Need a bit of a refresher? In this episode of Chalk Talk, Amelia Dalton chats with David Kuklinski from Neutrik about the OpticalCon advanced, OpticalCon LITE and Opticalcon DragonFly fiber optic solutions from Neutrik. They take a closer look at what benefits each of these solutions brings to the table, what kind of configurations are offered with each of these fiber optic solutions and what kind of performance you can expect when using them in your next design.

Click here for more information about Neutrik opticalCON® Fiber Optic Connector System