feature article
Subscribe Now

Realistic Edge-Device Testing

KnowThings Stresses the System – Intentionally

We have so many electronic devices in our lives these days. OK, file that under “D” for “Duh!!!,” but how many different kinds do we have? Well… phones, of course. Tablets? Check… Computers? You bet.

After those, things are less obvious. A smart watch? OK… mostly that’s a phone that looks like a watch (and maybe without the phone capabilities, but… who uses a phone as a phone anymore anyway??). But OK… next? (I’m talking regular folks, not buy-all-the-techy-things folks from inside the Silicon Valley bubble…)

Now, if you look into industrial applications, you start to get a lot more. Obviously, folks delivering packages have electronic gadgetry that lets you sign for receipt and lets them keep track of where all those packages go. You see city infrastructure folks with electronic devices that can measure where wires and cables and power are located underground. There are the machines that measure what’s happening within our cars. Etc. etc. etc.

So most of us use one of a dizzying number of available brands of a few kinds of items. Industry uses many more specialized, purpose-built devices. The question is, how do you test to be sure that all of these things communicate correctly and with the desired performance?

So Many Tests, So Little Time

For us consumers, it’s not so hard (tell that to the testing guy though…). You’ve got a very few protocols to worry about carrying generic data from things like browsers, apps, and phone calls. So you can test the bejeebus out of them. How do you model the traffic? Well, since there are only a few classes of platform, you can expend the effort to build a purpose-built data generator for use in testing – that is, if you can’t simply plug into a real data source coming out of the wall or the air.

What about those more specific industrial devices? Well, there you have a moderate number and a limited audience; you can probably also model that traffic pretty clearly.

So here we have either wide scope with a limited number of platforms or narrow scope and a greater number of platforms (although probably not a crazy number). What’s going to happen if the Internet of Things (IoT) explodes for consumers the way it’s been predicted? Or even for industry? Now we’ll have a wide scope and a huge number of purpose-built widgets, each of which is likely to have some level of proprietary behavior even if it uses industry-standard protocols for most network layers.

The challenge here is primarily scaling and stress: you can test a single unit and have it work just fine, but what if that single unit suddenly becomes a thousand or a million or more? Can the infrastructure handle the load with the required performance?

With network layers below the application layer, performance may be governed by well-known protocols. But at the app layer, the data patterns will be specific to that applications. Yes, on a phone, an “app” is but one program on a generic computing platform. But with the IoT, an app is an entire device with a dedicated purpose. Does every device maker have to spend the time and effort to design and validate testing models for each unit they design?

What’s more likely is that you build a few prototype units and test them. All well and good, but how do you test the device’s ability to scale? You don’t want to have to built thousands of prototypes for stress testing.

Or perhaps you’re adapting existing industrial equipment for the IoT – a so-called brownfield installation. Now thousands of devices exist, but they’re already installed at sites around the world. Do you have to buy a couple thousand to run the stress tests?

Traditionally, you would now be consigned to the woodshed to spend valuable time developing bespoke data generators rather than working on the next product that you can sell. But, instead of doing that, might you be able to summon up the Thing That Will Save Everything? Machine learning?

Stressing Out

KnowThings thinks that you can. They’ve got a system for generating models automatically, using AI to build the model.

There are two pieces to the models they can help you to simulate:

  • The network itself, down to the physical layer. This is a one-time effort that they say is pretty easy.
  • What they call the data acquisition module – that is, the IoT edge device – or the portion of it that senses something and then sends the data up. This model isn’t so easy to build manually. (You can, of course, but it can be a lot of work.)

AI’s role is to listen to data from a device. The model generator will identify the patterns in the data through a variety of usage scenarios, categorizing the various packets with machine-generated tags. You can then go in and edit those tags to make them more human-digestible.

(Image courtesy KnowThings)

But what if you don’t have a device yet? They’re also working on a manual modeling capability using a simple language to describe the model. Initially, components included in the model will be generic – that is, they won’t reflect the detailed behavior of one brand of device vs. another. But their vision includes getting to that level of detail, meaning that you could do, for example, second-source evaluations.

So now you have two capabilities: manual, for when you don’t have built hardware yet, and automatic, for when you do. The third capability manages the transition: letting the manual model inform and provide semantic clues to the Ai process as you go from a virtual unit to a real unit.

What you get with these models is the ability to push your devices and system to see where (or if) they break. But let’s say that you have a model with human-understandable tags so that you can go in and inspect the results of a data capture. What happens if some unknown or, even worse, unauthorized traffic comes through?

Unexpected or novel data patterns will still be characterized; they just won’t have a nice, easy-to-read tag for what they represent. They’ll be mystery patterns. You can then go figure out whether they represent some mode or use case that you hadn’t run across yet. If not, then it may indicate a security issue – someone injecting malicious traffic into your stream.

So if this pans out as promised, you may have an easier way to test your edge devices. Of, if the effort has had you skimping on the testing, you can skimp no more.

 

More info:

KnowThings

One thought on “Realistic Edge-Device Testing”

Leave a Reply

featured blogs
Oct 23, 2020
Processing a component onto a PCB used to be fairly straightforward. Through-hole products, or a single or double row surface mount with a larger centerline rarely offer unique challenges obtaining a proper solder joint. However, as electronics continue to get smaller and con...
Oct 23, 2020
[From the last episode: We noted that some inventions, like in-memory compute, aren'€™t intuitive, being driven instead by the math.] We have one more addition to add to our in-memory compute system. Remember that, when we use a regular memory, what goes in is an address '...
Oct 23, 2020
Any suggestions for a 4x4 keypad in which the keys aren'€™t wobbly and you don'€™t have to strike a key dead center for it to make contact?...
Oct 23, 2020
At 11:10am Korean time this morning, Cadence's Elias Fallon delivered one of the keynotes at ISOCC (International System On Chip Conference). It was titled EDA and Machine Learning: The Next Leap... [[ Click on the title to access the full blog on the Cadence Community ...

featured video

Demo: Low-Power Machine Learning Inference with DesignWare ARC EM9D Processor IP

Sponsored by Synopsys

Applications that require sensing on a continuous basis are always on and often battery operated. In this video, the low-power ARC EM9D Processors run a handwriting character recognition neural network graph to infer the letter that is written.

Click here for more information about DesignWare ARC EM9D / EM11D Processors

featured paper

Designing highly efficient, powerful and fast EV charging stations

Sponsored by Texas Instruments

Scaling the necessary power for fast EV charging stations can be challenging. One solution is to use modular power converters stacked in parallel. Learn more in our technical article.

Click here to download the technical article

Featured Chalk Talk

RX23W Bluetooth

Sponsored by Mouser Electronics and Renesas

Adding Bluetooth to your embedded design can be tricky for IoT developers. Bluetooth 5 brings a host of new capabilities that make Bluetooth integration more compelling than ever. In this episode of Chalk Talk, Amelia Dalton chats with Michael Sarpa from Renesas about the cool capabilities of Bluetooth 5, and how you can easily integrate them into your next project.

More information about Renesas Electronics RX23W 32-bit Microcontrollers