editor's blog
Subscribe Now

The Essence of Big Data

iStock_000064716741_Small.jpgThe biggest buzzword that every press release must reference in the title these days is “Internet of Things” (IoT). The second biggest buzzword would appear to be “Big Data”. (Although the IoT uses Big Data, resulting in a re-entrant ranking problem that’s too much for my brain after two conferences this week.)

The question I’ve struggled with, however, is, “What is Big Data?” It’s almost as hard as, “What is the IoT?” One answer might be, “A vague concept that helps make your product sound more sophisticated and leading-edge, if you can pull it off.” But, while possibly true, that’s not particularly helpful.

There could be many nuanced aspects to Big Data, so I’m going to zoom way out and define – OK, maybe not define, but characterize – it via metaphor.

You see… you’ve had this problem, albeit well under control. You think no one has noticed, but we have… we just haven’t said anything. You have an… acquisitive nature. Yeah, we see that UPS truck show up. Again and again. (You were probably bummed that you can’t arrange after-dark delivery.) And over time you ran out of space in your home. So you had to get a storage space for much of your junk. (Yeah, I went there… I called it junk. Am I wrong?)

But, being a person of foresight, you asked the obvious question: If I put this into storage and never get it again, why have it in the first place? That’s a question you probably don’t want to answer honestly, in that it would result in a rather dramatic lifestyle reevaluation. So instead, you ran with the fantasy that you will, in fact, make frequent trips to your little attic-away-from-the-attic to get stuff. And you wanted to be able to do so without rummaging; takes too long and leaves a mess.

So you thought through what would go into the storage space. And you designed and had installed shelving specifically sized for the different things. And you labeled the shelves, numbering positions and levels, and every time you put something in there (which was pretty often, but manageable), you took careful note of where it went.

And anytime you wanted to get something (it did really happen occasionally), you could simply go to the logbook index and see where the item was and retrieve it with nary a bead of sweat raised.

Then came the Difficult Times. Your Mad Uncle Tito passed (tough in its own right), but upon being bestowed the honor of managing his affairs, you discovered that your propensity for Getting and Keeping was genetic. Only yours was diluted as compared to the Mad Uncle.

Not only did he acquire stuff; he acquired houses too. And each of the houses was packed to the gills with stuff. It looked like some of it had value; simply solving the problem with a bulldozer and front loader felt rash. So you rented another very large storage locker and went about trying to move stuff into there.

The problem with your system is that you have to hire professionals to build the shelving and arrange things just so. And it takes people and time to do all the cataloguing and moving. It worked for your own stuff, but for his stuff, well, it just seemed overwhelming.

And that wasn’t even the hard part. When you designed your own shelving, you knew, approximately, what kind of stuff was going to go there. Because it was your stuff. But you had no idea what might be found lurking in Tito’s many closets and under his bed and in his basements. It could be anything. And, for the same reason, you had no idea what you might want to get at in the future.

It’s the fundamental problem of storing Other People’s Stuff. (You down with OPS?)

It hurt you to the very core, but you had to make a strong decision. There was no way to do this in an organized fashion. The houses needed to be emptied and sold faster than you could neatly arrange the contents. So you simply hired some cheap labor to load trucks with stuff. Stuff loaded any old way. And in the storage locker, you simply put it in a pile.

Perhaps you made multiple piles – one for each house, or furniture over here and state plates over there, mixed with other stoneware and flatware and bad hotel art. So you might have created a patina (or illusion) of organization, but that’s it.

And you locked the door and called it good.

And when you wanted to actually find stuff, well, you hired folks that were good at finding stuff. So many people had so much stuff that this had become a new cottage industry, and different companies specialized in finding different kinds of things. Those guys over there were good at finding clothing; that other group was good at finding LPs. (No one had yet cracked the problem of finding remotes.) They joked that they could practically create a market out of the stuff they found, and, in fact, they referred to themselves generically as “marts.”

And that, to me, is the essence of Big Data, as compared to Ye Olde Relaytionale Databayse. It’s a big ol’ pile of Other People’s Stuff. Schema schmema. Perhaps with a few tags and flags here and there so that you can tell which house it came from or which stuff was more likely to be state plates. Other than that, you don’t mess with it, and you damn sure don’t throw anything away. And when you need something, you overlay with a datamart to extract any good bits.

Which, naturally, you use to improve your advertising targeting. Cuz we’re all just dying to receive more advertising – especially if it has our name on it. Makes us feel special.

Leave a Reply

featured blogs
May 20, 2022
This year's NASA Turbulence Modeling Symposium is being held in honor of Philippe Spalart and his contributions to the turbulence modeling field. The symposium will bring together both academic and... ...
May 19, 2022
Learn about the AI chip design breakthroughs and case studies discussed at SNUG Silicon Valley 2022, including autonomous PPA optimization using DSO.ai. The post Key Highlights from SNUG 2022: AI Is Fast Forwarding Chip Design appeared first on From Silicon To Software....
May 12, 2022
By Shelly Stalnaker Every year, the editors of Elektronik in Germany compile a list of the most interesting and innovative… ...
Apr 29, 2022
What do you do if someone starts waving furiously at you, seemingly delighted to see you, but you fear they are being overenthusiastic?...

featured video

Synopsys PPA(V) Voltage Optimization

Sponsored by Synopsys

Performance-per-watt has emerged as one of the highest priorities in design quality, leading to a shift in technology focus and design power optimization methodologies. Variable operating voltage possess high potential in optimizing performance-per-watt results but requires a signoff accurate and efficient methodology to explore. Synopsys Fusion Design Platform™, uniquely built on a singular RTL-to-GDSII data model, delivers a full-flow voltage optimization and closure methodology to achieve the best performance-per-watt results for the most demanding semiconductor segments.

Learn More

featured paper

Reduce EV cost and improve drive range by integrating powertrain systems

Sponsored by Texas Instruments

When you can create automotive applications that do more with fewer parts, you’ll reduce both weight and cost and improve reliability. That’s the idea behind integrating electric vehicle (EV) and hybrid electric vehicle (HEV) designs.

Click to read more

featured chalk talk

Traveo II Microcontrollers for Automotive Solutions

Sponsored by Mouser Electronics and Infineon

Today’s automotive designs are more complicated than ever, with a slew of safety requirements, internal memory considerations, and complicated power issues to consider. In this episode of Chalk Talk, Amelia Dalton chats with Marcelo Williams Silva from Infineon about the Traveo™ II Microcontrollers that deal with all of these automotive-related challenges with ease. Amelia and Marcelo take a closer look at how the power efficiency, smart IO signal paths, and over the air firmware updates included with this new MCU family will make all the time-saving difference in your next automotive design.

Click here for more information about Cypress Semiconductor Traveo™ II 32-bit Arm Automotive MCUs