editor's blog
Subscribe Now

The Essence of Big Data

iStock_000064716741_Small.jpgThe biggest buzzword that every press release must reference in the title these days is “Internet of Things” (IoT). The second biggest buzzword would appear to be “Big Data”. (Although the IoT uses Big Data, resulting in a re-entrant ranking problem that’s too much for my brain after two conferences this week.)

The question I’ve struggled with, however, is, “What is Big Data?” It’s almost as hard as, “What is the IoT?” One answer might be, “A vague concept that helps make your product sound more sophisticated and leading-edge, if you can pull it off.” But, while possibly true, that’s not particularly helpful.

There could be many nuanced aspects to Big Data, so I’m going to zoom way out and define – OK, maybe not define, but characterize – it via metaphor.

You see… you’ve had this problem, albeit well under control. You think no one has noticed, but we have… we just haven’t said anything. You have an… acquisitive nature. Yeah, we see that UPS truck show up. Again and again. (You were probably bummed that you can’t arrange after-dark delivery.) And over time you ran out of space in your home. So you had to get a storage space for much of your junk. (Yeah, I went there… I called it junk. Am I wrong?)

But, being a person of foresight, you asked the obvious question: If I put this into storage and never get it again, why have it in the first place? That’s a question you probably don’t want to answer honestly, in that it would result in a rather dramatic lifestyle reevaluation. So instead, you ran with the fantasy that you will, in fact, make frequent trips to your little attic-away-from-the-attic to get stuff. And you wanted to be able to do so without rummaging; takes too long and leaves a mess.

So you thought through what would go into the storage space. And you designed and had installed shelving specifically sized for the different things. And you labeled the shelves, numbering positions and levels, and every time you put something in there (which was pretty often, but manageable), you took careful note of where it went.

And anytime you wanted to get something (it did really happen occasionally), you could simply go to the logbook index and see where the item was and retrieve it with nary a bead of sweat raised.

Then came the Difficult Times. Your Mad Uncle Tito passed (tough in its own right), but upon being bestowed the honor of managing his affairs, you discovered that your propensity for Getting and Keeping was genetic. Only yours was diluted as compared to the Mad Uncle.

Not only did he acquire stuff; he acquired houses too. And each of the houses was packed to the gills with stuff. It looked like some of it had value; simply solving the problem with a bulldozer and front loader felt rash. So you rented another very large storage locker and went about trying to move stuff into there.

The problem with your system is that you have to hire professionals to build the shelving and arrange things just so. And it takes people and time to do all the cataloguing and moving. It worked for your own stuff, but for his stuff, well, it just seemed overwhelming.

And that wasn’t even the hard part. When you designed your own shelving, you knew, approximately, what kind of stuff was going to go there. Because it was your stuff. But you had no idea what might be found lurking in Tito’s many closets and under his bed and in his basements. It could be anything. And, for the same reason, you had no idea what you might want to get at in the future.

It’s the fundamental problem of storing Other People’s Stuff. (You down with OPS?)

It hurt you to the very core, but you had to make a strong decision. There was no way to do this in an organized fashion. The houses needed to be emptied and sold faster than you could neatly arrange the contents. So you simply hired some cheap labor to load trucks with stuff. Stuff loaded any old way. And in the storage locker, you simply put it in a pile.

Perhaps you made multiple piles – one for each house, or furniture over here and state plates over there, mixed with other stoneware and flatware and bad hotel art. So you might have created a patina (or illusion) of organization, but that’s it.

And you locked the door and called it good.

And when you wanted to actually find stuff, well, you hired folks that were good at finding stuff. So many people had so much stuff that this had become a new cottage industry, and different companies specialized in finding different kinds of things. Those guys over there were good at finding clothing; that other group was good at finding LPs. (No one had yet cracked the problem of finding remotes.) They joked that they could practically create a market out of the stuff they found, and, in fact, they referred to themselves generically as “marts.”

And that, to me, is the essence of Big Data, as compared to Ye Olde Relaytionale Databayse. It’s a big ol’ pile of Other People’s Stuff. Schema schmema. Perhaps with a few tags and flags here and there so that you can tell which house it came from or which stuff was more likely to be state plates. Other than that, you don’t mess with it, and you damn sure don’t throw anything away. And when you need something, you overlay with a datamart to extract any good bits.

Which, naturally, you use to improve your advertising targeting. Cuz we’re all just dying to receive more advertising – especially if it has our name on it. Makes us feel special.

Leave a Reply

featured blogs
Dec 6, 2023
Optimizing a silicon chip at the system level is crucial in achieving peak performance, efficiency, and system reliability. As Moore's Law faces diminishing returns, simply transitioning to the latest process node no longer guarantees substantial power, performance, or c...
Dec 6, 2023
Explore standards development and functional safety requirements with Jyotika Athavale, IEEE senior member and Senior Director of Silicon Lifecycle Management.The post Q&A With Jyotika Athavale, IEEE Champion, on Advancing Standards Development Worldwide appeared first ...
Nov 6, 2023
Suffice it to say that everyone and everything in these images was shot in-camera underwater, and that the results truly are haunting....

featured video

Dramatically Improve PPA and Productivity with Generative AI

Sponsored by Cadence Design Systems

Discover how you can quickly optimize flows for many blocks concurrently and use that knowledge for your next design. The Cadence Cerebrus Intelligent Chip Explorer is a revolutionary, AI-driven, automated approach to chip design flow optimization. Block engineers specify the design goals, and generative AI features within Cadence Cerebrus Explorer will intelligently optimize the design to meet the power, performance, and area (PPA) goals in a completely automated way.

Click here for more information

featured paper

3D-IC Design Challenges and Requirements

Sponsored by Cadence Design Systems

While there is great interest in 3D-IC technology, it is still in its early phases. Standard definitions are lacking, the supply chain ecosystem is in flux, and design, analysis, verification, and test challenges need to be resolved. Read this paper to learn about design challenges, ecosystem requirements, and needed solutions. While various types of multi-die packages have been available for many years, this paper focuses on 3D integration and packaging of multiple stacked dies.

Click to read more

featured chalk talk

Optimize Performance: RF Solutions from PCB to Antenna
RF is a ubiquitous design element found in a large variety of electronic designs today. In this episode of Chalk Talk, Amelia Dalton and Rahul Rajan from Amphenol RF discuss how you can optimize your RF performance through each step of the signal chain. They examine how you can utilize Amphenol’s RF wide range of connectors including solutions for PCBs, board to board RF connectivity, board to panel and more!
May 25, 2023
23,535 views