feature article
Subscribe Now

Keeping it All in Memory

When you think “database,” a number of images might come to mind.

  • Giant servers with multiple IT experts required to make even the most minute of changes without breaking the entire thing and shutting the company down for weeks while it’s all rebuilt from scratch.
  • The outsized egos of self-styled demi-gods braving Tasmanian cyclones, breaking nary a bead of sweat.
  • Monster teams of consultants required to implement what you want in an app overlaid on a monster database on a monster schedule. (Which will slip.)
  • Or, on a more modest scale, you may have nightmares about being chased by the ugly, multi-appendaged DoCmd object, where you can’t really tell head from tail and aren’t really sure if you’re being eaten (but you’re pretty sure you’re being corrupted – and not in a good way).

Most of us have interacted with a database at some point. In fact, anyone who uses the internet will have, unless he or she exclusively visits old, static sites. But we’ve used more databases in more places than we might expect.

That call you made a few minutes ago on your smartphone? When you looked up the contact? Database. On the phone. Oh, and the call details while you’re in the middle of the call to be sure you’re billed “accurately”? Also a database – not a big, lumbering one, but one small and agile enough to maintain lots of in-process calls (and likely with an extremely efficient custom-crafted round-up function) before sending the info to a big, lumbering database when the call ends.

This is a special category, the in-memory database, that’s finding increasing use in embedded systems. Including very small embedded systems.

As the name implies, these databases aren’t stored on huge, redundant servers: they’re kept in live memory. This means that they’re optimized completely differently: disk I/O is no longer an issue.

It also means they’re not persistent. They’re like the opposite end of the spectrum from a RAID server: if the power goes out, so do they. Actually, that’s not necessarily true, since your contacts don’t disappear if you turn off your phone – but that’s because the “memory” in that case is flash.

Often these form “working” databases for use when you don’t need a persistent record of all of the working details of whatever the database is supporting.  And they can typically be rebuilt if something goes wrong, just as routing tables are rebuilt if a router goes down.

And they’re most useful when database transactions and lookups need to be fast: when setting up a call, things can’t take forever. (Even though it may seem that they do. In the old landline days when Bell ruled the phones, a call had to be set up in a specified time, a time far shorter than what cell phones do today. But… alas… that was back in the days when a voice sounded like, well, a voice…)

Not all databases for embedded systems are purely in-memory: there are also hybrid databases, where some tables are in memory and others are persistent.

There are actually a lot of players in this space – Wikipedia lists 33 companies, for what that’s worth. At first glance, one company I spoke to that claims to have a faster and smaller implementation than the others – McObject – would appear to be missing from the list. Then I saw that extremeDB – the product name, not the company name – is on the list. But… it’s red-linked – its page has been deleted.

(I even read through the debate on why it was deleted. Someone didn’t like the fact that company people wrote it – I’ve gotten in trouble for this myself, although not deleted – and the fact that there were links to EE Times and Dr. Dobbs articles, which were described as “spam spam spam.” Someone describing himself as a customer offered to rewrite it given specific examples that would pass muster… that was not provided, and the page was deleted. Ah… democracy in action… “No, I’m not going to tell you what I want, I’m simply going to tell you I don’t like what I have…” Budding CEO perhaps?)

I had a chance to talk with McObject’s Steve Graves at ESC. As a case in point, they claim a code footprint of 150K or less (I know, for some of you anything over 256 bytes is bloat…) and claim read/write accesses of a few microseconds. They attribute their speed to a couple things (other than memory-residence): they work directly with data stored in the form needed by the application, so there’s no futzing around with formats; and they have a native query language that’s faster than SQL. They have an SQL implementation, but it’s slower because the queries have to be parsed and interpreted.

At ESC, they were touting a new clustering capability. This is targeted at telecom guys, where numerous systems might be working off of the same database. Each gets a local copy to work from, and replication across the cluster is managed transparently by the runtime system. This also works with their MVCC capability, which eliminates locks. Since everyone’s working on a copy, there’s no need for locked doors. The only issue comes when more than one person makes a change to the same record. One will get in; the other(s) will be rejected. So coders need to use a loop on each transaction to retry if it doesn’t work the first time.

Their ACID claims (a kind of acid test for databases) may be a bit off the mark: they can presumably handle atomicity, consistency, and isolation (A, C, and I), but their very in-memory nature would suggest that they’re not durable (the D).

They also claim flexibility for those building a database. The tables don’t have to be normalized, reducing design time; they can also include structures in the tables. They support a variety of indexing schemes for different applications, including “R-trees,” useful for geographic information; “Patricia” trees (or tries), used for string searches – in particular, IP addresses; and k-d trees, used for multi-dimensional searches.

That variety of indexing for different applications highlights the range of uses that these kinds of databases are addressing. While a lot of this stuff might just seem like stupid, old-fashioned table look-up, in fact a table look-up is really a degenerate database query. (No, I’m not talking about Megan’s law databases…) If databases are big, huge, monstrous things to manage, then a table look-up seems like a nice way to go.

On the other hand, if databases are easy, small, and fast, then you can simplify your world by making everything look like a database.

As long as it’s not the kind that visits you by night…

Leave a Reply

featured blogs
Jan 15, 2021
I recently saw (what appears at first glance to be) a simple puzzle involving triangles. But is finding the solution going to be trickier than I think?...
Jan 15, 2021
It's Martin Luther King Day on Monday. Cadence is off. Breakfast Bytes will not appear. And, as is traditional, I go completely off-topic the day before a break. In the past, a lot of novelty in... [[ Click on the title to access the full blog on the Cadence Community s...
Jan 14, 2021
Learn how electronic design automation (EDA) tools & silicon-proven IP enable today's most influential smart tech, including ADAS, 5G, IoT, and Cloud services. The post 5 Key Innovations that Are Making Everything Smarter appeared first on From Silicon To Software....
Jan 13, 2021
Testing is the final step of any manufacturing process, and arguably the most important, and yet it can often be overlooked.  Releasing a poorly tested product onto the market has destroyed more than one reputation for quality, and this is even more important in an age when ...

featured paper

Overcoming Signal Integrity Challenges of 112G Connections on PCB

Sponsored by Cadence Design Systems

One big challenge with 112G SerDes is handling signal integrity (SI) issues. By the time the signal winds its way from the transmitter on one chip to packages, across traces on PCBs, through connectors or cables, and arrives at the receiver, the signal is very distorted, making it a challenge to recover the clock and data-bits of the information being transferred. Learn how to handle SI issues and ensure that data is faithfully transmitted with a very low bit error rate (BER).

Click here to download the whitepaper

Featured Chalk Talk

Nano Pulse Control Clears Issues in the Automotive and Industrial Markets

Sponsored by Mouser Electronics and ROHM Semiconductor

In EV and industrial applications, converting from high voltages on the power side to low voltages on the electronics side poses a big challenge. In order to convert big voltage drops efficiently, you need very narrow pulse widths. In this episode of Chalk Talk, Amelia Dalton chats with Satya Dixit from ROHM about new Nano Pulse Control technology that changes the game in DC to DC conversion.

More information about ROHM Semiconductor BD9V10xMUF Buck Converters