Last year, a new standard was overlaid on PCI Express (PCIe) to reset the way non-volatile memory (NVM) is accessed. To date, solid-state disk (SSD) access methodologies had been modeled around the existing mechanisms and limitations surrounding “spinning media” – hard drives. As solid-state memories start to proliferate in roles that used to be dominated by hard drives, those limitations and mechanisms change.

The new standard that accomplishes this is called NVM Express (NVMe), and it uses the basics of PCIe to handle moving the data around, since that’s often how these memory subsystems are connected to the CPU subsystem. But the higher layer adapts PCIe to a specific NVM context.

The standard sets up submission and completion queues – up to 64K of them, each of which can hold up to 64K 64-byte commands. Features include:

• End-to-end data protection
• No uncacheable memory-mapped I/O register reads in either the submission or completion path
• No more than one memory-mapped I/O write to submit a command
• Queue priority and arbitration
• Ability to do a 4K-byte read in a single 64-byte command
• A small basic command set (Read, Write, Write Uncorrectable, Flush, Compare, Dataset Mgmt)
• Support for interrupt aggregation (including message-signaled interrupts)
• Multiple namespaces – a device can be decoupled from a “volume”
• Support for I/O virtualization (like SR-IOV)
• Error reporting and management
• Ability to support low-power modes

There are register sets for:

• Declaring what a particular controller supports
• Device failure status
• Configuring an admin queue for managing I/O queues
• Doorbell registers for submission and completion queues

Cadence just announced their NVMe IP offering, which is based on their existing PCIe IP; the NVMe layer is new, along with the firmware needed to support it. They’ve optimized the underlying PCIe implementation for this particular context, making the overall implementation smaller. They’ve merged the APIs up to the top level so that there is one interface regardless of which layer might be accessed by any given operation. They’ve also coordinated their DMAs for smoother operation and less contention.

They’ve hardware-accelerated the basic commands; the command set itself can be extended through the firmware.

The PCIe PHY is hard IP; the rest is RTL and firmware. They’ve got a tool to configure the IP via an XML description that describes the configuration to their implementation tools.

You can find out more about Cadence’s NVMe IP in their announcement.