A Closer Look at Software-Defined Storage
Software-defined storage is one of the industry’s newest buzzwords, so I thought I would take a moment to explain what the term means.
Why We Started SwiftStack
The genesis for starting SwiftStack in 2011 was the pain we experienced first hand from deploying, managing and using vendor-specific storage systems. As users, what we wanted was more flexibility, less lock-in, better control and lower costs than what traditional storage systems could provide. We also heard the same thing from other organizations - and with data growing dramatically (but not IT budgets), there was absolute certainty that the pain would only exacerbate over time for not just us, but for pretty much everyone who stored and served data at scale.
So how did we end up here? Most existing storage systems are single, integrated systems. Over the years, they have gotten easier to deploy and frankly, quite reliable. It has evolved in this manner because the problem domain is constrained, and also because these systems only support specific hardware and software combinations. The problem, however, is that these storage systems are confined to a vendor-defined operating system and vendor-provided hardware. This means that system capacity growth lags behind the general hardware market and you are locked out of decreasing hardware prices. A self-contained storage system also lacks flexibility; it cannot take advantage of new hardware capabilities and platforms, and expansion is limited due to head unit offerings by the vendor.
Enter Software Defined Storage
When we started SwiftStack, our big idea was to provide an object storage system - OpenStack Swift - with a de-coupled management system so customers could achieve (1) amazing flexibility on how - and where - they deployed their storage, (2) control of their data without being locked-in to a vendor and (3) private storage at public cloud prices. At the time, we didn’t call it software defined storage (frankly, no one did) but we think the term perfectly illustrates the fundamental change this model represents.
Software Defined storage (SDS), which decouples software from the hardware, has many benefits. Customers can instantly take advantage of the changing hardware landscape and add newer components to expand performance and capacity as needed. But the difference from I think how we think of SDS from others is that needs to be more than running storage software on industry standard hardware. For to be truly software-defined, we believe the control - and the data access - also needs to be de-coupled from the underlying storage hardware. Through this de-coupling, customers can now make choices on how their storage is scaled and managed and how users can store and access data - all driven programmatically for the entire storage tier, independently where the storage resources are deployed. In addition, this allows organizations to deploy a storage infrastructure that is not just API compatible with public cloud storage, but architecturally identical and at lower cost.
So what are some of the main characteristics of SDS?
Typically, IT operations teams have already built up a tremendous amount of tools around a particular hardware vendor and operating system for their computing infrastructure. This includes everything from procurement process, racking/stacking, OS provisioning, operational support systems, etc. Software-defined storage systems enable IT teams to utilize infrastructure components that are already in place - and apply that to their storage tier. In this sense, it is just an extension of what IT operations teams are already doing. By leveraging the existing OS and familiar hardware, this greatly reduces complexity.
SDS shifts reliability to the software
With reliability shifted to the software, any single component can fail but the data will remain available and durable. SDS allows a storage system to span to a tightly-integrated storage stack and out to multiple nodes and racks – even multiple data centers.
Decoupled controller enables SDS
SDS can be centrally managed with no limits on cluster size. A decoupled controller coordinates all the nodes that form a complete storage platform. This is done by orchestrating data placement and using a single pane of glass for management of roles and health of each node and cluster. Capacity can be added by adding nodes, and likewise, old equipment can be seamlessly decommissioned. Rolling upgrades for both hardware and software are seamless.
So Who Needs SDS?
With all the buzz going on about SDS now there is no doubt some level of confusion in the market. The organizations who would benefit most from an SDS solution have a growing data set, an increasing number of users (internal or external) and probably a flat IT budget. From our experience here at SwiftStack, a LOT of organizations fall into this category but the early adopters are typically web, SaaS and mobile application companies - and enterprise organizations moving to a storage as-a-service model for their applications and users. What all these users have in common, however, is that they - as we did before starting SwiftStack - is to get more flexibility, less lock-in, better control and lower costs than what traditional storage systems can provide. With SwiftStack’s SDS solution, they can achieve that.