The data storage and backup sectors used to be the sleepiest parts of the computer business. The pace of innovation was relatively slow, with inc
remental improvements, primarily in capacity, coming every few years. While the performance of every other major computing subsystem – CPU, network, backplane, etc. – improved exponentially, disk speeds hardly improved at all. Over time, disk speed became a prohibitive bottleneck to business IT systems. In order to cope with this, it became traditional to address disk performance requirements by piling up as many disks – aka spindles – in arrays as were needed to meet the anticipated speed requirements of the computers connected to them. The math was pretty simple: If, for example, your database servers needed 10,000 IOPS (I/O operations per second) and the fastest spindles delivered 200 IOPS each, then you needed 50 spindles to meet the speed requirement, along with an array controller capable of managing that many drives. If the smallest capacity spindles that got 200 IOPS were 300 GB each, this meant you were buying 15 TB of storage (300 GB x 50) even if you might have only needed two or three TB to store the data from your servers. This is called over-provisioning, and it had become a way of life for many IT shops who were forced to buy capacity they would never use in order to get the speed they required. I once had a customer who was implementing an early version of a virtual desktop environment, with 1500 virtual workstations. Because of the high performance requirements associated with virtual desktops, the cost of the storage for the project was over $300,000. He got the speed he needed but he also bought capacity he would never use.
There are three key problems with over-provisioning, all relating to cost:
- Acquisition costs. Storage arrays were (and are) expensive, especially back in the days when some vendors were charging over $5,000 per TB of capacity.
- Operational costs. A storage array with lots of spinning disks uses a large mount of power, which is one of the key variable-cost challenges facing IT shops who are often under pressure from management to reduce Opex costs. Support and maintenance costs also tend to be high; the more complex a system is, the more parts that can break.
- The cost of space. IT shops have seen themselves under constant pressure to reduce their physical footprint, especially as more and more businesses migrate to co-location facilities where they are charged for the real estate they occupy. A large storage array takes up a lot of rack space.
The good news is that the storage world, especially in the last two years, has finally begun to rock & roll, driven by two key factors:
- The relatively economical viability of flash memory, either in the form of Solid State Disk (or SSD) or as flash-based cache. Flash has the tantalizing characteristics of extreme speed – 50 times (or more) faster than the fastest spinning disks – plus high physical density and low pow
er consumption. - The massive computing power of newer CPUs from Intel, AMD, etc. has enabled the design of very high performance storage controllers that can do things that were literally impossible a few years ago because of the associated performance costs, such as compression, deduplication and encryption.
In recent years flash memory costs have declined dramatically but, compared to disk costs, remain rather high. In response some very smart storage vendors such as Nimble Storage and NexGen Storage have engineered what are called “hybrid” storage arrays, which use Flash and inexpensive spinning disk in tandem, all governed by high-power controllers with multi-core processors. They have designed controller software that, put simply, uses Flash (or SSD) for stuff that benefits from being fast and regular disk for stuff that doesn’t. In addition, the horsepower of the CPUs allows the arrays to do inline compression or deduplication, which allows the storage of more data on far less disk space. The result is that these hybrid arrays manage the neat trick of being faster, physically denser and greener all at once, and not by just a little bit. They are also priced in such a way that smelling salts are not required when viewing a quote. In the backup space, Unitrends, the premier supplier of backup appliances, has come out with the Recovery-943 a flash-enabled hybrid backup appliance, which is the densest, greenest large-scale backup appliance ever built.
So now, not only are the storage and backup sectors no longer the sleepiest parts of the computer business, but some companies within these sectors are rocking and rolling, leading the way in the IT industry’s constant quest for better, faster & cheaper solutions.
For more information please contact Southern Data Storage at (888) 255-6889 or email me at mark@southerndatastorage.com.