Hu Yoshida

Flash Investments for the Long Term

Blog Post created by Hu Yoshida Employee on Nov 2, 2015

Over the last few years, I have been writing a series of blogs on IT trends for the coming new year. I will be beginning my series for 2016 next month. One of the things I predicted last year was the displacement of Solid State Devices (SSD) designed for commodity markets by flash modules that are enhanced with the processing power required to address enterprise storage requirements for performance, durability and increased capacity. Unlike disk drives, flash drives require a lot of software and a lot of processing power for mapping blocks to pages, wear leveling, extended ECC, data refresh, housekeeping, and other management tasks. No matter what you do in the storage controller, most of these functions have to be done at the flash device level. This year we are beginning to see more specialized flash devices as the demand for larger capacity and, consequently, lower-cost flash devices increases.


Having direct access to the hardware engineering resources of Hitachi Ltd., Hitachi Data System (HDS) delivered a specialized flash module in 2012 with a capacity of 1.6 TB when SSDs were still limited to 400 and 600 GB. Unlike other vendors who used standard SSD’s, which were designed for use with commodity systems like PCs, Hitachi Data Systems developed their own flash module designed specifically for the enterprise storage market with a quad-core processor that had enough power to support a usable flash capacity of 1.6TB and deliver four times the sustained throughput of the current generation solid-state drives that are based on multi-level cell (MLC) technology. A year later, in 2013, HDS began delivering 3.2 TB flash modules and as of the end of 2Q 2015 we have shipped over 210 PB of Flash capacity. While we are not currently in the all-flash array category, which Gartner has defined as flash arrays that cannot support tiering to hard disks, we have many customers that have all flash G1000 storage arrays.  Our shipments of flash capacity shows that we are the leader in enterprise flash deployment. We also support SSDs as shown in the following chart.



It is harder for smaller companies and even large companies like EMC to make the investments to build or customize their flash modules for enterprise storage. That is the primary handicap of all flash array vendors that use commodity SSDs. Their ability to differentiate their flash offering is limited to what they can do in the software of the storage controller. The only way that they can communicate with the flash module is through a SAS or SATA interface. Since Hitachi builds the storage controller as well as the flash device controller, we can offload functions to the device and initiate actions in the device that are not possible through standard interface commands. (See my previous post on data eradication for flash devices, which we initiate and report back through our storage controller: What Do You Do With Your Old Flash Drives?). Think of what we could do if we could do other functions in the flash controller, like compression. You would not have the overhead of compression/decompression in the storage controller and we would be able to keep it on all the time without impact to performance.


Even on the storage controller side, the AFA vendors lack the capability to provide enterprise functions like disaster recovery, or dual active availability.  The much-anticipated Pure Storage IPO debuted in October but ended the day down 5.8% from their IPO price of $17. Pure Storage uses commodity SSDs, but they are not profitable since they address only a subset of the enterprise market. Their stock did jump up past their IPO price recently as a ripple effect from the Dell buyout of EMC. Analysts attribute this increase to the theory that emerging storage vendors like Pure will be acquisition targets by larger companies. If the goal of an emerging storage company is to be acquired, how much investment will they be making for long-term development of enterprise storage capabilities?


As we go to larger capacity flash devices, whether we do that by adding more MLC dimms, triple level cell (TLC), or 3D NAND technology, we will need more management and functions in the flash device, and that means specialized flash devices as well as enhancements to enterprise storage controllers. The new larger capacity flash drives will be comparable to 15K high performance disk drives on a capacity cost basis, and will eliminate the need for these types of disk drives. As we increase the capacity of flash devices, costs will decrease and flash will displace more and more of the disk drive market. The end for disk drives is in sight.


Flash itself will give way to more durable and higher performance nonvolatile technologies in the future. Success will depend on the ability to develop new media controllers as well as storage controllers that will be able to integrate these technologies to address future business requirements. Hitachi Data Systems plans to be here for the long term and continue to invest in the research and development of current and future systems.