Hu Yoshida

Enter the Brave New World of Storage with HCP S10

Blog Post created by Hu Yoshida Employee on Jan 29, 2015

One of the new features that was introduced in our January Hitachi Content Portfolio announcement was an Innovative new storage system, the HCP S10, which is designed for the Brave New World of storage that is required for IoT (Internet of Things) where IT, Information Technology, and OT, Operational Technology, come together.


The Brave New World

Hitachi Content Platform S10 is a new type of storage system that is designed to support the explosion of data that is being driven by Social, Mobile, Analytics, Cloud and the Internet of things. In order to support this new world, storage systems must be able to scale to very large capacities at low cost, be easy to install and maintain, provide high availability and be accessible over web service interfaces. The enterprise storage paradigms that were built around cached storage controllers, RAID protected high capacity enterprise disks, tethered together with SAN or LAN connections will need to be changed as we move into this brave new world. At the core of the business there will still be a growing need for enterprise storage controllers, but as we extend into the cloud and internet connectivity, there will be new requirements for storage that will require a radical change in how we architect storage systems. Hard disk recording densities have started to top out and the price reductions gained from higher bit densities have leveled off. In order to drive down hard disk costs we have to turn to commodity disks where the volumes in the commodity market continue to drive down costs. However, commodity disks do not have the durability of enterprise disks and are expected to fail more frequently. This adds to another problem which is the time required to rebuild the data on large capacity disks. We have to look out side the disk itself to solve these problems. Here are some of the innovations that are introduced in the HCP S10 to address this new world.


HCP S10 is a Hitachi Content Platform Storage node

HCP is S10 is not a standalone storage device. It is a storage node in an HCP object storage eco-System. It connects to HCP over the internet, and the HCP adaptive cloud-tiering functionality is used to move object data in and out of the S10 and across internal and external tiers of HCP storage including a choice of leading cloud services like Amazon S3, Microsoft® Azure, Verizon Cloud, Google Cloud Storage, and Hitachi Cloud Services for Content Archiving. The S10 is used by HCP to store object data, but the object management is done by the HCP, including indexing, versioning, compression, encryption, replication, retention and Worm. This makes it very easy to use. Just plug it into an Ethernet switch, where it will be discovered by HCP. There is no formatting of disks, defining LUNs, creating RAID Groups, assigning ports, or mapping of cache. There is no cache. This is just a 4 U tray of 60 raw disks, with two Intel servers, four 10 GbE ports for connectivity and two 1 GbE management ports. The S10 introduces storage capacity as part of an ecosystem and not as a stand-alone commodity storage array.



HCP S10 Provides Storage through a REST Web Services Interface

HCP S10 connects to HCP over Ethernet using the S3 internet protocol which means a wireless connection to HCP. While theoretically, the S10 could be geographically dispersed, it is better suited as an on premise cloud storage for faster access than lower cost public cloud but at a much lower cost than SAN or NAS attached storage. In addition to the advantages of low latency for an on-premise cloud there is the security of knowing that your data is behind your fire walls, and that you are not charged for every access as you would in a public cloud. One to 80 S10 storage nodes can be attached to an HCP. Currently we have qualified the S10 as an S3 tier for HCP, and we are planning to finish qualification of the S10 as a low cost tier for our HNAS object file system, which also provides policy based tiering. The S10 could work with other S3 enabled platforms or through other REST interfaces in the future. This intelligence is provided in the software on the S10.


HCP S10 Erasure Coding Provides High Availability with Commodity Disk

The main cost in storage systems is the storage media. HCP S10 contains 56 (4 enterprise disk are included for systems information for a total of 60 disks) high capacity, commodity disks in a 4 U enclosure with two Intel 6 core processors. While high capacity commodity disks provide lower cost, they are less durable than enterprise disks, and rebuild times will cause days of disruption to service levels. In order to over come the deficiencies of large capacity commodity disks, the S10 software provides 20+6 erasure coding, which means it can sustain 6 concurrent errors for a data reliability of 15 nines with a storage efficiency of 77%.


The S10 does not use a file system. The software uses raw disks and slices them into 64 MB extents and a group of 20 data extents plus 6 protection extents are used to form an extent group. When a disk fails, only the damaged extents are rebuilt. New data is written fully protected and rebuild activity is distributed across all available disks. Rebuild priority goes to the extent group with the most number of damaged extents. There is no need to reserve disks for spares or wait to rebuild the whole disk. By rebuilding the data and not the disk, we have faster rebuild and less vulnerable.




HCP S10 Value Proposition

The S10 can be described as a software-defined commodity storage node. There is no purpose built storage controller, no RAID, no formatting of disks, no mirroring, no snap shots. You plug in the power and connect to an Ethernet switch. The software in the Intel processors provides erasure coding and S3 connectivity to an HCP.  The object management is provided by HCP. The value is in the HCP and the S series software and it is ready for the next generation HDS software. Today, it addresses the need for low cost commodity object storage for on-premise use with HCP. In the near future we will qualify the S10 for the HNAS object file system. The future for S10 will only be limited by the development of innovative software and the availability of higher capacity commodity disks.


This is a brave new world for storage where users are dispersed and larger capacities create greater risks, and the traditional storage tools of RAID protection and FC delivery guarantees are no longer able to sustain us. We can enter this brave new world with the HCP S10.