Gartner, IDC, and Forrester are predicting that digital transformation will be a key focus for CEOs in 2016 if they are to compete with digital disruptors like Amazon, Ali Baba, Fintechs, Airbnb, etc. In order to compete against the agility of these disruptors, they require an agile development and delivery process. They are looking to CIOs to enable this with the transformation of the IT culture as I predicted in my trends for 2016. “2016 will see a greater focus on application development and analytics to drive business value”.
As I meet with customers in 2016, I am seeing more CIOs adopting or planning to adopt Agile or DevOps methodologies to accelerate application development. Agile is a software development methodology where requirements and solutions evolve through collaboration between cross-functional teams in an iterative process. This enables rapid and flexible response to change. This is opposed to the traditional “waterfall” approach, which was adopted from hardware development where progress flowed sequentially from requirements, down through design, development, test, production, and maintenance. This approach was very rigid and changes became more disruptive as you moved down the waterfall. Since the early 2000s Agile software development frameworks like Scrum have been developed to facilitate software development and many associations and universities offer certification or degrees in this discipline.
DevOps grew out of Agile and involves the practice of operations (IT), Quality Assurance, and development engineers working together throughout the design, development, test, and delivery of application software. DevOps is a cultural change, which requires a change in people and process. Operations people look for stability and repeatability, while developers seek change; and testers seek risk reduction. In the waterfall approach developers would throw code over the wall to the testers, who would then request resources from operations, which would not be prepared for these “unplanned” requests. While there is no one DevOps tool, there are tool chains, which involve tools like Docker for containerization or micro services, Jenkins for continuous integration, and HCP for storing and distributing packages, as well as storing test results and logs.
Yes HCP, Hitachi Content Platform, which is recognized by the industry for cloud computing excellence, is also one of the tools in our DevOps tool chain in Waltham where we develop our HCP, HCP-Anywhere, HCI, and HCP-S series software. Hitachi Data Systems launched DevOps for the HCP team in 2012 and adopted the charter that they would be responsible for the organizational efficiency of the HCP portfolio’s engineering teams. Meaning they look for operational inefficiencies and automate them – for example they developed a system which allows engineers to automatically install an OS onto lab servers, collectively saving everyone from having to do it manually over and over again. They also developed and maintain automation test harnesses which the different QA/Automation teams can use to automate as many test cases as possible, allowing them to easily multiply how many test cases we can run through in a single release. There are plenty of other tools as well, but the spirit is the same: they develop internal tools, which make engineers’ lives way easier. The results have been evident in the quality and speed of delivery of enhancements to HCP, which have distanced us from the competition. See how HCP responded to Safe Harbor concerns last year and our latest enhancements, which we added 2 weeks ago.
HCP's main benefit for DevOps is that its HA capabilities can help insulate downstream test automation tools from software upgrades, hardware maintenance/failures, etc. as well as insulating from availability issues in the upstream tools as well. We use Jenkins for continuous integration and if it goes down or is being upgraded, the downstream test tools don't notice or care since they are fetching builds from the always-online HCP. HCP’s Metadata Query Engine (MQE) also helps abstract away where the artifacts are located and named in a namespace. As long as the objects are indexed, MQE will find them and present them to the client, regardless of the object name & path. Even further downstream, after the automated tests are run, we can again take advantage of HCP by storing the test results and logs on the HCP (preferably in a separate namespace than the build artifacts). HCP’s security and encryption features ensure a secure enterprise environment, which is not always available with DevOps tools. DevOps is about automation and HCP can automate managing the space consumption by taking advantage of retention & disposition to "age out" and delete old logs or old builds, or tier them off elsewhere for long-term storage (such as S10, Amazon, glacier, etc). HCP also provides an automated backup solution, using its replication feature as a way to get copies of the backups off-site for DR. If you are implementing a DevOps process see how HCP is used as a Continuous Integration Build Artifact storage system in this white paper written by our HCP developers.
The benefits of incorporating DevOps in the transformation of IT have been well documented. Puppet Labs, an IT Automation Software company, has been conducting yearly surveys on the state of DevOps. In their 2015 State of DevOps Report they surveyed over 20,000 tech professionals worldwide. Their survey concluded that:
“We validated that high-performing IT organizations deploy code 30 times more frequently than their peers with 50% fewer failures. And for the first time, we showed that IT really does matter to the business: Companies with high IT performance are twice as likely to exceed their profitability, market share and productivity goals.”
In order to support the digital transformation required by the business, CIO’s must transform IT into a collaborative culture built around the principles of Agile and DevOps. Technologies, tools and education are available. The greatest challenge will be in changing the people and process. The most important ingredient for transformation will be leadership.