State of DevOps 2019 Report: Three Key Takeaways

Actually, Netflix *is* a Technology Company
October 8, 2019
AMG Earnings Q3 19: The Future of the Tech Industry
October 31, 2019

Each year the DORA (DevOps Research and Assessment) Group publishes a “State of DevOps” report. Drawn on extensive surveys, the report describes the state of DevOps adoption across the IT industry. It’s a fascinating look into the benefits of DevOps practices and how widely they’ve spread.

Interestingly, the report does not define DevOps. Instead, it focuses on four key metrics by which one can determine how effective an organization’s SDLC practices are. For what it’s worth, here is Wikipedia’s DevOps definition:

Don't miss a single blog!

Don't depend on Twitter, Facebook, or Linkedin notifications. Sign up and get the latest blogs and videos delivered to your inbox asap!

I agree to have my personal information transfered to MailChimp ( more information )

I will never give away, trade or sell your email address. You can unsubscribe at any time.

DevOps is a set of practices that combines software development (Dev) and information-technology operations (Ops) which aims to shorten the systems development life cycle and provide continuous delivery with high software quality.”

The report divides the tech industry into quartiles that reflect how well each group realizes the activities that form the basis of DevOps.

The four metrics that DORA looks at to determine DevOps effectiveness are:

  • Deployment frequency: For the primary application or service you work on, how often does your organization deploy code to production or release it to end users?
  • Lead time for changes: For the primary application or service you work on, what is your lead time for changes (i.e., how long does it take to go from code committed to code successfully running in production)?
  • Time to restore service: For the primary application or service you work on, how long does it generally take to restore service when a service incident or a defect that impacts users occurs (e.g., unplanned outage or service impairment)?
  • Change failure rate: For the primary application or service you work on, what percentage of changes to production or released to users result in degraded service (e.g., lead to service impairment or service outage) and subsequently require remediation (e.g., require a hotfix, rollback, fix forward, patch)?

I think this is the right way to look at DevOps — focus on outcomes rather than on specific stages, names, or tools. As the old marketing saw goes, people don’t want to buy quarter inch drills, they want to buy quarter inch holes. Purchasing a drill bit is just a means of accomplishing the desired outcome. It is just the case that improvements in these metrics are directly associated with agile development, automated deployment, and deep telemetry, i.e., typical DevOps practices. 

The results of pursuing a DevOps initiative are impressive. This chart illustrates the vast difference between the top and bottom quartiles across the four key metrics:

It’s undeniable that practices associated with DevOps lead to improved IT outcomes. Originally pioneered by cloud-native companies, they are spreading throughout the industry quickly, as shown in this report chart that compares the quartiles in 2018 and 2019:

In all the coverage of this year’s DORA report, however, there are three things that go underemphasized that must be understood to fully comprehend the implications of what DevOps means to the direction of the tech industry and, by extension, to the broader economy. 

Cloud is Fundamental

The fourth Key Finding in the report is “Cloud continues to be a differentiator for elite performers and drives high performance,” so the importance of cloud computing to DevOps is not exactly hidden.

What’s interesting is how the report discusses the use of cloud computing. Rather than getting caught up in the industry sturm und drang of cloud — is it public, or multi, or whatever the specific speaker means when using the term “hybrid cloud” — the report sidesteps this by referring to the NIST cloud computing definition characteristics:

  • On-demand self-service
  • Broad network access
  • Resource pooling
  • Rapid elasticity
  • Measured service

The message of the report regarding infrastructure is this: it doesn’t matter what you call it, where you install it, or how you buy it, but your infrastructure must be agile, easy to access, and deliver “the illusion of infinite capacity.” If your infrastructure does not deliver these five characteristics, your ability to implement DevOps will be severely constrained, if not made impossible.

This makes sense. DevOps practices often call for use of resources beyond the minimum required for normal application operation; for example, in running blue/green deployments or dealing with traffic spikes due to seasonal or unexpected load. Traditional infrastructure is notoriously difficult for dealing with temporary resource requirements as this doesn’t fit very well with upfront capacity planning and capital investment along with lengthy manual change processes.

Just as a house cannot be built without a strong foundation, so too does DevOps require the right infrastructure environment. It isn’t the purpose of the report to hammer home the need for getting one’s infrastructure house in order, but it’s clearly obvious to the thoughtful reader that DevOps can’t work if tied to traditional infrastructure practices.

Implementation is Critical

The report devotes a number of pages to the “How” of DevOps — how can an organization pursue a DevOps initiative and make it stick? It describes several approaches, as seen in this chart:

The critical question is not how to start a DevOps initiative, but how can one ensure its spread throughout the organization, so that it becomes the default SDLC set of practices rather than being marooned in early adopter groups?

Essentially, the report criticizes approaches that set up segregated islands of expertise, whether in a training group or a center of expertise. The shortcomings of these approaches is twofold: first, it isolates knowledge in certain groups and makes it difficult for other groups spread across the organization to access methods of improvement; and second, this isolation can lead to an ivory tower syndrome in which the island of expertise loses touch with the day-to-day activities that comprise the main workload of groups delivering applications.

This perspective is right, as far as it goes. It’s important to — and the report discusses this — treat DevOps adoption as a process, and to incrementally increase the breadth and depth of DevOps practices as the organization gains experience. For that reason, it’s better to treat the different transformation strategies not as alternatives, but as stages through which an organization passes on the journey to full DevOps maturity.

Curiously, though, the report fails to address the elephant in the room: organizational politics and the importance of senior sponsorship of DevOps adoption. In my experience, the specific form of spreading DevOps practices is far less critical than the overt and (crucially) implied messages from above. 

Implementing DevOps or any other significant transformation is disruptive and inconvenient. It requires change and incremental work — not to mention increased spending — beyond the established set of practices that are in place. People are incredibly attuned to reading social messages, and if the messages from above don’t emphasize how important a DevOps initiative is, and back that up via repetition and reinforcement, they will move the hard work of transformation to the back burner in favor of those items that do receive repeated emphasis.

There are probably very good reasons for the report to avoid getting into organizational prescriptions, but anyone reading the report for transformation recommendations should recognize that the choice of practice sharing technique is less critical than ensuring organizational alignment from the top down.

This Has Real Economic Consequences

Perhaps the most interesting — and important — aspect of DORA research is how the group ties DevOps technical practices to improved financial outcomes for the parent companies within which DevOps implementers work. Both the 2018 and 2019 reports state that companies operating at the top quartile of DevOps practices achieve better financial results than bottom quartile companies. The report summarizes its finding in this pithy statement: “Our analysis this year shows elite performers are twice as likely to meet or exceed their organizational performance goals.”

Here are the goals for which operating at an elite DevOps level enables better results:

  • Profitability
  • Productivity
  • Market share
  • Number of customers
  • Quantity of products or services
  • Operating efficiency
  • Customer satisfaction
  • Quality of products or services provided
  • Achieving organization or mission goals

A couple of things stand out about these improvements.

First, these are first and foremost non-IT outcomes. In other words, they are things non-IT parts of the company care about. The report draws a connection between operating IT well and doing better delivering on business metrics that, on their face, are not directly connected to IT. 

This takes DevOps initiatives out of the realm of IT improvement and into the realm by which company performance is judged. Put another way, this means everyone has a stake in how IT operates. It also means that IT investment should be gauged not on its traditional measure — how little money can be spent on it while still keeping the lights on — but on how much the company is willing to spend to improve its marketplace performance.  

Second, the relationship between elite DevOps performance and improved company performance is correlation, not causation. While the researchers noted better company performance is associated with better DevOps performance, the report does not identify how this better performance is achieved. 

It’s important to reflect on how far we have come. Not so very long ago, IT was universally considered a cost center, at best a necessary evil and at worst a drag on company performance, since any money spent on IT couldn’t be directed toward improving the company’s market offerings or toward shareholder reward. 

Today there is evidence that better IT performance, which means spending more money on tech, actually leads to better company performance. Or, as I often put it, the role of IT has changed from “support the business” to “run the business.” I predict it won’t be too long before we see CEOs proudly announcing how much more their companies are spending on IT, given that IT budget will come to be viewed much like marketing dollars or capital investment.

However, as noted, the report leaves unexplained the transmission mechanism between better IT practices and company performance. This is not to be critical of the report, its methodology, or the researchers. It does mean that more work remains to be done to draw an explicit, rather than implicit, connection between the two. In my view, understanding the relationship between IT practices and business performance will be a burgeoning topic over the next five years

I have some thoughts on how IT practices are connected to business outcomes that I will be sharing over the next few blog posts. Suffice it to say that the value creation of tech is associated with the ongoing change in our economy from industrial to digital, and figuring out how to play by the rules of the digital economy will be vital for future business success.


1 Comment

  1. […] many still adhere to the established IPS model of cloud computing (which I discussed in my most recent posting), AMG have moved way beyond core computing and application services. Today, they are rolling out […]

Leave a Reply

Your email address will not be published. Required fields are marked *