Private and Public Cloud Costs: Everything You’ve Been Told is Wrong

My 3D Printing Adventure
April 24, 2017
Why the Harvard Business School is Wrong About Blockchain
May 31, 2017
Show all

Cloud Costs: Critical Yet Misunderstood

The topic of cloud computing pricing is evergreen: controversy about how much public cloud computing really costs and whether on-premise cloud environments can be less expensive is endless. And certainly it’s not obvious what the real answer is. If we look at analog situations, it’s clear that buying vs. renting can be less expensive over a given time period; for example, over a four-year period, it is cheaper to buy a car than rent it from Hertz.

There are many who assert that running your own hardware infrastructure, for the same reason that buying a car is cheaper than obtaining it on a rental basis, is bound to be less expensive than using a public cloud provider. The justifications for this run from the reasoned (as in Chuck Hollis’ blog) to the ludicrous (as in this recent blog posting that claimed running your own hardware is less expensive because you don’t have to hire anyone to manage it).

Don't miss a single blog!

Don't depend on Twitter, Facebook, or Linkedin notifications. Sign up and get the latest blogs and videos delivered to your inbox asap!

I will never give away, trade or sell your email address. You can unsubscribe at any time.

The rationale for those holding this opinion boils down to:

  • Cloud provider costs are roughly comparable to those for on-premise, so they have no resource cost advantage;
  • Cloud providers charge a premium for making the capacity available on an on-demand basis, while the on-premise offering does not need to impose a convenience surcharge; and
  • Cloud providers are for-profit businesses while internal IT organizations run on a non-profit basis and therefore don’t add a margin for profit

Taking the same cost structure, tacking on an on-demand surcharge, and adding on a margin for profit indicates that on-premise is bound to be less expensive. Right?

I have to say, I am not convinced by this reasoning. In fact, I’m not sure that any of the factors cited are true, and find it easy to believe that they’re almost always false — particularly for the webscale public providers, which are the only ones really worth discussing, and likely to be the only ones left in five years.

Let’s look at the cost input structure. If you examine the figure below, it lists the major inputs into running a cloud environment.

Cloud Computing Cost Factors

Taking each factor in turn:

  • Hardware infrastructure refers to the physical infrastructure used to house and operate a cloud environment. It includes both computing (e.g., servers, switches, and storage) and environment (e.g., data center building, HVAC, and racks). It’s not obvious at all that an internal IT group can achieve the same cost structure for hardware that a webscale provider can achieve. Providers buy in much larger quantities than almost every IT organization, so receive volume discounts. Moreover, most of them have moved to internal design and direct purchase from Asian ODMs, so they avoid paying the profit margin imposed by a systems vendor. And for sure, none of the webscale providers buy “enterprise-grade” equipment, unlike most enterprises. Cost advantage: public cloud provider.
  • Software infrastructure refers to the orchestration software used by the provider to operate its environment. All of the large providers write their own, which enables them to deliver a differentiated and highly scaled offering. Of course, this means that the providers have to employ sophisticated software developers, which internal clouds may be able to avoid. However, this isn’t as clear-cut as it might seem. Enterprises often choose an orchestration product from a software vendor who achieves 80% gross and 20% net margins, all of which adds to the internal cloud cost structure. Of course, there are open source orchestration products that an enterprise might choose, which avoids the license tax, but, given the relative immaturity of the open source offerings that are available, typically means that the implementing organization ends up employing software developers, thereby obviating the putative cost advantage of using an open source product. For the public providers, the cost of software development is amortized across enormous capacity volume, such that the effect of internal software development is relatively small; by contrast, proprietary license fees drop much less as scale grows (each new node requires additional license fees, albeit as volumes grow larger, per-node costs drop due to volume discounts). Cost advantage: public cloud provider.
  • Internet bandwidth refers to general connectivity to the outside world. Public cloud providers purchase Internet connectivity in much higher volumes than any individual enterprise and therefore achieve cost advantage. Moreover, Google and Amazon (and no doubt Microsoft) have moved to purchasing their own fiber, which further reduces their Internet bandwidth cost, as they no longer have to fund a carrier’s profit margin as part of that cost. Cost advantage: public cloud provider.
  • Energy refers to the cost of electricity used to run the cloud environment. This is clearly an area in which very large-scale cloud providers achieve better pricing than any enterprise. In a study originally published by Virtual Strategy magazine (alas, no longer available online), author Steve Denegri analyzed CSP energy costs in detail. In his analysis, he described how, when building its San Antonio data center, Microsoft negotiated a new, super-large user electricity rate lower than even the high-user rate previously available to the largest electricity users. This cost advantage does not even touch on the reduced energy costs available to very large providers via operating data centers at very high efficiency. Facebook achieves a sub-1.10 PUE in its Oregon data centers; the typical skilled enterprise operates at over 1.5 PUE. Cost advantage: public cloud provider.
  • Labor refers to how much the employees required to build and operate a cloud environment cost. It’s not immediately obvious whether enterprises or public cloud providers can achieve the lowest labor cost. From a pure salary viewpoint, it’s likely that public providers pay more; after all, they employ the cream-of-the-crop talent required to build the largest computing environments. By contrast, most enterprises select from a less-heady applicant pool and therefore probably don’t have to pay as much. On the other hand, the question isn’t the raw cost per employee, it’s how much labor cost is imputed for a given amount of computing capacity. Seen in this light, it’s likely that public providers have a significant cost advantage; because their environments are so highly automated they require far fewer employees per 1000 servers. While most enterprises strain to achieve server/sysadmin ratios of 30-1, public providers commonly achieve 10,000-1. Cost advantage: public cloud provider.
  • Cost of capital refers to how much an entity must spend to obtain investment capital. The cost of capital is typically a blended rate of what an entity pays to borrow money (its interest rate) combined with its equity valuation. A high-PE company has, in effect, a lower equity cost of capital compared to a low-PE company. Of the large providers, Microsoft has a relatively low PE ratio, while Amazon and Google sport sky-high ratios. Large balance sheet cash reserves reduce a company’s cost of capital, and Microsoft has an enormous warchest of savings. By contrast, enterprises (and most non-webscale cloud providers) have much higher costs — in fact, I once had a vendor explain to me how unfair the cloud marketplace is because “Amazon is subsidized by the stock market and can offer lower prices than we can.” Another way of saying that is Amazon has a much lower cost of capital than his company, and thereby benefits. Cost advantage: public cloud provider.

Reviewing the cost categories, it’s public cloud providers 6 – 0 over private cloud. So you can understand why I’m perplexed when I hear people assert that they can operate a private cloud less expensively than one of the large public providers. And this is not even to address the issue that most IT organizations have no detailed understanding of their costs; their budgets are often spread across several departments and are never analyzed to provide true marginal costs broken down to specific resource units, e.g., server-hour. In fact, most of the time, the putative cost advantage of a private cloud is breezily asserted with no evidence save for an unsupported certainty that, by being “on the side” of users (i.e., part of the same corporate entity), internal IT must be more cost-effective than a profit-seeking external entity.

The problem with that perspective is that it assumes cloud operating costs are  analogous to the situation described at the beginning of this post: renting a car vs. buying one. In the car world, while Hertz can buy a car less expensively than an end customer can, there’s only a small difference in actual cost to Hertz and an end user. I don’t believe that analogy is at all accurate; while the cost savings a cloud provider achieves by designing and directly procuring servers may only be 15%, the total cost savings available to a cloud provider summed across all six categories might be as much as 75%. That is to say, a virtual machine costs a cloud provider one quarter what it costs a private cloud operator — $.25 compared to every dollar the private provider costs. Even adding in a profit margin to the public provider of 10% leaves it less than half the cost of a private cloud alternative.

Cloud Computing Price Drops: Gated by Investment?

Of course, even if public providers have a cost advantage, that doesn’t necessarily mean that they will reduce their prices sufficiently to make using a public provider less expensive than an on-premise environment. Despite Amazon’s (and now Microsoft and Google’s) ongoing price cuts, there are those who assert (as in this piece by competitor ProfitBricks) that Amazon, far from offering really low prices, is actually milking its offering for very high margins. This perspective is somewhat supported by this article that quotes Gartner’s Kyle Hilgendorf that cloud users aren’t really interested in price cuts and are more focused on the richness of the provider services, which benefits Amazon as the provider with the richest offering functionality.

I think this perspective has some validity. The big three public providers (AMG) are building out capacity like crazy and Amazon, in particular, is experiencing sky-high growth. While Amazon is extremely secretive about its revenues and infrastructure numbers, its one public indicator  — the number of S3 objects it manages — indicates the scale and growth it is experiencing.

Using S3 object numbers  as a proxy for overall AWS scale and growth, it seems clear that, if ProfitBricks’ assertion is correct,  too-high AWS prices aren’t deterring users from embracing the service. In fact, it may not be clear that lower prices would do Amazon any good and might, in fact, alienate users. In my blog, I did an analysis of Amazon’s capital investment, based on its frequent statement that each day it installs as much AWS capacity as it used to run all of Amazon in 2003, and concluded that it had to install 6.5 million dollars of equipment each and every day to achieve that capacity. I could be off significantly from the real number, but it’s clear that Amazon is investing a very large amount in its computing infrastructure, day in and day out. It’s likely that just keeping up with that level of investment taxes Amazon’s ability to the limit; lowering prices further would just result in even more users piling on, which would make Amazon’s daily struggle even worse. In other words, Amazon can barely keep up with demand as it is, so there’s no need (or point) to cutting prices to some, supposedly “fairer” level.

So, public providers may not have prices as low today as they theoretically could be. I’m not convinced that even at these prices public providers are more expensive than private clouds, but am convinced that they are not done lowering prices. If you are an IT organization creating a business case that indicates you can provide cloud computing services less expensively than a public provider, you need to factor in what your business case would look like if the public provider costs were 50% of what they are today.  Put another way: can you drop your prices as low as your public cloud competition can?

Cloud Computing Prices: How Low Can They Go?

The big question is, just how low can public cloud computing prices go? There is, of course, an irreducible minimum price level that, absent a willingness to absorb losses on a long-term basis, below which a public provider cannot go. I think it’s fair to say that we are not yet at that level, and of course the level drops as future hardware gets cheaper through the workings of Moore’s Law. The providers treat their cost structures as proprietary information, so it’s unlikely that any of them will publish numbers.

However, this are some guidelines we can look to as to how public providers will behave regarding prices. In general, I believe public cloud providers will follow these rules in pricing:

  • Price at marginal cost. When I was in college, in my microeconomics class, the instructor spelled out this rule: a company should be willing to take any price that allows it to cover its marginal cost and make some contribution, no matter how small, to cover its capital cost. A more sophisticated way to put this is that a firm should accept any offer that allows it to pay for the variable cost of the offer as well as provide some payment against its fixed cost, even if the payment does not cover the fully loaded offer cost.
  • Increase utilization to reduce fixed cost. Of course, the amount of fixed cost that any cloud resource has to address varies according to the utilization rate of the provider’s infrastructure. This is because the overhead of the fixed cost is spread across all of those resources that have been sold at a level to cover their variable costs. The more of those variable cost-covering resources are sold, the less fixed cost overhead is assigned to each resource, which makes it profitable (or reduces its net loss) at a lower price. Consequently, we can expect to see public cloud providers execute sophisticated pricing schemes designed to increase infrastructure utilization rates to as high a level as is practicable in terms of service quality acceptability. We’ve seen this already with AWS’s reserved and spot instances, which are designed to drive up utilization of Amazon’s computing infrastructure. We are nowhere near the end of this sophisticated demand management; to understand its nuances we have only to look at the airline industry, where this activity is called yield management and accounts for situations in which a passenger in one seat pays ten times what the one in the next seat paid.
  • Use game theory to increase market share. Certainly Amazon and the other webscale providers have use price drop announcements to signal that they are in the provider market for good. However, one can expect them to increase the frequency of price reduction announcements to communicate to other providers, both current and potential, that this is a highly competitive market in which any participant must recognize that there will be unrelenting price pressure and that, unless a provider is prepared for a huge capital investment with an unknown outcome (i.e., whether it will be one of the ultimate winners or just another loser that retires after throwing away untold millions of dollars), it might be better to withdraw (or never enter) the market. We’ve already seen a couple of providers (e.g., GoDaddy) announce that they will leave the CSP market, and a couple of others announced a renewed focus on “their core markets” (e.g., hosting), with a clear implication that they will de-emphasize cloud computing. Essentially, the public providers will force or frighten other entrants from the market; once the easy marks are out of the game, then the AMG players will turn on one another with a vengeance.

This has been a relatively lengthy blog post, because I think price, despite all of the discussions, is an underappreciated aspect of cloud computing. To my mind, it is the most important characteristic of cloud computing, and the one all other characteristics flow from or are enabled by. As I noted in this blog post, everyone talks about how cloud computing enables agility, but the reality is that cloud computing makes agility affordable. Absent its cost advantages, its ability to enable users to more rapidly access resources and roll out applications would be meaningless.

We’re still early enough in the cloud computing revolution such that the realities of what its economic impact on running IT are not yet fully understood. As the new pricing levels and expectations are better comprehended, we will see an enormous shift in IT infrastructure investment and where applications are deployed. The next ten years will see more dislocation in IT than all the previous platform revolutions combined — and I may be understating the revolution.


  1. Bernard Golden says:

    Brian, Steve Kaplan, also of Nutanix, is supposed to publish some numbers soon. I stand by my position, which is most IT organizations do not really have a handle on their fully-allocated costs.

  2. Brian Cox says:

    I would love to see actual numbers comparing public cloud costs to on-premises IT costs across a variety of key workloads for large (e.g. Home Depot), medium (e.g. San Mateo County) and small (e.g. US Mortgage of Florida).

Leave a Reply

Your email address will not be published. Required fields are marked *