Skip navigation

Okay, so why is everyone "doing cloud"? If you're a consumer of cloud-based technology, like an IT shop, it's ultimately about stuff like reduced operational costs, increased response time, improved repeatability of best practices, improved flexibility to meet customer needs, yada yada. If you're a service provider, it's about a lot of the same things as well as about offering competitive new services, increasing revenue streams, attracting a new customer base, yada yada.


So, if you're pursuing cloud-based computing for those reasons, how do you know if you're actually making a difference? IT has a long and storied history of saying, "We're going to do this project and then we'll save xyz or increase productivity by abc…" and once the project gets funded and folks start working on it, hardly anybody ever looks back to see if the project ever achieved what it set out to do.


That's where benchmarking comes into play. Now, benchmarking fills most IT professionals with trepidation. It's a lot of work and hard to do and the data's hard to capture. Let alone capture in a consistent way. And, being technologists who like things to be logical and tidy, all those reasons turn into an excuse to not do any kind of benchmarking.


My response to that? Bah Humbug! (I'm the Curmudgeon so I get to use that phrase regardless of the season!)


Cloud-based computing really has the potential to achieve all the kinds of things I listed above. But it doesn't all happen by magic. You have to work at it. And chances are that some things are going to be more expensive while you transition. And some processes will likely be more chaotic as you get everyone onboard. But you do want to make sure things get better as you move along. And you can't really know where things need more attention unless you do some kind of benchmarking.


When I refer to benchmarking, I'm including both financial benchmarking and performance benchmarking. And I include using benchmarks against yourself to see if you're improving and as a measure against industry standards (when you can find that data). Ideally, you should be looking for ways to compare across a number of dimensions, such as:

  • Time to provision
  • Labor associated with provisioning and management
  • Software licensing costs
  • Hardware costs
  • CAPEX vs OPEX spend


As an enterprise IT shop, it's important to understand if you're getting your money's worth from public service providers. As the service provider, this is important in terms of understanding the financial and performance attributes of the service you're providing. Are you gaining the revenue efficiencies and margins where you're expecting to get them?


So the first thing to do is to take stock now so you can see how things are change as you move to cloud-based computing. How long to things take to do NOW ? How much are those servers costing NOW? How much are you spending on labor NOW? And then, as you transition, who are those measurements telling you?


Don't get waylaid by seeking perfection in your benchmarks. If you need to swag things to start, swag them. Something, anything, is better than not having this data at all - just remember to use that information with a grain of salt.


Ultimately, someone is going to argue against spending time trying to do benchmarking by playing the magic "get out of jail card": It doesn’t make sense to spend much time on this because we're comparing apples to oranges. Here's my answer to them (thanks to searching the web for quotes about comparisons):


How do you compare apples and oranges? By their nutritional value.

Marshall Elizer


While that's an amusing rejoinder, I do like the basic point. Don't get too hung up on the details because you need to look at the bigger picture. You can always compare current state with cloud-state at a higher-level - like how they're helping you provide a service. And even if you really don't buy the nutritional value argument, at some point, some "big dog" business person is going to ask about the ROI of moving to cloud-based computing. Wouldn't it be good to actually have *some* data as the basis for how you'll answer?


To weigh in, comment below! I'll begrudgingly read your thoughts. To suggest a rant - email me at