DevOps cloud cost control: an oxymoron? If you’re in DevOps, you may not think that cloud cost is your concern. When asked what your primary concern is, you might say speed of delivery, or integrations, or automation. However, if you’re using public cloud, cost should be on your list of problems to control.
The Cloud Waste Problem
If DevOps is the biggest change in IT process in decades, then renting infrastructure on demand is the most disruptive change in IT operations. With the switch from traditional datacenters to public cloud, infrastructure is now used like a utility. Like any utility, there is waste. (Think: leaving the lights on or your air conditioner running when you’re not home.)
How big is the problem? In 2016, enterprises spent $23B on public cloud IaaS services. We estimate that about $6B of that was wasted on unneeded resources. The excess expense known as “cloud waste” comprises several interrelated problems: services running when they don’t need to be, improperly sized infrastructure, orphaned resources, and shadow IT.
Everyone who uses AWS, Azure, and Google Cloud Platform is either already feeling the pressure — or soon will be — to reel in this waste. As DevOps teams are primary cloud users in many companies, DevOps cloud cost control processes become a priority.
4 Principles of DevOps Cloud Cost Control
Let’s put this idea of cloud waste in the framework of some of the core principles of DevOps. Here are four key DevOps principles, applied to cloud cost control:
1. Holistic Thinking
In DevOps, you cannot simply focus on your own favorite corner of the world, or any one piece of a project in a vacuum. You must think about your environment as a whole.
For one thing, this means that, as mentioned above, cost does become your concern. Businesses have budgets. Technology teams have budgets. And, whether you care or not, that means DevOps has a budget it needs to stay within. Whether it’s a concern upfront or doesn’t become one until you’re approached by your CTO or CFO, at some point, infrastructure cost is going to be under scrutiny – and if you go too far out of budget, under direct mandates for reduction.
Solving problems not only speedily and elegantly, but cost efficiently becomes a necessity. You can’t just be concerned about Dev and Ops, you need to think about BizDevOps.
Holistic thinking also means that you need to think about ways to solve problems outside of code… more on this below.
2. No Silos
The principle of “no silos” means not only no communication silos, but also, no silos of access. This applies to the problem of cloud cost control when it comes to issues like leaving compute instances running when they’re not needed. If only one person in your organization has the ability to turn instances on and off, then all responsibility to turn those instances off falls on his or her shoulders.
It also means that if you want to use an instance that is scheduled to be turned off… well, too bad. You either call the person with the keys to log in and turn your instance on, or you wait until it’s scheduled to come on. Or if you really need a test environment now, you spin up new instances – completely defeating the purpose of turning the original instances off.
The solution is eliminating the control silo by allowing users to access their own instances to turn them on when they need them and off when they don’t — of course, using governance via user roles and policies to ensure that cost control tactics remain uninhibited.
(In this case, we’re thinking of providing access to outside management tools like the one we provide, but this can apply to your public cloud accounts and other development infrastructure management portals as well.)
3. Rapid, Useful Feedback
In the case of eliminating cloud waste, the feedback you need is where, in fact, waste is occurring. Are your instances sized properly? Are they running when they don’t need to be? Are there orphaned resources chugging away, eating at your budget?
Useful feedback can also come in the form of total cost savings, percentages of time your instances were shut down over the past month, and overall coverage of your cost optimization efforts. Reporting on what is working for your environment helps you decide how to continually address the problem that you are working on next.
You need monitoring tools in place in order to discover the answers to these questions. Preferably, you should be able to see all of your resources in a single dashboard, to ensure that none of these budget-eaters slip through the cracks. Multi-cloud and multi-region environments make this even more important.
The principle of Automation means that you should not waste time creating solutions when you don’t have to. This relates back to the problem of solving problems outside of code mentioned above.
Also, when “whipping up a quick script”, always remember the time cost to maintain such a solution. More about why scripting isn’t always the answer.
So when automating, keep your eyes open and do your research. If there’s already an existing tool that does what you’re trying to code, it could be a potential time-saver and process-simplifier.
So take a look at your DevOps processes today, and see how you can incorporate a DevOps cloud cost control – or perhaps, “continuous cost control” – mindset to help with your continuous integration and continuous delivery pipelines. Automate cost control to reduce your cloud expenses and make your life easier.
Announcing: ParkMyCloud now integrates with Centrify for Single Sign-On (SSO). What, did you think we were finished with SSO integrations?
That brings the list of SSO providers you can use with your ParkMyCloud account to:
- Active Directory Federation Services (ADFS) – Microsoft
- Azure Active Directory – Microsoft
- Google G-Suite
- Okta (in Okta App Network)
- OneLogin (in App Catalog)
- Ping Identity (in App Catalog)
Stay tuned: ParkMyCloud will be listed in the Centrify marketplace shortly.
We have integrated with Centrify for Single Sign-On, as well as the other SSO providers, to make it simpler:
- For account administrators, who can use just-in-time provisioning to automatically add their organization members to ParkMyCloud as they are authenticated in Centrify – all you need to do as an administrator is share your organization’s unique ParkMyCloud login link with your users. This can be found in the ParkMyCloud management console.
- For users, who will not need separate login information and a password for ParkMyCloud.
For a step-by-step guide for setting up Centrify as a SAML IdP server for ParkMyCloud, please see this article on our support site. Note that you will already need to have your ParkMyCloud account created – though there’s no need to add additional users until you’ve connected with Centrify, at which point you can add them directly from the SSO provider.
If we still don’t support your SSO provider of choice, please leave a comment below or contact us – we’re all about meeting user needs, here!
We are happy to share that ParkMyCloud now supports Active Directory Federation Services (ADFS) for Single Sign-On (SSO).
Additionally, ParkMyCloud is now integrated into the Ping Identity App catalog, making it easier to configure your SSO options and add users from Ping accounts.
With these updates, you can now connect to ParkMyCloud through six major SSO providers:
- Active Directory Federation Services (ADFS) – Microsoft
- Azure Active Directory – Microsoft
- Google G-Suite
- Okta (in Okta App Network)
- OneLogin (in App Catalog)
- Ping Identity (in App Catalog)
All of these SSO providers are among the top of those ranked in Gartner’s 2016 Magic Quadrant for Identity and Access Management as as Service.
Using SSO simplifies processes for both users and administrators. Users need to track and remember fewer passwords, and administrators can control user access in the single location of their SSO provider dashboard, to simplify processes and tighten access control.
Through these SSO providers, ParkMyCloud supports just-in-time provisioning of new users. This means that users are automatically created in ParkMyCloud as they are authenticated from the SSO provider. All the administrator needs to do is email users the organization’s unique ParkMyCloud login link, which can be found in the ParkMyCloud management console.
For more information about configuring SSO for your ParkMyCloud account, please see this article in our support portal – there are instructions for each SSO provider. (You’ll need to have an active ParkMyCloud account in place before you can start adding users from your SSO provider – here’s the signup link if you need to create one first.)
Azure vs. AWS 2017: what’s the deal? There’s been a lot of speculation lately that Microsoft Azure may be outpacing Amazon Web Services (AWS). We think that’s interesting and therefore worth taking a look at these claims. After all, AWS has been dominating the public cloud market for so long, maybe the media is just bored of that story, and ready for an underdog to jump ahead. So let’s take a look.
Is Azure catching up to AWS?
You may have seen some of the recent reports on both Microsoft and Amazon’s recent quarterly earnings. There have certainly been some provocative headlines:
With Amazon and Microsoft reporting their quarterly earnings at the same time, this is a good time to analyze the numbers and see where they stand in relation to one another. Upon closer inspection, here’s what the recent quarterly earnings reports showed:
- AWS revenue grew 43% in the quarter, with quarterly earnings of $3.66 billion, annualized to $14.6 billion. Sales and earnings exceeded expectations given by analyst estimates. In the immediate wake of Amazon’s report, the stock went up.
- Microsoft reported that its Intelligent Cloud division grew 11% to $6.8 billion, and that the Commercial Cloud division has a annualized run rate of $15.2 billion. These reported earnings only met analyst expectations, and therefore the stock fell by nearly 2 percent within hours.
- We think it’s important to note when it comes to Microsoft’s reported earnings the Commercial Cloud business includes Office 365, not just Azure. We have never fully understood why the Office 365 business has been bundled in with Commercial Cloud, given that it’s a very different business than the IAAS services of Amazon and Google to which it is often compared.
- Microsoft stated that Azure’s growth rate was 93%, without providing an actual revenue number. Once again, we find this lack of lack of earnings clarity somewhat problematic.
So is Azure bigger than AWS?
Well, currently no. There is little evidence of Azure surpassing AWS, aside from a small research study which pales in comparison to a clear majority of data stating otherwise.
But is Azure growing quickly?
Yes. In this regard, it’s important to consider what factors are at play in Azure’s growth, and whether they hold any weight as far as surpassing Azure outpacing AWS in the future.
Where is Azure actually gaining ground?
Now let’s take a look at what is driving Azure’s growth, and where Azure is gaining ground.
First of all, as companies grow beyond dipping their toes in the water of public cloud, they become more interested in secondary options for diversity and different business cases. Just from our own conversations, we’re finding that more and more AWS users are using Azure as a secondary option. While users might be interested to see what Azure can offer them in comparison, this doesn’t necessarily indicate that it will ultimately surpass AWS.
Take, for example, the results of a research survey released by data analytics provider Sumo Logic and conducted by UBM Research. According to the survey of 230 IT professionals from 500+ employees, Azure actually beat AWS as the preferred primary cloud provider, taking the lead by a 10 percent margin, with 66 percent of participants preferring Azure as opposed to the 55 percent who relied in AWS.
This research is significant because it’s the first time that survey data on customer preferences has reported Azure taking a lead over AWS. However, the data also revealed that a significant number of enterprises are using more than one cloud provider. While Azure and AWS both take the lead, there is certainly an overlap in participants who use both, in addition to other up-and-coming providers.
Second, enterprises have been committed to a variety of Microsoft products for years. According to UBM Research survey data, over 50 percent of participants who preferred Azure as their primary cloud provider were coming from large enterprises with 10,000+ employees. This makes sense considering that Microsoft has a foothold in terms of relationships and enterprise agreements with these larger organizations and are able to cross-sell Azure.
Third, Azure has a strong base in Europe, where more users report using Azure rather than AWS as their primary provider. In a 451 Research Survey with 700 participants considered to be “IT decision makers,” AWS topped the list among all participants as the preferred provider among 39 percent of participants. While Azure saw an increase in users, it still landed in second place overall at 35 percent. However, among the European participants only, Azure took the top spot, with 43.7 percent naming Azure as their provider, and 32 percent sticking with AWS.
Why does the Azure vs. AWS debate matter?
Why does the Azure vs. AWS 2017 debate matter to, when choosing a new or secondary cloud provider? Well… in terms of market performance, it probably doesn’t. As always, the specific needs of your business are going to be what’s important.
One thing is for certain: the public cloud is growing and it’s here to stay. Let’s not forget that both Google and IBM both have growing public cloud offerings too (and Google is looking to expand their enterprise market this year.) All of this competition drives innovation, and therefore IaaS and PaaS offerings – and perhaps, better pricing.
For the customer, the basic questions remain the same when evaluating public cloud providers:
- How understandable are the public cloud offerings to new customers?
- How much do the products cost?
- Are there adequate customer support and growth options?
- Are there useful surrounding management tools?
- Will our DevOps processes translate to these offerings?
- Can the PaaS offerings speed time-to-value and simplify things sufficiently, to drive stickiness?
- What security measures does the cloud provider have in place?
Based upon the evidence we think it’s pretty clear that AWS is still the leader among public cloud providers.
We’ll continue to track the AWS vs. Azure comparison, and as the companies’ offerings and pricing options grow and change – we’ll be interested to see how this evaluation changes in 2018.
When you start looking for an instance management tool to help manage your cloud infrastructure costs, you’ll realize there are a lot of options. While evaluating such tools, you need to make sure to have a list of requirements to make sure the software fits your needs and will help you reduce cloud waste. Here are a few items you might want to have on your checklist:
1. High visibility
One factor that contributes to cloud waste is the inability to track cloud instances. In today’s world, cross-cloud and cross-region are must-haves in order to provide high availability and true redundancy. Any modern instance management tool must be able to see all of your instances in one place, or you’re sure to have some fall through the cracks.
You might hate making reports, but solid reporting can be the difference between a well-informed organization and a proverbial dumpster fire. With the help of a good tool, you can generate reports that show the data you need for decision-making, without wasting time.
3. Takes Action
Sure, reports and pretty graphs are nice, but something needs to actually be acted upon in order to make any real difference to your monthly AWS or Azure bill! A lot of tools will gather up that data for you, but you really need something that can actually turn off the lights, so to speak — not just tell you which lights haven’t been turned off.
4. Simple to use UI
The user experience of an application can sometimes go unnoticed, but it’s often the difference between a useful tool and shelfware. One of the main difficulties in determining how easy an interface is to use is that you need to understand who the actual end user will be. The IT administrator who is evaluating products may be able to figure out the interface, but if other team members will need to use it, then their needs must be taken into account.
5. APIs and Automation
With the rise of DevOps practices and automated infrastructures, API access is a must. By enabling inbound actions and outbound notifications, new tools can work seamlessly with existing operations to eliminate wasted resources. Automation should also take into account your naming conventions and tagging standards for optimal integration.
6. Schedule Overrides
Once you’ve started working on solving your cloud waste problem by scheduling resources to turn off when not needed, you need to be able to adapt to the changing needs of the user and the organization. Anyone with proper access to a system should be able to override a given schedule if necessary, since any tool you use should be helping your users get work done.
7. Team Governance
A huge concern when letting users run wild with any new tool is how you can make sure they aren’t going to break anything. Giving someone the minimum required access is a security best practice, but sometimes those access controls can be confusing. In addition to a simple UI, the role-based access controls should also be simple to set up, modify, and understand.
8. Single Sign-On
Some might consider this a nice-to-have, but most enterprises today have started requiring this for all products they use. Users find it easy to sign in without remembering a million credentials, and admins find it more secure and faster to deploy. If SSO is being used within your organization, then you should start picking tools that integrate with it easily.
This is a starting point, but of course when evaluating an instance management tool, make sure to incorporate any unique needs your organization. What else would you include on your checklist?
“Is that old cloud instance running?”
Perhaps you’ve heard this around the office. It shouldn’t be too surprising: anyone who’s ever tried to load the Amazon EC2 console has quickly found how difficult it is to keep a handle on everything that is running. Only one region gets displayed at a time, which makes it common for admins to be surprised when the bill comes at the end of the month. In today’s distributed world, it not only makes sense for different instances to be running in different geographical regions, but it’s encouraged from an availability perspective.
On top of this multi-region setup, many organizations are moving to a multi-cloud strategy as well. Many executives are stressing to their operations teams that it’s important to run systems in both Azure and AWS. This provides extreme levels of reliability, but also complicates the day-to-day management of cloud instances.
So is that old cloud instance running?
You may get a chuckle out of the idea that IT administrators can lose servers, but it happens more frequently than we like to admit. If you only ever log in to US-East1, then you might forget that your dev team that lives in San Francisco was using US-West2 as their main development environment. Or perhaps you set up a second cloud environment to make sure your apps all work properly, but forgot to shut them down prior to going back to your main cloud.
That’s where a single-view dashboard (like the view you get with ParkMyCloud) can provide administrators with unprecedented visibility into their cloud accounts. This is a huge benefit that leads to cost savings right off the bat, as the cloud servers running that you forgot about or thought you turned off can be seen in a single pane of glass. Knowledge is power: now that you know it exists, you can turn it off. You also get an easy view into how your environment changes over time, so you’ll be aware if instances get spun up in various regions.
This level of visibility also has a freeing effect, as it can lead you to utilizing more regions without fear of losing instances. Many folks know they should be distributed geographically, but don’t want to deal with the headache of keeping track of the sprawl. By tracking all of your regions and accounts in one easy-to-use view, you can start to fully benefit from cloud computing without wasting money on unused resources.
Now with ParkMyCloud’s core functionality available for free, it’s easy to get this single view of your AWS and Azure environments. We think you’ll get a new perspective on your existing cloud infrastructure – and maybe you’ll find a few lost servers! Get started with the free version of ParkMyCloud.