You made the obvious move and migrated to Amazon Web Services (AWS). Months later, the attractive glow of the move from CapEx to OpEx spend has been dimmed by the reality of increasing monthly AWS bills. Your CFO wants to know what’s up.
This is a real challenge, according to the the head of infrastructure at a mid-sized software company we spoke with recently. Let’s call him Steve.
“I need to reduce my AWS costs as quickly as possible,” Steve told us. “My CFO saw that our AWS spend started at $20k per month when we migrated last year. Now it’s over $100k per month, which makes it one of our biggest line item expenses. We’re under a direct mandate: We have to bring it down.”
The problem is clear. But how did it get so bad, so quickly?
Your Infrastructure is Probably Exploding
“As we started to dive into it, we found that a large part of our spend is simply on waste,” Steve said. “With the rapid growth in our AWS use, we didn’t have visibility and policies in place. Our developers aren’t properly cleaning up after themselves, and resources aren’t being tracked, so it’s easy for them to be left running. It’s something we want to change, but it takes time and energy to do that.”
This is a familiar story for many small-to-medium sized businesses, whose agility allowed them to adopt the cloud early. But sometimes, the other side of the “agility” coin is a lack of defined processes, which leads to waste.
As Steve put it, “AWS built this awesome playground – everyone can play, but everything costs money.”
Amazon recently announced that AWS will likely be a 10 billion dollar per year business by the end of 2016. This is partly from their rapid gain in customers – and partly due to each of their customers spending more and more each year in their massive playground.
Beat Your CFO to It
There’s no need to wait for your CFO to come knocking on your door to take a swing at controlling AWS costs.
Visibility – start with taking a look at your existing environments. If your environment is large, a single dashboard view that shows multiple accounts and AWS regions can be helpful.
Communication – talk with your team and set clear guidelines around provisioning appropriately sized instances, managing existing instances, and stopping non-production instances when they are not needed.
Control – start taking action on these guidelines to optimize your AWS costs. Here are a few recommendations:
I’m sure that most people have heard of the Law of Unintended Consequences.
How often have we seen some new technology or gadget with promise be used for more nefarious purposes?
For example, I am convinced that when Ernest Holmes invented the tow truck back in 1916, he had a higher purpose in mind than using them to shake down citizens for parking revenues, to fill municipal coffers.
Indeed, the Law of Unintended Consequences is always in the back of the minds of scientists, engineers, inventors and entrepreneurs as they bring new inventions and products to market. Will these products take on an unintended life of their own? Will they be used for good or for evil purposes?
Occasionally, though, something good is invented and users stumble upon an even better way of putting it to use, in a way the inventor never dreamed of.
Our Intention: Snooze for an Hour
A few months ago we released simple capability within ParkMyCloud called “Snoozing”.
First some context: When you attach a parking schedule to an AWS instance, that schedule will enforce itself and maintain the desired instance state. When the schedule says an instance is supposed to be turned off, it will be off. When it is supposed to be on, it will be turned on.
That’s great for saving money on AWS instances that don’t need to be running 24×7, however, it can be problematic when a developer needs to come in on the weekend to get some work done and everything is turned off. Or they need to continue to work late on a weeknight, but a schedule is in place that shuts down development environments by 6 pm.
These common scenarios led several of our customers to ask us for a “snooze button” – the ability to suspend schedule action for a set period of time. So, we obliged. In fact, we integrated that ability with on/off toggles, so that a developer can select a parked instance and hit the toggle button to turn it on. The system will prompt him/her, asking how long they want to suspend the schedule. It then snoozes the schedule for that period of time for the instance(s).
The Unintended Consequence: Sleep Until Waking
Since we released the Start/Stop/Snooze functionality back in January of this year, a couple of our customers have taken the concept of saving money by parking instances and turned it on its ear.
They have taken the interesting approach of turning ALL of their non-production environments to be STOPPED by default, using a single parking schedule. They then require their development teams to snooze the schedule for the instances they will be using for the amount of time they will be working.
Using this approach, these users have not only maximized their monthly AWS cost savings, but they redeemed the time by avoiding the need to hammer out a set of parking schedules that suited everyone.
Bravo to them, on this innovative, out-of-the-box thinking!
Today, we reach the fifth and final post in our series of ways to save money on AWS. Way #5 to save is… ParkMyCloud! Read on, or watch the video version of this post:
ParkMyCloud is purpose-built to do one thing well, and that’s to schedule on/off times for EC2 instances in non-production environments, without scripting. We call that “parking.” You can think of “parked” as a new instance state between running and stopped.
Depending on the schedule you use, ParkMyCloud can achieve savings of up to 50-73%, making it better than Reserved Instances for non-production, without the annual commitment, without concerns about price cuts, and without having to pay upfront. It provides the same savings as Spot Instances without the risk of abrupt termination.
ParkMyCloud vs. Reserved Instances
So let’s look at the comparison with Reserved Instances a little more closely. I’ll show you why I think ParkMyCloud is better than Reserved Instances for non-production.
You’ll notice that I’ve added ParkMyCloud this time. The first blue ParkMyCloud column shows the costs when you use a typical schedule of running 12 hours a day, parked 12 hours a day, and parked on weekends. That savings is around 64%. To match that with Reserved Instances, you would actually have to pay the three-year upfront cost for Reserved Instances. Most people, because of the concern over price cuts, don’t use the three-year contracts, they stick to the one-year term.
The concern over price cuts does not apply to ParkMyCloud. If you have the 30% cut that you see in the “On-Demand w/ Price Cut” column, with ParkMyCloud, you would still get that 64% savings, but it’s against that new price point — see the “PMC w/ Price Cut” column.
How to Use ParkMyCloud
Unlike managing Reserved Instances, ParkMyCloud is very simple to use. Essentially, all you do is create a schedule, name it and save it (and that’s if you don’t want to use one of the schedules we provide):
Then you go to the dashboard and attach that schedule to one or more of your non-production instances.
Once you’ve attached those, we will predict what your 30-day savings will be. If you leave the schedules on for a period of time, once they start parking, you’ll see what savings you’ve achieved to date.
So let’s summarize the 5 ways to save on AWS EC2 we discussed.
Reserved Instances will save you about 31-43%, but the downside is, you’re locked in for 1-3 years. Additionally, there’s no protection against future AWS price cuts, and it’s definitely a use-it-or-lose-it situation. Although they can be used in both production and non-production, production is probably the best place for them.
We also talked about Spot Instances, which can routinely save 70-90%. However, because of long delays in request fulfillment and termination of instances on short notice, and the need for complex mitigation strategies, these are much higher risk, but there are definite use cases for both production and non-production as we saw.
We looked at Auto Scaling, which takes advantage of all these instance purchasing options. It makes it hard to pin down the savings and the risk, which depends on the purchasing options, and the rules you have.
We talked about scripting, but honestly I don’t think scripting is particularly cost-effective. It would give similar savings to what ParkMyCloud would, but take a greater cost to get there.
Lastly, today we talked about ParkMyCloud, where you can get 50-73% savings pretty simply, without the risks of Spot or Reserved Instances. One limitation is that we don’t recommend that you use ParkMyCloud in production instances. You can’t park Spot Instances, because you can’t stop them – they just terminate. And we don’t currently park Auto Scaling groups, although we are looking into doing this. But those limitations are small compared to the amount and simplicity of savings.
If this is of interest to you, I encourage you to sign up for a 30-day free trial. You can create an account and pocket the savings for the next 30 days while you try it out.
So far in the “How to Save Money on AWS” series, we’ve looked at three ways to save using different AWS purchasing options. Today, let’s look at the first way outside of AWS: scheduling on/off times for your idle EC2 instances.
Read on, or watch the video version below:
Using AWS Scripting to Schedule On/Off Times
Often non-production environments like development and staging are running 24×7, even though they are not being used during off-peak hours like at nights and on weekends. Therefore, the simplest way to save money on these environments is to turn them off when not in use. Of course, that’s easier said than done.
For example, in AWS, you could use data pipeline and script it up, but there isn’t anything native to the platform, and AWS has no plans to add anything.
You might also turn to any number of the cloud analytics platforms in the market. One thing they’ll recommend is for you to turn off these non-production instances when not in use. However, they’re basically like tattle tales – they tell you what’s wrong but don’t fix it for you.
Now, there are some cloud management platforms that can not only identify instances but actually take action, but they can be bloated, expensive, and quite complicated, and there’s a steep learning curve associated with those.
So what do people do when there’s a lack of viable options? Well, as a recovering command-line guy, I know that when the going gets tough, the tough start scripting. Scripting tends to be in many of our comfort zones: I know I’ve scripted a lot of things, and I’m sure I’m not alone. I get the appeal: you’re in control of your own destiny, you get to get your hands dirty, and at the end of the process, you get the satisfaction of having built something from start to finish.
The Problem with AWS Scripting
However, AWS scripting is not cost-effective, for several reasons.
First, creating the script is only half the battle. Once it’s built, you have to maintain it as your environment changes. If you have a large environment that’s changing frequently, even with the help of Chef, Puppet, or SaltStack, there’s added cost to keep up with that environment. And if you don’t maintain your scripts, you’re missing out on instances you could have turned off, which has a big cost associated with it — perhaps even more than you’re spending to maintain the scripts.
Lastly, how do you justify the fact that you’re working on these scripts to your boss? That requires some way of being able to show the cost benefit and do the tracking, and that will take time. Heaven forbid that your boss actually likes the report, thus requiring you to implement a more formal cost-tracking system.
While scripting on/off times for your instances might be a relatively easy fix in the short term, it’s not a sustainable long-term solution to the cost problem.
Stay tuned for next week’s post, the final in our series on how to save money on AWS.