“How do I stop wasting money on Reserved Instances?”
It’s a question we’ve heard before from despairing AWS users. They were told Reserved Instances (RIs) would save them money, so they purchased them. Now, halfway into a three-year contract, they realize they’re not utilizing the RIs they’re paying for. Or worse… they may not even know what RIs they have.
Amazon offers Reserved Instances to ostensibly help get your cloud costs in control. The message is that RIs help you save money on your EC2 instances by offering discounted hourly rates in exchange for a 1- or 3-year commitment. Before we get into how you can cut your cloud spending with an AWS RI, here’s a bit of background and what you need to know about AWS EC2 Reserved Instance pricing.
How do EC2 Reserved Instance Purchasing Options Work?
When it comes to Reserved Instances purchasing options, you can either choose a 1- or 3-year contract. The longer the commitment, the greater the cost savings compared to On-Demand. By choosing one of these contracts, customers are promised savings of up to 75%.
There are a few risks that come with the longer commitment times. For starters, if AWS drops pricing, then the promised savings are reduced or may disappear. And when AWS introduces a new generation of an instance type family it may attract your users away from your contracts – these are based on the older generation. If you don’t know your future needs, it may be appealing to use the 1-year instead of a 3-year contract, which has savings vs. On Demand at about 31-40%.
There are three different types of EC2 Reserved Instances that customers can purchase – Standard Reserved Instances, Convertible Reserved Instances, or Scheduled Reserved Instances. With Standard Reserved Instances, customers would see the most significant savings. However, Convertible Reserved Instances are attractive to customers because it gives them added flexibility like the ability to use different instance families, operating systems, or tenancies over the term. Scheduled RIs allow you buy an RI that is only used at certain times each day in a recurring schedule.
When an RI expires, you are charged again at the normal rate. See the recently released option to queue RI purchases in advance. This may help provide the greatest savings by eliminating gaps in your coverage from reservations.
Additional Ways To Save
AWS also offers additional discounts if you have more than $500,000 worth of Reserved Instances in a region – the more Reserved Instances you have, the larger your discount.
You may also buy RIs on the Reserved Instance Marketplace from third-party sellers. The great thing about this is that these third parties tend to list their RIs at lower prices for a shorter period of time. And if you find you have too many RIs, you can sell them on the Marketplace as well.
There are three different payment plans offered with Reserved Instances. Payments can be made either All Upfront, Partial Upfront, or No Upfront. It is important to note that if you pay all up front, you will have greater savings because there are no other costs or additional charges during the term regardless of the usage hours.
Some may think that the need to pay upfront and be locked in undermines both “pay as you go” and the notion of being “elastic”- almost like a step backward to the old economic model.
An example of the savings offered by each EC2 RI option, along with the percent of savings each has over the On-Demand price is shown below. From these graphs, you can see that with a 3-year contract, your savings would be much greater. Other things to note is that you will have greater savings with Standard Instances, as well as if you choose the “All Upfront” payment plan. While you would receive discounted hourly rates for choosing Partial Upfront or No Upfront as a payment plan, if you can, All Upfront would be your best option with the most savings.
How should I use my Reserved Instances?
In non-production environments such as dev, test, QA, and training, Reserved Instances are not your best bet. Why is this the case? These environments are less predictable; you may not know how many instances you need and when you will need them, so it’s better to not waste spend on these usage charges. Instead, schedule such instances (preferably using ParkMyCloud). Scheduling instances to be only up 12 hours per day on weekdays will save you 65% – better than all but the most restrictive 3-year RIs!
Reserved Instances are very much a “use it or lose it” proposition. In other words, there are no rollover minutes – if you don’t use your reserved instances one month you don’t get extra time the next month. Here’s why they are like this:
The EC2 options available are specific to Region, Availability Zone, Instance Type (e.g. m5.large) with some exceptions, Platform Type (e.g. Linux or Windows), and Tenancy. AWS, behind the scenes, attempts to randomly match instances you launch to the Reserved Instance contracts you have in place, based on the specific criteria. When there is a match, the cost benefit is applied. It is not uncommon for people to believe they are launching instances that match all the criteria, when in fact they are not, so the contracts are under-utilized. And you won’t know what matches were made until you get your bill at the end of the month.
AWS decrements the contract amount for every hour when not used, meaning your return on investment diminishes.
For every hour in your RI term, you pay the fee for hourly usage regardless of whether there has been any usage during that hour.
Given all of the tradeoffs mentioned above, Reserved Instances make the most sense in a production environment, where instances need to always be “on.”
How ParkMyCloud Can Help Manage Your Reserved Instances
ParkMyCloud is an easy to use platform that allows users to automatically identify and eliminate wasted cloud spend. You can use the ParkMyCloud platform to fully optimize your non-production instances without committing to an AWS EC2 RI term that will go underutilized. The platform does this by scheduling, rightsizing, and identifying idle instances. Recently, we added the ability to view all your existing Reserved Instances in the platform so you can better track what commitments you have already made, with more optimization functionality coming soon.
With ParkMyCloud, you can create parking schedules that automatically turn EC2 instances on and off according to your specifications. ParkMyCloud provides customized parking recommendations based on criteria provided by the user, which makes identifying “parkable” instances easier – and you can automatically accept these recommendations if you like. Turning this into an automated process cuts down on time and costs, thus further optimizing your cloud environments. Another perk of ParkMyCloud is that the platform tracks costs, projected 30-day savings, and actual savings for the current month – giving you better visibility.
ParkMyCloud easily achieves EC2 savings of 50-73% with no annual commitment, upfront payment, or risk of instance termination or price cuts. In fact, we had a customer cancel a $10,000 order for AWS Reserved Instances in favor of EC2 instances that they could turn on and off after they found out just how easy and powerful this cost savings tool can be. Here are some of the advantages that come with using ParkMyCloud:
No commitment or upfront payment
Price cut protection
Try out ParkMyCloud for yourself and get started parking your non-production systems and RightSizing your resources to ensure that your environments are running in the most efficient way possible.
As we continue to evaluate ways to automate various aspects of software development, today we’ll take a look at Google Cloud Composer. This is a fully managed workflow orchestration service built on Apache Airflow that makes workflow management and creation simple and consistent.
The evolution of hybrid and multi-cloud environments continue to grow as enterprises want to take advantage of the cloud’s scalability, flexibility, and global reach. Of the three major providers, Google Cloud has been the most open to supporting this multi-cloud reality. For example, earlier this year, Google launchedAnthos, a new managed service offering for hybrid and multi-cloud environments to give enterprises operational consistency by running quickly on any existing hardware, leverage open APIs and give developers the freedom to modernize. But, implementing the management of these environments can be either an invaluable proposition for your company or one to completely challenge your infrastructure instead – which brings us to Google’s solution,Cloud Composer.
How does Google Cloud Composer work?
With Cloud Composer, you can monitor, schedule and manage workflows across your hybrid and multi-cloud environment. Here is how:
As part of Google Cloud Platform (GCP), Cloud Composer integrates with tools like BigQuery, Dataflow, Dataproc, Datastore, Cloud Storage, Pub/Sub and Cloud ML Engine, giving users the ability to orchestrate end-to-end GCP workloads.
It has one-click deployment built-in to give you instant and easy access to a range of connectors and graphical representations that show your workflow in action.
Cloud Composer allows you to pull workflows together from wherever they live, supporting a fully-functioning and connected cloud environment.
Since Cloud Composer is built on Apache Airflow – an open-source technology – it provides freedom from vendor lock-in as well as integration with a wide variety of platforms.
Simplifying hybrid and multi-cloud environment management
Cloud Composer is ideal for hybrid and multi-cloud management because it’s built on Apache Airflow and operated with the Python programming language. Using open-source technology and the “no lock-in” approach and portability gives users the flexibility to create and deploy workflows seamlessly across clouds for a unified data environment.
Setting up your environment is quick and simple. Pipelines created with Cloud Composer will be configured as DAGs with easy integration for any required Python libraries, giving users of almost any level the ability to create and schedule their own workflows. With the built-in one-click deployment, you get instant and easy access to a range of connectors and graphical representations that show your workflow in action.
However, costs can be a drawback to making the most of your cloud environment when using Cloud Composer. Landing on specific costs for Cloud Composer can be hard to calculate, as Google measures the resources your deployments use and add the total cost of your Apache Airflow deployments onto your wider GCP bill.
Cloud Composer Pricing
Pricing for Cloud Composer is based on the size of a Cloud Composer environment and the duration the environment runs, so you pay for what you use, as measured by vCPU/hour, GB/month, and GB transferred/month. Google offersmultiple pricing units for Cloud Composer because it uses several GCP products as building blocks. You can also use the Google Cloud Platformpricing calculator to estimate the cost of using Cloud Composer.
So, should you use Google Cloud Composer? Cloud Composer environments are meant to be long-running compute resources that are always online so that you can schedule repeating workflows whenever necessary. Unfortunately, since you can’t turn on and off a Cloud Composer environment; you can only create or destroy, it may not be right for every environment and could cost more than the advantages may be worth.
Organizations that utilize Microsoft Azure as their cloud service provider have free access to Microsoft Azure Cost Management as a part of their subscription. Much of this originates from cloud monitoring and analytics tool, Cloudyn, which Microsoft acquired in July 2017. After the acquisition, Microsoft started migrating Cloudyn features into their Azure Cost Management portal and began offering it to their paying customers. The tool helps you monitor your cloud spending, increase your organizational accountability, and optimize your cloud efficiency. Let’s take a look at each of these features and see how well it performs in each.
Monitor Your Cloud Spending
The reports available in Microsoft Azure Cost Management help you view your past usage and costs while also allowing you to project your future spending. These costs can be viewed in daily, monthly, or yearly views, so you can see trends and anomalies across smaller or larger time frames. This data is pulled straight from Azure (or AWS, if you want to pay 1% of your AWS bill), so it helps for breaking down your raw cloud bill information.
Increase Your Organizational Accountability
Microsoft Azure Cost Management reports have the ability to be broken down in different ways by using “cost entities” to split resources into different buckets. These entities are often aligned with specific projects or departments within your organization, and can correlate with users or Azure subscriptions. Further, you can create “cost models” to split resources based on tags from your raw billing information.
Once the cost entities and cost models are in place, true accountability comes from having users log directly into Azure Cost Management to see and explore the costs associated with the teams and projects that they are a part of. On top of this, Azure Budgets can be set to alert or limit individuals or teams from overspending (or at least attempt to prevent it through warnings).
Optimize Your Cloud Efficiency
Even though this is a core tenant of Microsoft Azure Cost Management, optimization is one of the weakest features of the product. The essence of the documentation around this is that you should manually eliminate waste, without going into much detail about what is being wasted or how to eliminate it. Plus, this expects manual intervention and review of each resource without giving direct actions to eliminate the waste.
At ParkMyCloud, we believe that continuous cost control comes from actual action. We’ve created this for our customers through a simple UI (with full RBAC), smart recommendations with one-click remediation, and an automatic policy engine that can schedule your resources by default based on your tagging or naming conventions. Our multi-cloud platform will help you reduce cloud waste and maximize the value of your cloud. Start a trial today to see the automation in action!
The AWS reserved instance (AWS RI) offerings got a recent upgrade with the release of a “queue” function. This means that you can now purchase reserved instances that, rather than going into effect immediately, are scheduled for future purchase. (Yes – despite the fact that RI’s have been available for a decade, this is a new feature!)
Back up – what was released?
If you haven’t used AWS RIs before, it’s worth a brief primer. When you purchase a reservation, you’re not buying a specific instance or even capacity: it’s a billing function. In exchange for a commitment over 1 or 3 years, you get an attractive discount. These discounts are applied on the back end of the billing process, and are allocated against specific instances on an hour-by-hour basis over the course of the month.
There are a few variations within the AWS RI purchasing options, such as the term; how much you pay upfront vs. monthly; the option for them to be scheduled; whether the scope of the discount covers instances in a single region or in a particular availability zone; etc.
More on those options and whether you should actually be using Reserved Instances, in this post. (TL;DR: RIs are the right choice when you have 24×7 long-term production workloads; otherwise they’re usually not.)
So, the new feature is the option to purchase these reservation discounts to begin on a future date rather than immediately. This is designed to make it easier for users to have uninterrupted reserved instance coverage. Previously, at the end of a 1- or 3-year term, many users would be unaware that their reservation expired and would have a spike in cost…which they may or may not notice.
How does queuing work?
Now, when planned correctly, you can avoid the lapse of Reserved Instance coverage for your workloads by scheduling a new reservation purchase to go into effect as soon as the previous one expires. The furthest in advance you can schedule a purchase is three years, which is also the longest RI term available.
Before queueing was available, customers had the option to either just go ahead and purchase a new reservation a few days/hours/weeks before the previous RI was due to expire, or set a reminder to go in and buy a new reservation after the previous one had lapsed. Either way, there was an extra cost – either a time window with too many RIs, or one with too few. So it is easy to see that RI queueing can save you money. Queueing can also save you some hassle, as you no longer have to set reminders and build your daily/weekly schedule around going in to buy a new RI. (Reminiscent of some late-night eBay sessions, waiting for the end of an auction to roll around.)
There are a few limitations. AWS RI purchases can be queued for regional Reserved Instances, but not zonal Reserved Instances. Regional RIs are the broader option as they cover any availability zone in a region, while zonal RIs are for a specific availability zone and actually reserve capacity as well.
Cancellation is an option: since payment is processed only at the scheduled purchase time in the queue, you can cancel a purchase at any time before it is processed.
We find it interesting that these are designed as new purchases rather than a “renewable” RIs – likely due to an idea that users may queue an evolving RI type or purchase profile, instead of the same instance type/duration/payment terms over time.
Beware the AWS RI Black Hole
Of course, the downside to queuing a purchase in advance is that you now have a new commitment to track – and one that may not meet your needs by the time the purchase goes into effect.
It’s already difficult to shine light on your existing reservations, especially with options in place such as instance size flexibility and the broad applicability of regional RIs.
That’s why ParkMyCloud has released our first support for Reserved Instances this week. You told us that RIs are the next biggest thing that need optimization help on your cloud bills, and we listened. Now, you can see all your AWS RIs – past, present, and queued future purchases – in one place in ParkMyCloud. Next, we’ll be working on more recommendations and optimization – stay tuned!
Azure market share appears to be growing within the cloud computing race – both at large and within our own customer base here at ParkMyCloud.
As multi-cloud enthusiasts, we keenly observe the various commentator speculations about the winners and losers in the three-horse race between AWS, Azure and GCP that is the public cloud market. When quarterly results are reported, the tech news cycle buzzes for days, and what they choose to highlight can set the tone in the news.
One of the side benefits of reviewing the utilization of our customers in the ParkMyCloud platform is to compare what we see to what the market sees. Our customer base is of course a non-random sample from the cloud IaaS market, but we definitely see a number of trend correlations which do seem to speak to changes in this highly competitive marketplace.
Azure Market Share Among ParkMyCloud Users
One trend we recently spotted was an uptick in the relative proportion of Azure accounts and resources being managed within ParkMyCloud. Over the last six months or so, the proportion of customers using Azure exclusively has increased from roughly 10% to 20% – not to mention the handful using Azure in addition to one of the other major providers. Meanwhile, the proportion of our customers using solely AWS decreased slightly, while Google Cloud and the multi-cloud combinations remained roughly flat.
Azure Market Share at Large
Is this growth reflected in the market at large? Last quarter’s earnings reports and market outlook align with what we observed in our small sample. According to a recent KeyBanc report, Amazon lost almost 6% stake, while Microsoft Azure went from 26% to 30% and Google successfully grew its share from 8% to 10% in the cloud business. As the report’s author stated:
“AWS has a formidable lead and first-mover advantage in IaaS and is maintaining AWS estimates for this year and next, but the slowdown warrants further investigation into multi-cloud competitive dynamics”.
Still, Microsoft’s Azure cloud computing unit reported incredible revenue gains in their filings with its revenue increasing by 91% in FY18 and 72% in FY19. This growth has underpinned the overall performance of the entire Microsoft business and the consensus seems to be that Azure’s cloud momentum is still in its early days of playing out within the company’s massive install base. As shown in the chart above Azure’s growth has consistently been above the current 65% growth rate, and for much of the last five years has been close to doubling annually. Some have argued that the growth is slowing, which it is, but nevertheless it’s still at an impressive rate and even if it dropped to AWS levels would still be remarkable even by tech standards. After all, there is a key size after which the growth requires such a huge segment of the available market that it’s impossible to maintain early adoption rates.
Another key indicator of growth is Microsoft’s stock price, which as of this week has nearly matched its all-time high. Many cite Azure as a key driver of this growth, also noting that Azure’s customer skew toward larger enterprises protect it from some of the market volatility that AWS and Google Cloud’s large proportion of startup customers leave them vulnerable to.
What’s Driving Azure’s Growth?
While AWS has long been seen as an innovator, Azure has the advantage of being the default option with the ability for large enterprises using other Microsoft products to roll Azure into existing contracts.
One interesting idea is whether Azure is growing its customer base at a risky rate compared to its infrastructure capacity. For example, we’ve seen anecdotal complaints regarding low availability of most sizes of low priority VMs, which may indicate a lack of excess capacity. On the other hand, we do not know of any widespread availability issues outside of this “spare capacity” offering, which indicates a razor’s edge balance of supply and demand thus far.
Join us to Talk All Things Azure at Microsoft Ignite
If you enjoy discussing Azure market share and features, then come and discuss not only how to optimize your public cloud spend in Azure but also your own views on this fascinating market. You will find us at Microsoft Ignite in a few weeks. For Microsoft Ignite, November 4-8, we’ll be joining our parent company Turbonomic at booth #1713 in the expo hall. Schedule a time to stop by – we’d love to chat.