As we continue to dive into AWS Reserved Instances, today we want to take a look at the AWS Reserved Instance Marketplace.
Reserved Instances are a great way to save money – unless they don’t get used, and you won’t really know until you get the bill. But just because you’re locked into that contract doesn’t mean that your unused RIs have to be a total waste of money. AWS has given users a place to sell them – theReserved Instance Marketplace.
Using the Reserved Instance Marketplace, you can list your reservation for other users to purchase. Of course, like any online marketplace, there’s no guarantee that you’ll actually sell them, but at least you have a shot at getting some of your money back.
AWS has some solid documentation for all the ins and outs of buying and selling in the Reserved Instance Marketplace, but we decided to highlight answers to some of the questions we most commonly see about how to get started with selling unused RIs. Read our FAQ below.
Selling on the Reserved Instance Marketplace
AWS customers and third-parties are free to use the marketplace to sell unused Standard RIs regardless of length terms or original pricing options.
When is it a good idea to sell unused RIs?
If you’re changing instance types (perhaps for rightsizing or better optimizing the instance type for its load or application), moving regions, your business needs have changed, your capacity needs have changed, or you just don’t need that instance type anymore – use the marketplace.
How do I become a seller?
To register as a seller, you’ll need to provide bank account and tax information. Once you’ve completed registration, you’ll receive a confirmation email.
Are there any restrictions or limitations to what I can sell?
Once you’ve registered as a seller, you’re free to sell any EC2 Standard Reserved Instances as long as your term length has at least one month remaining.
Convertible instances cannot be sold in the marketplace.
You can sell Standard RIs regardless of the purchasing plan (No Upfront, Partial Upfront, or All Upfront), but in the case of All Upfront – you must have made the full payment before you can sell, and the reservation must be active for at least 30 days before listing. AWS also charges a 12% service fee for upfront pricing.
Pricing is flexible – the minimum sale price is $0.00
You can’t modify or change a listing once it’s been made, but you can cancel it and create a new one.
What information does AWS share with buyers?
According to US regulations, buyers will be able to see your legal name on the buyer’s statement. In the event that AWS Support is contacted regarding invoices or tax purposes, the buyer may receive your email address to be able to communicate with you directly, along with your ZIP code and country.
How does selling work?
Once you list the RIs you want to sell in the marketplace, buyers will be able to see them. Instances are grouped by remainder of term length and hourly rate. The cheapest reservations are sold first, followed by the next cheapest, and so on until the buyer’s order is fulfilled. AWS handles the transaction and transfer of ownership. The instances are yours until they’re sold, and once you make a sale, you’ll go back to paying the on-demand rate whenever you use that instance type moving forward.
How do I list my RIs in the marketplace?
There’s a few ways you can list your unused RIs in the AWS Reserved Instances Marketplace. You can sell them all at once, in parts, or by instance type, platform, and scope. You can also cancel your listing, but you won’t get anything back on any portions that have already been sold. There are also several routes you can take for where and how to list your RIs: using the AWS Management Console, using the AWS CLI or Amazon EC2 API, and from the Listing State of the My Listings tab of the Reserved Instances page.
How do I price my RIs in the marketplace?
When selling an RI, the only fee that you can decide on is the upfront fee – the one-time fee that the buyer is charged for purchasing your instance. Usage and recurring fees cannot be specified – the buyer will pay what was charged for the original purchase. The minimum sales price allowed is $0.00 and the maximum you can sell per year is $50,000 (although AWS can grant you permission to sell more on a case-by-case basis).
AWS also sets a default pricing schedule for your listed RIs. Pricing decreases incrementally over a month-to-month period to account for the value of the RI decreasing over time. What you can do, however, is set upfront prices based on the point of sale for your RI (a set price if it’s sold with 5 months remaining in the term, 3 months remaining, etc).
What happens after I make a sale?
You’ll get an email notification anytime an RI has sold, and each day there is any activity on your account, such as creating or selling a listing. Once the buyer pays AWS for your RIs, you’ll get a message to your email account about the sold reservation. AWS sends a wire transfer to the bank account provided, typically 1-3 days from the date of sale, but you won’t be able to receive funds until after AWS has verified the account with your bank, which can take up to 2 weeks. You can also see your sales in the Reserved Instance disbursement report, where you can check the status of everything you’ve listed. Or you can track the status of your RI listings in the console (Reserved Instance > My Listings > Listing State) for a full breakdown of available listings, pending, sold, and canceled.
Reserved Instances can save money on your AWS bill, but can just as easily waste money by going unused. Luckily, the AWS Reserved Instances Marketplace can help by giving you a place to sell your unused RIs. To optimize your existing RIs, be sure to check out our Reserved Instance management solution.
Did we miss any of your questions in this AWS Reserved Instances Marketplace FAQ? Let us know!
In this blog we will look at the Google Cloud Committed Use discount program for customers that are willing to “commit” to a certain level of usage of the GCP Compute Engine.
The Committed Use purchasing option is particularly useful if you are certain that you will be continually operating instances in a region and project over the course of a year or more. If your instance usage does not add up to a full month, you may instead want to look at the Google Cloud Sustained Use discounts, which we discussed in a previous blog.
The Google Cloud Committed Use discount program has many similarities to the AWS and Azure Reserved Instances (RI) programs, and a couple unique aspects as well. A comparison of the various cloud providers’ reservation programs is probably worth a blog in itself, so for now, let’s focus on the Google Cloud Committed Use discounts, and the best times and places to use them.
Critical Facts about Google Cloud Committed Use
The Committed Use discount is best for a stable and predictable workload (you are committed to pay – regardless of whether you use the resources or not!)
Commitment periods are for either 1 or 3 years.
Commitments are for a specific region and a specific project. Zone assignment within a region is not a factor.
Discounts apply to the total number of vCPUs and amount of memory– not to a specific machine or machine type.
No pre-payment – the commitment cost is billed monthly.
GCP Committed Use discounts are available for all of the GCP instance families except the shared-core machine types, such as f1-micro and g1-small.
Committed Use discounts do not apply to the premium charges for sole-tenants, nor can they be used for Preemptible instances.
The commitments for General Purpose instances are distinct from those for Memory Optimized instances. If you have some of both types, you must buy two different types of Commitment. These types are:
Standard – n1-standard
High Memory – n1-highmem
High CPU – n1-highcpu
General purpose sole-tenant
How much does it cost?
Each Committed Use purchase must include a specific number of vCPUs and the amount of memory per vCPU. This combination of needing to commit to both a number of vCPUs and amount of Memory can make the purchase of a commitment a bit more complicated if you use a variety of machine types in your environment. The following table illustrates some GCP machine types and the amount of memory automatically provided per vCPU:
Memory per vCPU
0.9 – 6.5 GB
While the vCPU aspect is fairly straightforward, the memory commitment to purchase requires a bit of thought. Since it is not based on a specific machine type (like AWS and Azure), you must decide just how much memory to sign-up for. If your set of machine types is homogeneous, this is easy – just match the vCPU/memory ratio to what you run. The good news here is that you are just buying a big blob of memory – you are not restricted to rigidly holding to some vCPU/memory ratio. The billing system will “use up” a chunk of memory for one instance and then move on to the next.
Looking at a specific example, the n1-standard-8 in the Oregon region that we discussed in the Sustained Usage Blog, we can see that the Committed Use discount does amount to some savings, but one must maintain a usage level throughout the month to fully consume the commitment.
Recall from the earlier blog that the base price of this instance type in the GCP Price list already assumes a Sustained Usage discount over a full month, and that the actual “list price” of the instance type is $277.40, and Sustained Usage provides up to a maximum of a 30% discount. With that as a basis, we can see that the net savings for the Committed Use discount over 1 year is 37%, and over 3 years, rises to 55%. This is close to the advertised discount of 57% in the GCP pricing information, which varies by region.
The break-even points in this graph are about 365 hours/month for a 3 year commitment, and 603 hours/month for a 1 year commitment. In other words, if you are sure you will be using a resource less than 365 hours/month over the course of a year, then you probably want to avoid purchasing a 3 year Commitment.
Allocation of Commitments
Because Commitments are assigned on a vCPU/RAM basis, you cannot simply point at a specific instance, and say THAT instance is assigned to my Committed Use discount. Allocation of commitments is handled when your bill is generated, and your discount is applied in a very specific order:
To custom machine types
Sole-tenant node groups
Predefined machine types
This sequence is generally good for the customer, in that it applies the Commitment to the more expensive instances first. For example, an n1-standard-4 instance in Northern Virginia normally costs $109.35. If an equivalent server was constructed as a Custom instance, it would cost $114.76.
For sole-tenant node groups, you are typically paying for an entire physical machine, and the Committed Use discount serves to offset the normal cost for that node. For a sole-tenant node group that is expected to be operating 7x24x365, it makes the most sense to buy Committed Use for the entire system, as you will be paying for the entire machine, regardless of how many instances are running on it.
Commitments are allocated over the course of each hour in a month, distributing the vCPUs and RAM to all of the instances that are operating in that hour. This means you cannot buy a Commitment for 1 vCPU and 3.75 GB of RAM, and run two n1-standard-1 instances for the first half of the month, and then nothing for the second half of the month, expecting it all to be covered by the Commitment. In this scenario, you would be charged for one month at the committed rate, and two weeks at the regular rate (subject to whatever Sustained Usage discount you might accumulate for the second instance).
Thank you for….not…sharing?
Unlike AWS, where Reserved Instances are automatically shared across multiple linked accounts within an organization, GCP Commitments cannot be shared across projects within a billing account. For some companies, this can be a major decision point as to whether or not they commit to Commitments. Within the ParkMyCloud platform, we see customers with as many as 60 linked AWS accounts, all of which share in a pool of Reserved Instances. GCP customers do not have this flexibility with Commitments, being locked-in to the Project in which they were purchased. A number of our customers use AWS Accounts as a mechanism to track resources for teams and projects; GCP has Projects and Quotas for this purpose, and they are not quite as flexible for committed resource sharing. For a larger organization, this lack of sharing means each project needs to be much more careful about how they purchase Commitments.
Google Cloud Committed Use discounts definitely offer great savings for organizations that expect to maintain a certain level of usage of GCP and that expect to keep those resources within a stable set of regions and projects. Since GCP Commitments are assigned at the vCPU/Memory level, they provide excellent flexibility over machine-type-based assignments. With the right GCP usage profile over a year or more, purchase of Google Cloud Committed Use discounts is a no-brainer, especially since there are no up-front costs!
AWS offers a range of EC2 instance types optimized for various purposes. It’s great that they provide so much variety, but of course, it means one more thing that you have to learn.
We broke this down in a new video, which also compares EC2 purchasing options. Check it out here:
Or, read on for a look into each instance type. Remember that within each type, you’ll still need to choose instance sizes for your specific needs. Additionally, older generations within each instance types are available for purchase – for example, c5 is the latest “c” instance, but c4 and c3 are still available – but as the newer types tend to perform better at a cheaper price, you’ll only want to use the older types if you have an AMI or other dependency.
This image shows a quick summary of what we’ll cover:
These general purpose EC2 instance types are a good place to start, particularly if you’re not sure what type to use. There are two general purpose types.
t2 instance type
The t2 family is a burstable instance type. If you have an application that needs to run with some basic CPU and memory usage, you can choose t2. It also works well if you have an application that gets used sometimes but not others. When the resource is idle, you’ll generate CPU credit, which you’ll utilize when the resource is used. It’s a cheaper option that’s useful for things that come and go a lot, such as websites or development environments.
We’ll also add a mnemonic to help you remember the purpose of each instance type.
Mnemonic: t is for tiny or turbo.
m5 instance type
The m5 instance type is similar, but for more consistent workloads. It has a nice balance of CPU, memory, and disk. It’s not hard to see why almost half of EC2 workloads are on “m” instances.
There’s also an m5d option, which uses solid state drives (SSD) for the instance storage.
Mnemonic: m is for main choiceor happy medium.
c5 instance type
The c5 instance type has a high ratio of compute/CPU versus memory. If you have a compute-intensive application – maybe scientific modelling, intensive machine learning, or multiplayer gaming – these instances are a good choice. There is also the c5d option, which is SSD-backed.
Mnemonic: c is for compute (at least that one’s easy!)
r4 instance family
The r4 instance family is memory-optimized, which you might use for in-memory databases, real-time processing of unstructured big data, or Hadoop/Spark clusters. You can think of it as a kind of midpoint between the m5 and the x1e.
Mnemonic: r is for RAM.
x1e instance family
The x1e family has a much higher ratio of memory, so this is a good choice if you have a full in-memory application or a big data processing engine like Apache Spark or Presto.
Mnemonic: x is for xtreme, as in “xtreme RAM”seems to be generally accepted, but we think this is a bit weak. If you have any suggestions, comment below.
p3 instance type
If you need GPUs on your instances, p3 instances are a good choice. They are useful for video editing, and AWS also lists use cases of “computational fluid dynamics, computational finance, seismic analysis, speech recognition, autonomous vehicles” – so it’s fairly specialized.
Mnemonic: p is for pictures (graphics).
h1 instance type
The h1 type is HDD backed, with a balance of compute and memory. You might use it for distributed file systems, network file systems, or data processing applications.
Mnemonic: h is for HDD.
i3 instance type
The i3 instance type is similar to h1, but it is SSD backed, so if you need an NVMe drive, choose this type. Use it for NoSQL databases, in-memory databases, Elasticsearch, and more.
Mnemonic: i is for IOPS.
d2 instance type
d2 instances have an even higher ratio of disk to CPU and memory, which makes them a good fit for Massively Parallel Processing (MPP), MapReduce and Hadoop distributed computing, and similar applications.
Mnemonic: d is for dense.
What EC2 instance types should you use?
As AWS has continued to add options to EC2, there are now EC2 instance types for almost any application. If you have comparison questions around pricing, run them through the AWS monthly calculator. And if you don’t know, then generally starting with t2 or m5 is the way to go.
Among the various ways to lose money on idle or orphaned resources in the cloud, here’s another for AWS users to add to the list: unused reserved Instances. At first, the idea of wasting money on AWS Reserved Instances seems counterintuitive. After all, aren’t RIs meant to save money? The short answer is yes – but only if you use them efficiently.
How Unused Reserved Instances Occur
To understand how unused Reserved Instances contribute to cloud waste, consider how they work. With AWS Reserved Instances, you’re making a commitment of usage by renting instances for a fixed amount of time in exchange for a lower rate (per-hour or per-second) than on-demand. You’re still free to use all the same families, OS types, and instance sizes with either one, except with RIs your ability to use certain instance types is limited to the purchasing plan you choose.
The only real difference between an AWS On-Demand instance and an AWS Reserved Instance is how you get billed for them on the backend – and this is where it gets tricky. You don’t know if your Reserved Instances have been used until you get the bill. Instead, you run your instances as you always would, with no insight into what will get billed as reserved instances. It’s only when your bill is created the following month that AWS reviews your reservations alongside your usage to apply the Reserved Instances that match up with your workload. This leaves you with little visibility into what your costs will be, forcing you to track usage on your own, and running the risk of unused reservations that result in, you guessed it – wasted money.
Ways to Avoid Losing Money Unused Reserved Instances
Reserved Instances require commitment of usage, ongoing awareness and insight into your future costs, and the possibility of going unused if AWS can’t apply them sufficiently. But that doesn’t mean you should shy away from using them. Reserved Instances can be cost-effective if used with a few things in mind:
Pick the RI type that suits your usage and workload. The best plan is one of prevention. Before you get started with purchasing reservations, get a detailed look at your usage and the most optimal instance types for your workload (something you should already be doing as part of your cost control measures). By design, Reserved Instances work best with steady state workloads and consistent usage. Once you confirm that your usage makes you a good candidate, you’ll want to choose the RI instance type that will benefit your needs most:
Standard RIs – Recommended for steady-state usage, and provide the most savings.
Convertible RIs – a smaller discount from On-Demand instances, but in return provide flexibility to change families, OS types, and tenancies.
Scheduled RIs – similar to Standard RIs, but only apply to instances launched within the time windows you select, which can recur on a daily, weekly, or monthly schedule.
Sell unused reserved instances on the Reserved Instance Marketplace. Using the marketplace allows you to list your reservation for purchase by other users. The cheapest reservations are sold first, and once someone purchases yours, you’ll be the charged the on-demand rate whenever you use that instance type moving forward.
Purchase convertible reservations. With convertible reservations, you have the option to convert your reserved instances to other types, so long as the new type is more expensive. You won’t get as much of a discount, but flexibility and more options for use make up for the smaller savings.
The Lesson to be Learned
Just like any other idle or unused cloud resource, unused reserved instances can only do one thing – waste your money. Cloud services were meant to help you keep infrastructure costs in check, but only if you use them smartly. Does all this sound complex to manage? That’s because it is. Take the DIY route by keeping an eye on your infrastructure, or put our automated solution to work and check out our Reserved Instance manager.
Implementing DevOps practices in small organizations seems like standard practice, but what if you’re trying to utilize DevOps in large organizations? Trying to modernize workflows can be a challenge for any company, but there are different challenges, risks, and benefits for bigger companies. Let’s take a look at how enterprises might approach a DevOps transformation through a few of the core tenants of DevOps.
There are a few different forms of feedback that come with DevOps: automated feedback about specific code (typically through unit and integration testing software), personal feedback from other team members, consumer feedback from customers using your product, and cross-team feedback throughout the organization. Startups and small companies may find it easier to have open lines of communication between individual team members as well as across teams.
Large organizations will need to make a conscious effort to keep team communication open, On the other hand, they will have more resources available (both money and employees) to field customer and in-house feedback about individual services or larger products. They may also be able to better purchase and implement automated testing and CI/CD tools, which leads to…
One of the biggest tech benefits to a DevOps approach is automating away the manual tasks that bog down critical projects. Large organizations often have the time, money, and people to set up automated tools, like CI/CD pipelines, unit and integration test suites, and config management systems. The biggest challenge in the enterprise world is trying to make everyone happy.
One approach is to standardize on a single tool for each purpose, such as Jenkins or Chef. This can enable your IT staff to specialize in those tools, but may make some users unhappy with being forced into a tool they may not prefer. The alternative is to allow each team or business unit to use their own preferred software, but this can turn into a “toolset hell” with a mashup of every combination of applications within your organization. Each approach has its pros and cons, and often comes down to a management decision.
Having individual teams that handle their part of the puzzle and nothing else is the biggest hurdle that enterprises face when trying to apply DevOps principles. The combination of ‘dev’ and ‘ops’ (and other disciplines, like ‘sec’ and ‘fin’) is naturally split out in a large organization, so recombining them can be a huge undertaking. Then again, that gap is exactly the problem the DevOps approach seeks to solve.
Some companies solve this by having a separate team that handles the cross-team support and communication. Other companies break down these silos by enabling employees to seamlessly migrate between teams depending on the project or application. The more “devopsy” method is to utilize ChatOps and centralized documentation repositories for open communication and collaboration, which can help break down unify the distinct teams.
The idea of holistic thinking tends to come easier to larger organizations, as successful enterprises typically have a system in place for “big picture” thinking, either through a management or product team, or through a cross-functional committee. That said, communication of this vision down to the employees, along with communication up to that management team, is crucial for enabling outside-the-box thinking to get past any roadblocks and hurdles that are in the way of creating and deploying the end product. Sometimes, the hardest part is convincing programmers that not everything needs to be solved with code!
DevOps in Large Organizations: Challenging but Rewarding
Some folks think that DevOps only applies to startups and small companies, but we’re seeing more and more teams benefit from implementing DevOps in large organizations. The benefits of the above DevOps principles are numerous, but frequently come with a different set of challenges based on your organizational size. Once you are aware of those challenges and have a plan to overcome them, you can start to transform your enterprise to a DevOps shop.