When we talk about cloud migration challenges, the conversation is about a company switching their workloads from an on-premise datacenter to a public cloud environment. But what about cloud to cloud migration?
The Benefits of Cloud to Cloud Migration
Why would a company go through the trouble of moving its entire infrastructure to the cloud, investing in one cloud service provider only to switch to another?
The cloud shift is no longer anything new. Companies have accepted cloud adoption and are becoming more comfortable with using cloud services. Now with AWS, Azure, and Google Cloud Platform currently leading the market (plus others growing rapidly), and constantly offering new and better options in terms of pricing and services, switching providers could prove to be fruitful.
Choosing a cloud provider to begin with is a monumental task. Businesses have to make choices regarding a number of factors – cost, reliability, security, and more. But even with all factors considered, business environments are always changing. Cost can become more or less important, your geographical region might evolve (which affects cost and availability of services), and priorities can shift to the point where another platform might be a better fit.
Perhaps your migration to AWS a few years ago was driven mainly by reliability and risk mitigation. While other providers were up and coming, you wanted to go with the gold standard. A few years later, productivity tools like Google’s G Suite became useful to your business. You now have business partners using other platforms like Azure or Google Cloud. You realize that your needs for software have changed, business partnerships have influence, and it becomes clear that another provider could be of greater benefit. Not to mention, cloud services themselves are ever-changing, and you might find better pricing, service-level agreements, scalability, and improved performance with another provider as offerings change over time.
While all of this makes sense, theoretically speaking, let’s take a look at a real example:
The Case of GitLab
A number of users were up in arms over Microsoft’s acquisition of Github, so much so that hundreds of thousands have already moved to another Git-repository manager – GitLab. And in a twist of fate, GitLab has made the announcement that they’ve decided to swap Microsoft Azure for another cloud provider – Google Cloud Platform.
Ask Andrew Newdigate, the Google Cloud Platform Migration Project Lead at GitLab, about why they’re making the move to GCP and he’ll likely mention service performance, reliability, and something along the lines of Kubernetes is the future.
Kubernetes, the open source project first released by Google and designed for application management of multiple software containers “makes reliability at massive scale possible.” What’s also appealing is that GitLab gets to use Google Kubernetes Engine, a service designed to simplify operating a Kubernetes cluster, as part of their cloud migration. The use of GKE has been cited as another driving factor for GitLab, looking to focus on “bumping up the stability of scalability of GitLab.com, by moving our worker fleet across to Kubernetes using GKE.”
Sid Sijbrandij, CEO of GitLab, adds better pricing and superior performance as reasons behind the migration. In an interview with VentureBeat, he said:
“Google as a public cloud, they have more experience than the other public cloud providers because they basically made a cloud for themselves […] And you find that in things such as networking, where their network quality is ahead of everyone else. It’s more reliable, it has less jitter, and it’s just really, really impressive how they do that, and we’re happy to start hosting Gitlab.com on that.”
The Challenges of Cloud to Cloud Migration
There’s a long list of factors that influence a company’s decision in selecting a cloud provider, and they don’t stop once you start building infrastructure in a particular cloud. Over time, other providers may prove to be better for the needs of your business. But just as there are challenges with cloud adoption in the first place, similar challenges apply when making the switch from cloud to cloud:
- Data transfer. Transferring data between different cloud service providers is a complex task, to say the least. Like data transfer from enterprise to cloud, information is transferred over the internet, but between cloud providers instead from server to cloud. This presents the issue of speed at which data downloads, and as a rule of thumb you should avoid transferring large chunks of data at a time. There can even be massive transfer costs of moving the data out of or into a cloud.
- Potential downtime. Downtime is also a risk. It’s important to account for inconsistencies in data, examine network connections, and prepare for the real possibility of applications going down during the migration process.
- Adapting to technologies for the new cloud. You built an application for Azure, but now you’re going Google – it’s not as simple as picking it up from one platform and expecting it to run on another (and with the same benefits). Anticipate a heavy amount of time spent reconfiguring the application code to get the most out of your new platform.
- Keeping costs in check. Consider the time and costs to migrate to the cloud, which tend to be misunderstood or drastically understated. Again, the same applies for cloud to cloud migration. By now, you have a better understanding of cloud service offerings, pricing models, and the complexity of a cloud adoption budget – for the service you were using. Once again, you’ll have to evaluate all of these costs and look into options that will help you save post-migration, like optimization tools.
Cloud to Cloud Migration – Is it worth it?
Before shifting to the cloud, you probably asked yourself the same thing. And just like before, you’ll have to dive deeply into factors like costs, technologies, and risk versus reward to assess whether or not a cloud to cloud migration is the right move for your business.
At first glance, a cloud to cloud migration is just as complicated and time-consuming as moving to the cloud in the first place, and it might seem like it’s just not worth the effort. But why did you move to the cloud? If you did to save costs over time, create better business opportunities, improve reliability and performance – then why would you NOT go with another provider that will benefit your business more in those areas? Not to mention, the more time you spend with one provider, building more applications as you go, the harder it will be to make the switch.
So cloud to cloud migration – is it worth it? Yes – but only if you’ve considered all the factors to determine whether or not another cloud is better for your business.
This AWS Reserved Instances FAQ is a culmination of questions we are often asked about using Reserved Instances to save on your Amazon Web Services (AWS) costs.
First, let’s make sure you’re all on the same page. AWS Reserved Instances are a way to save money on your cloud bill by reserving capacity in advance, which you can choose to pay for all, some, or no upfront. Due to the different billing model and variety of options, these often raise questions. Let’s explore a few:
Q1: How do I know which instances are reserved?
Because of how AWS billing works, Reserved Instances aren’t applied until you’re actually being charged for the instances. Think of it more like a “voucher” than a specific virtual machine.
This means that you run your instances as you normally would throughout the month, without seeing which will be billed as reserved vs. on-demand. When your bill comes on the 5th of the next month, AWS will look at your reservations and your usage and automatically apply the reservations that match the instances.
This means that you don’t have to decide which instances are reserved, but it also means you don’t have the visibility into what your true costs are going to be. You’ll have to track this on your own, or use some tools to help you manage your Reserved Instances. AWS has included some reports in the Cost Explorer that can help you visualize your month-to-date usage of your reservations.
Q2: What if I don’t need that instance type anymore?
There’s a couple of options for unused reservations, depending on the reason you aren’t using them anymore. The first option is to sell the reservation on the Reserved Instance Marketplace. This marketplace lets you list your reservation and allows others to purchase it from you. The reservations are grouped together, with the cheapest being sold first in that group. Once someone purchases your reservation, you start getting charged at the on-demand rate (if you still use that type).
The other option is to purchase convertible reservations, which allow you to change the attributes of the reservation (as long as it’s to a more expensive instance type). These convertible types give you much more flexibility, especially as your company or application are growing in use, but they do come with a smaller savings rate over standard Reserved Instances.
Q3: How do Reserved Instances work if I have multiple AWS accounts?
Many users of AWS in large organizations have started using a multi-account strategy, either for security reasons or billing reasons. When managing multiple AWS accounts, billing and reservations can get confusing. The good news is that reservations can “float” across multiple linked accounts, even if they weren’t purchased in the master account. Amazon is smart enough to attempt to apply the reservation to the account it was purchased in, but if a suitable instance is not found, it can apply it to an instance in a different account that matches. You can also choose to exclude linked accounts from receiving these floating reservations if you’d like.
Q4: Are AWS Reserved Instances better than On Demand?
Great question. This is actually the subject of the most popular post on the ParkMyCloud blog. In short: usually yes, but only for production environments. Otherwise, On Demand instances with on/off schedules are typically better value.
Reserved Instances can help save a lot of money on your AWS bill, but require a different way of thinking about your instances than the normal pay-as-you-go that you may be used to with cloud computing. These reservations are typically used for steady-state workloads with consistent usage. They can be complex to manage – be sure to check out our Reserved Instances optimizer to get the maximum value out of your reservations.
We’re happy to announce that ParkMyCloud now supports Alibaba Cloud!
Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) users have saved millions of dollars on their cloud bills using ParkMyCloud’s automated cloud cost optimization platform. Customers like McDonald’s, Sysco, and Unilever use ParkMyCloud to automatically turn off idle cloud resources as part of their DevOps process.
Now, Alibaba Cloud customers can do the same.
Alibaba Cloud is experiencing rapid customer adoption and growth – in the 4th quarter of last year, they saw over 100% growth, with more than 300 products and features launched. The company is clearly expanding their horizons beyond retail and putting a focus on innovation and development in the cloud space – both in China where their core customer base is located, and throughout the world as companies globally choose Alibaba as their primary cloud provider or as part of a multi-cloud strategy.
But the real reason we’re here is to help cloud users solve the enormous problem of cloud waste.
We estimate that Alibaba users will waste $552 million on idle cloud resources this year – that’s $1.5 million per day that could easily be saved with automated cost optimization in place. There’s no time to lose in getting cost control measures in place.
See it In Action
Get a preview of ParkMyCloud – watch this 2-minute demo to see how it works. To see a full demo and get your questions answered, schedule a personalized demo now.
Try Now for Free
You can get started with Alibaba Cloud cost control now with a free 14-day trial of ParkMyCloud, with full access to premium features.
After your trial expires, you can choose to continue using the free tier, or upgrade to use premium features such as SmartParking, full API access, advanced reporting and SSO.
Cheers, and happy parking.
Hitachi ID Systems recently reached their first ParkMyCloud birthday – to celebrate, we chatted with Patrick McMaster about how they optimize training infrastructure and why he and his team said “We honestly couldn’t be happier with ParkMyCloud.”
Can you start by telling me about Hitachi ID Systems and what you and your team do within the company?
Hitachi ID Systems makes identity and access management (IAM) software. I am the training coordinator, so I handle getting clients and potential partners up to speed with how our software works, how to install it, how to administrate it, etc. For those who are more interested in learning about software, we set them up with a virtual environment and course materials or an instructor to get them up to speed with how the software works.
Can you describe how you’re using public cloud?
We use AWS exclusively. When we advertise that we’re running a course a few months ahead of time, our infrastructure starts seeing the registrations and will start creating VMs, applying patches, getting the latest version of the appropriate software installers on the desktop and getting everything ready for the students, who will be accessing geographically-local AWS infrastructure.
In the past, everything for this online training was very manual. We on the training team would spin up the VMs manually, do the updates manually, and send the information to the potential students., Then when the course was over we would go through and do the reverse – shutting the elements down and turning off the virtual machine on AWS.
What does the supporting training infrastructure in AWS look like?
We have a number of VMs running per student or team that only need to be active during the team’s local business hours, plus some additional supporting infrastructure which is required 24/7. We started to realize as we got more students and began offering self-paced training that our AWS fees were increasing from the 24/7 access we were providing, but also just the management of keeping track of which students are where, when they should be brought into the system, when they should be shut down, etc. We needed to find a solution pretty quickly as we experienced that period of rapid growth.
How did that lead you to finding ParkMyCloud?
We knew we needed to automate the manual processes for this. Of course, lots of organizations tend to come up with solutions internally first. We’re a software company, so we had the talent for that, but we never have enough time. I’ve come to terms more and more every year with the benefits of delegating to the other sources. I realized that we are probably not the first organization to have this problem, so I Googled and found ParkMyCloud.
It became quickly evident that the features that you offered were exactly what we were looking for.
Can you describe your experience as a ParkMyCloud user so far?
Sure! So just before our demo of ParkMyCloud, we were fighting with this issue of trying to figure out how we can manage multiple time zones and multiple geographic locations, and not pay for that time that VMs are just spun up.
Then we went through the ParkMyCloud demo process and started our trial. We connected to our system and looked at ways to set up different schedules and pull information from AWS. There was definitely a moment where everyone in the room looked each other and said, “we must be missing something” – there had to be some additional steps we hadn’t thought out because it seemed too easy to work. But it really was that easy.
It just took a week of monitoring to make sure everything was turning off when it was supposed to – the bulk of our effort was really in that first week, and the time we need to spend in the interface is so small. We can go into ParkMyCloud’s dashboard and make exceptions to the schedule when needed, but the time that we actually spend thinking about these things is now about 2 hours a week, whereas before it was something that members of our staff might struggle with for 1-2 days. It’s been a huge improvement.
We did some calculations just in terms of uptime versus what we were doing before, and having the different schedules at our disposal and being able to spend that one week coming up with every scenario we could come up with was time well-spent. Now there are very few exceptions. I don’t think we’ve had to create a new schedule in a long time. Everything is organized logically, it’s very easy for us to find everything we need.
Who is responsible for tracking your AWS spending in the organization? Have they had any feedback?
Our finance department. Since we started using ParkMyCloud, it’s been very very quiet. No news is good news from finance. We are saving about 40% of our bill.
Do you have any other cloud cost savings measures in place?
Not for this training infrastructure. We have a pretty unique use case here. Our next steps are going to be more towards automatic termination, automatic spinning things up, more time-saving measures rather than cost.
This summer ParkMyCloud is working on instance rightsizing, if that’s something that would be helpful for you.
That’s definitely something that we could use. We are always trying to find better ways of doing things.
OK, great, thanks Patrick! Appreciate your time.
Thank you, have a good one!
We have been talking about idle cloud resources for several years now. Typically, we’re talking about instances purchased On Demand that you’re using for non-production purposes like development, testing, QA, staging, etc. These resources can be “parked” when they’re not being used, such as on nights and weekend, saving 65% or more per resource each month. What we haven’t talked much about is how the problem of idle cloud resources extends beyond just your typical virtual machine.
Why Idle Cloud Resources are a Problem
If you think about it, the problem is pretty straightforward: if a resource is idle, you’re paying your cloud provider for something you’re not actually using. This adds up.
Most non-production resources can be parked about 65% of the time, that is, parked 12 hours per day and all day on weekends (this is confirmed by looking at the resources parked in ParkMyCloud – they’re scheduled to be off just under 65% of the time.) We see that our customers are paying their cloud providers an average list price of $220 per month for their instances. If you’re currently paying $220 per month for an instance and leaving it running all the time, that means you’re wasting $143 per instance per month.
Maybe that doesn’t sound like much. But if that’s the case for 10 instances, you’re wasting $1,430 per month. One hundred instances? You’re up to a bill of $14,300 for time you’re not using. And that’s just a simple micro example. At a macro level that’s literally billions of dollars in wasted cloud spend.
4 Types of Idle Cloud Resources
So what kinds of resources are typically left idle, consuming your budget? Let’s dig into that, looking at the big three cloud providers — Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP).
- On Demand Instances/VMs – this is the core of the conversation, and what we’ve addressed above. On demand resources – and their associated scale groups – are frequently left running when they’re not being used, especially those used for non-production purposes.
- Relational Databases – there’s no doubt that databases are frequently left running when not needed as well, in similar circumstances to the On Demand resources. The problem is whether you can park them to cut back on wasted spend. AWS allows you to park certain types of its RDS service, however, you can not park like idle database services in Azure (SQL Database) or GCP (SQL). In this case, you should review your database infrastructure regularly and terminate anything unnecessary – or change to a smaller size if possible.
- Load Balancers – AWS Elastic Load Balancers (ELB) cannot be stopped (or parked), so to avoid getting billed for the time you need to remove it. The same can be said for Azure Load Balancer and GCP Load Balancers. Alerts can be set up in Cloudwatch/Azure Metrics/Google Stackdriver when you have a load balancer with no instances, so be sure to make use of those alerts.
- Containers – optimizing container use is a project of its own, but there’s no doubt that container services can be a source of waste. In fact, we are evaluating the ability for ParkMyCloud to park container services including ECS and EKS from AWS, ACS and AKS from Azure, and GKE from GCP, and the ability to prune and park the underlying hosts. In the meantime, you’ll want to regularly review the usage of your containers and the utilization of the infrastructure, especially in non-production environments.
Cloud waste is a billion-dollar problem facing businesses today. Make sure you’re turning off idle cloud resources in your environment, by parking those that can be stopped and eliminating those that can’t, to do your part in optimizing cloud spend.
We recently discussed how orphaned volumes and snapshots contribute to cloud waste and what you can do about it, but those are just two examples of orphaned cloud resources that result in unnecessary charges. The public cloud is a pay-as-you-go utility, requiring full visibility of specific infrastructure – you don’t want to be charged for resources you aren’t using. Here are other types of orphaned cloud resources that contribute to cloud waste (and cost you money):
Unassociated Elastic IPs
Elastic IPs are reserved public IP addresses designed for dynamic cloud computing in AWS. As a static IPv4 address associated with your AWS account, Elastic IPs can continue running an EC2 instance, even if it is stopped and restarted, by quickly remapping the address to another one of your instances. You can allocate an Elastic IP address to any EC2 instance in a given region, until you decide to release it.
The advantage of having an Elastic IP (EIP) is the ability to mask the failure of an EC2 instance, but if you do not associate the address to your account – you’re still getting charged. To avoid incurring a needless hourly charge from AWS, remember to release any unassociated IPs you no longer need.
Elastic Load Balancers (with no instances)
Cloud load balancing allows users to distribute workloads and traffic with the benefit of the cloud’s scalability. All major cloud providers offer some type of load balancing – AWS users can balance workloads and distribute traffic among EC2 instances with its Elastic Load Balancer, Google Cloud can distribute traffic between VM instances with Google Cloud Load Balancing, and Azure’s Load Balancer distributes traffic across multiple data centers.
An AWS Elastic Load Balancer (ELB) will incur charges to your bill as long as it’s configured with your account. Like with Elastic IPs, whether you’re using it or not – you’re paying. If you have no instances associated with your ELB, delete to avoid paying needless charges on your monthly bill.
Unused Machine Images (AMIs)
A Machine Image provides the information required to launch an instance, which is a virtual server in the cloud. In AWS they’re called AMIs, in Azure they’re Managed Images, and in Google Cloud Platform they’re Custom Images.
As part of your measures to reduce unnecessary costs from orphaned volumes, delete unused machine images when you no longer need them. Unless deleted, the snapshot that was created when the image was first created will continue to incur storage costs.
One of the growing pains that organizations face is the management of isolated pools of data in their cloud environment. Fragmented storage can result from data coming from a number of sources used by applications and business processes. Object Storage was designed to break down silos into scalable, cost-effective storage to store data in its native format. AWS offers object storage solutions like Amazon S3 and Amazon Glacier, Google has Google Cloud Storage, and Azure calls its solution Azure Blob Storage. All options help you manage your storage in one place, keeping your data organized and your business more cost effective.
Although object storage in and of itself is a cost effective solution, there are still ways to optimize and reduce costs within this solution. Delete files you no longer need so that you’re not paying for them. Delete unused files which can be recreated. In S3, use the “lifecycle” feature to delete or overwrite older versions of data. Clean up incomplete uploads that were interrupted, resulting in partial objects that are taking up needless space. And compress your data before storing to give you better performance and reduce your storage requirements.
How to Avoid Wasted Spend on Orphaned Cloud Resources
Don’t let forgotten resources waste money on your cloud bill. Put a stop to cloud waste by eliminating orphaned cloud resources and inactive storage, saving space, time, and money in the process. Remember to:
- Release unassociated IPs you no longer need.
- Remove Elastic Load Balancers with no instances attached.
- Delete unused machine images when you no longer need them.
- Keep object storage minimal – optimize by “cleaning up” regularly, removing files you don’t need.
The premise of the cloud and the resources available to you were meant to be cost effective, but it’s up to you keep costs in check. Optimize your cloud spend with visibility, management, and cost automation tools like ParkMyCloud to get the most out of your cloud environment.