How should CI/CD tool cost scaling, language support, and platform support affect your implementation decisions? In a previous post, we looked at the factors you should consider when choosing between a SaaS CI/CD tool vs. a self-hosted CI/CD solution. In this post, we will take a look at a number of other factors that should be considered when evaluating a SaaS CI/CD tool to determine if it’s the right fit for your organization, including cost scalability and language/platform support.
CI/CD Tool Cost Scaling
One thing that is important to keep in mind when deciding to use a paid subscription-based service is how the cost scales with your usage. There are a number of factors that can affect cost. Particularly, some CI/CD SaaS services limit the number of build processes that can be run concurrently. For example, Codeship’s free plan allows only one concurrent build at a time. Travis CI’s travis-ci.org product offers up to 5 concurrent builds for open source projects, but (interestingly) their $69 USD/mo plan on travis-ci.com only offers 1 concurrent build. All of this means that increased throughput will likely result in increased cost. If you expect to maintain a steady level of throughput (that is, you don’t expect to add significantly more developers, which would require additional CI/CD throughput) then perhaps limits on the number of concurrent build processes is not a concern for you. However, if you’re planning on adding more developers to your team, you’ll likely end up having more build/test jobs that need to be executed. Limits may hamper your team’s productivity.
Another restriction you may run across is a limit on the total number of “build minutes” for a given subscription. In other words, the cumulative number of minutes that all build/test processes can run during a given subscription billing cycle (typically a month) is capped at a certain amount. For example, CircleCI’s free plan is limited to 1,500 build minutes per month, while their paid plans offer unlimited build minutes. Adding more developers to your team will likely result in additional build jobs, which will increase the required amount of build minutes per month, which may affect your cost. Additionally, increasing the complexity of your build/test process may result in longer build/test times, which will further increase the number of build minutes you’ll need during each billing cycle. The takeaway here is that if you have a solid understanding of how your team and your build processes are likely to scale in the future, then you should be well equipped to make a decision on whether the cost of a build minute-limited plan will scale adequately to meet your organization’s needs.
Though not directly related to cost scaling, it’s important to note that some CI/CD SaaS providers place a limit on the length of time allowed for any single build/test job, independent of any cumulative per-billing-cycle limitations. For example, Travis CI’s travis-ci.org product limits build jobs to 50 minutes, while jobs on their travis-ci.com product are limited to 120 minutes per build. Similarly, Atlassian’s Bitbucket Pipelines limits builds to 2 hours per job. These limits are probably more than sufficient for most teams, but if you have any long-running build/test processes, you should make sure that your jobs will fit within the time constraints set by your CI/CD provider.
CI/CD Language and Platform Support
Not all languages and platforms are supported by all SaaS CI/CD providers. Support for programming languages, operating systems, containers, and third-party software installation are just a few of the factors that need to be considered when evaluating a SaaS CI/CD tool. If your team requires Microsoft Windows build servers, you are immediately limited to a very small set of options, of which AppVeyor is arguably the most popular. If you need to build and test iOS or Android apps, you have a few more options, such as Travis CI, fastlane, and Bitrise, among others.
Programming languages are another area of consideration. Most providers support the most popular languages, but if you’re using a less popular language, you’ll need to choose carefully. For instance, Travis CI supports a huge list of programming languages, but most other SaaS CI/CD providers support only a handful by comparison. If your project is written in D, Erlang, Rust, or some other less mainstream language, many SaaS CI/CD providers may be a no-go right from the start.
Further consideration is required when dealing with Docker containers. Some SaaS CI/CD providers offer first-class support for Docker containers, while other providers do not support them at all. If Docker is an integral part of your development and build process, some providers may be immediately disqualified from consideration due to this point alone.
As you can see, when it comes to determining the CI/CD tool that’s right for your team, there are numerous factors that should be considered, especially with regard to CI/CD tool cost. Fortunately, many SaaS CI/CD providers offer a free version of their service, which gives you the opportunity to test drive the service to ensure that it supports the languages, platforms, and services that your team uses. Just remember to keep cost scaling in mind before making your decision, as the cost of “changing horses” can be expensive should you find that your CI/CD tool cost scales disproportionately with the rest of your business.
In a future post, we will explore third-party integrations with CI/CD tools, with a focus on continuous delivery.
We talked with Kurt Brochu, Senior Manager of the Cloud Enablement Team at Sysco Foods, about how his company has been using ParkMyCloud to empower end users to keep costs in check with the implementation of their cloud-only strategy.
Thanks for taking the time to speak with us today. I know we chatted before at re:Invent, where you gave us some great feedback, and we’re excited to hear more about your use of ParkMyCloud since it rolled out to your other teams.
To get started, can you describe your role at Sysco and what you do?
I’m a senior manager here in charge of the cloud enablement team. The focus is on public cloud offerings, where we function as the support tier for the teams that consume those services. I also have ownership of ensuring that cost containment and appropriateness of use is being performed, as well as security and connectivity, network services, authentication, and DNS.
We don’t consider ourselves IT, our department is referred to as Business Technology. Our CTO brought us on 3 or 4 years ago with the expectation that we understand the business needs, wants, and desires, to actually service them as they would need versus passively telling them that their server is up or down.
As well as security and the dev team, teams using cloud also include areas that are customer facing, like sales, or internal, like finance, business reporting, asset management, and the list goes on.
Tell us about your company’s cloud usage.
We’ve had our own private cloud since 2003, offered on-prem. We’ve been in public cloud since 2013. Now, our position has gone from a “cloud-first” to a “cloud-only” strategy in the sense that any new workload that comes along is primarily put in public cloud. We primarily use AWS and are adding workloads to Azure as well.
Talk to me about how cost control fits into your cloud-only strategy. How did you realize there was a problem?
We were seeing around 20% month over month growth in expenditure between our two public clouds. Our budget wasn’t prepared for that type of growth.
We realized that some of the teams that had the ability to auto-generate workloads weren’t best managing their resources. There wasn’t an easy way to show the expenses in a visual manner to present them to Sysco, or to give them some means to manage the state of their workloads.
The teams were good at building other pipelines for bringing workloads online but they didn’t have day-to-day capabilities.
How did you discover ParkMyCloud as a solution to your cost control problem?
We first stumbled upon ParkMyCloud at the 2016 AWS re:Invent conference and were immediately intrigued but didn’t have the cycles to look into it until this past summer, when we made the switch from cloud-first to a cloud-only strategy.
We’ve been running ParkMyCloud since the week before re:Invent in 2017. From there, we had our first presentation to our leadership team in December 2017, where we showed that the uptick in savings was dramatic. It’s leveled off right now because we have a lot of new workloads coming in, but the savings are still noticeable. We still have developers who think that their dev system has to always be on and at will, but they don’t understand that now that we have ParkMyCloud, making it “at will” is as simple as an API call or the click of a button. I expect to see our savings to grow over the rest of the calendar year.
We have 50+ teams and over 500 users on ParkMyCloud now.
That’s great to hear! So how much are you saving on your cloud costs with ParkMyCloud?
Our lifetime savings thus far is $28,000, and the tool has paid for itself pretty quickly.
We have one team who has over 40% savings on their workloads. They were spending on average about $10,000 a month, and now it’s at $5,800 because they leverage ParkMyCloud’s simplified scheduling start/stop capabilities.
What other benefits are you getting from your use of the platform?
What I really like is that we have given most of our senior directors, who actually own the budgets, access to the tool as well. It lets the senior directors, as well as the executives when I present to them, see the actual cost savings. It gives you the ability to shine light in places that people don’t like to have the light shine.
The development team at ParkMyCloud has also been very open to receiving suggestions and capabilities that will help us improve savings and increase user adoption.
That’s great, and please continue to submit your feedback and requests to us! And in that regard, have you tried our SmartParking feature to get recommended schedules based on your usage?
Yes, we have started to. When I’m asked by a team to show them how we suggest they use the tool, they get to decide whether or not to enforce it. I’ll say that they are exceedingly happy by the fact that they can go and see their usage. One developer is telling their team that the feature has to be on at all times.
Are there any other cost savings measures that you use in conjunction with ParkMyCloud or in addition?
We pull numbers and look at Amazon’s best prices guide for sizing. We also take the recommendations from ParkMyCloud and we cross compare those.
Do you have any other feedback for us?
The magic of ParkMyCloud is that it empowers the end user to make decisions for the betterment of business, and gives us the needed visibility to do our jobs effectively. That’s the bottom line. Each user has a decision: I can spend money on wasted resources or I can save it where I can and apply the savings to other projects. Once you start to understand that, then you have that “AHA” moment.
Before using ParkMyCloud, most developers have no awareness of the expense of their workloads. This tool allows me to unfilter that data so they can see, for example: this workload is $293 a month, every month. If you look at your entire environment, you’re spending $17,000 a month, but if you take it down just for the weekend, you could be saving $2-3,000 a month or more depending on how aggressive you want to be, without hurting your ability to support the business. It’s that “AHA” moment that is satisfying to watch.
That’s what we noticed immediately when we looked at the summary reports – the uptick that appears right after you have these presentations with the team makes your heart feel good.
Well thank you Kurt, again we really appreciate you taking the time to speak with us.
NCAA, Google Cloud? What does the cloud have to do with March Madness? Actually, public cloud is increasingly being used and promoted in sports. When you watch the tournament on NCAA, Google Cloud ads will show prominently. Plus, the NCAA has chosen to run its infrastructure on Google Cloud Platform (GCP).
(By the way, have your done your bracket yet? I just did mine – I went chalk and picked Villanova. Couldn’t see my WVU Mountaineers winning it all).
So we will see and hear a lot of Google Cloud in the coming weeks. Google recently announced a multiyear sponsorship deal with the NCAA and will run these ads throughout the upcoming NCAA basketball tournament. Google is hoping to expand its cloud business by taking complex topics such as cloud computing, machine learning and artificial intelligence and making them relatable to a wider audience.
So why does is matter that NCAA and Google Cloud will appear so prominently together this March Madness?
First of all, Google Cloud is always matching wits with the other major cloud providers — and in this case, they’ve had their hooks in various mainstream sporting leagues and events for several years. For example, did you notice the partnership between AWS and the National Football League (NFL)? Both AWS and NFL promote machine-learning capabilities — software that helps recognize patterns and make predictions — to quickly analyze data captured during games. The data could provide new kinds of statistics for fans and insights that could help coaches.
Second, there’s the infrastructure that supports these huge events. I can tell you as a sports fan that me and my mates will all be live streaming football, basketball, golf and soccer (yes the English Premier League) on our phones and tablets wherever we are. We do this while watching the kids play sports, working in the office, and even while we are playing golf – hook it up to cart (a buggy for my UK mates). Many of these content providers are using AWS, Microsoft Azure, GCP, and IBM Cloud to get this content to us in real time, and to analyze it and provide valuable insights for a better user experience.
Or take a look at the Masters golf tournament. Usually IBM and ATT are big sponsors, although the Masters is usually very hush hush about a lot of this. Last year there was a lot of talk of IBM Watson, the Masters and the surreal experience they were able to deliver. This is a really good read on what went on behind the scenes and how Watson and IBM’s cloud delivered that experience. IBM used Machine learning, Visual recognition, Speech-to-text, and cognitive computing to build a phenomenal user experience for Masters viewers and visitors.
The NCAA and Google Cloud are not just ad partners, but the NCAA is also a GCP customer. The NCAA is migrating 80+ years of historical and play-by-play data, from 90 championships and 24 sports to GCP. To start, the NCAA will tap into decades of historical basketball data using BigQuery, Cloud Spanner, Datalab, Cloud Machine Learning and Cloud Dataflow, to power the analysis of team and player performance. So Google Cloud not only gets advertising prominence for one of the most-watched events of the year, it gets a high-profile customer and one of the coolest use cases out there.
Enjoy the tournament – let’s go Cats!
In this blog we are going to examine how Microsoft Azure region pricing varies and how region selection can help you reduce cloud spending.
How Organizations Select Public Cloud Regions
There are many comparisons that go into pricing differences between AWS vs Azure vs GCP, etc. At the end of the day, however, most organizations select one primary cloud service provider (CSP) for most of their workloads, plus maybe another for multi-cloud redundancy of critical services. Once selected, organizations then typically put many of their workloads in the region closest to their offices, plus maybe some geographic redundancy in their production systems. In other situations, a certain region is selected because that is the first region to support some new CSP feature. As time goes by, other regions become options because either those new features are propagated through the system, or whole new regions are created.
CSP regions tend to cluster around certain larger geographic regions, that I will call “areas” for the purpose of this blog. Looking at Azure in particular, we can see that Azure has three major US areas (Western, Central, and Eastern). The Western and Eastern US areas each have two Azure regions, and the Central area has four Azure regions. The UK, Europe and Australia areas each have two Azure regions. There are a number of other Azure regions as well, but they are far enough dispersed that I would consider them to be areas with a single region.
How Does Azure Region Pricing Vary?
With this regional distribution as a starting point, let’s look next at costs for instances. Here is a somewhat random selection of Azure region pricing data, looking at a variety of instance types (cost data as of approximately March 1, 2018).
While this graphic is a bit busy, there are a couple things that jump out at us:
- Within most of the areas, there are clearly more expensive regions and less expensive regions.
- The least expensive regions, on average across these instance types are us-west-2, us-west-central, and korea-south.
- The most expensive regions are asia-pacific-east, japan-east, and australia-east.
- Windows instances are about 1.5-3 times more expensive than their Linux-based counterparts
Let’s zoom-in on Azure Standard_DS2_v2 instance type, which comprises almost 60% of the total population of Azure instances customers are managing in the ParkMyCloud platform.
We can clearly see the relative volatility in the cost of this instance type across regions. And, while the Windows instance is about 1.5-2 times the cost of the Linux instance, the volatility is fairly closely mirrored across the regions.
Of more interest, however, is how the costs can differ within a given area. From that comparison we can see that there is some real savings to be gained by careful region selection within an area:
Over the course of a year, strategic region selection of a Windows DS2 instance could save up to $578 for the asia-pacific regions, $298 for the us-east regions, and $228 for the Korean regions.
How to Save Using Regions
By comparing regions within your desired “area” as illustrated above, the savings over a quantity of instances can be significant. Good region selection is fundamental to controlling Azure costs, and for costs across the other clouds as well.
Today, we’re excited to announce the release of SmartParkingTM for Microsoft Azure! SmartParking allows Azure customers to automate cloud cost optimization by creating parking schedules optimized to your actual cloud usage based on Azure Monitor data.
Here’s how it works: ParkMyCloud analyzes your Azure Monitor data to find patterns in the usage for each of your virtual machines (VMs). Based on those patterns, ParkMyCloud creates recommended on/off schedules for each VM to turn them off when they are idle. This maximizes your savings by ensuring that no VM is running when it’s not needed — while also saving you the time and frustration of trying to figure out when your colleagues need their resources running.
We released SmartParking for AWS in January, and customers have had positive feedback — and SmartParking for Google Cloud Platform is coming soon.
Customize Your Recommendations like your 401K
Is it better to park aggressively, maximizing savings, or to park conservatively, ensuring that no VM is parked when a user might need it? Everyone will have a different preference, which is why we’ve created different options for SmartParking recommendations. Like an investment portfolio, you can choose to receive SmartParking schedules that are “conservative”, “balanced”, or “aggressive”. And like an investment, a bigger risk comes with the opportunity for a bigger reward.
An aggressive SmartParking schedule prioritizes the maximum savings amount. You will park instances – and therefore save money – for the most time, with the “risk” of occasional inconvenience by having something turned off when someone needs it. Not to worry, though — users can always “snooze” these schedules to override them if they need to use the instance when it’s parked.
On the other hand, a conservative SmartParking schedule will make it more likely that your instances are never parked when they might be needed. It will only recommend parked times when the instance is never used. Choose “balanced” for a happy medium.
Customer Feedback: Making Parking Better Than Ever
ParkMyCloud customer Sysco Foods has more than 500 users across 50 teams using ParkMyCloud to manage their AWS environments. “When I’m asked by a team how they should use the tool, they’re exceedingly happy that they can go in and see when systems are idle,” Kurt Brochu, Sysco Foods’ Senior Manager of the Cloud Enablement Team, said of SmartParking. “To me, the magic is that the platform empowers the end user to make decisions for the betterment of the business.”
Already a ParkMyCloud user? Log in to your account to try out SmartParking for Azure. Note that you’ll have to update the permissions that ParkMyCloud has to access your Azure data — see the user guide for instructions on that.
Not yet a ParkMyCloud user? Start a free trial here.
Google Cloud Platform user? Not to worry — Google Cloud SmartParking is coming next month. Let us know if you’re interested and we’ll notify you when it’s released.
We talked with Bill Gullicksen, Director of IT at Qcentive, about how the company is using ParkMyCloud to save money on their AWS costs while enabling cloud computing in healthcare.
Thanks for taking the time to speak with us today. Can you start by telling me about Qcentive and how you are using the cloud?
We are a 2-year-old healthcare startup founded through the venture capital arm of Blue Cross Blue Shield of Massachusetts (BCBSMA). We build systems for the healthcare industry to help reduce costs in healthcare and provide efficiencies. Our cloud-based payment platform will facilitate the development and management of value-based contracts between healthcare companies. We are excited to be one of the earliest vendors authorized to take healthcare information and move it securely to the cloud.
What do you think made Qcentive stand apart as the best option for moving data to the cloud?
Healthcare has historically been cloud-averse due to issues like privacy and security concerns. In order to prove the use case for cloud computing in healthcare, we needed to build out a prototype and go through many months of meetings–producing artifacts to prove that we could move data to the cloud in a HIPAA/HITECH-compliant and secure manner.
We’ve recently released our first prototype module of the application by taking years of patient and healthcare contract information, loading it all into AWS, and then putting our application on top of it. Our application allows our health plan customers and their value-based contracting provider partners to analyze healthcare claim records, emergency room visits, etc. and to quickly calculate how to potentially realize savings in those areas.
So as you’re helping healthcare companies transition to the cloud, how did you come to find ParkMyCloud as a useful tool for your mission?
We had a few architects just going to town on AWS during the first year we were in business. They were building away, and all of a sudden our monthly AWS costs began to ramp up. We were spending a lot of money on Amazon and we didn’t even have a working application yet!
Last summer I was put in charge of our AWS operations and was asked to address our AWS costs. I asked, “what can we do to get some of these costs under control?” We started out with some rightsizing exercises and scaled some stuff back and that got us some savings. We found areas where we have had some stability and used Reserved Instances there, allowing us to get a 30-40% discount, but we didn’t want to do long-term commitments so we only did those for a year.
For the remaining instances, I realized we pay by the minute and we really don’t need to be running instances 24/7. That’s when we started thinking about how to schedule instances to shut down. I could do that and turn them off with AWS tools, but then telling an instance to turn itself back on at 6 in the morning–I didn’t have a way to do that. And that’s when I found out about ParkMyCloud and said this looks perfect – we can schedule instances to get them running 12 hours a day, 5 days a week instead of 24/7 and we’ll probably cut our costs in half.
Have you discovered any other benefits while using ParkMyCloud?
ParkMyCloud was the perfect tool for what we needed at the time and it also gave us a side benefit where we could give developers, QA people, and even data analysts and business folks the ability to turn an instance off when they’re done, or turn it on without having to write a bunch of complex policies within AWS.
Before, if we only wanted certain people to be able to manipulate a handful of instances, I had to put those instance IDs in the policies. Instance IDs frequently change, so running custom policies was taking a lot of overhead and we got the benefit from ParkMyCloud of just assigning them teams. Now, whether the instance IDs change or not, there’s no extra work for our IT team.
That’s why we chose ParkMyCloud and why we’ve been using it for 6-7 months now. For me it was great, very simple to set up, simple to use, easy for non-technical users and with very little effort from me and my technical staff, so it’s been perfect.
Great. So it seems like you were using a good mix of different cost savings efforts between the reserved instances, the rightsizing, and ParkMyCloud. Is there anything else you’re doing to manage cloud re-infrastructure costs?
Those are the bulk of it. We have other cloud-tracking subscriptions that we use sometimes. They are very simple but I just use it for looking at the daily spend, seeing if there’s any unexpected spikes, things like that. I can use it for finding resources that are no longer being used. It’s nice to have for identifying orphaned volumes and gives me a simple, easy way to clean some of that up, but we get our biggest use out of ParkMyCloud.
What percent of your resources are currently on ParkMyCloud schedules?
We’ve taken some schedules off just to keep some systems up for a while, but our rule of thumb has been to put a schedule and a team on everything. Even if a schedule is running 24/7/365, we want to at least have a schedule on it and know that it’s a conscious business decision we made to keep that up versus “it just slipped through the cracks and we never looked at it.”
About how many people in your team or organization are using ParkMyCloud?
Somewhere around 15 or so users.
Where do those users sit within your organization?
I’m Director of IT and we’ve got a Director of DevOps and a DevOps engineer–we are the three technical resources around infrastructure. Then we’ve got around 10 or so software developers that all have access so they can spin up their dev environments and spin them down when they’re not working.
We have a flexible schedule. Some of our software developers do their best coding at 3 in the morning. If they get up with an idea and they want to code, they need the ability to start up instances, do what they need to do, and then turn them off when they’re done. So they’re all in there, our QA department and some business analysts that do a lot of data analysis and database querying are also using ParkMyCloud.
That makes sense. So, how much are you saving on your AWS bills using ParkMyCloud?
Our initial savings with ParkMyCloud were significant and the product paid for itself quickly. Based on business needs, our costs can escalate rapidly so we estimate we’re saving up to 20% on our costs on a monthly basis.
We’ve got a lot of instances that we keep normally parked now and we only turn them on when there’s a workload to run. And then we’ve got probably another 40 or 50% of our instances that only run Monday through Friday, from 7:00 AM to 7:00 PM, so we’re getting that savings there which to me is bigger savings than dealing with Reserved Instances.
Things like Reserved Instances look great the day you buy them, but then the first time you have to change the size on something, all of the sudden you’ve got Reserved Instances that you’re not using anymore. With ParkMyCloud that never happens, it’s all savings.
How did you first hear about ParkMyCloud?
We were interviewing an external technology company, G2 Technologies in Boston last summer that was being brought in to augment our CI/CD process. While they were in we asked, “hey, do you know any good methods for doing scheduling?” – and they said take a look at ParkMyCloud.
Any other feedback for us?
I was surprised how simple ParkMyCloud was to get up and running. It was a couple of hours from signing up for the trial to having most of the work done and realizing savings, which was great. The release of your mobile app has been fantastic because it’s nice if we need to turn something on for somebody that doesn’t have access on a Saturday when I’m 30 miles away from my computer. I can do it anywhere with the mobile app.
Glad to hear it! I think that wraps things up for now. Thank you Bill, I appreciate your time.