Predictive Scaling for EC2 & More AWS Announcements to Be Thankful For

Predictive Scaling for EC2 & More AWS Announcements to Be Thankful For

Amazon Web Services (AWS) has been pumping out announcements in the lead up to their AWS re:Invent conference next week – which is predicted to exceed 50,000 attendees this year. (See you there?) We’re excited to see what big news the cloud giant has for us next week!

In the meantime, here are three AWS announcements from the last few days that will interest anyone who’s concerned with cloud costs.

Predictive Scaling for EC2

AWS’s new predictive scaling for EC2 is a new and improved way to use Auto Scaling to optimize costs. Typically when you set up an Auto Scaling Group, you need to set scaling policies, such as rules for launching instances based on changes in capacity. Given the complexity of these requirements, some users we’ve talked to forgo them altogether, instead using Auto Scaling simply for instance health checks and replacements.

With predictive scaling for EC2, there is very little the user needs to set up. You will simply set up the group, and machine learning models will analyze daily and weekly scaling patterns to predictively scale. You’ll have choices to optimize for availability, or optimize for cost – making it easy to use Auto Scaling to save money.Of course, sometimes you’ll know better than the machine – for example, development and test instances may require on/off or scale-up/scale-down schedules based on when users need them, which won’t always be consistent. For that, use ParkMyCloud to schedule auto scaling groups to turn off or change scaling when you know they will have little or no utilization.

AWS Cost Explorer Forecasting

AWS has announced an improved forecasting engine for the AWS Cost Explorer. It now breaks down historical data based on charge type – distinguishing between On Demand and Reserved Instance charges – and applies machine learning to predict future spend.

They have extended the prediction range from three months to twelve months, which will certainly be of use for budget forecasting. It’s also accessible via the API – we see this being used to show budget predictions on team dashboards in your office, among other applications.

CloudWatch Automatic Dashboards

The third announcement from this week that we’re looking forward to using ourselves here at ParkMyCloud is the new series of CloudWatch Automatic Dashboards. This will make it remarkably easier to navigate through your CloudWatch metrics and monitor costs and performance, and help potential issues break through the noise.

Thanks, AWS!

Now, play around with AWS’s new predictive scaling for EC2, then take some time to relax.

Happy Thanksgiving! (And to our non-U.S. readers, enjoy your Thursday!)

Interview: How Dealer-FX Saves Sysadmins’ Sanity with Automated AWS Management

Interview: How Dealer-FX Saves Sysadmins’ Sanity with Automated AWS Management

We chatted with Steve Scott, Cloud Infrastructure Manager at Dealer-FX about how they use ParkMyCloud’s automated AWS management to save significant amounts of time and sanity.

Tell us about what Dealer-FX does, and what your team does within the company. 

Dealer-FX provides software solutions to dealerships. Our software is used at the service advisor level – the people that you see when you take your car in. They’re usually behind a monitor that you never get to see and they’re typing away all things associated with your car information, VIN, scheduling information, recall information, etc. Our software controls all of that across many different OEMs, which are the manufacturers, and thousands of dealerships across Canada and the US.

I am the manager of cloud operations here and my team is strictly at the cloud management level, fully invested in AWS. We started using AWS through one of the OEMs we work with and that’s how we got into the cloud a few years ago.

Can you describe more about how you’re using AWS?

We use AWS for all of our testing, development, staging, and production environments. We use it all, from the API level to the functional level with virtual servers and virtual environments – everything we have that’s customer facing resides with AWS today.

Before you started using ParkMyCloud, what challenges did you face in your use of AWS?

One of the biggest things is that we use a lot of servers. When we had somewhere around 400 servers, we started to look into scheduling, both for server maintenance and for things that were only required to be online during certain periods of time. There was no inherent AWS service that was easily configurable for the same function that ParkMyCloud offered.

We’ve been using ParkMyCloud for a few years for automated AWS management to schedule resources on and off. Our code is in a period of transition from legacy to more cloud native, so we don’t have the resources to use some of the more cost-effective offerings from AWS like reserved instances, but we’re getting there. ParkMyCloud is certainly helping us, as we rely on it for scheduling server maintenance, staging, testing, and development environments.

How did you find ParkMyCloud?

I was bugging our AWS rep for some type of scheduling functionality. They could do it, but it would have taken a lot of work, and it was kind of iffy whether or not it would work for us. He directed me to ParkMyCloud.

Do you see yourselves using more cost efficient resources like Reserved Instances in the future?

I wouldn’t say that exactly. One thing we will look into is more autoscaling functionality. We do all of that manually, except ParkMyCloud sets up the scheduling and does that beautifully. We currently use ParkMyCloud scheduling because we have a predictable workload. For example, we might have 8 servers online between a certain number of hours, and after a period of time bring it down to 7, then 6, and so on depending on the environment, and then bring them back up again the next day.

In the future, as we build new apps, we’ll still be utilizing ParkMyCloud as we always have. We have RDS functionality on the horizon, which we know we can also schedule with ParkMyCloud’s automated AWS management.

We also use ParkMyCloud for planning on/off times for our staging environments which are on-demand. We haven’t taken advantage of all the features yet, but we use ParkMyCloud for very strategic reasons, in very strategic places, and it works phenomenally.

How would you describe the benefits that Dealer-FX has gotten from ParkMyCloud?

From the sysadmin perspective, the main reason we wanted ParkMyCloud was the sheer ease of turning servers on and off. Before, we needed to wake up at certain times and do it ourselves, manually turning off and on hundreds of servers. Having to do those things is no one’s cup of tea!

Who was responsible for doing that previously?

It was 2-3 people on my team.

It sounds like that took a lot of time.

It was a significant amount of time, and due to the high volume of deployments and growth over time, it become more and more terrible to administrate. ParkMyCloud is saving us time and sanity all over the place, and it just works. We’ve never had an issue with it. The design is ultimately “set it and forget it.”

Any other feedback? 

I know there’s lots of things on the horizon that we’ll be using as needed, and I’d be happy to receive updates of new features. Any new tools, extensions, or anything you add I would love to hear about.

We’ll be sharing rightsizing shortly, so look forward to that next! We appreciate your time and feedback.

Sounds great! Thanks!

Alibaba ECS Instance Types Comparison

Alibaba ECS Instance Types Comparison

Alibaba Cloud offers a number of ECS instance types optimized to meet various needs at the enterprise level. Instance types are specialized for different purposes, and vary by virtual CPU (vCPU), disk capability, memory size, and other features, offering a number of options to match any workload.

ECS instance types at the enterprise level are available for computing on the x86-architecture or for heterogenous computing, with generous options among both. But with so much to choose from, how does one find the right ECS instance type to meet their individual needs? We drew up a comparison for a quick guide on the different options and what they offer. The chart below and written descriptions are a brief and easy reference, but remember that finding the right instance type for your workload will always depend on your needs.

General Purpose

All general purpose ECS instance types are I/O optimized, offer a CPU to memory ratio of 1:4 and come with support for for SSD and Ultra Cloud Disks.

g5

The g5 instance type is backed by a 2.5 GHz Intel Xeon Platinum 8163 processor. With an ultra high packet forwarding rate, higher computing specs and higher network performance, this type ideal for scenarios involving transfer of a large volume of packets, running enterprise-level apps, small to medium database systems, data analysis, and computing clusters and data processing that rely on memory.

sn2e

Similar to the g5, the sn2e instance is ideal for the same scenarios and has higher computing specifications and enhanced network performance. Backed by a 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) or Platinum 8163 (Skylake) processors.

hfg5 (high clock speed)

The hfg5 is centered upon stable performance. Having many features in common with the g5 and sn2e options, this type family also offers a 56 vCPU instance type for boosted performance. With a 3.1 GHz Intel Xeon Gold 6149 (Skylake) processor, this instance type is better suited for high-performance front end servers, science and engineering apps, and Massively Multiplayer Online (MMO) games and video coding.

Compute Optimized

Compute optimized ECS instance types are all I/O optimized with support for SSD Cloud Disks and Ultra Cloud Disks, but vary in their vCPU to memory ratios and ideal scenario uses. Read on to learn the nuances.

Se1ne (enhanced network performance)

The se1ne comes with higher computing specs for enhanced network performance, a vCPU ratio of 1:8, ultra high packet transfer rate for receiving or transmitting, and 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) or Platinum 8163 (Skylake) processors. This type is ideal for large pack transfers, high-performance and large memory databases, data analysis, mining, distributed memory cache, and enterprise-level apps with large memory requirements (Hadoop, Spark).

hfc5 (high clock speed)

Optimized for stable performances, the hfc5 type boasts 3.1 GHz Intel Xeon Gold 6149 (Skylake) processors, vCPU to memory ratio of 1:2, higher computing specs and performance, and ideal for high-performance Web front-end servers, science and engineering applications, and MMO gaming and video coding.

gn6v, gn5, gn5i, gn4, ga1 (with GPU)

The g-series ECS instance type families come are all optimized for GPU compute workloads, with varying GPU processors and compute to memory options. All are ideal for deep learning, scientific computing, high-performance computing, rendering, multi-media coding, decoding, and other GPU compute workloads.

  • gnv6: V100 GPU processors, vCPU to memory ratio of 1:4, 2.5 GHz Intel Xeon Platinum 8163 (Skylake) processors.
  • gn5: NVIDIA P100 GPU processors, no fixed ratio of vCPU to memory, high-performance local NVMe SSD disks, and 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors.
  • gn51: NVIDIA P4 GPU processors, vCPU to memory ratio of 1:4, and 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors.
  • gn4: NVIDIA M40 GPU processors, no fixed CPU to memory, and 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors
  • ga1: AMD S7150 GPU processors, vCPU to memory ratio of 1:2.5, and 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors, and high-performance local NVMe SSD disks. Also ideal for other server-end business scenarios that require powerful concurrent floating-point compute capabilities

f1, f2, f3 (with FPGA)

The f-series ECS instance type families are ideal for deep learning, genomics research, financial analysis, picture, transcoding, and computational workloads, including real-time video processing and security.

  • f1: Intel ARRIA 10 GX 1150 FPGA, vCPU to memory ratio of 1:7.5, 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors
  • f2: Xilinx Kintex UltraScale XCKU115, vCPU to memory ratio of 1:7.5, and 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors
  • F3: Xilinx 16nm Virtex UltraScale + VU9P, vCPU to memory ratio = 1:4, and 2.5 GHz Intel Xeon Platinum 8163 (Skylake) processors

Memory Optimized

Memory optimized ECS instance types are optimized to meet for needs for high-performance and high memory databases and come I/O optimized with support for SSD and Ultra Cloud disks.

re4

The re4 is ideal for memory-intensive applications and Big Data processing engines (Apache spark or Presto), with 2.2 GHz Intel Xeon E7 8880 v4 (Broadwell) processors, up to 2.4 GHz Turbo Boot, vCPU to memory ratio of 1:12, up to 1920.0 GiB memory, and ecs.re4.20xlarge and ecs.re4.40xlarge have been certified by SAP HANA.

se1ne, se1

The s-series instance types are good for transfers of large volumes of packets, data analysis and mining, distributed memory cache, and Hadoop, Spark, and other enterprise-level applications with large memory requirements.

  • se1ne: vCPU to memory ratio of 1:8, high packet transfer rate, and 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) or Platinum 8163 (Skylake) processors.
  • se1: vCPU to memory ratio of 1:8, 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors

Big Data

d1ne, d1

The d-series ECS instance types are I/O optimized with support for SSD and Ultra Cloud disks come with high computing specs and network performance. They’re best use use for Hadoop MapReduce, HDFS, Hive, HBase, etc, Spark in-memory computing, big data computing and storage analysis (i.e. internet and finance industries), Elasticsearch, logs, and so on.

  • d1ne: High-volume local SATA HDD disks with high I/O throughput and up to 35 Gbit/s of bandwidth for a single instance, vCPU to memory ratio of 1:4, 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors
  • D1: High-volume local SATA HDD disks with high I/O throughput and up to 17 Gbit/s of bandwidth for a single instance, vCPU to memory ratio of 1:4, and 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors

Intensive Compute

ic5

The ic5 instance types stands on it own with a vCPU to memory ratio of 1:1. I/O optimized with support for SSD Cloud Disks and Ultra Cloud Disks, a high packet transfer rate, and 2.5 GHz Intel Xeon Platinum 8163 (Skylake) processors. Ideal for web front-end servers, data analysis, batch compute, and MMO game front-ends.

With Local SSD

i2, i2g, i1

The i-series type family all come with high-performance local SSD disks with high IOPs, high I/O throughput, and low latency. They’re ideal for OLTP and high-performance relational databases, NoSQL databases (Cassandra and MongoDB), and search applications, such as Elasticsearch

  • i2: vCPU to memory ratio of 1:8, designed for high-performance databases, 2.5 GHz Intel Xeon Platinum 8163 (Skylake) processors
  • i2g: vCPU to memory ratio of 1:4, designed for high-performance databases, 2.5 GHz Intel Xeon Platinum 8163 (Skylake) processors
  • i1: vCPU to memory ratio of 1:4, designed for big data scenarios, 2.5 GHz Intel Xeon E5-2682 v4 (Broadwell) processors

What Alibaba ECS instance types are right for your workloads?

With a wide range of ECS instance types belonging to multiple families, how do you determine the ECS type is a good match for your workload? With plenty to choose from, it’s highly probably than one of them will meet your needs effectively, but first you need to know what those needs are. Once you have a clear and ongoing vision of your workload, usage trends, and business needs, use the guide to start looking for the ECS type that’s right for you.

Further reading:

4 Mistakes Cloud Users Make When Purchasing Amazon Reserved Instances

4 Mistakes Cloud Users Make When Purchasing Amazon Reserved Instances

Amazon Reserved Instances are a great way to save money on AWS. Whether you’re looking to save on EC2, RDS, Elasticache, Elasticsearch, or Redshift, there are options to save 30-70% compared to on-demand costs. Many customers know about the opportunity and purchase Reserved Instances, but don’t have a solid execution plan to manage them going forward, resulting in wasted spend. Here are some common pitfalls we see.

Mistake #1: Thinking that once you purchase Amazon Reserved Instances, the work is done

Your journey is just beginning! Amazon Reserved Instances are only as effective as their match to your environment. Your usage will shift to difference resource types, and services may be right sized or be revamped. Dev environments will come and go as the team starts and finishes testing new features. It’s essential to continuously monitor your environment to ensure you’re eliminating any wasted spend, as well as identifying growth in usage that opens up opportunities to add additional Reserved Instances to save money.

Mistake #2: Thinking the story ends at EC2

While EC2 is the most common use case for Amazon Reserved Instances, there are 4 other services you need to monitor as well. RDS, Elasticache, Elasticsearch, and Redshift all offer Reserved Instances in one form or another. Users often overlook the savings opportunities Reserved Instances provide for these service. Additionally, did you know that not every server type has a Reserved Instance option for it? By focusing on server types for which you have reservations, and some other simple changes you can unlock additional savings.

Mistake #3: Ignoring AWS’s Pricing Changes

AWS changes prices and you need to make sure that you’re taking full advantage of them. Whether it’s converting Reserved Instances to capture the lower price or knowing what the best savings options are when it comes time to renew your Reserved Instances – you want to make sure you’re on top of all of AWS’s pricing changes when it comes to Reserved Instances.

Mistake #4: Assuming all upfront payments generate equal savings

Amazon Reserved Instances offer partial and Full upfront payment options, which have the potential to save you more – but are you choosing the best ones? Make sure you’re putting your money to work as efficiently as possible by running through various scenarios to identify which mix of reservations is best for you. This will also vary by the service you’re buying Reserved Instances for. In some places, it might be a no brainer to put a little money upfront as your savings are greatly increased. Other scenarios might lead to only a minimal increase in savings.

An Easy Solution to Optimize Amazon Reserved Instances

The key to saving money with AWS RIs is continuously monitor your Reserved Instances Fleet and test different scenarios to identify the best mix of reservations to save you the most amount of money.

Does all this optimization sound like a lot of work? That’s because it is. Eliminate all of this work by using StratCloud, which manages the buying, selling, modifying and monitoring of all your Reserved Instances. We got tired of spreadsheets and hundreds of individual decisions a month, so we created a Reserved Instance Optimizer. It uses big data analytics to analyze millions of data points optimize Reserved Instances for maximum savings. Learn more and sign up for your free demo today.

New: Schedule Google Cloud SQL Databases with ParkMyCloud

New: Schedule Google Cloud SQL Databases with ParkMyCloud

The latest release of ParkMyCloud includes the ability to schedule Google Cloud SQL Databases, among other updates to help you save more money through cloud automation.

Save with Google Cloud SQL Parking

First up, ParkMyCloud can now park Google Cloud SQL Databases! This means you can automate start/stop on a schedule, so your databases used for development, testing, and other non-production purposes are only running when you actually need them – and you only pay for the hours you need. The average schedule in ParkMyCloud is OFF 65% of the time, which means 65% savings – that’s a lot of money.

You can also use ParkMyCloud’s policy engine to create rules that automatically assign your SQL databases to parking schedules and to teams, so they’re only accessible to the users who need them.

Google Cloud SQL databases are just the latest in the growing list of types of cloud resources you can park, which also includes Google VM instances, Google Managed Instance groups, AWS EC2 instances, AWS auto scaling groups, AWS RDS instances, Azure VMs, Azure Scale Sets, and Alibaba Cloud ECS instances.

So why now? A growing number of ParkMyCloud users base their infrastructure in Google Cloud – in fact, GCP users are our fastest-growing segment of users. We’ll continue to add ways to optimize your environment no matter what clouds you use, of course, but expect more GCP features to come. We’ve focused on databases in this release because databases are the biggest area of cloud spend after compute, accounting for about 15-20% of an average enterprise’s bill.

What Else is New in ParkMyCloud?

Users will enjoy a few other recent additions to the ParkMyCloud platform:

  • Automatically accept SmartParking recommendations – fully automate your resource optimization by using ParkMyCloud’s policy engine to automatically apply schedules (previously, these had to be manually applied). There are several settings you can tweak to suit your needs – more in the release notes.
  • Chat integrations – we most recently added chat integration for Google Hangouts and MS Teams, joining our existing Slack integration. You can receive notifications and perform override commands and more through your chat window
  • Join the ParkMyCloud User Community on Slack – feel free to join even if you’re not yet a customer!

How to Get Started

If you’re new to ParkMyCloud, you can get started with a free trial. After the full-featured 14-day trial, you can choose to subscribe to a premium plan, or use the free tier – visit our pricing page for more information.

If you already use ParkMyCloud, you’ll need to enable ParkMyCloud to discover and manage your Google Cloud SQL databases. Find the details about the updated limited access role permissions in our user guide. Two things to note: first, you’ll need to be subscribed to the Standard or Enterprise tier in order to access this feature.

As always, we welcome your feedback about this new addition to ParkMyCloud, and any features you’d like to see in the future – comment below or shoot us a note. Cheers!

 

5 Free Google Cloud Training Resources

5 Free Google Cloud Training Resources

If you’re looking to break into the cloud computing space, there’s an abundance of resources out there, including free Google Cloud training. If you know where to look, open source learning is a great way to get familiar with different cloud service providers. Combined with our previous blog posts on free training resources for AWS and Azure, you’ll be well on your way to expanding your cloud expertise and finding your own niche. No matter where you are in the learning process, there’s a training resource for every experience level and learning type – get started now with our list of 5 free Google Cloud training resources:  

1. Google Cloud Free Tier

For free, hands-on training there’s no better place to start than with Google Cloud Platform itself. GCP’s free tier option is a no-brainer thanks to its offerings:

  • Access to all GCP products. You’ll have everything you need to experiment with building and running apps, sites, and services. Firebase and the Google Maps API are included.
  • $300 credit is yours to spend for the next 12 months, an expansion from their previous 60-day period and a sizable offer in comparison to Azure’s $200 for 30 days, so take advantage.
  • No autocharges after the trial period ends – a rarity for most free trials, and a guarantee that this training resource is 100% free.
  • An always-free option. GCP’s free tier takes the cake with this an always-free tier that gives you enough power to run a small app despite limitations on product and usage, a perfect option for learning purposes.

And for help with navigating the platform as you use it, check out GCP’s documentation for a full overview, comparisons, tutorials, and more.

2. Coursera

On the Google Cloud training page, you’ll find plenty of classes to get technical skills and learn best practices for using the platform. Among those options, they have also teamed up with Coursera, an online learning platform founded by Stanford professors, to offer courses online so you can “skill up from anywhere.”

Coursera includes a number of free courses, and until 1/1/19, you can sign up and get your first month free on any select Google Cloud Specialization. Courses include topics in Machine Learning, Architecting, Data Engineering, Developing Applications, and the list goes on.  

3. Qwiklabs

In conjunction with Coursera, Google Cloud offers hands-on training with specialized labs available via Qwiklabs, a learning lab environment for developers. Choose a “quest” from their catalog to get started with 50+ hands-on labs from beginner to expert level, where you’ll learn new skills in a GCP environment and earn cloud badges along the way. Get started with GCP Essentials and work your way into more advanced, niche topics like Managing Cloud Infrastructure with Terraform, Machine Learning APIs, IoT in Google Cloud, and so on.

4. YouTube

You can’t go wrong with YouTube. An endless amount of free videos offers an abundance of Google Cloud training for those of you who prefer to watch the movie instead of reading the book (you know who you are). Some of the most popular YouTube channels for free Google Cloud training include:

  • Google Cloud Platform (243k subscribers) – “helping you build what’s next with secure infrastructure, developer tools, APIs, data analytics and machine learning.”
  • Simplilearn (164k subscribers) – one of the world’s leading certification training providers, with online training that includes Machine Learning, AWS, DevOps, Big Data, and Google Cloud Platform, among others. The course on Introduction To Google Cloud Platform Fundamentals Certification is a popular one with upwards of 99k views.
  • Edureka (537k subscribers) is a full-service, online learning platform with curated content in Big Data and Hadoop, DevOps, Blockchain, AI, Data Science, AWS, Google Cloud, and more. Their YouTube channel is a “gateway to high-quality videos, webinars, sample classes and lectures from industry practitioners and influencers.” If you’re jumping into GCP with no prior knowledge or experience, the What is Google Cloud Platform tutorial will get you started.

5. Blogs & Forums

While other resources keep you learning with hands-on training, tutorials, and certification prep, blogs keep your mind flowing with new insights, ideas, and the latest on all things cloud computing. Google Cloud and Qwiklab have blogs of their own, perfect for supplemented reading with their trainings. But for a more well-rounded blog with content on other service providers, check out Cloud Academy. We also cover Google Cloud on the ParkMyCloud blog – check out this guide to Google Cloud machine types, an explanation of sustained use discounts, and introduction to resource-based pricing. And be sure to subscribe to relevant discussion forums such as r/googlecloud on Reddit and the GCP Slack.

Take Advantage of These Free Google Cloud Training Resources

As it becomes clear that cloud computing is here to stay, free training resources only continue to emerge. We picked the 5 above for their reliability, variety, quality, and range of information. Whether you’re new to Google Cloud or consider yourself an expert, these resources will expand your knowledge and keep you up to date with what’s latest in the platform.

More Free Training Resources: