How to Use Azure Resource Groups for Better VM Management

How to Use Azure Resource Groups for Better VM Management

When you create a virtual machine in Microsoft Azure, you are required to assign it to an Azure Resource Group. This grouping structure may seem like just another bit of administrivia, but savvy users will utilize this structure for better governance and cost management for their infrastructure.

What are Azure Resources Groups?

Azure Resources Groups are logical collections of virtual machines, storage accounts, virtual networks, web apps, databases, and/or database servers. Typically, users will group related resources for an application, divided into groups for production and non-production — but you can subdivide further as needed.

They are part of the Azure resource group management model, which provides four levels, or “scopes” of management to help you organize your resources.

  • Management groups: These groups are containers that help you manage access, policy, and compliance for multiple subscriptions. All subscriptions in a management group automatically inherit the conditions applied to the management group. These are often used for grouping subscriptions by internal department or geographical region.
  • Subscriptions: A subscription associates user accounts and the resources that were created by those user accounts. Each subscription has limits or quotas on the amount of resources you can create and use. Organizations can use subscriptions to manage costs and the resources that are created by users, teams, or projects. A subscription is essentially a billing unit.
  • Resource groups: A resource group is a logical container into which Azure resources like web apps, databases, and storage accounts are deployed and managed.
  • Resources: Resources are instances of services that you create, like virtual machines, storage, or SQL databases.

One important factor to keep in mind when managing these scopes is that there is a difference between azure subscription vs management group. A management group cannot include an Azure Resource. It can only include other management groups or subscriptions. Azure Management Groups provide a level of organization above Azure Subscriptions. Also, there is no structure for a “nested” resource group in Azure – to “nest” groups for permissions, you will need to use a combination of permissions at the different levels listed above.

Be sure also to differentiate the concept of an Azure resource group from an “Azure availability set”. An availability set in Azure is a logical grouping of VMs to inform Azure how your application is built in order to protect the availability of your application.

Group structures like Azure’s exist at the other big public clouds — AWS, for example, offers optional Resource Groups, and Google Cloud “projects” define a level of grouping that falls someplace between Azure subscriptions and Azure Resource Groups.

The Azure Resource Manager

azure resource group consistent management layer diagram

You will manage resource groups through the “Azure Resource Manager”, which is the management layer for your resources. Benefits of the Azure Resource Manager include the ability to manage your infrastructure through declarative templates rather than through scripts; tagging management; deployment templates; dependency mapping; simplified role-based access control; and clarified cost management.

You can organize your resource groups for securing, managing, and tracking the costs related to your workflows. 

Ways to Create an Azure Resource Group

There are several ways to create an Azure Resource Group, or in other words, several ways to actually manage using the Azure Resource Manager. They are:

  • The Azure Portal
  • Azure PowerShell scripts
  • The Azure CLI
  • An ARM template

Azure Resource Group Best Practices

When organizing your resource groups, it is essential to understand that all the resources in a group should have the same life-cycle when including them. For instance, if an application requires different resources that need to be updated together, such as having a SQL database, a web app or a mobile app, then it makes sense to group these resources in the same resource group. However, for dev/test, staging, or production, it is important to use different resource groups as the resources in these groups have different lifecycles. 

Other things to consider when building your Azure list of resource groups: 

  • Resources can be added to or deleted from an Azure Resource Group. However, each of your resources should belong to an Azure Resource Group, so if you remove the resources from one Resource Group, you should add it to another one.
  • Keep in mind, not all resources can be moved to different resource groups.
  • Azure resource group regions: the resources you include in a resource group can be located in different Azure regions, and may be in different regions than the group itself. The group needs a location to specify where the metadata will be stored, which is necessary for some compliance policies. (Read more on pricing in different Azure regions.)
  • Grant access with resource groups:  you should use resource groups to control access to your resources – more on this below.
  • When a resource group is deleted, all resources in the group are deleted 
  • Group limits: you can deploy up to 800 instances of a resource type in each resource group – with some exceptions.

How to Use Azure Resource Groups Effectively for Governance

Azure resource groups are a handy tool for role-based access control (RBAC). Typically, you will want to grant user access at the resource group level – groups make this simpler to manage and provide greater visibility.

Azure resource group permissions help you follow the principle of least privilege. Users, processes, applications, and devices can be provided with the minimum permissions needed at the resource group level, rather than at the management group or subscription levels. For example, a policy relating to encryption key management can be applied at the management group level, while a start/stop scheduling policy might be applied at the resource group level.

Effective use of tagging allows you to identify resources for technical, automation, billing, and security purposes. Tags can extend beyond resource groups, which allows you to use tags to associate groups and resources that belong to the same project, application, or service. Be sure to apply tagging best practices, such as requiring a standard set of tags to be applied before a resource is deployed, to ensure you’re optimizing your resources.

Azure Resources Groups Simplify Cost Management

Azure Resource Groups also provide a ready-made structure for cost allocation  — resource groups make it simpler to identify costs at a project level than just relying on Azure subscriptions. You can use cost allocation tags to manage the costs of the resources within the group together. Additionally, you can use groups to manage resource scheduling and, when they’re no longer needed, termination. Make sure you don’t skip that step – keep waste in check.

You can do this manually, or through your cost optimization platform such as ParkMyCloud. Continuous cost control comes from actual action – which is what ParkMyCloud provides you through a simple UI (with full RBAC), smart recommendations with one-click remediation, and an automatic policy engine that can schedule your resources by default based on your tagging or naming conventions. For almost all Azure users, this means automatic assignment to teams, so you can provide governed user access to ParkMyCloud. It also means you can set on/off schedules at the group level, to turn your non-production groups off when they’re not needed to help you reduce cloud waste and maximize the value of your cloud. Start a trial today to see the automation in action.

A version of this article was published in April 2020. It has been updated and expanded for 2021.

Cloud Elasticity Was the Goal. Are You Achieving It?

Cloud Elasticity Was the Goal. Are You Achieving It?

There are three advantages of using public cloud: cloud elasticity, agility, and cost reduction. However – most public cloud users are not (yet!) achieving all three of these. While you may be aware if your costs are out of control, what you may not consider is how directly that’s tied to a misuse of the idea of “elasticity”. 

The True Meaning of Cloud Elasticity

There’s a reason “elastic” is in the name of EC2 and EBS. Users can provision what they want, when they want it, to easily scale up to meet demand. This ability is what has fueled the explosion of agility and enabled experimentation and innovation. 

But the other ingredient in elasticity is shrinking or turning off resources dynamically to meet demand, which is not nearly as easily achieved… and is often neglected. The problem with only being “half” elastic is fundamental to the purpose of the cloud. 

When you let resources in one direction only – up – in the name of “elasticity”, you neglect the promise of the cloud: only pay for what you need. You’ll be slapped in the face with rising costs. And yes – we’ve seen this happen time and time again. Organizations find that those easily-provisioned resources stick around, clogging their environments with waste.

Knowledge is Power, but Not Always Action

A recent Turbonomic survey found that optimizing existing cloud resources for performance and cost was the #1 goal for cloud users this year. The fact that this came in at #1 shows us that there’s certainly awareness among cloud customers that this is a problem they need to solve.

This is not news to you. You’ve perhaps seen the bill. You may have dashboards slicing and dicing your cloud bill. But too often, those dashboards are simply something to pull up for pretty data (or ugly, as the case may be) to share in a team meeting. The knowledge doesn’t 

We get it. We’re all busy. Our priorities in the workplace are always shifting, and honestly, it can be hard to make ✨achieving the promise of cloud elasticity✨ one of them. It’s definitely hard to make “reducing costs” one of them. 

But here’s the thing: knowledge – like the visibility you gain from cost dashboards – is power.

In our experience, most people do not act on that power. 

When Elasticity Goes Two Ways, You Can Optimize

What would it look like to take that knowledge and turn it into action?

You would truly only use the resources you need, when you need them. You would take advantage of public cloud’s usage-based pricing model, and achieve a cost-optimized environment.

And ideally, this would be automated – so the action just happens and you can spend your time on those more exciting benefits of cloud infrastructure: growing and innovating. When you eliminate wasted spend from your cloud environment, you open up your budget to achieve the business goals that really matter.

Here at ParkMyCloud, we help you with ensuring elasticity with easy-to-implement actions based on your utilization data. By “parking” non-production resources when not needed, such as nights and weekends, you can reduce the cost of those resources by 65% – an easy win for both elasticity and cost optimization.

Cloud Computing Trends 2021: Optimization is #1 Priority

Cloud Computing Trends 2021: Optimization is #1 Priority

Topping the list of cloud computing trends 2021 is…drumroll please…cloud optimization! 

This is according to data collected by the annual State of Multicloud Survey run by Turbonomic (that’s our parent brand). The survey, using data collected from more than 800 global IT professionals, examines the state of multicloud adoption, its drivers, and the technologies enabling it, including public cloud, containers, and edge computing. Here are a few highlights from the report.

Optimization Remains Top Priority for Public Cloud

As mentioned above, when asked their most important initiative for 2021, survey respondents ranked “optimize existing cloud resources for performance and cost” as their highest priority. We see this need playing out every day as we talk to enterprises, both newly migrated and with years of experience in the cloud. Those who are paying attention are realizing their infrastructure is overprovisioned, running when not needed, or otherwise wasted – to the tune of $26.6 billion wasted this year in all. By making optimization a priority, tech leaders can reallocate wasted spend and achieve more with their budgets.

Advancing a Multicloud Strategy Rated #2

After optimization, the next-highest ranked initiative was “advancing a multicloud strategy”(multiple public clouds with or without private clouds in the mix). Multicloud existence is the reality for most organizations – and 30% of survey respondents are using three or more public clouds today. Research by IDG finds that the most common reported reason for using multiple public clouds is to get the best platform and service options available.

Relevance of PaaS is Growing

Of organizations surveyed, 62% reported that public cloud PaaS will play a strategic role for their business within 18 months. This is supported by research from Gartner, which estimates the PaaS market will grow by 26% this year, driven at least in part by the need for remote workers to have access to scalable infrastructure in the form of modernized and cloud-native applications. 

Container Use is Advancing – But Difficult to Scale

For 61% of organizations surveyed, containerization will play a strategic role within 18 months. It is already strategic for nearly 20% today. Of those already implementing containers, 56% are using them in production. For the remainder, operations are difficult to scale due to the complexity introduced and the difficulty to optimize. 

The Full Report on Cloud Computing Trends 2021

Check out the report for the full data for each of these highlights, as well as results about AWS vs. Azure use, container deployment trends, edge computing, and how leaders compare to laggards in cloud services and strategy adoption. Read the 2021 State of Multicloud Report now.

If optimization is a priority for your cloud organization, check out the free trial of ParkMyCloud to start reducing costs through parking and rightsizing.

AWS vs Azure vs Google Cloud Market Share 2021: What the Latest Data Shows

AWS vs Azure vs Google Cloud Market Share 2021: What the Latest Data Shows

Today, we’ll take a look at the latest AWS vs Azure vs Google Cloud market share comparison, including the Q4 2020 earnings the ‘big three’ cloud providers have reported. Let’s take a look at all three providers side-by-side to see where they stand.

Note: several previous versions of this article have been published. It has been updated for February 2021.

AWS vs. Azure vs. Google Cloud Earnings

To level-set this comparison, first know that – unsurprisingly – the cloud market as a whole is bigger than ever. Gartner has predicted worldwide public cloud spend to grow 18% in 2021, with 70% of organizations using cloud to increase cloud spending in the wake of COVID-19. 

So within that market, let’s take a look at the AWS vs Azure vs Google Cloud market share breakdown and what each cloud provider’s reports shared.

AWS 

First, the big news, of course: Jeff Bezos is trading his CEO role for Executive Chair of the Amazon Board, while current CEO of AWS Andy Jassy will step up to the Amazon CEO role. No AWS CEO has yet been announced, but many bets are on Matt Garman, currently the Vice President of AWS Sales and Marketing, or else Peter DeSantis, AWS’s Vice President of Global Infrastructure.

Next, the bigger news: Amazon revenue. AWS 6 year financials

Amazon reported Amazon Web Services (AWS) revenue of $12.7 billion for Q4 2020, compared to $9.95 billion for Q4 2019. AWS revenue grew 28% in the quarter. 

Amazon as a whole had their first quarter over the $100 billion mark, at $125.56 billion. That’s an increase of 44% year-over-year, and beating predictions of $119.7 billion. Earnings per share were $14.09, compared to a $7.23 forecast.

Amazon as a whole benefitted from an astronomical online holiday shopping season due to COVID-19, and also from Prime Day being held in the fourth quarter.  And AWS? It made up 10% of Amazon’s sales for the quarter – and 52% of its operating income.AWS only continues to grow, and bolster the retail giant time after time.

One thing to keep in mind: you’ll see a couple of headlines pointing out that revenue growth is down and/or highlighting the fact that it’s flattening out, quoting that 28% number and comparing it to previous quarters’ growth rates, which peaked at 81% in 2015. However, that metric is of questionable value as AWS continues to increase revenue at this enormous scale (see Geekwire graph), and dominate the market (as we’ll see below). AWS added more revenue quarter-over-quarter and year-over-year than any quarter in its history. Dave Fildes, Director of Investor Relations, mentioned on the call that “If you account for this COVID anomaly this year of [AWS re:Invent] being virtual and free, AWS year-over-year revenue growth, if you look at it, actually accelerated adjusting for that from the third quarter to the fourth quarter,” an interesting tidbit both from the perspective of gaining a glimpse into what re:Invent actually does for the company, and that AWS revenue is accelerating. 

Azure

While Amazon specifies AWS revenue, Microsoft only reports on Azure’s growth rate. That number is 50% revenue growth over the previous quarter. This time last year, growth was reported at 62%. As mentioned above, comparing growth rates to growth rates is interesting, but not necessarily as useful a metric as actual revenue numbers – which we don’t have for Azure alone.

Here are the revenue numbers Microsoft does report. Azure is under the “Intelligent Cloud” business, which grew 23% to $14.6 billion. The operating group also includes server products and cloud services (26% growth). 

The lack of specificity around Azure frustrates many pundits as it simply can’t be compared directly to AWS, and inevitably raises eyebrows about how Azure is really doing. Of course, it also assumes that IaaS is the only piece of “cloud” that’s important, but then, that’s how AWS has grown to dominate the market. 

Nonetheless, Microsoft’s cloud business is clearly generating success for the company. Intelligent Cloud delivered the highest operating income of all segments this quarter at $6.4 billion, which is 36% of total consolidated operating income. 

Google Cloud

In more exciting news for public cloud followers, Alphabet has broken out Google Cloud revenue for the first time. Thus we learned that while Google Cloud revenue has increased over the last three years, so too have their operating losses. CFO Ruth Porat notes that these operating losses “reflect that we have meaningfully built out our organization, ahead of revenue.”

This quarter, Google Cloud reported revenue of $3.83 billion, an increase of 47% year-over-year. Operating losses were $1.24 billion compared to losses of $1.19 billion one year previously. For the full fiscal year 2020, Google Cloud’s revenue was $13 billion, with $5.6 billion operating losses.

Note that the Google Cloud unit includes not only Google Cloud Platform but also Google Workspace (formerly G Suite). 

One highlight was that deals over $250 million tripled during 2020, and several billion-dollar deals were closed during the year. 

Alibaba Cloud

We’ll add Alibaba Cloud to this list for the first time as the cloud computing division is profitable as of this quarter. The cloud computing arm of the Chinese retail giant earned $2.47 billion this quarter, an increase of 50% year-over-year. 

Cloud Computing Market Share Breakdown – AWS vs. Azure vs. Google Cloud

When we originally published this blog in 2018, we included a market share breakdown from analyst Canalys, which reported AWS in the lead owning about a third of the market, Microsoft in second with about 15 percent, and Google sitting around 5 percent.

In 2019, they reported an overall growth in the cloud infrastructure market of 42%. By provider, AWS had the biggest sales gain with a $2.3 billion YOY increase, but Canalys reported Azure and Google Cloud with bigger percentage increases.

As of February 2021, Canalys reports that the worldwide cloud market grew 32% this quarter to $39.9 billion. For the full year of 2020, cloud infrastructure spending grew 33% to $142 billion. AWS has 31% of the market, followed by Azure at 20%, Google at 7%, Alibaba Cloud close behind. 

Bezos has said, “AWS had the unusual advantage of a seven-year head start before facing like-minded competition. As a result, the AWS services are by far the most evolved and most functionality-rich.”

Our anecdotal experience talking to cloud customers often finds that true. It seems clear that in the case of AWS vs Azure vs Google Cloud market share – AWS still has a substantial lead, and their market share remains steady. With that said, all players are pushing growth and innovation and driving public cloud adoption across the board.

Overprovisioning & Always-On Resources Lead to $26.6 Billion in Public Cloud Waste Expected in 2021

Overprovisioning & Always-On Resources Lead to $26.6 Billion in Public Cloud Waste Expected in 2021

Overprovisioning and leaving cloud resources on are two enormous sources of wasted spend. 

Wasted spend drags down IT budgets – of particular importance as we enter 2021. The Flexera 2021 State of Tech Spend report found that the biggest change in key IT initiatives from 2020 to 2021 was in cost savings, with the percent of respondents ranking cost savings as a top initiative tripling year-over-year. 

It’s important that this is being recognized. Based on data collected by Gartner, we estimate that wasted spend will exceed $26.6 billion this year. 

Where the Wasted Cloud Spend is Coming From

Gartner estimates a total market spend of $304 billion on public cloud services end-user spending in 2021, as broken out in the table below. Their estimate for the proportion of that spent on Infrastructure as a Service (IaaS) is $65.3 billion. While wasted spend can be found in any area of cloud spend, customers tend to see the largest amount in these two areas, as well as finding it easiest to identify.

Idle Resources

Cloud resources can be considered “idle” when they are running while not being used. For example, when development servers are left running overnight and on weekends when they’re not needed. Since compute resources are paid for by the minute or second, that’s a large portion of the week they’re being paid for but not used (and yes, this applies even if you have reservations.)

Our data shows that about 44% of compute spend is on non-production resources. If we estimate that non-production resources are only needed during a 40-hour work week, the other 128 hours (76%), the resources are sitting idle. 

Applying that to the Gartner IaaS number, we estimate that up to $14.5 billion will be wasted on idle resources this year. 

Overprovisioned Resources

Overprovisioning occurs when a larger resource size is selected than is actually needed. There is a mindset of safety behind this, as of course, no one wants their applications to be under-resourced. 

But the overprovisioning occurring is far beyond what is necessary, given the elasticity of the cloud. About 40% of instances are sized at least one size larger than needed for their workloads. The cost can be cut in half by reducing an instance by one size, while downsizing by two sizes saves 75%.

Many of our customers show a large percentage of their resources are oversized, but bringing this to a conservative estimate of 40% of resources oversized by one size, giving us a savings per resource of 50%, we estimate that up to $8.7 billion is wasted due to overprovisioning. 

Orphaned Volumes and Snapshots

Another significant source of waste is orphaned volumes and snapshots. These are resources that have been detached from the infrastructure they were created to support, such as a volume detached from an instance or a snapshot with no volume attachment.

Our customers spend approximately 15% of their bills on storage, and we found that about 35% of that spend is on unattached volumes and snapshots. Applying that to the Gartner spending numbers, we estimate that up to $3.4 billion could be wasted this year on orphaned volumes and snapshots. 

Reducing Wasted Spend

Altogether, this gives us an estimate of $26.6 billion to be wasted on unused cloud resources in 2021. This waste estimate is just based on the three prominent sources of cloud waste. It does not include wasted spend on Platform as a Service (PaaS), which makes up $55 billion in cloud spend according to Gartner’s estimates, nor from SaaS, unused reservation commitments, inefficient containerization, and other areas of the bill. 

Attacking the three problem areas above is a great area to start for nearly all public cloud users. Here at ParkMyCloud, we’re on a mission to do just that. See how and try it out today, to do your part in reducing wasted cloud spend.