Every week, we find ourselves having a conversation about cost optimization with a wide variety of enterprises. In larger companies, we often talk to folks in the business unit that most people traditionally refer to as Information Technology (IT). These meetings usually include discussions about the centralization vs decentralization of IT and oftentimes they don’t realize it, as we are discussing cloud and how it’s built, run and managed in the organization.
Enterprises traditionally organized their IT team as a single department under the leadership of the CIO. The IT team works across organizational departments and supports the enterprise to meet various tooling and project needs requested by other business units or the executive team. Although there are significant efficiencies from this type of approach, there are some risks that can affect the entire organization, in particular, one that seems to stem from the ‘need for speed’ (agility). The LOB depends on IT to deliver services, hardware, software, and other ‘tools’, but this is not always done quickly and efficiently, mostly due to internal processes.
Benefits of Centralized IT Structures
The benefits of this type of organizational structure were often associated with increased purchasing power, improved information flow between IT team members, skilled hiring efficiencies, and a watchful view of the enterprise’s technical infrastructure from both an operational network and security perspective. Let’s dig into these in a bit more detail.
Lowered expenses and increased purchasing power – the centralized environment will always provide a business with more buying power at a lower cost by combining all of the needs of the business into a centralized buying pool.
Improved productivity for IT staff – IT teams are like any other team, they thrive with collaboration and mutual understanding and respect for each other’s skillsets. It also makes installations and technical resolution(s) easier as you’re addressing a centralized resource.
Enterprise-wide information dissemination – the centralized organization will build its network from the center out – LOBs will typically share the same networked resources – such as an ERP or CRM. This avoids the dangers of siloed information that could be critical to another LOB, but without access, there’s no visibility into the information that is available.
Despite the benefits stated above, a centralized team has several limitations and challenges – one of those challenges with the greatest enterprise-wide exposure is how best to prioritize project requests from each of the LOBs – enter decentralization and cloud — IaaS, PaaS and SaaS.
Decentralization is a type of organizational structure in which daily operations and decision-making responsibilities are delegated by top management to middle and lower-level managers and their respective business units. This frees up top management to focus more on major decisions. For a small business, growth may create the need to decentralize to continue efficient operations. Decentralization offers several advantages and is a practical approach when different departments or business units in a company have different IT needs and strategies.
Benefits of Decentralized IT Structures
The ability to tailor IT selection and configuration. When individual departments have IT decision-making power, they can choose and configure IT resources based on their own specific needs. For example, each department has its own servers optimized to run its required applications.
More fail-safes and organizational redundancy. Decentralizing makes servers and applications more resilient—and it can do the same for IT networks, too. If each department maintains its own server, one can function as a backup server in case another server fails. (Of course, this type of redundancy would need to be properly configured in advance.)
Respond faster to new IT trends. Since departments in decentralized organizations can make independent decisions, it’s easier for them to take advantage of new technology in the cloud.
One drawback of decentralized IT structures is that this model often leads to information silos – collections of data and information that cannot be easily shared across departments. Centralized IT structures help prevent these silos, leading to better knowledge-sharing and cooperation between departments. For example, using one centrally managed CRM system makes it possible for any employee in a company to access customer information from anywhere — think SalesForce.
The Reality is Hybrid IT
As we see above and in real life, there are many reasons an organization might be tempted to move toward or away from a centralized IT organizational structure but in practice many companies practice a hybrid model – some IT systems like your CRM and ChatOps are centralized, while others like your Cloud Provider and Orchestration tool may be decentralized (buy business unit). The top reasons for this hybrid model that come to mind are technical agility and the availability of tools through SaaS, IaaS and PaaS providers – IT no longer needs to build every solution and tool for you. And decentralized IT organizational structures are typically best for companies that rely on technical agility to remain competitive. These include newer, smaller companies (e.g., startups), and organizations that need to respond quickly to new IT developments (e.g., software and hardware companies or app developers). And, for larger companies that want to bring that mentality and model to their business, here is a great example, Capital One, a bank wanting to be a technology company.
What are your thoughts on the centralization vs decentralization of IT?
“How do I stop wasting money on Reserved Instances?”
It’s a question we’ve heard before from despairing AWS users. They were told Reserved Instances (RIs) would save them money, so they purchased them. Now, halfway into a three-year contract, they realize they’re not utilizing the RIs they’re paying for. Or worse… they may not even know what RIs they have.
Amazon offers Reserved Instances to ostensibly help get your cloud costs in control. The message is that RIs help you save money on your EC2 instances by offering discounted hourly rates in exchange for a 1- or 3-year commitment. Before we get into how you can cut your cloud spending with an AWS RI, here’s a bit of background and what you need to know about AWS EC2 Reserved Instance pricing.
How do EC2 Reserved Instance Purchasing Options Work?
When it comes to Reserved Instances purchasing options, you can either choose a 1- or 3-year contract. The longer the commitment, the greater the cost savings compared to On-Demand. By choosing one of these contracts, customers are promised savings of up to 75%.
There are a few risks that come with the longer commitment times. For starters, if AWS drops pricing, then the promised savings are reduced or may disappear. And when AWS introduces a new generation of an instance type family it may attract your users away from your contracts – these are based on the older generation. If you don’t know your future needs, it may be appealing to use the 1-year instead of a 3-year contract, which has savings vs. On Demand at about 31-40%.
There are three different types of EC2 Reserved Instances that customers can purchase – Standard Reserved Instances, Convertible Reserved Instances, or Scheduled Reserved Instances. With Standard Reserved Instances, customers would see the most significant savings. However, Convertible Reserved Instances are attractive to customers because it gives them added flexibility like the ability to use different instance families, operating systems, or tenancies over the term. Scheduled RIs allow you buy an RI that is only used at certain times each day in a recurring schedule.
When an RI expires, you are charged again at the normal rate. See the recently released option to queue RI purchases in advance. This may help provide the greatest savings by eliminating gaps in your coverage from reservations.
Additional Ways To Save
AWS also offers additional discounts if you have more than $500,000 worth of Reserved Instances in a region – the more Reserved Instances you have, the larger your discount.
You may also buy RIs on the Reserved Instance Marketplace from third-party sellers. The great thing about this is that these third parties tend to list their RIs at lower prices for a shorter period of time. And if you find you have too many RIs, you can sell them on the Marketplace as well.
There are three different payment plans offered with Reserved Instances. Payments can be made either All Upfront, Partial Upfront, or No Upfront. It is important to note that if you pay all up front, you will have greater savings because there are no other costs or additional charges during the term regardless of the usage hours.
Some may think that the need to pay upfront and be locked in undermines both “pay as you go” and the notion of being “elastic”- almost like a step backward to the old economic model.
An example of the savings offered by each EC2 RI option, along with the percent of savings each has over the On-Demand price is shown below. From these graphs, you can see that with a 3-year contract, your savings would be much greater. Other things to note is that you will have greater savings with Standard Instances, as well as if you choose the “All Upfront” payment plan. While you would receive discounted hourly rates for choosing Partial Upfront or No Upfront as a payment plan, if you can, All Upfront would be your best option with the most savings.
How should I use my Reserved Instances?
In non-production environments such as dev, test, QA, and training, Reserved Instances are not your best bet. Why is this the case? These environments are less predictable; you may not know how many instances you need and when you will need them, so it’s better to not waste spend on these usage charges. Instead, schedule such instances (preferably using ParkMyCloud). Scheduling instances to be only up 12 hours per day on weekdays will save you 65% – better than all but the most restrictive 3-year RIs!
Reserved Instances are very much a “use it or lose it” proposition. In other words, there are no rollover minutes – if you don’t use your reserved instances one month you don’t get extra time the next month. Here’s why they are like this:
The EC2 options available are specific to Region, Availability Zone, Instance Type (e.g. m5.large) with some exceptions, Platform Type (e.g. Linux or Windows), and Tenancy. AWS, behind the scenes, attempts to randomly match instances you launch to the Reserved Instance contracts you have in place, based on the specific criteria. When there is a match, the cost benefit is applied. It is not uncommon for people to believe they are launching instances that match all the criteria, when in fact they are not, so the contracts are under-utilized. And you won’t know what matches were made until you get your bill at the end of the month.
AWS decrements the contract amount for every hour when not used, meaning your return on investment diminishes.
For every hour in your RI term, you pay the fee for hourly usage regardless of whether there has been any usage during that hour.
Given all of the tradeoffs mentioned above, Reserved Instances make the most sense in a production environment, where instances need to always be “on.”
How ParkMyCloud Can Help Manage Your Reserved Instances
ParkMyCloud is an easy to use platform that allows users to automatically identify and eliminate wasted cloud spend. You can use the ParkMyCloud platform to fully optimize your non-production instances without committing to an AWS EC2 RI term that will go underutilized. The platform does this by scheduling, rightsizing, and identifying idle instances. Recently, we added the ability to view all your existing Reserved Instances in the platform so you can better track what commitments you have already made, with more optimization functionality coming soon.
With ParkMyCloud, you can create parking schedules that automatically turn EC2 instances on and off according to your specifications. ParkMyCloud provides customized parking recommendations based on criteria provided by the user, which makes identifying “parkable” instances easier – and you can automatically accept these recommendations if you like. Turning this into an automated process cuts down on time and costs, thus further optimizing your cloud environments. Another perk of ParkMyCloud is that the platform tracks costs, projected 30-day savings, and actual savings for the current month – giving you better visibility.
ParkMyCloud easily achieves EC2 savings of 50-73% with no annual commitment, upfront payment, or risk of instance termination or price cuts. In fact, we had a customer cancel a $10,000 order for AWS Reserved Instances in favor of EC2 instances that they could turn on and off after they found out just how easy and powerful this cost savings tool can be. Here are some of the advantages that come with using ParkMyCloud:
No commitment or upfront payment
Price cut protection
Try out ParkMyCloud for yourself and get started parking your non-production systems and RightSizing your resources to ensure that your environments are running in the most efficient way possible.
As more large enterprises adopt Azure cloud, especially those that have traditionally used Microsoft tools, we have observed growing interested in Microsoft Azure Enterprise Agreements, commonly known as EAs. We thought it would be useful to understand more about Microsoft EA’s, how they work with Azure, and what they mean to both the enterprise and the ISV.
What is an Azure Enterprise Agreement?
While you can create an Enterprise Agreement with Microsoft specifically for Azure, most companies using this option already have an EA in place for use of their software assets like Windows, Office, Sharepoint, System Center, etc. If you have an EA for other products, then you can simply add Azure to that existing agreement by making an upfront monetary commitment. You can then use eligible Azure cloud services throughout the year to meet the commitment. And you can pay for additional usage beyond the commitment, at the same rates. So, like any Enterprise License Agreement (ELA), including AWS’s EDP, you are committing to a contract term and volume to gain additional discounts.
According to Microsoft, the Enterprise Agreement is designed for organizations that want to license software and cloud services for a minimum three-year period. The Enterprise Agreement offers built-in savings ranging from 15 percent to 45 percent based on committed spend – and given how these commitments typically work, it is likely that the more you buy, the better your discount. The minimum listed commitment for an EA is 500 more users or devices for commercial companies (250 for public sector), and they specifically state this minimum does not apply to Server and Cloud Enrollment, an offering aimed at companies with EAs in place to help them standardize on Microsoft server and cloud technologies.
As it turns out, the Azure Enterprise commitment minimum is very low. You are required to make an upfront monetary commitment for each of the three years of the agreement, with a minimum order value of one “Monetary Commitment SKU” of $100 per month ($1,200/year). This low commitment make sense: once an enterprise is on a cloud platform, it’s sticky – land and expand is the name of the game for Azure, AWS, and Google. They expect infrastructure to grow significantly beyond the minimum, and just need to get a foot in the door. And of course,the starting point on the cloud is supposed to be much cheaper and flexible than on prem infrastructure.
Benefits of an Azure Enterprise Agreement… Beyond Pricing
There are certain Azure-specific EA benefits besides just price to entice users to move off of Pay-As-You-Go. You can create and manage multiple Azure subscriptions with a single EA. You can also roll up and manage all your subscriptions, giving you an enterprise view of how many resource minutes you’re using per subscription. In addition, you can assign subscription burn to accounting departments and cost centers so you can more easily manage budgets and see spend at various roll up levels.
EAs give you access to certain features that you’d otherwise be required to purchase separately. For example, an Azure EA gives you the option to purchase Azure Active Directory Premium, which will give you access to multi-factor authentication, 99.99% guaranteed uptime, and other features. Pay-As-You-Go only gives you access to the free version of Azure AD.
Besides getting the best pricing and discounts, what are some of the other added benefit an EA might provide to an enterprise:.
A common IT platform deployed across the organization.
Minimal up-front costs and the ability to budget more effectively by locking in pricing and spreading payments over three years.
Flexibility to choose from Microsoft cloud services, on-premises software, or a mix of both and migrate on your own terms.
Simplified purchasing with predictable payments through a single agreement for cloud services and software.
Managed licensing throughout the life of your agreement with the help of a Microsoft Certified Partner or a Microsoft representative.
Now, for vendors like ParkMyCloud, that need Azure pricing data to perform our service, how are we affected by the EA? Not adversely: the good news is that Microsoft makes EA pricing available through dedicated APIs and/or the Azure Price Sheet. We can match this information to a customer by using their Offer ID which defines their EA subscription and corresponding pricing (discounts).
How Else Can You Save Money on Azure?
Whether an Azure Enterprise Agreement makes sense for your organization is up to you to decide. Luckily, it’s not the only way to keep Azure costs in check. Here are a few others to explore:
One of the key drivers to a multi-cloud strategy is the fear of vendor lock-in. “Vendor lock-in” means that a customer is dependent on a particular vendor for products and services, and unable to use another vendor without substantial switching costs or operational impact. The vendor lock-in problem in cloud computing is the situation where customers are dependent (i.e. locked-in) on a single cloud service provider (CSP) technology implementation and cannot easily move to a different vendor without substantial costs or technical incompatibilities.
Vendor Lock-in: Public Cloud vs. Traditional Infrastructure
Before the cloud, IT was running in dedicated on-premises environments, requiring long-term capital investments and an array of software license commitments and never ending hardware refresh contracts. Based on that experience, it is understandable that a customer would be concerned about lock-in. Many large IT vendors like Oracle, IBM, HP, and Cisco would “lock” customers into 3-5-10 year Enterprise License Agreements (ELAs) or All You Can Eat (AYCE) hardware and software license agreements, promising huge discounts and greater buying power – but only for their products, of course. I used to sell these multi-year contracts. There is a common ground for sure, as the customer was locked-in to the vendor for years. But that was then and this is now. Is vendor lock-in really a concern for public cloud users?
Isn’t the point of cloud to provide organizations the agility to speed innovation and save costs by quickly scaling their infrastructure up and down? I mean, we get it – your servers, data, networking, user management, and much more are in the hands of one company, so the dependence on your CSP is huge. And if something goes wrong, it can be very detrimental to your business – your IT is in the cloud, and if you’re like us, your entire business is developed, built and run in the cloud. Most likely, some or all of your organization’s IT infrastructure where you are developing, building and running applications to power your business and generate revenue, is now off-premise, in the cloud. But although “lock-in” sounds scary, you are not stuck in the same way that you were with traditional hardware and software purchases.
Can You Really Get “Locked In” to Public Cloud?
Let’s talk about the realities of today’s public cloud-based world. Here are a couple of reasons why vendor lock-in isn’t as widespread a problem as you might think:
No Long-Term Commitments: Customers can adopt the cloud on their own terms. AWS, Azure, and Google Clouds are designed so customers only use the services when they see value, and they are free to use the technology of their choice. Pay-as-you-go pricing provides customers with the ability to shut down their environment, export their data and virtual machines (VMs), and walk away without ever incurring another expense. Customers are billed monthly without any required long-term commitments or contracts regardless of spend or support tier.
Customer Choice: Today’s cloud customers have alternatives to proprietary tools with advances in open source software technologies, along with a range of ‘as-a-service’ capabilities that can remake traditional IT — IaaS, PaaS, and even SaaS. A wide range of solutions that support industry standards allow customers to choose what they want to invest in and architect for application portability from the beginning, if they so choose.
Moving Into and Out of a CSP: Generally speaking, cloud services are built to support both migration into and out of their platforms, and CSPs and the industry at large provide many tools and documented techniques to make it easy to do both. Many cloud service providers offer tools to help move data between networks and technology partners. Customers can securely move information in and out of the cloud regardless of where that information is going: cloud-to-cloud or cloud-to-data center.
How to Mitigate Risk with a Multi-Cloud Strategy
Now the cloud is not without risk, and when we talk to customers the primary vendor lock-in concerns we hear are related to moving to another cloud service provider IF something goes awry. You hope that this never has to happen, but it’s a possibility. The general risks include:
Data transfer risk – it is not easy to move your data from CSP to another.
Application transfer risk – If you build an application on one CSP that leverages many of its offerings, the reconfiguration of this application to run natively on another provider can be an extremely expensive and difficult process
Infrastructure transfer risk – Every major CSP does things a little bit differently.
Human knowledge risk – simply put, AWS is not the same as Azure which is not the same as GCP, and your IT team has likely gained a lot of institutional knowledge about that provider’s tools and configurations.
To minimize the risk of vendor lock-in, your applications should be built or migrated to be as flexible and loosely coupled as possible. Cloud application components should be loosely linked with the application components that interact with them. And, adopt a multi-cloud strategy.
How Much Should You Worry About Vendor Lock-In?
Many companies are familiar with vendor lock-in from dealing with traditional enterprise companies mentioned above – you begin to use a service only to realize too late what the terms of that relationship brings with it in regards to cost and additional features and services. The same is not entirely true with selecting a cloud service provider like AWS, Azure, or Google Cloud. It’s difficult to avoid some form of dependence as you use the new capabilities and native tools of a given cloud platform. In our own case, we use AWS and we can’t just wake up tomorrow and use Azure or Google Cloud. However, it’s quite possible to set your business up to maintain enough freedom and mitigate risk, so you can feel good about your flexibility.
So how much should enterprises worry about vendor lock-in in public cloud? IMHO: they shouldn’t.
Shadow IT: you’ve probably heard of it. Also known as Stealth IT, this refers to information technology (IT) systems built and used within organizations without explicit organizational approval or deployed by departments other than the IT department.
A recent survey of IT decision makers ranked shadow IT as the lowest priority concern for 2019 out of seven possible options. Are these folks right not to worry? In the age of public cloud, how much of a problem is shadow IT?
What is Shadow IT?
So-called shadow IT includes any system employees are using for work that is not explicitly approved by the IT department. These unapproved systems are common, and chances are you’re using some yourself. One survey found that 86% of cloud applications used by enterprises are not explicitly approved.
A common example of shadow IT is the use of online cloud storage. With the numerous online or cloud-based storage services like Dropbox, Box, and Google Drive, users have quick and easy methods to store files online. These solutions may or may not have been approved and vetted by your IT department as “secure” and/or a “company standard”.
Another example is personal email accounts. Companies require their employees to conduct business using the corporate email system. However, users frequently use their personal email accounts either because they want to attach large files, connect using their personal devices, or because they think the provided email is too slow. One in three federal employees has stated they had used personal email for work. Another survey found that 4 in 10 employees overall used personal email for work.
After consumer applications, we come to the issue of public cloud. Companies employ infrastructure standards to make support manageable throughout the organization, manage costs, and protect data security. However, employees can find these limiting.
In our experience, the spread of technologies without approval comes down to enterprise IT not serving business needs well enough. Typically, the IT group is too slow or not responsive enough to the business users. Technology is too costly and doesn’t align well with the needs of the business. IT focuses on functional costs per unit as the value it delivers; but the business cares more about gaining quick functionality and capability to serve its needs and its customers’ needs. IT is also focused on security and risk management, and vetting of the numerous cloud-based applications takes time – assuming the application provider even makes the information available. Generally, enterprise IT simply doesn’t or cannot operate at the speed of the other business units it supports. So, business users build their own functionalities and capabilities through shadow IT purchases.
Individuals or even whole departments may turn to public cloud providers like AWS to have testing or even production environments ready to go in less time than their own IT departments, with the flexibility to deploy what they like, on demand.
Is Shadow IT a problem?
With the advent of SaaS, IaaS and PaaS services with ‘freemium’ offerings that anyone can start using (like Slack, GitHub, Google Drive, and even AWS), Shadow IT has become an adoption strategy for new technologies. Many of these services count on individuals to use and share their applications so they can grow organically within an organization. When one person or department decides one of these tools or solutions makes their job easier, shares that service with their co-workers, and that service grows from there, spreads from department to department, growing past the free tier, until IT’s hand is forced to explicit or implicit approve through support. In cases like these, shadow IT could be considered a route to innovation and official IT approval.
On the other hand, shadow IT solutions are not often in line with organizational requirements for control, documentation, security, and reliability. This can open up both security and legal risks for a company. Gartner predicted in 2016 that by 2020, a third of successful attacks experienced by enterprises will be on their shadow IT resources. It’s impossible for enterprises to secure what they’re not aware of.
There is also the issue of budgeting and spend. Research from Everest Group estimates that shadow IT comprises 50% or more of IT spending in large enterprises. While this could reduce the need for chargeback/showback processes by putting spend within individual departments, it makes technology spend far less trackable, and such fragmentation eliminates the possibility of bulk or enterprise discounting when services are purchased for the business as a whole.
Is it a problem?
As with many things, the answer is “it depends.” Any given Shadow IT project needs to be evaluated from a risk-management perspective. What is the nature of the data exposed in the project? Is it a sales engineer’s cloud sandbox where she is getting familiar with new technology? Or is it a marketing data mining and analysis project using sensitive customer information? Either way, the reaction to a Shadow IT “discovery” should not be to try to shame the users, but rather, to adapt the IT processes and provide more approved/negotiated options to the users in order to make their jobs easier. If Shadow IT is particularly prevalent in your organization, you may want to provide some risk management guidance and training of what is acceptable and what is not. In this way, Shadow IT can be turned into a strength rather than a weakness, by outsourcing the work to the end users.
But, of course, IT cannot evaluate the risk of systems it does not know about. The hardest part is still finding those in the shadows.