AWS Neptune Overview – Amazon’s Graph Database Service

AWS Neptune Overview – Amazon’s Graph Database Service

AWS Neptune is AWS’s managed graph database service, offered to give customers an option to easily build and run applications that work with highly connected datasets. It was first announced at AWS re:Invent 2017, and made generally available in May 2018.

Graph databases like AWS Neptune were created to address the limitations of relational databases, and offer an efficient way to work with complex data. 

What is a graph database?

A graph database is a database optimized to store and process highly connected data – in short, it’s about relationships. The data structure for these databases consists of vertices or nodes, and direct links called edges.

Use cases for such highly-connected data include social networking, restaurant recommendations, retail fraud detection, knowledge graphs, life sciences, and network & IT ops. For a restaurant recommendations use case, for example, you may be interested in the relationships between various users, where those users live, what types of restaurants those users like, where the restaurants are located, what sort of cuisine they serve, and more. With a graph database, you can use the relationships between these data points to provide contextual restaurant recommendations to users.

Details on the AWS Neptune Offering

AWS Neptune Pricing 

The AWS Neptune cost calculation depends on a few factors:

  • On-Demand instance pricing – you’ll need to pay for the compute instances needed for read-write workloads as well as Amazon Neptune replicas. These follow the general pricing for AWS On Demand instances.
  • Database Storage & I/Os – storage is also paid per usage with no upfront commitments. Storage is billed in per GB-month increments and I/Os are billed in per million request increments. 
  • Backup storage – you are charged for the storage associated with your automated database backups and database cluster snapshots. As per usual, increasing the retention period will cost more. 
  • Data transfer – you are charged per GB for data transferred in and out of AWS Neptune.

For this, as with most AWS services, pricing is confusing and difficult to predict. 

AWS Neptune Use Cases

Use cases for the AWS graph database and other similar offerings include:

  • Machine learning, such as intelligent image recognition, speech recognition, intelligent chatbots, and recommendation engines.
  • Social networking
  • Fraud detection – flexibility at scale makes graph databases useful to work with the huge amount of transactional data needed to detect fraud. 
  • Regulatory compliance – ever-more important as HIPPA, GDPR, and other regulations pose strict regulations on the way organizations use data about customers.
  • Knowledge graphs – such as advanced results for keyword searches and complex content searches.Life sciences – graph databases are uniquely suited to store models of disease and gene interactions, protein matterns, chemical compounds, and more. 
  • Network/IT Operations to keep networks secure, including identity and access management, detection of malicious file paths, and more. 
  • Supply chain transparency – graph databases are great for modeling complex supply chains that span the globe. 

Tired of SQL?

If you’re tired of SQL, AWS Neptune may be for you. A graph database is fundamentally different from SQL. There are no tables, columns, or rows – it feels like a NoSQL database. There are only two data types: vertices and edges, both of which have properties stored as key-value pairs.

AWS Neptune is fully managed, which means that database management tasks like hardware provisioning, software patching, setup, configuration, and backups are taken care of for you.

It’s also highly available and shows up in multiple availability zones. This is very similar to Aurora, the relational database from Amazon, in its architecture and availability.

Neptune supports Property Graph and W3C’s RDF. You can use these to build your own web of data sets that you care about, and build networks across the data sets in the way that makes sense for your data, not with arbitrary presets. You can do this using the graph models’ query languages: Apache TinkerPop Gremlin and SPARQL.

AWS Neptune Visualization is not built in natively. However, data can be visualized with Amazon SageMaker Jupyter notebooks, or third-party options like Metaphactory, Tom Sawyer Software, Cambridge Intelligence/Keylines, and Arcade. 

Other Graph Database Options

There’s certainly competition in the market for other graph database solutions. Here are a few that are frequently mentioned. 

AWS Neptune vs. Neo4j

Neo4j is a graph database that has been rated most popular by mindshare and adoption. Version 1.0 was released in February 2010. Unlike AWS Neptune, Neo4j is open source. Neo4j uses the language Cypher, which it originally developed. While there are several languages available in the graph database market, Cypher is widely known by now. 

Neo4j, unlike AWS Neptune, does actually come with graph visualization, which is a huge plus for working with this kind of data, though as mentioned above, there are several ways to visualize your Neptune data.

Other

Other graph databases include: AllegroGraph, AnzoGraph, ArangoDB, DataStax Enterprise Graph, InfiniteGraph, JanusGraph, MarkLogic, Microsoft SQL Server 2017, OpenLink Virtuoso, Oracle Spatial and Graph (part of Oracle Database), OrientDB, RedisGraph, SAP HANA, Sparksee, Sqrrl Enterprise, and Teradata Aster.

AWS Neptune – Getting Started

If you’re interested in the new service, you can check out more about AWS Neptune. As you get started, the AWS Neptune docs are a great resource. Or, check out some AWS Neptune Tutorials on YouTube

Once you’re on board, make sure you have cost control as a priority. ParkMyCloud can now park Neptune databases to ensure you’re only paying for what you’re actually using. Try it out for yourself!

Can Azure Dev/Test Pricing Save You Money?

Can Azure Dev/Test Pricing Save You Money?

Azure Dev/Test pricing is an option that Azure offers to give developers access to the tools that are necessary to support ongoing development and testing in Microsoft Azure services. This, hopefully, should give the user more control of their applications and environments reducing waste. 

Azure Dev/Test Pricing Options

With Azure Dev/Test pricing, three different options are available to users – Individual, Teams (Enterprise Agreement Customers), and another Teams option for those customers that don’t fall under the enterprise agreement. These pricing options are offered solely to active Visual Studio subscribers. We’ll dig in a little deeper to the pricing options and the benefits associated with each one. 

Option 1: Individuals

The individual option is meant to let users explore and get familiar with Azure’s services. As you can imagine, pricing for individuals is a little different than team pricing. Individuals are given the pricing option of monthly Azure credits for those who are subscribed to Visual Studio. If this pricing option is chosen, the individual is given a separate Azure subscription with a monthly credit balance ranging from $50-150.

You get to decide how you use your monthly credit. There are several Azure services that you can put the credit towards. The software included in your Visual Studio subscription can be used on Azure VMs for no additional charges, you pay a reduced rate for the VMs that you run. 

These monthly credits are ideal for personal workloads, but other options are more optimal for team workloads.

Option 2: Teams – Enterprise Agreement Customers

Teams that have an Enterprise Agreement in place have access to low Dev/Test rates for multiple subscriptions. The funds that are on the customer’s Enterprise Agreement will be used – there is no separate payment. A discount is given to customers at this level – all Windows and Windows Server, Virtual Machines, Cloud Services, and more are discounted off normal Enterprise Agreement rates. 

Unlike the option for Individuals, the team’s option for enterprise agreement customers allow end-users to access the application to provide feedback and to run tests – only Visual Studio subscribers can actually use the Azure resources running in this subscription. 

Option 3: Teams – All Other Customers

If a user isn’t an enterprise agreement customer but wants to use Azure for their teams, they would fall under this category. This rate offers a pay-as-you-go Dev/Test pricing option. This pricing option is very appealing because it allows users to quickly get their teams up and running with dev/test environments. Users are only allowed to use these environments for development and testing.  

This is a more flexible and inclusive option, it allows for multiple team members to interact with the resources, it’s not limited to just the account owners. 

Can Azure Dev/Test Save You Money?

All three options allow users to use the software that is included in their Visual Studio subscription for dev/testing. For VMs being run in environments in all three of these options, users are given a discounted price that is based on a Linux VM rate. 

Microsoft Azure users that are looking to save money on their cloud costs may want to use one of these options. These pricing options come with the benefit of no additional Microsoft software charges on Azure Virtual Machines and exclusive dev/test rates on other Azure services. 

Microsoft’s Start/Stop VM Solution vs. ParkMyCloud

Microsoft’s Start/Stop VM Solution vs. ParkMyCloud

Users looking to save money on public cloud may be in the market for a start/stop VM solution. While it sounds simple, there is huge savings potential available simply by stopping VMs, typically on a schedule. The basic idea is that non-production instances don’t need to run 24×7, so by turning VMs off when they’re not needed, you can save money.

If you use Microsoft Azure, perhaps you’ve seen the Start/Stop VM solution in the Azure Marketplace. You may want this tool if you want to configure Azure to start/stop VMs for the weekend or on weekday nights. It may also serve as a way to avoid creating a stop VM powershell.

Users of Azure have taken advantage of this option to start/stop VMs during off-hours, but have found that it is lacking some key functionality that they require for their business. Let’s take a look at what this Start/Stop tool offers and what it lacks, then compare it to ParkMyCloud’s comprehensive offering.

Azure Start/Stop VM Solution

Let’s take a look at Azure’s start/stop VM solution. The crux of this solution is the use of a few Azure services, specifically Automation and Log Analytics to schedule the VMs and Azure Monitor emails to let you know when a system was shut down or started. Both scheduling and keeping track of said schedules are important. 

As far as the backbone of Azure services, the use of native tools within Azure can be useful if you’re already baked into the Azure ecosystem, but can be prohibitive to exploring other cloud options. You may only use Azure at the moment, but having the flexibility to use other public clouds in the future is a strong reason to use cloud-agnostic tools today.

Next, this solution costs money, but it’s not very easy to estimate the cost (but does that surprise you?). The total cost is based on the underlying services (Automation, Log Analytics, and Azure Monitor), which means it could be very cheap or very expensive depending on what else you use and how often you’re scheduling resources. 

The schedules themselves can be based on time, but only for a single start and stop time – which is not practical for typical applications. The page claims it can be based on utilization, but in the initial setup there is no place to configure that. It also needs to be set up for 4 hours before it can show you any log or monitoring information. 

The interface for setting up schedules and automation is not very user-friendly. It requires creating automation scripts that are either for stopping or starting only, and only have one time attached. This is tedious, and the single-time configuration makes it difficult to maximize off time and therefore savings. 

To create new schedules, you have to create new scripts, which makes the interface confusing for those who aren’t used to the Azure portal. At the end of the setup, you’ll have at least a dozen new objects in your Azure subscription, which only grows if you have any significant number of VMs.

Users have noted numerous complaints in the solution’s reviews:

  • Great idea – painful to use – I don’t know why it couldn’t work like the auto shutdown built into the VM config with maybe a few more options (on/off weekdays vs. weekends). Feels like a painful set of scripts with no config options once it’s deployed (or I don’t understand how to use it).”
  • “Tried to boil the ocean – This solution is complex and bloated. It still supports classic VMs. The autostop solution only supports stop not start. Why bother using this?”
  • Start/Stop VM Azure – Difficult to do and harder to modify/change components. I’ll have difficulty to repeat to create another schedule for different VM.”

Luckily, there’s an easier option.

How it stacks up to ParkMyCloud

So if the Start/Stop VM Solution from Microsoft can start and stop Azure VMs, what more do you need? Well, we at ParkMyCloud have heard from customers (ranging from day-1 startups to Fortune 100 companies) that there are features necessary for a cloud cost optimization tool if it is going to get widespread adoption. 

That’s why we created ParkMyCloud: to provide simple, straightforward cost optimization that provides rapid ROI while being easy to use. You can use ParkMyCloud to save money through Azure start/stop VM schedules for non-production resources that are not needed evenings and weekends, as well as RightSizing overprovisioned resources.

Here are some of the features ParkMyCloud has that are missing from the Microsoft tool:

  • Single Pane of Glass – ParkMyCloud can work with multiple clouds, multiple accounts within each cloud, and multiple regions within each account, all in one easy-to-use interface.
  • Easy to change or override schedules – Users can change schedules or temporarily override them through the UI, our API, our Slackbot, or through our iOS app. 
  • Schedule recommendations – the Azure tool requires users to determine their own schedules. ParkMyClouds recommends on/off schedules based on keywords found in tags and names, and based on resource utilization history. 
  • Policy engine – ParkMyCloud can assign schedules automatically based on rules you create based on teams, names, or other criteria.
  • RightSizing – in addition to on/off schedules,  you can also save money with RightSizing. Our data shows that more than 95% of VMs are operating at less than 50% average CPU, which means they are oversized and wasting money.  Changing the VM size or family, or modernizing instance types, saves 50-75% of the cost of the instance.
  • User Management – Admins can delegate access to users and assign Team Leads to manage sub-groups within the organization, providing user governance over schedules and VMs. Admin, Team Lead, and Team Member roles are able to be modified to fit your organization’s needs.
  • No Azure-specific knowledge needed – Users don’t need to know details about setting up Automation Scripts or Log Analytics to get their servers up and running. Many ParkMyCloud administrators provide access to users throughout their organizations via the ParkMyCloud RBAC. This is useful for users who may need to, say, start and stop a demo environment on demand, but who do not have the knowledge necessary to do this through the Azure console.
  • Enterprise features – Single sign-on, savings reports, notifications straight to your email or chat group, and full support access helps your large organization save money quickly.
  • Integrations – use ParkMyCloud with your favorite SSO tools such as Ping and Okta. Get notifications and send commands back to ParkMyCloud through tools like Slack and Microsoft Teams.
  • Straightforward setup – it usually takes new users 15 minutes or less to set up a ParkMyCloud account, connect to Azure, and get started saving money. 
  • Reporting – with ParkMyCloud, users can view, download, and email savings reports covering costs, actions, and savings by team, credential, provider, resource, and more.
  • Notifications – users can get configurable notifications of ParkMyCloud updates & activities via email, webhook or ChatOps.
  • Huge cost savings and ROIhere are just a few examples from some of our customers.
    • A global fast food chain is managing 3,500+ resources in ParkMyCloud and saving more than $200,000 per month on their cloud spend
    • A global registry software company has saved more than $2.2 million on their cloud spend since signing up for ParkMyCloud – an ROI of 6173%
    • A global consumer goods company with 200+ ParkMyCloud users saves more than $100,000 per month on their cloud spend.

As you can tell, the Start/Stop VM solution from Microsoft can be useful for very specific cases, but most customers will find it lacking the features they really need to make cloud cost savings a priority. ParkMyCloud offers these features at a low cost, so try out the free trial now to see how quickly you can cut your Azure cloud bill.

Related Reading:

AWS Postgres Pricing Comparison

AWS Postgres Pricing Comparison

Maybe you’re looking to use PostgreSQL in your AWS environment – if so, you need to make sure to evaluate pricing and compare your options before you decide. A traditional “lift and shift” of your database can cause quite a headache, so your DBA team likely wants to do it right the first time (and who doesn’t?). Let’s take a look at some of your options for running PostgreSQL databases in AWS.

Option 1: Self-Managed Postgres on EC2

If you’re currently running your databases on-premises or in a private cloud, then the simplest conversion to public cloud in AWS is to stand up an EC2 virtual machine and install the Postgres software on that VM. Since PostgreSQL is open-source, there’s no additional charge for running the software, so you’ll just be paying for the VM (along with associated costs like storage and network transfer). AWS doesn’t have custom instance sizes, but they have enough different sizes across instance families that you can find an option to match your existing server.

As an example, let’s say you’d like to run an EC2 instance with 2 CPUs and 8 GB of memory and 100GB of storage in the us-east-1 region. An m5.large system would work for this, which would cost approximately $70 per month for compute, plus $10 per month for storage. On the plus side, there will be no additional costs if you are transferring existing data into the system (there’s only outbound data transfer costs for AWS).

The biggest benefit of running your own EC2 server with Postgres installed is that you can do any configuration changes or run external software as you see fit. Tools like pgbouncer for connection pooling or pg_jobmon for logging within transactions requires the self-management provided by this EC2 setup. Additional performance tuning that is based on direct access to the Postgres configuration files is also possible with this method.

Option 2: AWS Relational Database Service for Hosted Postgres Databases

If your database doesn’t require custom configuration or community projects to run, then using the AWS RDS service may work for you. This hosted service comes with some great options that you may not take the time to implement with your own installation, including:

    • Automated backups
    • Multi-AZ options (for automatic synchronization to a standby in another availability zone)
    • Behind-the-scenes patching to the latest version of Postgres
    • Monitoring via CloudWatch
    • Built-in encryption options

These features are all fantastic, but they do come at a price. The same instance size as above, an m5.large with 2 CPUs and 8 GB of memory, is approximately $130 per month for a single AZ, or $260 per month for a multi-AZ setup.

Option 3: Postgres-Compatible AWS Aurora

One additional option when looking at AWS Postgres pricing is AWS Aurora. This AWS-created database option is fully compatible with existing Postgres workloads, but enables auto-scaling and additional performance throughput. The price is also attractive, as a similar size of r5.db.large in a multi-AZ configuration would be $211 per month (plus storage and backup costs per GB). This is great if you’re all-in on AWS services, but might not work if you don’t like staying on the absolute latest Postgres version (or don’t want to become dependant on AWS).

AWS Postgres Pricing Comparison

Comparing these costs of these 3 options gives us: 

  • Self-managed EC2 – $80/month
  • Hosted RDS running Postgres in a single AZ – $130/month
  • Hosted RDS running Postgres in multiple AZ’s – $260/month
  • Hosted RDS running Aurora in multiple AZ’s – $211/month

Running an EC2 instance yourself is clearly the cheapest option from a pure cost perspective, but you better know how to manage and tune your settings in Postgres for this to work.  If you want your database to “just work” without worrying about losing data or accessibility, then the Aurora option is the best value, as the additional costs cover many more features that you’ll wonder how you ever lived without.

Azure AKS Overview: Azure’s Managed Kubernetes Service

Azure AKS Overview: Azure’s Managed Kubernetes Service

In today’s entry in our exploration of container services, we’ll look at Azure Kubernetes Service (AKS). Azure AKS manages your hosted Kubernetes environment, making it simple to deploy and manage containerized applications without container orchestration expertise, divesting much of that responsibility to Azure – much like EKS and GKE do for AWS and Google Cloud. Critical tasks like health monitoring of ongoing operations and maintenance by provisioning, upgrading, and scaling resources on demand are handled by Azure.

Azure AKS Overview

Azure AKS is, as of this writing, just over a year old, released for general availability in June 2018. With AKS, you can deploy, scale, and manage Docker containers and applications. Azure AKS gives developers greater flexibility, automation and reduced management overhead for administrators and developers. This is because it’s a managed service, which takes some of the management burden off the user.

As applications grow to span multiple containers deployed across multiple servers, operating them becomes more complex. To manage this complexity, Azure AKS provides an open source API to deploy, scale and manage Docker containers and container-based applications across a cluster of container hosts.

Use cases for AKS include:

  • Easily migrating existing applications to Kubernetes
  • Simplifying the deployment and management of microservices based applications
  • Easily integrated DevSecOps
  • IoT device deployment and management on demand
  • Machine Learning model training with AKS

If AKS is free, what do you pay for?

Yes, Azure AKS is a free service since there is no charge for managing Kubernetes clusters. However, you pay for the VM instances, storage and networking resources consumed by your Kubernetes cluster. These should be managed like any other cloud resources, with attention paid to potential areas of waste.

AKS vs. ACS

Microsoft’s experience with cluster orchestration began with Azure Container Service back in 2017, which supported Kubernetes, Docker Swarm and Mesosphere’s DC/OS. It was the simplest most open and flexible way to run container applications in the cloud then, and now followed by Azure Kubernetes Services (AKS), which was made generally available in 2018.

ACS users who run on Kubernetes can possibly migrate to AKS, but migration should be planned and reviewed for it to be successful as there are many key areas in which they are different. If considering migration, check out Azure’s guide to migrating from ACS to AKS here.

Should you use Azure AKS?

Chances are, you’re locked into a cloud provider – or have a preferred cloud provider – already, so you’re likely to use the container management service offered on your provider of choice. If you’re on Azure, AKS will be the natural choice as you increase use of microservices and app portability with containers.

Further reading: