BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Container-Native Storage - A New Market Segment with High Growth Potential

Following
This article is more than 4 years old.

The rise of Kubernetes has led to the creation of new market segments with huge potential. Container-native networking and storage are the fastest growing markets within the Kubernetes ecosystem. Interestingly, the market for these products didn’t even exist a few years ago.

Pixabay

When Kubernetes was first announced as an open source project by Google, it was perceived as a platform for web-scale companies running highly elastic and dynamic workloads. The initial focus of Kubernetes was on deploying and scaling stateless applications.

Modern application architecture deals with stateless and stateful workloads. Stateless workloads follow a shared-nothing pattern in which each instance of the workload runs independently. For example, a web application whose state is managed externally in a Redis cache and a MySQL database can scale to run a few hundreds of instances. Since each instance is autonomous and independent, multiple instances can be launched and terminated rapidly without worrying about keeping the state in sync across the running instances. The state is managed centrally in an in-memory database such as Redis or a relational database like MySQL.

When compared to stateless workloads such as API layers and web frontends, running stateful workloads in Kubernetes has been challenging. The platform did not support traditional storage operations such as cloning of volumes, automated snapshots, expanding volumes, backup and restore and migration of volumes across Kubernetes clusters.

The first generation of containerization targeted stateless services as they were easy to refactor. Stateful services such as databases and content management systems ran within VMs and cloud-based managed services. It was a common scenario to deploy an API and the web frontend in a Kubernetes cluster while running the database in Amazon RDS or Google Cloud SQL.

As Kubernetes matured to support a broad range of applications, it went beyond stateless applications to run enterprise line-of-business applications. Customers investing in Kubernetes are considering running end-to-end applications including stateless and stateful workloads within the cluster.

The maturity of Kubernetes resulted in a new standard called Container Storage Interface (CSI). Prior to CSI, Kubernetes followed the volume plugin mechanism that was tightly coupled and dependent on the internals of the orchestration engine. This approach forced storage providers to open source their implementations. The dependency on Kubernetes meant that the volume plugins had to be tested rigorously for a variety of scenarios before the integration. Bugs in volume plugins could potentially crash Kubernetes reducing the stability of the platform.

In January 2018, Kubernetes introduced the implementation of CSI which made it possible for third-party storage providers to develop solutions without the need to add to the core Kubernetes codebase.

CSI effectively decoupled storage providers from the underlying orchestration engine enabling them to expose a standard storage interface for Kubernetes, Mesos, Docker and Cloud Foundry. A year after its initial release, CSI for Kubernetes became generally available in January 2019.

The availability of CSI specification and the acceptance of Kubernetes by enterprises led to container-native storage platforms. These contemporary storage providers make it possible to run data-driven applications on Kubernetes. Enterprises can deploy traditional workloads such as SAP, PeopleSoft, SQL Server, Oracle and Hadoop on Kubernetes. The same storage backend can be leveraged for running data stores for microservices such as MongoDB, Cassandra, Redis, MySQL and PostgreSQL.

With Kubernetes becoming the preferred platform of hybrid cloud, enterprises will move mainstream workloads to it. Google is betting big on Anthos while IBM is pushing Cloud Private (ICP) into enterprises. Both Anthos and ICP use Kubernetes as the platform for running enterprise workloads in a hybrid environment.

Players such as Red Hat, Portworx, StorageOS, Robin Systems, Maya Data and Reduxio offer container-native storage infrastructure. Portworx, the Bay-area based startup, recently raised $27M Series C funding, which shows the investor confidence of this market.

The hybrid cloud push of Kubernetes creates a demand for a reliable container-native storage fabric that’s consistent across the enterprise data center and the public cloud. This storage fabric becomes the backbone for running enterprise line-of-business applications and mission-critical stateful workloads.

As Kubernetes makes inroads into the enterprise, the container-native storage ecosystem becomes important. Expect the players in this space to experience rapid growth in the coming months.

Follow me on Twitter or LinkedInCheck out my website