Why Kubernetes is Foundational for Fortune 500 Digital Transformation (and the Cloud Infrastructure Management Landscape)
About 18 months ago, tech publications began overflowing with news on containers and how they would radically transform enterprise IT. We set out to distill the hype in this new market by putting together a new series on our blog aptly called "The State of the Container." Based on our research, it became clear that this was a real market trend and represented a once in a decade transformation in enterprise infrastructure that would reshape the entire IT industry.
It was apparent that the container and application layer were only the beginning, and we began investigating where the gaps were – in areas like security, networking, management, and orchestration – to see where we could support the next generation of innovators. We spoke to more than one hundred corporate executives and founders focused on bringing a tidal wave of innovation to the enterprise IT stack, while making two investments along the way: CoreOS & Cockroach Labs. With more of the recent buzz building around Kubernetes, we wanted to take a moment to compare the pros and cons of other platforms, and why we believe Kubernetes will be crown jewel of container orchestration.
A Little History
In the not too distant past, virtualization ruled the world. Runbooks and deployment scripts were used to automate application delivery in large enterprises. Often prone to human error, a crop of configuration management tools sprouted up to address these problems in complex Fortune 500 enterprise development environments. But then, as we all know, software ate the world.
At this point, the internet giants of today like Google, Netflix, and Facebook were all already using containers and a microservices architecture, where small modular components modeled around specific business functions work together - enabling developers to continuously deliver new products, features, and enhancements at scale. The Fortune 500 continued to build with proprietary software stacks supported by large centrally controlled teams, often spending months to deliver new features and products, while spending significantly more to do so than their webscale counterparts.
Not only is enterprise application architecture extremely complex, but it’s also exceedingly difficult to scale on demand. Unlike a distributed, cloud based microservices architecture, the normal utilization rate of an enterprise datacenter is usually about only 20% - a wasteful, frustrating, and expensive problem. While Fortune 500 executives initially envied the datacenter operational expenditure enjoyed by internet companies, another huge business benefit was the speed at which webscale companies were delivering solutions for customers, partners, and employees. Datacenter utilization and product agility become game-changers at scale, and the Fortune 500 realized the need for digital transformation.
Which Brings Us To Today
Now, almost every Fortune 500 is transforming into a software-based business in an effort to compete in rapidly changing markets. Having fully realized the enterprise use cases for microservices, Fortune 500 executives know that agile development practices and containers are key to this digital transformation. However, the container format itself is just the beginning - a building block that has been largely commoditized. The foundational piece that remains key for Fortune 500 adoption is container management, and the battle is on for market share.
The following is our evaluation of such systems relevant to the cloud infrastructure management ecosystem, as well as some additional background on each:
Cloud Foundry: Cloud Foundry is an open source cloud computing Platform as a Service (PaaS). Originally developed by VMware and now owned by Pivotal Software - a joint venture by EMC, VMware, and General Electric. Cloud Foundry supports the full lifecycle, from initial development, through all testing stages, to deployment. It is therefore well-suited to the continuous delivery strategy but is not container centric in its approach.
OpenStack: OpenStack began in 2010 as a joint project of Rackspace Hosting and NASA. It’s a free and open-source platform for cloud computing, deployed as an Infrastructure-as-a-Service (IaaS) solution. OpenStack consists of a set of control panel APIs with software that allows companies to deploy VMs and other software defined infrastructure environments in their own datacenter. While valuable in being able to simplify heterogeneous enterprise development environments, OpenStack is often criticized for being extremely difficult to install. However, as demo’d recently, Openstack works really well with Kubernetes, making it as simple as installing any application running on Kubernetes, which in turn provides enterprises the benefits of both containers and virtualization based IaaS.
Docker Swarm: Docker Swarm is native container clustering for Docker, turning a pool of Docker hosts into a single, virtualized host. Docker’s new release includes a its own orchestration system named SwarmKit, which adds container management built right into the Docker container format. This brings developers a simple and easy way to manage containers but it still has a way to go to be ready for production. From a community perspective, Swarm is ultimately controlled by a single startup and not supported by an open-source community, which adds some risk down the line. Solutions like Mesos and Kubernetes (below) provide more robust feature-sets that appeal to larger enterprises.
Apache Mesos: Mesos is a open source software originally developed at the University of California at Berkeley. It sits between the application layer and the operating system and makes it easier and more efficient to deploy and manage applications in large-scale clustered environments. Mesos’ core is a generic scheduling system, but it integrates well with other schedulers. This enables systems like Hadoop and Marathon to work well together. Mesos is less focused on running containers because of its roots in scientific computing. It existed before the extreme interest in containers and has been re-based in areas to provide better container support.
Kubernetes: Kubernetes was designed from the ground up to be the ideal platform to build, manage, and orchestrate distributed applications using containers. It includes primitives for replication and service discovery as part of its core (these are added via frameworks in Mesos and requires some know-how to configure properly) and the vision for Kubernetes is to develop a system that allows enterprises to manage scalable application deployments with maximum efficiency, security, and ease.
With the enterprise use cases in place, Fortune 500 companies are now looking to move beyond “Docker science projects.” They are taking the next step by evaluating the replacement of legacy systems with the aforementioned solutions. Many are deploying several of these together, siloed by teams and use cases. An important word of caution: operating these systems in parallel will create significant hurdles in the future that will lead to serious cost. When considering the management, security, support, and human expertise required for each of these platforms - as well as being dependable and efficient on top of that - it easily becomes an operational nightmare to manage them all.
Forward thinking organizations have realized that Mesos and Kubernetes are the best platforms to abstract and pool datacenter resources. What makes Kubernetes stand apart from Mesos and all the other offerings here is largely due to the open source community that is forming around it, driving its open integration with other key infrastructure components (like Soundcloud’s Prometheus and CoreOS’s etcd) to form a multi-vendor supported enterprise stack. The reason for this, according to Apprenda executive Chris Gaun, is that Kubernetes provides a "very strong ecosystem that mimics the Hadoop model.” In this model, Hadoop became a success because it wasn’t dominated by a single large vendor.
Most of the other infrastructure platforms covered in this post are largely managed under the direction of one, or several large vendors, and they do not play nicely with other open source components. As described by Accel at a recent Open Source event at Work-Bench, they lack the requisite open source communities, network of contributors, and frontline developers and customers to become foundational technologies embraced by the masses. Kubernetes’ explosive growth, on the other hand, follows this notion of community adoption.
Furthermore, Kubernetes builds on 15 years of experience of running cloud infrastructure production workloads at Google, while also combining it with best-of-breed ideas and practices from its growing community of contributors and companies that are actively adding critical features. The Kubernetes community has been laying down a solid foundation for running distributed infrastructure at enterprise scale, while teams from companies like CoreOS have been busy combining Kubernetes with other powerful open source technologies to help businesses run containers with enterprise-grade security, reliability and support. Although it may seem like these companies are all vying for a slice of the pie, it’s unlikely to become a winner takes all situation as we’ve seen in the past with IT disruptions, and these platforms will evolve to support different scenarios for different customers. It will be fascinating to see how all of this plays out in the enterprise over the next few years, and we continue to be extremely impressed with each new Kubernetes release.comments powered by Disqus