Kubernetes Operators are a concept introduced by CoreOS project to extend the capabilities of Kubernetes, particularly the management of Stateful applications. It is supposed to simplify the entire lifecycle of stateful resources from packaging the application and deploying it to a Kubernetes cluster all the way up to managing and scaling the app.

As an example, some of the core Kubernetes components like etcd have their corresponding Operators made available by the CoreOs project. Etcd is a distributed key-value store that reflects the running state of the entire Kubernetes cluster at any given instant. Naturally, it is a stateful application and various Kubernetes controllers refer to etcd in order to figure out what their next step is going to be. For example, ReplicaSet controller will look at the number of pods running under a given selector and try to bring the number of running instances equal to the number specified by your ReplicaSet or Deployment YAML. The ReplicaSet refers to etcd which keeps track of the number of running pods and once the number of pods is changed to a desired value the etcd would update its record of it too.

But when it comes to Stateful applications, like etcd itself, we can’t spin up more pods across different nodes without some serious intervention. Because all the running instances must have data consistent with one another at all times. This is where Operators come in handy.

Prerequisites

If you wish to follow along in this tutorial you can start with something small like a Minikube installed on your laptop, or the Kubernetes distribution that comes with Docker for desktop.

The important thing is to have an understanding of the basic ideas of Kubernetes to begin with.

Etcd

Let’s create an Operator that would manage etcd across our Kubernetes cluster. We won’t be installing etcd as a Kubernetes component (that is to say in the kube-system namespace) but as a regular application. Because doing that would put the entire cluster at risk. However, once you are comfortable with Operators you can use them to deploy etcd in the kube-system as you are bootstrapping a new cluster.

I will be using Katacoda Playground here, and a closer inspection of the kube-system namespace would show you that we have one pod running etcd for us. But that is not something we will be fiddling with. We will install etcd in default namespace managed by etcd-operator

Starting out the default namespace has no pods running, we have a clean slate.

No resources found.

Now let’s install a new etcd instance in this namespace. We start by cloning the repository followed by a simple kubectl command.

Creating Etcd Operator

In the repo, there are several examples to operate on, the first would create a simple etcd operator using deployment.yaml file. Before we use that we first need to create a Role for the operator via which is can manage and scale the etcd cluster. You can create that Role using a shell script.

The operator object will be created by the last command although there will nothing to operate on. We don’t have an etcd cluster yet. So let’s create one.

This creates a cluster of etcd pods. You can see them using:

The first one in this list is the operator pod which would ensure that the etcd cluster maintains a certain state, as stated in the yaml files we used earlier. If you try deleting one of the example-etcd-cluster pods, another one will be created to take its place. That’s remarkably similar to what ReplicaSet does but here that pods are stateful!

Operators in General

As mentioned earlier, Operators are a general framework within which one can deploy and manage complex applications. The framework itself is what makes them useful and the particular examples like etcd operator or Prometheus operator that CoreOS provides are meant to act as a guide for you to develop your own application in a similar manner.

A few important aspects of Kubernetes Operators are the SDK used for writing, building and testing your own custom operator, the second is the idea of Operator Life Cycle Manager wherein you can think about all the various stages that your operator as well as the service it offers can go through.

The life cycle stages might include various updates, figuring out what operator is running in which namespaces and also updating the operators when a new version comes along.

References

You can read a lot more about this technology in:

1. CoreOS’ original post, and
2. The etcd operator can be explore here