K8 Operator – Maintenance with Scylla Manager

Now I’m going to show you how to do the proper maintenance, so we are going to use the
Scylla Manager, which on typical deployments like bare metal or cloud deployments looks like this
where you have the Scylla Manager server, which is doing all the maintenance work for you, there
is also a tiny Scylla cluster, dedicated for the data store for Scylla Manager and
administrators are using sctool to connect and talk with Scylla Manager, and Scylla Manager is managing
multiple Scylla clusters, so in Kubernetes there is also an additional component which basically
brings the integration for Scylla Manager and. Kubernetes and this controller is watching
the Scylla clusters deployed using Operator in your Kubernetes cluster and it is registering
these clusters in Scylla Manager and it is also responsible for synchronizing the tasks which are
specified in the Scylla cluster definition so if you specify a repair task
this controller will try do its best to add it or modify it inside Scylla Manager state, so
in order to deploy the Manager, you also apply a single file from the example directory
called manager,yaml and this basically installs the Manager, the Manager controller and
this Scylla cluster which is used as a data store, so we can check the Scylla Manager
system namespace, for the list of the pods and there should be three, one
for Scylla Manager, one for controller and one for the tiny Scylla cluster
and in order to schedule the jobs inside in Scylla Manager you basically
change your Scylla cluster and specify the list of the tasks here, so we can use kubectl edit
Scylla cluster, to change your existing cluster or you can create it with already list of the
repairs or list of the backups, so today I’m going to show you how to create a repair, so we have to
give it a human readable name, so let’s call it ‘weekly repair’ with an interval of seven days
there are a bunch of other parameters available, basically if you are familiar with Scylla Manager
and sctool, all of them are available here but also if you don’t know it, you can read it
in our documentation page, about all of them, so let’s just create this simple repair job
and let’s apply this Alternator,yaml
and it should
register this task and also the cluster inside the Manager
so let’s see whether Manager is ready, it’s not yet because the Scylla is still booting up
okay, now it’s ready, so let’s use the sctool inside the Scylla Manager pod
to check whether the cluster was automatically detected and registered and whether the task is
scheduled, so we are going to execute a command inside Manager pod
and we are going to use ‘sctool cluster list’
and there is a single cluster registered inside Scylla namespace
called ‘Scylla cluster’ so this is right, so let’s check the task list
and as you can see there is a bunch of health checks which are pre-registered for every
cluster and there is also a repair job, with some default arguments, which has already been done and
this repair executes every seven days, so we can check the task progress
using sctool by providing the ID of the repair and it shows that
it repaired the cluster in four seconds, this is because we only have
a single key space used for the game we played, so this is why it took four seconds
okay, for backups it’s very much the same, you just specify
the desired number of backups you want to execute and Scylla Manager controller will
register them in Scylla Manager, and. Scylla Manager will execute these backups