A deep dive into backup, how it’s performed, its benefits, and a CLI example of running it.
Transcript
The strategy that we’re using is like a Snapshot & Upload.
You can have a retention policy, as an integer
so you may say that I want my backup to be repeated every day but I
want to keep only two last days backed up.. It has similar to repair, it has retries
build in but there’s different layers, levels, Rclone have
their own retires, we have some HTTP retries for Scylla server to agent
communication and we also have backups and scheduler for full jobs. You can
control rate limit, so basically all those limits are, I would say DC aware, so
you may specify with glob patterns. DCs and for that DCs you may
specify different parameters, so for one. DC you may specify different rate limit
and for the other DC and you may also have DC filtering so you may
backup only single DC or a set of them and again similar as for repair we have
the flexible keyspace/table selection with glob patterns so you have
this -K or keyspace flag that lets you specify the list of patterns that would
match or exclude keyspaces and tables if you want to backup only one keyspace
or table. Alright, so the key features are low impact on your node as well
as security and one thing that I want to mention here is that similar
to like health check and other of the design decisions, we are
credentials agnostic, so we’re highly recommend to use IAM Roles, so you would
assign this IAM Role to your node and then you would assign this Role’s write privilege
to the bucket and that’s it otherwise you may specify your AWS
credentials on the node directly.
I wanted to show you some old CLI, how it looks in CLI, so basically that’s
very similar to what we have for repair. So you can
list your backups as well and the source of truth is Ask-free
bucket direct, so we don’t read that information from database we
read it from registry directly so in case of like a complete failure
you had just another cluster and then you can use this tool to connect it to
existing S3 bucket and restore from there