TTL, Migrating from other DBs

Regardless
of which database you guys are using,
there are some chances that some of you are
probably relying on TTL – “Time. To Live”, in order
to control your data expiration.
In that case,
considering you are not subject to any of the edge cases
that we have discussed, then we have good news for you.
But before
I tell you the good news, let me see the question
from Lucas, There are global table is ScyllaDB
because of data centres
applications, all tables are global.
I assume there is no other way than creating separate
Scylla cluster in each zone to get local/zone
table.
Yeah, that’s pretty much that Lucas, you cannot
currently at least you cannot specify
a table to live in a single availability zone.
In a typical production ScyllaDB deployment, we would spend
at least three availability zones,
although you have flexibility to configure that.
And when you configure
the replication factor under a single zone, yeah,
by default to the partitioner, the replication algorithm
will spread your data to ensure that at least one copy lives
in a different availability zone.
Yeah. 7 minutes left.
So let’s hurry people and let’s get back to the good news. So
the good news, if you are
working with time to live tables, is that they are
pretty much fairly convenient to migrate as you can leverage
the power of time to live your date expiration to
do all the heavy lifting work for you.
For example, considering your TTL
is at worst, a couple of months.
One strategy that you can employ is just simply configure
your client to write to your ScyllaDB cluster
while writing and reading from your existing source of truth.
Then what’s going to happen since your data is TTL’d,
it will eventually expire on the source table.
And then the what you have to do is simply switch over, okay?
That simple.
And of course this considers that your
expiration period is at worst some months,
but if your expiration times are like, years,
then you can employ some other strategies, such as
when a user reads depending on the time
frame, here is requesting data, you would point to Scylla, etc.
and so what.
So you have some flexibility here as well.
We finally got to different databases where
your existing source of truth may be another, you know
SQL, relational, graph or any other kind of database that
we haven’t covered here thus far.
So how do I migrate from database X to ScyllaDB?
There’s always someone who asks just kind of question
rightfully asked.
And in general, guys,
the guidance that we have seen in the overview in the beginning
of the presentation, pretty much applies here as well.
However, the drawback is that you,
you won’t have a migrator which you can use,
especially because it’s hard for us to support
every distinct database that exists in the world,
and every API that exists in the world.
So at this point, you would need to come up with your own
tooling in order to do the heavy-lifting work, but
always remember the
following key important concepts which you have seen here.
And that’s essentially a replay of what
we’ve seen in the overview section.
That’s important general guidance
in order for you to perform a successful migration.
First, carefully choose your schema
and your data modeling
because this main affect which is important in order
for you to succeed in database like ScyllaDB.
If you got your schema incorrectly, then
everything that comes after during the migration
work is going to be simply ineffective.
You’re probably going to eventually go back
and revisit everything again.
After you are done with your schema
and your data modeling, move an offline copy of your tables
to your destination. ScyllaDB cluster.
And after you are done moving your offline
copy, your snapshot, replay your changes,
and then
you are going to switch the clients and lastly,
of course, we hope that you enjoy ScyllaDB.