Building a Mutant Monitoring System

Overview

This course touches on different topics, providing a backstory that is relevant for the first few lessons. The lessons can be taken independently of one another. Some of the topics covered in the course are covered more in-depth in other courses. In such cases, a link is provided to other lessons and material.

If you are not familiar with the basic concepts of NoSQL and of ScyllaDB, it’s recommended you start with the ScyllaDB Essentials course.

Backstory

Mutants have emerged from the shadows and are wreaking havoc on the earth! Increased levels of malicious mutant behavior pose a threat to national security and the general public. Luckily, some mutants have teamed up with Government agencies to help protect people against the bad ones.

To better protect the citizens and understand more about the mutants, the Government enacted the Mutant Registration Act. As required by the act, each mutant must wear a small device that reports on their actions every second.

Your mission, as a new member of Division 3, is to help the Government keep the Mutants under control by building a Mutant Monitoring System (MMS). The MMS will consist of a database that will contain a Mutant Catalog and Monitoring system.

Choosing a Database

A NoSQL database is preferred for MMS because it has high availability, a scale-out data set, disaster recovery capabilities, and can accommodate IoT time-series workloads – all the requirements for the MMS database. There are many NoSQL database technologies to choose from, such as HBase, Apache Cassandra, MongoDB, Influx, and ScyllaDB.

ScyllaDB was chosen for the MMS because it leverages the best from Apache Cassandra in high availability, fault tolerance, and rich ecosystem. ScyllaDB provides a dramatically higher-performing and resource-effective NoSQL database to power modern and demanding applications with low latencies and none of the Java garbage collection woes found in other NoSQL databases.

A collection of data on each mutant needs to be imported into the database. The Mutant Catalog database will keep track of basic contact information for each mutant, including the mutant’s first name, last name, address, and photo.

To monitor the mutants, we need a separate tracking database consisting of time-series data such as the mutant’s location, speed, velocity, heat, and telepathy powers. The data will be gathered from various sources and imported into the database. After the data is gathered, we will be able to analyze the statistics.

Setting up a Cluster

Before we proceed, follow this procedure to set up a ScyllaDB cluster.

Building the Mutant Catalog

As we explained, the Mutant Catalog will consist of each mutant’s name, address, and picture. Let’s connect to a ScyllaDB node to create it. To manipulate the ScyllaDB database, we will be using the cqlsh tool. Cqlsh is a command-line tool written in Python that allows us to run Cassandra Query Language commands on ScyllaDB.

docker exec -it scylla-node1 cqlsh

The first task is to create the keyspace for the catalog.

CREATE KEYSPACE catalog WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy','DC1' : 3};

The catalog keyspace uses the NetworkTopologyStrategy because we will need to think about replicating the data to other data centers for disaster recovery purposes later. The data replica count is three, which means that each node in the cluster will have a copy of the data.

Now that the keyspace is created, it is time to create the table.

use catalog;

CREATE TABLE mutant_data (
   first_name text,
   last_name text,
   address text,
   picture_location text,
   PRIMARY KEY((first_name, last_name)));

The data types used in the catalog are text which is a string of data. The “PRIMARY KEY” section allows us to query the database by the mutant’s first and last name after we begin to gather data.

Now let’s add a few mutants to the catalog with the following statements:

insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Bob','Loblaw','1313 Mockingbird Lane', 'http://www.facebook.com/bobloblaw');
insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Bob','Zemuda','1202 Coffman Lane', 'http://www.facebook.com/bzemuda');
insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Jim','Jeffries','1211 Hollywood Lane', 'http://www.facebook.com/jeffries');

We can see all of the data inserted with the following statement:

select * from mutant_data;

To query the database for “Bob Loblaw”:

select * from mutant_data where first_name='Bob' AND last_name='Loblaw';

Summary

We now have the ScyllaDB infrastructure in place to start building out the Mutant Monitoring System. The Mutant Catalog is the first keyspace that was created to gather basic metrics for each mutant, such as name and contact information. With the latter part of the training, you learned how to add and query data from this keyspace. In a following lesson, we will look at building the Tracking System. The faster that we get this system up and running, the better for the fate of our country. Please remain safe out there.