Coding with Python Part 3 – Data Types

In two previous lessons, Coding with Python – Part 1 and Part 2, we learned how to use the Python driver to create applications that interact with ScyllaDB. Division 3 has decided to explore the Python programming language a bit further and came across an exciting feature of ScyllaDB that would allow us to store files in the database. With this ability, we can save images of the mutants in the catalog keyspace. With the images stored, Division 3 can see what the mutant looks like whenever they want and even share the image and tracking details with local law enforcement officials if needed.

A table in ScyllaDB supports a wide array of data types such as timestamp, text, integer, UUID, blob, map, and more. For Division 3’s use case, we will add a new column to the catalog.mutant_data table and store images for each mutant there using a Python application. The new column will be of a type called “map,” where the key of the map will be a file name, and value will store the photo of the mutant in a binary format called “blob.” Since ScyllaDB is a distributed system with fault protection and resiliency, storing files in ScyllaDB will have the same benefits as our existing data based on the replication factor of the keyspace. To get started, we will first need to set up the ScyllaDB cluster.

Starting the ScyllaDB Cluster

follow this procedure to remove previous clusters and set up a new cluster.

Next, create the keyspace catalog and the table mutant_data:

docker exec -it scylla-node1 cqlsh

CREATE KEYSPACE catalog WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy','DC1' : 3};

use catalog;

CREATE TABLE mutant_data (
   first_name text,
   last_name text,
   address text,
   website text,
   picture_file map<text, blob>,
   PRIMARY KEY((first_name, last_name))
);

Building the Python Example Application

If you previously built the Python Docker app, you can skip directly to “Running the Application.”

Otherwise, to build the application in Docker, change into the python subdirectory in scylla-code-samples:

exit

cd scylla-code-samples/mms/python

Now we can build the container:

docker build -t python-app .

Running the Application

Finally, run the sample Python application:

cd python-datatypes

docker run -d --net=mms_web --name some-python-app python-app python_data_types.py

Viewing the Run Results

Once the Python app is run in the container, we can see the output by printing the logs:

docker logs some-python-app

The output of the application will be:

Let’s dive a little bit deeper to see what the code is doing. When the application is run, it will initiate a connection to the ScyllaDB cluster and initialize prepared statements for the supported operations: insert new mutant, delete existing mutant from the DB and get the photo of the mutant using it’s first and last name.

In the container, there is an image file for each mutant:

• peter_parker.jpg
• maximus_lobo.png

When adding new mutants to the database, the application will read the files from the container filesystem and insert them into the table. To store the file, we will use the picture_file column. The following method will read the file, store it in  a data variable and then insert a row into the table using the self.insert_ps prepared statement:

The final step for the Python application is to read the data from ScyllaDB and write the data to the /tmp directory. The  save_mutant_photo  method is used to fetch the relevant row for the mutant and save all the photos to the disk.

To retrieve the images from the container and to verify that everything worked correctly, we can run the following Docker command to copy the newly written images out of the container and to our machine:

docker cp some-python-app:/tmp/ .

Using your favorite image viewer, you can verify that each image is intact.

Conclusion

In this lesson, we went over the different data types that one can use in their database tables and learned how to store binary files in ScyllaDB with a simple Python application. By being able to store images in ScyllaDB, Division 3 will be able to quickly see what a mutant looks like and can share the details with local law enforcement if needed. With the ability to store files in the Mutant Monitoring System, the possibilities are endless for how Division 3 can continue to evolve the system. Please be safe out there and continue to monitor the mutants!