kingofsnake

Use Jupyter, SciPy, pandas, and scikit-learn without installing any Pythons.

abstract

kingofsnake is a template for a reproducible data analysis lab which:

• serves Jupyter notebooks from a Docker container.
• never conflicts with existing Python(s), Anaconda, or virtualenvs.
• includes pinned versions of these packages and their dependencies:

networkx
notebook
pandas[all]
requests
scikit-learn
scipy
seaborn

See the books folder for examples of:

• data cleaning
• unsupervised clustering
• supervised classification
• principal component analysis
• force-directed graph drawing

basics

Generate a new repo from this template.

bake a python environment

1. Open a terminal and cd to this folder.
2. Edit the Dockerfile to choose a Python version.
3. Edit requirements.txt to choose Python packages.
4. Run ./kitchen bake to build a kingofsnake:latest Docker image.
5. Run ./kitchen freeze to update requirements.txt and rebuild.

serve jupyter notebooks

1. Open a terminal and cd to this folder.
2. Run ./kitchen serve to start a Jupyter server.
3. Open a web browser and enter localhost:8888 in the address bar.

This runs Jupyter in a container, publishes port 8888, and mounts some folders from this repo:

• etc/ipython is mounted as /home/kos/.ipython
• etc/jupyter is mounted as /home/kos/.jupyter
• books is mounted as /home/kos/books
• code is mounted as /home/kos/code
• data is mounted as /home/kos/data

Jupyter security: On the first run, Jupyter might ask you to copypaste a token and create a password. It will save the hashed password and any custom settings to etc/ipython and etc/jupyter in this repo. If those folders do not exist, they will be created automatically. Git ignores the contents of both folders.

delete everything and start over

1. Open a terminal and cd to this folder.
2. ./kitchen clean stops and deletes all kingofsnake containers.
3. ./kitchen eightysix deletes the kingofsnake:latest image.

The clean command is rarely necessary because kingofsnake containers self-destruct.

contents

books

The books folder contains example notebooks:

• classify.ipynb trains and tests an sklearn.linear_model classifier.
• clean.ipynb standardizes, sorts, and filters pandas DataFrames.
• cluster.ipynb finds clusters with scipy.cluster.hierarchy.
• components.ipynb finds principal components with sklearn.decomposition.PCA.
• graph.ipynb draws graphs using the ForceAtlas2 energy model.
• plot.ipynb uses matplotlib to visualize data.

code

The code folder contains example Python modules:

• classify.py for classification
• cluster.py for clustering
• graph.py for graph drawing
• plot.py for data visualization
• tools.py for constants and convenience methods

data

This folder is for storing data files. Git ignores everything in it except a few examples.

dependencies

kingofsnake has one dependency:

1. Docker for Linux or Mac or Windows.

Windows users may need to edit the kitchen script for path compatibility.

examples

Show all available kitchen commands.

./kitchen help

Run a container as root without Jupyter, folder mounts, or published ports.

./kitchen runit latest

Bake another image called kingofsnake:karl, freeze it, and serve Jupyter.

./kitchen bake karl
./kitchen freeze karl
./kitchen serve karl

Delete the kingofsnake:karl image and its containers:

./kitchen eightysix karl

faq

How do I install kingofsnake?

Don't install anything. Use this repo as a template.

How do I make it stop?

Click on the terminal running Jupyter and press CTRL-C.

Can I run Jupyter in the background?

Yes. See the Docker run reference.

Do I need all these notebooks and modules?

No. Delete them if you want to.