What is this

Contrast api repo

About this project

Relevant docs:

• parameters list
• structure

Licensing

We believe in open-science! The data is licensed under  CC-BY-SA-4.0 creative commons open license and the code here under the GNU GPLv3 open source license

Contributing

see CONTRIBUTING

Technology and setup

python 3.11 Django + drf + django-filter DB is postgresql Django configurations for settings file Common practice middlewares (timezones, querycount)

setup

Setup for development

1. Python virtual environment:
   We are using poetry to manage the projects dependencies.
   Install Poetry - https://python-poetry.org/docs/#installation

2. Get the code:
   Clone this project

   git clone git@github.com:Mudrik-Lab/Contrast2.git
   

3. Install dependencies:
   enter projects directory and install dependencies using Poetry. Poetry will look for pyproject.toml file

   cd contrast-api
   poetry install
   

   And enter the virtual env created by Poetry:

   poetry shell
   

---

From this point in the setup you should run the commands while you are inside the virtual env / poetry shell

---

4. Database:
   We are currently using postgres. You need to set up a user,

   • After you have installed postgres, enter postgres cli client:

   sudo su - postgres
   psql
   

   • create a database, a user and a role

   CREATE DATABASE contrast_api_db;
   CREATE USER contrast_api_user WITH PASSWORD 'contrast_api_pass';
   ALTER ROLE contrast_api_user SET client_encoding TO 'utf8';
   GRANT ALL PRIVILEGES ON DATABASE contrast_api_db TO contrast_api_user;
   ALTER ROLE contrast_api_user CREATEDB;
   ALTER DATABASE contrast_api_db OWNER TO contrast_api_user;
   

   • to exit postgres cli:
     Ctrl+D

     and then exit superuser shell
     exit

   • Now you can migrate the data:

   python manage.py migrate   
   

5. To load the pre-existing data you need to: 6.

   • Download this file in .xlsx format for ConTrast, or
   • Download this file in .xlsx format for UnConTrast
   7. • Save the file in /studies/data directory for ConTrast, or
      • Save the file in /uncontrast_studies/data directory for UnConTrast

   8. • Run this command for ConTrast:

         python manage.py load_historic_data 
        

      • Run this command for UnConTrast:

         python manage.py load_uncon_data 
        

Note: if you need to do this in bash (for example when migrating on the server): 1. Copy the file to your drive and temporarily allow open access to viewer to view 2. Follow the technique here 6. Create a superuser for yourself to start working

 python manage.py createsuperuser 

7. Run the dev server

   python manage.py runserver
   

tests

poetry run python manage.py test

Production

Currently this project is deployed to Heroku, via github actions ci

Setup for use in local environment as a "black box"

e.g when you work on the frontend

docker-compose up --build

# or to run in the background
docker-compose up -d --build

# You might drop --build, in case no changes in the poetry.lock file, but I'd suggest not to 

First run would be quite long because of docker building

Postgres has some issues currently with start order, so if you see errors in the logs, just restart the compose a few times until it work

For doing the initial data load while running in compose:

After copying the file as above, and after verifying compose up --build as above

docker exec -it web_contrast_api python manage.py load_historic_data
# This runs the load, but inside the already running containers

CI

As part of deploying this app we also deploy a separately build react app, and serve it from the django project with django-spa.

Linting and formatting

With ruff, not automated yet as part of CI

ruff check . --fix
ruff format

We're using ruff format for formatting please adjust prs accordingly

Upgrading a local Postgres cluster to a new major version

Postgres refuses to start on a data directory created by a different major version, so data/postgres cannot be carried across an image bump. Dump with the new version's client, recreate the cluster, restore:

# 1. with the OLD container still running, dump using the NEW client
docker run --rm -e PGPASSWORD=contrast_api_pass -v "$PWD/data:/out" postgres:18 \
  pg_dump -h host.docker.internal -p 5433 -U contrast_api_user -d contrast_api_db \
  --no-owner --no-privileges -Fc -f /out/pgdump-$(date +%Y%m%d).dump

# 2. keep the old cluster around until the new one is verified
docker compose down
mv data/postgres data/postgres-old

# 3. bump the image in docker-compose.yml, then let initdb build a new cluster
docker compose up -d db

# 4. restore
docker exec -i postgres_contrast_api pg_restore -U contrast_api_user \
  -d contrast_api_db --no-owner --no-privileges < data/pgdump-YYYYMMDD.dump

Rollback is mv data/postgres-old data/postgres plus reverting the image tag.

Two things that bite here:

• postgres:18 moved its default PGDATA from /var/lib/postgresql/data to /var/lib/postgresql/18/docker. docker-compose.yml pins PGDATA back to the old path so the ./data/postgres bind mount is still the real cluster. Without that pin the data silently lives in an anonymous volume.
• Dump with the newer pg_dump. A dump produced by pg_dump 18 contains SET transaction_timeout = 0;, which only exists in Postgres 17+; restoring such a dump into an older server errors on that line.

Local seed data (paradigms, atlas tags, measure types) is not reproducible from this repo — it is not in git and the load_*_data commands need CSVs that are gitignored. Take a dump before touching data/postgres.