From eb576fa08d0c5528292f0bb7aa9a3cb5145e3cf6 Mon Sep 17 00:00:00 2001 From: "F. Daniel Nicolalde" Date: Wed, 14 May 2025 17:14:55 -0500 Subject: [PATCH 1/6] Changing labkey name to LabKey --- docker/README.md | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/docker/README.md b/docker/README.md index d71c10987..e27f42706 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,18 +1,18 @@ # Creating and Using Docker Images -This folder contains a set of Docker images and a Docker Compose service definition to start and run a LabKey server like the one used at the WNPRC. Each of the subfolders corresponds to a particular service/image used in the Compose definition (e.g., `postgres/` contains configuration information for the PostgreSQL service), and the Gradle build file helps to build the custom images that do not come from any online Docker repository (such as LabKey and our own custom cron service). +This folder contains a set of folders with Dockerfiles and a Compose file which defining services to start and run a LabKey server like the one used at the WNPRC. Each of the subfolders corresponds to a particular service/image used in the Compose definition (e.g., `postgres/` contains configuration information for the PostgreSQL service), and the Gradle build file helps to build the custom images that do not come from any online Docker repository (such as LabKey and our own custom cron service). -Any service-specific configuration needs to be defined in a `.env` file in this directory, with a pre-built example file provided in `default.env`. All the variables in the `default.env` file has a prefix to the corresponding service (e.g. LK = labkey, PG = postgres) and they are all organized alphabetically to make it easier to group all variables that affect the different services. Before deploying the services with Compose, you will need to create this `.env` file (e.g., by copying and renaming `default.env`). +Any service-specific configuration needs to be defined in a `.env` file in this directory, with a pre-built example file provided in `default.env`. All the variables in the `default.env` file has a prefix to the corresponding service (e.g. LK = labkey, PG = postgres) and they are all organized alphabetically to make it easier to group all variables that affect the different services. Before deploying the services with Compose, you will need to create the `.env` file (e.g., by copying and renaming `default.env`). The following files need to be rename to use SSL certificates in your local development machine: `cert.pem.default` and `key.pem.default` both files have to be rename to remove the .default . The names have to match the names in `.env` file. ## Downloading Docker Images from Docker Hub -WNPRC maintains a service contract with Docker Hub. This allows the IDS unit to build images in this cloud service thus not requiring to locally build images in our production server, test environment and developer machines. The contract allows for five accounts to be associated with the WNPRCEHR Organization. The `idsshared` account can be used to download and access our private LabKey images (i.e. [labkeysnapshot](https://hub.docker.com/repository/docker/wnprcehr/labkeysnapshot/general) and [labkey](https://hub.docker.com/repository/docker/wnprcehr/labkey/general)), the token and password for that account can be found in `Keypass-IDS.kdbx` in the `wnprc.dirve.wisc.edu` shared folder. +WNPRC maintains a service contract with Docker Hub. This allows the IDS unit to build images in this cloud service thus not requiring to locally build images in our production server, test environment and developer machines. The contract allows for five accounts to be associated with the WNPRCEHR Organization. The `idsshared` account can be used to download and access our private LabKey images (i.e. [labkeysnapshot](https://hub.docker.com/repository/docker/wnprcehr/labkeysnapshot/general) and [labkey](https://hub.docker.com/repository/docker/wnprcehr/labkey/general)), the token and password for that account can be found in `Keypass-IDS.kdbx` in the `wnprc.drive.wisc.edu` shared folder. Another alternative is to login via the Docker CLI (`docker login`) with the shared username and password. Gradle tasks can login to Docker Hub without the need to type the password. -All the docker images can be downloaded from Docker Hub using the following commands, user has to be login into Docker Hub. It is best to use a token and/or a password saved on the user's home folder file called `~/.gradle/gradle.properties` by adding the following lines. +All the docker images can be downloaded from Docker Hub using the following commands, user has to be login into Docker Hub. It is best to use a token and/or a password saved on the user's home folder file called `~/.gradle/gradle.properties`, this is the same file used during the LabKey configuration. Add the following lines replacing the data inside brackets. ``` dockerhubUsername=idsshared @@ -20,7 +20,7 @@ dockerhubPassword= dockertokenpath= ``` -Gradle tasks to interact with Docker have two versions, one using a [plugin](https://github.com/bmuschko/gradle-docker-plugin) and the second one uses direct command line via the Docker CLI. All the tasks defined in the `build.gradle` file have the two flavors. Either of the following command downloads all the custom images manage by the IDS unit. +Gradle tasks to interact with Docker have two versions, one using a [plugin](https://github.com/bmuschko/gradle-docker-plugin) and the second one uses direct command line via the Docker CLI. Thus, all the tasks defined in the `build.gradle` file have two versions. Either of the following command downloads all the custom images manage by the IDS unit. ``` ./gradlew downloadAll @@ -29,11 +29,11 @@ Gradle tasks to interact with Docker have two versions, one using a [plugin](htt To download a specific images from a feature branch use the following commands replacing the Labkey version (i.e. XX.YY = 22.11) and the name of the branch inside the brackets: ``` -./gradlew downloadLabkey -PdockerString= -./gradlew dowloadLabkeyPlug -PdockerString= +./gradlew downloadLabkey -PbranchName= +./gradlew dowloadLabkeyPlug -PbranchName= -./gradlew downloadEhrcron -PdockerString= -./gradlew downloadEhrcronPlug -PdockerString= +./gradlew downloadEhrcron -PbranchName= +./gradlew downloadEhrcronPlug -PbranchName= ``` For a list of all the task use the follwowing commands: @@ -46,17 +46,20 @@ For a list of all the task use the follwowing commands: To build the custom images from a stand-alone clone, navigate to the **docker** folder (**not** the repository root) and execute the following command: ``` -./gradlew buildAll -PdockerString= +./gradlew buildAll -PbranchName= ``` From a clone embedded inside a LabKey platform source code, you will need to execute the command from the LabKey root, with the appropriate adjustments to the project path: ``` -./gradlew :externalModules:wnprc-modules:docker:buildall -PdockerString= +./gradlew :externalModules:wnprc-modules:docker:buildall -PbranchName= ``` -Each of the custom images has its own build task as well (e.g., `buildLabkey`, `buildEhrcron`) and all have corresponding tasks using the pluging (e.g. `buildEhrcronPlug`, `buildPostfixPlug`). The Labkey images depends on a hook (`docker/labkey/hooks/build`) which is used in Docker Hub to correctly interprete GitHub branches naming convencion. This same hook is used by the gradle task to download the correct LabKey installer and create the Docker image. This build tasks does not have a companion option using the plugin. +Each of the custom images has its own build task as well (e.g., `buildLabkey`, `buildEhrcron`) and all have corresponding tasks using the pluging (e.g. `buildEhrcronPlug`, `buildPostfixPlug`). The Labkey and ehrcranrnutils images depends on hooks (`~/hooks/build`) which is used in Docker Hub to correctly interprete GitHub branches naming convencion and build the image for the correct architecture (i.e., arm64 and adm64). This same hook is used by the gradle task to download the correct LabKey installer and create the Docker image. This build tasks does not have a companion option using the plugin version. + +For newer Apple Silicon all docker images can be build for ARM processors or as multi-platform builds by using Other than using Gradle, the images can each be built directly using Docker by executing a command like this: ``` docker build -t wnprcehr/ehrcron:vX.X.X ehrcron +docker build --builder container --platform linux/arm64 -t wnprcehr/cranrnutils:cranrnutils_24.11_livebackuptestserver --load cranrnutils ``` If changes are only committed to TeamCity or a new based LabKey build needs to be create, use --no-cache option. To build localy, you must obtain the URL to download the installer from TeamCity. The Dockerfile connect to TeamCity using a set of credentials and downloads the LabKey installer. ``` @@ -160,12 +163,15 @@ docker compose up -d postgres # for taking down just one of the services (e.g., postgres) docker compose stop postgres --timeout 60 +# for removing just one of the services this makes sure the system uses the latest version (e.g., postgres) +docker compose rm postgres --timeout 60 + # for accesing running services to inspect changes use the following commands (e.g., labkey or postgres) docker compose exec labkey /bin/bash ``` All other Docker Compose commands (`logs`, `ps`, etc.) work also. -*Note that sometimes the postgres container closes before the database itself is completely shut down. Be sure to disconnect your pgAdmin and IntelliJ database connections, if any, stop labkey, and then do a shutdown. Otherwise the next time postgres starts it will go into an automatic recovery mode and take a long time to start back up. By adding a timeout of 60 seconds it allows the database to close gracefully and avoid the recovery process. +*Note that sometimes the postgres container closes before the database itself is completely shut down. Be sure to disconnect your pgAdmin and IntelliJ database connections, if any, stop labkey, and then do a shutdown. Otherwise the next time postgres starts it will go into an automatic recovery mode and take a long time to start back up. By adding a timeout of 60 seconds it allows the database to close every connection, shutdown gracefully and avoid the recovery process. ## Docker setup in production EHR Running EHR in the production mode, requires different images for ehrcron and the **SNAPSHOT** version of LabKey. The verison of LabKey is controlled by the following variables stored in `.env` file: `LK_PROD`, `LK_VERSION` and `LK_FB`. These varaibles get replace in the following string `wnprcehr/labkey${LK_PROD}:$LK_VERSION${LK_FB}` during runtime. The string becomes `wnprcehr/labkeysnapshot:24.11` which are the tags in this [repo](https://hub.docker.com/repository/docker/wnprcehr/labkeysnapshot/tags). The version of ehrcron image is also controlled by the `.env` file. The variables `PERL_PROD`, `LK_VERSION` and `LK_FB` are used to modify the name of the image defined in the `production.yaml` file from `wnprcehr/ehrcron$PERL_PROD:$LK_VERSION${LK_FB}` to `wnprcehr/ehrcronprod:24.11`. These are the tags defined in this [repo](https://hub.docker.com/repository/docker/wnprcehr/ehrcronprod/general). The ehrcronprod image has the following scheduled jobs: @@ -176,17 +182,17 @@ Running EHR in the production mode, requires different images for ehrcron and t ## Running multiple instances of LabKey in same Server -We created a folder called `development` in this repo. This folder contains a simplify version of the main `compose.yaml` file. It only has two services: labkey and ngnix. To start a secondary version of labkey in the test server. Copy the development folder, and rename it to particular project. Within the new folder, you have to edit three of files: +We created a folder called `development` in this repo. This folder contains a simplify version of the main `compose.yaml` file. It only has two services: labkey and ngnix. To start a secondary version of labkey in the test server. Copy the development folder, and rename it to particular project. Within the new folder, you have to edit three files: 1. `.env` 1. `nginx/nginx.conf` 1. `compose.yaml` -In the `.env` file, edit the following variables: `LK_DANGER_PORT` to other number than 8080, this is the port which Labkey service will use outside the Docker container. `LK_SECURE_PORT` this port is the one user will need to add to the test server URL to access your instance of LabKey (e.g. https://.primate.wisc.edu:8443). List of ports and databases used for each instance of LabKey in the test-server can be found in this private page: [Test_Servers](https://github.com/WNPRC-EHR-Services/EHR_Documentation/blob/master/sop/Test_Servers.md). Update the list once your instance is up and running. `LK_BASE_URL` to a unique name for your new labkey service, it has to match the name you will modify in the `compose.yaml` file. `PG_NAME` to a database you are planning to use with your new instances of LabKey. +In the `.env` file, edit the following variables: `LK_DANGER_PORT` to other number than 8080, this is the port which LabKey service will use outside the Docker container. `LK_SECURE_PORT` this port is the one users will need to add to the test server URL to access your instance of LabKey (e.g. https://.primate.wisc.edu:8443). List of ports and databases used for each instance of LabKey in the test-server can be found in this private page: [Test_Servers](https://github.com/WNPRC-EHR-Services/EHR_Documentation/blob/master/sop/Test_Servers.md). Update the list once your instance is up and running. `LK_BASE_URL` to a unique name for your new LabKey service, it has to match the name you will modify in the `compose.yaml` file. `PG_NAME` to a database you are planning to use with your new instances of LabKey. In the `ngnix.conf` file you need to edit the following: `proxy_pass` at the end of the file, to the name you have selected for your new service, it also has to match the name on your `compose.yaml` and `.env` files. -Finally, in your `compose.yaml` file edit the name of the labkey service, it should be unique, therefore check other development folder for all the names used. +Finally, in your `compose.yaml` file edit the name of the LabKey service, it should be unique, therefore check other development folder for all the names used. All the auxiliary LabKey instances can be manage via the manage_all_continers.sh script. This script accepts two values (-s || -d), s starts all the containers in the docker folder. Starts with the primary which contains postgres and than looks for any folder that starts with dev. From c4e921731fe153ed4ab6f4b1cfc178205e273c50 Mon Sep 17 00:00:00 2001 From: "F. Daniel Nicolalde" Date: Wed, 14 May 2025 17:15:38 -0500 Subject: [PATCH 2/6] Removing tomcat for cranrnutils --- docker/build.gradle | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docker/build.gradle b/docker/build.gradle index 0bdd08834..86e933a3e 100644 --- a/docker/build.gradle +++ b/docker/build.gradle @@ -98,7 +98,7 @@ tasks.register("buildEhrCronPlug", DockerBuildImage){ group 'DockerPlugin' } -tasks.register("buildTomcat") { Task t -> +tasks.register("buildCranr") { Task t -> doLast { if (!project.hasProperty("nocache")){ ext.nocache = "false" @@ -125,32 +125,32 @@ tasks.register("buildTomcat") { Task t -> if (nocache.equals("true")){ println "building image without cache" t.project.exec{ - workingDir "$projectDir/tomcat" + workingDir "$projectDir/cranrnutils" executable "docker" - args "build", ".", "--no-cache", "-t", "wnprcehr/tomcat:tomcat9_"+branch + args "build", "--platform","linux/arm64,linux/amd64" ,"--no-cache", "-t", "wnprcehr/cranrnutils:cranrnutils_"+branch, "." } }else{ t.project.exec{ - workingDir "$projectDir/tomcat" + workingDir "$projectDir/cranrnutils" executable "docker" - args "build", ".", "-t", "wnprcehr/tomcat:tomcat9_"+branch + args "build", "--platform","linux/arm64,linux/amd64" ,"-t", "wnprcehr/cranrnutils:cranrnutils_"+branch, "." } } }else { - logger.warn 'Must provide docker string via command line. (ex. ./gradlew buildTomcat -PbranchName=22.11_fb)' + logger.warn 'Must provide docker string via command line. (ex. ./gradlew buildCranr -PbranchName=22.11_fb)' } } } -configure(buildTomcat){ +configure(buildCranr){ group 'Docker' description 'Generates the Tomcat docker image (with R)' dependsOn "checkDocker", "dockerLogin", "processbranchName" onlyIf {buildType != ''} } -tasks.register("buildTomcatPlug", DockerBuildImage){ +tasks.register("buildCranrPlug", DockerBuildImage){ def dockerStr = providers.gradleProperty("branchName") if (dockerStr.present){ // String branch = "" @@ -169,13 +169,13 @@ tasks.register("buildTomcatPlug", DockerBuildImage){ break; } inputDir.set(file("tomcat/")) - images.add("wnprcehr/tomcat:tomcat9_${branch}") + images.add("wnprcehr/cranrnutils:cranrnutils_${branch}") }else { - logger.warn 'Must provide docker string via command line. (ex. ./gradlew buildTomcatPlug -PbranchName=22.11_fb)' + logger.warn 'Must provide docker string via command line. (ex. ./gradlew buildCranrPlug -PbranchName=22.11_fb)' } } -configure(buildTomcatPlug){ +configure(buildCranrPlug){ group 'DockerPlugin' description 'Generates the Tomcat docker image (with R)' dependsOn "checkDocker", "dockerLogin", "processbranchName" @@ -184,7 +184,7 @@ configure(buildTomcatPlug){ //Building LabKey with hooks/build for consistancy with Docker Hub and used of environment variables duirng the build process //Source: https://stackoverflow.com/questions/36322536/how-to-set-an-environment-variable-from-a-gradle-build / https://stackoverflow.com/a/63140816 tasks.register("buildLabkey") { Task t -> -//task("buildLabkey", group: "Docker", description: "Generates the LabKey docker image", dependsOn: ["buildTomcat", "buildEhrCron", "buildPostfix"]) { Task t -> +//task("buildLabkey", group: "Docker", description: "Generates the LabKey docker image", dependsOn: ["buildCranr", "buildEhrCron", "buildPostfix"]) { Task t -> doLast { def dockerStr = providers.gradleProperty("branchName") @@ -317,7 +317,7 @@ configure(tagPostfixPlug){ tasks.register("buildAll"){ group 'Docker' description 'Generates all the docker images in the subfolders' - dependsOn "buildTomcat", "buildEhrcron", "buildLabkey", "buildPostfix" + dependsOn "buildCranr", "buildEhrcron", "buildLabkey", "buildPostfix" } From f12206b5fa91a85b484bbdd8f865c28229dfcfd7 Mon Sep 17 00:00:00 2001 From: "F. Daniel Nicolalde" Date: Tue, 20 May 2025 14:13:40 -0500 Subject: [PATCH 3/6] Updating documentation of Docker --- docker/README.md | 42 ++++++++++++++++++++++++++---------------- 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/docker/README.md b/docker/README.md index e27f42706..e55e48660 100644 --- a/docker/README.md +++ b/docker/README.md @@ -2,32 +2,35 @@ This folder contains a set of folders with Dockerfiles and a Compose file which defining services to start and run a LabKey server like the one used at the WNPRC. Each of the subfolders corresponds to a particular service/image used in the Compose definition (e.g., `postgres/` contains configuration information for the PostgreSQL service), and the Gradle build file helps to build the custom images that do not come from any online Docker repository (such as LabKey and our own custom cron service). -Any service-specific configuration needs to be defined in a `.env` file in this directory, with a pre-built example file provided in `default.env`. All the variables in the `default.env` file has a prefix to the corresponding service (e.g. LK = labkey, PG = postgres) and they are all organized alphabetically to make it easier to group all variables that affect the different services. Before deploying the services with Compose, you will need to create the `.env` file (e.g., by copying and renaming `default.env`). +Any service-specific configuration needs to be defined in a `.env` file in this directory, with a pre-built example file provided in `default.env`. All the variables in the `default.env` file has a prefix to the corresponding service (e.g. LK = LabKey, PG = postgres) and they are all organized alphabetically to make it easier to group all variables that affect the different services. Before deploying the services with Compose, you will need to create the `.env` file (e.g., by copying and renaming `default.env`). The following files need to be rename to use SSL certificates in your local development machine: `cert.pem.default` and `key.pem.default` both files have to be rename to remove the .default . The names have to match the names in `.env` file. ## Downloading Docker Images from Docker Hub -WNPRC maintains a service contract with Docker Hub. This allows the IDS unit to build images in this cloud service thus not requiring to locally build images in our production server, test environment and developer machines. The contract allows for five accounts to be associated with the WNPRCEHR Organization. The `idsshared` account can be used to download and access our private LabKey images (i.e. [labkeysnapshot](https://hub.docker.com/repository/docker/wnprcehr/labkeysnapshot/general) and [labkey](https://hub.docker.com/repository/docker/wnprcehr/labkey/general)), the token and password for that account can be found in `Keypass-IDS.kdbx` in the `wnprc.drive.wisc.edu` shared folder. +WNPRC maintains a service contract with Docker Hub. This contract allows the IDS unit to build images in this cloud service thus not requiring to locally build images in our production server, test environment and developer machines. The contract allows for five accounts to be associated with the WNPRCEHR Organization. The `idsshared` account can be used to download and access our private LabKey images (i.e. [labkeysnapshot](https://hub.docker.com/repository/docker/wnprcehr/labkeysnapshot/general) and [labkey](https://hub.docker.com/repository/docker/wnprcehr/labkey/general)), the token and password for that account can be found in `Keypass-IDS.kdbx` in the `wnprc.drive.wisc.edu` shared folder. -Another alternative is to login via the Docker CLI (`docker login`) with the shared username and password. Gradle tasks can login to Docker Hub without the need to type the password. - -All the docker images can be downloaded from Docker Hub using the following commands, user has to be login into Docker Hub. It is best to use a token and/or a password saved on the user's home folder file called `~/.gradle/gradle.properties`, this is the same file used during the LabKey configuration. Add the following lines replacing the data inside brackets. +Another alternative is to login via the Docker CLI (`docker login`) with the shared username and password. Gradle tasks can login to Docker Hub without the need to type the password but the credentials need to be stored in the gradle.properties. t is best to use a token and/or a password saved on the user's home folder file called `~/.gradle/gradle.properties`, this is the same file used during the LabKey development setup. Add the following lines replacing the data inside brackets. ``` dockerhubUsername=idsshared dockerhubPassword= dockertokenpath= ``` +For a list of all the task use the following commands: + +``` +./gradlew tasks +``` -Gradle tasks to interact with Docker have two versions, one using a [plugin](https://github.com/bmuschko/gradle-docker-plugin) and the second one uses direct command line via the Docker CLI. Thus, all the tasks defined in the `build.gradle` file have two versions. Either of the following command downloads all the custom images manage by the IDS unit. +Docker images can be downloaded from Docker Hub using the following commands, user has to be login into Docker Hub as explain in the previous chapter. All Gradle tasks to interact with Docker engine locally have two versions, one using a [plugin](https://github.com/bmuschko/gradle-docker-plugin) and the second one uses direct command line via the Docker CLI. Thus, all the tasks defined in the `build.gradle` file have two versions. Either of the following command downloads all the custom images manage by the IDS unit. ``` ./gradlew downloadAll ./gradlew downloadAllPlug ``` -To download a specific images from a feature branch use the following commands replacing the Labkey version (i.e. XX.YY = 22.11) and the name of the branch inside the brackets: +To download a specific images from a feature branch use the following commands replacing the Labkey version (i.e. XX.YY = 24.11) and the name of the branch inside the brackets: ``` ./gradlew downloadLabkey -PbranchName= ./gradlew dowloadLabkeyPlug -PbranchName= @@ -36,25 +39,22 @@ To download a specific images from a feature branch use the following commands r ./gradlew downloadEhrcronPlug -PbranchName= ``` -For a list of all the task use the follwowing commands: - -``` -./gradlew tasks -``` - ## Building the Custom Images To build the custom images from a stand-alone clone, navigate to the **docker** folder (**not** the repository root) and execute the following command: ``` ./gradlew buildAll -PbranchName= ``` -From a clone embedded inside a LabKey platform source code, you will need to execute the command from the LabKey root, with the appropriate adjustments to the project path: +From a clone embedded inside a LabKey development setup with all the source code, you will need to execute the command from the LabKey root, with the appropriate adjustments to the project path: ``` ./gradlew :externalModules:wnprc-modules:docker:buildall -PbranchName= ``` -Each of the custom images has its own build task as well (e.g., `buildLabkey`, `buildEhrcron`) and all have corresponding tasks using the pluging (e.g. `buildEhrcronPlug`, `buildPostfixPlug`). The Labkey and ehrcranrnutils images depends on hooks (`~/hooks/build`) which is used in Docker Hub to correctly interprete GitHub branches naming convencion and build the image for the correct architecture (i.e., arm64 and adm64). This same hook is used by the gradle task to download the correct LabKey installer and create the Docker image. This build tasks does not have a companion option using the plugin version. +Each of the custom images has its own build task as well (e.g., `buildLabkey`, `buildEhrcron`) and all have corresponding tasks using the pluging (e.g. `buildEhrcronPlug`, `buildPostfixPlug`). The Labkey and ehrcranrnutils images depends on hooks (`~/hooks/build`) which is used in Docker Hub to correctly interprete GitHub branches naming convencion and build the image for the correct architecture (i.e., arm64 and adm64). This same hook is used by the gradle task to download the correct LabKey installer from TeamCity and create the corresponding Docker image. These build tasks does not have a companion option using the plugin version. -For newer Apple Silicon all docker images can be build for ARM processors or as multi-platform builds by using +For newer Apple Silicon all docker images can be build for ARM processors or as multi-platform builds by using: +``` +--platform linux/arm64,linux/amd64 +``` Other than using Gradle, the images can each be built directly using Docker by executing a command like this: ``` @@ -217,6 +217,16 @@ The script has very few options, as shown in these examples: ``` The use of the `-p` flag allows us to use this script to manage multiple instances of the LabKey PostgreSQL container on the same server, provided that each instance is run from its own folder with its own .env file (to specify ports, data file locations, etc.) +## Configuration of nightly-ehr.primate.wisc.edu + +This server is configured to update every night after the production server completes a complete backup and move the created file to a long term ITSS storage (i.e., `PrimateFS`). The script called `load_database_update_testserver.sh` is based on `load_database_backup.sh` and it is configured to run as a cron job in the `nightly-ehr.primate.wisc.edu` server by the root user. + +To check the current configuraion type: `sudo crontab -l`. To modify the configuration type: `sudo crontab -e`. + +The script uses multiple parameters: `-postgres` - location of postgres executable (i.e., /usr/lib/postgresql/15/bin/), `--dbname` - name of the databse to replace, `--jobs` - number of processes to run the backup, `--production` - restore a complete database, `--path` - location of the backup files (~/labkey_backup/database/daily/). + +The script also downloads the latest image of LabKeySnapshot from Docker Hub and cleans all the old images from the local image repository. + ## Additional Configurations In some instance, the shared memory and effective cache size should be modified for dev machines. In the docker/postgres/postgresql.cong file modify line shared_buffers and effective_cache_size to 1024MB and 2048MB respectively. From d5c9403e92224f06819a89f5a154ba3178f3bcf7 Mon Sep 17 00:00:00 2001 From: aschmidt34 <124093649+aschmidt34@users.noreply.github.com> Date: Tue, 20 May 2025 15:44:23 -0500 Subject: [PATCH 4/6] Update README.md --- docker/README.md | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/docker/README.md b/docker/README.md index e55e48660..75a6f4e2c 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,8 +1,8 @@ # Creating and Using Docker Images -This folder contains a set of folders with Dockerfiles and a Compose file which defining services to start and run a LabKey server like the one used at the WNPRC. Each of the subfolders corresponds to a particular service/image used in the Compose definition (e.g., `postgres/` contains configuration information for the PostgreSQL service), and the Gradle build file helps to build the custom images that do not come from any online Docker repository (such as LabKey and our own custom cron service). +This folder contains a set of folders with Dockerfiles and a Compose file which define services to start and run a LabKey server like the one used at the WNPRC. Each of the subfolders corresponds to a particular service/image used in the Compose definition (e.g., `postgres/` contains configuration information for the PostgreSQL service), and the Gradle build file helps to build the custom images that do not come from any online Docker repository (such as LabKey and our own custom cron service). -Any service-specific configuration needs to be defined in a `.env` file in this directory, with a pre-built example file provided in `default.env`. All the variables in the `default.env` file has a prefix to the corresponding service (e.g. LK = LabKey, PG = postgres) and they are all organized alphabetically to make it easier to group all variables that affect the different services. Before deploying the services with Compose, you will need to create the `.env` file (e.g., by copying and renaming `default.env`). +Any service-specific configuration needs to be defined in a `.env` file in this directory, with a pre-built example file provided in `default.env`. All the variables in the `default.env` file have a prefix to the corresponding service (e.g. LK = LabKey, PG = postgres) and they are all organized alphabetically to make it easier to group all variables that affect the different services. Before deploying the services with Compose, you will need to create the `.env` file (e.g., by copying and renaming `default.env`). The following files need to be rename to use SSL certificates in your local development machine: `cert.pem.default` and `key.pem.default` both files have to be rename to remove the .default . The names have to match the names in `.env` file. @@ -10,7 +10,7 @@ The following files need to be rename to use SSL certificates in your local deve WNPRC maintains a service contract with Docker Hub. This contract allows the IDS unit to build images in this cloud service thus not requiring to locally build images in our production server, test environment and developer machines. The contract allows for five accounts to be associated with the WNPRCEHR Organization. The `idsshared` account can be used to download and access our private LabKey images (i.e. [labkeysnapshot](https://hub.docker.com/repository/docker/wnprcehr/labkeysnapshot/general) and [labkey](https://hub.docker.com/repository/docker/wnprcehr/labkey/general)), the token and password for that account can be found in `Keypass-IDS.kdbx` in the `wnprc.drive.wisc.edu` shared folder. -Another alternative is to login via the Docker CLI (`docker login`) with the shared username and password. Gradle tasks can login to Docker Hub without the need to type the password but the credentials need to be stored in the gradle.properties. t is best to use a token and/or a password saved on the user's home folder file called `~/.gradle/gradle.properties`, this is the same file used during the LabKey development setup. Add the following lines replacing the data inside brackets. +Another alternative is to login via the Docker CLI (`docker login`) with the shared username and password. Gradle tasks can login to Docker Hub without the need to type the password but the credentials need to be stored in the gradle.properties. It is best to use a token and/or a password saved on the user's home folder file called `~/.gradle/gradle.properties`, this is the same file used during the LabKey development setup. Add the following lines replacing the data inside brackets. ``` dockerhubUsername=idsshared @@ -23,14 +23,14 @@ For a list of all the task use the following commands: ./gradlew tasks ``` -Docker images can be downloaded from Docker Hub using the following commands, user has to be login into Docker Hub as explain in the previous chapter. All Gradle tasks to interact with Docker engine locally have two versions, one using a [plugin](https://github.com/bmuschko/gradle-docker-plugin) and the second one uses direct command line via the Docker CLI. Thus, all the tasks defined in the `build.gradle` file have two versions. Either of the following command downloads all the custom images manage by the IDS unit. +Docker images can be downloaded from Docker Hub using the following commands, user has to be logged into Docker Hub as explained in the previous chapter. All Gradle tasks to interact with the Docker engine locally have two versions, one using a [plugin](https://github.com/bmuschko/gradle-docker-plugin) and the second one uses direct command line via the Docker CLI. Thus, all the tasks defined in the `build.gradle` file have two versions. Either of the following commands download all the custom images managed by the IDS unit. ``` ./gradlew downloadAll ./gradlew downloadAllPlug ``` -To download a specific images from a feature branch use the following commands replacing the Labkey version (i.e. XX.YY = 24.11) and the name of the branch inside the brackets: +To download a specific image from a feature branch use the following commands replacing the Labkey version (i.e. XX.YY = 24.11) and the name of the branch inside the brackets: ``` ./gradlew downloadLabkey -PbranchName= ./gradlew dowloadLabkeyPlug -PbranchName= @@ -49,9 +49,9 @@ From a clone embedded inside a LabKey development setup with all the source code ``` ./gradlew :externalModules:wnprc-modules:docker:buildall -PbranchName= ``` -Each of the custom images has its own build task as well (e.g., `buildLabkey`, `buildEhrcron`) and all have corresponding tasks using the pluging (e.g. `buildEhrcronPlug`, `buildPostfixPlug`). The Labkey and ehrcranrnutils images depends on hooks (`~/hooks/build`) which is used in Docker Hub to correctly interprete GitHub branches naming convencion and build the image for the correct architecture (i.e., arm64 and adm64). This same hook is used by the gradle task to download the correct LabKey installer from TeamCity and create the corresponding Docker image. These build tasks does not have a companion option using the plugin version. +Each of the custom images has its own build task as well (e.g., `buildLabkey`, `buildEhrcron`) and all have corresponding tasks using the pluging (e.g. `buildEhrcronPlug`, `buildPostfixPlug`). The Labkey and ehrcranrnutils images depend on hooks (`~/hooks/build`) which is used in Docker Hub to correctly interpret GitHub branches naming convencion and build the image for the correct architecture (i.e., arm64 and adm64). This same hook is used by the gradle task to download the correct LabKey installer from TeamCity and create the corresponding Docker image. These build tasks does not have a companion option using the plugin version. -For newer Apple Silicon all docker images can be build for ARM processors or as multi-platform builds by using: +For newer Apple Silicon all docker images can be built for ARM processors or as multi-platform builds by using: ``` --platform linux/arm64,linux/amd64 ``` @@ -61,7 +61,7 @@ Other than using Gradle, the images can each be built directly using Docker by e docker build -t wnprcehr/ehrcron:vX.X.X ehrcron docker build --builder container --platform linux/arm64 -t wnprcehr/cranrnutils:cranrnutils_24.11_livebackuptestserver --load cranrnutils ``` -If changes are only committed to TeamCity or a new based LabKey build needs to be create, use --no-cache option. To build localy, you must obtain the URL to download the installer from TeamCity. The Dockerfile connect to TeamCity using a set of credentials and downloads the LabKey installer. +If changes are only committed to TeamCity or a new based LabKey build needs to be created, use --no-cache option. To build localy, you must obtain the URL to download the installer from TeamCity. The Dockerfile connects to TeamCity using a set of credentials and downloads the LabKey installer. ``` docker build --build-arg LABKEY_TEAMCITY_USERNAME= --build-arg LABKEY_TEAMCITY_PASSWORD= --build-arg TEAMCITY_URL= --build-arg TOMCAT_IMAGE= --build-arg LK_VERSION= --no-cache --rm=true -t wnprcehr/labkey:XX.YY labkey ``` @@ -93,7 +93,7 @@ docker build \ --build-arg LABKEY_TEAMCITY_PASSWORD= \ -t wnprcehr/labkey:XX.X labkey ``` -If you want to build an images for an specific branch within Github, you should pass one additional argument `--build-arg TOMCAT_IMAGE`. Your commands will look something like this, use the name of the branch without the fb prefix, the name should match as how TeamCity creates the image: +If you want to build an image for a specific branch within Github, you should pass one additional argument `--build-arg TOMCAT_IMAGE`. Your commands will look something like this, use the name of the branch without the fb prefix, the name should match as how TeamCity creates the image: ``` docker build \ --build-arg LABKEY_TEAMCITY_USERNAME= \ @@ -102,7 +102,7 @@ docker build \ -t wnprcehr/labkeyDev:XX.X labkey ``` -The LabKey image depends on the Tomcat image, which can be dowload from Docker Hub or build locally. This image takes a long time to build from scratch, it is best to download it from Docker Hub. Here are the commands to download or build this image. +The LabKey image depends on the Tomcat image, which can be dowloaded from Docker Hub or built locally. This image takes a long time to build from scratch, it is best to download it from Docker Hub. Here are the commands to download or build this image. ``` ./gradlew downloadTomcat -PbranchName= ./gradlew downloadTomcatPlug -PbranchName= @@ -115,9 +115,9 @@ docker build --no-cache -t wnprcehr/tomcat:tomcat9_ tomcat ## Deploying the Docker Compose Services -There are several services controlled by the compose.yaml and production.yaml files. Spliting the Docker services in these two files allows to use the same GitHub repository in two different server without having to make changes locally except for changes in the `.env` file. +There are several services controlled by the compose.yaml and production.yaml files. Splitting the Docker services in these two files allows us to use the same GitHub repository in two different servers without having to make changes locally except for changes in the `.env` file. -The Docker services are production EHR, nightly-ehr and test servers ran are the follwoing: +The Docker services ran in production EHR, nightly-ehr and test servers are the following: ||Service|Functionality|YAML File|Repository| |---|---|---|---|---| |1|postgres|database|compose.yaml|[postgres](https://hub.docker.com/_/postgres)| @@ -140,7 +140,7 @@ To deploy the services, you again either use Gradle or use Docker Compose direct # for tearing down all the services ./gradlew :docker:down ``` -To use Docker Compose, you can execute commands like the following (*from this directory*, where your `.env` file is located), this commands will work on production server as well as other servers: +To use Docker Compose, you can execute commands like the following (*from this directory*, where your `.env` file is located), these commands will work on the production server as well as the other servers: ``` # for spinning up all the services in production server docker compose -f compose.yaml -f production.yaml up -d @@ -149,7 +149,7 @@ docker compose -f compose.yaml -f production.yaml up -d docker compose -f compose.yaml -f production.yaml down --timeout 60 ``` -Add `-f compose.yaml -f production.yaml` to make changes in the production server. If this is not added the system will provide a warning that there is an orphan services running. +Add `-f compose.yaml -f production.yaml` to make changes in the production server. If this is not added the system will provide a warning that there are orphan services running. ``` # for spinning up all the services docker compose up -d @@ -188,17 +188,17 @@ We created a folder called `development` in this repo. This folder contains a si 1. `nginx/nginx.conf` 1. `compose.yaml` -In the `.env` file, edit the following variables: `LK_DANGER_PORT` to other number than 8080, this is the port which LabKey service will use outside the Docker container. `LK_SECURE_PORT` this port is the one users will need to add to the test server URL to access your instance of LabKey (e.g. https://.primate.wisc.edu:8443). List of ports and databases used for each instance of LabKey in the test-server can be found in this private page: [Test_Servers](https://github.com/WNPRC-EHR-Services/EHR_Documentation/blob/master/sop/Test_Servers.md). Update the list once your instance is up and running. `LK_BASE_URL` to a unique name for your new LabKey service, it has to match the name you will modify in the `compose.yaml` file. `PG_NAME` to a database you are planning to use with your new instances of LabKey. +In the `.env` file, edit the following variables: `LK_DANGER_PORT` to a number other than 8080, this is the port which LabKey service will use outside the Docker container. `LK_SECURE_PORT` this port is the one users will need to add to the test server URL to access your instance of LabKey (e.g. https://.primate.wisc.edu:8443). List of ports and databases used for each instance of LabKey in the test-server can be found in this private page: [Test_Servers](https://github.com/WNPRC-EHR-Services/EHR_Documentation/blob/master/sop/Test_Servers.md). Update the list once your instance is up and running. `LK_BASE_URL` to a unique name for your new LabKey service, it has to match the name you will modify in the `compose.yaml` file. `PG_NAME` to a database you are planning to use with your new instances of LabKey. In the `ngnix.conf` file you need to edit the following: `proxy_pass` at the end of the file, to the name you have selected for your new service, it also has to match the name on your `compose.yaml` and `.env` files. -Finally, in your `compose.yaml` file edit the name of the LabKey service, it should be unique, therefore check other development folder for all the names used. +Finally, in your `compose.yaml` file edit the name of the LabKey service, it should be unique, therefore check other development folders for all the names used. -All the auxiliary LabKey instances can be manage via the manage_all_continers.sh script. This script accepts two values (-s || -d), s starts all the containers in the docker folder. Starts with the primary which contains postgres and than looks for any folder that starts with dev. +All the auxiliary LabKey instances can be managed via the manage_all_continers.sh script. This script accepts two values (-s || -d), s starts all the containers in the docker folder. Starts with the primary which contains postgres and than looks for any folder that starts with dev. ## Loading a Database Backup Using the Script -Along with the Docker-specific utilities in this folder, there is a (Bash-only) script to restore a database backup into a local Docker container: **load_database_backup.sh**. By default, this script will download the latest backup from the production server (assumed to have been created the same day at 1AM) and restore that backup into a PostgreSQL container as defined in the docker-compose.yml and .env files in this folder. Depending on resource on local machine or server, it is possible to increase the number of processors for the restore process. Change the number in line 132 right after -j option, by default is set to 4 processes. +Along with the Docker-specific utilities in this folder, there is a (Bash-only) script to restore a database backup into a local Docker container: **load_database_backup.sh**. By default, this script will download the latest backup from the production server (assumed to have been created the same day at 1AM) and restore that backup into a PostgreSQL container as defined in the docker-compose.yml and .env files in this folder. Depending on the resources on the local machine or server, it is possible to increase the number of processors for the restore process. Change the number in line 132 right after -j option, which by default is set to 4 processes. The script has very few options, as shown in these examples: ```bash @@ -219,11 +219,11 @@ The use of the `-p` flag allows us to use this script to manage multiple instanc ## Configuration of nightly-ehr.primate.wisc.edu -This server is configured to update every night after the production server completes a complete backup and move the created file to a long term ITSS storage (i.e., `PrimateFS`). The script called `load_database_update_testserver.sh` is based on `load_database_backup.sh` and it is configured to run as a cron job in the `nightly-ehr.primate.wisc.edu` server by the root user. +This server is configured to update every night after the production server completes a complete backup and moves the created file to a long term ITSS storage (i.e., `PrimateFS`). The script called `load_database_update_testserver.sh` is based on `load_database_backup.sh` and it is configured to run as a cron job in the `nightly-ehr.primate.wisc.edu` server by the root user. To check the current configuraion type: `sudo crontab -l`. To modify the configuration type: `sudo crontab -e`. -The script uses multiple parameters: `-postgres` - location of postgres executable (i.e., /usr/lib/postgresql/15/bin/), `--dbname` - name of the databse to replace, `--jobs` - number of processes to run the backup, `--production` - restore a complete database, `--path` - location of the backup files (~/labkey_backup/database/daily/). +The script uses multiple parameters: `-postgres` - location of postgres executable (i.e., /usr/lib/postgresql/15/bin/), `--dbname` - name of the database to replace, `--jobs` - number of processes to run the backup, `--production` - restore a complete database, `--path` - location of the backup files (~/labkey_backup/database/daily/). The script also downloads the latest image of LabKeySnapshot from Docker Hub and cleans all the old images from the local image repository. From 4726f567b7e8ba216154d3b9f42ff24d409540e5 Mon Sep 17 00:00:00 2001 From: "F. Daniel Nicolalde" Date: Mon, 28 Jul 2025 12:18:54 -0500 Subject: [PATCH 5/6] removing text for tomcat in the gradle build --- docker/build.gradle | 12 ++++++------ docker/labkey/Dockerfile | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docker/build.gradle b/docker/build.gradle index 86e933a3e..d31ae1a5c 100644 --- a/docker/build.gradle +++ b/docker/build.gradle @@ -445,7 +445,7 @@ tasks.register("downloadPostfixPlug",DockerPullImage){ dependsOn 'checkDocker' } -tasks.register("downloadTomcat"){ +tasks.register("downloadCranr"){ doLast { def dockerStr = providers.gradleProperty("branchName") if (dockerStr.present) @@ -454,11 +454,11 @@ tasks.register("downloadTomcat"){ String branchName = dockerStr.get() t.project.exec { executable "docker" - args "pull", "wnprcehr/tomcat:tomcat9_${branchName}" + args "pull", "wnprcehr/cranrnutils:cranrnutils_${branch}" } } else{ - logger.warn 'Must provide docker string via command line. (ex. ./gradlew downloadTomcat -PbranchName=22.11_fb)' + logger.warn 'Must provide docker string via command line. (ex. ./gradlew downloadCranr -PbranchName=22.11_fb)' } } group 'Docker' @@ -467,8 +467,8 @@ tasks.register("downloadTomcat"){ } -tasks.register("downloadTomcatPlug",DockerPullImage){ - image = "wnprcehr/tomcat:tomcat9_${branchName}" +tasks.register("downloadCranrPlug",DockerPullImage){ + image = "wnprcehr/cranrnutils:cranrnutils_${branch}" onComplete{ def dockerStr = providers.gradleProperty("branchName") @@ -478,7 +478,7 @@ tasks.register("downloadTomcatPlug",DockerPullImage){ } else{ - logger.warn 'Must provide docker string via command line. (ex. ./gradlew downloadTomcatPlug -PbranchName=22.11_fb)' + logger.warn 'Must provide docker string via command line. (ex. ./gradlew downloadCranrPlug -PbranchName=22.11_fb)' } } doLast{ diff --git a/docker/labkey/Dockerfile b/docker/labkey/Dockerfile index eea97b456..bdf136625 100644 --- a/docker/labkey/Dockerfile +++ b/docker/labkey/Dockerfile @@ -22,7 +22,7 @@ RUN echo -e "Downloading LabKey build from \033[1;33m${Z}\033[0m" \ # ----------------------------------------------------------------------------- # MAIN IMAGE BUILD DEFINITION -FROM --platform=$BUILDPLATFORM wnprcehr/cranrnutils:cranrnutils_${LK_VERSION}${FB_NAME} +FROM wnprcehr/cranrnutils:cranrnutils_${LK_VERSION}${FB_NAME} # creating folders for LabKey installation RUN mkdir -p /labkey/labkey From 59ac0f8edd4127d26b28f0f9666536968218cbbc Mon Sep 17 00:00:00 2001 From: "F. Daniel Nicolalde" Date: Mon, 28 Jul 2025 12:19:34 -0500 Subject: [PATCH 6/6] improving documention for mulitplatform builds --- docker/README.md | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/docker/README.md b/docker/README.md index 75a6f4e2c..d32aca3ba 100644 --- a/docker/README.md +++ b/docker/README.md @@ -51,15 +51,20 @@ From a clone embedded inside a LabKey development setup with all the source code ``` Each of the custom images has its own build task as well (e.g., `buildLabkey`, `buildEhrcron`) and all have corresponding tasks using the pluging (e.g. `buildEhrcronPlug`, `buildPostfixPlug`). The Labkey and ehrcranrnutils images depend on hooks (`~/hooks/build`) which is used in Docker Hub to correctly interpret GitHub branches naming convencion and build the image for the correct architecture (i.e., arm64 and adm64). This same hook is used by the gradle task to download the correct LabKey installer from TeamCity and create the corresponding Docker image. These build tasks does not have a companion option using the plugin version. -For newer Apple Silicon all docker images can be built for ARM processors or as multi-platform builds by using: +For newer Apple Silicon all docker images can be built for ARM processors or as multi-platform builds by using the platform argument. Before building a multi-platform version a new builder has to be created. ``` +docker buildx create --name=container +``` +This builder uses an emulator to create images for AMD processor. In the Apple Silicon the build process for AMD images take longer therefore is best to just use arm64 or the default builder. +``` +--platform linux/arm64 --platform linux/arm64,linux/amd64 ``` Other than using Gradle, the images can each be built directly using Docker by executing a command like this: ``` docker build -t wnprcehr/ehrcron:vX.X.X ehrcron -docker build --builder container --platform linux/arm64 -t wnprcehr/cranrnutils:cranrnutils_24.11_livebackuptestserver --load cranrnutils +docker build --builder container --platform linux/arm64 -t wnprcehr/cranrnutils:cranrnutils_YY.MM_featureBranch --load cranrnutils ``` If changes are only committed to TeamCity or a new based LabKey build needs to be created, use --no-cache option. To build localy, you must obtain the URL to download the installer from TeamCity. The Dockerfile connects to TeamCity using a set of credentials and downloads the LabKey installer. ``` @@ -88,10 +93,14 @@ labkeyTeamcityPassword= To build using Docker directly, you will need to pass those same credentials as run-time build arguments on the command line: ``` -docker build \ - --build-arg LABKEY_TEAMCITY_USERNAME= \ - --build-arg LABKEY_TEAMCITY_PASSWORD= \ - -t wnprcehr/labkey:XX.X labkey +docker build \ +--builder container --platform linux/arm64,linux/amd64 \ +--build-arg LABKEY_TEAMCITY_USERNAME= \ +--build-arg LABKEY_TEAMCITY_PASSWORD= \ +--build-arg TEAMCITY_URL= \ +--build-arg FB_NAME= \ +--build-arg LK_VERSION= --no-cache --rm=true --load \ +-t wnprcehr/labkey:YY.MM_Feature_Branch labkey ``` If you want to build an image for a specific branch within Github, you should pass one additional argument `--build-arg TOMCAT_IMAGE`. Your commands will look something like this, use the name of the branch without the fb prefix, the name should match as how TeamCity creates the image: ``` @@ -104,11 +113,11 @@ docker build \ The LabKey image depends on the Tomcat image, which can be dowloaded from Docker Hub or built locally. This image takes a long time to build from scratch, it is best to download it from Docker Hub. Here are the commands to download or build this image. ``` -./gradlew downloadTomcat -PbranchName= +./gradlew downloadCranr -PbranchName= ./gradlew downloadTomcatPlug -PbranchName= -./gradlew buildTomcat -PbranchName= -./gradlew buildTomcatPlug -PbranchName= +./gradlew buildCranr -PbranchName= +./gradlew buildCranrPlug -PbranchName= docker build --no-cache -t wnprcehr/tomcat:tomcat9_ tomcat ``` @@ -194,7 +203,7 @@ In the `ngnix.conf` file you need to edit the following: `proxy_pass` at the end Finally, in your `compose.yaml` file edit the name of the LabKey service, it should be unique, therefore check other development folders for all the names used. -All the auxiliary LabKey instances can be managed via the manage_all_continers.sh script. This script accepts two values (-s || -d), s starts all the containers in the docker folder. Starts with the primary which contains postgres and than looks for any folder that starts with dev. +All the auxiliary LabKey instances can be managed via the manage_all_continers.sh script. This script accepts two values (i.e., -s || -d), `-s` - starts all the containers in the docker folder and `-d` - shuts down all the instances running in the server. This script starts with the primary which contains postgres and than looks for any folder that has the prefix dev. ## Loading a Database Backup Using the Script