From b14dbcc93406672e7fbf587f61c046cce7bfe61b Mon Sep 17 00:00:00 2001 From: Markus Frei Date: Wed, 17 Jun 2026 17:02:26 +0200 Subject: [PATCH] feat(roles/duplicity): support Debian/Ubuntu and remove the build toolchain Security win: backup hosts no longer get a C compiler. duplicity's Swift backend pulled the deprecated, source-only netifaces (via the unused pyrax stack), which forced gcc and the Python -devel headers onto every backup host. Pinning a modern oslo.* stack drops netifaces, so the role now installs no build toolchain at all - a significant reduction of the attack surface on production machines. * python_venv: add an optional per-venv pip_constraints key (writes a constraints file and passes pip --constraint), plus meta/argument_specs.yml. * duplicity: pin a modern oslo.* stack via pip_constraints. This also fixes the collections.Mapping crash on Python 3.10+, so the manual oslo.config workaround is gone. package_requirements is now interpreter-only (no gcc, no -devel, no librsync-devel). * duplicity: add Debian 12/13 and Ubuntu 22.04/24.04/26.04 support, proven on all of them plus RHEL 8/9/10 via containers. Install gnupg from the role. * duplicity: add meta/argument_specs.yml, load platform vars under the always tag, drop a duplicate gpg --import task, set owner/group on all templates, and align the tags with the LFOps vocabulary (duplicity:script folded into duplicity:configure, new duplicity:dump for the backup schedule). * Mark Debian and Ubuntu proven for duplicity in COMPATIBILITY. --- CHANGELOG.md | 4 + COMPATIBILITY.md | 2 +- roles/duplicity/README.md | 12 +- roles/duplicity/defaults/main.yml | 2 +- roles/duplicity/meta/argument_specs.yml | 143 ++++++++++++++++++ roles/duplicity/tasks/main.yml | 45 +++--- roles/duplicity/vars/Debian.yml | 3 + roles/duplicity/vars/RedHat.yml | 3 + roles/duplicity/vars/Ubuntu.yml | 3 + roles/duplicity/vars/main.yml | 65 +++++--- roles/python_venv/README.md | 16 +- roles/python_venv/meta/argument_specs.yml | 38 +++++ roles/python_venv/tasks/create-venv.yml | 18 ++- .../opt/python-venv/constraints.txt.j2 | 5 + 14 files changed, 306 insertions(+), 53 deletions(-) create mode 100644 roles/duplicity/meta/argument_specs.yml create mode 100644 roles/duplicity/vars/Debian.yml create mode 100644 roles/duplicity/vars/RedHat.yml create mode 100644 roles/duplicity/vars/Ubuntu.yml create mode 100644 roles/python_venv/meta/argument_specs.yml create mode 100644 roles/python_venv/templates/opt/python-venv/constraints.txt.j2 diff --git a/CHANGELOG.md b/CHANGELOG.md index df9b43482..43f1ec199 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,18 +21,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +* **role:duplicity**: Add Debian and Ubuntu support (proven on Debian 12, Debian 13, Ubuntu 22.04, Ubuntu 24.04 and Ubuntu 26.04). The role now installs the `gnupg` package itself, so backups also work on minimal installs that ship without `gpg`. +* **role:python_venv**: Add an optional per-venv `pip_constraints` key that pins transitive dependencies through a pip constraints file, without having to list them as direct packages. * **role:matomo_import_logs**: New role that imports Apache access logs into Matomo on a schedule, one systemd timer per site, and ships the Matomo log-analytics import script (`import_logs.py`). The `token_auth` is provided via a per-site auth file instead of the command line (passing `--token-auth`, `--login` or `--password` is deprecated, since they are visible in the process list and now log a deprecation warning). The script also supports the Traefik access-log format and fixes a possible endless loop when reading a config file. * **role:glances**: Add RHEL 10 / Rocky 10 / Alma 10 support by installing glances into a Python venv via the `python_venv` role, since the package is not available in EPEL 10. RHEL 10 is now marked proven (`x`) in COMPATIBILITY. * **role:graylog_datanode**: Add `graylog_datanode__http_publish_uri` to set the REST API URI the DataNode advertises, needed when the bind address is not directly reachable (multiple interfaces, a NAT gateway, or a `0.0.0.0` bind address). ### Changed +* **role:duplicity**: Validate the role variables at start, and align the task tags with the LFOps vocabulary. The `duplicity:script` tag is gone (the `duba` script now deploys under `duplicity:configure`), and the new `duplicity:dump` tag manages the backup schedule. * **role:collabora**: Support Collabora Online CODE 25.04.10. The role ships one `coolwsd.xml` template per CODE release and had none for this version, so it aborted the deploy on hosts that had updated to it. * **role:clamav**: Send notification mails through `sendmail` (provided by postfix) instead of the `mail` command (mailx). One invocation works across distributions, and delivery no longer depends on mailx being installed. * **role:icingadb, role:icingaweb2, role:icingaweb2_module_reporting, role:icingaweb2_module_x509, role:mariadb_server**: Move the MariaDB tasks from the deprecated `community.mysql` collection to its replacement `ansible.mysql`. Behaviour is unchanged, but the deprecation warnings printed on every run are gone and the roles keep working once `community.mysql` is removed upstream. ### Fixed +* **role:duplicity**: Swift backups now work out of the box on Python 3.10 and newer (all supported RHEL, Debian and Ubuntu releases). The role pins a modern `oslo.*` stack in the venv, which fixes the `collections.Mapping` crash (the previously required manual workaround is gone) and drops the deprecated, source-only `netifaces` dependency. As a result the role no longer installs a C compiler (`gcc`) or development headers on backup hosts: the build toolchain is gone from production machines, which is a significant reduction of the attack surface. * **role:monitoring_plugins**: A source install now deploys the sudoers drop-in as `/etc/sudoers.d/linuxfabrik-monitoring-plugins`, the same file name the rpm/deb packages use. Both install methods remove the drop-in under the former name `/etc/sudoers.d/monitoring-plugins`, so sudo no longer warns about a duplicate `Cmnd_Alias` on hosts that got the drop-in twice (for example after switching the install method or after running the monitoring-plugins one-liner installer). * **role:collect_rpmnew_rpmsave**: Stop emitting an Ansible deprecation warning on every run by making the `when` conditions explicitly boolean. Keeps the role working on Ansible 2.19 and later. * **role:kvm_vm**: Use `kvm_vm__connect_url` for every libvirt operation. Disk resizes (`virsh blockresize`) and a few other steps previously ignored the configured connection URL and always talked to the local default, so they failed or acted on the wrong libvirt when `kvm_vm__connect_url` pointed at a non-default or remote host. diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 6de4f3b64..50ad7e346 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -29,7 +29,7 @@ Which Ansible role is proven to run on which OS? | dnf_makecache | | | x | x | x | | | | | | dnf_versionlock | | | x | x | | | | | Fedora 40 | | docker | | | x | (x) | (x) | | | | | -| duplicity | | | x | x | x | | | | Fedora 35 | +| duplicity | x | x | x | x | x | x | x | x | Fedora 35 | | elastic_agent | (x) | (x) | (x) | x | (x) | (x) | x | (x) | | | elastic_agent_fleet_server | (x) | (x) | (x) | x | (x) | (x) | x | (x) | | | elasticsearch | (x) | (x) | x | x | (x) | (x) | x | (x) | | diff --git a/roles/duplicity/README.md b/roles/duplicity/README.md index befc50504..0b9f5f9f2 100644 --- a/roles/duplicity/README.md +++ b/roles/duplicity/README.md @@ -18,7 +18,7 @@ Note that this role does not support running with `--check`, as it first creates Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/README.md) that installs this role runs these for you. Optional ones can be disabled via the playbook's skip variables. -* On RHEL-compatible systems, the EPEL repository must be enabled (role: [linuxfabrik.lfops.repo_epel](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_epel)). On Rocky 9+, the CRB ("Code Ready Builder") repository must also be enabled (role: [linuxfabrik.lfops.repo_baseos](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_baseos)) so `python3-virtualenv` can be installed. +* On RHEL-compatible systems, the EPEL repository must be enabled (role: [linuxfabrik.lfops.repo_epel](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_epel)). On Rocky 9+, the CRB ("Code Ready Builder") repository must also be enabled (role: [linuxfabrik.lfops.repo_baseos](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_baseos)) so `python3-virtualenv` can be installed. On Debian and Ubuntu, no additional repositories are required. * `duplicity`, `python-swiftclient` and `python-keystoneclient` must be installed into a Python 3 virtual environment in `/opt/python-venv/duplicity` (role: [linuxfabrik.lfops.python_venv](https://github.com/Linuxfabrik/lfops/tree/main/roles/python_venv)). **Attention** @@ -43,12 +43,12 @@ Manual steps: `duplicity:configure` -* Deploys the configuration for duplicity. +* Deploys the configuration for duplicity, including the `duba` script. * Triggers: none. -`duplicity:script` +`duplicity:dump` -* Just deploys the `duba` script. +* Manages the daily backup schedule (deploys and enables the `duba` systemd timer). * Triggers: none. `duplicity:state` @@ -276,10 +276,6 @@ duplicity__timer_enabled: true * Make sure your `duplicity__gpg_encrypt_master_key_block` is correct and has an empty line after the `-----BEGIN PGP PUBLIC KEY BLOCK-----`. -**`duplicity` fails with `AttributeError: module 'collections' has no attribute 'Mapping'` in `oslo_config/cfg.py`** - -* Manually install `'oslo.config>=9'`, e.g. `/opt/python-venv/duplicity/bin/pip install 'oslo.config>=9'`. - ## License [The Unlicense](https://unlicense.org/) diff --git a/roles/duplicity/defaults/main.yml b/roles/duplicity/defaults/main.yml index 1d438a9c1..1a8695443 100644 --- a/roles/duplicity/defaults/main.yml +++ b/roles/duplicity/defaults/main.yml @@ -51,4 +51,4 @@ duplicity__loglevel: 'notice' duplicity__swift_authurl: 'https://swiss-backup02.infomaniak.com/identity/v3' duplicity__swift_authversion: '3' -duplicity__swift_tenantname: 'sb_project_{{ duplicity__swift_login["username"] }}' +duplicity__swift_tenantname: 'sb_project_{{ duplicity__swift_login["username"] | d("") }}' diff --git a/roles/duplicity/meta/argument_specs.yml b/roles/duplicity/meta/argument_specs.yml new file mode 100644 index 000000000..840aa0904 --- /dev/null +++ b/roles/duplicity/meta/argument_specs.yml @@ -0,0 +1,143 @@ +# argument_specs validates required variables and types automatically at role entry. +# use this for simple "is defined" / type checks. for complex validations +# (value ranges, cross-variable logic), use ansible.builtin.assert in the tasks. +argument_specs: + main: + options: + + duplicity__backup_backend: + type: 'str' + required: false + default: 'swift' + choices: + - 'sftp' + - 'swift' + description: 'The backup backend being used.' + + duplicity__backup_dest: + type: 'str' + required: false + description: 'The backup destination, combined with the backup source path to form the target URL for duplicity.' + + duplicity__backup_dest_container: + type: 'str' + required: false + description: 'The Swift container, used to separate backups on the destination.' + + duplicity__backup_full_if_older_than: + type: 'str' + required: false + default: '30D' + description: 'After how long a full backup instead of an incremental one is done.' + + duplicity__backup_retention_time: + type: 'str' + required: false + default: '30D' + description: 'The retention time of the backups.' + + duplicity__backup_sources__dependent_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: 'Directories to back up. Dependent-role injection.' + + duplicity__backup_sources__group_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: 'Directories to back up. Group-level override.' + + duplicity__backup_sources__host_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: 'Directories to back up. Host-level override.' + + duplicity__excludes: + type: 'list' + elements: 'str' + required: false + default: + - '**/*.git*' + - '**/*.svn*' + - '**/*.temp' + - '**/*.tmp' + - '**/.cache' + - '**/cache' + - '**/log' + description: 'Global exclude shell patterns for duplicity.' + + duplicity__gpg_encrypt_master_key: + type: 'str' + required: true + description: 'The long key ID of the master GPG key.' + + duplicity__gpg_encrypt_master_key_block: + type: 'str' + required: true + description: 'The ASCII-armored public master GPG key, used in addition to the local key to encrypt the backups.' + + duplicity__loglevel: + type: 'str' + required: false + default: 'notice' + choices: + - 'debug' + - 'error' + - 'info' + - 'notice' + - 'warning' + description: 'The duplicity log level.' + + duplicity__logrotate: + type: 'int' + required: false + description: 'Number of days log files are kept before being rotated out.' + + duplicity__on_calendar: + type: 'str' + required: false + description: 'The OnCalendar definition for the daily systemd timer.' + + duplicity__on_calendar_hour: + type: 'str' + required: false + default: '23' + description: 'Shorthand to set the hour of duplicity__on_calendar.' + + duplicity__sftp_password: + type: 'str' + required: false + description: 'Password for the SSH user used by the SFTP connection. Required when duplicity__backup_backend is sftp.' + + duplicity__swift_authurl: + type: 'str' + required: false + default: 'https://swiss-backup02.infomaniak.com/identity/v3' + description: 'The authentication URL for Swift.' + + duplicity__swift_authversion: + type: 'str' + required: false + default: '3' + description: 'The authentication version for Swift.' + + duplicity__swift_login: + type: 'dict' + required: false + description: 'The Swift username and password (subkeys username and password). Required when duplicity__backup_backend is swift.' + + duplicity__swift_tenantname: + type: 'str' + required: false + description: 'The Swift tenant name.' + + duplicity__timer_enabled: + type: 'bool' + required: false + default: true + description: 'The state of the daily systemd timer.' diff --git a/roles/duplicity/tasks/main.yml b/roles/duplicity/tasks/main.yml index deb42a1d9..7fd2b1212 100644 --- a/roles/duplicity/tasks/main.yml +++ b/roles/duplicity/tasks/main.yml @@ -5,6 +5,17 @@ name: 'shared' tasks_from: 'platform-variables.yml' + tags: + - 'always' + + +- block: + + - name: 'Install required packages' + ansible.builtin.package: + name: '{{ __duplicity__required_packages }}' + state: 'present' + - name: 'mkdir /etc/duba' ansible.builtin.file: path: '/etc/duba' @@ -19,7 +30,7 @@ - block: - - name: 'Combined Paths:' + - name: 'Combined Paths' ansible.builtin.debug: var: 'duplicity__backup_sources__combined_var' @@ -43,11 +54,6 @@ register: 'duplicity__gpg_import_result' changed_when: '"not changed" not in duplicity__gpg_import_result.stderr' - - name: 'gpg --import /tmp/public-master-key' - ansible.builtin.command: 'gpg --import /tmp/public-master-key' - register: 'duplicity__gpg_import_result' - changed_when: '"not changed" not in duplicity__gpg_import_result.stderr' - - name: 'rm -f /tmp/public-master-key' ansible.builtin.file: path: '/tmp/public-master-key' @@ -66,6 +72,8 @@ backup: true src: 'etc/duba/duba.json.j2' dest: '/etc/duba/duba.json' + owner: 'root' + group: 'root' mode: 0o600 # file contains secrets - name: 'Deploy /etc/systemd/system/duba.service' @@ -77,15 +85,14 @@ group: 'root' mode: 0o644 - - name: 'Deploy /etc/systemd/system/duba.timer' + - name: 'Deploy /usr/local/bin/duba' ansible.builtin.template: backup: true - src: 'etc/systemd/system/duba.timer.j2' - dest: '/etc/systemd/system/duba.timer' + src: 'usr/local/bin/duba.j2' + dest: '/usr/local/bin/duba' owner: 'root' group: 'root' - mode: 0o644 - register: 'duplicity__systemd_duba_timer_result' + mode: 0o755 - name: 'Deploy /etc/logrotate.d/duplicity' ansible.builtin.template: @@ -93,6 +100,7 @@ src: 'etc/logrotate.d/duplicity.j2' dest: '/etc/logrotate.d/duplicity' owner: 'root' + group: 'root' mode: 0o644 tags: @@ -102,18 +110,20 @@ - block: - - name: 'Deploy /usr/local/bin/duba' + - name: 'Deploy /etc/systemd/system/duba.timer' ansible.builtin.template: backup: true - src: 'usr/local/bin/duba.j2' - dest: '/usr/local/bin/duba' + src: 'etc/systemd/system/duba.timer.j2' + dest: '/etc/systemd/system/duba.timer' owner: 'root' - mode: 0o755 + group: 'root' + mode: 0o644 + register: 'duplicity__systemd_duba_timer_result' tags: - 'duplicity' - 'duplicity:configure' - - 'duplicity:script' + - 'duplicity:dump' - block: @@ -122,9 +132,10 @@ ansible.builtin.systemd: name: 'duba.timer' state: '{{ duplicity__timer_enabled | bool | ternary("started", "stopped") }}' - enabled: '{{ duplicity__timer_enabled }}' + enabled: '{{ duplicity__timer_enabled | bool }}' daemon_reload: '{{ duplicity__systemd_duba_timer_result | d({}) is changed }}' tags: - 'duplicity' + - 'duplicity:dump' - 'duplicity:state' diff --git a/roles/duplicity/vars/Debian.yml b/roles/duplicity/vars/Debian.yml new file mode 100644 index 000000000..bf705ca3f --- /dev/null +++ b/roles/duplicity/vars/Debian.yml @@ -0,0 +1,3 @@ +# gpg is required for key generation, key import and duplicity's encryption at backup time. +__duplicity__required_packages: + - 'gnupg' diff --git a/roles/duplicity/vars/RedHat.yml b/roles/duplicity/vars/RedHat.yml new file mode 100644 index 000000000..f84187b25 --- /dev/null +++ b/roles/duplicity/vars/RedHat.yml @@ -0,0 +1,3 @@ +# gpg is required for key generation, key import and duplicity's encryption at backup time. +__duplicity__required_packages: + - 'gnupg2' diff --git a/roles/duplicity/vars/Ubuntu.yml b/roles/duplicity/vars/Ubuntu.yml new file mode 100644 index 000000000..bf705ca3f --- /dev/null +++ b/roles/duplicity/vars/Ubuntu.yml @@ -0,0 +1,3 @@ +# gpg is required for key generation, key import and duplicity's encryption at backup time. +__duplicity__required_packages: + - 'gnupg' diff --git a/roles/duplicity/vars/main.yml b/roles/duplicity/vars/main.yml index 167f0793d..02c215db4 100644 --- a/roles/duplicity/vars/main.yml +++ b/roles/duplicity/vars/main.yml @@ -1,16 +1,41 @@ +# duplicity ships prebuilt manylinux wheels, so nothing here needs to be compiled and no +# compiler or development headers are required: package_requirements only installs the Python +# interpreter (plus python3-venv on Debian/Ubuntu, which provides the venv module). +# +# pip_constraints force a modern oslo.* stack. Without them, duplicity's unconditional pyrax +# dependency (the Rackspace backend we never use) drags in an ancient python-keystoneclient and +# oslo.utils. That old oslo.utils both requires the deprecated, source-only `netifaces` (which +# would need a compiler) and uses `collections.Mapping`, removed in Python 3.10, which crashes +# the Swift backend. Pinning modern oslo.* avoids netifaces entirely and fixes the crash. +# +# The single `Debian` key (os_family) covers both Debian and Ubuntu, where the distribution +# default `python3` is the right interpreter and the package names do not differ by version. __duplicity__python_venv__venvs__dependent_var: + Debian: + - name: 'duplicity' + packages: + - 'duplicity' + package_requirements: + - 'python3-venv' + pip_constraints: + - 'oslo.config>=9' + - 'oslo.i18n>=5' + - 'oslo.serialization>=5' + - 'oslo.utils>=7' + python_executable: 'python3' + exposed_binaries: + - 'duplicity' RedHat8: - name: 'duplicity' packages: - 'duplicity' - # If we add these, we get "SetuptoolsDeprecationWarning: Invalid dash-separated options", - # and current versions of duplicity (now) installs these as well: - # - 'python-keystoneclient' - # - 'python-swiftclient' package_requirements: - - 'gcc' - - 'librsync-devel' - - 'python3.11-devel' + - 'python3.11' + pip_constraints: + - 'oslo.config>=9' + - 'oslo.i18n>=5' + - 'oslo.serialization>=5' + - 'oslo.utils>=7' python_executable: 'python3.11' exposed_binaries: - 'duplicity' @@ -18,14 +43,13 @@ __duplicity__python_venv__venvs__dependent_var: - name: 'duplicity' packages: - 'duplicity' - # If we add these, we get "SetuptoolsDeprecationWarning: Invalid dash-separated options", - # and current versions of duplicity (now) installs these as well: - # - 'python-keystoneclient' - # - 'python-swiftclient' package_requirements: - - 'gcc' - - 'librsync-devel' - - 'python3.11-devel' + - 'python3.11' + pip_constraints: + - 'oslo.config>=9' + - 'oslo.i18n>=5' + - 'oslo.serialization>=5' + - 'oslo.utils>=7' python_executable: 'python3.11' exposed_binaries: - 'duplicity' @@ -33,14 +57,13 @@ __duplicity__python_venv__venvs__dependent_var: - name: 'duplicity' packages: - 'duplicity' - # If we add these, we get "SetuptoolsDeprecationWarning: Invalid dash-separated options", - # and current versions of duplicity (now) installs these as well: - # - 'python-keystoneclient' - # - 'python-swiftclient' package_requirements: - - 'gcc' - - 'librsync-devel' - - 'python3.13-devel' + - 'python3.13' + pip_constraints: + - 'oslo.config>=9' + - 'oslo.i18n>=5' + - 'oslo.serialization>=5' + - 'oslo.utils>=7' python_executable: 'python3.13' exposed_binaries: - 'duplicity' diff --git a/roles/python_venv/README.md b/roles/python_venv/README.md index dcf21f796..377a831b7 100644 --- a/roles/python_venv/README.md +++ b/roles/python_venv/README.md @@ -61,6 +61,11 @@ Manual steps: * Mandatory. These packages will be installed in the virtual environment using `pip`. * Type: List. + * `pip_constraints`: + + * Optional. List of pip [constraints](https://pip.pypa.io/en/stable/user_guide/#constraints-files) applied when installing `packages`. They are written to `/opt/python-venv/name/constraints.txt` and passed to `pip` via `--constraint`. Use this to pin transitive dependencies (for example to force a version that drops a deprecated, compiler-requiring sub-dependency) without listing them as direct `packages`. + * Type: List. + * `system_site_packages`: * Optional. Allows the virtual environment to access the system site-packages dir. @@ -101,11 +106,14 @@ python_venv__venvs__host_var: - name: 'duplicity' packages: - 'duplicity' - - 'python-swiftclient' - - 'python-keystoneclient' package_requirements: - - 'gcc' - - 'librsync-devel' + - 'python3.11' + pip_constraints: + - 'oslo.config>=9' + - 'oslo.i18n>=5' + - 'oslo.serialization>=5' + - 'oslo.utils>=7' + python_executable: 'python3.11' exposed_binaries: - 'duplicity' python_venv__venvs__group_var: [] diff --git a/roles/python_venv/meta/argument_specs.yml b/roles/python_venv/meta/argument_specs.yml new file mode 100644 index 000000000..d74950be2 --- /dev/null +++ b/roles/python_venv/meta/argument_specs.yml @@ -0,0 +1,38 @@ +# argument_specs validates required variables and types automatically at role entry. +# use this for simple "is defined" / type checks. for complex validations +# (value ranges, cross-variable logic), use ansible.builtin.assert in the tasks. +argument_specs: + main: + options: + + python_venv__pip_cert: + type: 'str' + required: false + description: 'Path to a PEM-encoded CA certificate bundle for pip to use instead of its built-in certificates.' + + python_venv__pip_conf_global_retries: + type: 'int' + required: false + default: 0 + description: 'Number of retries pip performs on a failed network operation, written to /etc/pip.conf.' + + python_venv__venvs__dependent_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: 'Virtual environments to manage. Dependent-role injection.' + + python_venv__venvs__group_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: 'Virtual environments to manage. Group-level override.' + + python_venv__venvs__host_var: + type: 'list' + elements: 'dict' + required: false + default: [] + description: 'Virtual environments to manage. Host-level override.' diff --git a/roles/python_venv/tasks/create-venv.yml b/roles/python_venv/tasks/create-venv.yml index 0a607baf2..315f09797 100644 --- a/roles/python_venv/tasks/create-venv.yml +++ b/roles/python_venv/tasks/create-venv.yml @@ -27,12 +27,28 @@ virtualenv: '/opt/python-venv/{{ venv["name"] }}' extra_args: '{{ ("--cert " ~ python_venv__pip_cert) if python_venv__pip_cert is defined and python_venv__pip_cert | length }}' +# A constraints file pins transitive dependencies (e.g. to force a modern version that drops a +# deprecated, compiler-requiring sub-dependency) without having to list them as direct packages. +- name: 'Deploy /opt/python-venv/{{ venv["name"] }}/constraints.txt' + ansible.builtin.template: + backup: true + src: 'opt/python-venv/constraints.txt.j2' + dest: '/opt/python-venv/{{ venv["name"] }}/constraints.txt' + owner: 'root' + group: 'root' + mode: 0o644 + when: + - 'venv["pip_constraints"] | d([]) | length > 0' + - name: 'Install packages using pip' ansible.builtin.pip: name: '{{ venv["packages"] }}' state: 'present' virtualenv: '/opt/python-venv/{{ venv["name"] }}' - extra_args: '{{ ("--cert " ~ python_venv__pip_cert) if python_venv__pip_cert is defined and python_venv__pip_cert | length }}' + extra_args: '{{ + (("--cert " ~ python_venv__pip_cert) if (python_venv__pip_cert is defined and python_venv__pip_cert | length > 0) else "") + ~ ((" --constraint /opt/python-venv/" ~ venv["name"] ~ "/constraints.txt") if (venv["pip_constraints"] | d([]) | length > 0) else "") + }}' - name: 'Link choosen binaries to /usr/local/bin' ansible.builtin.file: diff --git a/roles/python_venv/templates/opt/python-venv/constraints.txt.j2 b/roles/python_venv/templates/opt/python-venv/constraints.txt.j2 new file mode 100644 index 000000000..8b3c4beca --- /dev/null +++ b/roles/python_venv/templates/opt/python-venv/constraints.txt.j2 @@ -0,0 +1,5 @@ +# {{ ansible_managed }} +# 20260617 +{% for constraint in venv["pip_constraints"] | d([]) %} +{{ constraint }} +{% endfor %}