feat(roles/duplicity): support Debian/Ubuntu and remove the build toolchain

CHANGELOG.md

@@ -21,18 +21,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+* **role:duplicity**: Add Debian and Ubuntu support (proven on Debian 12, Debian 13, Ubuntu 22.04, Ubuntu 24.04 and Ubuntu 26.04). The role now installs the `gnupg` package itself, so backups also work on minimal installs that ship without `gpg`.
+* **role:python_venv**: Add an optional per-venv `pip_constraints` key that pins transitive dependencies through a pip constraints file, without having to list them as direct packages.
 * **role:matomo_import_logs**: New role that imports Apache access logs into Matomo on a schedule, one systemd timer per site, and ships the Matomo log-analytics import script (`import_logs.py`). The `token_auth` is provided via a per-site auth file instead of the command line (passing `--token-auth`, `--login` or `--password` is deprecated, since they are visible in the process list and now log a deprecation warning). The script also supports the Traefik access-log format and fixes a possible endless loop when reading a config file.
 * **role:glances**: Add RHEL 10 / Rocky 10 / Alma 10 support by installing glances into a Python venv via the `python_venv` role, since the package is not available in EPEL 10. RHEL 10 is now marked proven (`x`) in COMPATIBILITY.
 * **role:graylog_datanode**: Add `graylog_datanode__http_publish_uri` to set the REST API URI the DataNode advertises, needed when the bind address is not directly reachable (multiple interfaces, a NAT gateway, or a `0.0.0.0` bind address).
 
 ### Changed
 
+* **role:duplicity**: Validate the role variables at start, and align the task tags with the LFOps vocabulary. The `duplicity:script` tag is gone (the `duba` script now deploys under `duplicity:configure`), and the new `duplicity:dump` tag manages the backup schedule.
 * **role:collabora**: Support Collabora Online CODE 25.04.10. The role ships one `coolwsd.xml` template per CODE release and had none for this version, so it aborted the deploy on hosts that had updated to it.
 * **role:clamav**: Send notification mails through `sendmail` (provided by postfix) instead of the `mail` command (mailx). One invocation works across distributions, and delivery no longer depends on mailx being installed.
 * **role:icingadb, role:icingaweb2, role:icingaweb2_module_reporting, role:icingaweb2_module_x509, role:mariadb_server**: Move the MariaDB tasks from the deprecated `community.mysql` collection to its replacement `ansible.mysql`. Behaviour is unchanged, but the deprecation warnings printed on every run are gone and the roles keep working once `community.mysql` is removed upstream.
 
 ### Fixed
 
+* **role:duplicity**: Swift backups now work out of the box on Python 3.10 and newer (all supported RHEL, Debian and Ubuntu releases). The role pins a modern `oslo.*` stack in the venv, which fixes the `collections.Mapping` crash (the previously required manual workaround is gone) and drops the deprecated, source-only `netifaces` dependency. As a result the role no longer installs a C compiler (`gcc`) or development headers on backup hosts: the build toolchain is gone from production machines, which is a significant reduction of the attack surface.
 * **role:monitoring_plugins**: A source install now deploys the sudoers drop-in as `/etc/sudoers.d/linuxfabrik-monitoring-plugins`, the same file name the rpm/deb packages use. Both install methods remove the drop-in under the former name `/etc/sudoers.d/monitoring-plugins`, so sudo no longer warns about a duplicate `Cmnd_Alias` on hosts that got the drop-in twice (for example after switching the install method or after running the monitoring-plugins one-liner installer).
 * **role:collect_rpmnew_rpmsave**: Stop emitting an Ansible deprecation warning on every run by making the `when` conditions explicitly boolean. Keeps the role working on Ansible 2.19 and later.
 * **role:kvm_vm**: Use `kvm_vm__connect_url` for every libvirt operation. Disk resizes (`virsh blockresize`) and a few other steps previously ignored the configured connection URL and always talked to the local default, so they failed or acted on the wrong libvirt when `kvm_vm__connect_url` pointed at a non-default or remote host.

COMPATIBILITY.md

@@ -29,7 +29,7 @@ Which Ansible role is proven to run on which OS?
 | dnf_makecache                         |        |        |   x    |   x    |    x    |           |           |           |                                              |
 | dnf_versionlock                       |        |        |   x    |   x    |         |           |           |           | Fedora 40                                    |
 | docker                                |        |        |   x    |  (x)   |   (x)   |           |           |           |                                              |
-| duplicity                             |        |        |   x    |   x    |    x    |           |           |           | Fedora 35                                    |
+| duplicity                             |   x    |   x    |   x    |   x    |    x    |     x     |     x     |     x     | Fedora 35                                    |
 | elastic_agent                         |  (x)   |  (x)   |  (x)   |   x    |   (x)   |    (x)    |     x     |    (x)    |                                              |
 | elastic_agent_fleet_server            |  (x)   |  (x)   |  (x)   |   x    |   (x)   |    (x)    |     x     |    (x)    |                                              |
 | elasticsearch                         |  (x)   |  (x)   |   x    |   x    |   (x)   |    (x)    |     x     |    (x)    |                                              |

roles/duplicity/README.md

@@ -18,7 +18,7 @@ Note that this role does not support running with `--check`, as it first creates
 
 Any [LFOps playbook](https://github.com/Linuxfabrik/lfops/blob/main/playbooks/README.md) that installs this role runs these for you. Optional ones can be disabled via the playbook's skip variables.
 
-* On RHEL-compatible systems, the EPEL repository must be enabled (role: [linuxfabrik.lfops.repo_epel](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_epel)). On Rocky 9+, the CRB ("Code Ready Builder") repository must also be enabled (role: [linuxfabrik.lfops.repo_baseos](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_baseos)) so `python3-virtualenv` can be installed.
+* On RHEL-compatible systems, the EPEL repository must be enabled (role: [linuxfabrik.lfops.repo_epel](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_epel)). On Rocky 9+, the CRB ("Code Ready Builder") repository must also be enabled (role: [linuxfabrik.lfops.repo_baseos](https://github.com/Linuxfabrik/lfops/tree/main/roles/repo_baseos)) so `python3-virtualenv` can be installed. On Debian and Ubuntu, no additional repositories are required.
 * `duplicity`, `python-swiftclient` and `python-keystoneclient` must be installed into a Python 3 virtual environment in `/opt/python-venv/duplicity` (role: [linuxfabrik.lfops.python_venv](https://github.com/Linuxfabrik/lfops/tree/main/roles/python_venv)).
 
 **Attention**
@@ -43,12 +43,12 @@ Manual steps:
 
 `duplicity:configure`
 
-* Deploys the configuration for duplicity.
+* Deploys the configuration for duplicity, including the `duba` script.
 * Triggers: none.
 
-`duplicity:script`
+`duplicity:dump`
 
-* Just deploys the `duba` script.
+* Manages the daily backup schedule (deploys and enables the `duba` systemd timer).
 * Triggers: none.
 
 `duplicity:state`
@@ -276,10 +276,6 @@ duplicity__timer_enabled: true
 
 * Make sure your `duplicity__gpg_encrypt_master_key_block` is correct and has an empty line after the `-----BEGIN PGP PUBLIC KEY BLOCK-----`.
 
-**`duplicity` fails with `AttributeError: module 'collections' has no attribute 'Mapping'` in `oslo_config/cfg.py`**
-
-* Manually install `'oslo.config>=9'`, e.g. `/opt/python-venv/duplicity/bin/pip install 'oslo.config>=9'`.
-
 ## License
 
 [The Unlicense](https://unlicense.org/)

roles/duplicity/defaults/main.yml

@@ -51,4 +51,4 @@ duplicity__loglevel: 'notice'
 
 duplicity__swift_authurl: 'https://swiss-backup02.infomaniak.com/identity/v3'
 duplicity__swift_authversion: '3'
-duplicity__swift_tenantname: 'sb_project_{{ duplicity__swift_login["username"] }}'
+duplicity__swift_tenantname: 'sb_project_{{ duplicity__swift_login["username"] | d("") }}'

roles/duplicity/meta/argument_specs.yml

@@ -0,0 +1,143 @@
+# argument_specs validates required variables and types automatically at role entry.
+# use this for simple "is defined" / type checks. for complex validations
+# (value ranges, cross-variable logic), use ansible.builtin.assert in the tasks.
+argument_specs:
+  main:
+    options:
+
+      duplicity__backup_backend:
+        type: 'str'
+        required: false
+        default: 'swift'
+        choices:
+          - 'sftp'
+          - 'swift'
+        description: 'The backup backend being used.'
+
+      duplicity__backup_dest:
+        type: 'str'
+        required: false
+        description: 'The backup destination, combined with the backup source path to form the target URL for duplicity.'
+
+      duplicity__backup_dest_container:
+        type: 'str'
+        required: false
+        description: 'The Swift container, used to separate backups on the destination.'
+
+      duplicity__backup_full_if_older_than:
+        type: 'str'
+        required: false
+        default: '30D'
+        description: 'After how long a full backup instead of an incremental one is done.'
+
+      duplicity__backup_retention_time:
+        type: 'str'
+        required: false
+        default: '30D'
+        description: 'The retention time of the backups.'
+
+      duplicity__backup_sources__dependent_var:
+        type: 'list'
+        elements: 'dict'
+        required: false
+        default: []
+        description: 'Directories to back up. Dependent-role injection.'
+
+      duplicity__backup_sources__group_var:
+        type: 'list'
+        elements: 'dict'
+        required: false
+        default: []
+        description: 'Directories to back up. Group-level override.'
+
+      duplicity__backup_sources__host_var:
+        type: 'list'
+        elements: 'dict'
+        required: false
+        default: []
+        description: 'Directories to back up. Host-level override.'
+
+      duplicity__excludes:
+        type: 'list'
+        elements: 'str'
+        required: false
+        default:
+          - '**/*.git*'
+          - '**/*.svn*'
+          - '**/*.temp'
+          - '**/*.tmp'
+          - '**/.cache'
+          - '**/cache'
+          - '**/log'
+        description: 'Global exclude shell patterns for duplicity.'
+
+      duplicity__gpg_encrypt_master_key:
+        type: 'str'
+        required: true
+        description: 'The long key ID of the master GPG key.'
+
+      duplicity__gpg_encrypt_master_key_block:
+        type: 'str'
+        required: true
+        description: 'The ASCII-armored public master GPG key, used in addition to the local key to encrypt the backups.'
+
+      duplicity__loglevel:
+        type: 'str'
+        required: false
+        default: 'notice'
+        choices:
+          - 'debug'
+          - 'error'
+          - 'info'
+          - 'notice'
+          - 'warning'
+        description: 'The duplicity log level.'
+
+      duplicity__logrotate:
+        type: 'int'
+        required: false
+        description: 'Number of days log files are kept before being rotated out.'
+
+      duplicity__on_calendar:
+        type: 'str'
+        required: false
+        description: 'The OnCalendar definition for the daily systemd timer.'
+
+      duplicity__on_calendar_hour:
+        type: 'str'
+        required: false
+        default: '23'
+        description: 'Shorthand to set the hour of duplicity__on_calendar.'
+
+      duplicity__sftp_password:
+        type: 'str'
+        required: false
+        description: 'Password for the SSH user used by the SFTP connection. Required when duplicity__backup_backend is sftp.'
+
+      duplicity__swift_authurl:
+        type: 'str'
+        required: false
+        default: 'https://swiss-backup02.infomaniak.com/identity/v3'
+        description: 'The authentication URL for Swift.'
+
+      duplicity__swift_authversion:
+        type: 'str'
+        required: false
+        default: '3'
+        description: 'The authentication version for Swift.'
+
+      duplicity__swift_login:
+        type: 'dict'
+        required: false
+        description: 'The Swift username and password (subkeys username and password). Required when duplicity__backup_backend is swift.'
+
+      duplicity__swift_tenantname:
+        type: 'str'
+        required: false
+        description: 'The Swift tenant name.'
+
+      duplicity__timer_enabled:
+        type: 'bool'
+        required: false
+        default: true
+        description: 'The state of the daily systemd timer.'

roles/duplicity/tasks/main.yml

@@ -5,6 +5,17 @@
       name: 'shared'
       tasks_from: 'platform-variables.yml'
 
+  tags:
+    - 'always'
+
+
+- block:
+
+  - name: 'Install required packages'
+    ansible.builtin.package:
+      name: '{{ __duplicity__required_packages }}'
+      state: 'present'
+
   - name: 'mkdir /etc/duba'
     ansible.builtin.file:
       path: '/etc/duba'
@@ -19,7 +30,7 @@
 
 - block:
 
-  - name: 'Combined Paths:'
+  - name: 'Combined Paths'
     ansible.builtin.debug:
       var: 'duplicity__backup_sources__combined_var'
 
@@ -43,11 +54,6 @@
     register: 'duplicity__gpg_import_result'
     changed_when: '"not changed" not in duplicity__gpg_import_result.stderr'
 
-  - name: 'gpg --import /tmp/public-master-key'
-    ansible.builtin.command: 'gpg --import /tmp/public-master-key'
-    register: 'duplicity__gpg_import_result'
-    changed_when: '"not changed" not in duplicity__gpg_import_result.stderr'
-
   - name: 'rm -f /tmp/public-master-key'
     ansible.builtin.file:
       path: '/tmp/public-master-key'
@@ -66,6 +72,8 @@
       backup: true
       src: 'etc/duba/duba.json.j2'
       dest: '/etc/duba/duba.json'
+      owner: 'root'
+      group: 'root'
       mode: 0o600 # file contains secrets
 
   - name: 'Deploy /etc/systemd/system/duba.service'
@@ -77,22 +85,22 @@
       group: 'root'
       mode: 0o644
 
-  - name: 'Deploy /etc/systemd/system/duba.timer'
+  - name: 'Deploy /usr/local/bin/duba'
     ansible.builtin.template:
       backup: true
-      src: 'etc/systemd/system/duba.timer.j2'
-      dest: '/etc/systemd/system/duba.timer'
+      src: 'usr/local/bin/duba.j2'
+      dest: '/usr/local/bin/duba'
       owner: 'root'
       group: 'root'
-      mode: 0o644
-    register: 'duplicity__systemd_duba_timer_result'
+      mode: 0o755
 
   - name: 'Deploy /etc/logrotate.d/duplicity'
     ansible.builtin.template:
       backup: true
       src: 'etc/logrotate.d/duplicity.j2'
       dest: '/etc/logrotate.d/duplicity'
       owner: 'root'
+      group: 'root'
       mode: 0o644
 
   tags:
@@ -102,18 +110,20 @@
 
 - block:
 
-  - name: 'Deploy /usr/local/bin/duba'
+  - name: 'Deploy /etc/systemd/system/duba.timer'
     ansible.builtin.template:
       backup: true
-      src: 'usr/local/bin/duba.j2'
-      dest: '/usr/local/bin/duba'
+      src: 'etc/systemd/system/duba.timer.j2'
+      dest: '/etc/systemd/system/duba.timer'
       owner: 'root'
-      mode: 0o755
+      group: 'root'
+      mode: 0o644
+    register: 'duplicity__systemd_duba_timer_result'
 
   tags:
     - 'duplicity'
     - 'duplicity:configure'
-    - 'duplicity:script'
+    - 'duplicity:dump'
 
 
 - block:
@@ -122,9 +132,10 @@
     ansible.builtin.systemd:
       name: 'duba.timer'
       state: '{{ duplicity__timer_enabled | bool | ternary("started", "stopped") }}'
-      enabled: '{{ duplicity__timer_enabled }}'
+      enabled: '{{ duplicity__timer_enabled | bool }}'
       daemon_reload: '{{ duplicity__systemd_duba_timer_result | d({}) is changed }}'
 
   tags:
     - 'duplicity'
+    - 'duplicity:dump'
     - 'duplicity:state'

roles/duplicity/vars/Debian.yml

@@ -0,0 +1,3 @@
+# gpg is required for key generation, key import and duplicity's encryption at backup time.
+__duplicity__required_packages:
+  - 'gnupg'

roles/duplicity/vars/RedHat.yml

@@ -0,0 +1,3 @@
+# gpg is required for key generation, key import and duplicity's encryption at backup time.
+__duplicity__required_packages:
+  - 'gnupg2'

roles/duplicity/vars/Ubuntu.yml

@@ -0,0 +1,3 @@
+# gpg is required for key generation, key import and duplicity's encryption at backup time.
+__duplicity__required_packages:
+  - 'gnupg'