From d1e3b2cc447c66b294dcb9f8e13e208284828357 Mon Sep 17 00:00:00 2001 From: Jinpei Su Date: Fri, 15 May 2026 09:18:20 +0000 Subject: [PATCH 1/2] add MySQL MGR solutions migrated from internal wiki Adds five solution docs migrated from the internal Confluence MGR knowledge base, translated to English and validated against MySQL 8.0 on Alauda Application Services 4.x: - Troubleshooting MySQL initialization failure due to fs.aio-max-nr - Configure CONNECTION_CONTROL plugin for failed-login lockout - How to deploy CloudBeaver for MySQL management - How to import and export MySQL data (generic mysqldump workflow) - How to validate InnoDB Cluster import/export with the employees sample Co-Authored-By: Claude Opus 4.7 (1M context) --- ...Control_Plugin_for_Failed_Login_Lockout.md | 114 +++++++++++ ...Deploy_CloudBeaver_for_MySQL_Management.md | 151 ++++++++++++++ .../How_to_Import_and_Export_MySQL_Data.md | 101 ++++++++++ ...MySQL_InnoDB_Cluster_Data_Import_Export.md | 186 ++++++++++++++++++ ...nitialization_Failure_due_to_aio_max_nr.md | 74 +++++++ 5 files changed, 626 insertions(+) create mode 100644 docs/en/solutions/Configure_MySQL_Connection_Control_Plugin_for_Failed_Login_Lockout.md create mode 100644 docs/en/solutions/How_to_Deploy_CloudBeaver_for_MySQL_Management.md create mode 100644 docs/en/solutions/How_to_Import_and_Export_MySQL_Data.md create mode 100644 docs/en/solutions/How_to_Validate_MySQL_InnoDB_Cluster_Data_Import_Export.md create mode 100644 docs/en/solutions/Troubleshooting_MySQL_Initialization_Failure_due_to_aio_max_nr.md diff --git a/docs/en/solutions/Configure_MySQL_Connection_Control_Plugin_for_Failed_Login_Lockout.md b/docs/en/solutions/Configure_MySQL_Connection_Control_Plugin_for_Failed_Login_Lockout.md new file mode 100644 index 00000000..cd74b991 --- /dev/null +++ b/docs/en/solutions/Configure_MySQL_Connection_Control_Plugin_for_Failed_Login_Lockout.md @@ -0,0 +1,114 @@ +--- +kind: + - How To +products: + - Alauda Application Services +ProductsVersion: + - 4.0,4.1,4.2,4.3 +id: KB260515003 +--- + +# Configure MySQL Connection Control Plugin for Failed Login Lockout + +## Issue + +By default, MySQL accepts an unlimited number of password retries against any account. To mitigate brute-force attacks, operators often need to introduce a progressive delay after a configurable number of consecutive failed login attempts. This how-to enables the upstream `CONNECTION_CONTROL` plugin on a MySQL 8.0 instance managed by Alauda Application Services for MySQL (Group Replication topology) and verifies the throttling behavior. + +> The plugin works for MGR because the Router performs transparent L4 routing and preserves the original MySQL handshake. The same plugin is not effective behind a ProxySQL fronted Percona XtraDB Cluster (PXC) deployment because ProxySQL terminates and rewraps the authentication exchange. + +## Environment + +- Alauda Application Services for MySQL 4.0 and later +- A MySQL Group Replication (MGR) instance backed by the `Mysql` CR +- Cluster access via `kubectl` + +## Resolution + +### 1. Plugin variables + +The plugin exposes three variables that govern the delay applied after consecutive failed connection attempts: + +| Variable | Default | Description | +| --- | --- | --- | +| `connection_control_failed_connections_threshold` | `3` | Number of consecutive failed attempts before the delay kicks in. `0` disables the feature. Range: `0`–`2147483647`. | +| `connection_control_min_connection_delay` | `1000` ms | Minimum delay added before the server responds to a failed attempt once the threshold is reached. Range: `1000`–`2147483647`. | +| `connection_control_max_connection_delay` | `2147483647` ms | Upper bound of the delay. Range: `1000`–`2147483647`. | + +### 2. Enable the plugin via the `Mysql` CR + +Edit the MGR instance YAML and add the plugin entries under `spec.paras.mysqld`. The following example locks out an account after 5 failed attempts and holds each subsequent failure open for at least 5 minutes (300 000 ms): + +```yaml +spec: + paras: + mysqld: + plugin-load: connection_control.so + connection-control-failed-connections-threshold: "5" + connection-control-min-connection-delay: "300000" +``` + +Apply the change. The operator performs a rolling restart of the MGR pods to load the plugin. + +### 3. Verify the plugin is active + +After the rolling restart completes, exec into the Router pod and connect to the read-only service: + +```bash +kubectl -n get svc | grep -read +# Note the cluster-ip of -read-only and use it as the connection host. + +kubectl -n exec -it -router- -c router -- bash +mysql -uroot -h -P 3306 -p"$MYSQL_PASSWORD" +``` + +Check the plugin status and the current variables: + +```sql +SELECT PLUGIN_NAME, PLUGIN_STATUS +FROM information_schema.plugins +WHERE PLUGIN_NAME LIKE 'CONNECTION%'; + +SHOW VARIABLES LIKE 'connection_control%'; +``` + +Expected output: + +``` ++------------------------------------------+---------------+ +| PLUGIN_NAME | PLUGIN_STATUS | ++------------------------------------------+---------------+ +| CONNECTION_CONTROL | ACTIVE | +| CONNECTION_CONTROL_FAILED_LOGIN_ATTEMPTS | ACTIVE | ++------------------------------------------+---------------+ + ++-------------------------------------------------+------------+ +| Variable_name | Value | ++-------------------------------------------------+------------+ +| connection_control_failed_connections_threshold | 5 | +| connection_control_max_connection_delay | 2147483647 | +| connection_control_min_connection_delay | 300000 | ++-------------------------------------------------+------------+ +``` + +### 4. Validate the lockout behavior + +Trigger several failed logins with a deliberately wrong password and time each attempt. The connection time should grow once the threshold is exceeded: + +```bash +for i in $(seq 1 8); do + time mysql -uroot -h -P 3306 -p"wrongpass" -e "SELECT 1" || true +done +``` + +Inspect the per-account failed-attempt counter from any session: + +```sql +SELECT * +FROM performance_schema.connection_control_failed_login_attempts; +``` + +Each row shows the user/host pair and the number of consecutive failures currently held against it. A successful login resets the counter for that pair. + +### 5. Roll back + +To disable the lockout, remove the `plugin-load` and `connection-control-*` entries from `spec.paras.mysqld` and re-apply the CR. The operator restarts the pods and the plugin is no longer loaded. diff --git a/docs/en/solutions/How_to_Deploy_CloudBeaver_for_MySQL_Management.md b/docs/en/solutions/How_to_Deploy_CloudBeaver_for_MySQL_Management.md new file mode 100644 index 00000000..8b7452ee --- /dev/null +++ b/docs/en/solutions/How_to_Deploy_CloudBeaver_for_MySQL_Management.md @@ -0,0 +1,151 @@ +--- +kind: + - How To +products: + - Alauda Application Services +ProductsVersion: + - 4.0,4.1,4.2,4.3 +id: KB260515005 +--- + +# How to Deploy CloudBeaver for MySQL Management + +## Issue + +You need a web-based SQL client for browsing and querying MySQL instances managed by Alauda Application Services, without installing a desktop tool on every operator's workstation. CloudBeaver is the open-source web edition of DBeaver and runs as a single-pod Deployment on Kubernetes. This how-to deploys CloudBeaver and connects it to a MySQL Group Replication (MGR) instance. + +## Environment + +- Any Kubernetes cluster reachable to operators (NodePort exposure is used below; Ingress works equally well) +- A `StorageClass` capable of provisioning ReadWriteOnce PVCs — used to persist CloudBeaver workspace state across pod restarts +- Network reachability from the CloudBeaver pod to the target MySQL Router service + +## Resolution + +### 1. Prepare the manifest + +Save the following to `cloudbeaver.yaml`. Adjust `storageClassName`, image registry, and resource requests to match your environment: + +```yaml +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: cloudbeaver +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: sc-topolvm # replace with any RWO StorageClass available in the cluster + volumeMode: Filesystem +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: cloudbeaver +spec: + replicas: 1 + selector: + matchLabels: + app: cloudbeaver + strategy: + type: RollingUpdate + rollingUpdate: + maxSurge: 1 + maxUnavailable: 1 + template: + metadata: + labels: + app: cloudbeaver + spec: + containers: + - name: cloudbeaver + image: docker-mirrors.alauda.cn/dbeaver/cloudbeaver:latest + imagePullPolicy: Always + ports: + - name: web + containerPort: 8978 + protocol: TCP + resources: + limits: + cpu: "1" + memory: 1Gi + requests: + cpu: 100m + memory: 256Mi + volumeMounts: + - name: cloudbeaver-data + mountPath: /opt/cloudbeaver/workspace + volumes: + - name: cloudbeaver-data + persistentVolumeClaim: + claimName: cloudbeaver +--- +apiVersion: v1 +kind: Service +metadata: + name: cloudbeaver +spec: + type: NodePort + selector: + app: cloudbeaver + ports: + - name: web + port: 8978 + targetPort: 8978 + protocol: TCP +``` + +> CloudBeaver stores its administrator password, saved connections, and query history under `/opt/cloudbeaver/workspace`. Without a persistent volume, everything is lost on every pod restart. Confirm the chosen `storageClassName` exists in the target cluster before applying. + +### 2. Deploy + +```bash +kubectl -n apply -f cloudbeaver.yaml +kubectl -n rollout status deploy/cloudbeaver +``` + +### 3. Discover the access URL + +The Service uses NodePort, so any node IP plus the allocated NodePort exposes the UI: + +```bash +namespace= +HOST=$(kubectl -n "$namespace" get pod -l app=cloudbeaver \ + -o jsonpath='{.items[0].status.hostIP}') +PORT=$(kubectl -n "$namespace" get svc cloudbeaver \ + -o jsonpath='{.spec.ports[0].nodePort}') +echo "http://$HOST:$PORT" +``` + +Open the URL in a browser. + +### 4. Initial setup + +1. On first launch CloudBeaver prompts you to set the administrator password. Choose a strong password and store it safely — this account governs all subsequent server-side configuration. +2. Log in with the administrator user. +3. (Optional) Switch the UI language under the user menu in the top-right. + +### 5. Connect to a MySQL instance + +1. Click **New Connection** and choose **MySQL**. +2. Fill **Host** with the Router service of the target MGR instance and **Port** with the read-write port. From outside the cluster, retrieve the NodePort: + + ```bash + kubectl -n get svc -router + ``` + +3. Enter the application database user and password. +4. Under **Driver Properties**, set `allowPublicKeyRetrieval` to `TRUE` so the MySQL 8 driver can complete the `caching_sha2_password` handshake against a non-TLS endpoint. Do not enable this against an untrusted network — it lets the client retrieve the server's public key over an unencrypted channel. +5. Under **Access Management**, grant the current CloudBeaver user permission to use the new connection. +6. Save and test the connection. SQL editors can now be opened from the browser and queries executed against the MGR cluster. + +### 6. Uninstall + +```bash +kubectl -n delete -f cloudbeaver.yaml +``` + +The PVC is removed alongside the rest of the manifest. To preserve CloudBeaver state for a future redeploy, delete only the Deployment and Service and re-attach the existing PVC during the next install. diff --git a/docs/en/solutions/How_to_Import_and_Export_MySQL_Data.md b/docs/en/solutions/How_to_Import_and_Export_MySQL_Data.md new file mode 100644 index 00000000..514d43a2 --- /dev/null +++ b/docs/en/solutions/How_to_Import_and_Export_MySQL_Data.md @@ -0,0 +1,101 @@ +--- +kind: + - How To +products: + - Alauda Application Services +ProductsVersion: + - 4.0,4.1,4.2,4.3 +id: KB260515002 +--- + +# How to Import and Export MySQL Data + +## Issue + +You need to move business data between two MySQL databases — for example, migrating data from a self-hosted MySQL instance into a managed MySQL cluster on the ACP platform. The procedure must preserve referential integrity (triggers, routines, events) without polluting the destination with the source's system tables. + +## Environment + +- Source: any MySQL 5.7 or 8.0 database +- Destination: a MySQL 8.0 database, including an MGR-based MySQL cluster running under Alauda Application Services 4.x +- `mysqldump` utility from a MySQL release greater than or equal to the destination version + +## Resolution + +### 1. Plan the migration + +1. **Provision the destination cluster first.** Reserve enough storage to hold the logical dump in addition to the imported data set. +2. **Decide on a consistent cutover.** If the application cannot tolerate inconsistencies between the source and destination, stop application writes to the source before taking the dump. Otherwise rely on `--single-transaction` for a transactionally consistent snapshot of InnoDB tables. +3. **Do not dump system databases.** Restoring `mysql`, `information_schema`, `performance_schema`, or `sys` from another MySQL instance can corrupt the destination's privilege and metadata catalogues. Only dump the business schemas. +4. **Recreate application users on the destination.** The destination starts with its own privilege catalogue, so application-facing accounts must be created explicitly. Do not reuse `root` for application traffic; create dedicated accounts with the minimum required grants. + +### 2. Choose a `mysqldump` version + +The `mysqldump` client must be at least the version of the destination server. Two common arrangements: + +- Run `mysqldump` from inside a destination pod / host, which already ships a compatible binary; or +- Install a standalone `mysqldump` of an appropriate version on a jump host with network access to the source. + +Confirm both server and client versions before starting: + +```bash +mysql --version +mysqldump --version +``` + +### 3. Export the source business schemas + +Use a single transactional dump that captures triggers, routines, and events, and writes the source GTID set so the destination can replay binlog positions if needed: + +```bash +mysqldump \ + --host= \ + --user=root \ + --password='' \ + --single-transaction \ + --source-data=1 \ + --set-gtid-purged=AUTO \ + --triggers \ + --routines \ + --events \ + --databases ... \ + > _fullbackup.sql +``` + +Flag notes: + +- `--single-transaction` — consistent snapshot of InnoDB tables without table locks. +- `--source-data=1` — embed `CHANGE MASTER` / binlog position metadata in the dump. +- `--set-gtid-purged=AUTO` — preserve GTID information when the source uses GTIDs. +- `--triggers --routines --events` — include stored programs and event scheduler entries. +- `--databases` — dump only the listed schemas; never use `--all-databases`. + +For MySQL 5.7 sources, replace `--source-data=1` with `--master-data=1`. + +### 4. Import into the destination + +Run `mysql` against the destination using the same credentials and database list. The dump file is self-contained, so a single invocation re-creates schemas, tables, and stored programs: + +```bash +mysql \ + --host= \ + --user=root \ + --password='' \ + < _fullbackup.sql +``` + +For very large dumps, prefer running the import close to the destination (same VPC / same node) to reduce network round-trip overhead. + +### 5. Post-import verification + +1. Count rows in a sample of business tables on both sides: + ```sql + SELECT COUNT(*) FROM .; + ``` +2. Re-create application users on the destination with the minimum required grants: + ```sql + CREATE USER 'app'@'%' IDENTIFIED BY ''; + GRANT SELECT, INSERT, UPDATE, DELETE ON .* TO 'app'@'%'; + FLUSH PRIVILEGES; + ``` +3. Point the application at the destination and confirm read/write traffic before decommissioning the source. diff --git a/docs/en/solutions/How_to_Validate_MySQL_InnoDB_Cluster_Data_Import_Export.md b/docs/en/solutions/How_to_Validate_MySQL_InnoDB_Cluster_Data_Import_Export.md new file mode 100644 index 00000000..d3dd455f --- /dev/null +++ b/docs/en/solutions/How_to_Validate_MySQL_InnoDB_Cluster_Data_Import_Export.md @@ -0,0 +1,186 @@ +--- +kind: + - How To +products: + - Alauda Application Services +ProductsVersion: + - 4.0,4.1,4.2,4.3 +id: KB260515004 +--- + +# How to Validate MySQL InnoDB Cluster Data Import and Export + +## Issue + +You want a repeatable, end-to-end exercise to validate logical import and export against a MySQL InnoDB Cluster (Group Replication) instance: loading a sample dataset onto the PRIMARY, enabling `secure_file_priv` and `local_infile` so client tools can read and write files, and round-tripping data with a workbench-style client or `mysqldump`. This how-to walks through the procedure using the public `employees` sample database. + +## Environment + +- Alauda Application Services for MySQL 4.0 and later +- A running MGR instance (`Mysql` CR) with at least one ONLINE PRIMARY and two ONLINE SECONDARY members +- Cluster access via `kubectl` +- A workstation with the `mysql` client, `tar`, and optionally MySQL Workbench + +## Resolution + +### 1. Stage the `employees` sample database + +Download and extract the sample dataset on any host that can reach the cluster: + +```bash +wget https://launchpadlibrarian.net/24493586/employees_db-full-1.0.6.tar.bz2 +tar -xjvf employees_db-full-1.0.6.tar.bz2 +``` + +Edit `employees_db/employees.sql` and comment out the legacy `storage_engine` directives that no longer exist in MySQL 8.0; otherwise the loader fails with an unknown-variable error: + +```sql +-- set storage_engine = InnoDB; +-- set storage_engine = MyISAM; +-- set storage_engine = Falcon; +-- set storage_engine = PBXT; +-- set storage_engine = Maria; +-- select CONCAT('storage engine: ', @@storage_engine) as INFO; +``` + +### 2. Identify the PRIMARY member + +The `employees.sql` loader executes DDL and DML, so it must run against the PRIMARY: + +```bash +kubectl -n get pod -owide + +kubectl -n exec -it -0 -c mysql -- \ + mysql -uroot -p"$MYSQL_PASSWORD" -e \ + "SELECT MEMBER_HOST, MEMBER_PORT, MEMBER_STATE, MEMBER_ROLE + FROM performance_schema.replication_group_members;" +``` + +Expected output: + +``` ++---------------------+-------------+--------------+-------------+ +| MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | ++---------------------+-------------+--------------+-------------+ +| -0. | 3306 | ONLINE | PRIMARY | +| -1. | 3306 | ONLINE | SECONDARY | +| -2. | 3306 | ONLINE | SECONDARY | ++---------------------+-------------+--------------+-------------+ +``` + +Note the pod that hosts the PRIMARY (``) for the next step. + +### 3. Copy the sample dataset to the PRIMARY pod + +Only `/var/lib/mysql` inside the container is writable by the `mysql` user. + +```bash +kubectl cp -n -c mysql ./employees_db :/var/lib/mysql/ +kubectl -n exec -it -c mysql -- ls -lh /var/lib/mysql/employees_db +``` + +### 4. Load the sample database on the PRIMARY + +```bash +kubectl -n exec -it -c mysql -- bash -c \ + 'cd /var/lib/mysql/employees_db && mysql -uroot -p"$MYSQL_PASSWORD" < employees.sql' +``` + +Successful output includes `CREATING DATABASE STRUCTURE` followed by repeated `LOADING
` lines. + +After the load succeeds, remove the staging directory so MySQL does not treat it as a database directory: + +```bash +kubectl -n exec -it -c mysql -- rm -rf /var/lib/mysql/employees_db +``` + +### 5. Enable file import and export + +By default the server restricts both `secure_file_priv` and `local_infile`: + +```sql +SELECT @@secure_file_priv, @@local_infile; +-- +-----------------------+----------------+ +-- | /var/lib/mysql-files/ | 0 | +-- +-----------------------+----------------+ +``` + +Edit the `Mysql` CR and set the values under `spec.paras.mysqld`: + +```yaml +spec: + paras: + mysqld: + secure_file_priv: "" # empty string = unrestricted; or set to a directory to scope LOAD DATA / SELECT ... INTO OUTFILE + local_infile: "1" # enable LOAD DATA LOCAL INFILE +``` + +Re-apply the CR. The operator performs a rolling restart. Re-check: + +```sql +SELECT @@secure_file_priv, @@local_infile; +-- +--------------------+----------------+ +-- | | 1 | +-- +--------------------+----------------+ +``` + +> Setting `secure_file_priv` to an empty string removes all server-side restrictions on file paths used by `LOAD DATA`, `SELECT ... INTO OUTFILE`, and `LOAD_FILE()`. In production, restrict it to a dedicated directory rather than leaving it unrestricted. + +### 6. Connect a workbench client through the Router + +Retrieve the Router service (NodePort or LoadBalancer) of the instance: + +```bash +kubectl -n get svc -router +``` + +In MySQL Workbench (or any compatible GUI client), create a connection to the Router host and the read-write port using the root credentials. + +### 7. Export a table + +Sanity-check the source data first: + +```sql +SELECT COUNT(*) FROM employees.departments; +``` + +The `employees` schema is heavily foreign-key constrained; suspend constraints when dropping tables for round-trip tests and re-enable them afterwards: + +```sql +SET FOREIGN_KEY_CHECKS = 0; +-- ... drop / import operations ... +SET FOREIGN_KEY_CHECKS = 1; +``` + +Equivalent CLI export with `mysqldump`: + +```bash +mysqldump -h -P -uroot -p"$MYSQL_PASSWORD" \ + --set-gtid-purged=OFF \ + employees departments > employees_departments.sql +``` + +Workbench equivalent: **Server → Data Export**, select `employees.departments`, and write to `employees_departments.sql`. The resulting dump begins with the standard `mysqldump` header and contains `CREATE TABLE` plus `INSERT` statements. + +### 8. Import the dump back + +Drop the table on the cluster, then re-import: + +```sql +SET FOREIGN_KEY_CHECKS = 0; +DROP TABLE employees.departments; +SET FOREIGN_KEY_CHECKS = 1; +``` + +```bash +mysql -h -P -uroot -p"$MYSQL_PASSWORD" \ + employees < employees_departments.sql +``` + +Workbench equivalent: **Server → Data Import → Import from Self-Contained File**, choose `employees_departments.sql`, target schema `employees`, then **Start Import**. + +Verify the row count matches the pre-export value: + +```sql +SELECT COUNT(*) FROM employees.departments; +``` diff --git a/docs/en/solutions/Troubleshooting_MySQL_Initialization_Failure_due_to_aio_max_nr.md b/docs/en/solutions/Troubleshooting_MySQL_Initialization_Failure_due_to_aio_max_nr.md new file mode 100644 index 00000000..839f4b50 --- /dev/null +++ b/docs/en/solutions/Troubleshooting_MySQL_Initialization_Failure_due_to_aio_max_nr.md @@ -0,0 +1,74 @@ +--- +kind: + - How To +products: + - Alauda Application Services +ProductsVersion: + - 4.0,4.1,4.2,4.3 +id: KB260515001 +--- + +# Troubleshooting MySQL Initialization Failure Caused by Exhausted Async I/O Slots + +## Issue + +A MySQL Pod (single-instance, PXC, or MGR) fails to initialize and the container log shows `io_setup() failed`. Inspecting the host shows that `/proc/sys/fs/aio-nr` is at or close to `/proc/sys/fs/aio-max-nr`, meaning the kernel's pool of async I/O contexts is exhausted and MySQL cannot register the AIO contexts it needs to open InnoDB tablespaces. + +This typically appears on hosts that already serve many AIO-heavy workloads (multiple NFS or distributed-storage mounts, other database containers, virtualized I/O paths), where the default `fs.aio-max-nr = 65536` is no longer sufficient. + +## Environment + +- Alauda Application Services for MySQL on ACP (any topology: standalone MySQL, MySQL-PXC, MySQL-MGR) +- Linux kernel with libaio (any supported distribution) +- Host has reached or is close to the kernel default `fs.aio-max-nr = 65536` + +## Resolution + +### 1. Confirm the symptom + +On the node hosting the failing Pod, check the current AIO usage: + +```bash +cat /proc/sys/fs/aio-nr +cat /proc/sys/fs/aio-max-nr +``` + +If `aio-nr` is at or near `aio-max-nr`, the host has run out of AIO contexts and any new MySQL container scheduled on it will fail with `io_setup() failed`. + +### 2. Raise `fs.aio-max-nr` temporarily + +This unblocks initialization without rebooting the node: + +```bash +echo 1048576 > /proc/sys/fs/aio-max-nr +cat /proc/sys/fs/aio-max-nr +``` + +After the value is raised, delete the failing MySQL Pod so the operator schedules a fresh one; initialization should now succeed. + +### 3. Persist the change across reboots + +Add the setting to `/etc/sysctl.conf` (or a drop-in under `/etc/sysctl.d/`): + +```bash +echo 'fs.aio-max-nr = 1048576' >> /etc/sysctl.conf +sysctl -p +cat /proc/sys/fs/aio-max-nr +``` + +Apply this on every node that may host a MySQL Pod — if scheduling moves the Pod onto a node where the value is still the default, the failure recurs. + +### 4. Choose an appropriate value + +`fs.aio-max-nr` caps the number of outstanding async I/O requests the kernel will accept system-wide. The default `65536` is sized for a light desktop workload, not a database host. + +| Host profile | Suggested value | +| --- | --- | +| Dedicated database host with fast storage (NVMe/flash) | `1048576` or higher | +| General-purpose Kubernetes node also running databases | `262144` | +| Hosts with multiple NFS / distributed-storage mounts | `1048576` (each backend consumes contexts) | + +Notes: + +- Each AIO context costs a small amount of kernel memory; on nodes with less than 16 GB of RAM, prefer the lower value to avoid wasting memory. +- High-performance storage benefits most from the larger value — without it, the storage layer will be unable to drive enough concurrent requests to saturate the device. From 90912f5e7a031619e14c87a6aaefc5746801626a Mon Sep 17 00:00:00 2001 From: Jinpei Su Date: Fri, 15 May 2026 11:04:38 +0000 Subject: [PATCH 2/2] add MySQL audit log filter plugin solution Adds a how-to for enabling Percona Server's audit_log_filter.so plugin on a MySQL Group Replication instance managed by Alauda Application Services. Covers loading via spec.params.mysql.mysqld.plugin_load_add, filter creation, user attachment, and audit-log inspection. Migrated from the internal Confluence MGR knowledge base. Co-Authored-By: Claude Opus 4.7 (1M context) --- ...Configure_MySQL_Audit_Log_Filter_Plugin.md | 160 ++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 docs/en/solutions/Configure_MySQL_Audit_Log_Filter_Plugin.md diff --git a/docs/en/solutions/Configure_MySQL_Audit_Log_Filter_Plugin.md b/docs/en/solutions/Configure_MySQL_Audit_Log_Filter_Plugin.md new file mode 100644 index 00000000..2e9418d9 --- /dev/null +++ b/docs/en/solutions/Configure_MySQL_Audit_Log_Filter_Plugin.md @@ -0,0 +1,160 @@ +--- +kind: + - How To +products: + - Alauda Application Services +ProductsVersion: + - 4.0,4.1,4.2,4.3 +id: KB260515006 +--- + +# Configure MySQL Audit Log Filter Plugin + +## Issue + +You need to capture an audit trail of connection events, DDL, and DML on a MySQL 8.0 instance. MySQL Community Server does not ship the proprietary `audit_log.so` plugin from MySQL Enterprise, but Percona Server provides a drop-in replacement, `audit_log_filter.so`, that emits per-event JSON records and supports event-class filtering. This how-to enables the plugin on a `Mysql` CR-managed instance, defines a sensible default filter set, and verifies the output. + +## Environment + +- Alauda Application Services for MySQL 4.0 and later +- MySQL Server 8.0.36 or later (the plugin requires this minimum patch level) +- Cluster access via `kubectl` + +> The plugin is shipped as a preview feature in MySQL 8.0 and is generally available in MySQL 8.4. Treat it as feature-stable for the audit pipeline but expect minor variable renames between point releases. + +## Resolution + +### 1. Load the plugin via the `Mysql` CR + +Set the plugin and its configuration variables under `spec.params.mysql.mysqld`. Use `loose_` prefixed variables so the server does not refuse to start if the plugin is not yet loaded: + +```yaml +spec: + params: + mysql: + mysqld: + plugin_load_add: "audit_log_filter.so" + loose_audit_log_filter_format: "JSON" + loose_audit_log_filter_rotate_on_size: "104857600" +``` + +Variable meaning: + +- `plugin_load_add` — additionally load the named plugin at startup. +- `loose_audit_log_filter_format` — output format. `JSON` is recommended for downstream parsing. +- `loose_audit_log_filter_rotate_on_size` — file-rotation size in bytes (here, 100 MiB). The plugin keeps up to 1 GiB of rotated history by default. + +Full variable reference: [Percona audit log filter variables](https://docs.percona.com/percona-server/8.0/audit-log-filter-variables.html). + +Apply the change. The operator performs a rolling restart of the MySQL pods. + +### 2. Verify the plugin is loaded + +After the rolling restart completes, connect to the read-write service and check the plugin status plus its variables: + +```sql +SELECT PLUGIN_NAME, PLUGIN_STATUS +FROM information_schema.PLUGINS +WHERE PLUGIN_NAME LIKE '%audit%'\G + +SHOW GLOBAL VARIABLES LIKE 'audit_log_filter%'; +``` + +Confirm that `AUDIT_LOG_FILTER` is `ACTIVE` and that the variable values match the YAML you applied. + +### 3. Initialize the plugin + +The plugin ships an initialization script that creates its filter tables in the `mysql` system schema. Run it once against the read-write endpoint: + +```bash +mysql -h -read-write -uroot -p \ + < /usr/share/percona-server/audit_log_filter_linux_install.sql +``` + +Enter the root password when prompted. + +### 4. Define and attach filters + +Connect to the read-write endpoint and create two filters: + +- `quiet` for the operator's health-check and management users, to avoid flooding the log with no-op queries. +- `default` for every other user, recording connect/disconnect events, general statements, and write-side table access. + +Run the following against the read-write endpoint: + +```sql +SET @quiet = ' +{ + "filter": { + "class": [ + { + "name": "table_access", + "event": [ + { "name": "insert" }, + { "name": "delete" }, + { "name": "update" } + ] + } + ] + } +}'; + +SELECT audit_log_filter_set_filter('quiet', @quiet); +SELECT audit_log_filter_set_user('exporter@localhost', 'quiet'); +SELECT audit_log_filter_set_user('manage@localhost', 'quiet'); +SELECT audit_log_filter_set_user('healthchecker@localhost', 'quiet'); + +SET @default = ' +{ + "filter": { + "class": [ + { + "name": "connection", + "event": [ + { "name": "connect" }, + { "name": "disconnect" } + ] + }, + { "name": "general" }, + { + "name": "table_access", + "event": [ + { "name": "insert" }, + { "name": "delete" }, + { "name": "update" } + ] + } + ] + } +}'; + +SELECT audit_log_filter_set_filter('default', @default); +SELECT audit_log_filter_set_user('%', 'default'); +``` + +Adjust the filter JSON to match the auditing scope you actually need. The full schema is documented at [audit log filter definitions](https://dev.mysql.com/doc/refman/8.4/en/audit-log-filter-definitions.html). + +### 5. Detach or remove filters + +Detach a filter from a user (`%` matches all otherwise-unconfigured users; substitute an actual `user@host` to detach for a single account): + +```sql +SELECT audit_log_filter_remove_user('%'); +``` + +Delete a filter definition: + +```sql +SELECT audit_log_filter_remove_filter('default'); +``` + +### 6. Inspect the audit log files + +The audit log files live alongside the MySQL data directory inside every MySQL pod. List them on a given pod: + +```bash +kubectl -n exec -it -c mysql -- \ + ls -lh /var/lib/mysql/audit_filter* +``` + +Each rotated file is JSON-line formatted and ready to ship to a log aggregator.