When we have to restart a Docker Splunk 10.2.7 instance, sometimes we see the following error:
TASK [splunk_common : Check for previous ConfigMap-owned values in indexes.conf] ***
ok: [localhost]
Tuesday 01 September 2026 21:24:09 +0000 (0:00:00.257) 0:00:44.298 *****
TASK [splunk_common : Load previous ConfigMap-owned values in indexes.conf] ****
fatal: [localhost]: FAILED! => {
"ansible_facts": {
"previous_config_map_stanzas": {}
},
"ansible_included_var_files": [],
"changed": false,
"message": "an error occurred while trying to read the file '/opt/splunk/etc/system/local/.indexes.conf.splunk-ansible-managed.yml': [Errno 13] Permission denied: b'/opt/splunk/etc/system/local/.indexes.conf.splunk-ansible-managed.yml'. [Errno 13] Permission denied: b'/opt/splunk/etc/system/local/.indexes.conf.splunk-ansible-managed.yml'"
}
Observed behavior:
The first initialization creates a Splunk-Ansible state file such as:
- /opt/splunk/etc/system/local/.inputs.conf.splunk-ansible-managed.yml
The file is created with the following ownership and permissions:
- -rw-rw---- 1 41812 41812 25 .inputs.conf.splunk-ansible-managed.yml
The numeric ownership corresponds to:
- splunk:splunk
- UID 41812
- GID 41812
- mode 0660
The container initialization process runs as the ansible user:
- uid=999(ansible) gid=999(ansible) groups=999(ansible),1000(sudo)
- The ansible user is not a member of group splunk and therefore cannot read the persisted state file on the next initialization.
The subsequent startup fails with:
TASK [splunk_common : Load previous ConfigMap-owned values in inputs.conf]
fatal: [localhost]: FAILED! => {
"ansible_facts": {
"previous_config_map_stanzas": {}
},
"ansible_included_var_files": [],
"changed": false,
"message": "an error occurred while trying to read the file '/opt/splunk/etc/system/local/.inputs.conf.splunk-ansible-managed.yml': [Errno 13] Permission denied: b'/opt/splunk/etc/system/local/.inputs.conf.splunk-ansible-managed.yml'. [Errno 13] Permission denied: b'/opt/splunk/etc/system/local/.inputs.conf.splunk-ansible-managed.yml'"
}
Splunk-Ansible task behavior:
In both the 9.4.15 and 10.2.7 images, the state file is loaded using this task:
- name: Load previous ConfigMap-owned values in {{ conf_file }}
include_vars:
file: "{{ conf_directory }}/.{{ conf_file }}.splunk-ansible-managed.yml"
name: previous_config_map_stanzas
when: config_map_managed_values_file.stat.exists
no_log: "{{ hide_password }}"
This task does not appear to use privilege escalation or become_user: splunk.
The file is created using:
- name: Record ConfigMap-owned values in {{ conf_file }}
copy:
content: "{{ conf_stanzas | to_nice_yaml }}"
dest: "{{ conf_directory }}/.{{ conf_file }}.splunk-ansible-managed.yml"
owner: "{{ splunk.user }}"
group: "{{ splunk.group }}"
mode: '0660'
when:
- reconcile_config_map_values | default(false) | bool
- conf_stanzas | length > 0
become: yes
become_user: "{{ splunk.user }}"
no_log: "{{ hide_password }}"
This creates an apparent mismatch:
- The file is written as splunk:splunk with mode 0660.
- The next initialization attempts to read the file as ansible.
- The ansible user is not a member of the splunk group.
- The read fails with EACCES.
Example default.yml:
splunk:
http_enableSSL: true
http_port: 8443
conf:
- key: inputs
value:
directory: /opt/splunk/etc/system/local
content:
default:
host: test-host
Simplified reproduction procedure:
Start splunk/splunk:10.2.7 with a persistent volume mounted at /opt/splunk/etc.
Supply a default.yml containing a splunk.conf entry for inputs.conf.
Allow the first initialization to complete.
Confirm that this file exists:
- ls -ln /opt/splunk/etc/system/local/.inputs.conf.splunk-ansible-managed.yml
Expected result:
- -rw-rw---- 1 41812 41812 ...
Confirm the container runtime identity: docker exec id
Observed result:
- uid=999(ansible) gid=999(ansible) groups=999(ansible),1000(sudo)
- Restart the container:
- docker restart
- Splunk-Ansible initialization fails while loading the persisted managed-values file.
Deyault.yml
The current Splunk Operator documentation continues to support default.yml through the defaults and defaultsUrl CR fields. For example, a ConfigMap can be mounted and referenced as:
spec:
volumes:
- name: defaults
configMap:
name: splunk-defaults
defaultsUrl: /mnt/defaults/default.yml
Because SOK uses the same Splunk Enterprise image, the same code path may be reachable when default.yml contains splunk.conf entries.
Actual Kubernetes exposure may depend on the effective Pod security context. A Pod configured with fsGroup: 41812 or supplementalGroups: [41812] may not experience the failure.
When we have to restart a Docker Splunk 10.2.7 instance, sometimes we see the following error:
TASK [splunk_common : Check for previous ConfigMap-owned values in indexes.conf] ***
ok: [localhost]
Tuesday 01 September 2026 21:24:09 +0000 (0:00:00.257) 0:00:44.298 *****
TASK [splunk_common : Load previous ConfigMap-owned values in indexes.conf] ****
fatal: [localhost]: FAILED! => {
"ansible_facts": {
"previous_config_map_stanzas": {}
},
"ansible_included_var_files": [],
"changed": false,
"message": "an error occurred while trying to read the file '/opt/splunk/etc/system/local/.indexes.conf.splunk-ansible-managed.yml': [Errno 13] Permission denied: b'/opt/splunk/etc/system/local/.indexes.conf.splunk-ansible-managed.yml'. [Errno 13] Permission denied: b'/opt/splunk/etc/system/local/.indexes.conf.splunk-ansible-managed.yml'"
}
Observed behavior:
The first initialization creates a Splunk-Ansible state file such as:
The file is created with the following ownership and permissions:
The numeric ownership corresponds to:
The container initialization process runs as the ansible user:
The subsequent startup fails with:
TASK [splunk_common : Load previous ConfigMap-owned values in inputs.conf]
fatal: [localhost]: FAILED! => {
"ansible_facts": {
"previous_config_map_stanzas": {}
},
"ansible_included_var_files": [],
"changed": false,
"message": "an error occurred while trying to read the file '/opt/splunk/etc/system/local/.inputs.conf.splunk-ansible-managed.yml': [Errno 13] Permission denied: b'/opt/splunk/etc/system/local/.inputs.conf.splunk-ansible-managed.yml'. [Errno 13] Permission denied: b'/opt/splunk/etc/system/local/.inputs.conf.splunk-ansible-managed.yml'"
}
Splunk-Ansible task behavior:
In both the 9.4.15 and 10.2.7 images, the state file is loaded using this task:
include_vars:
file: "{{ conf_directory }}/.{{ conf_file }}.splunk-ansible-managed.yml"
name: previous_config_map_stanzas
when: config_map_managed_values_file.stat.exists
no_log: "{{ hide_password }}"
This task does not appear to use privilege escalation or become_user: splunk.
The file is created using:
copy:
content: "{{ conf_stanzas | to_nice_yaml }}"
dest: "{{ conf_directory }}/.{{ conf_file }}.splunk-ansible-managed.yml"
owner: "{{ splunk.user }}"
group: "{{ splunk.group }}"
mode: '0660'
when:
become: yes
become_user: "{{ splunk.user }}"
no_log: "{{ hide_password }}"
This creates an apparent mismatch:
Example default.yml:
splunk:
http_enableSSL: true
http_port: 8443
conf:
- key: inputs
value:
directory: /opt/splunk/etc/system/local
content:
default:
host: test-host
Simplified reproduction procedure:
Start splunk/splunk:10.2.7 with a persistent volume mounted at /opt/splunk/etc.
Supply a default.yml containing a splunk.conf entry for inputs.conf.
Allow the first initialization to complete.
Confirm that this file exists:
Expected result:
Confirm the container runtime identity: docker exec id
Observed result:
Deyault.yml
The current Splunk Operator documentation continues to support default.yml through the defaults and defaultsUrl CR fields. For example, a ConfigMap can be mounted and referenced as:
spec:
volumes:
- name: defaults
configMap:
name: splunk-defaults
defaultsUrl: /mnt/defaults/default.yml
Because SOK uses the same Splunk Enterprise image, the same code path may be reachable when default.yml contains splunk.conf entries.
Actual Kubernetes exposure may depend on the effective Pod security context. A Pod configured with fsGroup: 41812 or supplementalGroups: [41812] may not experience the failure.