[bug-fix] Fix reinstall-overwrites-kept-config: preserve config on plain reinstall after --keep-config#3449
Conversation
…all after --keep-config Apply the remediation from the bug assessment on issue #3427. Before the unconditional shutil.rmtree(dest_dir), scan dest_dir for any *-config.yml and *-config.local.yml files and hold their contents in memory. After shutil.copytree succeeds, write them back so user-customized values always win over the packaged defaults. This mirrors the existing backup/restore logic for the --force reinstall path but handles the case where remove --keep-config left config files behind in an unregistered extension directory. Refs #3427 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Fixes extension config loss when reinstalling after remove --keep-config.
Changes:
- Rescues and restores preserved extension configuration files.
- Adds regression tests for standard and local configs.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/extensions/__init__.py |
Adds preserved-config rescue and restoration. |
tests/test_extensions.py |
Adds reinstall config-preservation tests. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Medium
| # Restore stranded configs rescued before the rmtree above. | ||
| for filename, content in stranded_configs.items(): | ||
| (dest_dir / filename).write_bytes(content) |
There was a problem hiding this comment.
Fixed in 5507df0. The stranded-config restore loop is now placed immediately after shutil.copytree, before command, skill, and hook registration — so a failure in any of those steps leaves the preserved config in place.
Posted on behalf of @mnriem by GitHub Copilot (model: claude-sonnet-4.5, autonomous).
| stranded_configs: dict[str, bytes] = {} | ||
| if dest_dir.exists() and not self.registry.is_installed(manifest.id): | ||
| for cfg_file in ( | ||
| list(dest_dir.glob("*-config.yml")) | ||
| + list(dest_dir.glob("*-config.local.yml")) | ||
| ): | ||
| if cfg_file.is_file() and not cfg_file.is_symlink(): | ||
| stranded_configs[cfg_file.name] = cfg_file.read_bytes() |
There was a problem hiding this comment.
Fixed in 5507df0 and bd55c59. stranded_configs now stores (bytes, mode) tuples (capturing cfg_file.stat().st_mode before the rmtree). After copytree the original mode is reapplied via target.chmod(mode & 0o660) — the mask strips setuid/setgid and world-write bits so only user/group read-write permissions are restored.
Posted on behalf of @mnriem by GitHub Copilot (model: claude-sonnet-4.5, autonomous).
…eserve file mode - Restore missing `test_install_force_without_existing` method declaration in tests/test_extensions.py so pytest collects it as a separate test. - Move stranded-config restoration to immediately after `copytree`, before command/skill/hook registration, so a failed registration step can't leave preserved configs permanently lost. - Store `(bytes, mode)` tuples instead of bare bytes when rescuing stranded configs, and reapply the original file mode after writing so permission bits (e.g. 0600 for credential files) are faithfully restored. Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)
Only preserve user/group read-write bits (mode & 0o660) to avoid restoring setuid, setgid, or world-writable permissions from a user-modified config file. Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)
| # Restore stranded configs rescued before the rmtree above. | ||
| for filename, (content, mode) in stranded_configs.items(): |
There was a problem hiding this comment.
Fixed in fix: add copytree rollback path and strengthen regression test with packaged default config. The copytree call is now wrapped in a try/except BaseException block. If it raises (disk full, permission error, etc.), the rescued configs are written back — recreating dest_dir with mkdir(parents=True, exist_ok=True) if it was left absent or only partially created — before the exception is re-raised.
Posted on behalf of @mnriem by GitHub Copilot (model: claude-sonnet-4.6, autonomous).
| # Install once | ||
| manager.install_from_directory( | ||
| extension_dir, "0.1.0", register_commands=False | ||
| ) |
There was a problem hiding this comment.
Fixed in the same commit. test_reinstall_after_keep_config_preserves_config now writes a default test-ext-config.yml (with model: default-model) into extension_dir before the first install. The user then customizes the installed copy to custom-model. After remove --keep-config + plain reinstall, the test asserts that custom-model is present and default-model is not — so a "restore only when absent" implementation would fail.
Posted on behalf of @mnriem by GitHub Copilot (model: claude-sonnet-4.6, autonomous).
…ackaged default config - Wrap shutil.copytree in a try/except BaseException so stranded configs rescued before rmtree are written back even if copytree fails mid-way (addresses review comment: configs were permanently lost on copy failure) - Add a packaged default config to extension_dir in the regression test so a naive 'restore only when absent' implementation would fail; assert the user's customized values beat the packaged defaults after reinstall Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
| target = dest_dir / filename | ||
| target.write_bytes(content) | ||
| target.chmod(mode & 0o660) |
There was a problem hiding this comment.
Addressed in 5f9da76. Restores now write to a securely created temporary file with constrained mode before data is written, then atomically replace the target, so preserved config bytes are never written under permissive target modes.
| target = dest_dir / filename | ||
| target.write_bytes(content) | ||
| # Mask to only user/group read-write bits to avoid restoring | ||
| # setuid/setgid or world-writable permissions. | ||
| target.chmod(mode & 0o660) |
There was a problem hiding this comment.
Addressed in 5f9da76. The successful-install restore path now uses the same secure temp-file + atomic replace flow with mode applied before writing content.
Assisted-by: GitHub Copilot (model: gpt-5, autonomous)
| mode = preserved_mode & 0o660 | ||
| tmp_path: Path | None = None | ||
| try: | ||
| with tempfile.NamedTemporaryFile( | ||
| mode="wb", | ||
| dir=target.parent, | ||
| prefix=f".{target.name}.", | ||
| delete=False, | ||
| ) as tmp: | ||
| tmp_path = Path(tmp.name) | ||
| tmp_path.chmod(mode) | ||
| tmp.write(content) | ||
| os.replace(tmp_path, target) |
There was a problem hiding this comment.
Fixed in the latest commit. _restore_stranded_config_file now writes content while the temp file holds its OS-default secure mode (typically 0600 on POSIX), then applies preserved_mode directly — without any mask — after the file is fully written and the with block closes, immediately before os.replace. This faithfully restores 0644, executable bits, and world permissions exactly as they were on the original file.
Posted on behalf of @mnriem by GitHub Copilot (model: claude-sonnet-4.6, autonomous).
| except BaseException: | ||
| # copytree failed — dest_dir may be absent or only partially | ||
| # created. Write the rescued configs back now so they are not | ||
| # permanently lost even though the install did not complete. | ||
| if stranded_configs: |
There was a problem hiding this comment.
Added test_copytree_failure_restores_stranded_config in the same commit. It monkeypatches shutil.copytree to create a partial dest_dir and then raise OSError("simulated disk full"), exercises the rollback path, and asserts that the preserved config bytes are restored, the original file mode is reapplied (POSIX only via stat.S_IMODE comparison), and the extension remains unregistered after the failed install.
Posted on behalf of @mnriem by GitHub Copilot (model: claude-sonnet-4.6, autonomous).
…e-failure test - _restore_stranded_config_file: write content while temp file is at its secure OS-default mode (typically 0600 on POSIX), then apply the original preserved_mode after the file is fully written and before the atomic os.replace. Removes the & 0o660 mask that was silently stripping world-read and executable bits (e.g. 0644 → 0640). - Add test_copytree_failure_restores_stranded_config: patches shutil.copytree to create a partial destination then raise OSError, then asserts that the preserved config bytes and file mode are restored by the rollback path and that the extension remains unregistered. Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
…rt' and 'import from'' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
| from pathlib import Path | ||
| from datetime import datetime, timezone | ||
| from unittest.mock import MagicMock | ||
| import specify_cli.extensions as _ext_module |
| stranded_configs: dict[str, tuple[bytes, int]] = {} | ||
| if dest_dir.exists() and not self.registry.is_installed(manifest.id): |
Bug fix — reinstall-overwrites-kept-config
Proposed fix for issue #3427, applying the remediation from the bug assessment.
Verdict: Valid · Severity: medium
Summary
When
specify extension remove <ext> --keep-configis used, the extension is unregistered but its*-config.ymlfiles survive in the extension directory. A subsequent plainspecify extension add <ext>unconditionally deleted that directory before copying the fresh extension in, silently discarding the preserved config. The fix rescues those stranded config files into memory before thermtreeand writes them back aftercopytree, so user-customized values always win over the packaged defaults.Changes
src/specify_cli/extensions/__init__.pyrmtree(dest_dir), collect any*-config.yml/*-config.local.ymlfiles from an unregistereddest_dirinto memory; restore them aftercopytreetests/test_extensions.pytest_reinstall_after_keep_config_preserves_configandtest_reinstall_after_keep_config_preserves_local_configTests Added or Updated
tests/test_extensions.py::TestInstallFromDirectory::test_reinstall_after_keep_config_preserves_config— pins that a customized*-config.ymlsurvives aremove --keep-config→ plain reinstall cycletests/test_extensions.py::TestInstallFromDirectory::test_reinstall_after_keep_config_preserves_local_config— same for*-config.local.ymloverride filesLocal Verification
--forcereinstall path, applying it to the previously-unhandled "unregistered but config-bearing directory" case.Deviations from Assessment
None. The implementation follows the preferred remediation exactly as described.
Risks & Review Notes
not self.registry.is_installed(manifest.id)guard ensures we only rescue configs when the extension is genuinely unregistered, avoiding picking up stale files from a different install.shutil.copytree, so packaged defaults are always superseded by the user's values — no risk of defaults silently winning.install_from_zip()delegates toinstall_from_directory(), so it is covered without additional changes.install_from_directory().Refs #3427 · cc
@grafvonbremove --keep-config#3427