Skip to content

[v2] Lazily import all built-in plugins#10298

Draft
aemous wants to merge 22 commits into
aws:v2from
aemous:lazy-plugins-events-imports-lazycommand
Draft

[v2] Lazily import all built-in plugins#10298
aemous wants to merge 22 commits into
aws:v2from
aemous:lazy-plugins-events-imports-lazycommand

Conversation

@aemous
Copy link
Copy Markdown
Contributor

@aemous aemous commented May 11, 2026

Description of changes:

PrefixTrie

  • Made PrefixTrie in vendored botocore public.
  • Created unit tests for PrefixTrie.

LazyCommand

  • Implemented LazyCommand for deferred plugin importing and invocation until the command is invoked or its help docs are generated.
  • Implement unit tests and functional tests for LazyCommand.

LazyInitEmitter

  • Implemented LazyInitEmitter, which loads a plugin registry to lazily initialize plugins, and automatically wrap customizations in LazyCommands.
  • Created unit tests for LazyInitEmitter.

Auto-generated plugin registry

  • Created new script to auto-generate plugins registry, and used it to migrate handlers.py monolith to a plugin-events registry (script not committed to repo. It's a one-time use though).
  • Committed auto-generated plugins registry.
  • For customizations that register against building-command-table.main and other events, split their registrations so that a single function contains registrations against building-command-table.main, and another function contains all other registrations.

Lazy loading

  • Use LazyInitEmitter and pass it to Session in botocore.
  • Updated plugin.py to use LazyInitEmitter as a default event emitter instead of HierarchicalEmitter.
  • Update PyInstaller hook so that it uses the plugins registry to dynamically include lazy-loaded plugins in built exexutables.
  • Update handlers.py so it no longer imports and registers all plugins. It uses LazyInitEmitter to facilitate lazy loading of all plugins.
  • Update existing tests that depended on botocore's builtin event handlers (in awscli/botocore/handlers.py) to be registered before event handlers defined in the tests, to use register_last instead of register, to preserve behavior of the handlers defined in the tests to be invoked after the builtin handlers in botocore.
  • Add missing test cases to tests/functional/test_lazy.py.
  • Add changelog entry.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

aemous added 22 commits May 8, 2026 12:58
…and merge two for-loops in lazy_emitter.py for improved readability.
… portable in preparation to migrate it to a separate repository.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant