-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Description
As identified in PR #32 review, the large number of generated models could benefit from lazy loading to reduce memory usage, especially for users who only use a subset of the API.
Current State
- All generated models are imported at module initialization
- Memory footprint includes all models even if unused
- Impacts startup time and memory usage
Proposed Solution
Implement lazy loading using import hooks and a registry pattern:
class LazyModelLoader:
def __init__(self):
self._models = {}
self._loaded = set()
def __getattr__(self, name):
if name not in self._loaded:
# Dynamically import the model
module = importlib.import_module(f'esologs._generated.{name}')
self._models[name] = module
self._loaded.add(name)
return self._models[name]
# Usage
models = LazyModelLoader()
# Model only loaded when accessed
character_data = models.get_character_by_idImplementation Ideas
- Use Python's import hooks for transparent lazy loading
- Track which models are commonly used together
- Provide option to preload common models
- Consider using
__init__.pywith__getattr__for module-level lazy loading
Benefits
- Reduced memory footprint for applications using subset of API
- Faster initial import time
- No change to public API
Performance Metrics to Track
- Import time before/after
- Memory usage for minimal usage patterns
- Memory usage for full API usage
Considerations
- Ensure thread safety for lazy loading
- Consider impact on type checking and IDE support
- May need to update documentation about import behavior
References
- PR Release v0.2.0b1 - First Beta Release #32 review feedback
- Python import hooks documentation
Metadata
Metadata
Assignees
Labels
No labels