Skip to content

Decorating bound methods in profiling utils breaks instance isolation #13462

@ViktoriiaRomanova

Description

@ViktoriiaRomanova

Describe the bug

Trying to profile the ltx2 pipeline encountered an error:

Image

Root Cause: Interaction of Two Factors

When wrapping pipeline component methods (e.g., scheduler.step) using a decorator (annotate), the method is retrieved as a bound method:

method = getattr(component, method_name, None)
if method is None:
continue
setattr(component, method_name, annotate(method, label))

The decorator then wraps this bound method, capturing its __self__ (the original scheduler instance) inside the closure.

As a result:

  • The wrapped step becomes a plain function, not a descriptor.
  • The original self is frozen inside the wrapper.

Later, when duplicating the scheduler (a peculiarity of the ltx2 pipeline):

audio_scheduler = copy.deepcopy(self.scheduler)

the wrapped function is copied as-is, including the captured reference to the original scheduler.

As a result: calling audio_scheduler.step(...) actually executes self.scheduler.step(...):

latents = self.scheduler.step(noise_pred_video, t, latents, return_dict=False)[0]
# NOTE: for now duplicate scheduler for audio latents in case self.scheduler sets internal state in
# the step method (such as _step_index)
audio_latents = audio_scheduler.step(noise_pred_audio, t, audio_latents, return_dict=False)[0]

This leads to:

  • Shared internal state (_step_index) between supposedly independent schedulers
  • Incorrect stepping behavior
  • Index out-of-bounds errors during inference

The following shows debugger output from the first loop iteration, right after executing audio_scheduler.step():

Image

The object ID inside both step calls matches the ID of self.scheduler:

Image

Fix Proposal

Decorate the unbound function and rebind it:

    for component_name, method_name, label in annotations:
        component = getattr(pipe, component_name, None)
        if component is None:
            continue
        component_cls = type(component)
        method = getattr(component_cls, method_name, None)
        if method is None:
            continue
        setattr(component, method_name, annotate(method, label).__get__(component, component_cls))

I can open a PR if this solution is suitable.

Reproduction

Run ltx2 profiling for any number of steps:
python examples/profiling/profiling_pipelines.py --pipeline ltx2 --mode eager --num_steps 2

Logs

System Info

Diffusers version: 0.38.0.dev0

  • Platform: Linux-6.8.0-84-generic-x86_64-with-glibc2.39
  • Running on Google Colab?: No
  • Python version: 3.12.3
  • PyTorch version (GPU?): 2.10.0+cu128 (True)
  • Flax version (CPU?/GPU?/TPU?): not installed (NA)
  • Jax version: not installed
  • JaxLib version: not installed
  • Huggingface_hub version: 1.9.2
  • Transformers version: 5.5.1
  • Accelerate version: 1.13.0
  • PEFT version: not installed
  • Bitsandbytes version: not installed
  • Safetensors version: 0.7.0
  • xFormers version: not installed
  • Accelerator: NVIDIA A100-SXM4-80GB, 81920 MiB

Who can help?

@sayakpaul, @dg845

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds-code-exampleWaiting for relevant code example to be providedpipelines

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions