[Modular] update the auto pipeline blocks doc#13148
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| ``` | ||
|
|
||
| It is **very** important to include a `description` to avoid any confusion over how to run a block and what inputs are required. While [`~modular_pipelines.AutoPipelineBlocks`] are convenient, it's conditional logic may be difficult to figure out if it isn't properly explained. | ||
| It is **very** important to include a `description` to avoid any confusion over how to run a block and what inputs are required. While [`~modular_pipelines.AutoPipelineBlocks`] are convenient, its conditional logic may be difficult to figure out if it isn't properly explained. |
There was a problem hiding this comment.
Should we consider providing some kinda of templating in this case? DOesn't have to be in this PR, though.
|
|
||
| ```py | ||
| auto_blocks.get_execution_blocks("mask") | ||
| auto_blocks.get_execution_blocks(mask=True) |
There was a problem hiding this comment.
Out of curiosity. What was the reasoning behind shifting towards a flag based configuration as opposed to strings (mask=True as opposed to just "mask")?
There was a problem hiding this comment.
It supports conditions based on value, e.g. get_execution_blocks(type="a") — AutoPipeline was quite limited for more than 2 workflows beause it only supported a single trigger keyword. I wanted a class that can support more flexible conditional paths where we're able to traverse into each one. It was also the easiest to implement this way: https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/modular_pipeline.py#L704
We could probably extend AutoPipeline to support trigger keyward sets though, it's the most common use cases we have
sayakpaul
left a comment
sayakpaul
left a comment
There was a problem hiding this comment.
Super neat features. I would like to write a test suite around conditional blocks and the workflow selection. Is it okay if I opened a PR?
|
@sayakpaul sounds good
|
| auto_blocks.get_execution_blocks(mask=True) | ||
| ``` | ||
|
|
||
| ## ConditionalPipelineBlocks |
There was a problem hiding this comment.
is this a new type of block? 🤩
would be nice to include in this API reference section here https://moon-ci-docs.huggingface.co/docs/diffusers/pr_13148/en/api/modular_diffusers/pipeline_blocks
|
|
||
| ## ConditionalPipelineBlocks | ||
|
|
||
| [`~modular_pipelines.AutoPipelineBlocks`] is a special case of [`~modular_pipelines.ConditionalPipelineBlocks`]. While [`~modular_pipelines.AutoPipelineBlocks`] selects blocks based on whether a trigger input is provided or not, [`~modular_pipelines.ConditionalPipelineBlocks`] lets you implement a `select_block` method with any custom selection logic. |
There was a problem hiding this comment.
| [`~modular_pipelines.AutoPipelineBlocks`] is a special case of [`~modular_pipelines.ConditionalPipelineBlocks`]. While [`~modular_pipelines.AutoPipelineBlocks`] selects blocks based on whether a trigger input is provided or not, [`~modular_pipelines.ConditionalPipelineBlocks`] lets you implement a `select_block` method with any custom selection logic. | |
| [`~modular_pipelines.AutoPipelineBlocks`] is a special case of [`~modular_pipelines.ConditionalPipelineBlocks`]. While [`~modular_pipelines.AutoPipelineBlocks`] selects blocks based on whether a trigger input is provided or not, [`~modular_pipelines.ConditionalPipelineBlocks`] is able to select a block based on custom selection logic provided in the `select_block` method. |
|
|
||
| ## Workflows | ||
|
|
||
| Pipelines that contain conditional blocks ([~modular_pipelines.AutoPipelineBlocks] or [~modular_pipelines.ConditionalPipelineBlocks]) can support multiple workflows — for example, our SDXL modular pipeline supports a dozen workflows all in one pipeline. But this also means it can be confusing for users to know what workflows are supported and how to run them. For pipeline builders, it's useful to be able to extract only the blocks relevant to a specific workflow. |
There was a problem hiding this comment.
| Pipelines that contain conditional blocks ([~modular_pipelines.AutoPipelineBlocks] or [~modular_pipelines.ConditionalPipelineBlocks]) can support multiple workflows — for example, our SDXL modular pipeline supports a dozen workflows all in one pipeline. But this also means it can be confusing for users to know what workflows are supported and how to run them. For pipeline builders, it's useful to be able to extract only the blocks relevant to a specific workflow. | |
| Pipelines that contain conditional blocks ([`~modular_pipelines.AutoPipelineBlocks`] or [`~modular_pipelines.ConditionalPipelineBlocks]`) can support multiple workflows — for example, our SDXL modular pipeline supports a dozen workflows all in one pipeline. But this also means it can be confusing for users to know what workflows are supported and how to run them. For pipeline builders, it's useful to be able to extract only the blocks relevant to a specific workflow. |
| } | ||
| ``` | ||
|
|
||
| All of our built-in modular pipelines come with workflows defined. You can list them all supported workflows like this: |
There was a problem hiding this comment.
| All of our built-in modular pipelines come with workflows defined. You can list them all supported workflows like this: | |
| All of our built-in modular pipelines come with pre-defined workflows. The `available_workflows` property lists all supported workflows: |
| # ['text2image', 'image2image', 'inpaint'] | ||
| ``` | ||
|
|
||
| And retrieve the execution blocks for a specific workflow. This returns a [`~modular_pipelines.SequentialPipelineBlocks`] containing only the blocks that would actually execute for that workflow, which is useful for inspecting and debugging. |
There was a problem hiding this comment.
| And retrieve the execution blocks for a specific workflow. This returns a [`~modular_pipelines.SequentialPipelineBlocks`] containing only the blocks that would actually execute for that workflow, which is useful for inspecting and debugging. | |
| Retrieve a specific workflow with `get_workflow` to inspect and debug a specific block that executes the workflow. |
4 participants
No description provided.