Fix(backend & frontend): Present Flux.2 reference images in the same order as shown in the frontend#8923
Fix(backend & frontend): Present Flux.2 reference images in the same order as shown in the frontend#8923lstein wants to merge 3 commits intoinvoke-ai:mainfrom
Conversation
* Initial plan * Fix: Preserve insertion order of items in CollectInvocation Items directly connected to a CollectInvocation were previously sorted by (iteration_path, source_node_id). Since source_node_id is a random UUID, this caused non-deterministic ordering for non-iterated direct connections (e.g. Flux.2 reference images). Fix: use insertion index as the secondary sort key instead of source_node_id, preserving the left-to-right visual order from the frontend. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
* Initial plan * Add drag-and-drop reordering for reference images Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix: disable native img drag in RefImagePreview to allow DnD reordering Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Add info log when ref images are reordered Co-authored-by: lstein <111189+lstein@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
|
Instead of doing it this way, why not make a "linked" version of the collect node that allows for chaining? An image collection linked node already exists in https://github.com/skunkworxdark/collection_tools. |
@JPPhoto Are you suggesting I copy Skunkworx's linked node code into the main repo? Will this cause any conflicts or confusion? |
I think all of the linked nodes are great, I use them all of the time, and they enforce ordering. So (with permission) I think they should all be in the main repo, then the frontend could chain them in the graph builder to create proper ordering. |
Summary
The Flux.2 klein models allow you to pass multiple reference images and refer to them explicitly as "image 1", "image 2", etc. However, the
CollectInvocationitems were being sorted by(iteration_path, source_node_id)when building collections. This has no relationship to the left to right visual order in which the images are displayed in the front end. As a result, referring to "image 1" in the prompt would not always correspond to the leftmost image.This PR fixes the problem by sorting the collection by its
input_edgesindex, which corresponds to the left to right ordering of the images visually. As an added bonus, I've added a drag-and-drop function that lets you change the order of the reference images on the front end.Related Issues / Discussions
QA Instructions
You should see that the generated image more-or-less follows the visual order of the images. However, it appears that the model pays more attention to the first image and progressively less to subsequent ones, so in general the leftmost part of the generated image will correspond well to image 1 and the other two panels will suffer semantic leakage.
You can also try prompts like "The subject of image 1 is shaking hands with the subject of image 2."
Merge Plan
Simple merge
Checklist
What's Newcopy (if doing a release after this PR)