Skip to content

Set consuming_received_data=True during bundle setup#38136

Draft
hjtran wants to merge 1 commit intoapache:masterfrom
hjtran:pr/consuming-received-data-init
Draft

Set consuming_received_data=True during bundle setup#38136
hjtran wants to merge 1 commit intoapache:masterfrom
hjtran:pr/consuming-received-data-init

Conversation

@hjtran
Copy link
Copy Markdown
Contributor

@hjtran hjtran commented Apr 9, 2026

Problem

If an sdk worker is processing a stage with long bundle startup times, it's not currently possible for a runner to tell whether the worker is idle and ready for work or if it's still running setup_bundle. Consider the following order of events:

Runner sends ProcessBundleRequest
Runner sends data on the input data channel
Runner sends a ProcessBundleProgressRequest to see if the worker is done and ready for more inputs

If the worker returns a ProcessBundleProgressResponse with consuming_received_data=False, the worker might actually still be just running setup_bundle. In the worse case, a runner may misinterpret this signal and repeatedly queue up a large queue of work for the worker that it could've distributed over many workers

Summary of changes

  • Initialize consuming_received_data to True at the start of process_bundle, so runners polling ProcessBundleProgress can distinguish "still setting up / actively processing" from "idle and caught up with all received data."
  • Previously both setup and idle reported False, making them indistinguishable.
  • After setup completes, the flag is set to False before entering the data processing loop.

The semantic meaning of consuming_received_data is not really respected now (for example, a runner could send a ProcessBundleRequest and never send any input data and yet the worker would briefly report that it's "consuming received data" when it has received no data), but I think this better implements the intent of the field.

Without this change, it's impossible for a runner to definitively know whether a python sdk worker is actually idle

Runners polling ProcessBundleProgress could not distinguish between
"still setting up" and "idle/caught up" since both reported
consuming_received_data=False. Now the flag is True from the start of
process_bundle through setup, then set to False once setup completes
and before entering the data processing loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request improves the observability of bundle processing by refining the state reporting mechanism. By explicitly setting the consumption flag during the setup phase, it resolves ambiguity for runners that monitor progress, ensuring that an idle status is only reported when the worker has truly finished processing all received data.

Highlights

  • State tracking improvement: Initialized the consuming_received_data flag to True at the start of bundle processing to explicitly signal that the worker is in a setup or active state.
  • Runner visibility: Enabled runners to distinguish between the initial setup phase and an idle state where all received data has been processed.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant