Skip to content

fix(relay): Refresh task-local organization options from the shared c… - #127929

Closed
ranmori wants to merge 1 commit into
getsentry:masterfrom
ranmori:fix/relay-stale-org-options
Closed

ranmori wants to merge 1 commit into
getsentry:masterfrom
ranmori:fix/relay-stale-org-options

Conversation

@ranmori

@ranmori ranmori commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Organization option changes, such as new data scrubbing fields, never reach Relay when the project config is rebuilt by a long-running taskworker. The worker keeps rebuilding configs from the options it read first, until the process is replaced.

Fixes #127554

compute_projectkey_config calls OrganizationOption.objects.reload_task_local_cache() before building a config (added in #117602), but that method only drops the task-local entry when the shared cache is empty. Saving an option goes through reload_cache, which repopulates the shared cache with the new values, so in the worker the shared cache is never empty and the stale local entry is kept. Configs are then rebuilt, and later invalidations rewrite them, with the old options.

This makes the method do what its comment describes: copy the shared cache into the local cache when it is populated, and drop the local entry otherwise so the next read queries the database.

An alternative would be clearing the whole local option cache at task boundaries, the way request_finished does for web requests. That is broader and touches every task, so I kept this to the existing reload hook.

ProjectOption has the same kind of process-local cache and no equivalent reload in this task, so project-level settings may be affected in the same way. I left that out of scope here and can follow up if wanted.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

…ache

reload_task_local_cache only dropped a task's local copy of an
organization's options when the shared cache was empty. Saving an option
repopulates the shared cache with the new values, so a long-running
worker kept the values it read first and rebuilt project configs from
them. Data scrubbing changes then never reached Relay until the worker
was replaced.

Copy the shared cache into the local cache when it is populated, as the
existing comment describes, and drop the local copy otherwise.

Fixes getsentry#127554
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 17, 2026
@ranmori
ranmori marked this pull request as ready for review October 17, 2026 08:32
@ranmori
ranmori requested a review from a team as a code owner October 17, 2026 08:32
@ranmori
ranmori marked this pull request as draft October 17, 2026 08:35
@marius-mbravio

Copy link
Copy Markdown

Thanks for picking this up, @ranmori. Your analysis of the organization side matches what we saw when we filed #127554.

We need project-level settings fixed too. relay/datascrubbing.py merges project options such as sentry:relay_pii_config and sentry:sensitive_fields with the organization's, and ProjectOption has the same process-local cache with no refresh in compute_projectkey_config. We've opened #127930, which moves the corrected reload_task_local_cache into OptionManager and calls it for both, with tests for each. Whether it supersedes this PR, or the project half is folded in here, is up to the Sentry team.

@ranmori

ranmori commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @marius-mbravio. #127930 includes this change and covers project options as well, so I'll close this in favour of it to keep review in one place.

@ranmori ranmori closed this Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Data scrubbing changes never reach Relay: taskworker rebuilds project configs from stale organization options

2 participants