Skip to content

Docs: Add concurrent writes guide to write documentation - #4282

Open
jey1987-cmd wants to merge 1 commit into
apache:mainfrom
jey1987-cmd:docs-concurrent-writes
Open

jey1987-cmd wants to merge 1 commit into
apache:mainfrom
jey1987-cmd:docs-concurrent-writes

Conversation

@jey1987-cmd

Copy link
Copy Markdown

Rationale for this change

The commit retry and concurrency validation added in #3320 are described in the configuration reference, but the write documentation doesn't explain what happens when several writers commit to one table. This adds a "Concurrent writes" section under "Write to a table".

Related to #1008.

What changes

A new "Concurrent writes" section in mkdocs/docs/api.md covering:

  • how each operation behaves on a concurrent commit (append, overwrite and delete, metadata-only changes);
  • how a retry works: backoff with jitter, the retry and timeout limits, the stable snapshot ID, cleanup of uncommitted files, and that a failed transaction cannot be reused;
  • how the isolation levels decide what counts as a conflict;
  • an example of handling CommitFailedException and ValidationException, and of setting the retry property;
  • a short list of ways to reduce conflicts.

The statements were checked against Transaction.commit_transaction and _SnapshotProducer._validate_concurrency on current main. Note: #4023 would change the last-attempt and cleanup behavior described here; this section would need a small update if it merges first.

Are these changes tested?

Documentation only. codespell passes on the changed file. I could not run the docs build or markdownlint locally; relying on CI.

Are there any user-facing changes?

Documentation only.

This section was drafted with Claude Code (Claude Sonnet 5.5). I reviewed it and checked it against the code before submitting.

🤖 Generated with Claude Code

Explain how append, overwrite and delete behave on a concurrent commit, how
commit retries work, how isolation levels decide conflicts, and how to handle
CommitFailedException and ValidationException.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant