Skip to content

storage: fix 404 when setting headers or metadata on keys with special characters - #922

Merged
pierre-emmanuelJ merged 4 commits into
masterfrom
pej/sc-201630/cli-setting-headers-on-an-existing-object-reports-a-404
Oct 2, 2026
Merged

pierre-emmanuelJ merged 4 commits into
masterfrom
pej/sc-201630/cli-setting-headers-on-an-existing-object-reports-a-404

Conversation

@pierre-emmanuelJ

Copy link
Copy Markdown
Member

Description

  • Problem: exo storage headers add / metadata add reported a 404 on an object that exists when its key holds a percent-encoded sequence (e.g. a literal %26) or a +. These commands rewrite the object with an in-place copy, and the copy source was sent unencoded, so the server decoded it into another key.
  • Fix: the copy source is now URL-encoded for headers and metadata updates, + included. storage copy / move already encoded it but left + as is and hit the same 404 on such keys; they are fixed too.
  • Compatibility: no behaviour change for keys that already worked; no flag or output change.

[sc-201630]

Checklist

(For exoscale contributors)

  • Changelog updated (under Unreleased block, and add the Pull Request #number for each bit you add to the CHANGELOG.md)
  • Testing

Testing

Unit tests, a new e2e scenario (storage_headers_special_chars.txtar), and a manual run against SOS: 404 with the binary built from master, success with this branch.


Note

AI assistance: code, tests, PR description.

🤖 Generated with Claude Code

pierre-emmanuelJ and others added 2 commits October 2, 2026 12:32
Headers and metadata are updated with an in-place CopyObject whose copy
source was sent unencoded. The server URL-decodes it, so a key holding a
percent-encoded sequence such as `%26` pointed at another object and the
command reported a 404 although the object exists.

The copy source is now built by the helper `copy` and `move` already
use. That helper left `+` as is, which the server reads as a space: it
is encoded too, which fixes the same 404 on `copy` and `move`.

[sc-201630]

AI-assisted: true
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AI-assisted: true
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pierre-emmanuelJ
pierre-emmanuelJ requested review from a team October 2, 2026 12:36
Comment thread pkg/storage/sos/move.go Outdated
Comment on lines +125 to +127
// copySource returns the URL-encoded source of a copy operation. The server
// decodes it the way it decodes a query string, so "+" has to be encoded too
// or it would be read as a space.

@kobajagi kobajagi Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would url.QueryEscape do the same thing?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and it is simpler: url.QueryEscape encodes + itself, so the manual replace is gone. The one difference is that a space is now sent as + instead of %20, which the server decodes back to a space. Changed, and the e2e scenario (key with &, space, + and %26) passes against SOS.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pierre-emmanuelJ and others added 2 commits October 2, 2026 14:07
…etting-headers-on-an-existing-object-reports-a-404

# Conflicts:
#	CHANGELOG.md
AI-assisted: true
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pierre-emmanuelJ
pierre-emmanuelJ merged commit 4b1858e into master Oct 2, 2026
7 checks passed
@pierre-emmanuelJ
pierre-emmanuelJ deleted the pej/sc-201630/cli-setting-headers-on-an-existing-object-reports-a-404 branch October 2, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants