Skip to content

feat(gax): implement rest-special-uri-chars security guidelines for fallback transcode - #8976

Draft
danieljbruce wants to merge 1 commit into
mainfrom
rest-special-uri-chars-13020989777614489317
Draft

feat(gax): implement rest-special-uri-chars security guidelines for fallback transcode#8976
danieljbruce wants to merge 1 commit into
mainfrom
rest-special-uri-chars-13020989777614489317

Conversation

@danieljbruce

Copy link
Copy Markdown
Contributor

This pull request implements the approved language-neutral security guidelines from go/client-libraries:rest-special-uri-chars to solve the path traversal and parameter injection vulnerability in our REST fallback layer.

We modified the URI transcoding behavior in core/packages/gax/src/transcoding.ts and added robust test coverage in core/packages/gax/test/unit/transcoding.ts. All 386 tests pass successfully.


PR created automatically by Jules for task 13020989777614489317 started by @danieljbruce

…allback transcode

Implements the approved language-neutral security guidelines from go/client-libraries:rest-special-uri-chars to solve the path traversal and parameter injection vulnerability in our REST fallback layer.

Specifically, we:
- Modified `encodeWithoutSlashes` (Single Asterisk `*` matching) to fail immediately and throw on "." or ".." with an error, and percent-encode all non-unreserved characters.
- Modified `encodeWithSlashes` (Double Asterisk `**` matching) to execute the segment-based validation algorithm to prevent complete path consumption and throw an error for unsafe traversals, and percent-encode all characters except `[-_.~/0-9a-zA-Z]`.
- Added comprehensive unit tests validating single/double asterisk patterns, unsafe and safe traversals, and percent-encoding.

Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements REST fallback security guidelines by adding path traversal validation to the URI encoding functions and swapping the slash-encoding behaviors of encodeWithSlashes and encodeWithoutSlashes. The reviewer points out that swapping the behaviors of these exported functions is highly counter-intuitive and could introduce breaking changes. They suggest renaming the functions to encodeUriPath and encodeUriSegment to make their behaviors unambiguous, standardizing the error message formats, and updating all corresponding call sites.

Comment on lines +152 to 187
export function encodeWithSlashes(
str: string,
propertyName = 'resource ID',
): string {
const segments = str.split('/');
if (segments.some(segment => segment === '.' || segment === '..')) {
let leftoverSegments = 0;
const unseenSegments = segments.length;

for (let i = 0; i < segments.length; i++) {
const segment = segments[i];
const remaining = unseenSegments - 1 - i;
if (segment === '..') {
leftoverSegments--;
} else if (segment !== '.' && segment !== '') {
leftoverSegments++;
}

if (leftoverSegments < 0) {
throw new Error(`Invalid value "${str}" for ${propertyName}`);
}
if (leftoverSegments > remaining) {
break; // Validation succeeds early
}
}

if (leftoverSegments === 0) {
throw new Error(`Invalid value "${str}" for ${propertyName}`);
}
}

return str
.split('')
.map(c => (c.match(/[-_.~0-9a-zA-Z]/) ? c : encodeURIComponent(c)))
.map(c => (c.match(/[-_.~/0-9a-zA-Z]/) ? c : encodeURIComponent(c)))
.join('');
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Swapping the behaviors of encodeWithSlashes and encodeWithoutSlashes is highly counter-intuitive and can cause breaking changes for any external callers since these functions are exported. Instead, let's rename them to unambiguous names:

  • encodeUriPath (which preserves slashes, previously encodeWithoutSlashes in the original code, now encodeWithSlashes in this PR)
  • encodeUriSegment (which encodes slashes, previously encodeWithSlashes in the original code, now encodeWithoutSlashes in this PR)

Additionally, let's make the error messages consistent by using double quotes around the invalid value in both functions.

export function encodeUriPath(
  str: string,
  propertyName = 'resource ID',
): string {
  const segments = str.split('/');
  if (segments.some(segment => segment === '.' || segment === '..')) {
    let leftoverSegments = 0;
    const unseenSegments = segments.length;

    for (let i = 0; i < segments.length; i++) {
      const segment = segments[i];
      const remaining = unseenSegments - 1 - i;
      if (segment === '..') {
        leftoverSegments--;
      } else if (segment !== '.' && segment !== '') {
        leftoverSegments++;
      }

      if (leftoverSegments < 0) {
        throw new Error(`Invalid value "${str}" for ${propertyName}`);
      }
      if (leftoverSegments > remaining) {
        break; // Validation succeeds early
      }
    }

    if (leftoverSegments === 0) {
      throw new Error(`Invalid value "${str}" for ${propertyName}`);
    }
  }

  return str
    .split('')
    .map(c => (c.match(/[-_.~\/0-9a-zA-Z]/) ? c : encodeURIComponent(c)))
    .join('');
}

Comment on lines +189 to 200
export function encodeWithoutSlashes(
str: string,
propertyName = 'resource ID',
): string {
if (str === '.' || str === '..') {
throw new Error(`Invalid value ${str} for ${propertyName}`);
}
return str
.split('')
.map(c => (c.match(/[-_.~0-9a-zA-Z/]/) ? c : encodeURIComponent(c)))
.map(c => (c.match(/[-_.~0-9a-zA-Z]/) ? c : encodeURIComponent(c)))
.join('');
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Rename encodeWithoutSlashes to encodeUriSegment to match the new unambiguous naming convention, and make the error message format consistent with encodeUriPath.

Suggested change
export function encodeWithoutSlashes(
str: string,
propertyName = 'resource ID',
): string {
if (str === '.' || str === '..') {
throw new Error(`Invalid value ${str} for ${propertyName}`);
}
return str
.split('')
.map(c => (c.match(/[-_.~0-9a-zA-Z/]/) ? c : encodeURIComponent(c)))
.map(c => (c.match(/[-_.~0-9a-zA-Z]/) ? c : encodeURIComponent(c)))
.join('');
}
export function encodeUriSegment(
str: string,
propertyName = 'resource ID',
): string {
if (str === '.' || str === '..') {
throw new Error(`Invalid value "${str}" for ${propertyName}`);
}
return str
.split('')
.map(c => (c.match(/[-_.~0-9a-zA-Z]/) ? c : encodeURIComponent(c)))
.join('');
}

Comment on lines +130 to 133
`${prefix}${encodeWithoutSlashes(key, key)}=${encodeWithoutSlashes(
value.toString(),
key,
)}`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the call to use the renamed encodeUriSegment function.

Suggested change
`${prefix}${encodeWithoutSlashes(key, key)}=${encodeWithoutSlashes(
value.toString(),
key,
)}`,
`${prefix}${encodeUriSegment(key, key)}=${encodeUriSegment(
value.toString(),
key,
)}`,

Comment on lines +142 to 145
`${prefix}${encodeWithoutSlashes(key, key)}=${encodeWithoutSlashes(
request[key] === null ? 'null' : request[key]!.toString(),
key,
)}`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the call to use the renamed encodeUriSegment function.

Suggested change
`${prefix}${encodeWithoutSlashes(key, key)}=${encodeWithoutSlashes(
request[key] === null ? 'null' : request[key]!.toString(),
key,
)}`,
`${prefix}${encodeUriSegment(key, key)}=${encodeUriSegment(
request[key] === null ? 'null' : request[key]!.toString(),
key,
)}`,

Comment on lines 211 to 213
if (!pattern || pattern === '*') {
return encodeWithSlashes(fieldValue);
return encodeWithoutSlashes(fieldValue, propertyName);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the call to use the renamed encodeUriSegment function.

Suggested change
if (!pattern || pattern === '*') {
return encodeWithSlashes(fieldValue);
return encodeWithoutSlashes(fieldValue, propertyName);
}
if (!pattern || pattern === '*') {
return encodeUriSegment(fieldValue, propertyName);
}

}

return encodeWithoutSlashes(fieldValue);
return encodeWithSlashes(fieldValue, propertyName);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the call to use the renamed encodeUriPath function.

Suggested change
return encodeWithSlashes(fieldValue, propertyName);
return encodeUriPath(fieldValue, propertyName);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant