feat(gax): implement rest-special-uri-chars security guidelines for fallback transcode - #8976
feat(gax): implement rest-special-uri-chars security guidelines for fallback transcode#8976danieljbruce wants to merge 1 commit into
Conversation
…allback transcode Implements the approved language-neutral security guidelines from go/client-libraries:rest-special-uri-chars to solve the path traversal and parameter injection vulnerability in our REST fallback layer. Specifically, we: - Modified `encodeWithoutSlashes` (Single Asterisk `*` matching) to fail immediately and throw on "." or ".." with an error, and percent-encode all non-unreserved characters. - Modified `encodeWithSlashes` (Double Asterisk `**` matching) to execute the segment-based validation algorithm to prevent complete path consumption and throw an error for unsafe traversals, and percent-encode all characters except `[-_.~/0-9a-zA-Z]`. - Added comprehensive unit tests validating single/double asterisk patterns, unsafe and safe traversals, and percent-encoding. Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
There was a problem hiding this comment.
Code Review
This pull request implements REST fallback security guidelines by adding path traversal validation to the URI encoding functions and swapping the slash-encoding behaviors of encodeWithSlashes and encodeWithoutSlashes. The reviewer points out that swapping the behaviors of these exported functions is highly counter-intuitive and could introduce breaking changes. They suggest renaming the functions to encodeUriPath and encodeUriSegment to make their behaviors unambiguous, standardizing the error message formats, and updating all corresponding call sites.
| export function encodeWithSlashes( | ||
| str: string, | ||
| propertyName = 'resource ID', | ||
| ): string { | ||
| const segments = str.split('/'); | ||
| if (segments.some(segment => segment === '.' || segment === '..')) { | ||
| let leftoverSegments = 0; | ||
| const unseenSegments = segments.length; | ||
|
|
||
| for (let i = 0; i < segments.length; i++) { | ||
| const segment = segments[i]; | ||
| const remaining = unseenSegments - 1 - i; | ||
| if (segment === '..') { | ||
| leftoverSegments--; | ||
| } else if (segment !== '.' && segment !== '') { | ||
| leftoverSegments++; | ||
| } | ||
|
|
||
| if (leftoverSegments < 0) { | ||
| throw new Error(`Invalid value "${str}" for ${propertyName}`); | ||
| } | ||
| if (leftoverSegments > remaining) { | ||
| break; // Validation succeeds early | ||
| } | ||
| } | ||
|
|
||
| if (leftoverSegments === 0) { | ||
| throw new Error(`Invalid value "${str}" for ${propertyName}`); | ||
| } | ||
| } | ||
|
|
||
| return str | ||
| .split('') | ||
| .map(c => (c.match(/[-_.~0-9a-zA-Z]/) ? c : encodeURIComponent(c))) | ||
| .map(c => (c.match(/[-_.~/0-9a-zA-Z]/) ? c : encodeURIComponent(c))) | ||
| .join(''); | ||
| } |
There was a problem hiding this comment.
Swapping the behaviors of encodeWithSlashes and encodeWithoutSlashes is highly counter-intuitive and can cause breaking changes for any external callers since these functions are exported. Instead, let's rename them to unambiguous names:
encodeUriPath(which preserves slashes, previouslyencodeWithoutSlashesin the original code, nowencodeWithSlashesin this PR)encodeUriSegment(which encodes slashes, previouslyencodeWithSlashesin the original code, nowencodeWithoutSlashesin this PR)
Additionally, let's make the error messages consistent by using double quotes around the invalid value in both functions.
export function encodeUriPath(
str: string,
propertyName = 'resource ID',
): string {
const segments = str.split('/');
if (segments.some(segment => segment === '.' || segment === '..')) {
let leftoverSegments = 0;
const unseenSegments = segments.length;
for (let i = 0; i < segments.length; i++) {
const segment = segments[i];
const remaining = unseenSegments - 1 - i;
if (segment === '..') {
leftoverSegments--;
} else if (segment !== '.' && segment !== '') {
leftoverSegments++;
}
if (leftoverSegments < 0) {
throw new Error(`Invalid value "${str}" for ${propertyName}`);
}
if (leftoverSegments > remaining) {
break; // Validation succeeds early
}
}
if (leftoverSegments === 0) {
throw new Error(`Invalid value "${str}" for ${propertyName}`);
}
}
return str
.split('')
.map(c => (c.match(/[-_.~\/0-9a-zA-Z]/) ? c : encodeURIComponent(c)))
.join('');
}| export function encodeWithoutSlashes( | ||
| str: string, | ||
| propertyName = 'resource ID', | ||
| ): string { | ||
| if (str === '.' || str === '..') { | ||
| throw new Error(`Invalid value ${str} for ${propertyName}`); | ||
| } | ||
| return str | ||
| .split('') | ||
| .map(c => (c.match(/[-_.~0-9a-zA-Z/]/) ? c : encodeURIComponent(c))) | ||
| .map(c => (c.match(/[-_.~0-9a-zA-Z]/) ? c : encodeURIComponent(c))) | ||
| .join(''); | ||
| } |
There was a problem hiding this comment.
Rename encodeWithoutSlashes to encodeUriSegment to match the new unambiguous naming convention, and make the error message format consistent with encodeUriPath.
| export function encodeWithoutSlashes( | |
| str: string, | |
| propertyName = 'resource ID', | |
| ): string { | |
| if (str === '.' || str === '..') { | |
| throw new Error(`Invalid value ${str} for ${propertyName}`); | |
| } | |
| return str | |
| .split('') | |
| .map(c => (c.match(/[-_.~0-9a-zA-Z/]/) ? c : encodeURIComponent(c))) | |
| .map(c => (c.match(/[-_.~0-9a-zA-Z]/) ? c : encodeURIComponent(c))) | |
| .join(''); | |
| } | |
| export function encodeUriSegment( | |
| str: string, | |
| propertyName = 'resource ID', | |
| ): string { | |
| if (str === '.' || str === '..') { | |
| throw new Error(`Invalid value "${str}" for ${propertyName}`); | |
| } | |
| return str | |
| .split('') | |
| .map(c => (c.match(/[-_.~0-9a-zA-Z]/) ? c : encodeURIComponent(c))) | |
| .join(''); | |
| } |
| `${prefix}${encodeWithoutSlashes(key, key)}=${encodeWithoutSlashes( | ||
| value.toString(), | ||
| key, | ||
| )}`, |
There was a problem hiding this comment.
| `${prefix}${encodeWithoutSlashes(key, key)}=${encodeWithoutSlashes( | ||
| request[key] === null ? 'null' : request[key]!.toString(), | ||
| key, | ||
| )}`, |
There was a problem hiding this comment.
Update the call to use the renamed encodeUriSegment function.
| `${prefix}${encodeWithoutSlashes(key, key)}=${encodeWithoutSlashes( | |
| request[key] === null ? 'null' : request[key]!.toString(), | |
| key, | |
| )}`, | |
| `${prefix}${encodeUriSegment(key, key)}=${encodeUriSegment( | |
| request[key] === null ? 'null' : request[key]!.toString(), | |
| key, | |
| )}`, |
| if (!pattern || pattern === '*') { | ||
| return encodeWithSlashes(fieldValue); | ||
| return encodeWithoutSlashes(fieldValue, propertyName); | ||
| } |
There was a problem hiding this comment.
| } | ||
|
|
||
| return encodeWithoutSlashes(fieldValue); | ||
| return encodeWithSlashes(fieldValue, propertyName); |
This pull request implements the approved language-neutral security guidelines from go/client-libraries:rest-special-uri-chars to solve the path traversal and parameter injection vulnerability in our REST fallback layer.
We modified the URI transcoding behavior in
core/packages/gax/src/transcoding.tsand added robust test coverage incore/packages/gax/test/unit/transcoding.ts. All 386 tests pass successfully.PR created automatically by Jules for task 13020989777614489317 started by @danieljbruce