Skip to content

fix: skip marker-constrained uninstalled packages in Python requirements #459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ruromero wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: skip marker-constrained uninstalled packages in Python requirements #459

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions src/providers/python_controller.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ export default class Python_controller {

/**
* Parse the requirements.txt file using tree-sitter and return structured requirement data.
* @return {Promise<{name: string, version: string|null}[]>}
* @return {Promise<{name: string, version: string|null, hasMarker: boolean}[]>}
*/
async #parseRequirements() {
const content = fs.readFileSync(this.pathToRequirements).toString();
Expand All @@ -109,7 +109,8 @@ export default class Python_controller {
const version = versionMatches.length > 0
? versionMatches[0].captures.find(c => c.name === 'version').node.text
: null;
return { name, version };
const hasMarker = reqNode.children.some(c => c.type === 'marker_spec');
return { name, version, hasMarker };
}));
}

Expand Down Expand Up @@ -224,7 +225,10 @@ export default class Python_controller {
CachedEnvironmentDeps[packageName.replace("_", "-")] = pipDepTreeEntryForCache
})
}
parsedRequirements.forEach(({ name: depName, version: manifestVersion }) => {
parsedRequirements.forEach(({ name: depName, version: manifestVersion, hasMarker }) => {
if(hasMarker && CachedEnvironmentDeps[depName.toLowerCase()] === undefined) {
return
}
Comment on lines +229 to +231
Copy link
Copy Markdown

@qodo-code-review qodo-code-review bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Skips required marker deps 🐞 Bug ≡ Correctness

In Python_controller#getDependenciesImpl, any requirement with a marker is silently skipped when not
found in the installed-packages cache, without evaluating whether the marker expression is false for
the current environment. This can drop required dependencies (marker evaluates true but package is
missing) and produce an incomplete SBOM instead of failing fast.
Agent Prompt 
### Issue description
Marker-constrained requirements are skipped solely based on absence from the installed package cache (`pip show`/`pipdeptree`), but the code never checks whether the marker actually evaluates to `false` for the current environment. This can silently omit dependencies that should apply (marker evaluates `true`) but are missing due to an incomplete/broken environment.

### Issue Context
- `#parseRequirements()` currently returns `{ name, version, hasMarker }`, but not the marker expression.
- `#getDependenciesImpl()` uses `hasMarker && notInstalled => return`.

### Fix Focus Areas
- src/providers/python_controller.js[98-114]
- src/providers/python_controller.js[228-246]

### Suggested fix
1. Extend `#parseRequirements()` to also return the marker text (e.g., locate the `marker_spec` child node and capture its `.text`).
2. When `hasMarker && notInstalled`, evaluate the marker against the current environment and:
   - If marker evaluates **false**: skip (current intended behavior).
   - If marker evaluates **true**: keep current behavior for missing deps (error out via existing flow), so SBOM isn’t silently incomplete.
3. If marker evaluation cannot be performed (e.g., missing evaluator), fail closed (do not skip) or gate skipping behind an explicit option.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

if(matchManifestVersions === "true" && manifestVersion != null) {
let installedVersion
if(CachedEnvironmentDeps[depName.toLowerCase()] !== undefined) {
Expand Down
38 changes: 38 additions & 0 deletions test/providers/python_pip.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,41 @@ suite('testing the python-pip data provider with virtual environment', () => {
})

}).beforeAll(() => {clock = useFakeTimers(new Date('2023-10-01T00:00:00.000Z'))}).afterAll(()=> clock.restore());

suite('testing python-pip PEP 508 marker handling', () => {
const markerTestCase = 'pip_requirements_txt_marker_skip'

/** Verify that packages with environment markers (PEP 508) that are not installed
* in the current environment are silently skipped, while marker-constrained
* packages that ARE installed are still included in the SBOM. */
test('verify marker-constrained uninstalled packages are skipped in component analysis', async () => {
// given: pip environment where only six and certifi are installed (pywin32 is Windows-only)
const pipFreezeOutput = 'six==1.16.0\ncertifi==2023.7.22\n'
const pipShowOutput =
'Name: certifi\nVersion: 2023.7.22\nSummary: Python package for providing Mozilla\'s CA Bundle.\nRequires: \nRequired-by: ' +
'\n---\n' +
'Name: six\nVersion: 1.16.0\nSummary: Python 2 and 3 compatibility utilities\nRequires: \nRequired-by: '

process.env['TRUSTIFY_DA_PIP_FREEZE'] = Buffer.from(pipFreezeOutput).toString('base64')
process.env['TRUSTIFY_DA_PIP_SHOW'] = Buffer.from(pipShowOutput).toString('base64')

try {
// when: component analysis is run against a manifest with a Windows-only marker package
let expectedSbom = fs.readFileSync(`test/providers/tst_manifests/pip/${markerTestCase}/expected_component_sbom.json`).toString().trim()
expectedSbom = JSON.stringify(JSON.parse(expectedSbom))

let result = await pythonPip.provideComponent(`test/providers/tst_manifests/pip/${markerTestCase}/requirements.txt`, {})

// then: SBOM contains six and certifi but not pywin32
expect(result).to.deep.equal({
ecosystem: 'pip',
contentType: 'application/vnd.cyclonedx+json',
content: expectedSbom
})
} finally {
delete process.env['TRUSTIFY_DA_PIP_FREEZE']
delete process.env['TRUSTIFY_DA_PIP_SHOW']
}
}).timeout(10000)

}).beforeAll(() => clock = useFakeTimers(new Date('2023-10-01T00:00:00.000Z'))).afterAll(() => clock.restore());
48 changes: 48 additions & 0 deletions ...providers/tst_manifests/pip/pip_requirements_txt_marker_skip/expected_component_sbom.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"metadata": {
"timestamp": "2023-10-01T00:00:00.000Z",
"component": {
"name": "default-pip-root",
"version": "0.0.0",
"purl": "pkg:pypi/default-pip-root@0.0.0",
"type": "application",
"bom-ref": "pkg:pypi/default-pip-root@0.0.0"
}
},
"components": [
{
"name": "certifi",
"version": "2023.7.22",
"purl": "pkg:pypi/certifi@2023.7.22",
"type": "library",
"bom-ref": "pkg:pypi/certifi@2023.7.22"
},
{
"name": "six",
"version": "1.16.0",
"purl": "pkg:pypi/six@1.16.0",
"type": "library",
"bom-ref": "pkg:pypi/six@1.16.0"
}
],
"dependencies": [
{
"ref": "pkg:pypi/default-pip-root@0.0.0",
"dependsOn": [
"pkg:pypi/certifi@2023.7.22",
"pkg:pypi/six@1.16.0"
]
},
{
"ref": "pkg:pypi/certifi@2023.7.22",
"dependsOn": []
},
{
"ref": "pkg:pypi/six@1.16.0",
"dependsOn": []
}
]
}
3 changes: 3 additions & 0 deletions test/providers/tst_manifests/pip/pip_requirements_txt_marker_skip/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
six==1.16.0
certifi==2023.7.22 ; python_version >= "3"
pywin32==306 ; platform_system == "Windows"
Loading