[BUG] Environment can produce indirect dependency cycles due to extras

[camblomquist]

Describe the bug

When scanning the requires of a given package, cyclonedx-python appears to include a given package in dependsOn if that package exists in the environment even if the extra was not actually specified when installing the package. This can create indirect dependency cycles in the SBOM which tools like Sunshine do not handle well.

To Reproduce

This is easiest to see in a package like matplotlib where it has dependencies that will have some test or plot extra that declare matplotlib as a dependency. Install matplotlib into an empty virtual environment and generate an SBOM from that environment. matplotlib will be listed as a dependency in multiple components.

Expected behavior

The SBOM only includes includes a dependency in dependsOn when it's known that the dependency was requested by the component.

Environment

• cyclonedx-py version: 7.3.1
• Python version: 3.10.12
• OS: Ubuntu Linux

Additional context

This isn't a trivial problem to solve. A virtual environment doesn't appear to expose a means of querying what extras were chosen for a given package. One could choose a heuristic like "only include the dependencies if all of the dependencies specified by that extra exist in the environment," but that heuristic fails in the MRE: matplotlib depends on fonttools which has a plot extra whose only requirement is matplotlib.

Contribution

• I am willing to provide a fix
• I will wait until somebody else fixes it

If I had a better understanding of the internals, I might have been able to provide a fix but the nature of this problem might require more discussion.

[jkowalleck]

this seam to so be a very interesting report.

could you provide/pullrequest a testbed to showcase this behaviour? testbeds shall to here: https://github.com/CycloneDX/cyclonedx-python/tree/main/tests/_data/infiles/environment

you could use this as an example: https://github.com/CycloneDX/cyclonedx-python/tree/main/tests/_data/infiles/environment/with-extras

[jkowalleck]

matplotlib depends on fonttools which has a plot extra whose only requirement is matplotlib.

dont see any plot extra in https://github.com/matplotlib/matplotlib/blob/main/pyproject.toml.

and i see fonttools as a mandatory runtime dependency as of https://github.com/matplotlib/matplotlib/blob/a99bc61d82b76eab80509a57e39fa20a556d1985/pyproject.toml#L35

[jkowalleck]

hm, gave this topic a quick thought, my current ideas:
we should not care how a package was installed - per (extras) dependency, or manually - all we should care about:

• does a package A declare any dependency to package B, and is this package B installed?
  • it yes, then A depends on B in the SBOM
  • else, A does not depend on B in the SBOM
• is a package C installed any other package declares a dependency on it
  • if true: all other installed packages that depend on C shall have a dependency in t the SBOM.
  • else: no packages hall depend on C in the SBOM

we dont care for extras - they are pure install-time information. package managers might care,
but the environment analysis does not - it simply checks what is there and weather a dependency was declared.

[camblomquist]

matplotlib depends on fonttools which has a plot extra whose only requirement is matplotlib.

dont see any plot extra in https://github.com/matplotlib/matplotlib/blob/main/pyproject.toml.

and i see fonttools as a mandatory runtime dependency as of https://github.com/matplotlib/matplotlib/blob/a99bc61d82b76eab80509a57e39fa20a556d1985/pyproject.toml#L35

I realized while writing that tracking the cycles would be hard to follow so I apologize for not trying to fix that.

matplotlib has the hard dependency on fonttools.

In fonttools, matplotlib is specified as a dependency in the plot extra https://github.com/fonttools/fonttools/blob/e74dcee6b032586c1b15f8481bd341bc9e3b4a0b/setup.py#L156

This shows up in the METADATA for fonttools as Requires-Dist: matplotlib; extra == "plot"

There are a few cycles like this specifically with matplotlib's dependencies. fonttools just happened to be one that made it easy to describe how the heuristic would fail.

[jkowalleck]

anyway, the original request was about indirect dependency cycles.
lets see if there is any bug here.