Support Custom Artifactory Repositories for Package Metadata by voidpetal · Pull Request #258 · aboutcode-org/python-inspector

voidpetal · 2026-01-21T15:31:20Z

When using --index-url with a custom Artifactory repository, dependency resolution works but the packages array comes back empty. This happens because get_pypi_data_from_purl() hardcodes https://pypi.org/pypi for the JSON API endpoint. Internal packages that don't exist on PyPI.org return 404 and are silently skipped.

The fix includes deriving the JSON API base URL from the provided repository instead of hardcoding PyPI.org
It is also necessary to match distribution files by filename (standardized per PEP 427/491) instead of full URL, since URL paths can differ between Simple API and JSON API endpoints.

pombredanne

Thanks. This PR really looks like it was in good part generated by a coding agent. I appreciate your efforts, but overall, do not force me to review code that you have not thoroughly reviewed yourself, otherwise you are effectively outsourcing AI generation review to me, a human, which could be considered bad form and not super polite or nice.

Time is a precious thing that is not a commodity for me. And reviewing AI slop is a waste of my time.

We are evolving our AI policy but at a high level:

do not use AI to generate commit messages and PR bodies messages. These are used to communicate between people, so be nice and genuine: type these yourself with your own words. Here please amend your commit messages and PR body.
adopt our style for commits messages and code. In particular for tests.
we need doc and changelog updates
do not add unused code. Remove it.

pombredanne · 2026-03-11T08:14:22Z

+    if repos:
+        # Convert to list if needed and use first repo's index_url
+        repos_list = list(repos) if not isinstance(repos, list) else repos
+        base_path = repos_list[0].index_url.replace("/simple", "/pypi")


What is this doing? this is weird: what if you have 10 repos?

Good point, I initially opted out for a half-solution to get the first repository. However now I updated it to try all available repositories until it hits a success. Let me know if this aligns with your expectations.

pombredanne · 2026-03-11T08:15:29Z

+    for url_entry in response.get("urls") or []:
+        url = url_entry.get("url")
+        if url:
+            # Resolve relative URLs (from Artifactory) to absolute URLs


Do you have Artifactory examples? and what about nexus or others?

I am not aware of any public Artifactory URL to share freely.

pombredanne · 2026-03-11T08:17:50Z


+def get_file_match_key(url: str, sha256: Optional[str] = None) -> tuple:
+    """
+    Extract a match key (filename, sha256) for comparing distribution files.


Use our comment style: `Return ....

This function was unused, sorry I overlooked it..

pombredanne · 2026-03-11T08:20:10Z

            continue

-        url_data = urls.get(dist_url)
+        url_data = urls_by_filename[filename]


Are you sure that this will not fail?

You're right, replaced with urls_by_filename.get(filename)

voidpetal · 2026-03-11T10:36:45Z

Hi @pombredanne,

Thanks a lot for your review!
I agree with you and I sincerely apologize for making you feel your time is wasted. I will do my best to take into consideration your advice for the future contributions. Since this is my first contribution, I hope that you can forgive me for the oversight.

I have amended the PR and commit descriptions as you proposed and the amended code is ready for your further review.

Please let me know if you would like me to do anything else!

pombredanne · 2026-05-08T09:59:55Z

@voidpetal we have now #261 that was merged, which has a similar goal. There are a few issues in both cases, as there are cases where #261 and this code here may not work all the times. In particular, Artifactory is peculiar in that it does not support the simple API, but Nexus does, and of course PyPI does too. We will need to refine all that with proper tests as we cannot assume that we have to always a (slow) pypi/ endpoint over a a (faster) simple/ endpoint, especially when not using Artifactory

voidpetal · 2026-05-15T08:03:32Z

Hi @pombredanne, thanks for letting me know! I checked and it seems that with the new release our issues were resolved. So I am fine also closing this PR.

Let me know if there is something you'd like to keep from this PR.

Artifactory URLs have different path structures than PyPI URLs, but filenames are identical. When exact URL match fails, fall back to matching by filename to ensure packages array is populated. Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

Artifactory might return `project_urls: null` even for public packages. Now iterates through all index_urls and falls back to PyPI.org to get VCS metadata when Artifactory has incomplete data. Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

This helps ORT resolve VCS provenance when artifact download fails. Populated from code_view_url Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

Extract sdist URL, hash, and filename from PyPI response and include in extra_data.source_artifact for downstream consumers that need source distribution information. Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

voidpetal · 2026-05-28T14:18:47Z

Hi @pombredanne, actually the issue wasn't fully resolved, but I updated the PR with a fix which resolves the missing metadata issue for us.

In total the PR contains the following fixes:

URL filename matching - Artifactory URLs differ from PyPI URLs (different path structure), but filenames are identical. Now we fallback to filename matching when exact URL match fails, otherwise packages array comes back empty.
VCS URL extraction - vcs_url is now populated from code_view_url. ORT tries both [ARTIFACT, VCS] for provenance resolution, so when artifact fails it falls back to VCS. Without vcs_url, both paths fail.
Artifactory project_urls fallback - Artifactory might return project_urls: null even for public packages. We now continue to PyPI.org to get VCS metadata when Artifactory has incomplete data.
Source artifact metadata - Added extra_data.source_artifact with sdist URL/hash for downstream consumers.

Please let me know what you think!

voidpetal force-pushed the fix-custom-artifactory-metadata branch 2 times, most recently from 7bae318 to 2321963 Compare January 22, 2026 10:53

pombredanne requested changes Mar 11, 2026

View reviewed changes

voidpetal force-pushed the fix-custom-artifactory-metadata branch 3 times, most recently from 3cafbe0 to 672f569 Compare March 11, 2026 10:35

voidpetal marked this pull request as draft March 11, 2026 13:28

voidpetal force-pushed the fix-custom-artifactory-metadata branch from 672f569 to e5ddeeb Compare March 11, 2026 13:42

voidpetal marked this pull request as ready for review March 11, 2026 13:54

voidpetal changed the title ~~Support Custom PyPI-Compatible Repositories for Package Metadata~~ Support Custom Artifactory Repositories for Package Metadata Mar 11, 2026

voidpetal force-pushed the fix-custom-artifactory-metadata branch 3 times, most recently from 40dbfa4 to 9fdb092 Compare March 12, 2026 07:50

voidpetal force-pushed the fix-custom-artifactory-metadata branch from 9fdb092 to 0d45a34 Compare May 28, 2026 13:41

voidpetal added 7 commits May 28, 2026 16:13

Extract vcs_url from project_urls.Source

22f705b

This helps ORT resolve VCS provenance when artifact download fails. Populated from code_view_url Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

Add source artifact metadata to extra_data

6df7fab

Extract sdist URL, hash, and filename from PyPI response and include in extra_data.source_artifact for downstream consumers that need source distribution information. Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

Add unit tests

e85d615

Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

Regenerate test fixtures

a6a2a7c

Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

Update CHANGELOG Artifactory metadata support

9b10285

Signed-off-by: Kai Hodžić <hodzic.e.k@outlook.com>

voidpetal force-pushed the fix-custom-artifactory-metadata branch from 8824f77 to 9b10285 Compare May 28, 2026 14:18

Uh oh!

Conversation

voidpetal commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pombredanne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

voidpetal commented Mar 11, 2026 • edited by mjherzog Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pombredanne commented May 8, 2026

Uh oh!

voidpetal commented May 15, 2026

Uh oh!

voidpetal commented May 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

voidpetal commented Jan 21, 2026 •

edited

Loading

voidpetal commented Mar 11, 2026 •

edited by mjherzog

Loading