Skip to content

Regenerate Pipfile.lock against Python 3.13 - #183

Open
caglarpir wants to merge 1 commit into
mapillary:mainfrom
caglarpir:deps-full-refresh
Open

Regenerate Pipfile.lock against Python 3.13#183
caglarpir wants to merge 1 commit into
mapillary:mainfrom
caglarpir:deps-full-refresh

Conversation

@caglarpir

Copy link
Copy Markdown
Contributor

Refreshes the whole lockfile from a single resolve, clearing the remaining Python security advisories and the section drift that was preventing Dependabot from fully fixing them.

Why a full regenerate rather than more per-package bumps

Pipfile.lock had drifted so that default and develop pinned different versions of the same package:

urllib3    default==2.7.0    develop==2.5.0
protobuf   default==6.33.5   develop==6.31.1
click      default==8.2.1    develop==8.5.0

Dependabot bumps one section at a time, so the runtime side got patched while a contributor running pipenv install --dev still received the vulnerable versions — and the alerts never cleared. Five of the nine open advisories were this, not a missing bump. Regenerating resolves both sections from one solve, and the new lock has no version conflicts between sections.

What changed

  • 74 → 67 packages, 36 version changes, no prereleases (despite allow_prereleases)
  • Clears the remaining advisories: urllib3 2.5.0→2.7.0, protobuf 6.31.1→6.33.6, requests→2.34.2, idna→3.19, pytest→9.1.1, pygments→2.21.0, jaraco.context→6.1.2
  • Drops 8 backports unnecessary on 3.13: tomli, typing-extensions, exceptiongroup, importlib-metadata, zipp, pytz, tzdata, backports.tarfile
  • Larger majors picked up: pandas 2.3.1→3.0.5, numpy→2.5.2, scipy 1.15.3→1.18.1, sphinx 8.1.3→9.1.0, twine→7.0.0

Also corrects the comment I left on the python-version pin, which is now stale — the scipy/OpenBLAS problem it described is fixed by this refresh. The pin stays, but for the durable reason: CI should not drift onto an interpreter the lockfile has never been resolved against.

How this was verified

Beyond the repo suite (17 tests, only 4 of which touch a third-party library), I built a differential harness: 22 probes exercising each dependency directly and through the SDK, driven from a recorded 1.4 MB vector tile so results are reproducible. The same harness runs against the old and new locks and the outputs are diffed — any offline difference is attributable to the dependency change alone. Determinism was confirmed by three runs under different PYTHONHASHSEED producing byte-identical output.

Probes cover: mercantile tile math; shapely geometry ops; haversine; geojson; turfpy (and via it numpy/scipy); pandas; protobuf → mapbox-vector-tile → vt2geojson tile decoding; requests/urllib3 session construction; dateutil; plus the SDK’s URL builders, extract_properties, the filter.pipeline chain, format utils, GeoJSON models, an import sweep of all 20 modules, and exception paths. Three live probes hit the real API.

Result: one behavioural difference across all 22 probes.

*** OFFLINE DIFF *** pandas.csv_and_frame
    .dtypes[0]: object -> str

That is pandas 3.0’s new default string dtype. It does not affect this project: the SDK never imports pandas — it appears only in tests/test_interface.py:53, via .tolist(), which is unchanged. Everything else is identical.

Test plan

Locally, on Python 3.13, against the regenerated lock:

  • pytest17 passed
  • python -m build — sdist + wheel built
  • flake8 src/mapillary --select=E9,F63,F7,F82 — 0
  • 22/22 harness probes ok, including the 3 live API probes

I also checked all 67 pins against PyPI: every one is either a pure-Python wheel or ships cp314 manylinux wheels, with turfpy==0.0.7 the only sdist-only dependency (pure Python, no build step). So Python 3.14 is now unblocked — worth a separate PR once this settles, not folded in here.

CI on this PR is the cross-check that the lock resolves on Linux, since it was generated on macOS.

Pipfile.lock had drifted: the default and develop sections pinned
different versions of urllib3, protobuf and click, so Dependabot's
per-section bumps left the develop side vulnerable even after the
runtime side was patched. Regenerating resolves both sections from the
same solve.

Notable changes:
- resolves all remaining Python security advisories (urllib3 2.5.0
  -> 2.7.0, protobuf 6.31.1 -> 6.33.6, requests -> 2.34.2, idna -> 3.19,
  pytest -> 9.1.1, pygments -> 2.21.0, jaraco.context -> 6.1.2)
- drops 8 backport packages no longer needed on 3.13 (tomli,
  typing-extensions, exceptiongroup, importlib-metadata, zipp, pytz,
  tzdata, backports.tarfile)
- 74 -> 67 packages, 36 version changes, no prereleases
- no remaining version conflicts between default and develop

Also corrects the now-stale comment on the python-version pin: the
scipy/OpenBLAS problem it described is resolved by this refresh
(scipy 1.18.1 ships cp314 wheels). The pin stays so CI cannot drift
onto an interpreter the lockfile was never resolved against.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant