Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .bazelignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ bazel-testlogs
# treated as directories with valid BUILD files for the main repo.
# Any directory with a WORKSPACE in it should be added here, with
# an entry like `bazel-{workspacename}`
examples/build_file_generation/bazel-bin
examples/build_file_generation/bazel-build_file_generation
examples/build_file_generation/bazel-out
examples/build_file_generation/bazel-testlogs
examples/bzlmod/bazel-bin
examples/bzlmod/bazel-bzlmod
examples/bzlmod/bazel-out
Expand All @@ -28,7 +32,7 @@ gazelle/bazel-gazelle
gazelle/examples/bzlmod_build_file_generation/bazel-bin
gazelle/examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation
gazelle/examples/bzlmod_build_file_generation/bazel-out
gazelle/examples/bzlmod_build_file_generation/bazel-testlog
gazelle/examples/bzlmod_build_file_generation/bazel-testlogs
sphinxdocs
tests/integration/bzlmod_lockfile/bazel-bzlmod_lockfile
tests/integration/compile_pip_requirements/bazel-compile_pip_requirements
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ Some checked-in files are generated and need to be updated when a new PR is
merged:

* **requirements lock files**: These are usually generated by a
`compile_pip_requirements` update target, which is usually in the same directory.
`lock` update target, which is usually in the same directory.
e.g. `bazel run //dev:requirements.update`

## Binary artifacts
Expand Down
115 changes: 54 additions & 61 deletions docs/pypi/lock.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,68 +11,12 @@ Currently `rules_python` only supports `requirements.txt` format.

## requirements.txt

### pip compile
### uv pip compile (bzlmod)

Generally, when working on a Python project, you'll have some dependencies that themselves have
other dependencies. You might also specify dependency bounds instead of specific versions.
So you'll need to generate a full list of all transitive dependencies and pinned versions
for every dependency.

Typically, you'd have your project dependencies specified in `pyproject.toml` or `requirements.in`
and generate the full pinned list of dependencies in `requirements_lock.txt`, which you can
manage with {obj}`compile_pip_requirements`:

```starlark
load("@rules_python//python:pip.bzl", "compile_pip_requirements")

compile_pip_requirements(
name = "requirements",
src = "pyproject.toml",
requirements_txt = "requirements_lock.txt",
)
```

This rule generates two targets:
- `bazel run [name].update` will regenerate the `requirements_txt` file
- `bazel test [name]_test` will test that the `requirements_txt` file is up to date

Once you generate this fully specified list of requirements, you can install the requirements ([bzlmod](./download)/[WORKSPACE](./download-workspace)).

:::{warning}
If you're specifying dependencies in `pyproject.toml`, make sure to include the
`[build-system]` configuration, with pinned dependencies.
`compile_pip_requirements` will use the build system specified to read your
project's metadata, and you might see non-hermetic behavior if you don't pin the
build system.

Not specifying `[build-system]` at all will result in using a default
`[build-system]` configuration, which uses unpinned versions
([ref](https://peps.python.org/pep-0518/#build-system-table)).
:::


#### pip compile Dependency groups

pip-compile doesn't yet support pyproject.toml dependency groups. Follow
[pip-tools #2062](https://github.com/jazzband/pip-tools/issues/2062)
to see the status of their support.

In the meantime, support can be emulated by passing multiple files to `srcs`:

```starlark
compile_pip_requirements(
srcs = ["pyproject.toml", "requirements-dev.in"]
...
)
```

### uv pip compile (bzlmod only)

We also have an experimental setup for the `uv pip compile` way of generating lock files.
This is well tested with the public PyPI index, but you may hit some rough edges with private
mirrors.

#### Example usage
When working on a Python project, you will have dependencies that themselves
have transitive dependencies. You can generate a full list of transitive
dependencies and pinned versions in `requirements_lock.txt` using the
{obj}`lock` rule with `uv`:

```starlark
load("@rules_python//python/uv:lock.bzl", "lock")
Expand Down Expand Up @@ -119,4 +63,53 @@ lock(
the {obj}`lock` docs for how to add one manually using `diff_test` from `bazel_skylib`.
:::

### pip compile (WORKSPACE)

For WORKSPACE projects or when using `pip-compile`, you can manage pinned
dependencies with {obj}`compile_pip_requirements`:

```starlark
load("@rules_python//python:pip.bzl", "compile_pip_requirements")

compile_pip_requirements(
name = "requirements",
src = "pyproject.toml",
requirements_txt = "requirements_lock.txt",
)
```

This rule generates two targets:
- `bazel run [name].update` will regenerate the `requirements_txt` file
- `bazel test [name]_test` will test that the `requirements_txt` file is up to date

Once you generate this fully specified list of requirements, you can install the requirements ([bzlmod](./download)/[WORKSPACE](./download-workspace)).

:::{warning}
If you're specifying dependencies in `pyproject.toml`, make sure to include the
`[build-system]` configuration, with pinned dependencies.
`compile_pip_requirements` will use the build system specified to read your
project's metadata, and you might see non-hermetic behavior if you don't pin the
build system.

Not specifying `[build-system]` at all will result in using a default
`[build-system]` configuration, which uses unpinned versions
([ref](https://peps.python.org/pep-0518/#build-system-table)).
:::


#### pip compile Dependency groups

pip-compile doesn't yet support pyproject.toml dependency groups. Follow
[pip-tools #2062](https://github.com/jazzband/pip-tools/issues/2062)
to see the status of their support.

In the meantime, support can be emulated by passing multiple files to `srcs`:

```starlark
compile_pip_requirements(
srcs = ["pyproject.toml", "requirements-dev.in"]
...
)
```

For more documentation see {obj}`lock`.
12 changes: 4 additions & 8 deletions examples/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# The following is experimental API and currently not intended for use outside this example.
load("@rules_python//python/uv/private:lock.bzl", "lock") # buildifier: disable=bzl-visibility
load("@rules_python//python/uv:lock.bzl", "lock")

licenses(["notice"]) # Apache 2.0

Expand All @@ -35,8 +34,7 @@ lock(
out = "bzlmod/requirements_windows_3_12.txt",
args = [
"--emit-index-url",
"--python-platform",
"windows",
"--python-platform=windows",
"--python-version=3.12",
],
python_version = "3.12",
Expand All @@ -60,8 +58,7 @@ lock(
out = "bzlmod/requirements_windows_3_13.txt",
args = [
"--emit-index-url",
"--python-platform",
"windows",
"--python-platform=windows",
"--python-version=3.13",
],
python_version = "3.13",
Expand All @@ -85,8 +82,7 @@ lock(
out = "bzlmod/requirements_windows_3_14.txt",
args = [
"--emit-index-url",
"--python-platform",
"windows",
"--python-platform=windows",
"--python-version=3.14",
],
python_version = "3.14",
Expand Down
34 changes: 24 additions & 10 deletions examples/bzlmod/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,37 @@
load("@bazel_skylib//rules:build_test.bzl", "build_test")
load("@pip//:requirements.bzl", "all_data_requirements", "all_requirements", "all_whl_requirements", "requirement")
load("@python_3_13//:defs.bzl", py_test_with_transition = "py_test")
load("@python_versions//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements")
load("@rules_python//python:py_binary.bzl", "py_binary")
load("@rules_python//python:py_library.bzl", "py_library")
load("@rules_python//python:py_test.bzl", "py_test")
load("@rules_python//python/uv:lock.bzl", "lock")

# This stanza calls a rule that generates targets for managing pip dependencies
# with pip-compile for a particular python version.
compile_pip_requirements_3_10(
# with uv for a particular python version.
lock(
name = "requirements_3_10",
timeout = "moderate",
src = "pyproject.toml",
extra_args = [
"--extra-index-url",
"https://pypi.org/simple/",
srcs = ["pyproject.toml"],
out = "requirements_lock_3_10.txt",
args = [
"--emit-index-url",
"--extra-index-url=https://pypi.org/simple/",
"--universal",
"--python-version=3.10",
],
requirements_txt = "requirements_lock_3_10.txt",
requirements_windows = "requirements_windows_3_10.txt",
python_version = "3.10",
)

lock(
name = "requirements_windows_3_10",
srcs = ["pyproject.toml"],
out = "requirements_windows_3_10.txt",
args = [
"--emit-index-url",
"--extra-index-url=https://pypi.org/simple/",
"--python-platform=windows",
"--python-version=3.10",
],
python_version = "3.10",
)

# The rules below are language specific rules defined in
Expand Down
13 changes: 9 additions & 4 deletions examples/bzlmod/other_module/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
load("@rules_python//python/uv:lock.bzl", "lock")

# NOTE: To update the requirements, you need to uncomment the rules_python
# override in the MODULE.bazel.
compile_pip_requirements(
lock(
name = "requirements",
src = "pyproject.toml",
srcs = ["pyproject.toml"],
out = "requirements_lock_3_11.txt",
args = [
"--emit-index-url",
"--universal",
"--python-version=3.11",
],
python_version = "3.11",
requirements_txt = "requirements_lock_3_11.txt",
)
9 changes: 9 additions & 0 deletions examples/bzlmod/other_module/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,12 @@ pip.parse(
requirements_lock = ":requirements_lock_3_11.txt",
)
use_repo(pip, "other_module_pip")

# DO NOT MERGE: consider making uv auto-used before switching
# everything over to it.
Comment on lines +44 to +45

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about uv not being present in WORKSPACE?

uv = use_extension(
"@rules_python//python/uv:uv.bzl",
"uv",
dev_dependency = True,
)
uv.configure()
10 changes: 4 additions & 6 deletions examples/bzlmod/other_module/requirements_lock_3_11.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# This file was autogenerated by uv via the following command:
# bazel run //:requirements.update
#
--index-url https://pypi.org/simple

absl-py==1.4.0 \
--hash=sha256:0d3fe606adfa4f7db64792dd4c7aee4ee0c38ab75dfd353b7a83ed3e957fcb47 \
--hash=sha256:d2c244d01048ba476e7c080bd2c6df5e141d211de80223460d5b3b8a2a58433d
# via other_module (pyproject.toml)
# via other-module (pyproject.toml)
Loading
Loading