Add redirect packages, fix tests, and prep for sub-package release by skrawcz · Pull Request #1588 · apache/hamilton

skrawcz · 2026-05-18T03:58:14Z

Add sf-hamilton-{sdk,ui,lsp,contrib}-redirect packages
Fix LSP: pygls>=2.0 compat (remove deprecated loop kwarg)
Fix UI: remove broken hamilton-ui entrypoint (use hamilton ui CLI)
Fix UI: settings_mini STATICFILES_DIRS to build/assets/ (Vite)
Fix UI: ProjectOut.role type annotation for pydantic 2.x
Fix SDK test: update polars stats expected values for polars 1.34+
Fix UI test: relax build directory string check
Add scripts/verify-sub-packages.md for release validation

Changes

changes to release subpackages

How I tested this

Notes

Checklist

PR has an informative and human-readable title (this will be pulled into the release notes)
Changes are limited to a single goal (no scope creep)
Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
Any change in functionality is tested
New functions are documented (with a description, list of inputs, and expected output)
Placeholder code is flagged / future TODOs are captured in comments
Project documentation has been updated if adding/changing functionality.

Copilot

Pull request overview

This PR prepares the Hamilton sub-packages (SDK/UI/LSP/Contrib) for release under the Apache package names by adding “redirect” packages for legacy sf-hamilton-* names, updating compatibility for newer dependency versions (e.g., pygls>=2.0, polars>=1.34), and adjusting UI mini-mode static asset handling.

Changes:

Add sf-hamilton-{sdk,ui,lsp,contrib} redirect package skeletons (README + templated pyproject + build scripts).
Fix/relax tests for optional dependencies (skip when missing) and for dependency behavior changes (Polars date quantiles, UI build path string checks).
Update UI mini settings to use Vite’s build/assets/ output and adjust LSP to remove deprecated loop usage for pygls>=2.0.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
ui/sdk/tests/tracking/test_pyspark_stats.py	Skip module when `pyspark` isn’t installed to keep test suite runnable in minimal envs.
ui/sdk/tests/tracking/test_polars_stats.py	Relax date-quantile assertions for Polars 1.34+ behavior differences.
ui/sdk/tests/tracking/test_langchain_stats.py	Skip module when `langchain_core` isn’t installed.
ui/sdk/tests/tracking/test_ibis_stats.py	Skip module when `ibis` isn’t installed.
ui/sdk/requirements-test.txt	Add Python version markers to avoid installing unsupported deps on Python 3.14+.
ui/backend/tests/test_build.py	Relax settings-content check for build dir strings (assets/static).
ui/backend/server/trackingserver_projects/schema.py	Make `ProjectOut.role` explicitly nullable for Pydantic v2 typing consistency.
ui/backend/server/server/settings_mini.py	Point `STATICFILES_DIRS` at `build/assets/` (Vite output).
ui/backend/pyproject.toml	Remove the `hamilton-ui` console script entrypoint.
sf-hamilton-ui-redirect/README.md	Document redirect from `sf-hamilton-ui` to `apache-hamilton-ui`.
sf-hamilton-ui-redirect/pyproject.toml.template	Template for redirect package metadata/dependency pinning.
sf-hamilton-ui-redirect/build.sh	Build helper to stamp version + build + twine-check redirect artifact.
sf-hamilton-sdk-redirect/README.md	Document redirect from `sf-hamilton-sdk` to `apache-hamilton-sdk`.
sf-hamilton-sdk-redirect/pyproject.toml.template	Template for SDK redirect package metadata/dependency pinning.
sf-hamilton-sdk-redirect/build.sh	Build helper for SDK redirect artifact.
sf-hamilton-lsp-redirect/README.md	Document redirect from `sf-hamilton-lsp` to `apache-hamilton-lsp`.
sf-hamilton-lsp-redirect/pyproject.toml.template	Template for LSP redirect package metadata/dependency pinning.
sf-hamilton-lsp-redirect/build.sh	Build helper for LSP redirect artifact.
sf-hamilton-contrib-redirect/README.md	Document redirect from `sf-hamilton-contrib` to `apache-hamilton-contrib`.
sf-hamilton-contrib-redirect/pyproject.toml.template	Template for contrib redirect package metadata/dependency pinning.
sf-hamilton-contrib-redirect/build.sh	Build helper for contrib redirect artifact.
scripts/verify-sub-packages.md	New release-validation guide for sub-packages (install/test/acceptance checks).
dev_tools/language_server/pyproject.toml	Bump `pygls` dependency to `>=2.0`.
dev_tools/language_server/hamilton_lsp/server.py	Remove deprecated `loop` kwarg usage for `pygls>=2.0`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

skrawcz · 2026-05-24T15:03:20Z

+```bash
+# Set the versions being released
+export SDK_VERSION=0.9.0
+export UI_VERSION=0.0.18
+export LSP_VERSION=0.2.0
+export CONTRIB_VERSION=0.0.9
+```


Fixed — added export HAMILTON_VERSION=1.90.0 to the Quick Start section.

skrawcz · 2026-05-24T15:03:30Z

+# Step 1: Open the UI in your browser and create a project.
+# The `hamilton ui` command opens http://localhost:8242 automatically.
+# Create a project and note its ID (shown in the URL, e.g. /dashboard/project/1).
+
+# Step 2: Run the hamilton_ui example against the server
+cd examples/hamilton_ui
+pip install -r requirements.txt
+python run.py --username <your_username> --project-id <project_id>
+# Expected: "Captured execution run. Results can be found at ..."
+
+# Step 3: Verify in the UI
+# Navigate to http://localhost:8242/dashboard/project/<id>/runs
+# You should see the DAG run with node-level details, attributes, etc.


Fixed — all URLs now consistently reference port 8241 (mini mode serves both API and frontend on the same port).

skrawcz · 2026-05-24T15:03:38Z

    def test_settings_mini_mode_configured(self):
        """Verify that Django settings are correctly configured for mini mode."""
        settings_file = get_ui_backend_dir() / "server" / "server" / "settings.py"
        assert settings_file.exists(), f"Django settings file not found at {settings_file}"

        settings_content = settings_file.read_text()

        # Check for mini mode configuration
        assert 'HAMILTON_ENV == "mini"' in settings_content, (
            "Django settings missing mini mode configuration"
        )

-        # Check for STATICFILES_DIRS pointing to build/static/ or build/assets/
-        assert "build/static/" in settings_content or "build/assets/" in settings_content, (
-            "Django settings missing build/static/ or build/assets/ in STATICFILES_DIRS"
+        # Check for STATICFILES_DIRS pointing to build/static or build/assets
+        assert "build/static" in settings_content or "build/assets" in settings_content, (
+            "Django settings missing build/static or build/assets in STATICFILES_DIRS"
        )


Good catch. Updated the test to also validate settings_mini.py directly (checks that it has build/static or build/assets in STATICFILES_DIRS). The settings.py check remains since it contains the mini mode branching logic.

Dev-iL · 2026-05-18T05:47:33Z

+    def __init__(self, server: str = "HamiltonServer", version: str = __version__):
+        super().__init__(server, version)


Do we need a regression test for this?

The existing test_commands.py validates server initialization works with pygls 2.x — that's the regression coverage. The change is just removing a deprecated kwarg, so the test confirming the server still starts is sufficient.

Dev-iL · 2026-05-18T06:43:26Z

+export SDK_VERSION=0.9.0
+export UI_VERSION=0.0.18
+export LSP_VERSION=0.2.0
+export CONTRIB_VERSION=0.0.9


Wouldn't it be better to have a script that reads the versions from wherever they're stored (like on L45 below)? Fewer manual steps = fewer opportunities for human error.

Dev-iL · 2026-05-18T06:45:31Z

+python -m venv /tmp/verify-sdk && source /tmp/verify-sdk/bin/activate
+pip install apache-hamilton apache-hamilton-sdk==${SDK_VERSION}


Shouldn't we use uv for such things?

Dev-iL · 2026-05-18T06:48:46Z

+pip install -r requirements.txt -r requirements-test.txt
+pip install -e .


Didn't we remove requirement txts? I think we should install with uv sync --group test

Dev-iL · 2026-05-18T06:51:29Z

+hamilton ui --settings-file mini --no-open --port 8241 &
+# Or equivalently from the CLI:
+# python -m hamilton.cli ui --settings-file mini --no-open --port 8241


uv run migrate-ui could be nice

Agreed — doesn't exist yet. Filed as a follow-up idea. Would need a CLI command wired into the hamilton_ui package.

Dev-iL · 2026-05-18T07:09:28Z

+        actual_q = actual["observability_value"][col].pop("quantiles")
+        expected_q = expected_stats["observability_value"][col].pop("quantiles")
+        # Accept either empty or populated quantiles for Date columns
+        assert actual_q == expected_q or actual_q == {}


Perhaps we should branch on the polars version to avoid a false pass? This might even be doable as a test parameterization.

Dev-iL · 2026-05-18T07:13:28Z

@@ -1,10 +1,10 @@
-ibis-framework
-langchain_core
+ibis-framework; python_version < "3.14"


Why? According to ibis' manifest, it supports python >=3.10

Dev-iL · 2026-05-18T07:14:55Z

-ibis-framework
-langchain_core
+ibis-framework; python_version < "3.14"
+langchain_core; python_version < "3.14"


Why? 3.14 is supported: https://github.com/langchain-ai/langchain/blob/master/libs/core/pyproject.toml#L19

Dev-iL · 2026-05-18T07:19:22Z

 pyarrow
-pyarrow_hotfix  # required for ibis tests
+pyarrow_hotfix; python_version < "3.14"


Would it be that bad to set the minimum pyarrow version to 14.0.1 (released on Nov 2023) to avoid pyarrow_hotfix entirely?

Dev-iL · 2026-05-18T07:20:01Z

+pyarrow_hotfix; python_version < "3.14"
 pydantic
-pyspark
+pyspark; python_version < "3.14"


Dev-iL

LGTM with one nit

Dev-iL · 2026-05-24T15:02:07Z

-pyarrow
-pyarrow_hotfix  # required for ibis tests
+pyarrow>=14.0.1
+pyarrow_hotfix


pyarrow_hotfix is included in pyarrow>=14.0.1, so can be removed

Suggested change

pyarrow_hotfix

Checked — pyarrow_hotfix is NOT bundled in pyarrow>=14.0.1. It's a separate package. ibis-framework 12.0.0 unconditionally does import pyarrow_hotfix in 16+ files (backends, formats, etc.) but doesn't declare it as a dependency. We need to keep it in our test requirements until ibis removes those imports.

Checked — pyarrow_hotfix is NOT bundled in pyarrow>=14.0.1. It's a separate package.

Right - it's not a matter of bundled - the hotfix package fixes a CVE present in pyarrow itself. If we're on pyarrow>=14.0.1 we can import pyarrow directly.

ibis-framework 12.0.0 unconditionally does import pyarrow_hotfix in 16+ files (backends, formats, etc.) but doesn't declare it as a dependency. We need to keep it in our test requirements until ibis removes those imports.

Yuck. Perhaps I'll raise a PR in ibis-framework to stop doing that.

BTW, ibis does mention it under many optional dependencies (e.g. here).

BTW2: feat: make pyarrow hot fix optional ibis-project/ibis#11977

- Add sf-hamilton-{sdk,ui,lsp,contrib}-redirect packages - Fix LSP: pygls>=2.0 compat (remove deprecated loop kwarg) - Fix UI: remove broken hamilton-ui entrypoint (use hamilton ui CLI) - Fix UI: settings_mini STATICFILES_DIRS to build/assets/ (Vite) - Fix UI: ProjectOut.role type annotation for pydantic 2.x - Fix SDK test: update polars stats expected values for polars 1.34+ - Fix UI test: relax build directory string check - Add scripts/verify-sub-packages.md for release validation

skrawcz requested review from elijahbenizzy, jernejfrank and zilto May 18, 2026 04:39

skrawcz force-pushed the stefan/release-sub-packages-prep branch 2 times, most recently from add8ac6 to 61852c4 Compare May 18, 2026 05:00

skrawcz requested review from Dev-iL and Copilot May 18, 2026 05:10

Copilot started reviewing on behalf of skrawcz May 18, 2026 05:10 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

Dev-iL reviewed May 18, 2026

View reviewed changes

elijahbenizzy approved these changes May 22, 2026

View reviewed changes

skrawcz force-pushed the stefan/release-sub-packages-prep branch 5 times, most recently from 13da058 to 35fbd11 Compare May 24, 2026 14:38

Dev-iL approved these changes May 24, 2026

View reviewed changes

skrawcz force-pushed the stefan/release-sub-packages-prep branch from 35fbd11 to cbe6250 Compare May 24, 2026 15:03

skrawcz merged commit 5665257 into main May 24, 2026
18 checks passed

skrawcz deleted the stefan/release-sub-packages-prep branch May 24, 2026 15:24

		def __init__(self, server: str = "HamiltonServer", version: str = __version__):
		super().__init__(server, version)

		python -m venv /tmp/verify-sdk && source /tmp/verify-sdk/bin/activate
		pip install apache-hamilton apache-hamilton-sdk==${SDK_VERSION}

		pip install -r requirements.txt -r requirements-test.txt
		pip install -e .

Conversation

skrawcz commented May 18, 2026

Changes

How I tested this

Notes

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dev-iL left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dev-iL May 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Dev-iL May 24, 2026 •

edited

Loading