Skip to content

feat(helm): connect Vermeer operations health #3434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
imbajin merged 1 commit into from
Oct 29, 2026
Merged

feat(helm): connect Vermeer operations health #3434

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions helm/hugegraph/docs/vermeer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,49 @@ view; the Vermeer service token does not grant them this administrator access.
Setting `hubble.enabled=false` keeps Vermeer available through its native entry.
`hubble.mode=direct` cannot be combined with an enabled native Vermeer integration.

### Cluster health in Hubble

The Hubble **System & Operations → Cluster Overview** keeps Server, PD and Store
as the main topology. Vermeer appears as a compact compute-cluster summary;
select it to switch to the Master-above-Worker view, then use **Return to main
cluster** to restore the primary topology. This view describes the deployment,
not the graph selected elsewhere in Hubble. Both Vermeer deployment modes use
the same view, including standalone's one Master and one Worker.

With `vermeer.enabled=true`, the chart also writes these Hubble properties:

| Property | Chart-managed value |
|---|---|
| `operations.vermeer.enabled` | `true` |
| `operations.vermeer.address` | The release's internal Master HTTP Service origin |
| `operations.vermeer.token_file` | `/var/run/hugegraph/vermeer/token` |

Operations and batch analysis reuse the same generated service token. Hubble
mounts that token read-only; its main container never mounts the signing factor.
No extra password, externally exposed Worker port or Kubernetes API permission
is needed. The existing network policies allow Hubble to reach only the Master
HTTP entry. Disabling Vermeer explicitly clears both integrations' endpoint and
token-file properties and sets `operations.vermeer.enabled=false`, including
when the image contains old defaults.

Cluster health requires a paired Hubble and Vermeer build that supports the
authenticated `GET /api/v1/admin/cluster` snapshot (`schema_version: 1`). The
published Vermeer pin documented below predates that endpoint; retain its
immutable digest until a reviewed replacement has been published, then select
the paired images using `hubble.image` and `vermeer.image` (clear or replace the
old `vermeer.image.digest` when selecting another tag). Do not infer capability
from a tag alone. An older image that returns 404 is shown as unsupported, not
healthy. A failed or unauthorized snapshot is reported as unavailable and does
not turn the main graph cluster into a database outage.

Worker counts are **ready / registered**, not ready / desired Helm replicas.
Registration alone does not establish readiness: the Master checks its reverse
connection and all three task streams. Process CPU is expressed in cores and
memory as RSS bytes; these are not Pod utilization percentages. Missing resource
measurements do not imply zero usage or make an otherwise ready Worker offline.
This deployment-wide snapshot is separate from the space-aware allocation
check at `/api/v1/workers/availability`.

## Optional native administration

Master HTTP defaults to NodePort in easy mode and ClusterIP in strict mode.
Expand Down
10 changes: 8 additions & 2 deletions helm/hugegraph/templates/hubble-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,8 +168,11 @@ spec:
{{- end }}
while IFS= read -r LINE || [[ -n "${LINE}" ]]; do
case "${LINE}" in
vermeer.url=*|vermeer.token_file=*)
# Written once below, including explicit blanks when disabled.
vermeer.url=*|vermeer.token_file=*|\
operations.vermeer.enabled=*|operations.vermeer.address=*|operations.vermeer.token_file=*)
# Native analysis and operations share the same internal
# service identity. Write every key once below, clearing
# stale image defaults when the component is disabled.
;;
pd.enabled=*)
printf 'pd.enabled={{ $pdMode }}\n' >>"${TMP}"
Expand Down Expand Up @@ -225,8 +228,11 @@ spec:
{{- if $vermeerEnabled }}
printf 'vermeer.url=%s\nvermeer.token_file=%s\n' \
"${HG_HUBBLE_VERMEER_URL}" "${HG_HUBBLE_VERMEER_TOKEN_FILE}" >>"${TMP}"
printf 'operations.vermeer.enabled=true\noperations.vermeer.address=%s\noperations.vermeer.token_file=%s\n' \
"${HG_HUBBLE_VERMEER_URL}" "${HG_HUBBLE_VERMEER_TOKEN_FILE}" >>"${TMP}"
{{- else }}
printf 'vermeer.url=\nvermeer.token_file=\n' >>"${TMP}"
printf 'operations.vermeer.enabled=false\noperations.vermeer.address=\noperations.vermeer.token_file=\n' >>"${TMP}"
{{- end }}
if [[ "${FOUND_PD_ENABLED}" == false ]]; then
printf 'pd.enabled={{ $pdMode }}\n' >>"${TMP}"
Expand Down
46 changes: 42 additions & 4 deletions helm/test_helm_vermeer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -435,12 +435,19 @@ def test_disabled_integration_clears_image_defaults_and_does_not_mount_token(sel
script = "\n".join(app.get("args", []))
self.assertIn("vermeer.url", script)
self.assertIn("vermeer.token_file", script)
self.assertIn("operations.vermeer.enabled=false", script)
self.assertIn("operations.vermeer.address=", script)
self.assertIn("operations.vermeer.token_file=", script)
self.assertNotIn("fixture-vermeer-master", script)

def test_hubble_wrapper_replaces_stale_and_duplicate_vermeer_properties(self):
for enabled in (False, True):
with self.subTest(enabled=enabled):
documents = self.documents({} if enabled else None)
cases = ((None, {}), ({}, {}),
({"mode": "cluster", "master": {"httpPort": 7688}},
{"fullnameOverride": "analysis"}))
for vermeer, overrides in cases:
enabled = vermeer is not None
with self.subTest(vermeer=vermeer, overrides=overrides):
documents = self.documents(vermeer, **overrides)
hubble, = resources(documents, "Deployment", "hubble")
container, = pod(hubble)["spec"]["containers"]
environment = dict(os.environ)
Expand All @@ -457,7 +464,13 @@ def test_hubble_wrapper_replaces_stale_and_duplicate_vermeer_properties(self):
properties.write_text("unrelated.setting=preserved\n"
"vermeer.url=http://stale-image-default\n"
"vermeer.token_file=/stale-token\n"
"vermeer.url=http://duplicate-default\n")
"vermeer.url=http://duplicate-default\n"
"operations.vermeer.enabled=true\n"
"operations.vermeer.address=http://stale-operations-default\n"
"operations.vermeer.token_file=/stale-operations-token\n"
"operations.vermeer.enabled=false\n"
"operations.vermeer.address=http://duplicate-default\n"
"operations.vermeer.token_file=/duplicate-token\n")
start = root / "bin/start-hubble.sh"
start.write_text("#!/bin/sh\nexit 0\n")
start.chmod(0o755)
Expand All @@ -473,6 +486,12 @@ def test_hubble_wrapper_replaces_stale_and_duplicate_vermeer_properties(self):
[line for line in final if line.startswith("vermeer.url=")])
self.assertEqual(["vermeer.token_file=" + expected_token],
[line for line in final if line.startswith("vermeer.token_file=")])
self.assertEqual(["operations.vermeer.enabled=" + str(enabled).lower()],
[line for line in final if line.startswith("operations.vermeer.enabled=")])
self.assertEqual(["operations.vermeer.address=" + expected_url],
[line for line in final if line.startswith("operations.vermeer.address=")])
self.assertEqual(["operations.vermeer.token_file=" + expected_token],
[line for line in final if line.startswith("operations.vermeer.token_file=")])

def test_persistence_inheritance_and_explicit_temporary_storage(self):
for mode in ("standalone", "cluster"):
Expand Down Expand Up @@ -529,6 +548,25 @@ def test_strict_network_covers_reverse_rpc_and_peer_rpc(self):
self.assertTrue(policy_allows(policies, labels[destination], labels[source], port, "ingress"))
self.assertTrue(policy_allows(policies, labels[source], labels[destination], port, "egress"))

def test_operations_needs_only_master_http_not_worker_or_rpc_access(self):
for mode in ("standalone", "cluster"):
with self.subTest(mode=mode):
documents = self.documents({"mode": mode, "master": {"httpPort": 7688,
"grpcPort": 7689},
"worker": {"httpPort": 7788, "grpcPort": 7789}},
mode="strict")
policies = resources(documents, "NetworkPolicy")
workloads = resources(documents, "StatefulSet") + resources(documents, "Deployment")
labels = {workload["metadata"]["labels"][COMPONENT]: pod(workload)["metadata"]["labels"]
for workload in workloads}
hubble, master = labels["hubble"], labels["vermeer-master"]
self.assertTrue(policy_allows(policies, hubble, master, 7688, "egress"))
self.assertTrue(policy_allows(policies, master, hubble, 7688, "ingress"))
worker = labels["vermeer-worker"] if mode == "cluster" else master
for target, port in ((master, 7689), (worker, 7788), (worker, 7789)):
self.assertFalse(policy_allows(policies, hubble, target, port, "egress"))
self.assertFalse(policy_allows(policies, target, hubble, port, "ingress"))

def test_backend_connections_need_no_extra_rules_for_either_worker_pod_layout(self):
for topology in ("standalone", "cluster"):
for entry_mode, policy_enabled in (("easy", None), ("strict", None),
Expand Down
Loading