diff --git a/content/cn/docs/quickstart/computing/hugegraph-vermeer.md b/content/cn/docs/quickstart/computing/hugegraph-vermeer.md index f32225842b..c8f42ad10e 100644 --- a/content/cn/docs/quickstart/computing/hugegraph-vermeer.md +++ b/content/cn/docs/quickstart/computing/hugegraph-vermeer.md @@ -14,7 +14,8 @@ Vermeer 是使用 Go 编写的高性能内存优先图计算框架,支持一 master 是负责通信、转发、汇总的节点,计算量和占用资源量较少。worker 是计算节点,用于存储图数据和运行计算任务,占用大量内存和 cpu。grpc 和 rest 模块分别负责内部通信和外部调用。 -启动时,程序先设置内置默认值,再读取工作目录下 `config/.ini` 的 `[default]` 节,最后由显式命令行参数覆盖对应配置;例如 `--env=master` 读取 `config/master.ini`。Docker 镜像把仓库中的配置复制到 `/go/bin/config/`,并以 `/go/bin/` 为工作目录。挂载宿主机目录到 `/go/bin/config` 会遮住镜像自带文件,因此该目录必须包含实际使用的 ini 文件。配置读取器不从环境变量取值。 +启动时,程序先设置内置默认值,再读取工作目录下 `config/.ini` 的 `[default]` 节,最后由显式命令行参数覆盖对应配置;例如 `--env=master` 读取 `config/master.ini`。Docker 镜像把仓库中的配置复制到 `/go/bin/config/`,并以 `/go/bin/` 为工作目录。挂载宿主机目录到 `/go/bin/config` 会遮住镜像自带文件,因此该目录必须包含实际使用的 ini 文件。普通 INI 配置键不会自动映射环境变量。部署专用凭据设置还支持 +`VERMEER_PD_PASSWORD_FILE`、`VERMEER_PD_PEERS`、`VERMEER_HUGEGRAPH_CREDENTIALS_FILE` 及文档中的签名因子/密码文件设置。 默认端口如下: @@ -37,10 +38,44 @@ flowchart LR ### 1.2 运行方法 +> [!DETAILS]- Helm:与 HugeGraph、Hubble 一起部署 Vermeer +> HugeGraph Helm Chart 支持一键拉起原生 Vermeer 内存图计算引擎,默认保持关闭。 +> 在 HugeGraph 仓库根目录下,开启 Vermeer 并启动: +> +> ```bash +> helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ +> -f helm/hugegraph/values-single.yaml \ +> --set vermeer.enabled=true --set vermeer.mode=standalone \ +> --wait --timeout 20m +> ``` +> +> ```filetree {wrap=true title="Vermeer 核心文件与配置结构"} +> helm/hugegraph/ +> examples/ +> values-vermeer-standalone.yaml # 单机预设(单个 Pod 同时运行 Master 与 Worker 容器) +> values-vermeer-cluster.yaml # 集群预设(独立 Master Pod + 多 Worker Pods 分布式计算) +> templates/ +> vermeer-master.yaml # Master 调度节点编排 +> vermeer-worker.yaml # Worker 内存计算节点编排 +> vermeer-auth-secret.yaml # 管理员密码与 Hubble 内部调用服务令牌 +> docs/ +> vermeer.md # Vermeer 算法调用、参数映射与运维指引 +> ``` +> +> **架构与运行模式**: +> - **单机模式(standalone)**:Master 与 Worker 运行在同一个 Pod 中,开销小,适合小规模图计算测试。 +> - **集群模式(cluster)**:设 `vermeer.mode=cluster` 与 `vermeer.worker.replicas=3`,提供独立的 Master 协调节点与多个分布式 Worker 计算节点。 +> - **图服务协同**:Master 通过 gRPC 向 PD 查询分区信息,Worker 直连 Store 扫描图数据切片;Hubble 会自动注入 Master 地址与通信令牌,在可视化界面中一键加载图并执行 15+ 种图算法。 +> +> **核心注意事项**: +> 1. **保持元数据命名空间一致**:保留 Chart 默认的 `metadataCluster: hg-test`。该值统一了 Server 的 `HG_SERVER_CLUSTER` 与 Store 的 `PD_CLUSTER`,确保 Schema 正确读取;切勿随意修改该值,以免计算加载时报 `Schema missing`。 +> 2. **访问原生 Web 控制台**:原生 UI 默认以私有 ClusterIP 运行。如需直连管理,可通过 `kubectl port-forward svc/-vermeer-master 6688:6688` 转发,并在 **Admin credential** 处填入 `-vermeer-admin` Secret 中的 `password`(无需用户名),登录后获得 24 小时有效会话。 +> 3. 详细镜像固定策略、持久卷配置及运维命令可参考 [Helm 可选引擎指南](/cn/docs/quickstart/hugegraph/hugegraph-helm/#optional-engines) 与 [Vermeer 部署指南](https://github.com/hugegraph/hugegraph/blob/master/helm/hugegraph/docs/vermeer.md)。 + > [!WARNING] > 生产环境必须启用 HugeGraph Server [认证与授权](/cn/docs/config/config-authentication/)、IP 白名单和最小权限授权,并保留 `audit-*.log` 审计记录。Server Auth 不会保护 Vermeer、PD 和 Store 的独立接口;这些 HTTP、gRPC 端口必须限制在可信网络及调用方范围内,Vermeer 对外入口需配置访问控制。 > -> Vermeer 的 `master.ini` 默认 `auth=none`,普通及管理 API 均未启用鉴权。本机快速上手只发布回环端口;远程访问前必须启用 `auth=token`,或通过受保护网络、网关将访问限制到可信调用方。 +> 独立手动部署时,默认 `master.ini` 配置为 `auth=none`,下方 Docker 示例仅发布至回环地址;若需远程访问,务必在配置文件中启用 `auth=token`。Helm 部署则已默认开启 Token 鉴权与私有 Service 隔离。 下面两种 Docker 启动方式都需要先准备一个宿主机配置目录。请在 Vermeer 仓库根目录执行,将项目提供的 [`master.ini`](https://github.com/apache/hugegraph-computer/blob/master/vermeer/config/master.ini) 和 [`worker.ini`](https://github.com/apache/hugegraph-computer/blob/master/vermeer/config/worker.ini) 模板复制到该目录;挂载会覆盖镜像里的 `/go/bin/config`,所以不要把空目录或整个用户主目录挂进去: diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-helm-operations.md b/content/cn/docs/quickstart/hugegraph/hugegraph-helm-operations.md index 0dfaa73839..62efeaab9d 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-helm-operations.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-helm-operations.md @@ -11,7 +11,7 @@ search_keywords: [HugeGraph Helm 运维, local PV, NetworkPolicy, PD, Store] ## 1 存储和内网镜像 {#1-storage-and-private-images} -默认安装需要七个持久卷。动态存储需要正常的 provisioner,静态 local 存储需要预建目录和绑定节点的匹配 PV。 +基础安装需要七个持久卷。动态存储需要正常的 provisioner,静态 local 存储需要预建目录和绑定节点的匹配 PV。 [完整例子](https://github.com/apache/hugegraph/tree/master/helm/hugegraph/examples)包含七个 local PV、`WaitForFirstConsumer`、镜像仓库和 GitOps Secret。 `WaitForFirstConsumer` 的 PVC 初期 Pending 可以是正常等待;持续 Pending 应检查 PVC/Pod 事件和 PV 匹配。 PVC 已绑定但目录不存在或权限不匹配时可能挂载失败,不能把故障 PVC 静默替换成临时目录。 @@ -25,6 +25,11 @@ kubectl get pv,storageclass 出现 `ImagePullBackOff` 时检查 Pod 事件、仓库连通性和拉取凭据。默认只需四个组件镜像,等待 Store/Helm test 复用已包含 curl 的组件镜像。 如显式覆盖 helper 镜像,该镜像也需要同步到内网仓库。 +启用 SeaTunnel 还需配置的 `apache/seatunnel` 镜像;启用 Vermeer 使用按 digest 固定的 `hugegraph/vermeer:helm-30b67d93e7d0`, +Master、Worker 和令牌初始化共用它。[官方发布运行](https://github.com/hugegraph/actions/actions/runs/40653109759)记录源码与镜像 digest, +详见 [Vermeer 部署指南](https://github.com/hugegraph/hugegraph/blob/e0f6b071c31e645127226ee5c1217a4fb4196732/helm/hugegraph/docs/vermeer.md)。 +自定义镜像覆盖还需同步更新 `vermeer.image.digest`,或显式清空以按标签选择;非空 digest 优先。 +基础 local PV 示例不包含 Vermeer 卷:单机增加两个,集群增加一个 Master 卷及每 Worker 一个卷,默认增加四个。 ## 2 端口和健康检查 {#2-ports-and-health} @@ -34,6 +39,26 @@ PD `/v1/ready` 需要 Raft leader,`/v1/health` 只说明监听器已启动,S Server 等待存储的默认上限为 300 秒,可以用 `server.storageWaitTimeoutSeconds` 调整。 本地模板渲染不能验证集群连通性、卷挂载或 JVM 运行容量。 +### 保留元数据命名空间 {#metadata-namespace} + +`metadataCluster: hg-test` 用于声明 Server 和 Store 访问 PD 时的元数据前缀(注入至 Server 的 `HG_SERVER_CLUSTER` 与 Store 的 `PD_CLUSTER`)。 +它并非 Raft 分组名,修改该值不会自动迁移历史元数据。**切勿**为了启用 Vermeer 将已有集群的前缀随意变更。若遇到 Server 查询正常但 Vermeer 扫描 Store 报 `Schema missing` 或 `no such vertex label`,应优先核查各组件的元数据前缀是否完全一致。 + +若历史自建环境使用了非默认前缀,升级前需在 Server Deployment 显式标注确认: + +```bash +kubectl -n hugegraph annotate deployment hugegraph-server \ + hugegraph.apache.org/metadata-cluster='<已核准的历史前缀>' +``` + +并在 values 中将 `metadataCluster` 设为同一值。对于采用 `store.updateStrategy.type=OnDelete` 策略的 Store 节点,需按[受控滚动升级流程](#6-rolling-store-images-safely)逐个重建 Pod,新配置方可生效。 + +> [!DETAILS]- 可选引擎运维:探针、凭据与拓扑切换 +> - **端口与健康状态**:SeaTunnel HTTP/UI 端口为 `8080`;Vermeer Master HTTP/gRPC 为 `6688/6689`,Worker 为 `6788/6789`。两款引擎的原生接口在两种模式下均保持私有 ClusterIP。 +> - **密码与会话**:初次部署生成的管理密码在 easy 模式下为 6 位字符,strict 模式下为 16 位;后续升级与模式切换均自动保留既有凭据。Vermeer 原生控制台直接在 **Admin credential** 填入 `-vermeer-admin` Secret 的 `password` 即可登录;Hubble 内部服务凭据已自动注入。 +> - **拓扑与副本切换**:修改 `seatunnel.mode` 或 `vermeer.mode` 前,请确保所有计算与导入批任务已结束并卸载内存图副本。单机与集群使用独立的数据卷声明,拓扑切换不会自动迁移本地文件。 +> - 完整恢复指引与进阶参数详见 [Vermeer 部署指南](https://github.com/apache/hugegraph/blob/master/helm/hugegraph/docs/vermeer.md) 与 [SeaTunnel 部署指南](https://github.com/apache/hugegraph/blob/master/helm/hugegraph/docs/seatunnel.md)。 + ### Helm test 检查范围 {#helm-test-coverage} `helm test` 逐 Server Pod 检查图绑定(默认检查 `DEFAULT-hugegraph`,可通过 `server.testGraphAlias` 指定图别名),并要求当前可见关联索引处于 `CREATED` 状态。 @@ -87,10 +112,11 @@ PD 会持久化分区设置,初始化后修改 chart 值不会覆盖已有元 ## 5 NetworkPolicy {#5-networkpolicy} -easy 模式及普通集群预设默认关闭,strict 模式默认开启;显式 `networkPolicy.enabled` 在任何模式都优先。 +基础服务的完整隔离在 easy 和普通集群预设下默认关闭,strict 下开启。 +SeaTunnel 和 Vermeer 的私有 ClusterIP、登录及原生 HTTP/RPC 入口限制始终生效,`networkPolicy.enabled=false` 也不例外。 集群网络插件必须支持策略执行,才有实际隔离效果。 生成策略允许组件间通信和 DNS,外部客户端、Ingress 控制器需要适当的 `networkPolicy..extraIngress`, -外部发现地址可能还需要 `extraEgress`。空列表合法但可能阻止外部访问;支持 Kubernetes 原生规则,包括允许全部的 `{}`,不要求额外来源确认。 +外部发现地址可能还需要 `extraEgress`。基础服务规则的空列表合法但可能阻止外部访问,并支持 Kubernetes 原生规则,包括允许全部的 `{}`。 按实际来源地址或 selector 配置,并考虑 NodePort SNAT 和控制器行为。 ```yaml @@ -114,6 +140,9 @@ networkPolicy: port: 8088 ``` +原生引擎的 ingress 只接受显式内部 Pod/namespace selector,拒绝 IP block 和允许全部的规则。 +上面的基础服务示例用于 Server,不用于对外暴露原生引擎;外部数据源可按需配置引擎 `extraEgress`,strict 进一步限制出站流量。 + ## 6 逐个升级 Store {#6-rolling-store-images-safely} 集群预设使用 `store.updateStrategy.type=OnDelete` 便于维护时控制逐个替换。 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-helm.md b/content/cn/docs/quickstart/hugegraph/hugegraph-helm.md index 94341fa9b9..a27dc870cf 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-helm.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-helm.md @@ -6,10 +6,11 @@ search_keywords: [HugeGraph Helm, Kubernetes, K8s, NodePort, Hubble, 内网部 --- 本文适用于 HugeGraph **1.8.0 及以后版本**,需要支持 OCI 的 Helm **3.10+** 和 Kubernetes (K8s) **1.23+**。 -可从 GHCR 或 Docker Hub 安装 `hugegraph-helm`。Chart 与 HugeGraph 服务分别维护版本,例如 `0.2.x` 的 Chart 可以部署 `1.8.x` 的服务。 +可从 GHCR 或 Docker Hub 安装 `hugegraph-helm`。Chart 与 HugeGraph 服务分别维护版本,例如 `0.4.x` 的 Chart 可以部署 `1.8.x` 的服务。 > [!WARNING] -> 当前开发 Chart 使用 `latest` 镜像,适合测试。 +> 当前开发 Chart 的 HugeGraph 基础组件使用 `latest` 镜像,适合测试。 +> 可选 Vermeer 已固定为经过验证的已发布镜像及 digest,见[可选引擎](#optional-engines)。 > 对应 HugeGraph 服务正式发布后,生产环境请选择稳定 Chart,并检查其中的镜像版本。 > 镜像标签可能变化;需要固定镜像内容时,请配置各组件的 `image.digest`。 @@ -60,7 +61,8 @@ helm install hugegraph ./hugegraph-helm -n hugegraph \ --create-namespace --wait --timeout 10m ``` -单节点试用预设替代默认安装,将各组件设为 1 副本并放宽调度限制,适合资源受限环境。 +单节点试用预设使用 HStore 的 1 PD + 1 Store + 1 Server 与 Hubble,放宽调度限制,适合资源受限环境。 +SeaTunnel 和 Vermeer 仍保持关闭。 ## 2 模式和访问入口 @@ -80,12 +82,13 @@ hugegraph-helm/ server-deployment.yaml # Server 引擎无状态 Deployment(注入 serviceId / graphSpace) hubble-deployment.yaml # 可视化控制台 Deployment tests/ # 冒烟验证测试钩子(helm test,支持自定义 testGraphAlias) - examples/ # 7 种生产场景完整示例(含 local PV、GitOps 凭据、Ingress 等) + examples/ # 部署示例(local PV、GitOps 凭据、可选引擎、Ingress 等) docs/operations.md # 生命周期进阶运维与故障恢复手册 ``` 默认 `mode: easy`,Server 和 Hubble 自动分配 NodePort,PD 保持 ClusterIP。可以直接使用 HTTP,无需域名、证书或额外确认参数。 -`mode: strict` 默认将 Server/Hubble 设为 ClusterIP,并开启 NetworkPolicy。两个模式都保留鉴权和持久化。 +`mode: strict` 默认将 Server/Hubble 设为 ClusterIP,并开启完整 NetworkPolicy 隔离。 +SeaTunnel 和 Vermeer 的原生 Service 在两种模式下始终为私有 ClusterIP,访问始终需要登录。两个模式都保留鉴权和持久化。 优先级是组件显式参数高于全局参数,全局参数高于模式默认值,显式 `false` 也会生效。 全局参数包括 `persistence.enabled` 和 `global.imageRegistry`,严格预设为 `values-strict.yaml`。 @@ -107,11 +110,64 @@ kubectl -n hugegraph get secret hugegraph-admin \ 默认 `externalTrafficPolicy: Cluster` 下,访问的节点不必运行目标 Pod。ClusterIP 是集群内部的虚拟服务地址,不是节点 IP,普通笔记本通常不能直接访问。 也可以运行 `kubectl -n hugegraph port-forward svc/hugegraph-hubble 8088:8088`,在本机打开 `http://127.0.0.1:8088`。 -三个客户端 Service 都支持 `externalTrafficPolicy`、`loadBalancerClass`、`loadBalancerIP`、`loadBalancerSourceRanges`、`externalIPs` +基础 PD/Server/Hubble 三类客户端 Service 支持 `externalTrafficPolicy`、`loadBalancerClass`、`loadBalancerIP`、`loadBalancerSourceRanges`、`externalIPs` 和 `allocateLoadBalancerNodePorts`,具体行为取决于集群及负载均衡实现。LoadBalancer 专用字段应配合对应类型使用。 NodePort 省略或设为 `0` 时自动分配,固定端口由 Kubernetes 根据集群配置的范围校验,chart 不再强制 `30000–32767`。 原 HTTP 暴露确认参数保留兼容,但不再阻止渲染。 +### 可选引擎:默认关闭 {#optional-engines} + +Chart 默认仅部署核心图服务(PD + Store + Server + Hubble)。针对数据集成与超大规模内存图计算场景,Chart 提供了 SeaTunnel 与 Vermeer 两款原生可选引擎,默认均保持关闭: + +| 可选引擎 | 默认状态 | 核心用途 | 运行模式 (`*.mode`) | 访问控制 | +|---|---|---|---|---| +| **Hubble** | 开启 (`true`) | 图数据可视化、多图空间与任务监控 | 单副本 Deployment | 跟随全局 `mode`(NodePort 或 ClusterIP) | +| **SeaTunnel** | 关闭 (`false`) | 异构数据源批流导入与跨图整库迁移 | `standalone` / `cluster` | 私有 ClusterIP(HTTP Basic 鉴权) | +| **Vermeer** | 关闭 (`false`) | 内存优先的高性能分布式图计算算法 | `standalone` / `cluster` | 私有 ClusterIP(Token 鉴权,Hubble 自动集成) | + +在 values 中将对应引擎的 `enabled` 设为 `true` 即可启用。两款引擎的原生接口均保留为私有 ClusterIP,管理密码与 Server 独立生成。 + +> [!DETAILS]- SeaTunnel:按需开启原生数据导入、导出与迁移 +> ```bash +> helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ +> -f helm/hugegraph/values-single.yaml \ +> --set seatunnel.enabled=true --set seatunnel.mode=standalone \ +> --wait --timeout 30m +> ``` +> +> - **运行模式**:`standalone` 启动 1 个 Hybrid 节点(轻量测试);`cluster` 模式启动多个同构节点(高并发批处理)。 +> - **访问与凭据**:Web 控制台使用独立 HTTP Basic 认证(默认用户 `admin`),密码保存在 `hugegraph-seatunnel-auth` Secret 中。端口 `5801` 仅供集群内部通信,严禁对外暴露。 +> - 完整作业配置、内置示例验证及跨图迁移流程见 [SeaTunnel 部署章节](#6-start-seatunnel) 或 [SeaTunnel 部署指南](https://github.com/apache/hugegraph/blob/master/helm/hugegraph/docs/seatunnel.md)。 + +> [!DETAILS]- Vermeer:按需开启内存图计算,自动接入 Hubble +> ```bash +> helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ +> -f helm/hugegraph/values-single.yaml \ +> --set vermeer.enabled=true --set vermeer.mode=standalone \ +> --wait --timeout 20m +> ``` +> +> ```text +> helm/hugegraph/ +> examples/ +> values-vermeer-standalone.yaml # 单机预设(单个 Pod 同时包含 Master 与 Worker) +> values-vermeer-cluster.yaml # 集群预设(1 个 Master Pod + 3 个 Worker Pods) +> docs/ +> vermeer.md # Vermeer 算法调用、参数映射与进阶运维手册 +> ``` +> +> - **运行模式**:`standalone` 单机模式在同一个 Pod 中运行 Master 协调与 Worker 计算容器;`cluster` 模式提供独立的 Master Pod 与可水平扩展的 Worker Pod 集群(默认 3 副本)。 +> - **Hubble 一键集成**:Hubble 会自动注入 Master 内部 Service 地址与访问令牌。管理员在 Hubble 控制台即可直接触发 15+ 种图算法,计算结果保留在内存副本中,不污染原图属性。 +> - **原生控制台直连**:原生 UI 默认以私有 ClusterIP 运行,可通过 `kubectl -n hugegraph port-forward svc/hugegraph-vermeer-master 6688:6688` 转发,并在 **Admin credential** 输入框中填入 `-vermeer-admin` Secret 的 `password` 登录。 +> - 镜像固定策略与进阶凭据管理见 [Vermeer 部署指南](https://github.com/hugegraph/hugegraph/blob/master/helm/hugegraph/docs/vermeer.md)。 + +### PD 元数据命名空间 {#metadata-namespace} + +顶层参数 `metadataCluster`(默认 `hg-test`)定义了 PD 中元数据的集群前缀。Chart 会将该值同步注入 Server 的 `HG_SERVER_CLUSTER` 与 Store 的 `PD_CLUSTER`,使得 Store 在接收 Vermeer 扫描请求时能准确定位 Server 写入的 Schema。 + +> [!IMPORTANT] +> `metadataCluster` 是元数据存储的前缀空间,而非 Raft 分组名。修改该值**不会**自动迁移已有图数据。开启 Vermeer 时必须保留原有集群的前缀值;若加载图时遇到 `Schema missing` 或 `no such vertex label`,请优先检查各组件的元数据前缀是否一致。自定义前缀的核验流程请参考 [元数据命名空间运维指引](./hugegraph-helm-operations.md#metadata-namespace)。 + ### 服务身份与图空间 默认 `server.serviceId` 与 `server.graphSpace` 均为 `DEFAULT`。如需将 Server 注册到指定的业务服务名或图空间,可通过 values 指定: @@ -141,6 +197,10 @@ GitOps 凭据、仅 API 部署以及无域名的 HTTP Ingress。 等待 PD 的 init container 复用 Store 镜像,Helm test 复用 Server 镜像。这两个镜像已包含 `curl`,无需额外 curl 镜像或启动时安装软件。 自定义镜像可以继续使用显式 `store.waitImage` 和 `server.waitImage` 覆盖。 +启用 SeaTunnel 时还需同步其配置的 `apache/seatunnel` 镜像,启用 Vermeer 时使用上文固定的已发布构建。 +Vermeer 的 Master、Worker 和服务令牌初始化复用同一镜像;所有已启用镜像及显式 helper 覆盖都应同步。 +使用自定义 Vermeer 镜像时,还需更新 `vermeer.image.digest`,或在按标签选择时显式清空;非空 digest 优先于标签。 + ### 存储配置 `storageClassName` 为空时继承集群默认 StorageClass,`"-"` 表示显式不使用 StorageClass。 @@ -149,6 +209,11 @@ local PV 需要预建宿主机目录、匹配容量、`nodeAffinity` 和 `kubern 新建临时实例可以设置 `persistence.enabled=false` 使用 `emptyDir`,组件存储开关优先。 PVC 失败不会自动降级为临时存储,已经初始化的 PD/Store 切换存储模式会被拒绝。 +Vermeer 的 Master 和 Worker 使用独立数据卷,单 Pod 模式也不共享 Pebble 目录。 +单机模式增加 2 个卷;集群模式为 1 个 Master 卷加每 Worker 1 个卷,默认增加 4 个。 +因此默认基础拓扑加 Vermeer 单机/集群分别需要 9/11 个卷,轻量 HugeGraph 预设加 Vermeer 单机需要 5 个。 +七卷 local PV 示例仅覆盖基础安装;SeaTunnel checkpoint 仍使用临时本地文件。 + ### 离线安装和私库 先选择最新版本,再按该版本下载 Chart,将 `.tgz` 搬入内网。可以直接安装,也可以接入已有的 OCI 或 HTTP 私库。 @@ -208,6 +273,21 @@ GitOps 使用加密的固定值,或预先创建 Secret 并配置 `pd.auth.exis Hubble 默认开启,当前镜像的登录仍依赖 Server 鉴权。只使用 API 可以设置 `hubble.enabled=false`, 如需关闭 API 鉴权,再设置 `server.auth.enabled=false`。未使用的凭据配置可以留在 values 中。 +可选引擎的原生入口始终需要登录,easy 模式也不例外。新生成的独立管理员密码在 easy 模式下为 6 位随机字母数字,strict 模式下为 16 位。 +SeaTunnel 使用 HTTP Basic,Vermeer 使用 `-vermeer-admin` Secret 与 **Admin credential** 登录。 +升级保留既有密码,切换模式也不会自动轮换。 +Vermeer 管理密码可通过 `vermeer.auth.admin.password` 或 `vermeer.auth.admin.existingSecret/key` 配置,仅 Master 读取。 +原生管理按需使用经过授权的 port-forward,Hubble 的服务凭据自动配置。 + +> [!DETAILS]- 高级:Vermeer 部署凭据与 GitOps +> GitOps 需保留稳定的管理员凭据和独立的内部服务身份。 +> 服务签名因子通过 `vermeer.auth.value` 或 `vermeer.auth.existingSecret/key` 配置,Hubble 自动生成服务令牌; +> 日常 Hubble 使用和原生密码登录都不需要手工获取该令牌。 +> 更改外部管理密码 Secret 只需重启 Master;服务签名因子轮换需重启 Master 与 Hubble,无需重启 Worker。 +> Worker Schema 读取默认复用部署管理员,也可通过 `vermeer.hugegraphCredentials.existingSecret/key` 指向现有用户的受保护用户名密码 JSON。 +> 该用户需具备目标图的 Schema 权限,Chart 不会自动创建账户。 + + ## 5 升级和验证 使用 Helm 原生值复用保留已有部署的 UI、入口、存储和凭据设置。 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md index f00f2bb539..919a3b1af9 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -110,6 +110,9 @@ set -a; . ./.env; set +a 单独运行当前主线构建的 Store 时,将示例地址换成 Store 与 PD 均可路由的实际地址: +下面的 `mkdir -p` 和非递归 `chown` 仅适用于新建空目录。复用旧 root 镜像写入的数据时,先停止 Store, +执行一次 `sudo chown -R 10001:10001 /path/to/storage`,再启动 UID 10001 镜像。 + ```bash # master 构建的镜像以 UID 10001 运行,宿主机数据目录必须对该用户可写 mkdir -p /path/to/storage && sudo chown 10001:10001 /path/to/storage diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-pd.md b/content/cn/docs/quickstart/hugegraph/hugegraph-pd.md index 11b4ae136e..1f444b53b5 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-pd.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-pd.md @@ -81,6 +81,9 @@ docker build -f hugegraph-pd/Dockerfile -t hugegraph/pd:local . 单独运行当前主线构建的 PD 时,先设置部署专用密钥,并把示例地址替换为容器可达的真实地址: +下面的 `mkdir -p` 和非递归 `chown` 仅适用于新建空目录。复用旧 root 镜像写入的数据时,先停止 PD, +执行一次 `sudo chown -R 10001:10001 /path/to/data`,再启动 UID 10001 镜像。 + ```bash export HG_PD_AUTH_SECRET_KEY="$(openssl rand -hex 24)" # master 构建的镜像以 UID 10001 运行,宿主机数据目录必须对该用户可写 diff --git a/content/cn/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source.md b/content/cn/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source.md index b528728072..e9b24e0fab 100644 --- a/content/cn/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source.md +++ b/content/cn/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source.md @@ -19,6 +19,9 @@ SeaTunnel 3.0+ 提供 HugeGraph Source,支持图模型自动发现和多标签 这两个任务只迁移指定标签和属性,不会完整复制源图的索引、TTL 等全部 Schema 配置。运行期间应暂停源图写入,避免两个任务读到不同时间的数据;完成后核对顶点、边数量及抽样属性。 +需要保留普通边标签的 Schema ID 和索引定义时,可参考 Helm Chart 中的 +[Schema 复制示例](https://github.com/apache/hugegraph/tree/96ce802a25672f8b7b0d08c49e68db694231f5e4/helm/hugegraph/examples/seatunnel/clone)。 + [![跨图迁移时重建主键可能改变顶点 ID;使用 CUSTOMIZE_STRING 保留原 ID,确保边端点匹配](/cn/docs/images/seatunnel/seatunnel-preserve-ids-zh.png)](/cn/docs/images/seatunnel/seatunnel-preserve-ids-zh.png) ### 1.1 先迁移顶点 diff --git a/content/en/docs/quickstart/computing/hugegraph-vermeer.md b/content/en/docs/quickstart/computing/hugegraph-vermeer.md index 4a48997f9f..4abb496940 100644 --- a/content/en/docs/quickstart/computing/hugegraph-vermeer.md +++ b/content/en/docs/quickstart/computing/hugegraph-vermeer.md @@ -14,7 +14,8 @@ Vermeer is a high-performance, memory-first graph computing framework written in The master handles communication, forwarding, and aggregation, with modest computation and resource usage. Workers store graph data and execute tasks, consuming most memory and CPU. gRPC handles internal communication; REST provides external APIs. -At startup, built-in defaults are loaded first, followed by the `[default]` section of `config/.ini` under the working directory, then explicit command-line overrides. For example, `--env=master` reads `config/master.ini`. The Docker image copies repository configuration to `/go/bin/config/` and works from `/go/bin/`. A host bind mount hides the bundled files, so it must contain every ini file used. The configuration reader does not read environment variables. +At startup, built-in defaults are loaded first, followed by the `[default]` section of `config/.ini` under the working directory, then explicit command-line overrides. For example, `--env=master` reads `config/master.ini`. The Docker image copies repository configuration to `/go/bin/config/` and works from `/go/bin/`. A host bind mount hides the bundled files, so it must contain every ini file used. Ordinary INI keys are not mapped automatically from environment variables. Dedicated deployment credential settings also support +`VERMEER_PD_PASSWORD_FILE`, `VERMEER_PD_PEERS`, `VERMEER_HUGEGRAPH_CREDENTIALS_FILE` and the documented token-factor/password file settings. Default ports: @@ -37,10 +38,44 @@ flowchart LR ### 1.2 Running Vermeer +> [!DETAILS]- Helm: Deploy Vermeer with HugeGraph and Hubble +> The HugeGraph Helm Chart supports deploying the native Vermeer in-memory graph computing engine, disabled by default. +> From the HugeGraph repository root, enable Vermeer with: +> +> ```bash +> helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ +> -f helm/hugegraph/values-single.yaml \ +> --set vermeer.enabled=true --set vermeer.mode=standalone \ +> --wait --timeout 20m +> ``` +> +> ```filetree {wrap=true title="Vermeer Core Files and Layout"} +> helm/hugegraph/ +> examples/ +> values-vermeer-standalone.yaml # Standalone preset (Master and Worker run in a single Pod) +> values-vermeer-cluster.yaml # Cluster preset (dedicated Master Pod + distributed Worker Pods) +> templates/ +> vermeer-master.yaml # Master scheduling and coordination workload +> vermeer-worker.yaml # Worker in-memory computing workload +> vermeer-auth-secret.yaml # Admin credentials and Hubble internal service tokens +> docs/ +> vermeer.md # Advanced algorithm parameters, API mapping and operations +> ``` +> +> **Architecture & Topologies**: +> - **Standalone Mode (`standalone`)**: Master and Worker run as distinct containers in the same Pod with minimal overhead, suitable for local exploration and small-scale testing. +> - **Cluster Mode (`cluster`)**: Set `vermeer.mode=cluster` and `vermeer.worker.replicas=3` to provision a dedicated Master coordinator Pod and multiple distributed Worker computing Pods. +> - **Graph Service Integration**: Master queries PD via gRPC for partition topology, while Workers scan graph slices directly from Store via gRPC. Hubble automatically injects the Master URL and communication token, allowing users to load graphs and execute 15+ graph algorithms directly from the UI. +> +> **Key Operational Considerations**: +> 1. **Preserve Metadata Namespace**: Retain the Chart default `metadataCluster: hg-test`. This key synchronizes Server `HG_SERVER_CLUSTER` and Store `PD_CLUSTER` so that Store can properly resolve schemas. Do not arbitrarily change this value on existing clusters to avoid `Schema missing` errors during load tasks. +> 2. **Access Native Web Dashboard**: The native UI runs as a private ClusterIP Service. Use `kubectl port-forward svc/-vermeer-master 6688:6688` to access it locally, and log in by entering the password from the `-vermeer-admin` Secret directly into **Admin credential** (no username required), which grants a 24-hour versioned session. +> 3. For image pinning, persistent volume sizing, and recovery procedures, refer to the [Helm Optional Engines Guide](/docs/quickstart/hugegraph/hugegraph-helm/#optional-engines) and [Vermeer Deployment Guide](https://github.com/hugegraph/hugegraph/blob/master/helm/hugegraph/docs/vermeer.md). + > [!WARNING] > In production, enable [Server authentication and authorization](/docs/config/config-authentication/), an IP allowlist, and minimum permissions; retain `audit-*.log`. Server Auth does not protect independent Vermeer, PD, or Store APIs. Restrict these HTTP/gRPC ports to trusted networks and callers, and configure access controls at the Vermeer external entry point. > -> `master.ini` defaults to `auth=none`, disabling authentication for ordinary and administrative APIs. Local examples publish loopback only. Before remote access, enable `auth=token` or restrict callers through a protected network or gateway. +> Manual standalone deployments default to `auth=none` in `master.ini`, and the Docker examples below publish to loopback only. When deploying manually for remote access, always enable `auth=token`. The Helm deployment enables token authentication and private ClusterIP isolation by default. Both Docker options need a host configuration directory. From the Vermeer repository root, copy the supplied [`master.ini`](https://github.com/apache/hugegraph-computer/blob/master/vermeer/config/master.ini) and [`worker.ini`](https://github.com/apache/hugegraph-computer/blob/master/vermeer/config/worker.ini) templates. The mount hides image configuration under `/go/bin/config`, so never mount an empty directory or your entire home directory there: diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-helm-operations.md b/content/en/docs/quickstart/hugegraph/hugegraph-helm-operations.md index c4ceec088c..dc10c1488e 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-helm-operations.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-helm-operations.md @@ -11,7 +11,7 @@ Commands assume release/namespace `hugegraph`; substitute the actual workload na ## 1 Storage and private images -The default installation needs seven persistent volumes. Dynamic storage requires a working provisioner; static local storage requires prepared directories +The base installation needs seven persistent volumes. Dynamic storage requires a working provisioner; static local storage requires prepared directories and matching PVs on the named nodes. Use the [complete examples](https://github.com/apache/hugegraph/tree/master/helm/hugegraph/examples) for all seven local PVs, `WaitForFirstConsumer`, registry mirrors and GitOps Secrets. An initially Pending PVC with `WaitForFirstConsumer` can be normal; persistent Pending needs PVC/Pod events and PV matching checks. @@ -26,6 +26,13 @@ kubectl get pv,storageclass For `ImagePullBackOff`, inspect Pod events and registry access/credentials. The default image inventory contains four component images; Store wait and Helm test reuse existing component images with `curl` already installed. Explicit helper-image overrides must also be mirrored. +Enabled SeaTunnel adds its configured `apache/seatunnel` image. Enabled Vermeer uses `hugegraph/vermeer:helm-30b67d93e7d0`, pinned by digest +and reused by Master, Worker and token initialization. Its [official publishing run](https://github.com/hugegraph/actions/actions/runs/40653109759) +records the source and image digest; see the +[Vermeer deployment guide](https://github.com/hugegraph/hugegraph/blob/e0f6b071c31e645127226ee5c1217a4fb4196732/helm/hugegraph/docs/vermeer.md). +Custom image overrides must also update `vermeer.image.digest`, or explicitly clear it to select by tag; digest takes priority. +The base local-PV example omits Vermeer claims: standalone adds two, +while cluster adds one Master claim plus one per Worker (four additional claims by default). ## 2 Ports and health @@ -35,6 +42,26 @@ PD `/v1/ready` requires a Raft leader; `/v1/health` only confirms the listener. Server waits up to `server.storageWaitTimeoutSeconds` (default 300) before its application startup; adjust it for slow storage bootstrap. No local Helm rendering verifies live cluster reachability, volumes or JVM capacity. +### Preserve the metadata namespace {#metadata-namespace} + +`metadataCluster: hg-test` sets the PD metadata namespace prefix (injected into Server `HG_SERVER_CLUSTER` and Store `PD_CLUSTER`). +It is not a Raft group name; changing it will not automatically migrate existing metadata. **Never** arbitrarily change this prefix on an initialized cluster to enable Vermeer. If Server REST queries succeed but Vermeer scanning Store fails with `Schema missing` or `no such vertex label`, check that the namespace prefix matches across all components. + +If a legacy environment used a custom prefix, record it explicitly on the Server Deployment before upgrading: + +```bash +kubectl -n hugegraph annotate deployment hugegraph-server \ + hugegraph.apache.org/metadata-cluster='' +``` + +and ensure `metadataCluster` matches this value in your values file. For Store nodes using `store.updateStrategy.type=OnDelete`, follow the [rolling upgrade procedure](#6-rolling-store-images-safely) to recreate Pods one by one so the new settings take effect. + +> [!DETAILS]- Optional Engines Operations: Probes, Credentials, and Topology Switching +> - **Ports & Health**: SeaTunnel HTTP/UI runs on `8080`; Vermeer Master HTTP/gRPC listens on `6688/6689`, and Worker on `6788/6789`. Both engines stay behind private ClusterIP in both modes. +> - **Credentials & Sessions**: Generated admin passwords use 6 characters in easy mode or 16 in strict mode, retained across upgrades and mode changes. In Vermeer native UI, enter the password from Secret `-vermeer-admin` into **Admin credential**; Hubble internal service tokens are auto-wired. +> - **Topology Changes**: Ensure active computations and batch ingestion jobs are complete and in-memory graphs are unloaded before changing `seatunnel.mode` or `vermeer.mode`. Standalone and cluster use separate PVC claims, so mode changes do not migrate local files. +> - See the [Vermeer Guide](https://github.com/apache/hugegraph/blob/master/helm/hugegraph/docs/vermeer.md) and [SeaTunnel Guide](https://github.com/apache/hugegraph/blob/master/helm/hugegraph/docs/seatunnel.md) for full recovery and configuration details. + ### Helm test coverage `helm test` validates graph binding on every Server Pod (`DEFAULT-hugegraph` by default, or configured via `server.testGraphAlias`), and requires @@ -93,11 +120,12 @@ before changing a live cluster; the chart does not migrate existing shard metada ## 5 NetworkPolicy -Easy mode, including the ordinary cluster preset, defaults policies off. Strict mode enables them; explicit `networkPolicy.enabled` wins in either mode. +Easy mode and the ordinary cluster preset leave full base-service isolation disabled by default; strict enables it. +SeaTunnel and Vermeer always keep private ClusterIP Services, login and native HTTP/RPC ingress restrictions, even with `networkPolicy.enabled=false`. The cluster network plugin must enforce NetworkPolicy for these resources to have an effect. The generated policies admit component traffic and DNS. External clients and Ingress controllers need suitable `networkPolicy..extraIngress` rules; external discovery addresses may need matching `extraEgress`. -An empty list is valid and can block external traffic. Native Kubernetes rules are accepted, including `{}` to allow all; +For base-service rules, an empty list can block outside traffic and native Kubernetes rules include `{}` to allow all; there is no separate source-acknowledgement gate. Use your actual source addresses/selectors, considering NodePort SNAT and controller behavior. ```yaml @@ -121,6 +149,10 @@ networkPolicy: port: 8088 ``` +Native engine ingress accepts explicit internal Pod/namespace selectors only; IP blocks and allow-all forms are rejected. +The base-service example above applies to Server, not native engine exposure. External data destinations can use engine `extraEgress`; +strict mode additionally enforces egress isolation. + ## 6 Rolling Store images safely Use `store.updateStrategy.type=OnDelete` for a controlled maintenance rollout, as the cluster preset does. diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-helm.md b/content/en/docs/quickstart/hugegraph/hugegraph-helm.md index 994bf5f437..49e566f8dc 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-helm.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-helm.md @@ -7,10 +7,11 @@ search_keywords: [HugeGraph Helm, Kubernetes, K8s, NodePort, Hubble, OCI, GHCR, This guide covers HugeGraph **1.8.0 and later**, using Helm **3.10+** with OCI support and Kubernetes (K8s) **1.23+**. Install `hugegraph-helm` from GHCR or Docker Hub. Chart and HugeGraph versions are maintained separately: -for example, a `0.2.x` Chart can deploy a `1.8.x` service. +for example, a `0.4.x` Chart can deploy a `1.8.x` service. > [!WARNING] -> The current development Chart uses `latest` images and is intended for testing. +> The current development Chart uses `latest` images for the base HugeGraph components and is intended for testing. +> Optional Vermeer is pinned to a verified published image and digest; see [optional engines](#optional-engines). > For production, choose a stable Chart once the corresponding HugeGraph service has been released, and review its image versions. > Image tags can change; set each component's `image.digest` to keep using the same image content. @@ -64,7 +65,8 @@ helm install hugegraph ./hugegraph-helm -n hugegraph \ --create-namespace --wait --timeout 10m ``` -The single-node preset replaces the default installation with one replica per component and relaxed scheduling constraints. +The single-node preset selects 1 PD + 1 Store + 1 Server using HStore, plus Hubble, with relaxed scheduling constraints. +It leaves SeaTunnel and Vermeer disabled. ## 2 Modes and entry points @@ -84,12 +86,14 @@ hugegraph-helm/ server-deployment.yaml # Server Deployment (injects serviceId and graphSpace) hubble-deployment.yaml # Hubble UI Deployment tests/ # Smoke validation hook (helm test, supports custom testGraphAlias) - examples/ # Seven complete production scenario examples (local PVs, GitOps secrets, Ingress) + examples/ # Deployment examples: local PVs, GitOps credentials, optional engines and Ingress docs/operations.md # Lifecycle operations and recovery guide ``` `mode: easy` is the default. Server and Hubble use automatic NodePorts; PD stays ClusterIP. HTTP works without a domain, certificate or acknowledgement. -`mode: strict` defaults Server/Hubble to ClusterIP and enables NetworkPolicy. Both modes keep authentication and persistence. +`mode: strict` defaults Server/Hubble to ClusterIP and enables full NetworkPolicy isolation. +SeaTunnel and Vermeer native Services stay private ClusterIP services and require login in both modes. +Both modes keep authentication and persistence. Explicit component parameters take priority over global parameters, which take priority over mode defaults; explicit `false` remains an override. The global parameters are `persistence.enabled` and `global.imageRegistry`. Strict mode is also available through `values-strict.yaml`. @@ -111,11 +115,66 @@ Open Hubble on its assigned NodePort and log in as `admin` with this password. A The internal ClusterIP is a virtual Service address, not a node IP; a laptop normally cannot reach it directly. Use `kubectl -n hugegraph port-forward svc/hugegraph-hubble 8088:8088` for a local browser at `http://127.0.0.1:8088`. -Each client Service supports `externalTrafficPolicy`, `loadBalancerClass`, `loadBalancerIP`, `loadBalancerSourceRanges`, `externalIPs` +The base PD/Server/Hubble client Services support `externalTrafficPolicy`, `loadBalancerClass`, `loadBalancerIP`, `loadBalancerSourceRanges`, `externalIPs` and `allocateLoadBalancerNodePorts`, subject to the cluster/controller's capabilities. Use LoadBalancer fields with that Service type. NodePort omission or `0` means automatic allocation. Fixed ports are validated against the cluster's configured range by Kubernetes; the chart does not force `30000–32767`. Legacy HTTP acknowledgement fields remain accepted but no longer gate rendering. +### Optional engines: disabled by default {#optional-engines} + +By default, the Chart provisions only the core graph services (PD + Store + Server + Hubble). For data migration and distributed memory graph computing, the Chart offers SeaTunnel and Vermeer as native optional engines, both disabled by default: + +| Engine | Default | Purpose | Topologies (`*.mode`) | Access & Security | +|---|---|---|---|---| +| **Hubble** | Enabled (`true`) | Graph visualization, multi-graphspace management & metrics | Single-replica Deployment | Follows global `mode` (NodePort or ClusterIP) | +| **SeaTunnel** | Disabled (`false`) | Heterogeneous data ingestion & full graph migration | `standalone` / `cluster` | Private ClusterIP (HTTP Basic auth) | +| **Vermeer** | Disabled (`false`) | Memory-first high-performance distributed graph algorithms | `standalone` / `cluster` | Private ClusterIP (Token auth, auto-wired to Hubble) | + +Enable either engine by setting its `enabled` key to `true` in values. Both engines maintain private ClusterIP services, and their administrative passwords are generated independently from Server. + +> [!DETAILS]- SeaTunnel: On-demand Data Import, Export, and Migration +> ```bash +> helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ +> -f helm/hugegraph/values-single.yaml \ +> --set seatunnel.enabled=true --set seatunnel.mode=standalone \ +> --wait --timeout 30m +> ``` +> +> - **Topologies**: `standalone` runs 1 Hybrid node for lightweight testing; `cluster` runs multiple homogeneous nodes for high-throughput batch jobs. +> - **Authentication**: Web dashboard uses independent HTTP Basic auth (default `admin`), stored in the `hugegraph-seatunnel-auth` Secret. Port `5801` is strictly for cluster internal communication. +> - See the [SeaTunnel Deployment Section](#6-start-seatunnel) or [SeaTunnel Guide](https://github.com/apache/hugegraph/blob/master/helm/hugegraph/docs/seatunnel.md) for full job configs and migration steps. + +> [!DETAILS]- Vermeer: In-Memory Graph Computing Integrated with Hubble +> ```bash +> helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ +> -f helm/hugegraph/values-single.yaml \ +> --set vermeer.enabled=true --set vermeer.mode=standalone \ +> --wait --timeout 20m +> ``` +> +> ```text +> helm/hugegraph/ +> examples/ +> values-vermeer-standalone.yaml # Standalone preset (Master and Worker combined in one Pod) +> values-vermeer-cluster.yaml # Cluster preset (1 dedicated Master Pod + 3 distributed Worker Pods) +> docs/ +> vermeer.md # Advanced algorithm parameters, API mapping and operations +> ``` +> +> - **Topologies**: `standalone` runs Master and Worker in one Pod; `cluster` provides a separate Master coordinator Pod and scalable Worker computing Pods (default 3 replicas). +> - **Hubble Integration**: Hubble automatically acquires the internal Master Service URL and service tokens. Operators can trigger 15+ graph algorithms directly from Hubble without manual token configuration. +> - **Native Dashboard Access**: Forward the private Service via `kubectl -n hugegraph port-forward svc/hugegraph-vermeer-master 6688:6688` and sign in with the password from Secret `-vermeer-admin` under **Admin credential**. +> - For image pinning and credentials management, see the [Vermeer Deployment Guide](https://github.com/hugegraph/hugegraph/blob/master/helm/hugegraph/docs/vermeer.md). + +### PD metadata namespace {#metadata-namespace} + +The top-level `metadataCluster` parameter (default `hg-test`) specifies the PD metadata cluster prefix. The Chart injects this value into Server `HG_SERVER_CLUSTER` and Store `PD_CLUSTER`, ensuring Store correctly locates the Schema initialized by Server when processing Vermeer graph scans. + +> [!IMPORTANT] +> `metadataCluster` is a metadata namespace prefix, not a Raft group name. Modifying it **does not** automatically migrate existing graph data. Always preserve the original cluster prefix when enabling Vermeer; if graph load tasks fail with `Schema missing` or `no such vertex label`, check that the namespace prefix matches across all components. Refer to the [Metadata Namespace Operations Guide](./hugegraph-helm-operations.md#metadata-namespace) for details. +Standard older releases using `hg-test` need no namespace-adoption step. For a custom legacy prefix, follow +[metadata namespace adoption](./hugegraph-helm-operations.md#metadata-namespace) before upgrading. + ### Server service identity and graph space Both `server.serviceId` and `server.graphSpace` default to `DEFAULT`. To register the Server with a specific service name or graph space in PD: @@ -146,6 +205,10 @@ The default mirror inventory is only `hugegraph/pd`, `hugegraph/store`, `hugegra Store's PD wait init container and the Helm test reuse Store and Server images respectively, which already contain `curl`. No separate curl image or runtime package installation is required. Explicit `store.waitImage` / `server.waitImage` overrides remain supported. +Enabled SeaTunnel additionally needs its configured `apache/seatunnel` image; enabled Vermeer uses the published build pinned above. +Vermeer reuses that image for Master, Worker and service-token initialization. Mirror every enabled image and explicit helper override. +For a custom Vermeer image, also update `vermeer.image.digest`, or explicitly clear it when selecting by tag; a nonempty digest takes priority. + ### Storage configuration For storage, empty `storageClassName` inherits the cluster default, while `"-"` explicitly selects no StorageClass. @@ -154,6 +217,11 @@ Local PVs require prepared host directories, matching capacities, `nodeAffinity` For a new temporary installation, set `persistence.enabled=false` to use `emptyDir`; component storage flags override it. The chart never replaces a failed PVC with temporary storage and rejects PD/Store storage-mode changes on initialized releases. +Vermeer uses separate Master and Worker claims, even in its single-Pod mode: two additional volumes in standalone, +or one Master volume plus one per Worker in cluster mode. With the default base topology this means 9 or 11 volumes respectively. +The small HugeGraph preset plus standalone Vermeer needs 5. The seven-volume local example covers only the base installation. +SeaTunnel checkpoints remain temporary local files. + ### Offline installation and internal repositories Select the latest version, download that Chart and move the `.tgz` into your network. @@ -215,6 +283,21 @@ update the Secret/source to that same password, then restart the Server Deployme Hubble is enabled by default and its current login requires Server authentication. Set `hubble.enabled=false` for API-only access; optionally also set `server.auth.enabled=false` for an unauthenticated API. Inactive credential settings may remain in your values file. +Enabled optional engines always require native login, including in easy mode. Newly generated administrator passwords use 6 random alphanumeric +characters in easy mode or 16 in strict mode. SeaTunnel uses HTTP Basic, while Vermeer uses Secret `-vermeer-admin` and its +**Admin credential** login. Upgrades retain existing passwords, including when changing mode; mode changes do not rotate them automatically. +Vermeer password sources are `vermeer.auth.admin.password` or `vermeer.auth.admin.existingSecret/key`; +only Master reads that password. Native administration uses authorized port-forwarding, and Hubble service credentials are automatic. + +> [!DETAILS]- Advanced: Vermeer deployment credentials and GitOps +> For GitOps, preserve stable administrator credentials and the separate internal service identity. +> The service signing factor uses `vermeer.auth.value` or `vermeer.auth.existingSecret/key`; Hubble generates its service token automatically. +> Ordinary Hubble use and native password login do not require retrieving that token. +> Changing an external native administrator Secret needs a Master restart only. Service-factor rotation requires Master and Hubble, +> not Workers. Worker Schema reads reuse the deployment administrator unless `vermeer.hugegraphCredentials.existingSecret/key` +> supplies protected username/password JSON for an existing account with the required Schema permissions. No account is created automatically. + + ## 5 Upgrade and verification Use native Helm value reuse to preserve an existing deployment's UI, entry, storage and credentials: diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md index 708931b9e7..5d39d077ee 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-hstore.md @@ -111,6 +111,9 @@ Multi-node service names are `pd0`–`pd2`, `store0`–`store2`, and `server0` To run master-built Store alone, replace example addresses with actual routable Store and PD addresses: +The `mkdir -p` and nonrecursive `chown` below are for a new empty directory. Before reusing data written by a root image, stop Store and run +`sudo chown -R 10001:10001 /path/to/storage` once, then start the UID 10001 image. + ```bash # Master-built images run as UID 10001; the host data directory must be writable by it mkdir -p /path/to/storage && sudo chown 10001:10001 /path/to/storage diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-pd.md b/content/en/docs/quickstart/hugegraph/hugegraph-pd.md index 85d14103da..ce534da44f 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-pd.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-pd.md @@ -82,6 +82,9 @@ docker build -f hugegraph-pd/Dockerfile -t hugegraph/pd:local . To run master-built PD alone, first set a deployment-specific secret and replace example addresses with addresses reachable by the container: +The `mkdir -p` and nonrecursive `chown` below are for a new empty directory. Before reusing data written by a root image, stop PD and run +`sudo chown -R 10001:10001 /path/to/data` once, then start the UID 10001 image. + ```bash export HG_PD_AUTH_SECRET_KEY="$(openssl rand -hex 24)" # Master-built images run as UID 10001; the host data directory must be writable by it diff --git a/content/en/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source.md b/content/en/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source.md index 7b5437ac89..b4768bcd2f 100644 --- a/content/en/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source.md +++ b/content/en/docs/quickstart/toolchain/export-migration/hugegraph-seatunnel-source.md @@ -20,6 +20,9 @@ The following example migrates `person` vertices and `knows` edges from a source These two jobs migrate only the selected labels and properties. They do not copy every source schema setting, such as indexes and TTLs. Pause writes to the source graph during the migration so both jobs read a consistent point in time. Afterward, compare vertex and edge counts and sample properties. +To retain schema IDs and indexes for ordinary edge labels, see the Chart's +[schema-copy examples](https://github.com/apache/hugegraph/tree/96ce802a25672f8b7b0d08c49e68db694231f5e4/helm/hugegraph/examples/seatunnel/clone). + [![Regenerating a primary key can change 1:marko to 2:marko; CUSTOMIZE_STRING preserves the original ID so edge endpoints still resolve](/docs/images/seatunnel/seatunnel-preserve-ids-en.png)](/docs/images/seatunnel/seatunnel-preserve-ids-en.png) ### 1.1 Migrate vertices first