Skip to content

optimize HBase 1.x request-path CPU usage - #330

Merged
WeiXinChan merged 4 commits into
oceanbase:masterfrom
hhlgt:optimize/client-cpu-performance
Sep 1, 2026
Merged

optimize HBase 1.x request-path CPU usage#330
WeiXinChan merged 4 commits into
oceanbase:masterfrom
hhlgt:optimize/client-cpu-performance

Conversation

@hhlgt

@hhlgt hhlgt commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR optimizes the request and response hot paths of the HBase 1.x client.

It reduces CPU usage, byte-array copying, and temporary object allocation for
Get, Batch Get, Scan, and Put operations while preserving existing HBase client
semantics and compatibility paths.

Changes

Read path

  • Consume compact K/Q/T/V result batches provided by obkv-table-client-java.
  • Avoid materializing intermediate per-field ObObj values when compact results
    are available.
  • Introduce OHBaseResultCell, a lightweight immutable Cell implementation
    backed directly by decoded field arrays.
  • Optimize point Get result assembly by:
    • validating the returned row key once
    • reusing the canonical row-key array
    • avoiding temporary KeyValue objects and row-key clones
  • Optimize closestRowBefore by retaining only cells belonging to the greatest
    returned row key without cloning row keys from temporary cells.
  • Consume compact Scan rows directly and construct HBase 1.x-compatible
    KeyValue results without rebuilding generic rows.
  • Avoid intermediate family/qualifier arrays for TableGroup
    family\0qualifier columns.
  • Preallocate result containers and reuse a static HBase cell comparator.

Batch Get

  • Add a dedicated result-processing path for pure Get batches.
  • Preserve request/result ordering without maintaining an unnecessary
    per-operation result mapping.
  • Consume compact K/Q/T/V batches directly.
  • Preserve empty-result and per-operation exception behavior.
  • Validate malformed result counts and K/Q/T/V payloads.
  • Keep generic ObObj decoding as a compatibility fallback.

Put path

  • When autoFlush is enabled, execute synchronous Puts directly without passing
    through the BufferedMutator queue, heap-size accounting, temporary linked
    lists, and an additional flush cycle.
  • Flush existing buffered mutations before entering the direct path to preserve
    write ordering.
  • Fall back to the buffered path if pending buffered state cannot be drained.
  • Build Put V2 requests using compact cell arrays.
  • Preallocate row, column-family, and cell containers.
  • Reuse qualifier/value backing-array regions on the synchronous autoFlush path.
  • Clone qualifier/value data on delayed-flush paths where the request outlives
    the current call.
  • Avoid duplicate value cloning when encoding cells with TTL.
  • Validate Put mutations using the original family/cell map without
    materializing deprecated compatibility structures.
  • Preserve the existing server-version gate for multi-column-family writes.

Default behavior

The optimizations are applied automatically when the request shape and lifetime
make them safe. No new public tuning switches are introduced.

In particular:

  • compact Batch Get decoding is selected automatically for eligible HBase
    requests
  • compact Put encoding is enabled by default
  • zero-copy Put cell handling is limited to synchronous autoFlush requests
  • buffered writes continue to use owned copies
  • Get uses lightweight result cells while Scan preserves the HBase 1.x
    KeyValue result contract

Compatibility

  • Preserve the existing HBase 1.x public APIs.
  • Preserve Get, Batch Get, Scan, and Put result semantics.
  • Preserve TableGroup and single-column-family behavior.
  • Preserve existence-only and empty-result handling.
  • Preserve write ordering between buffered and direct Puts.
  • Keep legacy decoding paths when compact results are unavailable.

Dependency

  • Upgrade obkv-table-client from 2.4.0 to 2.4.1-SNAPSHOT.
  • This PR depends on the corresponding optimization PR in
    obkv-table-client-java.

Dependent PR: oceanbase/obkv-table-client-java#442

Tests

Added 47 focused unit test cases covering:

  • compact point Get and closestRowBefore processing
  • compact Batch Get result conversion and ordering
  • compact Scan result consumption
  • TableGroup family/qualifier range handling
  • synchronous autoFlush direct Put execution
  • preservation of pending buffered-write ordering
  • compact Put encoding with and without TTL
  • zero-copy and sliced cell-array handling
  • delayed-flush cell cloning
  • Put validation and maximum cell-size boundaries
  • lightweight result-cell compatibility
  • malformed and empty result handling

hhlgt added 4 commits August 5, 2026 17:26
Optimize point Get and closestRowBefore, consume compact KQTV batches, avoid TableGroup intermediate copies, and use lightweight result cells for Get and Scan.
Reduce HBase client CPU usage by simplifying synchronous Put execution and integrating compact Put request encoding.
Use a dedicated pure-Get batch result loop and preallocate result containers. Consume compact K/Q/T/V batches directly without rebuilding results from per-field ObObj values. Enable the compact Batch Get decoder by default while retaining an explicit configuration fallback.
@hhlgt

hhlgt commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

AI Coding Review

审查范围:
Base: origin/master
Head: optimize/client-cpu-performance
Commits: 4
Target: HBase 1.x

[P1] BufferedMutatorParams.maxKeyValueSize 被静默忽略

位置:
src/main/java/com/alipay/oceanbase/hbase/util/OHBufferedMutatorImpl.java:173
branch: optimize/client-cpu-performance
OHBufferedMutatorImpl 仍然在构造函数中读取并保存:
this.maxKeyValueSize = params.getMaxKeyValueSize() ...
但 Put 校验被改成:
ohTable.validatePutMutation((Put) mt);
该方法使用的是 OHTable.maxKeyValueSize,而不是当前 BufferedMutatorParams 设置的 maxKeyValueSize。因此 mutator 字段虽然存在,却不再被使用。

影响:

  • 用户为单个 BufferedMutator 设置更小限制时,超限 Cell 可能被接受并发送。
  • 用户设置更大限制时,合法 Cell 可能仍被 OHTable 默认限制拒绝。
  • 破坏 HBase BufferedMutatorParams.maxKeyValueSize 的既有参数语义。
  • 建议增加带显式限制的校验入口:
    ohTable.validatePutMutation((Put) mt, maxKeyValueSize);
    或者保留:
    OHTable.validatePut((Put) mt, maxKeyValueSize);
    再单独执行现有的单 CF/多 CF 校验。

建议增加两个回归用例:
Table 默认值较大、Mutator 参数较小时,超出 Mutator 限制必须失败。
Table 默认值较小、Mutator 参数较大时,应按 Mutator 参数进行判断。

依赖风险

pom.xml 将依赖升级为:
<table.client.version>2.4.1-SNAPSHOT</table.client.version>
该 PR 依赖前一个 Table Client PR。合入前需要确保:
2.4.1-SNAPSHOT 已发布到 CI 可访问的 snapshot 仓库;或者
Table Client 发布正式版本后改为正式版本。

Review 结论

合入前需要修复 BufferedMutatorParams.maxKeyValueSize 参数失效问题,并确认 Table Client snapshot 的发布顺序。

@WeiXinChan

Copy link
Copy Markdown
Contributor

LGTM

@WeiXinChan
WeiXinChan merged commit e70be83 into oceanbase:master Sep 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants