feat: add more read checks

.agents/languages/rust.md

@@ -279,6 +279,11 @@ Load this file when changing `rust/` or Rust xlang behavior.
   compile-time selection hooks whose bodies must disappear after monomorphization.
 - If breakage is explicitly acceptable during a Rust module refactor, rewire macros, tests, and sibling crates directly to the new boundaries instead of adding compatibility re-exports.
 - For panic-safety in hot paths, preserve TLS context reuse. Add scoped guards or owned fallbacks rather than per-call context allocation, and reset reused contexts at entry and successful exit.
+- Read depth and per-root generic/reference state use root reset as their only failure-cleanup
+  owner. Nested readers and skippers increment depth before reading children and decrement only
+  after every child succeeds; an error must retain the failed path's depth and transient state until
+  root reset. Do not use `Drop`, RAII, scope guards, or match-error cleanup to decrement or pop that
+  read-side state on failure. This rule does not change write-side cleanup.
 - Compatible scalar, list-array, and binary/uint8-array adaptations are immediate-field-only. Keep recursive matched-field shape classification owned by `fory-core/src/meta/type_meta.rs`; collection elements, array elements, map keys, and map values must require exact nullability, ref tracking, generic arity, and type shape except documented user-type family normalization.
 - Root deserialization graph memory budget state belongs to `ReadContext` and is initialized by the
   root `Fory` read methods before the header is consumed. Use the fixed `128 MiB` default unless a

.agents/skills/fory-performance-optimization/SKILL.md

@@ -34,6 +34,12 @@ Deliver measurable performance improvements in Apache Fory without protocol drif
 - Never add public hacky API for performance shortcuts; keep optimization helpers internal/private and conceptually clean.
 - Do not hide regressions behind unsafe compiler flags or benchmark-only code paths.
 - Keep optimization surfaces nested-safe; avoid root-only shortcuts unless they are architecturally valid and requested.
+- Do not add reader-side validation solely to produce an earlier or more precise malformed-input
+  error. A necessary crash, panic, undefined-behavior, out-of-bounds, resource-amplification,
+  no-progress, state-pollution, type, or policy guard must keep its hot success path to a primitive
+  branch and move exception allocation and message formatting into a cold no-inline helper when
+  supported. If an existing bounds-safe downstream operation already raises a controlled root
+  error, do not duplicate its validation on the hot path.
 
 ## Execute Workflow
 

AGENTS.md

@@ -29,10 +29,55 @@ This is the entry point for AI guidance in Apache Fory. Read this file first, th
 ## Agent Operating Rules
 
 - Preserve architecture. Do not introduce new layers, parallel flows, or public APIs unless explicitly requested; prefer local repair in the existing owner over shared-infra expansion, and stop if a fix conflicts with an ADR, spec, or invariant.
+- Do not change an existing `RefReader`/`RefWriter` architecture or API to support compatible skip. Compatible skip must not add alternate reference slots or tables, alternate reference lookup or publication methods, or forwarding APIs in read/write contexts, builders, serializers, or generated-code plumbing. Keep ordinary reference publication and lookup unchanged and resolve the case in the existing compatible generated owner. For an authorized removed-field read of an unregistered Struct, the empty object created by the skip reader is that path's final owner: publish that same object for `RefValue`, consume the Struct fields, and let later `RefFlag` values resolve to it. This preserves reference numbering and identity without registering the Struct; an independent dynamic root still requires normal registration. Do not add parallel reference state, a sentinel, a rejection, or a common-path branch for this case.
 - Respect ownership. Keep logic, state, and helpers in their natural owner, and do not move serializer-local, context-local, runtime-type-local, or protocol-local problems into global utilities.
 - Check the spec before implementation. For wire behavior and xlang mapping, use the specs as the source of truth and never copy one runtime's bug into another runtime just to make tests pass.
 - Do not make assumptions about runtime behavior, ownership, registration, metadata construction, protocol semantics, or test coverage. Read the current code, owning docs/specs, and relevant tests before making a design judgment or implementation decision. If the evidence is incomplete, inspect more or state the uncertainty explicitly instead of filling gaps from memory or analogy with another runtime.
 - For untrusted deserialization, read `docs/security/deserialization.md` before changing allocation, stream filling, skip, reference, metadata, or policy validation behavior. Variable-length deserialization must not allocate or reserve backing/output capacity from attacker-declared lengths or counts before the byte owner has proven proportional readable bytes with `checkReadableBytes` or the runtime equivalent. Root graph memory reservation is accounting only and may happen before that byte check, but it must not replace the byte check.
+- Malformed input must surface as a controlled root-operation error and still run
+  root cleanup, but the exact exception type, error code, message, detection
+  layer, and detection point are not contracts unless a public API or
+  specification explicitly says otherwise. An existing bounded downstream
+  buffer, type, reference, depth, or serializer error is sufficient. Do not add
+  hot-path branches, helper APIs, allocations, or generated-code expansion
+  solely to make an error earlier, more specific, or more uniform, and do not
+  write tests that force such error normalization.
+- Never add a reader-side check solely to produce a more precise malformed-input
+  error. Retain or add a check only when the unchecked path has a concrete
+  consequence such as a crash, panic, undefined behavior, out-of-bounds access,
+  disproportionate work, no progress, persistent state pollution, or a real
+  type or policy violation. If a necessary check is reachable from a hot or
+  generated path, keep the success path to a primitive branch and move exception
+  creation, message formatting, and other failure work into a cold no-inline
+  helper when the language supports it. A bounds-safe downstream operation that
+  already raises a controlled root error is sufficient; do not duplicate it for
+  error precision.
+- Before reporting or fixing a robustness finding, prove that the current path
+  causes at least one concrete consequence: crash, panic, undefined behavior,
+  or out-of-bounds access; disproportionate allocation, CPU work, or stream
+  growth; a no-progress loop; persistent state, reference-table, or cache
+  pollution; later-root corruption or a failed-root cleanup leak; or a concrete
+  type, registration, callable, or deserialization-policy violation. Protocol
+  strictness alone is out of scope. Do not change code merely because a
+  malformed or noncanonical flag, enum value, marker, length form, or reserved
+  value is accepted, rejected late, decoded differently, or produces a less
+  precise error.
+- Arbitrary-precision binary Decimal codecs accept only scales in
+  `[-10_000, 10_000]` and an absolute unscaled magnitude of at most `10_000`
+  binary bytes. The Java standalone `BigInteger` serializer uses the same
+  magnitude limit. This is a value-range rule, not a wire-format change:
+  `magnitude` means the canonical unsigned bytes of the absolute value, not a
+  signed two's-complement prefix, protocol headers, decimal digits, or the Fory
+  JSON `10_000`-character limit. Readers must validate the range before
+  allocating magnitude storage, constructing arbitrary-precision values, or
+  expanding scale, while retaining the existing readable-byte, negative-length,
+  overflow, and canonical checks. Writers must validate symmetrically before
+  emitting any part of the value and before materializing an oversized
+  magnitude. Compare scale directly with both bounds; do not use `abs(scale)`,
+  which can overflow for the minimum integer. Fixed-range Decimal carriers keep
+  their stricter native ranges and reject oversized magnitudes before copying or
+  construction. Compatible scalar conversion keeps its independent `256`-digit
+  and scale/output-expansion limits.
 - Root deserialization graph memory budgets are approximate gates for materialized graph owners,
   not exact heap accounting, input byte accounting, or raw element counts. `maxGraphMemoryBytes`
   defaults to fixed `128 MiB`; positive values override the default; explicit non-positive values
@@ -73,6 +118,7 @@ This is the entry point for AI guidance in Apache Fory. Read this file first, th
 - When a user corrects a non-obvious invariant, encode it in the nearest source comment before continuing, and also update `AGENTS.md`, `.agents/**`, docs, or specs when the rule is reusable beyond one file. Do not rely only on chat history, task notes, commit messages, or benchmark logs for corrections that protect security, protocol behavior, ownership, naming, or hot-path performance.
 - Reject semantic hacks. Do not bypass broken semantics by deleting cases, simplifying callers, adding coercion hooks, or using workaround fallbacks; fix the underlying bug and prove it with focused tests.
 - Protect hot paths. Avoid per-call allocations, callback objects, result tuples or records, unnecessary runtime branches, and wrapper-class substitutions in hot codec/runtime paths; prefer conditional imports and allocation-free concrete implementations where they fit the language.
+- Decoder depth and the generic-type stack paired with that depth use root-operation failure cleanup. Nested decoders decrement depth and pop generic types only after successful child reads; do not add nested `try/finally` to restore them after exceptions. The root operation's `finally`/reset must clear both decoder depth and the generic-type stack.
 - Keep public APIs minimal. Public APIs must match user ownership and mental model, not internal implementation details; generated flows stay type-owned, while custom serializer registration stays explicit.
 - Use semantic naming only. Name things after protocol or domain concepts, not history, runtime origin, or workaround style; avoid vague names such as `Internal`, `java_style_*`, `Runtime`, `Session`, `Plan`, `Payload`, or `Binding` when they do not name the real concept. Keep class, method, function, and variable names concise; do not encode the whole scenario or implementation history into one identifier. Never name a class or method with a `Plan` suffix; use the real domain concept instead. For Fory codec/read APIs, do not use generic `payload` naming; name the exact owner and data shape, such as bytes, body, frame, field, string, list, map, compressed bytes, or primitive-array encoding.
 - Keep one implementation path. Do not keep parallel helpers, serializers, harnesses, wrappers, or registration flows for the same concept; extend the existing owner path instead of inventing another one.

cpp/fory/meta/BUILD

@@ -6,6 +6,7 @@ cc_library(
     hdrs = glob(["*.h"]),
     strip_include_prefix = "/cpp",
     deps = [
+        "//cpp/fory/thirdparty:libmmh3",
         "//cpp/fory/type:fory_type",
         "//cpp/fory/util:fory_util",
     ],

cpp/fory/meta/CMakeLists.txt

@@ -52,6 +52,7 @@ target_include_directories(fory_meta
 
 target_link_libraries(fory_meta
     PUBLIC
+        fory_thirdparty
         fory_util
 )
 

cpp/fory/meta/meta_string.cc

@@ -19,10 +19,12 @@
 
 #include "fory/meta/meta_string.h"
 
+#include "fory/thirdparty/MurmurHash3.h"
 #include "fory/util/buffer.h"
 
 #include <algorithm>
 #include <cctype>
+#include <cstring>
 
 namespace fory {
 namespace meta {
@@ -236,6 +238,30 @@ MetaStringDecoder::decode_lower_upper_digit_special_char(uint8_t value) const {
 
 MetaStringTable::MetaStringTable() = default;
 
+int64_t compute_meta_string_hash(const std::vector<uint8_t> &bytes,
+                                 MetaEncoding encoding) {
+  static constexpr uint8_t k_empty_input = 0;
+  const uint8_t *data = bytes.empty() ? &k_empty_input : bytes.data();
+  uint64_t hash_out[2] = {0, 0};
+  MurmurHash3_x64_128(data, static_cast<int>(bytes.size()), 47, hash_out);
+
+  uint64_t hash = hash_out[0];
+  if ((hash & (uint64_t{1} << 63)) != 0) {
+    // Unsigned negation matches Java Math.abs(long) bit-for-bit, including
+    // Long.MIN_VALUE wrapping to itself without signed overflow.
+    hash = uint64_t{0} - hash;
+  }
+  if (hash == 0) {
+    hash += 256;
+  }
+  hash &= UINT64_C(0xffffffffffffff00);
+  hash |= static_cast<uint8_t>(encoding);
+
+  int64_t signed_hash;
+  std::memcpy(&signed_hash, &hash, sizeof(signed_hash));
+  return signed_hash;
+}
+
 Result<std::string, Error>
 MetaStringTable::read_string(Buffer &buffer, const MetaStringDecoder &decoder) {
   Error error;
@@ -265,14 +291,13 @@ MetaStringTable::read_string(Buffer &buffer, const MetaStringDecoder &decoder) {
   if (len > k_small_threshold) {
     // Big string layout in Java MetaStringResolver:
     //   header (len<<1 | flags) + hash_code(int64) + data[len]
-    // The original encoding is not transmitted explicitly. For cross-language
-    // purposes we treat the payload bytes as UTF8 and let callers handle any
-    // higher-level semantics.
     int64_t hash_code = buffer.read_int64(error);
     if (FORY_PREDICT_FALSE(!error.ok())) {
       return Unexpected(std::move(error));
     }
-    (void)hash_code; // hash_code is only used for Java-side caching.
+    FORY_TRY(encoded, to_meta_encoding(static_cast<uint8_t>(
+                          static_cast<uint64_t>(hash_code))));
+    encoding = encoded;
     if (len > 0) {
       if (FORY_PREDICT_FALSE(!buffer.ensure_readable(len, error))) {
         return Unexpected(std::move(error));
@@ -283,7 +308,10 @@ MetaStringTable::read_string(Buffer &buffer, const MetaStringDecoder &decoder) {
         return Unexpected(std::move(error));
       }
     }
-    encoding = MetaEncoding::UTF8;
+    if (FORY_PREDICT_FALSE(compute_meta_string_hash(bytes, encoding) !=
+                           hash_code)) {
+      return Unexpected(Error::invalid_data("Malformed meta string hash"));
+    }
   } else {
     // Small string layout: data[len] with an encoding byte when len > 0.
     // Java omits the encoding byte for empty strings.

cpp/fory/meta/meta_string.h

@@ -106,6 +106,12 @@ struct EncodedMetaString {
   std::vector<uint8_t> bytes;
 };
 
+// Compute the canonical wire hash for a meta string. Large meta strings encode
+// MetaEncoding in the low byte of this hash instead of writing a separate
+// encoding byte.
+int64_t compute_meta_string_hash(const std::vector<uint8_t> &bytes,
+                                 MetaEncoding encoding);
+
 // Encoder for meta strings used by xlang type metadata.
 // This mirrors the behavior of Java's MetaStringEncoder.
 class MetaStringEncoder {

cpp/fory/meta/meta_string_test.cc

@@ -414,6 +414,89 @@ TEST_F(MetaStringTest, MetaStringTableEmptyString) {
   EXPECT_EQ(result.value(), "");
 }
 
+TEST_F(MetaStringTest, MetaStringTableReadLargeNames) {
+  MetaStringEncoder namespace_encoder{'.', '_'};
+  MetaStringDecoder namespace_decoder{'.', '_'};
+  MetaStringEncoder type_name_encoder{'$', '_'};
+  MetaStringDecoder type_name_decoder{'$', '_'};
+  const std::vector<MetaEncoding> namespace_encodings = {
+      MetaEncoding::UTF8, MetaEncoding::ALL_TO_LOWER_SPECIAL,
+      MetaEncoding::LOWER_UPPER_DIGIT_SPECIAL};
+  const std::vector<MetaEncoding> type_name_encodings = {
+      MetaEncoding::UTF8, MetaEncoding::ALL_TO_LOWER_SPECIAL,
+      MetaEncoding::LOWER_UPPER_DIGIT_SPECIAL,
+      MetaEncoding::FIRST_TO_LOWER_SPECIAL};
+  const std::string namespace_name =
+      "org.apache.fory.serialization.longnamespace";
+  const std::string type_name = "RecursiveCollectionNode";
+
+  auto encoded_namespace =
+      namespace_encoder.encode(namespace_name, namespace_encodings);
+  auto encoded_type_name =
+      type_name_encoder.encode(type_name, type_name_encodings);
+  ASSERT_TRUE(encoded_namespace.ok());
+  ASSERT_TRUE(encoded_type_name.ok());
+  ASSERT_GT(encoded_namespace.value().bytes.size(), 16);
+  ASSERT_GT(encoded_type_name.value().bytes.size(), 16);
+
+  Buffer buffer;
+  auto write_large = [&buffer](const EncodedMetaString &encoded) {
+    buffer.write_var_uint32(static_cast<uint32_t>(encoded.bytes.size()) << 1);
+    buffer.write_int64(
+        compute_meta_string_hash(encoded.bytes, encoded.encoding));
+    buffer.write_bytes(encoded.bytes.data(), encoded.bytes.size());
+  };
+  write_large(encoded_namespace.value());
+  write_large(encoded_type_name.value());
+
+  MetaStringTable table;
+  buffer.reader_index(0);
+  auto decoded_namespace = table.read_string(buffer, namespace_decoder);
+  auto decoded_type_name = table.read_string(buffer, type_name_decoder);
+  ASSERT_TRUE(decoded_namespace.ok());
+  ASSERT_TRUE(decoded_type_name.ok());
+  EXPECT_EQ(decoded_namespace.value(), namespace_name);
+  EXPECT_EQ(decoded_type_name.value(), type_name);
+  EXPECT_EQ(buffer.reader_index(), buffer.writer_index());
+  EXPECT_EQ(compute_meta_string_hash(encoded_type_name.value().bytes,
+                                     encoded_type_name.value().encoding),
+            INT64_C(0x1f8637e8459afd04));
+}
+
+TEST_F(MetaStringTest, MetaStringTableRejectsLargeHash) {
+  MetaStringEncoder type_name_encoder{'$', '_'};
+  MetaStringDecoder type_name_decoder{'$', '_'};
+  const std::string type_name = "RecursiveCollectionNode";
+  auto encoded = type_name_encoder.encode(
+      type_name, {MetaEncoding::UTF8, MetaEncoding::ALL_TO_LOWER_SPECIAL,
+                  MetaEncoding::LOWER_UPPER_DIGIT_SPECIAL,
+                  MetaEncoding::FIRST_TO_LOWER_SPECIAL});
+  ASSERT_TRUE(encoded.ok());
+  ASSERT_GT(encoded.value().bytes.size(), 16);
+
+  const int64_t canonical =
+      compute_meta_string_hash(encoded.value().bytes, encoded.value().encoding);
+  Buffer malformed;
+  malformed.write_var_uint32(static_cast<uint32_t>(encoded.value().bytes.size())
+                             << 1);
+  malformed.write_int64(canonical ^ INT64_C(0x100));
+  malformed.write_bytes(encoded.value().bytes.data(),
+                        encoded.value().bytes.size());
+
+  MetaStringTable table;
+  malformed.reader_index(0);
+  auto result = table.read_string(malformed, type_name_decoder);
+  ASSERT_FALSE(result.ok());
+  EXPECT_EQ(result.error().code(), ErrorCode::InvalidData);
+
+  Buffer reference;
+  reference.write_var_uint32((1u << 1) | 1u);
+  reference.reader_index(0);
+  auto unpublished = table.read_string(reference, type_name_decoder);
+  EXPECT_FALSE(unpublished.ok());
+  EXPECT_EQ(unpublished.error().code(), ErrorCode::InvalidData);
+}
+
 // ============================================================================
 // Special character encoding tests
 // ============================================================================

cpp/fory/serialization/any_serializer.h

@@ -122,7 +122,7 @@ template <> struct Serializer<std::any> {
       return std::any();
     }
 
-    return type_info->harness.any_read_fn(ctx);
+    return read_value(ctx, *type_info);
   }
 
   static inline std::any read_data(ReadContext &ctx) {
@@ -142,7 +142,26 @@ template <> struct Serializer<std::any> {
       return std::any();
     }
 
-    return type_info.harness.any_read_fn(ctx);
+    return read_value(ctx, type_info);
+  }
+
+private:
+  static inline std::any read_value(ReadContext &ctx,
+                                    const TypeInfo &type_info) {
+    // std::any is a dynamic materialization boundary: its wire type can select
+    // another registered std::any-bearing object recursively. Keep concrete
+    // smart-pointer layouts out of this dynamic-depth policy.
+    auto depth_result = ctx.increase_dyn_depth();
+    if (FORY_PREDICT_FALSE(!depth_result.ok())) {
+      ctx.set_error(std::move(depth_result).error());
+      return std::any();
+    }
+    std::any value = type_info.harness.any_read_fn(ctx);
+    if (FORY_PREDICT_FALSE(ctx.has_error())) {
+      return value;
+    }
+    ctx.decrease_dyn_depth();
+    return value;
   }
 };