From 6415107ca7030f958de4b043503434b8bbdef61b Mon Sep 17 00:00:00 2001 From: andreatp Date: Wed, 4 Nov 2026 17:19:13 +0000 Subject: [PATCH 1/6] docs: fix stale and incorrect statements SIMD is interpreter-only: the bytecode compilers fail on v128, so the Quick Start requirement and the SIMD page overstated support. The compiler pages claimed parity with the interpreter's spec tests, which no longer holds. Also remove a dangling example reference in Linking, a license link pointing at a page without licenses, a copy-pasted stdout redirect, and release-time phrasing elsewhere. Add writing rules to the docs README. --- docs/README.md | 9 +++++++++ docs/docs/advanced/memory-customization.md | 4 ---- docs/docs/advanced/simd.md | 14 +++----------- docs/docs/advanced/tools.md | 3 +-- docs/docs/core/linking.md | 5 ++--- docs/docs/execution/build-time-compiler.md | 9 +++------ docs/docs/execution/runtime-compiler.md | 13 ++++--------- docs/docs/index.md | 2 +- docs/docs/security/best-practices.md | 2 +- docs/docs/wasi/index.md | 4 +--- 10 files changed, 25 insertions(+), 40 deletions(-) diff --git a/docs/README.md b/docs/README.md index 9d6cc86a6..1bba3401b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -24,3 +24,12 @@ $ npm run build This command generates static content into the `build` directory and can be served using any static contents hosting service. + +### Writing docs + +- Describe how to use a feature, not how it is implemented. Internals belong in Javadoc or in the pull request description. +- No release-time or migration statements. Those go in a blog post; breaking changes relative to Chicory go in the migration guide. +- One code sample per option. Show a builder once per page. +- Body headings start at `##`; the frontmatter `title` is the page heading. Open every page with one plain sentence saying what it covers, which becomes its description in `llms.txt`. +- Names: Wasm, interpreter, runtime compiler, build-time compiler. Version placeholder: `${endive.version}`. +- Every page is executed by `npm test`: keep the hidden `//DEPS` and `docs.FileOps.writeResult(...)` blocks, and update `tests/approvals/` when a page moves. diff --git a/docs/docs/advanced/memory-customization.md b/docs/docs/advanced/memory-customization.md index 3f6db6850..29b6c4813 100644 --- a/docs/docs/advanced/memory-customization.md +++ b/docs/docs/advanced/memory-customization.md @@ -10,10 +10,6 @@ title: Advanced Wasm Memory Customization docs.FileOps.copyFromWasmCorpus("count_vowels.rs.wasm", "count_vowels.rs.wasm"); -System.setOut(new PrintStream( - new BufferedOutputStream( - new FileOutputStream("docs/examples/rust.md.result")))); - import run.endive.wasm.Parser; import run.endive.runtime.Instance; import run.endive.runtime.ByteArrayMemory; diff --git a/docs/docs/advanced/simd.md b/docs/docs/advanced/simd.md index c3387e4f3..d401519d9 100644 --- a/docs/docs/advanced/simd.md +++ b/docs/docs/advanced/simd.md @@ -4,22 +4,14 @@ sidebar_label: SIMD title: SIMD support --- -SIMD is supported out of the box by the interpreter. +SIMD (`v128`) instructions are supported by the interpreter on every supported JDK, with no extra dependency or configuration. -All WebAssembly `v128` instructions use the scalar interpreter implementation on every supported -JDK. No extra dependency, JVM flag, or machine-factory configuration is required. +The runtime and build-time compilers do not support SIMD: compiling a module that uses `v128` instructions fails, so such modules must run in the interpreter. :::warning -SIMD support **REQUIRES** validation. Disabling validation (`WasmModule.builder().withValidation(false)`) is likely to produce incorrect results. +SIMD execution relies on validation. Keep it enabled (the default) for modules that use `v128`. ::: -### Migration - -The `run.endive:simd` module is gone. If your application declares it and uses -`SimdInterpreterMachine`, remove that dependency and the explicit -`withMachineFactory(SimdInterpreterMachine::new)` call; the default machine from -`run.endive:runtime` now executes `v128` instructions. - diff --git a/docs/docs/annotations/_category_.yml b/docs/docs/annotations/_category_.yml deleted file mode 100644 index 3f46b5dc7..000000000 --- a/docs/docs/annotations/_category_.yml +++ /dev/null @@ -1,2 +0,0 @@ -label: Annotations -position: 35 diff --git a/docs/docs/annotations/index.md b/docs/docs/core/annotations.md similarity index 97% rename from docs/docs/annotations/index.md rename to docs/docs/core/annotations.md index 5bdb4c079..58588abfb 100644 --- a/docs/docs/annotations/index.md +++ b/docs/docs/core/annotations.md @@ -1,5 +1,5 @@ --- -sidebar_position: 100 +sidebar_position: 4 sidebar_label: Annotations title: Annotations --- @@ -143,6 +143,6 @@ Exactly how this is done depends on the build system you are using, for instance diff --git a/docs/docs/core/execution-modes.md b/docs/docs/core/execution-modes.md deleted file mode 100644 index 958efa352..000000000 --- a/docs/docs/core/execution-modes.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -sidebar_position: 4 -sidebar_label: Execution modes -title: Execution modes ---- - -## Overview - -| Mode | Performance | Dynamic Module Loading | Requirements | Output Format | Ideal Use Case | -|---|---|---|---|---|---| -| **Interpreter** | 🐒 Slow | βœ… Supported | None | None - fully interpreted | Default mode; highly portable; suitable for development and environments requiring dynamic loading. | -| **Runtime Compilation** | πŸ‡ Fast | βœ… Supported | Requires reflection and ASM dependency | In-memory Java Bytecode | Enhanced performance; suitable when dynamic loading is needed and the usage of reflection is fine. | -| **Build time Compilation** | πŸ‡ Fast | ❌ Not Supported | Build-time tools (e.g., Maven or Gradle plugins) | Plain Java Bytecode | Optimal performance; no dynamic loading; ideal for production with static modules. | - -## Summary - -- **Interpreter**: Executes WebAssembly (Wasm) modules directly without prior compilation. It's the default mode in Endive, offering maximum portability and simplicity. However, it has slower execution speed compared to compiled modes. - -- **Runtime Compilation**: Compiles Wasm modules to Java bytecode at runtime for fast execution. This mode requires one additional dependency on [ASM](https://asm.ow2.io/), it uses reflection, and it loads bytecode dynamically. It fully supports loading new Wasm modules on-the-fly, but it might not be supported on some platforms (such as Android, or GraalVM's native-image). - -- **Build time Compilation**: Compiles Wasm modules to Java bytecode during the build process using tools like Maven or Gradle plugins. This mode offers the best performance and eliminates the need for dynamic loading and additional dependencies, making it ideal for production environments with static modules. - - diff --git a/docs/docs/core/host-functions.md b/docs/docs/core/host-functions.md index 3e9d41227..bcfd664a6 100644 --- a/docs/docs/core/host-functions.md +++ b/docs/docs/core/host-functions.md @@ -99,5 +99,5 @@ logIt.apply(); ``` :::tip -For an easier way to write host function and interact with a Wasm module, see [Annotations](../annotations/index.md). +For an easier way to write host function and interact with a Wasm module, see [Annotations](annotations.md). ::: diff --git a/docs/docs/core/memory.md b/docs/docs/core/memory.md index 7a163a347..097a13783 100644 --- a/docs/docs/core/memory.md +++ b/docs/docs/core/memory.md @@ -24,6 +24,10 @@ docs.FileOps.copyFromWasmCorpus("count_vowels.rs.wasm", "count_vowels.wasm"); ``` --> +:::tip +A module like this one is a Rust library built with `rustc --target=wasm32-unknown-unknown --crate-type=cdylib`. Use `--target=wasm32-wasip1` for programs that need [WASI](../wasi/index.md). +::: + Build and instantiate this module: ```java diff --git a/docs/docs/examples/_category_.yml b/docs/docs/examples/_category_.yml deleted file mode 100644 index e2c291542..000000000 --- a/docs/docs/examples/_category_.yml +++ /dev/null @@ -1,2 +0,0 @@ -label: Examples -position: 70 diff --git a/docs/docs/examples/rust.md b/docs/docs/examples/rust.md deleted file mode 100644 index 08ecfc296..000000000 --- a/docs/docs/examples/rust.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -sidebar_position: 1 -sidebar_label: Rust -title: Using Rust with Endive ---- -## Compile Rust to Wasm - -Compiling a Rust library to Wasm is easy and can be performed using standard `rustc` options: - -```bash -rustc --target=wasm32-unknown-unknown --crate-type=cdylib -``` - -when you need to add support for WASI preview 1 (typically when using CLIs) you can use: - -```bash -rustc --target=wasm32-wasi --crate-type=bin -``` - -:::tip -For production usage, make sure to produce an optimized Wasm module by using the standard compiler options `-C opt-level = 3` (speed) or `-C opt-level = "z"` (size). -::: - -## Using in Endive - - - -```java -import run.endive.wasm.Parser; -import run.endive.runtime.Instance; - -var instance = Instance.builder(Parser.parse(new File("count_vowels.rs.wasm"))).build(); - -var alloc = instance.export("alloc"); -var dealloc = instance.export("dealloc"); -var countVowels = instance.export("count_vowels"); - -var memory = instance.memory(); -var message = "Hello, World!"; -byte[] bytes = message.getBytes(); -int len = bytes.length; -int ptr = (int) alloc.apply(len)[0]; - -memory.write(ptr, bytes); - -var result = countVowels.apply(ptr, len)[0]; -System.out.println(result); -``` diff --git a/docs/docs/execution/build-time-compiler.md b/docs/docs/execution/build-time-compiler.md index 27b892dd6..9a42a4f9d 100644 --- a/docs/docs/execution/build-time-compiler.md +++ b/docs/docs/execution/build-time-compiler.md @@ -111,7 +111,7 @@ mvn help:describe -DgroupId=run.endive -DartifactId=endive-compiler-maven-plugin ### Generating module exports and imports The build-time compiler can also generate typed Java wrappers for a module's exports and imports, -eliminating the need for the [`@WasmModuleInterface` annotation](../annotations/index.md#wasmmoduleinterface) and the annotation processor setup. +eliminating the need for the [`@WasmModuleInterface` annotation](../core/annotations.md#wasmmoduleinterface) and the annotation processor setup. Add the `moduleInterface` parameter to the plugin configuration: diff --git a/docs/docs/execution/index.md b/docs/docs/execution/index.md new file mode 100644 index 000000000..9d051fbea --- /dev/null +++ b/docs/docs/execution/index.md @@ -0,0 +1,27 @@ +--- +sidebar_position: 0 +sidebar_label: Overview +title: Compilation +--- + +Endive can run a Wasm module in three ways: interpreted, compiled to JVM bytecode when the instance is created, or compiled to JVM bytecode at build time. + +| Mode | Performance | Dynamic module loading | Requirements | Use when | +|---|---|---|---|---| +| **Interpreter** | 🐒 Slow | βœ… | None | The default. Maximum portability, and the only mode that runs SIMD (`v128`) modules. | +| **[Runtime compiler](runtime-compiler.md)** | πŸ‡ Fast | βœ… | ASM dependency and reflection | Modules are loaded dynamically and reflection is acceptable. | +| **[Build-time compiler](build-time-compiler.md)** | πŸ‡ Fast | ❌ | Maven or Gradle plugin | Modules are known at build time. Best startup, no extra runtime dependency, works with `native-image`. | + +The interpreter needs nothing beyond `run.endive:runtime`. +Both compilers translate each Wasm function to a JVM method and can leave functions too large to compile to the interpreter. +Neither supports SIMD yet, and SIMD execution relies on validation, so keep it enabled for modules that use `v128`. + +[Redline](../experimental/redline.md) is an experimental fourth option that compiles the module to native code at build time. + + diff --git a/docs/docs/security/best-practices.md b/docs/docs/security/best-practices.md index cb2340988..d80ceb2fe 100644 --- a/docs/docs/security/best-practices.md +++ b/docs/docs/security/best-practices.md @@ -61,20 +61,7 @@ var options = WasiOptions.builder() Wasm modules can consume unbounded CPU and memory. Always set limits when running untrusted code. -**CPU timeout via thread interruption:** - -```java title="Example" -var executor = Executors.newSingleThreadExecutor(); -var future = executor.submit(() -> instance.export("run").apply()); - -try { - var result = future.get(5, TimeUnit.SECONDS); -} catch (TimeoutException e) { - future.cancel(true); // interrupts the Wasm execution -} -``` - -See [CPU Limits](/docs/advanced/cpu-limits) for more patterns. +Run untrusted code under a timeout and interrupt it when the deadline passes; see [CPU Limits](/docs/security/cpu-limits) for the patterns. ## Compiler Security diff --git a/docs/docs/advanced/cpu-limits.md b/docs/docs/security/cpu-limits.md similarity index 77% rename from docs/docs/advanced/cpu-limits.md rename to docs/docs/security/cpu-limits.md index 04e60c384..38eac5dad 100644 --- a/docs/docs/advanced/cpu-limits.md +++ b/docs/docs/security/cpu-limits.md @@ -1,17 +1,10 @@ --- -sidebar_position: 1 -sidebar_label: CPU -title: CPU +sidebar_position: 3 +sidebar_label: CPU Limits +title: CPU Limits --- -# Limiting CPU usage -:::warning[Security Consideration] -Wasm modules can contain infinite loops. When running untrusted code, always set execution timeouts via thread interruption or an ExecutorService with a deadline. Without timeouts, a malicious module can consume 100% CPU indefinitely. -::: - -Often, when running untrusted user code in our infrastructure, we want to have strong guarantees around the termination of the program. - -To achieve this result there are, currently, two mechanisms in Endive: +Wasm modules can loop forever, so untrusted code needs a timeout. Endive offers two mechanisms to bound execution. ## Interrupts @@ -86,6 +79,6 @@ var instance = diff --git a/docs/docs/security/overview.md b/docs/docs/security/overview.md index ed35bb4c3..fe2365a9b 100644 --- a/docs/docs/security/overview.md +++ b/docs/docs/security/overview.md @@ -44,7 +44,7 @@ The critical trust boundary is between **Wasm guest code** and **host functions* ## What the Sandbox Does NOT Guarantee -- **CPU limits**: Wasm modules can execute infinite loops. The host must enforce timeouts (see [CPU Limits](/docs/advanced/cpu-limits)). +- **CPU limits**: Wasm modules can execute infinite loops. The host must enforce timeouts (see [CPU Limits](/docs/security/cpu-limits)). - **Memory growth limits**: Modules can request memory growth up to the declared maximum. The host should set appropriate limits. - **Post-compilation verification**: The build-time and runtime compilers translate Wasm to JVM bytecode without a separate verification pass. For maximum assurance with untrusted code, prefer the interpreter. - **Cache integrity**: The [directory-based compiler cache](/docs/execution/compiler-cache/#the-directory-cache) does not verify bytecode integrity on load. Protect cache directories with restrictive permissions. diff --git a/docs/docusaurus.config.ts b/docs/docusaurus.config.ts index 17f7cfe3f..c0871e352 100644 --- a/docs/docusaurus.config.ts +++ b/docs/docusaurus.config.ts @@ -39,8 +39,8 @@ const config: Config = { { redirects: [ { - to: '/docs/annotations', - from: ['/docs/experimental/host-modules', '/docs/usage/annotations'], + to: '/docs/core/annotations', + from: ['/docs/experimental/host-modules', '/docs/usage/annotations', '/docs/annotations'], }, { to: '/docs/execution/runtime-compiler', @@ -56,27 +56,23 @@ const config: Config = { }, { to: '/docs/core/memory', - from: ['/docs/usage/memory'], + from: ['/docs/usage/memory', '/docs/examples/rust'], }, { to: '/docs/core/linking', from: ['/docs/usage/linking'], }, { - to: '/docs/core/execution-modes', - from: ['/docs/usage/execution_modes'], + to: '/docs/execution', + from: ['/docs/usage/execution_modes', '/docs/core/execution-modes', '/docs/usage/simd', '/docs/advanced/simd'], }, { to: '/docs/wasi', from: ['/docs/usage/wasi'], }, { - to: '/docs/advanced/cpu-limits', - from: ['/docs/usage/cpu'], - }, - { - to: '/docs/advanced/simd', - from: ['/docs/usage/simd'], + to: '/docs/security/cpu-limits', + from: ['/docs/usage/cpu', '/docs/advanced/cpu-limits'], }, { to: '/docs/advanced/tools', diff --git a/docs/tests/approvals/docs-advanced-cpu-limits.md.approved.txt b/docs/tests/approvals/docs-core-annotations.md.approved.txt similarity index 100% rename from docs/tests/approvals/docs-advanced-cpu-limits.md.approved.txt rename to docs/tests/approvals/docs-core-annotations.md.approved.txt diff --git a/docs/tests/approvals/docs-core-execution-modes.md.approved.txt b/docs/tests/approvals/docs-core-execution-modes.md.approved.txt deleted file mode 100644 index c6cac6926..000000000 --- a/docs/tests/approvals/docs-core-execution-modes.md.approved.txt +++ /dev/null @@ -1 +0,0 @@ -empty diff --git a/docs/tests/approvals/docs-examples-rust.md.approved.txt b/docs/tests/approvals/docs-examples-rust.md.approved.txt deleted file mode 100644 index 00750edc0..000000000 --- a/docs/tests/approvals/docs-examples-rust.md.approved.txt +++ /dev/null @@ -1 +0,0 @@ -3 diff --git a/docs/tests/approvals/docs-advanced-simd.md.approved.txt b/docs/tests/approvals/docs-execution-index.md.approved.txt similarity index 100% rename from docs/tests/approvals/docs-advanced-simd.md.approved.txt rename to docs/tests/approvals/docs-execution-index.md.approved.txt diff --git a/docs/tests/approvals/docs-annotations-index.md.approved.txt b/docs/tests/approvals/docs-security-cpu-limits.md.approved.txt similarity index 100% rename from docs/tests/approvals/docs-annotations-index.md.approved.txt rename to docs/tests/approvals/docs-security-cpu-limits.md.approved.txt From 8a415cbc738eca1533837a55a6745ef237b39b2c Mon Sep 17 00:00:00 2001 From: andreatp Date: Wed, 4 Nov 2026 17:28:49 +0000 Subject: [PATCH 4/6] docs: consistency pass One heading convention (frontmatter title, body sections from h2), one version placeholder, one set of names for the execution modes, shorter security admonitions that defer to the Security section, and a descriptive first sentence on every page for llms.txt. The Quick Start keeps one dependency snippet and defers to Installation. The Annotations type table now lists the string, Instance and Memory parameters the processor accepts. --- docs/blog/2025-02-20-chicory-1.1.0.md | 2 +- docs/docs/advanced/logging.md | 2 - docs/docs/advanced/memory-customization.md | 11 ++--- docs/docs/advanced/tools.md | 17 ++++---- docs/docs/ai-friendly-docs.md | 2 - docs/docs/core/annotations.md | 25 ++++++------ docs/docs/core/linking.md | 11 +---- docs/docs/core/memory.md | 7 +--- docs/docs/execution/compiler-cache.md | 4 +- docs/docs/experimental/cli.md | 6 +-- docs/docs/experimental/redline.md | 18 ++++----- docs/docs/experimental/why.md | 2 - docs/docs/getting-started/installation.md | 2 - docs/docs/index.md | 47 ++++++++-------------- docs/docs/migration/from-chicory.md | 2 - docs/docs/security/best-practices.md | 2 - docs/docs/security/overview.md | 2 - docs/docs/wasi/index.md | 26 ++++++------ 18 files changed, 70 insertions(+), 118 deletions(-) diff --git a/docs/blog/2025-02-20-chicory-1.1.0.md b/docs/blog/2025-02-20-chicory-1.1.0.md index b06301415..59f6026af 100644 --- a/docs/blog/2025-02-20-chicory-1.1.0.md +++ b/docs/blog/2025-02-20-chicory-1.1.0.md @@ -33,5 +33,5 @@ With this release, we’re introducing a **β€œWho Uses Chicory?”** section in Try out **Chicory 1.1.0** today, and let us know what you think! -[simd]: /docs/advanced/simd +[simd]: /docs/execution [memory]: /docs/advanced/memory-customization diff --git a/docs/docs/advanced/logging.md b/docs/docs/advanced/logging.md index 50f10caf1..47913743e 100644 --- a/docs/docs/advanced/logging.md +++ b/docs/docs/advanced/logging.md @@ -3,8 +3,6 @@ sidebar_position: 4 sidebar_label: Logging title: Logging --- -# Logging - For maximum compatibility and to avoid external dependencies we use, by default, the JDK Platform Logging ([JEP 264](https://openjdk.org/jeps/264)). The Platform Logging falls back to the [`java.util.logging` API](https://docs.oracle.com/en/java/javase/21/core/java-logging-overview.html) by default. diff --git a/docs/docs/advanced/memory-customization.md b/docs/docs/advanced/memory-customization.md index 29b6c4813..06d68d6df 100644 --- a/docs/docs/advanced/memory-customization.md +++ b/docs/docs/advanced/memory-customization.md @@ -1,7 +1,7 @@ --- sidebar_position: 5 -sidebar_label: Memory -title: Advanced Wasm Memory Customization +sidebar_label: Memory Customization +title: Memory Customization --- -Different Wasm workloads will be using the memory in very different ways. - -Make sure you don't do changes without proper benchmarking and analysis. - -It's possible to provide a custom implementation of the entire Memory used by the Wasm module: +You can replace the `Memory` implementation backing a module through `withMemoryFactory`. +Workloads use memory in very different ways, so benchmark before changing it: ```java var instance = Instance.builder(module).withMemoryFactory(limits -> { diff --git a/docs/docs/advanced/tools.md b/docs/docs/advanced/tools.md index 662f50fc5..351c26ccb 100644 --- a/docs/docs/advanced/tools.md +++ b/docs/docs/advanced/tools.md @@ -1,34 +1,31 @@ --- sidebar_position: 3 -sidebar_label: WABT -title: WABT +sidebar_label: Tools +title: Tools --- -# Tools +Endive publishes a few WebAssembly tools as Maven artifacts, compiled to Wasm and run with Endive itself. ## WebAssembly Binary Toolkit -Since we use them in the build of the project, we publish a few [wabt](https://github.com/WebAssembly/wabt) tools compiled at build time with Endive. -Adding support for more tools is easy and we welcome contributions in this direction. -The relevant module can be added to the build with: +A few [wabt](https://github.com/WebAssembly/wabt) tools, the ones Endive's own build uses: ```xml run.endive wabt - latest-release + ${endive.version} ``` ## Wasm Tools -As we need it to catch up with the latest development of the Wasm spec we publish a few [wasm-tools](https://github.com/bytecodealliance/wasm-tools) compiled at build time with Endive. -The relevant module can be added to the build with: +A few [wasm-tools](https://github.com/bytecodealliance/wasm-tools) binaries, which track the latest Wasm proposals: ```xml run.endive wasm-tools - latest-release + ${endive.version} ``` diff --git a/docs/docs/ai-friendly-docs.md b/docs/docs/ai-friendly-docs.md index 358ebec33..4390f2426 100644 --- a/docs/docs/ai-friendly-docs.md +++ b/docs/docs/ai-friendly-docs.md @@ -4,8 +4,6 @@ sidebar_label: AI-Friendly Docs title: AI-Friendly Documentation --- -# AI-Friendly Documentation - Endive documentation is designed to be easily consumed by LLMs, AI agents, and developer tools. All machine-readable files are auto-generated at build time from the same source as the human-facing site, using the [llms.txt specification](https://llmstxt.org/). ## Available Endpoints diff --git a/docs/docs/core/annotations.md b/docs/docs/core/annotations.md index 58588abfb..c0a8e279e 100644 --- a/docs/docs/core/annotations.md +++ b/docs/docs/core/annotations.md @@ -5,9 +5,8 @@ title: Annotations --- ## Host Modules -Instead of writing host functions by hand, you can write a class containing annotated methods -and let the Endive annotation processor generate the host functions for you. This is especially -useful when you have many host functions. +Instead of writing host functions by hand, you can annotate the methods of a class and let the Endive annotation processor generate the host functions for you. +This is especially useful when you have many host functions. -### Loading and Instantiating Wasm Modules - -First your Wasm module must be loaded from disk and then instantiated. Let's [download a test module](https://raw.githubusercontent.com/bytecodealliance/endive/main/wasm-corpus/src/main/resources/compiled/iterfact.wat.wasm) . -This module contains some code to compute factorial: +## Load and instantiate a module -Download from the link or with curl: +[Download a test module](https://raw.githubusercontent.com/bytecodealliance/endive/main/wasm-corpus/src/main/resources/compiled/iterfact.wat.wasm) that computes a factorial, from the link or with curl: ```bash curl https://raw.githubusercontent.com/bytecodealliance/endive/main/wasm-corpus/src/main/resources/compiled/iterfact.wat.wasm > factorial.wasm @@ -56,7 +45,7 @@ docs.FileOps.copyFromWasmCorpus("iterfact.wat.wasm", "factorial.wasm"); ``` --> -Load this module and instantiate it: +Parse it and create an instance: ```java import run.endive.runtime.ExportFunction; @@ -70,25 +59,21 @@ var module = Parser.parse(new File("./factorial.wasm")); Instance instance = Instance.builder(module).build(); ``` +The module is inert code; the instance is a virtual machine ready to execute it. + :::note[Threading] `Instance` and `Store` are not thread-safe. Create separate instances per thread, or synchronize access externally. Memory operations (used by the Wasm threads proposal) are thread-safe. ::: -You can think of the `module` as of inert code, and the `instance` -is the run-time representation of that code: a virtual machine ready to execute. - -### Invoking a Wasm Function +## Call a function -Wasm modules, like all code modules, can export functions to the outside -world. This module exports a function called `"iterFact"`. -We can get a handle to this function using `Instance#export(String)`: +The module exports a function called `iterFact`. Get a handle to it with `Instance#export(String)`: ```java ExportFunction iterFact = instance.export("iterFact"); ``` -`iterFact` can be invoked with the `apply()` method. We must map any Java types to raw `long`s and do the reverse -when we want to go back to Java. +Call it with `apply()`. Arguments and results are raw `long`s, so map Java types yourself on both sides: ```java var result = iterFact.apply(5)[0]; @@ -102,5 +87,5 @@ docs.FileOps.writeResult("docs", "index.md.result", "" + result); --> :::note -Functions in Wasm can return multiple values, hence the array. This function only returns one value, so we take the first value. +Wasm functions can return multiple values, hence the array. This one returns a single value, so we take the first. ::: diff --git a/docs/docs/migration/from-chicory.md b/docs/docs/migration/from-chicory.md index 6f6cedce6..b555e75f6 100644 --- a/docs/docs/migration/from-chicory.md +++ b/docs/docs/migration/from-chicory.md @@ -4,8 +4,6 @@ sidebar_label: From Chicory title: Migrating from Chicory --- -# Migrating from Chicory to Endive - Endive is a fork of [Chicory](https://github.com/dylibso/chicory) by Dylibso, Inc. This guide documents all breaking changes for users migrating from Chicory. diff --git a/docs/docs/security/best-practices.md b/docs/docs/security/best-practices.md index d80ceb2fe..85f55b82e 100644 --- a/docs/docs/security/best-practices.md +++ b/docs/docs/security/best-practices.md @@ -4,8 +4,6 @@ sidebar_label: Best Practices title: Security Best Practices --- -# Security Best Practices - Practical guidance for running Wasm modules securely with Endive. ## Writing Safe Host Functions diff --git a/docs/docs/security/overview.md b/docs/docs/security/overview.md index fe2365a9b..428d7247c 100644 --- a/docs/docs/security/overview.md +++ b/docs/docs/security/overview.md @@ -4,8 +4,6 @@ sidebar_label: Security Model title: Security Model --- -# Security Model - Endive executes WebAssembly modules inside the JVM. Understanding the security boundaries is essential when running untrusted code. ## The Wasm Sandbox diff --git a/docs/docs/wasi/index.md b/docs/docs/wasi/index.md index 258810a3c..f23632d17 100644 --- a/docs/docs/wasi/index.md +++ b/docs/docs/wasi/index.md @@ -3,12 +3,6 @@ sidebar_position: 50 sidebar_label: WASI P1 title: Wasi Preview 1 --- -# WASI Preview 1 - -:::warning[Security Consideration] -WASI file access does not enforce path sandboxing by default. Always use a virtual filesystem (e.g., [ZeroFs](https://github.com/roastedroot/zerofs) or [Jimfs](https://github.com/google/jimfs)) to restrict guest access to pre-opened directories. Passing the host filesystem directly exposes all files the JVM process can access. See [Security Best Practices](/docs/security/best-practices). -::: - + +## Calling exports that use references + +Use `applyWithRefs` instead of `apply` for an export whose signature contains a reference type; `apply` throws on those. +The first array has one slot per parameter and carries the numeric arguments; the second carries the reference arguments, in order: + +```java +import run.endive.runtime.Instance; +import run.endive.runtime.WasmStruct; +import run.endive.wasm.Parser; +import java.io.File; + +var instance = Instance.builder(Parser.parse(new File("./gc.wasm"))).build(); + +// (param i32 i32) (result (ref $point)) +var point = (WasmStruct) instance.export("new_point").applyWithRefs(new long[] {3, 4}, null).refResult(0); + +// (param (ref null $point)) (result i32) +var sum = instance.export("sum").applyWithRefs(new long[] {0}, new Object[] {point}).longResult(0); +assert(7L == sum); +``` + +The returned `CallResult` exposes numeric results through `longResult(i)` and reference results through `refResult(i)`. + +## Reading and building structs and arrays + +`WasmStruct` exposes its fields by index: `field(i)` for numeric fields and `fieldRef(i)` for reference fields. +`WasmArray` does the same with `get(i)` and `getRef(i)`. +To create one on the host, use the builder and add the fields in declaration order: + +```java +assert(3L == point.field(0)); + +var origin = WasmStruct.builder(). + typeIdx(point.typeIdx()). + addField(0). + addField(0). + build(); +assert(0L == instance.export("sum").applyWithRefs(new long[] {0}, new Object[] {origin}).longResult(0)); +``` + + + +## Host functions receiving references + +A host function whose signature contains reference types implements `applyWithRefs` on `WasmFunctionHandle`, and is then wrapped in a `HostFunction` like any other: + +```java +import run.endive.runtime.CallResult; +import run.endive.runtime.WasmFunctionHandle; + +var handle = new WasmFunctionHandle() { + @Override + public long[] apply(Instance instance, long... args) { + throw new UnsupportedOperationException("called with references, use applyWithRefs"); + } + + @Override + public CallResult applyWithRefs(Instance instance, long[] args, Object[] refArgs) { + var p = (WasmStruct) refArgs[0]; + return CallResult.of(new long[] {p.field(0) + p.field(1)}, null); + } +}; +``` diff --git a/docs/docs/core/memory.md b/docs/docs/core/memory.md index f6ebb6f4f..60baeccf2 100644 --- a/docs/docs/core/memory.md +++ b/docs/docs/core/memory.md @@ -69,6 +69,22 @@ dealloc.apply(ptr, len); assert(3L == result); // 3 vowels in Hello, World! ``` +## Limiting memory growth + +A module declares the initial and maximum size of its memory in 64 KiB pages, and the maximum defaults to 4 GiB when it declares none. +To cap how far an untrusted module can grow, override the limits of its memory when building the instance: + +```java +import run.endive.wasm.types.MemoryLimits; + +// keep the declared 17 initial pages, allow growth up to 32 pages (2 MiB) +var limited = Instance.builder(Parser.parse(new File("./count_vowels.wasm"))). + withMemoryLimits(new MemoryLimits(17, 32)). + build(); +``` + +Past the maximum, `memory.grow` fails inside the module. The override applies to the memory the module defines; an imported memory keeps the limits it was created with. + -You can replace the `Memory` implementation backing a module through `withMemoryFactory`. -Workloads use memory in very different ways, so benchmark before changing it: +Endive ships two implementations of linear memory, and `withMemoryFactory` selects which one backs an instance: ```java -var instance = Instance.builder(module).withMemoryFactory(limits -> { - return new ByteArrayMemory(limits); - }).build(); +var instance = Instance.builder(module). + withMemoryFactory(ByteArrayMemory::new). + build(); ``` -:::note -Endive provides two built-in Memory implementations: -- `ByteArrayMemory`: An optimized memory implementation, recommended for recent OpenJDK systems -- `ByteBufferMemory`: Recommended for different Java runtimes (in particular, on Android VMs) -::: +## Choosing an implementation + +| Implementation | Plain loads and stores | Atomic instructions | Works on | +|---|---|---|---| +| `ByteBufferMemory` (default) | Good | Always take a lock | Every JDK and Android | +| `ByteArrayMemory` | Fastest on every JDK | Lock-free up to JDK 21, locked from JDK 23 | Every JDK | + +Atomic instructions only matter for modules built with the threads proposal that share their memory between threads. +For everything else `ByteArrayMemory` is the faster choice on a standard JVM, and `ByteBufferMemory` is the safe default where that cannot be assumed, such as Android. +From JDK 23 the JDK no longer provides lock-free multi-byte atomics on byte arrays, so a shared memory performs the same with either implementation there. + +Workloads use memory in very different ways, so benchmark before changing the default. + +[Redline](../experimental/redline.md#native-memory) keeps linear memory off-heap with lock-free atomics; that memory comes with the Redline runners and is not selectable here. + +Any class implementing `Memory` can be passed to `withMemoryFactory`, for example an off-heap implementation of your own. +The factory receives the module's declared limits, or the ones set with [`withMemoryLimits`](../core/memory.md#limiting-memory-growth). -This module expects us to fulfil an import with the name `console.log`. -As the name implies, this function allows a caller to log a message to standard output. -We could write it as the host function: +The module expects an import named `console.log` that prints a message, written as a host function like this: ```java -import run.endive.log.SystemLogger; import run.endive.wasi.WasiOptions; import run.endive.wasi.WasiPreview1; import run.endive.wasm.Parser; @@ -72,7 +71,6 @@ import run.endive.runtime.Store; import java.io.File; -var logger = new SystemLogger(); // let's just use the default options for now var options = WasiOptions.builder().build(); // create our instance of wasip1 @@ -121,7 +119,7 @@ var wasi = WasiPreview1.builder().withOptions(wasiOpts).build(); // greet-wasi is a rust program that greets the string passed in stdin var store = new Store().addFunction(wasi.toHostFunctions()); // instantiating will execute the module if it's a WASI command-pattern module -store.instantiate("hello-wasi", Parser.parse(new File("greet-wasi.wasm"))); +store.instantiate("greet-wasi", Parser.parse(new File("greet-wasi.wasm"))); // check that we output the greeting