Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ But first, read this page (including the small print at the end).
+ [Coding Guidelines](#coding-guidelines)
+ [Continuous Integration](#continuous-integration)
+ [Tests and documentation are not optional](#tests-and-documentation-are-not-optional)
+ [Writing docs](#writing-docs)
* [Reporting an issue](#reporting-an-issue)
* [AI Tool Use Policy](#ai-tool-use-policy)
* [Legal](#legal)
Expand Down Expand Up @@ -159,6 +160,17 @@ To automatically apply and approve e new version of the "Golden samples" used by
APPROVAL_TESTS_USE_REPORTER=AutoApproveReporter
```

### Writing docs

The website lives in `docs/` (see [its README](docs/README.md) for the build). Every page under `docs/docs/` follows these rules:

- Describe how to use a feature, not how it is implemented. Internals belong in Javadoc or in the pull request description.
- No release-time or migration statements. Those go in a blog post; breaking changes relative to Chicory go in the migration guide.
- One code sample per option. Show a builder once per page.
- Body headings start at `##`; the frontmatter `title` is the page heading. Open every page with one plain sentence saying what it covers, which becomes its description in `llms.txt`.
- Names: Wasm, interpreter, runtime compiler, build-time compiler. Version placeholder: `${endive.version}`.
- Every page is executed by `npm test` in `docs/`: keep the hidden `//DEPS` and `docs.FileOps.writeResult(...)` blocks, and update `docs/tests/approvals/` when a page moves.

## Reporting an issue

This project uses GitHub issues to manage the issues. Open an issue directly in GitHub.
Expand Down
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,5 @@ $ npm run build

This command generates static content into the `build` directory and can be served using any static contents hosting service.


The rules for writing pages are in [CONTRIBUTING.md](../CONTRIBUTING.md#writing-docs).
2 changes: 1 addition & 1 deletion docs/blog/2025-02-20-chicory-1.1.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,5 @@ With this release, we’re introducing a **“Who Uses Chicory?”** section in

Try out **Chicory 1.1.0** today, and let us know what you think!

[simd]: /docs/advanced/simd
[simd]: /docs/execution
[memory]: /docs/advanced/memory-customization
2 changes: 0 additions & 2 deletions docs/docs/advanced/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ sidebar_position: 4
sidebar_label: Logging
title: Logging
---
# Logging

For maximum compatibility and to avoid external dependencies we use, by default, the JDK Platform Logging ([JEP 264](https://openjdk.org/jeps/264)).
The Platform Logging falls back to the [`java.util.logging` API](https://docs.oracle.com/en/java/javase/21/core/java-logging-overview.html) by default.

Expand Down
42 changes: 23 additions & 19 deletions docs/docs/advanced/memory-customization.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
sidebar_position: 5
sidebar_label: Memory
title: Advanced Wasm Memory Customization
sidebar_label: Memory Customization
title: Memory Customization
---
<!--
```java
Expand All @@ -10,10 +10,6 @@ title: Advanced Wasm Memory Customization

docs.FileOps.copyFromWasmCorpus("count_vowels.rs.wasm", "count_vowels.rs.wasm");

System.setOut(new PrintStream(
new BufferedOutputStream(
new FileOutputStream("docs/examples/rust.md.result"))));

import run.endive.wasm.Parser;
import run.endive.runtime.Instance;
import run.endive.runtime.ByteArrayMemory;
Expand All @@ -23,23 +19,31 @@ var module = Parser.parse(new File("count_vowels.rs.wasm"));
```
-->

Different Wasm workloads will be using the memory in very different ways.

Make sure you don't do changes without proper benchmarking and analysis.

It's possible to provide a custom implementation of the entire Memory used by the Wasm module:
Endive ships two implementations of linear memory, and `withMemoryFactory` selects which one backs an instance:

```java
var instance = Instance.builder(module).withMemoryFactory(limits -> {
return new ByteArrayMemory(limits);
}).build();
var instance = Instance.builder(module).
withMemoryFactory(ByteArrayMemory::new).
build();
```

:::note
Endive provides two built-in Memory implementations:
- `ByteArrayMemory`: An optimized memory implementation, recommended for recent OpenJDK systems
- `ByteBufferMemory`: Recommended for different Java runtimes (in particular, on Android VMs)
:::
## Choosing an implementation

| Implementation | Plain loads and stores | Atomic instructions | Works on |
|---|---|---|---|
| `ByteBufferMemory` (default) | Good | Always take a lock | Every JDK and Android |
| `ByteArrayMemory` | Fastest on every JDK | Lock-free up to JDK 21, locked from JDK 23 | Every JDK |

Atomic instructions only matter for modules built with the threads proposal that share their memory between threads.
For everything else `ByteArrayMemory` is the faster choice on a standard JVM, and `ByteBufferMemory` is the safe default where that cannot be assumed, such as Android.
From JDK 23 the JDK no longer provides lock-free multi-byte atomics on byte arrays, so a shared memory performs the same with either implementation there.

Workloads use memory in very different ways, so benchmark before changing the default.

[Redline](../experimental/redline.md#native-memory) keeps linear memory off-heap with lock-free atomics; that memory comes with the Redline runners and is not selectable here.

Any class implementing `Memory` can be passed to `withMemoryFactory`, for example an off-heap implementation of your own.
The factory receives the module's declared limits, or the ones set with [`withMemoryLimits`](../core/memory.md#limiting-memory-growth).

<!--
```java
Expand Down
29 changes: 0 additions & 29 deletions docs/docs/advanced/simd.md

This file was deleted.

20 changes: 8 additions & 12 deletions docs/docs/advanced/tools.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,31 @@
---
sidebar_position: 3
sidebar_label: WABT
title: WABT
sidebar_label: Tools
title: Tools
---
# Tools
Endive publishes a few WebAssembly tools as Maven artifacts, compiled to Wasm and run with Endive itself.

## WebAssembly Binary Toolkit

Since we use them in the build of the project, we publish a few [wabt](https://github.com/WebAssembly/wabt) tools compiled at build time with Endive.
Adding support for more tools is easy and we welcome contributions in this direction.
The relevant module can be added to the build with:
A few [wabt](https://github.com/WebAssembly/wabt) tools, the ones Endive's own build uses:

```xml
<dependency>
<groupId>run.endive</groupId>
<artifactId>wabt</artifactId>
<version>latest-release</version>
<version>${endive.version}</version>
</dependency>
```

## Wasm Tools

As we need it to catch up with the latest development of the Wasm spec we publish a few [wasm-tools](https://github.com/bytecodealliance/wasm-tools) compiled at build time with Endive.
The relevant module can be added to the build with:
A few [wasm-tools](https://github.com/bytecodealliance/wasm-tools) binaries, which track the latest Wasm proposals:

```xml
<dependency>
<groupId>run.endive</groupId>
<artifactId>wasm-tools</artifactId>
<version>latest-release</version>
<version>${endive.version}</version>
</dependency>
```

Expand All @@ -49,8 +46,7 @@ System.setOut(new PrintStream(

## wat2wasm

In Endive, we don't have a Wasm text format parser just yet.
To overcome this limitation you can use `wat2wasm`, for example:
Endive does not parse the Wasm text format. Use `wat2wasm` to convert text to binary:

```java
import run.endive.wabt.Wat2Wasm;
Expand Down
5 changes: 1 addition & 4 deletions docs/docs/ai-friendly-docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ sidebar_label: AI-Friendly Docs
title: AI-Friendly Documentation
---

# AI-Friendly Documentation

Endive documentation is designed to be easily consumed by LLMs, AI agents, and developer tools. All machine-readable files are auto-generated at build time from the same source as the human-facing site, using the [llms.txt specification](https://llmstxt.org/).

## Available Endpoints
Expand All @@ -29,8 +27,7 @@ Every documentation page also has a **raw Markdown** version available by append
Fetch `llms-full.txt` and include it in your prompt context:

```bash
curl -s https://endive.run/llms-full.txt | wc -c
# ~86 KB — fits easily in any modern LLM context window
curl -s https://endive.run/llms-full.txt
```

### Let an AI agent discover relevant pages
Expand Down
2 changes: 0 additions & 2 deletions docs/docs/annotations/_category_.yml

This file was deleted.

94 changes: 47 additions & 47 deletions docs/docs/annotations/index.md → docs/docs/core/annotations.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,43 @@
---
sidebar_position: 100
sidebar_position: 4
sidebar_label: Annotations
title: Annotations
---
## Host Modules
The annotation processor generates host functions from annotated Java methods, and typed Java bindings from a Wasm module, so you do not write the `HostFunction` boilerplate by hand.

Instead of writing host functions by hand, you can write a class containing annotated methods
and let the Endive annotation processor generate the host functions for you. This is especially
useful when you have many host functions.
## Setup

Add the annotations dependency:

```xml
<dependency>
<groupId>run.endive</groupId>
<artifactId>annotations</artifactId>
<version>${endive.version}</version>
</dependency>
```

and register `annotations-processor` as an annotation processor of the Java compiler, with Maven:

```xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<annotationProcessorPaths>
<path>
<groupId>run.endive</groupId>
<artifactId>annotations-processor</artifactId>
<version>${endive.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
```

## Host modules

Annotate the methods of a class and the processor generates one host function per method:

<!--
```java
Expand All @@ -19,9 +49,8 @@ useful when you have many host functions.
```java
@HostModule("demo")
public final class Demo {

public Demo() {}

private final Random random = new Random();

@WasmExport
public long add(int a, int b) {
return a + b;
Expand Down Expand Up @@ -76,14 +105,16 @@ var imports = ImportValues.builder().addFunction(demo.toHostFunctions()).build()

### Type conversions

The following conversions are supported:
Parameters and return values are converted as follows:

| Java type | Wasm type |
|-----------|-----------|
| `int` | `i32` |
| `long` | `i64` |
| `float` | `f32` |
| `double` | `f64` |

| Java Type | Wasm Type |
|-------------------|------------|
| `int` | `i32` |
| `long` | `i64` |
| `float` | `f32` |
| `double` | `f64` |
Parameters can also be a `String` annotated with `@Buffer` (passed as an `i32` pointer and an `i32` length) or `@CString` (an `i32` pointer to a NUL-terminated string), both read from the module's memory, and `Instance` or `Memory`, which receive the calling instance or its memory without a Wasm parameter.

## WasmModuleInterface

Expand All @@ -110,39 +141,8 @@ This annotation generates several things, depending on the provided module:

You can find examples of how to use the generated code in the [`annotations/it` folder of the repository](https://github.com/bytecodealliance/endive/tree/main/annotations/it).

## Enabling the Annotation Processor

In order to use host modules, you need to import the relevant annotations, e.g. in Maven:

```xml
<dependency>
<groupId>run.endive</groupId>
<artifactId>annotations</artifactId>
<version>latest-release</version>
</dependency>
```

and configure the Java compiler to include the Endive `annotations-processor` as an annotation processor.
Exactly how this is done depends on the build system you are using, for instance, with Maven:

```xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<annotationProcessorPaths>
<path>
<groupId>run.endive</groupId>
<artifactId>annotations-processor</artifactId>
<version>latest-release</version>
</path>
</annotationProcessorPaths>
</configuration>
</plugin>
```

<!--
```java
docs.FileOps.writeResult("docs/annotations", "index.md.result", "empty");
docs.FileOps.writeResult("docs/core", "annotations.md.result", "empty");
```
-->
29 changes: 0 additions & 29 deletions docs/docs/core/execution-modes.md

This file was deleted.

Loading
Loading