Develop

.github/workflows/check-docs.yml

@@ -14,8 +14,25 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v4
 
+      # Jena 6 requires Java 21; the runner defaults to 17
+      - name: Set up Java 21
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: '21'
+
       - name: RDF syntax check
         uses: AtomGraph/RDF-syntax-check@v1.0.5
+        with:
+          jena-version: 6.1.0
+
+      - name: Install Linux packages
+        run: sudo apt-get update && sudo apt-get install -y libxml2-utils
+
+      # riot checks rdf:XMLLiteral well-formedness; canonical form is only
+      # enforced up to Jena 4.7.0, so it is checked separately
+      - name: XMLLiteral canonical form check
+        run: ./check-xmlliterals.sh
 
       - name: Check links
         run: ./check-links.sh

.gitignore

@@ -4,3 +4,4 @@ linkeddatahub/.DS_Store
 docs/docs.trix
 docs/files.xml
 docs/html
+docs/timestamps.xml

CLAUDE.md

@@ -34,6 +34,35 @@ make validate          # Validate documents
 make ttl-to-html      # Convert Turtle files to HTML
 ```
 
+### Documentation media (screencast/)
+
+Screenshots and clips for the docs are produced by a scripted Playwright rig, never taken by hand.
+
+```bash
+cd screencast
+node docs/shoot.mjs --base … --cert-file … --cert-password-file … [--only reference]
+make docs-publish   # optimise docs/out/ into ../docs/, printing sha1 per published file
+```
+
+- `docs/manifest.mjs` is the shot list — one entry per `div.screenshot-placeholder` in the `.ttl` sources, bound to its file, line and caption verbatim. New slots go there, not in the runner.
+- Each entry's `want` is asserted after `act` and before the capture, so a shot that never reached its state is reported missed rather than written.
+- `blocked` slots stay as placeholders with their reason recorded. Do not fill one by hand.
+- The shoot writes **masters** — 2880px lossless PNG, a `.webm` and an `.mp4` per clip — into `docs/out/`, plus an `index.json` of slot, caption, file and outcome. It never edits a `.ttl`.
+- `make docs-publish` derives the web assets into `docs/`: stills to WebP at 2240px (twice the docs' content column), `.mp4`s copied since they already carry CRF 20 / `yuv420p` / faststart, `.webm`s dropped.
+
+**Optimise before hashing.** Uploads are content-addressed at `{base}uploads/{sha1}`, so a reference is only valid for the bytes that ship — never hash a master.
+
+References are document-relative, one `../` per path segment of the page, and live inside `rdf:XMLLiteral` bodies that must stay canonical (attributes alphabetical, explicit end tags, each start tag on one line):
+
+```xml
+<img alt="The document tree with a container expanded" src="../../uploads/{sha1}"></img>
+<video aria-label="Browsing and navigating data" controls="controls" preload="metadata" src="../../uploads/{sha1}"></video>
+```
+
+A clip is a `<video>`, never an `<object>`. `Item.java` serves every upload with `Content-Security-Policy: default-src 'none'; sandbox` (the LNK-011 stored-XSS fix). An `<object>` loads the file as a nested *document*, so that CSP governs the document's own media load and Chrome blocks it — Firefox does not, which makes the breakage look browser-specific when it is not. A `<video>` is a subresource of the page, governed by the page's CSP, and loads normally. Images are unaffected for the same reason.
+
+Shipping them needs no script change: `docs/update-folder.sh` already uploads every non-`.ttl`/`.sh` file in a folder to that folder's container, and `ttl-to-html.xsl` rewrites `uploads/{sha1}` to `files/{name}` for the static build, terminating on a hash it cannot find — so a clean `make ttl-to-html` proves every baked hash resolves.
+
 ### Prerequisites
 All installation scripts require LinkedDataHub CLI scripts in PATH:
 ```bash
@@ -78,4 +107,5 @@ Some applications like SKOS use custom XSLT stylesheets and CSS that need to be
 - `.rq` - SPARQL query files
 - `.csv` - Data import files
 - `.sh` - Shell scripts for installation and deployment
+- `.webp` / `.mp4` - Documentation screenshots and clips, published by `make docs-publish`
 - `Makefile` - Interactive installation and build targets

README.md

@@ -4,12 +4,14 @@ System, demo, and user-submitted applications built on LinkedDataHub. Completely
 
 ## Prerequisites
 
-The installation scripts in this repository use [LinkedDataHub's CLI scripts](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/reference/command-line-interface/). You need to add the `bin` (and its subfolders) of your LinkedDataHub fork or clone to `$PATH`, for example:
+The installation scripts in this repository use [LinkedDataHub's `ldh` CLI](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/reference/command-line-interface/). Build it from your LinkedDataHub fork or clone and put it on `$PATH`:
 
 ```shell
-export PATH="$(find bin -type d -exec realpath {} \; | tr '\n' ':')$PATH"
+cd ../LinkedDataHub/cli && mvn package && export PATH="$PWD/bin:$PATH"
 ```
 
+The `bin` shell scripts that `ldh` replaces are deprecated. The certificate and WebID tooling in `bin` (`webid-keygen.sh`, `server-cert-gen.sh` and friends) talks to no API and is not.
+
 __Note that app installation scripts are not idempotent. Subsequent runs might continue adding data but are not guaranteed to succeed.__
 
 ## Apps
@@ -30,6 +32,31 @@ __Note that app installation scripts are not idempotent. Subsequent runs might c
     <dd>164 lines of installation shell scripts</dd>
 </dl>
 
+#### Screenshots and screen recordings
+
+The documentation's images and clips are not taken by hand. Every
+`div.screenshot-placeholder` in the `.ttl` sources is a slot in
+[`screencast/docs/manifest.mjs`](screencast/docs/manifest.mjs), which binds it to a
+scripted Playwright run against a live dataspace — so a shot is reproducible, and a
+slot that cannot be shot against the demo data says why instead of going missing.
+
+```shell
+cd screencast
+node docs/shoot.mjs --base … --cert-file … --cert-password-file …   # into docs/out/
+make docs-publish                                                   # into ../docs/
+```
+
+The shoot writes masters — 2880px lossless PNG, a `.webm` and an `.mp4` per clip.
+`make docs-publish` derives the shipping copies into `docs/`, converting the stills
+to WebP at twice the docs' content width and copying the already-optimised `.mp4`s,
+then prints each published file's SHA-1. That hash is its address: uploads are
+content-addressed at `{base}uploads/{sha1}`, so the bytes have to be final before
+anything references them. The references themselves are plain XHTML in the literal
+bodies — `<img src="../../uploads/{sha1}">` and `<video src="../../uploads/{sha1}">`
+— and `docs/install.sh` uploads the files along with the documents.
+
+[More on the recording rig →](screencast/README.md)
+
 ### Northwind Traders
 
 ![Set-based (parallax) navigation](demo/northwind-traders/screenshot.gif "Set-based (parallax) navigation")
@@ -101,41 +128,34 @@ __You need to request append/write access to be able to create/edit the data.__
 
 **Reusable vocabulary packages that add domain-specific functionality to LinkedDataHub dataspaces.**
 
-Packages provide ontology imports and custom XSLT templates for rendering specific RDF vocabularies. They use **installation-time composition** - integrating content during installation via JAX-RS endpoints rather than loading dynamically at runtime.
+Packages provide ontology imports and custom XSLT templates for rendering specific RDF vocabularies. They are **composed at request time** out of a declaration: a single `ldh:import` triple in the application's settings, resolved on the next request.
 
 ### Structure
 
 Each package consists of:
 - **`ns.ttl`** - Ontology with vocabulary imports (`owl:imports`) and property views (`ldh:view` / `ldh:inverseView`)
-- **`layout.xsl`** - XSLT stylesheet with custom rendering templates using system modes
+- **A stylesheet** - XSLT with custom rendering templates using system modes. The filename is whatever the package's `ac:stylesheet` names.
 
 ### Installation
 
-Packages are installed via CLI or from application install scripts:
+The declaration *is* the installation:
 
 ```bash
-install-package.sh \
-  -b https://localhost:4443/ \
-  -f ssl/owner/cert.pem \
-  -p Password \
-  --package https://packages.linkeddatahub.com/skos/#this
+ldh packages list
+ldh packages add --package https://packages.linkeddatahub.com/editor/taxonomy/#this
 ```
 
-Installation integrates the package by:
-1. Downloading and storing the ontology as a SPARQL document
-2. Adding `owl:imports` to the namespace ontology
-3. Saving the stylesheet and updating master stylesheets with `xsl:import`
-4. Clearing caches to load the new content
+From the next request onwards the server resolves it: the package ontology joins the application's `owl:imports` closure, and its stylesheet is composed into the application stylesheet. No restart, and nothing is copied into the webapp.
 
 ### Available Packages
 
-- **[skos](packages/skos/)** - SKOS vocabulary support (concepts, schemes, collections)
+- **[editor/taxonomy](packages/editor/taxonomy/)** - Taxonomy editing on SKOS (concepts, schemes, collections)
 
 ### Architecture
 
 - **Declarative only** - RDF + XSLT, no Java code
-- **Installation-time composition** - Pre-composed before loading, no runtime overhead
+- **Request-time composition** - resolved from the `ldh:import` declaration, nothing copied into the webapp
 - **Property views** (`ldh:view` / `ldh:inverseView`) - SPARQL-based views attached to properties
-- **XSLT overrides** - Custom rendering using system modes (`bs2:*`, `xhtml:*`, etc.)
+- **XSLT overrides** - Custom rendering using system modes (`ac:*`, `ldh:*`, `xhtml:*`, etc.)
 
 [Read the full packages documentation →](packages/README.md)

check-links.sh

@@ -15,7 +15,16 @@ print(result[len('http://x/'):])
 
 while IFS= read -r -d '' ttl_file; do
     rel="${ttl_file#"$REPO_DIR"/}"
-    url_path="${rel%.ttl}/"
+
+    # A .ttl is served at <its path minus .ttl>/ — except root.ttl, which install.sh PUTs
+    # at the application base itself rather than at <dir>/root/. Resolving a root.ttl's
+    # relative links against <dir>/root/ would send them one level too deep.
+    if [[ "$(basename "$rel")" == "root.ttl" ]]; then
+        url_path="$(dirname "$rel")/"
+        [[ "$url_path" == "./" ]] && url_path=""
+    else
+        url_path="${rel%.ttl}/"
+    fi
 
     while IFS= read -r url; do
         [[ "$url" =~ ^https?:// ]]  && continue

check-xmlliterals.sh

@@ -0,0 +1,80 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Checks that every rdf:XMLLiteral lexical form in the RDF corpus is in exclusive
+# canonical XML form, which RDF 1.1 requires of the datatype's lexical space.
+#
+# Jena 4.7.0 enforced this while validating; Jena 5 and later only check
+# well-formedness. Without this check a non-canonical literal passes both
+# `make validate` and the CI syntax check while still being invalid.
+#
+# The rules that follow from canonical form, when authoring XHTML literals:
+# attributes in alphabetical order, explicit end tags (<br></br>, never <br/>),
+# and each start tag on a single line.
+#
+# Usage: check-xmlliterals.sh [directory]   (defaults to the script's own directory)
+
+ROOT_DIR="$(cd "${1:-$(dirname "$0")}" && pwd)"
+EXIT_CODE=0
+
+if ! command -v xmllint > /dev/null; then
+    echo "check-xmlliterals.sh: xmllint is required (Debian/Ubuntu: libxml2-utils)" >&2
+    exit 2
+fi
+
+WORK_DIR="$(mktemp -d)"
+trap 'rm -rf "$WORK_DIR"' EXIT
+
+cd "$ROOT_DIR"
+
+while IFS= read -r -d '' file; do
+    rm -f "$WORK_DIR"/lit.*
+
+    # Write the lexical form of each """..."""^^rdf:XMLLiteral literal to its own
+    # file under $WORK_DIR, and report the literal's number and starting line.
+    awk -v out="$WORK_DIR" '
+        function emit(body, start,    path) {
+            path = out "/lit." ++n
+            printf "%s", body > path
+            close(path)
+            print n, start
+        }
+        {
+            pos = 1
+            while ((idx = index(substr($0, pos), "\"\"\"")) > 0) {
+                at = pos + idx - 1
+                if (inside) {
+                    body = body substr($0, pos, at - pos)
+                    inside = 0
+                    if (substr($0, at + 3) ~ /^[ \t]*\^\^[ \t]*rdf:XMLLiteral/) emit(body, start)
+                } else {
+                    inside = 1
+                    start = NR
+                    body = ""
+                }
+                pos = at + 3
+            }
+            if (inside) body = body substr($0, pos) "\n"
+        }
+    ' "$file" > "$WORK_DIR/manifest"
+
+    while read -r literal start; do
+        lexical_form="$WORK_DIR/lit.$literal"
+
+        if ! xmllint --exc-c14n "$lexical_form" > "$WORK_DIR/canonical" 2> "$WORK_DIR/parse-error"; then
+            echo "$file:$start: rdf:XMLLiteral is not well-formed XML" >&2
+            sed 's|^|    |' "$WORK_DIR/parse-error" >&2
+            EXIT_CODE=1
+            continue
+        fi
+
+        if ! cmp -s "$lexical_form" "$WORK_DIR/canonical"; then
+            echo "$file:$start: rdf:XMLLiteral is not canonical XML" >&2
+            # diff exits non-zero when the files differ, which set -e would treat as fatal
+            diff -u "$lexical_form" "$WORK_DIR/canonical" | sed -e '1,2d' -e 's|^|    |' >&2 || true
+            EXIT_CODE=1
+        fi
+    done < "$WORK_DIR/manifest"
+done < <(find . \( -name '*.ttl' -o -name '*.trig' \) -type f -not -path '*/.git/*' -not -path '*/html/*' -print0)
+
+exit "$EXIT_CODE"

demo/copenhagen/bicycle-parkings.ttl

@@ -14,12 +14,11 @@
 # Intro XHTML
 <#cycling-intro> a ldh:XHTML ;
     rdf:value """<div xmlns="http://www.w3.org/1999/xhtml">
-        <h3>Cycling infrastructure</h3>
-        <p>Copenhagen is renowned as one of the world's most bicycle-friendly cities, with extensive cycling infrastructure
-           serving hundreds of thousands of daily cyclists. The city's bicycle parking facilities are strategically located
-           throughout the urban area to support this sustainable transportation network.</p>
-        <p>Explore the map below to discover bicycle parking locations across Copenhagen, including capacity information
-           and geographic distribution patterns.</p>
+        <div class="ldh-section">
+            <h2>Where the bikes go</h2>
+            <p>4,390 municipal bicycle racks, each with its capacity, street address and city district. Capacity
+               is the number the rack was built for, so the map reads as supply rather than as points.</p>
+        </div>
     </div>"""^^rdf:XMLLiteral .
 
 # Object block
@@ -28,5 +27,6 @@
 
 # View block with MapMode
 <#select-children-view> a ldh:View ;
+    dct:title "All bicycle parkings" ;
     spin:query ldh:SelectChildren ;
     ac:mode ac:MapMode .

demo/copenhagen/charging-stations.ttl

@@ -8,13 +8,25 @@
 # Main container document
 <> a dh:Container ;
     dct:title "Charging stations" ;
-    rdf:_1 <#select-children> .
+    rdf:_1 <#charging-intro> ;
+    rdf:_2 <#select-children> .
+
+# Intro XHTML
+<#charging-intro> a ldh:XHTML ;
+    rdf:value """<div xmlns="http://www.w3.org/1999/xhtml">
+        <div class="ldh-section">
+            <h2>Where the cars charge</h2>
+            <p>248 public charging points, each with the operator that runs it, its street address and how many
+               vehicles it takes at once.</p>
+        </div>
+    </div>"""^^rdf:XMLLiteral .
 
 # Object block
 <#select-children> a ldh:Object ;
     rdf:value <#select-children-view> .
 
 # View block with MapMode
 <#select-children-view> a ldh:View ;
+    dct:title "All charging stations" ;
     spin:query ldh:SelectChildren ;
     ac:mode ac:MapMode .

demo/copenhagen/libraries.ttl

@@ -14,12 +14,11 @@
 # Intro XHTML
 <#libraries-intro> a ldh:XHTML ;
     rdf:value """<div xmlns="http://www.w3.org/1999/xhtml">
-        <h3>Public libraries</h3>
-        <p>Copenhagen's public library system serves as a cornerstone of community life, providing free access to
-           knowledge, culture, and digital resources. From the iconic central library to neighborhood branches,
-           these facilities support learning, creativity, and social connection across all city districts.</p>
-        <p>Use the interactive map below to locate libraries throughout Copenhagen and discover their unique
-           characteristics and services.</p>
+        <div class="ldh-section">
+            <h2>The branch network</h2>
+            <p>Nineteen public libraries, each with its address, postal district, contact address and its page on
+               the city's library site. Open a branch to follow the link out.</p>
+        </div>
     </div>"""^^rdf:XMLLiteral .
 
 # Object block
@@ -28,5 +27,6 @@
 
 # View block with MapMode
 <#select-children-view> a ldh:View ;
+    dct:title "All libraries" ;
     spin:query ldh:SelectChildren ;
     ac:mode ac:MapMode .

demo/copenhagen/parking-facilities.ttl

@@ -9,8 +9,9 @@
 # Main container document
 <> a dh:Container ;
     dct:title "Parking facilities" ;
-    rdf:_1 <#largest-parkings-block> ;
-    rdf:_2 <#select-children> .
+    rdf:_1 <#parking-intro> ;
+    rdf:_2 <#largest-parkings-block> ;
+    rdf:_3 <#select-children> .
 
 # Chart block (wrapped in ldh:Object per ContentMode contract)
 <#largest-parkings-block> a ldh:Object ;
@@ -43,11 +44,22 @@ WHERE
 ORDER BY DESC(?spaces)
 LIMIT 100""" .
 
+# Intro XHTML
+<#parking-intro> a ldh:XHTML ;
+    rdf:value """<div xmlns="http://www.w3.org/1999/xhtml">
+        <div class="ldh-section">
+            <h2>The city's car parks</h2>
+            <p>Eighteen public car parks with their operator, address and capacity. Capacity is what ranks them in
+               the chart below and what the map is worth reading for.</p>
+        </div>
+    </div>"""^^rdf:XMLLiteral .
+
 # Object block
 <#select-children> a ldh:Object ;
     rdf:value <#select-children-view> .
 
 # View block with MapMode
 <#select-children-view> a ldh:View ;
+    dct:title "All parking facilities" ;
     spin:query ldh:SelectChildren ;
     ac:mode ac:MapMode .