A browser application that trains small causal language models from scratch and compares how they continue the same natural-language task after different clause-order training histories. Training and inference execute in WebGPU. There is no inference API, PyTorch installation, server-side training, or scripted response fallback.
Live site: https://samg-coder.github.io/lingpath-studio-webshader/
Open it in Chrome or Edge with hardware acceleration. Training and weights stay in your browser. GitHub Pages is HTTPS, which satisfies the WebGPU origin requirement (file:// does not).
- Extract the ZIP to a normal writable directory.
- Double-click START_WINDOWS.bat. Python 3.10+ is needed only for a local static server and the first dependency download. No Python packages are installed.
- Open the displayed local address in Chrome or Edge with hardware acceleration enabled.
- Leave CUDA → WebShader (upstream compiler) selected. Choose the two language profiles, press Train matched pair, then open Compare language.
The first launch retrieves the compiler source from SamG-Coder/cuda-webshader at commit c272bb782f8458761af2894e8771517f64ecdae6. Each file is checked against its Git blob checksum; a SHA-256 manifest is stored locally. Subsequent launches use that cache. An internet connection is required only for that first compiler download. No model/data upload occurs.
Do not open index.html directly using file://. WebGPU and JavaScript modules need a suitable origin; localhost or HTTPS is the supported setup.
Alternative: python server.py. For a dependency-free Node static server: node server.mjs. That Node server does not install the compiler; run python server.py --vendor-only first, or use the explicitly labelled reference engine.
Copy this entire directory to showcases/lingpath-studio/ in your repository and serve the repository normally. The application detects that URL path and imports your repository's src/compiler/compiler.js directly. It does not download a second compiler in that mode. The supplied integration ZIP already has that directory structure.
Open showcases/lingpath-studio/index.html. No existing repository file needs to be overwritten. See docs/INTEGRATION.md for a minimal explorer link. This standalone repository is published at https://github.com/SamG-Coder/lingpath-studio-webshader.
- Two independently trained, byte-identically initialised autoregressive transformers.
- Whole-clause ordering in actual English sentences, edited by preset, drag-and-drop or arrow controls.
- A balanced condition that samples all 24 clause permutations.
- Matching semantic story IDs, minibatch order, continuation text, vocabulary and non-padding token budgets.
- Twenty CUDA kernels for embedding, tiled matrix multiplication, RMSNorm, causal attention, SiLU, cross-entropy, complete backpropagation, global gradient clipping and AdamW.
- Live measured cross-entropy, held-out native/common-format loss, progress, GPU details and session logs.
- Natural-language prompts and actual autoregressive text generation.
- Candidate continuation likelihoods, displayed as relative scores, not calibrated action probabilities.
- Token-prefix next-word distributions, distribution entropy and a causal attention heatmap.
- Local IndexedDB saving, binary checkpoint import/export, optimizer restoration and paused-run continuation.
- Import/export of paired story corpora in JSONL, and JSON exports of comparisons and prefix traces.
- Browser diagnostics for causal masking, gradients, genuine loss reduction and checkpoint/optimizer restoration.
The included corpus has 2,048 training stories and 256 disjoint held-out combinations, drawn from six domains: safety, lost property, repair, navigation, causal evidence and resources. The training-only word tokenizer has around 135 tokens.
Default model sizes are intentionally small: approximately 19,000 parameters (Micro), 87,000 (Standard) and 253,000 (Desktop), at context 64 and the bundled vocabulary. These are small language models, not general assistants. They learn the supplied story language. Unfamiliar words are visibly mapped to <unk>; this is not an unrestricted English chat model.
600 updates is an initial training block, not a guarantee of fluent or correct responses. Start with held-out examples, inspect validation loss and add training as needed. The Micro preset is suitable for a first check. Model A and B are trained sequentially within each matched step, not as hundreds of concurrent processes.
The stories contain authored continuations explaining actions. Both models see exactly the same continuations. The experiment tests how representation order affects learning and prediction; it does not discover values from an otherwise value-free world.
demo-pair.lps is a small pair trained in this build's explicitly labelled WGSL reference engine for a functional demonstration. Click Load trained demo, then open Compare language. You can also import the file manually. It is not an experimental result or a recommended grammar. Its model/corpus/seed/training metadata are stored inside the checkpoint. Start a new run to choose your own profiles.
START_OFFLINE_REFERENCE.bat skips downloading the compiler. In the application choose Offline WGSL reference. This is a separately labelled mechanical WGSL mirror of this project's twenty owned kernels, not output secretly attributed to cuda-webshader. It performs the same real WebGPU training.
The local tests ran actual browser WebGPU computation through Chromium's SwiftShader software adapter, using the reference kernels. The upstream compiler could be inspected through the GitHub connector but its full source tree could not be downloaded into this authoring runtime. Consequently the complete upstream-compiled path was not executed here, nor was physical RTX 5080 performance measured. First training automatically runs numerical checks in the engine you select. Compiler failures are surfaced; there is no silent switch to reference mode.
Read docs/VALIDATION.md and the included JSON reports. CUDA sources passed C++ syntax checks with CUDA declarations stubbed; this is not an NVCC/native-CUDA execution claim.
It changes the order of complete English clauses, not all grammatical properties of an artificial language. Native prompt format and common prompt format are different evaluation conditions. The common format is itself a defined sentence order, not a language-neutral representation.
A single seed's difference is not evidence of a general grammar effect. Compare multiple seeds, native/common prompts, same-grammar control pairs and balanced controls. Different templates/architectures could produce different outcomes. Attention maps are not causal explanations; generated text is not a record of internal reasoning. Nothing here establishes a fact about human neural pathways, national cultures or moral behaviour.
The most reliable starting point is Export paired corpus. Edit its clause fields and continuation, then import it before creating models. Each record must have clauses.person, .event, .object, .context and a natural-language continuation. Optional choices are used only for inference comparison.
Imported records are deduplicated by exact clause content, sorted by stable IDs and split deterministically (every tenth record held out). The importer recomputes this split; an input split field is not blindly trusted. Maximum 12,000 records, 8 MB and 512 word tokens. Stories that exceed the chosen context produce an error rather than being truncated. This importer does not claim to automatically parse/rewrite arbitrary books into semantically equivalent grammar variants.
npm test
python -m unittest discover -s tests -p 'test_*.py'
python server.py --no-vendorThen open /tests/index.html to test reference WebGPU or /tests/index.html?engine=webshader to check the installed upstream compiler. The app's Controls & kernels tab offers the same engine-specific checks.
src/model.js training graph · src/kernel-specs.js CUDA sources and independent reference emitter · kernels/ readable CUDA/WGSL mirrors · src/corpus.js matched language generation · src/gpu.js compiler integration and WebGPU host · src/inference.js language generation and traces · tests/ numerical and data checks.
Upstream: https://github.com/SamG-Coder/cuda-webshader Pinned source: https://github.com/SamG-Coder/cuda-webshader/tree/c272bb782f8458761af2894e8771517f64ecdae6