VisionDemo: live object detection from a SKaiNET cartridge

.github/workflows/visiondemo-ci.yml

@@ -0,0 +1,48 @@
+name: VisionDemo CI
+
+on:
+  pull_request:
+    paths:
+      - "VisionDemo/**"
+      - "skainet-ui/**"
+      - ".github/workflows/visiondemo-ci.yml"
+  push:
+    branches:
+      - develop
+    paths:
+      - "VisionDemo/**"
+      - "skainet-ui/**"
+      - ".github/workflows/visiondemo-ci.yml"
+
+concurrency:
+  group: visiondemo-ci-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repo
+        uses: actions/checkout@v4
+
+      - name: Set up JDK
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: "21"
+
+      - name: Set up Gradle
+        uses: gradle/actions/setup-gradle@v4
+
+      - name: Make gradlew executable
+        run: chmod +x VisionDemo/gradlew
+
+      # The debug APK assembly proves the whole chain — the published cartridge artifacts
+      # resolve from Central, the KMP library compiles against the yolo/v1 contract, and the
+      # JNI runner packaging merges. Deliberately no packs and no weights in CI (they are
+      # AGPL-materialized locally; bundlePacks is a no-op on a bare checkout), and no
+      # instrumented tests (needs a device).
+      - name: Build
+        working-directory: VisionDemo
+        run: ./gradlew :androidApp:assembleDebug --console=plain

README.md

@@ -147,6 +147,20 @@ Confidence: 98.3%
 
 📂 [`MnistJavaDemo/`](MnistJavaDemo/) · command-line · Java 21+
 
+### 📷 Vision Demo
+
+Live camera **object detection** from a SKaiNET cartridge: YOLOv8n (COCO, 80 classes)
+compiled through IREE runs on the phone's GPU — ~57 ms per frame (≈17 fps) on a Pixel 8a —
+with a glass HUD, FPS sparkline, CPU/Vulkan backend switch, capture gallery and detections
+viewer. The app consumes the published `sk.ainet.cartridge` artifacts (code only); the
+pack it runs is materialized locally from the
+[detection blueprint](https://github.com/SKaiNET-developers/SKaiNET-cartridge-blueprints) —
+the AGPL-3.0 weights never ship, and a fresh clone builds and runs pack-less.
+
+📂 [`VisionDemo/`](VisionDemo/) · runs on Android
+
+---
+
 ### 🌸 Iris Classifier
 
 The classic Iris dataset as the smallest possible **training loop**: a two-layer dense

VisionDemo/.gitignore

@@ -0,0 +1,18 @@
+*.iml
+.kotlin
+.gradle
+**/build/
+xcuserdata
+!src/**/build/
+local.properties
+.idea
+.DS_Store
+captures
+.externalNativeBuild
+.cxx
+*.xcodeproj/*
+!*.xcodeproj/project.pbxproj
+!*.xcodeproj/xcshareddata/
+!*.xcodeproj/project.xcworkspace/
+!*.xcworkspace/contents.xcworkspacedata
+**/xcshareddata/WorkspaceSettings.xcsettings

VisionDemo/README.md

@@ -0,0 +1,75 @@
+# VisionDemo — live object detection from a SKaiNET cartridge
+
+Live camera object detection on Android: YOLOv8n (COCO, 80 classes) compiled through IREE,
+running on the phone's GPU (Vulkan) or CPU, consumed as a **SKaiNET cartridge**. The app is
+the published cartridge API end to end — glass HUD with FPS readout and sparkline, backend
+switch (CPU / VULKAN / VK·320), capture gallery, a detections viewer and an image-picker
+path beside the live feed.
+
+Verified on a Pixel 8a: **~57 ms per frame (≈17 fps) live** on the 320-input Vulkan pack,
+~174 ms on the 640 one; boxes and labels map 1:1 onto the analyzed frames.
+
+## How it consumes the model
+
+This example owns **no model code and no weights**. It depends on two small libraries from
+Maven Central, published out of
+[SKaiNET-cartridge-blueprints](https://github.com/SKaiNET-developers/SKaiNET-cartridge-blueprints):
+
+```kotlin
+implementation("sk.ainet.cartridge:detection-yolov8-iree:0.2.2")   // opens, verifies, runs packs
+// brings sk.ainet.cartridge.profiles:yolo-v1 (the task-profile contract) transitively
+```
+
+What actually runs is a **pack** — compiled graphs (`.vmfb`), converted parameters (`.irpa`)
+and the native runtime, under a signed manifest — and packs are **never distributed**: the
+YOLOv8 weights are AGPL-3.0 with `redistribution: restricted`, so you materialize your own,
+on your machine, from the blueprint:
+
+```sh
+git clone https://github.com/SKaiNET-developers/SKaiNET-cartridge-blueprints
+cd SKaiNET-cartridge-blueprints
+./gradlew :blueprints:detection-yolov8-iree:materializeCartridge \
+    -Pprofile=profiles/mine.json   # start from profiles/example.cpu-arm64.json
+```
+
+The [blueprint README](https://github.com/SKaiNET-developers/SKaiNET-cartridge-blueprints/tree/main/blueprints/detection-yolov8-iree)
+is the step-by-step guide (fetch → convert → export → compile → measure → sign), including the
+Vulkan and 320-input speed variants this demo's numbers come from.
+
+The blueprint downloads the weights from their publisher (pinned by revision + SHA-256,
+under terms **you** accept — it never accepts a license for you), converts, compiles and
+signs. The pack lands in `blueprints/detection-yolov8-iree/build/cartridge/`.
+
+## Run it
+
+```sh
+./gradlew :androidApp:installDebug
+```
+
+`bundlePacks` copies any packs it finds into the APK assets. By default it looks in a
+sibling `SKaiNET-cartridge-blueprints` checkout (`../../SKaiNET-cartridge-blueprints/
+blueprints/detection-yolov8-iree/build/cartridge`); point it elsewhere with
+`-PpacksDir=/path/to/packs`. **No packs is fine**: a fresh clone builds, installs and runs
+with no weights — the app then shows the materialization steps instead of the camera feed.
+
+Camera permission is asked on first start; the image-picker path works without it.
+
+## Licensing
+
+- App code: MIT, like the rest of this repository.
+- `sk.ainet.cartridge:*` artifacts: MIT, code only.
+- Orbitron display font: SIL OFL-1.1 (`androidApp/src/main/assets/fonts/OFL-1.1.txt`).
+- YOLOv8n weights: **AGPL-3.0** (Ultralytics), `redistribution: restricted`. They are not in
+  this repository, not on Maven, and not in CI. A pack you materialize derives from them:
+  bundling it into your locally built APK is private use, but **distributing** such an APK
+  carries the AGPL obligations with it. Don't publish pack-bundled builds.
+
+## Structure
+
+- `composeApp/` — Compose Multiplatform library. All UI is `commonMain`, written against the
+  `yolo/v1` contract (`YoloEngine`, `Detection`). The platform seams are five
+  `expect` declarations — camera feed, pack discovery, engine opening, image picker, display
+  font — and Android (CameraX frames-as-preview + the IREE module runner) is the first wired
+  platform; a desktop or iOS target needs only its own actuals.
+- `androidApp/` — the APK shell (AGP 9 does not allow a KMP module to be the application):
+  manifest, Orbitron asset, the `bundlePacks` task. No code.

VisionDemo/androidApp/build.gradle.kts

@@ -0,0 +1,58 @@
+// The APK shell for :composeApp — no code of its own, only the manifest, the Orbitron font
+// asset and the bundled packs. AGP 9 requires the application module to be separate from the
+// KMP module (see the library's build script header).
+plugins {
+    alias(libs.plugins.androidApplication)
+}
+
+android {
+    namespace = "sk.ainet.samples.visiondemo.app"
+    compileSdk = libs.versions.android.compileSdk.get().toInt()
+
+    defaultConfig {
+        applicationId = "sk.ainet.samples.visiondemo"
+        minSdk = libs.versions.android.minSdk.get().toInt()
+        targetSdk = libs.versions.android.targetSdk.get().toInt()
+        versionCode = 1
+        versionName = "0.1"
+
+        ndk {
+            // The ABIs the runner AAR (libskainet_iree_module.so) actually carries.
+            abiFilters += listOf("arm64-v8a", "armeabi-v7a")
+        }
+    }
+
+    packaging {
+        // SKaiNET registers kernels through ServiceLoader: merge, never drop, the entries.
+        resources { merges += "META-INF/services/**" }
+    }
+
+    compileOptions {
+        sourceCompatibility = JavaVersion.VERSION_17
+        targetCompatibility = JavaVersion.VERSION_17
+    }
+
+    // Locally materialized packs ride into the APK (assets/packs/<pack_dir>/…) — see
+    // bundlePacks below and the licensing section in the README (the packs derive from
+    // AGPL-3.0 weights; your build, your obligations — they never enter git, and the app
+    // runs without them, showing how to materialize one).
+    sourceSets.getByName("main").assets.srcDir("build/bundled-assets")
+}
+
+dependencies {
+    implementation(projects.composeApp)
+}
+
+// Copies materialized packs into the APK assets. Point -PpacksDir= at a directory holding
+// pack_dirs (descriptor.json + manifest.json + artifacts/); the default is the blueprint's
+// output in a sibling SKaiNET-cartridge-blueprints checkout. Missing directory = no packs
+// bundled = the app still builds and runs (pack-less screen).
+val packsDir: File = providers.gradleProperty("packsDir").orNull
+    ?.let { rootProject.file(it) }
+    ?: rootProject.file("../../SKaiNET-cartridge-blueprints/blueprints/detection-yolov8-iree/build/cartridge")
+
+val bundlePacks = tasks.register<Copy>("bundlePacks") {
+    from(packsDir) { include("detection-yolov8-iree-*/**") }
+    into(layout.buildDirectory.dir("bundled-assets/packs"))
+}
+tasks.named("preBuild") { dependsOn(bundlePacks) }

VisionDemo/androidApp/src/main/AndroidManifest.xml

@@ -0,0 +1,23 @@
+<?xml version="1.0" encoding="utf-8"?>
+<manifest xmlns:android="http://schemas.android.com/apk/res/android">
+
+    <uses-permission android:name="android.permission.CAMERA" />
+    <uses-feature android:name="android.hardware.camera.any" android:required="false" />
+
+    <application
+        android:label="SKaiNET Vision"
+        android:icon="@mipmap/ic_launcher"
+        android:roundIcon="@mipmap/ic_launcher_round"
+        android:theme="@android:style/Theme.Material.NoActionBar"
+        android:allowBackup="false">
+        <activity
+            android:name="sk.ainet.samples.visiondemo.MainActivity"
+            android:exported="true"
+            android:configChanges="orientation|screenSize|keyboardHidden">
+            <intent-filter>
+                <action android:name="android.intent.action.MAIN" />
+                <category android:name="android.intent.category.LAUNCHER" />
+            </intent-filter>
+        </activity>
+    </application>
+</manifest>

VisionDemo/androidApp/src/main/assets/fonts/OFL-1.1.txt

@@ -0,0 +1,43 @@
+SIL OPEN FONT LICENSE
+
+Version 1.1 - 26 February 2007
+
+PREAMBLE
+
+The goals of the Open Font License (OFL) are to stimulate worldwide development of collaborative font projects, to support the font creation efforts of academic and linguistic communities, and to provide a free and open framework in which fonts may be shared and improved in partnership with others.
+
+The OFL allows the licensed fonts to be used, studied, modified and redistributed freely as long as they are not sold by themselves. The fonts, including any derivative works, can be bundled, embedded, redistributed and/or sold with any software provided that any reserved names are not used by derivative works. The fonts and derivatives, however, cannot be released under any other type of license. The requirement for fonts to remain under this license does not apply to any document created using the fonts or their derivatives.
+
+DEFINITIONS
+
+"Font Software" refers to the set of files released by the Copyright Holder(s) under this license and clearly marked as such. This may include source files, build scripts and documentation.
+
+"Reserved Font Name" refers to any names specified as such after the copyright statement(s).
+
+"Original Version" refers to the collection of Font Software components as distributed by the Copyright Holder(s).
+
+"Modified Version" refers to any derivative made by adding to, deleting, or substituting — in part or in whole — any of the components of the Original Version, by changing formats or by porting the Font Software to a new environment.
+
+"Author" refers to any designer, engineer, programmer, technical writer or other person who contributed to the Font Software.
+
+PERMISSION & CONDITIONS
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of the Font Software, to use, study, copy, merge, embed, modify, redistribute, and sell modified and unmodified copies of the Font Software, subject to the following conditions:
+
+1) Neither the Font Software nor any of its individual components, in Original or Modified Versions, may be sold by itself.
+
+2) Original or Modified Versions of the Font Software may be bundled, redistributed and/or sold with any software, provided that each copy contains the above copyright notice and this license. These can be included either as stand-alone text files, human-readable headers or in the appropriate machine-readable metadata fields within text or binary files as long as those fields can be easily viewed by the user.
+
+3) No Modified Version of the Font Software may use the Reserved Font Name(s) unless explicit written permission is granted by the corresponding Copyright Holder. This restriction only applies to the primary font name as presented to the users.
+
+4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font Software shall not be used to promote, endorse or advertise any Modified Version, except to acknowledge the contribution(s) of the Copyright Holder(s) and the Author(s) or with their explicit written permission.
+
+5) The Font Software, modified or unmodified, in part or in whole, must be distributed entirely under this license, and must not be distributed under any other license. The requirement for fonts to remain under this license does not apply to any document created using the Font Software.
+
+TERMINATION
+
+This license becomes null and void if any of the above conditions are not met.
+
+DISCLAIMER
+
+THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE FONT SOFTWARE.

VisionDemo/androidApp/src/main/res/drawable/ic_launcher_monochrome.xml

@@ -0,0 +1,17 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Themed-icon (Android 13) layer: the lens mark reduced to one color — detection
+     brackets, lens ring, trigger dot. Kept inside the 66dp adaptive safe zone. -->
+<vector xmlns:android="http://schemas.android.com/apk/res/android"
+    android:width="108dp" android:height="108dp"
+    android:viewportWidth="108" android:viewportHeight="108">
+    <path
+        android:strokeColor="#FFFFFF" android:strokeWidth="3.5"
+        android:strokeLineCap="round"
+        android:pathData="M31,41 V35 q0,-4 4,-4 h6 M67,31 h6 q4,0 4,4 v6 M77,67 v6 q0,4 -4,4 h-6 M41,77 h-6 q-4,0 -4,-4 v-6" />
+    <path
+        android:strokeColor="#FFFFFF" android:strokeWidth="3.5"
+        android:pathData="M38,54 a16,16 0 1,0 32,0 a16,16 0 1,0 -32,0" />
+    <path
+        android:fillColor="#FFFFFF"
+        android:pathData="M47,54 a7,7 0 1,0 14,0 a7,7 0 1,0 -14,0" />
+</vector>

VisionDemo/androidApp/src/main/res/mipmap-anydpi-v26/ic_launcher.xml

@@ -0,0 +1,6 @@
+<?xml version="1.0" encoding="utf-8"?>
+<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
+    <background android:drawable="@color/ic_launcher_background" />
+    <foreground android:drawable="@mipmap/ic_launcher_foreground" />
+    <monochrome android:drawable="@drawable/ic_launcher_monochrome" />
+</adaptive-icon>

VisionDemo/androidApp/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml

@@ -0,0 +1,6 @@
+<?xml version="1.0" encoding="utf-8"?>
+<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
+    <background android:drawable="@color/ic_launcher_background" />
+    <foreground android:drawable="@mipmap/ic_launcher_foreground" />
+    <monochrome android:drawable="@drawable/ic_launcher_monochrome" />
+</adaptive-icon>

VisionDemo/androidApp/src/main/res/values/colors.xml

@@ -0,0 +1,6 @@
+<?xml version="1.0" encoding="utf-8"?>
+<resources>
+    <!-- Mid-point of the icon art's own background gradient; the foreground layer carries
+         the full gradient, so this only shows during adaptive-icon parallax overshoot. -->
+    <color name="ic_launcher_background">#222325</color>
+</resources>

VisionDemo/build.gradle.kts

@@ -0,0 +1,7 @@
+plugins {
+    alias(libs.plugins.androidApplication) apply false
+    alias(libs.plugins.androidMultiplatformLibrary) apply false
+    alias(libs.plugins.kotlinMultiplatform) apply false
+    alias(libs.plugins.composeMultiplatform) apply false
+    alias(libs.plugins.composeCompiler) apply false
+}