diff --git a/.github/workflows/visiondemo-ci.yml b/.github/workflows/visiondemo-ci.yml new file mode 100644 index 0000000..52dc299 --- /dev/null +++ b/.github/workflows/visiondemo-ci.yml @@ -0,0 +1,48 @@ +name: VisionDemo CI + +on: + pull_request: + paths: + - "VisionDemo/**" + - "skainet-ui/**" + - ".github/workflows/visiondemo-ci.yml" + push: + branches: + - develop + paths: + - "VisionDemo/**" + - "skainet-ui/**" + - ".github/workflows/visiondemo-ci.yml" + +concurrency: + group: visiondemo-ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + build: + runs-on: ubuntu-latest + + steps: + - name: Checkout repo + uses: actions/checkout@v4 + + - name: Set up JDK + uses: actions/setup-java@v4 + with: + distribution: temurin + java-version: "21" + + - name: Set up Gradle + uses: gradle/actions/setup-gradle@v4 + + - name: Make gradlew executable + run: chmod +x VisionDemo/gradlew + + # The debug APK assembly proves the whole chain — the published cartridge artifacts + # resolve from Central, the KMP library compiles against the yolo/v1 contract, and the + # JNI runner packaging merges. Deliberately no packs and no weights in CI (they are + # AGPL-materialized locally; bundlePacks is a no-op on a bare checkout), and no + # instrumented tests (needs a device). + - name: Build + working-directory: VisionDemo + run: ./gradlew :androidApp:assembleDebug --console=plain diff --git a/README.md b/README.md index 13ed42a..51c7684 100644 --- a/README.md +++ b/README.md @@ -147,6 +147,20 @@ Confidence: 98.3% 📂 [`MnistJavaDemo/`](MnistJavaDemo/) · command-line · Java 21+ +### 📷 Vision Demo + +Live camera **object detection** from a SKaiNET cartridge: YOLOv8n (COCO, 80 classes) +compiled through IREE runs on the phone's GPU — ~57 ms per frame (≈17 fps) on a Pixel 8a — +with a glass HUD, FPS sparkline, CPU/Vulkan backend switch, capture gallery and detections +viewer. The app consumes the published `sk.ainet.cartridge` artifacts (code only); the +pack it runs is materialized locally from the +[detection blueprint](https://github.com/SKaiNET-developers/SKaiNET-cartridge-blueprints) — +the AGPL-3.0 weights never ship, and a fresh clone builds and runs pack-less. + +📂 [`VisionDemo/`](VisionDemo/) · runs on Android + +--- + ### 🌸 Iris Classifier The classic Iris dataset as the smallest possible **training loop**: a two-layer dense diff --git a/VisionDemo/.gitignore b/VisionDemo/.gitignore new file mode 100644 index 0000000..7d9c0e4 --- /dev/null +++ b/VisionDemo/.gitignore @@ -0,0 +1,18 @@ +*.iml +.kotlin +.gradle +**/build/ +xcuserdata +!src/**/build/ +local.properties +.idea +.DS_Store +captures +.externalNativeBuild +.cxx +*.xcodeproj/* +!*.xcodeproj/project.pbxproj +!*.xcodeproj/xcshareddata/ +!*.xcodeproj/project.xcworkspace/ +!*.xcworkspace/contents.xcworkspacedata +**/xcshareddata/WorkspaceSettings.xcsettings diff --git a/VisionDemo/README.md b/VisionDemo/README.md new file mode 100644 index 0000000..8604dbc --- /dev/null +++ b/VisionDemo/README.md @@ -0,0 +1,75 @@ +# VisionDemo — live object detection from a SKaiNET cartridge + +Live camera object detection on Android: YOLOv8n (COCO, 80 classes) compiled through IREE, +running on the phone's GPU (Vulkan) or CPU, consumed as a **SKaiNET cartridge**. The app is +the published cartridge API end to end — glass HUD with FPS readout and sparkline, backend +switch (CPU / VULKAN / VK·320), capture gallery, a detections viewer and an image-picker +path beside the live feed. + +Verified on a Pixel 8a: **~57 ms per frame (≈17 fps) live** on the 320-input Vulkan pack, +~174 ms on the 640 one; boxes and labels map 1:1 onto the analyzed frames. + +## How it consumes the model + +This example owns **no model code and no weights**. It depends on two small libraries from +Maven Central, published out of +[SKaiNET-cartridge-blueprints](https://github.com/SKaiNET-developers/SKaiNET-cartridge-blueprints): + +```kotlin +implementation("sk.ainet.cartridge:detection-yolov8-iree:0.2.2") // opens, verifies, runs packs +// brings sk.ainet.cartridge.profiles:yolo-v1 (the task-profile contract) transitively +``` + +What actually runs is a **pack** — compiled graphs (`.vmfb`), converted parameters (`.irpa`) +and the native runtime, under a signed manifest — and packs are **never distributed**: the +YOLOv8 weights are AGPL-3.0 with `redistribution: restricted`, so you materialize your own, +on your machine, from the blueprint: + +```sh +git clone https://github.com/SKaiNET-developers/SKaiNET-cartridge-blueprints +cd SKaiNET-cartridge-blueprints +./gradlew :blueprints:detection-yolov8-iree:materializeCartridge \ + -Pprofile=profiles/mine.json # start from profiles/example.cpu-arm64.json +``` + +The [blueprint README](https://github.com/SKaiNET-developers/SKaiNET-cartridge-blueprints/tree/main/blueprints/detection-yolov8-iree) +is the step-by-step guide (fetch → convert → export → compile → measure → sign), including the +Vulkan and 320-input speed variants this demo's numbers come from. + +The blueprint downloads the weights from their publisher (pinned by revision + SHA-256, +under terms **you** accept — it never accepts a license for you), converts, compiles and +signs. The pack lands in `blueprints/detection-yolov8-iree/build/cartridge/`. + +## Run it + +```sh +./gradlew :androidApp:installDebug +``` + +`bundlePacks` copies any packs it finds into the APK assets. By default it looks in a +sibling `SKaiNET-cartridge-blueprints` checkout (`../../SKaiNET-cartridge-blueprints/ +blueprints/detection-yolov8-iree/build/cartridge`); point it elsewhere with +`-PpacksDir=/path/to/packs`. **No packs is fine**: a fresh clone builds, installs and runs +with no weights — the app then shows the materialization steps instead of the camera feed. + +Camera permission is asked on first start; the image-picker path works without it. + +## Licensing + +- App code: MIT, like the rest of this repository. +- `sk.ainet.cartridge:*` artifacts: MIT, code only. +- Orbitron display font: SIL OFL-1.1 (`androidApp/src/main/assets/fonts/OFL-1.1.txt`). +- YOLOv8n weights: **AGPL-3.0** (Ultralytics), `redistribution: restricted`. They are not in + this repository, not on Maven, and not in CI. A pack you materialize derives from them: + bundling it into your locally built APK is private use, but **distributing** such an APK + carries the AGPL obligations with it. Don't publish pack-bundled builds. + +## Structure + +- `composeApp/` — Compose Multiplatform library. All UI is `commonMain`, written against the + `yolo/v1` contract (`YoloEngine`, `Detection`). The platform seams are five + `expect` declarations — camera feed, pack discovery, engine opening, image picker, display + font — and Android (CameraX frames-as-preview + the IREE module runner) is the first wired + platform; a desktop or iOS target needs only its own actuals. +- `androidApp/` — the APK shell (AGP 9 does not allow a KMP module to be the application): + manifest, Orbitron asset, the `bundlePacks` task. No code. diff --git a/VisionDemo/androidApp/build.gradle.kts b/VisionDemo/androidApp/build.gradle.kts new file mode 100644 index 0000000..bc51d19 --- /dev/null +++ b/VisionDemo/androidApp/build.gradle.kts @@ -0,0 +1,58 @@ +// The APK shell for :composeApp — no code of its own, only the manifest, the Orbitron font +// asset and the bundled packs. AGP 9 requires the application module to be separate from the +// KMP module (see the library's build script header). +plugins { + alias(libs.plugins.androidApplication) +} + +android { + namespace = "sk.ainet.samples.visiondemo.app" + compileSdk = libs.versions.android.compileSdk.get().toInt() + + defaultConfig { + applicationId = "sk.ainet.samples.visiondemo" + minSdk = libs.versions.android.minSdk.get().toInt() + targetSdk = libs.versions.android.targetSdk.get().toInt() + versionCode = 1 + versionName = "0.1" + + ndk { + // The ABIs the runner AAR (libskainet_iree_module.so) actually carries. + abiFilters += listOf("arm64-v8a", "armeabi-v7a") + } + } + + packaging { + // SKaiNET registers kernels through ServiceLoader: merge, never drop, the entries. + resources { merges += "META-INF/services/**" } + } + + compileOptions { + sourceCompatibility = JavaVersion.VERSION_17 + targetCompatibility = JavaVersion.VERSION_17 + } + + // Locally materialized packs ride into the APK (assets/packs//…) — see + // bundlePacks below and the licensing section in the README (the packs derive from + // AGPL-3.0 weights; your build, your obligations — they never enter git, and the app + // runs without them, showing how to materialize one). + sourceSets.getByName("main").assets.srcDir("build/bundled-assets") +} + +dependencies { + implementation(projects.composeApp) +} + +// Copies materialized packs into the APK assets. Point -PpacksDir= at a directory holding +// pack_dirs (descriptor.json + manifest.json + artifacts/); the default is the blueprint's +// output in a sibling SKaiNET-cartridge-blueprints checkout. Missing directory = no packs +// bundled = the app still builds and runs (pack-less screen). +val packsDir: File = providers.gradleProperty("packsDir").orNull + ?.let { rootProject.file(it) } + ?: rootProject.file("../../SKaiNET-cartridge-blueprints/blueprints/detection-yolov8-iree/build/cartridge") + +val bundlePacks = tasks.register("bundlePacks") { + from(packsDir) { include("detection-yolov8-iree-*/**") } + into(layout.buildDirectory.dir("bundled-assets/packs")) +} +tasks.named("preBuild") { dependsOn(bundlePacks) } diff --git a/VisionDemo/androidApp/src/main/AndroidManifest.xml b/VisionDemo/androidApp/src/main/AndroidManifest.xml new file mode 100644 index 0000000..0b0f72a --- /dev/null +++ b/VisionDemo/androidApp/src/main/AndroidManifest.xml @@ -0,0 +1,23 @@ + + + + + + + + + + + + + + + diff --git a/VisionDemo/androidApp/src/main/assets/fonts/OFL-1.1.txt b/VisionDemo/androidApp/src/main/assets/fonts/OFL-1.1.txt new file mode 100644 index 0000000..6fe84ee --- /dev/null +++ b/VisionDemo/androidApp/src/main/assets/fonts/OFL-1.1.txt @@ -0,0 +1,43 @@ +SIL OPEN FONT LICENSE + +Version 1.1 - 26 February 2007 + +PREAMBLE + +The goals of the Open Font License (OFL) are to stimulate worldwide development of collaborative font projects, to support the font creation efforts of academic and linguistic communities, and to provide a free and open framework in which fonts may be shared and improved in partnership with others. + +The OFL allows the licensed fonts to be used, studied, modified and redistributed freely as long as they are not sold by themselves. The fonts, including any derivative works, can be bundled, embedded, redistributed and/or sold with any software provided that any reserved names are not used by derivative works. The fonts and derivatives, however, cannot be released under any other type of license. The requirement for fonts to remain under this license does not apply to any document created using the fonts or their derivatives. + +DEFINITIONS + +"Font Software" refers to the set of files released by the Copyright Holder(s) under this license and clearly marked as such. This may include source files, build scripts and documentation. + +"Reserved Font Name" refers to any names specified as such after the copyright statement(s). + +"Original Version" refers to the collection of Font Software components as distributed by the Copyright Holder(s). + +"Modified Version" refers to any derivative made by adding to, deleting, or substituting — in part or in whole — any of the components of the Original Version, by changing formats or by porting the Font Software to a new environment. + +"Author" refers to any designer, engineer, programmer, technical writer or other person who contributed to the Font Software. + +PERMISSION & CONDITIONS + +Permission is hereby granted, free of charge, to any person obtaining a copy of the Font Software, to use, study, copy, merge, embed, modify, redistribute, and sell modified and unmodified copies of the Font Software, subject to the following conditions: + +1) Neither the Font Software nor any of its individual components, in Original or Modified Versions, may be sold by itself. + +2) Original or Modified Versions of the Font Software may be bundled, redistributed and/or sold with any software, provided that each copy contains the above copyright notice and this license. These can be included either as stand-alone text files, human-readable headers or in the appropriate machine-readable metadata fields within text or binary files as long as those fields can be easily viewed by the user. + +3) No Modified Version of the Font Software may use the Reserved Font Name(s) unless explicit written permission is granted by the corresponding Copyright Holder. This restriction only applies to the primary font name as presented to the users. + +4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font Software shall not be used to promote, endorse or advertise any Modified Version, except to acknowledge the contribution(s) of the Copyright Holder(s) and the Author(s) or with their explicit written permission. + +5) The Font Software, modified or unmodified, in part or in whole, must be distributed entirely under this license, and must not be distributed under any other license. The requirement for fonts to remain under this license does not apply to any document created using the Font Software. + +TERMINATION + +This license becomes null and void if any of the above conditions are not met. + +DISCLAIMER + +THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE FONT SOFTWARE. diff --git a/VisionDemo/androidApp/src/main/assets/fonts/Orbitron.ttf b/VisionDemo/androidApp/src/main/assets/fonts/Orbitron.ttf new file mode 100644 index 0000000..1fc54ce Binary files /dev/null and b/VisionDemo/androidApp/src/main/assets/fonts/Orbitron.ttf differ diff --git a/VisionDemo/androidApp/src/main/res/drawable/ic_launcher_monochrome.xml b/VisionDemo/androidApp/src/main/res/drawable/ic_launcher_monochrome.xml new file mode 100644 index 0000000..3729e72 --- /dev/null +++ b/VisionDemo/androidApp/src/main/res/drawable/ic_launcher_monochrome.xml @@ -0,0 +1,17 @@ + + + + + + + diff --git a/VisionDemo/androidApp/src/main/res/mipmap-anydpi-v26/ic_launcher.xml b/VisionDemo/androidApp/src/main/res/mipmap-anydpi-v26/ic_launcher.xml new file mode 100644 index 0000000..69fc468 --- /dev/null +++ b/VisionDemo/androidApp/src/main/res/mipmap-anydpi-v26/ic_launcher.xml @@ -0,0 +1,6 @@ + + + + + + diff --git a/VisionDemo/androidApp/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml b/VisionDemo/androidApp/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml new file mode 100644 index 0000000..69fc468 --- /dev/null +++ b/VisionDemo/androidApp/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml @@ -0,0 +1,6 @@ + + + + + + diff --git a/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher.png b/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher.png new file mode 100644 index 0000000..aa6e4da Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher_foreground.png b/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher_foreground.png new file mode 100644 index 0000000..7d5d134 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher_foreground.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher_round.png b/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher_round.png new file mode 100644 index 0000000..997d061 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-hdpi/ic_launcher_round.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher.png b/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher.png new file mode 100644 index 0000000..4e55247 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher_foreground.png b/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher_foreground.png new file mode 100644 index 0000000..ebbc577 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher_foreground.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher_round.png b/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher_round.png new file mode 100644 index 0000000..35d46f9 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-mdpi/ic_launcher_round.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher.png b/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher.png new file mode 100644 index 0000000..446baa6 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher_foreground.png b/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher_foreground.png new file mode 100644 index 0000000..037469b Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher_foreground.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher_round.png b/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher_round.png new file mode 100644 index 0000000..e006472 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xhdpi/ic_launcher_round.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher.png b/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher.png new file mode 100644 index 0000000..4e2b27b Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.png b/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.png new file mode 100644 index 0000000..3645ee9 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher_foreground.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher_round.png b/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher_round.png new file mode 100644 index 0000000..6cd4503 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xxhdpi/ic_launcher_round.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher.png new file mode 100644 index 0000000..0c29941 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.png b/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.png new file mode 100644 index 0000000..9dfb349 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher_foreground.png differ diff --git a/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png b/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png new file mode 100644 index 0000000..1bcdee4 Binary files /dev/null and b/VisionDemo/androidApp/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png differ diff --git a/VisionDemo/androidApp/src/main/res/values/colors.xml b/VisionDemo/androidApp/src/main/res/values/colors.xml new file mode 100644 index 0000000..824edbf --- /dev/null +++ b/VisionDemo/androidApp/src/main/res/values/colors.xml @@ -0,0 +1,6 @@ + + + + #222325 + diff --git a/VisionDemo/build.gradle.kts b/VisionDemo/build.gradle.kts new file mode 100644 index 0000000..7a58a0c --- /dev/null +++ b/VisionDemo/build.gradle.kts @@ -0,0 +1,7 @@ +plugins { + alias(libs.plugins.androidApplication) apply false + alias(libs.plugins.androidMultiplatformLibrary) apply false + alias(libs.plugins.kotlinMultiplatform) apply false + alias(libs.plugins.composeMultiplatform) apply false + alias(libs.plugins.composeCompiler) apply false +} diff --git a/VisionDemo/composeApp/build.gradle.kts b/VisionDemo/composeApp/build.gradle.kts new file mode 100644 index 0000000..575b64d --- /dev/null +++ b/VisionDemo/composeApp/build.gradle.kts @@ -0,0 +1,67 @@ +import org.jetbrains.kotlin.gradle.dsl.JvmTarget + +// All UI lives in commonMain against the published KMP cartridge API (YoloEngine, Detection); +// the camera feed, pack discovery and engine opening are expect/actual, with Android +// (CameraX + the IREE module runner) as the first wired platform — a desktop or iOS target +// needs only its own actuals. AGP 9 does not allow a KMP module to be the application +// itself, so :androidApp is the (code-free) APK shell. +plugins { + alias(libs.plugins.kotlinMultiplatform) + alias(libs.plugins.androidMultiplatformLibrary) + alias(libs.plugins.composeMultiplatform) + alias(libs.plugins.composeCompiler) +} + +kotlin { + jvmToolchain(21) + + android { + namespace = "sk.ainet.samples.visiondemo" + compileSdk = libs.versions.android.compileSdk.get().toInt() + minSdk = libs.versions.android.minSdk.get().toInt() + compilerOptions { jvmTarget.set(JvmTarget.JVM_17) } + } + + sourceSets { + commonMain.dependencies { + implementation(compose.runtime) + implementation(compose.foundation) + implementation(compose.material3) + implementation(compose.ui) + implementation(libs.kotlinx.coroutines) + implementation("sk.ainet.ui:skainet-ui") + implementation(libs.cartridge.detection.yolov8) + } + androidMain.dependencies { + implementation(compose.materialIconsExtended) + implementation(libs.androidx.activity.compose) + implementation(libs.androidx.camera.camera2) + implementation(libs.androidx.camera.lifecycle) + } + } +} + +// The versions the UI shows ("Powered by SKaiNET …", pack-less screen), taken from the +// version catalog at build time so a dependency bump can never leave a stale number behind. +val generateBuildInfo by tasks.registering { + val skainetVersion = versionCatalogs.named("libs").findVersion("skainet").get().requiredVersion + val cartridgeVersion = versionCatalogs.named("libs").findVersion("cartridge").get().requiredVersion + val outDir = layout.buildDirectory.dir("generated/buildInfo/kotlin") + inputs.property("skainetVersion", skainetVersion) + inputs.property("cartridgeVersion", cartridgeVersion) + outputs.dir(outDir) + doLast { + val file = outDir.get().file("sk/ainet/samples/visiondemo/BuildInfo.kt").asFile + file.parentFile.mkdirs() + file.writeText( + """ + |package sk.ainet.samples.visiondemo + | + |/** Generated from gradle/libs.versions.toml by generateBuildInfo; do not edit. */ + |const val SKAINET_VERSION: String = "$skainetVersion" + |const val CARTRIDGE_VERSION: String = "$cartridgeVersion" + |""".trimMargin(), + ) + } +} +kotlin.sourceSets.getByName("commonMain").kotlin.srcDir(generateBuildInfo) diff --git a/VisionDemo/composeApp/src/androidMain/kotlin/sk/ainet/samples/visiondemo/MainActivity.kt b/VisionDemo/composeApp/src/androidMain/kotlin/sk/ainet/samples/visiondemo/MainActivity.kt new file mode 100644 index 0000000..8000c91 --- /dev/null +++ b/VisionDemo/composeApp/src/androidMain/kotlin/sk/ainet/samples/visiondemo/MainActivity.kt @@ -0,0 +1,51 @@ +package sk.ainet.samples.visiondemo + +import android.Manifest +import android.content.pm.PackageManager +import android.os.Bundle +import androidx.activity.ComponentActivity +import androidx.activity.compose.setContent +import androidx.activity.enableEdgeToEdge +import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.filled.Bolt +import androidx.compose.material.icons.filled.Close +import androidx.compose.material.icons.filled.FlipCameraAndroid +import androidx.compose.material.icons.filled.Memory +import androidx.compose.material.icons.filled.MonitorHeart +import androidx.compose.material.icons.filled.Pause +import androidx.compose.material.icons.filled.PhotoCamera +import androidx.compose.material.icons.filled.PhotoLibrary +import androidx.compose.material.icons.filled.PlayArrow +import androidx.compose.material.icons.filled.Upload + +class MainActivity : ComponentActivity() { + override fun onCreate(savedInstanceState: Bundle?) { + super.onCreate(savedInstanceState) + appContext = applicationContext + enableEdgeToEdge() + if (checkSelfPermission(Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) { + requestPermissions(arrayOf(Manifest.permission.CAMERA), 1) + } + setContent { + App( + AppIcons( + bolt = Icons.Filled.Bolt, + cpu = Icons.Filled.Memory, + activity = Icons.Filled.MonitorHeart, + camera = Icons.Filled.PhotoCamera, + images = Icons.Filled.PhotoLibrary, + flip = Icons.Filled.FlipCameraAndroid, + upload = Icons.Filled.Upload, + close = Icons.Filled.Close, + play = Icons.Filled.PlayArrow, + pause = Icons.Filled.Pause, + ), + ) + } + } + + override fun onRequestPermissionsResult(requestCode: Int, permissions: Array, grantResults: IntArray) { + super.onRequestPermissionsResult(requestCode, permissions, grantResults) + if (requestCode == 1 && grantResults.firstOrNull() == PackageManager.PERMISSION_GRANTED) recreate() + } +} diff --git a/VisionDemo/composeApp/src/androidMain/kotlin/sk/ainet/samples/visiondemo/Platform.android.kt b/VisionDemo/composeApp/src/androidMain/kotlin/sk/ainet/samples/visiondemo/Platform.android.kt new file mode 100644 index 0000000..f450cab --- /dev/null +++ b/VisionDemo/composeApp/src/androidMain/kotlin/sk/ainet/samples/visiondemo/Platform.android.kt @@ -0,0 +1,155 @@ +package sk.ainet.samples.visiondemo + +import android.annotation.SuppressLint +import android.content.Context +import android.graphics.Bitmap +import android.graphics.BitmapFactory +import android.graphics.Matrix +import android.net.Uri +import androidx.activity.compose.rememberLauncherForActivityResult +import androidx.activity.result.contract.ActivityResultContracts +import androidx.camera.core.CameraSelector +import androidx.camera.core.ImageAnalysis +import androidx.camera.lifecycle.ProcessCameraProvider +import androidx.compose.runtime.Composable +import androidx.compose.runtime.DisposableEffect +import androidx.compose.runtime.rememberCoroutineScope +import androidx.compose.runtime.rememberUpdatedState +import androidx.compose.runtime.getValue +import androidx.compose.ui.graphics.ImageBitmap +import androidx.compose.ui.graphics.asImageBitmap +import androidx.compose.ui.platform.LocalContext +import androidx.compose.ui.platform.LocalLifecycleOwner +import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.font.Typeface +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.launch +import kotlinx.coroutines.withContext +import sk.ainet.cartridge.detection.yolov8.PackDir +import sk.ainet.cartridge.detection.yolov8.YoloV8Cartridge +import sk.ainet.cartridge.profile.yolo.v1.YoloEngine +import sk.ainet.cartridge.profile.yolo.v1.YoloImage +import java.io.File +import java.util.concurrent.Executors + +/** Set by [MainActivity] before composition. */ +@SuppressLint("StaticFieldLeak") +internal lateinit var appContext: Context + +actual fun preparePacks(): List { + // Bundled packs (assets/packs//…) unpack into files/packs once; the IREE runtime + // opens files by path, so assets must reach the file system first. adb-loaded packs land in + // the same place (push to /data/local/tmp, then run-as cp — debug builds). + try { + for (name in appContext.assets.list("packs").orEmpty()) { + YoloV8Cartridge.packFromAssets(appContext, "packs/$name") + } + } catch (_: Exception) { + // no bundled packs + } + return File(appContext.filesDir, "packs") + .listFiles { f: File -> File(f, "descriptor.json").isFile } + .orEmpty() + .sortedBy { it.name } + .map { PackRef(it.name, it.absolutePath) } +} + +actual fun openEngine(pack: PackRef): YoloEngine = YoloV8Cartridge.open(PackDir(pack.path)) + +@Composable +actual fun CameraFeed(active: Boolean, front: Boolean, onFrame: (Frame) -> Unit) { + val context = LocalContext.current + val lifecycleOwner = LocalLifecycleOwner.current + val scope = rememberCoroutineScope() + val currentOnFrame by rememberUpdatedState(onFrame) + + DisposableEffect(active, front) { + if (!active) return@DisposableEffect onDispose { } + val executor = Executors.newSingleThreadExecutor() + var provider: ProcessCameraProvider? = null + val future = ProcessCameraProvider.getInstance(context) + future.addListener({ + val p = try { future.get() } catch (_: Exception) { return@addListener } + provider = p + val analysis = ImageAnalysis.Builder() + .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST) + .setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888) + .build() + analysis.setAnalyzer(executor) { proxy -> + val rotation = proxy.imageInfo.rotationDegrees + val bitmap = proxy.toBitmap() + proxy.close() + val matrix = Matrix().apply { + if (rotation != 0) postRotate(rotation.toFloat()) + if (front) postScale(-1f, 1f) // mirror, like a selfie preview + } + val upright = + if (rotation == 0 && !front) bitmap + else Bitmap.createBitmap(bitmap, 0, 0, bitmap.width, bitmap.height, matrix, true) + // Convert for the engine here, on the camera thread — the UI never re-reads pixels. + val frame = Frame(upright.asImageBitmap(), upright.toYolo()) + scope.launch(Dispatchers.Main) { currentOnFrame(frame) } + } + try { + p.unbindAll() + p.bindToLifecycle( + lifecycleOwner, + if (front) CameraSelector.DEFAULT_FRONT_CAMERA else CameraSelector.DEFAULT_BACK_CAMERA, + analysis, + ) + } catch (_: Exception) { + // no camera / permission denied — picker path still works + } + }, context.mainExecutor) + onDispose { + provider?.unbindAll() + executor.shutdown() + } + } +} + +@Composable +actual fun rememberImagePicker(onPicked: (ImageBitmap) -> Unit): () -> Unit { + val context = LocalContext.current + val scope = rememberCoroutineScope() + val launcher = rememberLauncherForActivityResult(ActivityResultContracts.GetContent()) { uri: Uri? -> + if (uri == null) return@rememberLauncherForActivityResult + scope.launch { + val bmp = withContext(Dispatchers.IO) { decodeBounded(context, uri) } + if (bmp != null) onPicked(bmp.asImageBitmap()) + } + } + return { launcher.launch("image/*") } +} + +/** Bounded decode: big camera photos add nothing past the letterbox, only conversion time. */ +private fun decodeBounded(context: Context, uri: Uri, maxEdge: Int = 2048): Bitmap? = try { + val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true } + context.contentResolver.openInputStream(uri)?.use { BitmapFactory.decodeStream(it, null, bounds) } + var sample = 1 + while (maxOf(bounds.outWidth, bounds.outHeight) / (sample * 2) >= maxEdge / 2) sample *= 2 + val opts = BitmapFactory.Options().apply { inSampleSize = sample } + context.contentResolver.openInputStream(uri)?.use { BitmapFactory.decodeStream(it, null, opts) } +} catch (_: Exception) { + null +} + +/** ARGB bitmap → the profile's RGB888 [YoloImage], straight from the pixel array. */ +private fun Bitmap.toYolo(): YoloImage { + val argb = IntArray(width * height) + getPixels(argb, 0, width, 0, 0, width, height) + val rgb = ByteArray(argb.size * 3) + var o = 0 + for (p in argb) { + rgb[o++] = (p shr 16 and 0xFF).toByte() + rgb[o++] = (p shr 8 and 0xFF).toByte() + rgb[o++] = (p and 0xFF).toByte() + } + return YoloImage(rgb, width, height) +} + +actual fun displayFontFamily(): FontFamily = try { + FontFamily(Typeface(android.graphics.Typeface.createFromAsset(appContext.assets, "fonts/Orbitron.ttf"))) +} catch (_: Exception) { + FontFamily.Default +} diff --git a/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/App.kt b/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/App.kt new file mode 100644 index 0000000..e651469 --- /dev/null +++ b/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/App.kt @@ -0,0 +1,723 @@ +package sk.ainet.samples.visiondemo + +import androidx.compose.animation.core.InfiniteRepeatableSpec +import androidx.compose.animation.core.LinearEasing +import androidx.compose.animation.core.RepeatMode +import androidx.compose.animation.core.animateFloat +import androidx.compose.animation.core.rememberInfiniteTransition +import androidx.compose.animation.core.tween +import androidx.compose.foundation.Canvas +import androidx.compose.foundation.Image +import androidx.compose.foundation.background +import androidx.compose.foundation.isSystemInDarkTheme +import androidx.compose.foundation.border +import androidx.compose.foundation.clickable +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.Spacer +import androidx.compose.foundation.layout.aspectRatio +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.height +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.safeDrawingPadding +import androidx.compose.foundation.layout.size +import androidx.compose.foundation.layout.width +import androidx.compose.foundation.lazy.grid.GridCells +import androidx.compose.foundation.lazy.grid.LazyVerticalGrid +import androidx.compose.foundation.lazy.grid.items +import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.shape.CircleShape +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.foundation.verticalScroll +import androidx.compose.material3.Icon +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.Text +import androidx.compose.material3.darkColorScheme +import androidx.compose.material3.lightColorScheme +import androidx.compose.runtime.Composable +import androidx.compose.runtime.CompositionLocalProvider +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.rememberCoroutineScope +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.draw.alpha +import androidx.compose.ui.draw.clip +import androidx.compose.ui.draw.shadow +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.graphics.ImageBitmap +import androidx.compose.ui.graphics.Path +import androidx.compose.ui.graphics.drawscope.Stroke +import androidx.compose.ui.graphics.graphicsLayer +import androidx.compose.ui.graphics.toPixelMap +import androidx.compose.ui.graphics.vector.ImageVector +import androidx.compose.ui.layout.ContentScale +import androidx.compose.ui.text.TextStyle +import androidx.compose.ui.text.drawText +import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.text.rememberTextMeasurer +import androidx.compose.ui.text.style.TextAlign +import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.sp +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.launch +import kotlinx.coroutines.withContext +import sk.ainet.cartridge.profile.yolo.v1.Detection +import sk.ainet.ui.components.PoweredBySKaiNET +import sk.ainet.cartridge.profile.yolo.v1.YoloEngine +import sk.ainet.cartridge.profile.yolo.v1.YoloImage +import sk.ainet.cartridge.profile.yolo.v1.YoloResult + +data class Shot( + val id: Long, + val image: ImageBitmap, + val dets: List, + val ms: Int, + val backend: String, +) + +private enum class Tab { Live, Images } + +@Composable +fun App(icons: AppIcons) { + val dark = isSystemInDarkTheme() + val palette = if (dark) DarkPalette else LightPalette + // Material defaults tuned to the palette — only skainet-ui's components read them; the + // app's own surfaces come from LocalDt. + val scheme = (if (dark) darkColorScheme() else lightColorScheme()).copy( + surface = palette.surface, + onSurfaceVariant = palette.mutedFg, + ) + MaterialTheme(colorScheme = scheme) { + CompositionLocalProvider(LocalDt provides palette) { AppContent(icons) } + } +} + +@Composable +private fun AppContent(icons: AppIcons) { + val dt = LocalDt.current + val display = remember { displayFontFamily() } + val scope = rememberCoroutineScope() + @Suppress("OPT_IN_USAGE") + val serial = remember { Dispatchers.Default.limitedParallelism(1) } + + var packs by remember { mutableStateOf>(emptyList()) } + var scanned by remember { mutableStateOf(false) } + var selected by remember { mutableStateOf(null) } + var engine by remember { mutableStateOf(null) } + var tab by remember { mutableStateOf(Tab.Live) } + var paused by remember { mutableStateOf(false) } + var front by remember { mutableStateOf(false) } + var frame by remember { mutableStateOf(null) } + var dets by remember { mutableStateOf>(emptyList()) } + var inferMs by remember { mutableStateOf(0) } + var history by remember { mutableStateOf>(emptyList()) } + var shots by remember { mutableStateOf>(emptyList()) } + var viewer by remember { mutableStateOf(null) } + var flash by remember { mutableStateOf(false) } + var busy by remember { mutableStateOf(false) } + var note by remember { mutableStateOf(null) } + + fun select(pack: PackRef) { + if (pack == selected) return + selected = pack + history = emptyList() + scope.launch { + note = "opening ${pack.short}…" + withContext(serial) { + engine?.close() + engine = null + engine = openEngine(pack) + } + note = null + } + } + + LaunchedEffect(Unit) { + val found = withContext(Dispatchers.Default) { preparePacks() } + packs = found + scanned = true + (found.firstOrNull { it.isAccelerated } ?: found.firstOrNull())?.let(::select) + } + LaunchedEffect(flash) { if (flash) { kotlinx.coroutines.delay(160); flash = false } } + + fun inferLive(f: Frame) { + val e = engine ?: return + if (busy || paused) return + busy = true + scope.launch { + try { + val (result, ms) = withContext(serial) { + val yolo = f.yolo + val t0 = nowMillis() + val r = e.infer(yolo) + r to (nowMillis() - t0).toInt() + } + if (result is YoloResult.Detections) { + dets = result.boxes + inferMs = ms + history = (history + (1000f / ms.coerceAtLeast(1))).takeLast(40) + } + } catch (ex: Exception) { + note = ex.message + } finally { + busy = false + } + } + } + + val pickImage = rememberImagePicker { image -> + val e = engine + scope.launch { + if (e == null) { note = "no pack open"; return@launch } + note = "running ${selected?.short}…" + val (result, ms) = withContext(serial) { + val t0 = nowMillis() + val r = e.infer(image.toYolo()) + r to (nowMillis() - t0).toInt() + } + note = null + if (result is YoloResult.Detections) { + val shot = Shot(nowMillis(), image, result.boxes, ms, selected?.short ?: "?") + shots = listOf(shot) + shots + viewer = shot + } + } + } + + CameraFeed(active = tab == Tab.Live && packs.isNotEmpty(), front = front) { f -> + frame = f.image + inferLive(f) + } + + Column( + Modifier.fillMaxSize().background(dt.background).safeDrawingPadding(), + ) { + Header(display, icons, packs, selected, ::select) + + Box(Modifier.weight(1f).padding(horizontal = 12.dp)) { + if (tab == Tab.Live) { + if (scanned && packs.isEmpty()) { + PackLessScreen(display) + } else { + LiveView(display, frame, if (paused) emptyList() else dets, inferMs, history, + paused, flash, selected?.short ?: "–", note, dets.size) + } + } else { + Gallery(display, icons, shots, onOpen = { viewer = it }, onPick = pickImage) + } + } + + // Controls + Row( + Modifier.fillMaxWidth().padding(horizontal = 32.dp, vertical = 14.dp), + horizontalArrangement = Arrangement.SpaceBetween, + verticalAlignment = Alignment.CenterVertically, + ) { + RoundBtn(if (paused) icons.play else icons.pause, enabled = tab == Tab.Live) { paused = !paused } + CaptureButton(icons, uploadMode = tab == Tab.Images) { + if (tab == Tab.Images) pickImage() else { + val img = frame ?: return@CaptureButton + shots = listOf(Shot(nowMillis(), img, dets, inferMs, selected?.short ?: "?")) + shots + flash = true + } + } + RoundBtn(icons.flip, enabled = tab == Tab.Live) { front = !front } + } + + // Tabs + Row(Modifier.fillMaxWidth().background(dt.surface)) { + TabBtn(display, icons.camera, "LIVE", tab == Tab.Live, Modifier.weight(1f)) { tab = Tab.Live } + TabBtn(display, icons.images, "IMAGES · ${shots.size}", tab == Tab.Images, Modifier.weight(1f)) { tab = Tab.Images } + } + // The line every sample shows, from the shared design system; the version is generated + // from the catalog (generateBuildInfo), so it is always the real one. + PoweredBySKaiNET(SKAINET_VERSION, Modifier.background(dt.surface)) + } + + viewer?.let { shot -> + Viewer(display, icons, shot) { viewer = null } + } +} + +// ---------- pieces ---------- + +/** What a fresh clone shows: no packs are bundled or downloadable — materialization is yours. */ +@Composable +private fun PackLessScreen(display: FontFamily) { + val dt = LocalDt.current + Column( + Modifier.fillMaxSize().clip(RoundedCornerShape(18.dp)) + .border(1.dp, dt.border, RoundedCornerShape(18.dp)).background(dt.surface) + .padding(24.dp), + horizontalAlignment = Alignment.CenterHorizontally, + verticalArrangement = Arrangement.Center, + ) { + Text("NO PACKS INSTALLED", fontFamily = display, fontWeight = FontWeight.Bold, + fontSize = 14.sp, letterSpacing = 3.sp, color = dt.foreground) + Spacer(Modifier.height(12.dp)) + Text( + "This demo runs a cartridge you materialize yourself — the YOLOv8 weights are " + + "AGPL-3.0 and never ship with the app or its Maven artifacts (cartridge " + + "$CARTRIDGE_VERSION is code only).", + fontSize = 12.sp, color = dt.secondaryFg, textAlign = TextAlign.Center, + ) + Spacer(Modifier.height(16.dp)) + Column( + Modifier.fillMaxWidth().background(dt.card, RoundedCornerShape(12.dp)) + .border(1.dp, dt.border, RoundedCornerShape(12.dp)).padding(16.dp), + verticalArrangement = Arrangement.spacedBy(6.dp), + ) { + for (line in listOf( + "git clone …/SKaiNET-cartridge-blueprints", + "./gradlew materializeCartridge \\", + " -Pprofile=profiles/mine.json", + "# then reinstall this app:", + "./gradlew :androidApp:installDebug", + )) Text(line, fontSize = 11.sp, fontFamily = FontFamily.Monospace, color = dt.mutedFg) + } + Spacer(Modifier.height(12.dp)) + Text("Full steps: VisionDemo/README.md", fontSize = 11.sp, color = dt.mutedFg) + } +} + +@Composable +private fun Header( + display: FontFamily, + icons: AppIcons, + packs: List, + selected: PackRef?, + onSelect: (PackRef) -> Unit, +) { + val dt = LocalDt.current + Row( + Modifier.fillMaxWidth().padding(start = 16.dp, end = 16.dp, top = 10.dp, bottom = 8.dp), + horizontalArrangement = Arrangement.SpaceBetween, + verticalAlignment = Alignment.CenterVertically, + ) { + Row( + Modifier.weight(1f), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.spacedBy(8.dp), + ) { + Box( + Modifier.size(32.dp) + .shadow(10.dp, RoundedCornerShape(8.dp), spotColor = dt.primaryGlow, ambientColor = dt.primaryGlow) + .background(dt.primary, RoundedCornerShape(8.dp)), + contentAlignment = Alignment.Center, + ) { Icon(icons.bolt, null, Modifier.size(16.dp), tint = Color.White) } + Column { + Text("SKaiNET", fontFamily = display, fontWeight = FontWeight.Bold, fontSize = 14.sp, + letterSpacing = 3.sp, color = dt.foreground, maxLines = 1, softWrap = false) + Text("YOLO · DETECT", fontFamily = display, fontSize = 9.sp, letterSpacing = 2.sp, + color = dt.mutedFg, maxLines = 1, softWrap = false) + } + } + BackendSwitch(display, icons, packs, selected, onSelect) + } +} + +@Composable +private fun BackendSwitch( + display: FontFamily, + icons: AppIcons, + packs: List, + selected: PackRef?, + onSelect: (PackRef) -> Unit, +) { + val dt = LocalDt.current + Row( + Modifier.border(1.dp, dt.border, RoundedCornerShape(50)).background(dt.card, RoundedCornerShape(50)).padding(2.dp), + ) { + for (pack in packs) { + val active = pack == selected + Row( + Modifier.clip(RoundedCornerShape(50)) + .background(if (active) dt.primary else Color.Transparent, RoundedCornerShape(50)) + .clickable { onSelect(pack) } + .padding(horizontal = 12.dp, vertical = 6.dp), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.spacedBy(4.dp), + ) { + Icon(if (pack.isAccelerated) icons.activity else icons.cpu, null, Modifier.size(12.dp), + tint = if (active) Color.White else dt.mutedFg) + Text(pack.pillLabel, fontFamily = display, fontSize = 10.sp, + fontWeight = FontWeight.Bold, letterSpacing = 1.sp, maxLines = 1, softWrap = false, + color = if (active) Color.White else dt.mutedFg) + } + } + if (packs.isEmpty()) { + Text("NO PACKS", fontFamily = display, fontSize = 10.sp, letterSpacing = 1.sp, + color = dt.mutedFg, modifier = Modifier.padding(horizontal = 12.dp, vertical = 6.dp)) + } + } +} + +private fun Modifier.glass(dt: Palette, shape: RoundedCornerShape) = + background(dt.scrim, shape).border(1.dp, dt.glassBorder, shape) + +@Composable +private fun LiveView( + display: FontFamily, + frame: ImageBitmap?, + dets: List, + inferMs: Int, + history: List, + paused: Boolean, + flash: Boolean, + backend: String, + note: String?, + objCount: Int, +) { + val dt = LocalDt.current + Box( + Modifier.fillMaxSize().clip(RoundedCornerShape(18.dp)) + .border(1.dp, dt.border, RoundedCornerShape(18.dp)).background(dt.surface), + ) { + frame?.let { + Image(it, null, Modifier.fillMaxSize(), contentScale = ContentScale.Crop) + BoxesOverlay(dets, it.width, it.height, crop = true, Modifier.fillMaxSize()) + } + if (flash) Box(Modifier.fillMaxSize().background(dt.foreground.copy(alpha = 0.7f))) + + // HUD + Row( + Modifier.fillMaxWidth().padding(12.dp), + horizontalArrangement = Arrangement.SpaceBetween, + ) { + Column(Modifier.glass(dt, RoundedCornerShape(12.dp)).padding(horizontal = 12.dp, vertical = 8.dp)) { + Row(verticalAlignment = Alignment.Bottom, horizontalArrangement = Arrangement.spacedBy(4.dp)) { + Text( + if (inferMs > 0) (1000f / inferMs).format1() else "–", + fontFamily = display, fontWeight = FontWeight.Black, fontSize = 28.sp, color = dt.onGlass, + ) + Text("FPS", fontFamily = display, fontSize = 10.sp, letterSpacing = 2.sp, + color = dt.primary, modifier = Modifier.padding(bottom = 4.dp)) + } + Sparkline(history, Modifier.width(96.dp).height(22.dp)) + } + Column( + Modifier.glass(dt, RoundedCornerShape(12.dp)).padding(horizontal = 12.dp, vertical = 8.dp), + verticalArrangement = Arrangement.spacedBy(4.dp), + ) { + Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) { + Stat(display, "INFER", if (inferMs > 0) "$inferMs ms" else "–") + Stat(display, "AVG", if (history.isNotEmpty()) history.average().toFloat().format1() + " fps" else "–") + } + Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) { + Stat(display, "OBJS", "$objCount") + Stat(display, "MODE", if (paused) "paused" else "live") + } + } + } + + // Status strip + Row( + Modifier.align(Alignment.BottomCenter).fillMaxWidth().padding(12.dp) + .glass(dt, RoundedCornerShape(12.dp)).padding(horizontal = 12.dp, vertical = 8.dp), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.spacedBy(8.dp), + ) { + val pulse by rememberInfiniteTransition().animateFloat( + 0.4f, 1f, + InfiniteRepeatableSpec(tween(800, easing = LinearEasing), RepeatMode.Reverse), + ) + Box( + Modifier.size(8.dp).alpha(if (paused) 1f else pulse) + .background(if (paused) dt.mutedFg else dt.primary, CircleShape), + ) + Text( + note ?: "$backend · $inferMs ms · ${summary(dets)}", + color = dt.onGlass, fontSize = 12.sp, fontFamily = FontFamily.Monospace, maxLines = 1, + ) + } + } +} + +@Composable +private fun Stat(display: FontFamily, k: String, v: String) { + val dt = LocalDt.current + Column { + Text(k, fontSize = 9.sp, letterSpacing = 2.sp, color = dt.mutedFg) + Text(v, fontSize = 11.sp, fontWeight = FontWeight.Bold, fontFamily = FontFamily.Monospace, color = dt.onGlass) + } +} + +@Composable +private fun Sparkline(values: List, modifier: Modifier) { + val dt = LocalDt.current + Canvas(modifier) { + if (values.size < 2) return@Canvas + val max = (values.max()) * 1.1f + val min = (values.min()) * 0.9f + val span = (max - min).takeIf { it > 0f } ?: 1f + val path = Path() + values.forEachIndexed { i, v -> + val x = i / 39f * size.width + val y = size.height - (v - min) / span * size.height + if (i == 0) path.moveTo(x, y) else path.lineTo(x, y) + } + drawPath(path, dt.primary, style = Stroke(width = 1.5.dp.toPx())) + } +} + +/** Detections in source-pixel coordinates, mapped by the same fit the image uses. */ +@Composable +private fun BoxesOverlay(dets: List, imgW: Int, imgH: Int, crop: Boolean, modifier: Modifier) { + val dt = LocalDt.current + val measurer = rememberTextMeasurer() + Canvas(modifier) { + if (imgW == 0 || imgH == 0) return@Canvas + val sx = size.width / imgW + val sy = size.height / imgH + val s = if (crop) maxOf(sx, sy) else minOf(sx, sy) + val ox = (size.width - imgW * s) / 2f + val oy = (size.height - imgH * s) / 2f + dets.forEachIndexed { i, d -> + val color = dt.boxColor(d.classId) + val l = ox + d.box.x1 * s + val t = oy + d.box.y1 * s + val w = (d.box.x2 - d.box.x1) * s + val h = (d.box.y2 - d.box.y1) * s + drawRoundRect( + color, + topLeft = androidx.compose.ui.geometry.Offset(l, t), + size = androidx.compose.ui.geometry.Size(w, h), + cornerRadius = androidx.compose.ui.geometry.CornerRadius(6.dp.toPx()), + style = Stroke(2.dp.toPx()), + ) + val label = "${d.label ?: "class-${d.classId}"} ${d.score.format2()}" + val layout = measurer.measure( + label, + TextStyle(color = Color.White, fontSize = 10.sp, fontFamily = FontFamily.Monospace, fontWeight = FontWeight.Bold), + ) + val ly = (t - layout.size.height - 2.dp.toPx()).coerceAtLeast(0f) + val lx = l.coerceIn(0f, (size.width - layout.size.width - 8.dp.toPx()).coerceAtLeast(0f)) + drawRoundRect( + color, + topLeft = androidx.compose.ui.geometry.Offset(lx, ly), + size = androidx.compose.ui.geometry.Size(layout.size.width + 8.dp.toPx(), layout.size.height + 2.dp.toPx()), + cornerRadius = androidx.compose.ui.geometry.CornerRadius(3.dp.toPx()), + ) + drawText(layout, topLeft = androidx.compose.ui.geometry.Offset(lx + 4.dp.toPx(), ly + 1.dp.toPx())) + } + } +} + +@Composable +private fun RoundBtn(icon: ImageVector, enabled: Boolean, onClick: () -> Unit) { + val dt = LocalDt.current + Box( + Modifier.size(48.dp).alpha(if (enabled) 1f else 0.3f) + .border(1.dp, dt.border, CircleShape).background(dt.card, CircleShape) + .clip(CircleShape).clickable(enabled = enabled, onClick = onClick), + contentAlignment = Alignment.Center, + ) { Icon(icon, null, Modifier.size(20.dp), tint = dt.foreground) } +} + +@Composable +private fun CaptureButton(icons: AppIcons, uploadMode: Boolean, onClick: () -> Unit) { + val dt = LocalDt.current + Box( + Modifier.size(72.dp).border(4.dp, dt.foreground.copy(alpha = 0.8f), CircleShape) + .padding(6.dp).clip(CircleShape).clickable(onClick = onClick), + contentAlignment = Alignment.Center, + ) { + Box( + Modifier.fillMaxSize() + .shadow(14.dp, CircleShape, spotColor = dt.primaryGlow, ambientColor = dt.primaryGlow) + .background(dt.primary, CircleShape), + contentAlignment = Alignment.Center, + ) { if (uploadMode) Icon(icons.upload, null, Modifier.size(24.dp), tint = Color.White) } + } +} + +@Composable +private fun TabBtn( + display: FontFamily, + icon: ImageVector, + label: String, + active: Boolean, + modifier: Modifier, + onClick: () -> Unit, +) { + val dt = LocalDt.current + Box(modifier.clickable(onClick = onClick), contentAlignment = Alignment.Center) { + if (active) { + Box( + Modifier.align(Alignment.TopCenter).padding(horizontal = 40.dp).fillMaxWidth().height(2.dp) + .shadow(8.dp, RoundedCornerShape(1.dp), spotColor = dt.primaryGlow) + .background(dt.primary, RoundedCornerShape(1.dp)), + ) + } + Row( + Modifier.padding(vertical = 12.dp), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.spacedBy(8.dp), + ) { + Icon(icon, null, Modifier.size(16.dp), tint = if (active) dt.foreground else dt.mutedFg) + Text(label, fontFamily = display, fontSize = 11.sp, fontWeight = FontWeight.Bold, + letterSpacing = 2.sp, color = if (active) dt.foreground else dt.mutedFg) + } + } +} + +@Composable +private fun Gallery( + display: FontFamily, + icons: AppIcons, + shots: List, + onOpen: (Shot) -> Unit, + onPick: () -> Unit, +) { + val dt = LocalDt.current + LazyVerticalGrid( + columns = GridCells.Fixed(3), + horizontalArrangement = Arrangement.spacedBy(8.dp), + verticalArrangement = Arrangement.spacedBy(8.dp), + ) { + item { + Column( + Modifier.aspectRatio(0.75f).border(1.dp, dt.border, RoundedCornerShape(12.dp)) + .clip(RoundedCornerShape(12.dp)).clickable(onClick = onPick), + horizontalAlignment = Alignment.CenterHorizontally, + verticalArrangement = Arrangement.Center, + ) { + Icon(icons.upload, null, Modifier.size(20.dp), tint = dt.mutedFg) + Spacer(Modifier.height(4.dp)) + Text("Pick image", fontSize = 10.sp, color = dt.mutedFg) + } + } + items(shots, key = { it.id }) { shot -> + Box( + Modifier.aspectRatio(0.75f).clip(RoundedCornerShape(12.dp)) + .border(1.dp, dt.border, RoundedCornerShape(12.dp)) + .clickable { onOpen(shot) }, + ) { + Image(shot.image, null, Modifier.fillMaxSize(), contentScale = ContentScale.Crop) + Text( + "${shot.dets.size} obj · ${shot.ms} ms", + Modifier.align(Alignment.BottomCenter).fillMaxWidth().padding(4.dp) + .glass(dt, RoundedCornerShape(6.dp)).padding(horizontal = 6.dp, vertical = 2.dp), + fontSize = 9.sp, fontFamily = FontFamily.Monospace, color = dt.onGlass, + ) + } + } + } +} + +@Composable +private fun Viewer(display: FontFamily, icons: AppIcons, shot: Shot, onClose: () -> Unit) { + val dt = LocalDt.current + Column(Modifier.fillMaxSize().background(dt.background).safeDrawingPadding()) { + Row( + Modifier.fillMaxWidth().padding(horizontal = 16.dp, vertical = 12.dp), + horizontalArrangement = Arrangement.SpaceBetween, + verticalAlignment = Alignment.CenterVertically, + ) { + Text("DETECTIONS", fontFamily = display, fontSize = 12.sp, fontWeight = FontWeight.Bold, + letterSpacing = 3.sp, color = dt.foreground) + Box( + Modifier.size(36.dp).border(1.dp, dt.border, CircleShape).background(dt.card, CircleShape) + .clip(CircleShape).clickable(onClick = onClose), + contentAlignment = Alignment.Center, + ) { Icon(icons.close, null, Modifier.size(16.dp), tint = dt.foreground) } + } + Column(Modifier.padding(horizontal = 12.dp).verticalScroll(rememberScrollState())) { + Box( + Modifier.fillMaxWidth().aspectRatio(shot.image.width.toFloat() / shot.image.height) + .clip(RoundedCornerShape(18.dp)).border(1.dp, dt.border, RoundedCornerShape(18.dp)), + ) { + Image(shot.image, null, Modifier.fillMaxSize(), contentScale = ContentScale.Fit) + BoxesOverlay(shot.dets, shot.image.width, shot.image.height, crop = false, Modifier.fillMaxSize()) + } + Row(Modifier.padding(top = 12.dp), horizontalArrangement = Arrangement.spacedBy(8.dp)) { + StatCard(display, "LATENCY", "${shot.ms} ms", Modifier.weight(1f)) + StatCard(display, "≈ FPS", (1000f / shot.ms.coerceAtLeast(1)).format1(), Modifier.weight(1f)) + StatCard(display, "BACKEND", if ("vulkan" in shot.backend) "VULKAN" else "CPU", Modifier.weight(1f)) + } + Column(Modifier.padding(top = 12.dp, bottom = 16.dp), verticalArrangement = Arrangement.spacedBy(4.dp)) { + shot.dets.forEachIndexed { i, d -> + Row( + Modifier.fillMaxWidth().background(dt.card, RoundedCornerShape(10.dp)) + .padding(horizontal = 12.dp, vertical = 8.dp), + horizontalArrangement = Arrangement.SpaceBetween, + verticalAlignment = Alignment.CenterVertically, + ) { + Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp)) { + Box(Modifier.size(10.dp).background(dt.boxColor(d.classId), RoundedCornerShape(2.dp))) + Text(d.label ?: "class-${d.classId}", fontSize = 14.sp, color = dt.foreground) + } + Text("${(d.score * 100).toInt()}%", fontSize = 12.sp, fontFamily = FontFamily.Monospace, color = dt.mutedFg) + } + } + } + } + } +} + +@Composable +private fun StatCard(display: FontFamily, k: String, v: String, modifier: Modifier) { + val dt = LocalDt.current + Column( + modifier.border(1.dp, dt.border, RoundedCornerShape(12.dp)) + .background(dt.card, RoundedCornerShape(12.dp)).padding(horizontal = 12.dp, vertical = 8.dp), + ) { + Text(k, fontSize = 9.sp, letterSpacing = 2.sp, color = dt.mutedFg) + Text(v, fontFamily = display, fontSize = 15.sp, fontWeight = FontWeight.Bold, color = dt.foreground) + } +} + +// ---------- helpers ---------- + +/** The icon set, supplied per platform entry point (material-icons-extended on Android). */ +data class AppIcons( + val bolt: ImageVector, + val cpu: ImageVector, + val activity: ImageVector, + val camera: ImageVector, + val images: ImageVector, + val flip: ImageVector, + val upload: ImageVector, + val close: ImageVector, + val play: ImageVector, + val pause: ImageVector, +) + +private fun summary(dets: List): String = + if (dets.isEmpty()) "nothing" else dets.groupingBy { it.label ?: "class-${it.classId}" } + .eachCount().entries.joinToString { "${it.value} ${it.key}" } + +private val appOrigin = kotlin.time.TimeSource.Monotonic.markNow() +internal fun nowMillis(): Long = appOrigin.elapsedNow().inWholeMilliseconds + +private fun Float.format1(): String = ((this * 10).toInt() / 10f).toString() +private fun Float.format2(): String { + val h = (this * 100).toInt() + return "${h / 100}.${(h % 100).toString().padStart(2, '0')}" +} + +/** ARGB pixels → the profile's RGB888 [YoloImage]. */ +internal fun ImageBitmap.toYolo(): YoloImage { + val pm = toPixelMap() + val buf = pm.buffer + val rgb = ByteArray(width * height * 3) + var o = 0 + for (y in 0 until height) { + var i = pm.bufferOffset + y * pm.stride + for (x in 0 until width) { + val p = buf[i++] + rgb[o++] = (p shr 16 and 0xFF).toByte() + rgb[o++] = (p shr 8 and 0xFF).toByte() + rgb[o++] = (p and 0xFF).toByte() + } + } + return YoloImage(rgb, width, height) +} diff --git a/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/Platform.kt b/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/Platform.kt new file mode 100644 index 0000000..fd7ac39 --- /dev/null +++ b/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/Platform.kt @@ -0,0 +1,50 @@ +package sk.ainet.samples.visiondemo + +import androidx.compose.runtime.Composable +import androidx.compose.ui.graphics.ImageBitmap +import androidx.compose.ui.text.font.FontFamily +import sk.ainet.cartridge.profile.yolo.v1.YoloEngine +import sk.ainet.cartridge.profile.yolo.v1.YoloImage + +/** A materialized pack the platform found (bundled assets unpacked, or loaded by hand). */ +data class PackRef(val id: String, val path: String) { + /** detection-yolov8-iree-vulkan-arm64 → "vulkan-arm64" */ + val short: String get() = id.removePrefix("detection-yolov8-iree-") + val isAccelerated: Boolean get() = "vulkan" in id + + /** CPU, VULKAN, VK·320 — the backend pill text (variants shorten so three pills fit). */ + val pillLabel: String + get() { + val res = short.substringAfterLast('-').takeIf { it.all(Char::isDigit) } + return when { + res == null -> if (isAccelerated) "VULKAN" else "CPU" + isAccelerated -> "VK·$res" + else -> "CPU·$res" + } + } +} + +/** Unpacks bundled packs if needed and lists what is available. Call off the UI thread. */ +expect fun preparePacks(): List + +/** Opens a pack as a `yolo/v1` engine on this platform's runner. Call off the UI thread. */ +expect fun openEngine(pack: PackRef): YoloEngine + +/** One upright camera frame: what to draw, and the same pixels already converted for the engine. */ +class Frame(val image: ImageBitmap, val yolo: YoloImage) + +/** + * Feeds upright camera frames while [active]; emits on the UI thread. Rendering is the caller's — + * the frames shown are the frames analyzed, so detection overlays map 1:1, and the [Frame] carries + * the engine-ready pixels so the UI never re-reads the bitmap. No-op where the platform has no + * camera (the picker path still works there). + */ +@Composable +expect fun CameraFeed(active: Boolean, front: Boolean, onFrame: (Frame) -> Unit) + +/** A launcher for the platform image picker; invoke the returned function to open it. */ +@Composable +expect fun rememberImagePicker(onPicked: (ImageBitmap) -> Unit): () -> Unit + +/** Orbitron, the SKaiNET display face (falls back to the platform default). */ +expect fun displayFontFamily(): FontFamily diff --git a/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/Theme.kt b/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/Theme.kt new file mode 100644 index 0000000..4850e91 --- /dev/null +++ b/VisionDemo/composeApp/src/commonMain/kotlin/sk/ainet/samples/visiondemo/Theme.kt @@ -0,0 +1,68 @@ +package sk.ainet.samples.visiondemo + +import androidx.compose.runtime.Immutable +import androidx.compose.runtime.staticCompositionLocalOf +import androidx.compose.ui.graphics.Color + +/** + * The SKaiNET design system of the lovable prototype, converted from its oklch tokens — in a dark + * and a light rendition. The glass HUD that floats over camera frames keeps its dark scrim in both + * themes (text over video needs it), which is what [onGlass] is for. + */ +@Immutable +data class Palette( + val background: Color, + val foreground: Color, + val card: Color, + val surface: Color, + val primary: Color, + val primaryGlow: Color, + val secondaryFg: Color, + val mutedFg: Color, + val border: Color, + val success: Color, + val warning: Color, + val scrim: Color, + val glassBorder: Color, + val onGlass: Color, +) { + /** Box palette: primary red, cyan-blue, green, warning amber, pink. */ + val boxes: List get() = listOf(primary, Color(0xFF00B5F5), success, warning, Color(0xFFE068D8)) + fun boxColor(i: Int) = boxes[((i % boxes.size) + boxes.size) % boxes.size] +} + +val DarkPalette = Palette( + background = Color(0xFF06070A), + foreground = Color(0xFFF2F2F2), + card = Color(0xFF101215), + surface = Color(0xFF0A0C0F), + primary = Color(0xFFDC2626), + primaryGlow = Color(0xFFEF4444), + secondaryFg = Color(0xFFD7D7D7), + mutedFg = Color(0xFF6B727E), + border = Color(0xFF25292F), + success = Color(0xFF4BCB71), + warning = Color(0xFFF0B135), + scrim = Color(0xFF030304).copy(alpha = 0.62f), + glassBorder = Color(0xFFF2F2F2).copy(alpha = 0.10f), + onGlass = Color(0xFFF2F2F2), +) + +val LightPalette = Palette( + background = Color(0xFFF6F6F8), + foreground = Color(0xFF0B0C0F), + card = Color(0xFFFFFFFF), + surface = Color(0xFFECEDF1), + primary = Color(0xFFDC2626), + primaryGlow = Color(0xFFEF4444), + secondaryFg = Color(0xFF2A2C31), + mutedFg = Color(0xFF6B727E), + border = Color(0xFFD9DBE1), + success = Color(0xFF1F9D4D), + warning = Color(0xFFB07B10), + scrim = Color(0xFF030304).copy(alpha = 0.62f), // glass over camera stays dark + glassBorder = Color(0xFFF2F2F2).copy(alpha = 0.14f), + onGlass = Color(0xFFF2F2F2), +) + +val LocalDt = staticCompositionLocalOf { DarkPalette } diff --git a/VisionDemo/docs/icons/ios/AppIcon.appiconset/AppIcon-1024.png b/VisionDemo/docs/icons/ios/AppIcon.appiconset/AppIcon-1024.png new file mode 100644 index 0000000..96d0232 Binary files /dev/null and b/VisionDemo/docs/icons/ios/AppIcon.appiconset/AppIcon-1024.png differ diff --git a/VisionDemo/docs/icons/ios/AppIcon.appiconset/Contents.json b/VisionDemo/docs/icons/ios/AppIcon.appiconset/Contents.json new file mode 100644 index 0000000..d011537 --- /dev/null +++ b/VisionDemo/docs/icons/ios/AppIcon.appiconset/Contents.json @@ -0,0 +1,14 @@ +{ + "images": [ + { + "filename": "AppIcon-1024.png", + "idiom": "universal", + "platform": "ios", + "size": "1024x1024" + } + ], + "info": { + "author": "xcode", + "version": 1 + } +} \ No newline at end of file diff --git a/VisionDemo/docs/icons/playstore-icon-512.png b/VisionDemo/docs/icons/playstore-icon-512.png new file mode 100644 index 0000000..36d8e42 Binary files /dev/null and b/VisionDemo/docs/icons/playstore-icon-512.png differ diff --git a/VisionDemo/gradle.properties b/VisionDemo/gradle.properties new file mode 100644 index 0000000..6cb20de --- /dev/null +++ b/VisionDemo/gradle.properties @@ -0,0 +1,9 @@ +kotlin.code.style=official + +#Gradle +org.gradle.jvmargs=-Xmx6g -XX:MaxMetaspaceSize=1536m +kotlin.daemon.jvmargs=-Xmx4g + +#Android +android.nonTransitiveRClass=true +android.useAndroidX=true diff --git a/VisionDemo/gradle/libs.versions.toml b/VisionDemo/gradle/libs.versions.toml new file mode 100644 index 0000000..a3e2879 --- /dev/null +++ b/VisionDemo/gradle/libs.versions.toml @@ -0,0 +1,40 @@ +[versions] +# Must match skainet-ui's AGP (included build): one AGP version per Gradle invocation. +agp = "9.3.1" +android-compileSdk = "37" +# The cartridge consumer library is built with minSdk 29. +android-minSdk = "29" +#noinspection EditedTargetSdkVersion +android-targetSdk = "37" +androidx-activityCompose = "1.12.2" +camerax = "1.4.2" +# The two cartridge artifacts out of SKaiNET-cartridge-blueprints move together. +cartridge = "0.2.2" +# Compose Multiplatform 1.12 (androidx compose 1.12) requires compileSdk 37. +composeMultiplatform = "1.12.2" +# Matches the Kotlin the cartridge artifacts (and SKaiNET 0.62.x) are compiled with. +kotlin = "2.4.21" +kotlinx-coroutines = "1.11.0" +# The engine under the cartridge runtime — resolved transitively, named here for the +# "Powered by SKaiNET" line (generateBuildInfo in composeApp). +skainet = "0.62.1" + +[libraries] +kotlin-test = { module = "org.jetbrains.kotlin:kotlin-test", version.ref = "kotlin" } +androidx-activity-compose = { module = "androidx.activity:activity-compose", version.ref = "androidx-activityCompose" } +androidx-camera-camera2 = { module = "androidx.camera:camera-camera2", version.ref = "camerax" } +androidx-camera-lifecycle = { module = "androidx.camera:camera-lifecycle", version.ref = "camerax" } +kotlinx-coroutines = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-core", version.ref = "kotlinx-coroutines" } + +# The blueprint's consumer library: opens, verifies and runs materialized packs. It +# api-exposes the yolo/v1 task-profile types (sk.ainet.cartridge.profiles:yolo-v1) the UI +# renders, and brings the IREE module runner + SKaiNET engine transitively. Code only — +# model weights and packs are never on Maven (see the README's licensing section). +cartridge-detection-yolov8 = { module = "sk.ainet.cartridge:detection-yolov8-iree", version.ref = "cartridge" } + +[plugins] +androidApplication = { id = "com.android.application", version.ref = "agp" } +androidMultiplatformLibrary = { id = "com.android.kotlin.multiplatform.library", version.ref = "agp" } +composeMultiplatform = { id = "org.jetbrains.compose", version.ref = "composeMultiplatform" } +composeCompiler = { id = "org.jetbrains.kotlin.plugin.compose", version.ref = "kotlin" } +kotlinMultiplatform = { id = "org.jetbrains.kotlin.multiplatform", version.ref = "kotlin" } diff --git a/VisionDemo/gradle/wrapper/gradle-wrapper.jar b/VisionDemo/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 0000000..eddabd2 Binary files /dev/null and b/VisionDemo/gradle/wrapper/gradle-wrapper.jar differ diff --git a/VisionDemo/gradle/wrapper/gradle-wrapper.properties b/VisionDemo/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 0000000..0ffa37a --- /dev/null +++ b/VisionDemo/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,9 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-9.7.0-bin.zip +distributionSha256Sum=84fbba45c7f4c64abc77460e1c00f541e9f960e3c7ed2538f1ede19eacd873ae +networkTimeout=10000 +retries=0 +retryBackOffMs=500 +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/VisionDemo/gradlew b/VisionDemo/gradlew new file mode 100755 index 0000000..249efbb --- /dev/null +++ b/VisionDemo/gradlew @@ -0,0 +1,248 @@ +#!/bin/sh + +# +# Copyright © 2015 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +############################################################################## +# +# gradlew start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh gradlew +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/3d91ce3b8caaf77ad09f381f43615b715b53f72c/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +# This is normally unused +# shellcheck disable=SC2034 +APP_BASE_NAME=${0##*/} +# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036) +APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Collect all arguments for the java command: +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments, +# and any embedded shellness will be escaped. +# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be +# treated as '${Hostname}' itself on the command line. + +set -- \ + "-Dorg.gradle.appname=$APP_BASE_NAME" \ + -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \ + "$@" + +# Stop when "xargs" is not available. +if ! command -v xargs >/dev/null 2>&1 +then + die "xargs is not available" +fi + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/VisionDemo/gradlew.bat b/VisionDemo/gradlew.bat new file mode 100644 index 0000000..a51ec4f --- /dev/null +++ b/VisionDemo/gradlew.bat @@ -0,0 +1,82 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem +@rem SPDX-License-Identifier: Apache-2.0 +@rem + +@if "%DEBUG%"=="" @echo off +@rem ########################################################################## +@rem +@rem gradlew startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables, and ensure extensions are enabled +setlocal EnableExtensions + +set DIRNAME=%~dp0 +if "%DIRNAME%"=="" set DIRNAME=. +@rem This is normally unused +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if %ERRORLEVEL% equ 0 goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +"%COMSPEC%" /c exit 1 + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +"%COMSPEC%" /c exit 1 + +:execute +@rem Setup the command line + + + +@rem Execute gradlew +@rem endlocal doesn't take effect until after the line is parsed and variables are expanded +@rem which allows us to clear the local environment before executing the java command +endlocal & "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %* & call :exitWithErrorLevel + +:exitWithErrorLevel +@rem Use "%COMSPEC%" /c exit to allow operators to work properly in scripts +"%COMSPEC%" /c exit %ERRORLEVEL% diff --git a/VisionDemo/settings.gradle.kts b/VisionDemo/settings.gradle.kts new file mode 100644 index 0000000..f6aba9e --- /dev/null +++ b/VisionDemo/settings.gradle.kts @@ -0,0 +1,35 @@ +rootProject.name = "VisionDemo" +enableFeaturePreview("TYPESAFE_PROJECT_ACCESSORS") + +pluginManagement { + repositories { + google { + mavenContent { + includeGroupAndSubgroups("androidx") + includeGroupAndSubgroups("com.android") + includeGroupAndSubgroups("com.google") + } + } + mavenCentral() + gradlePluginPortal() + } +} + +dependencyResolutionManagement { + repositories { + google { + mavenContent { + includeGroupAndSubgroups("androidx") + includeGroupAndSubgroups("com.android") + includeGroupAndSubgroups("com.google") + } + } + mavenCentral() + } +} + +// Shared SKaiNET design system (theme + components), same as the sibling examples. +includeBuild("../skainet-ui") + +include(":composeApp") +include(":androidApp")