Skip to content

v1.14 — Next Actions Batch: health integration, backup engine, insights, unified search#31

Merged
atomantic merged 67 commits intomainfrom
dev
Feb 27, 2026
Merged

v1.14 — Next Actions Batch: health integration, backup engine, insights, unified search#31
atomantic merged 67 commits intomainfrom
dev

Conversation

@atomantic
Copy link
Owner

Summary

Milestone v1.14 "Next Actions Batch" — 5 phases delivering cross-domain data integration and discovery features for PortOS.

Phase 1: Genome Migration Cleanup

  • Completed meatspace data migration — genome, ClinVar, and epigenetic services read/write from data/meatspace/, fixing stale paths in services, routes, client nav, and docs

Phase 2: Data Backup & Recovery

  • Rsync-based incremental backup engine with SHA-256 manifest, cron scheduler (daily 2 AM), REST API, and BackupWidget on Dashboard with status indicator, manual trigger, snapshot list, and inline restore with dry-run preview

Phase 3: Apple Health Integration

  • Full ingest pipeline for Health Auto Export JSON/XML — Zod-validated, deduplicated, day-partitioned storage
  • Query service with daily aggregation, summary stats, date range, and cross-metric correlation endpoints
  • Health metric cards (HR, HRV, Steps, Sleep) and correlation charts (Activity-Blood, Alcohol-HRV)
  • Import tab with JSON upload UI and XML streaming import

Phase 4: Cross-Domain Insights Engine

  • Rule-based genome-health correlations grouping genetic markers by clinical category and matching blood test values
  • LLM-powered taste-identity themes and cross-domain narrative with caching and diff tracking
  • Tabbed Insights page at /insights/:tab with Overview, Genome-Health, Taste-Identity, and Cross-Domain views

Phase 5: Unified Search

  • Fan-out search across Brain, CoS Memory, Apps, History, and Health metrics via Promise.allSettled
  • Cmd+K/Ctrl+K Spotlight-style overlay with debounced search-as-you-type, grouped results, keyboard nav, and deep-link navigation

Additional

  • Fixed HRV metric name mismatch between Health Auto Export JSON format and query layer
  • System health endpoints relocated from /api/health to /api/system/health

Test Plan

  • Verify genome data loads from /meatspace/genome routes
  • Test backup run/restore cycle via Dashboard widget
  • Import Apple Health JSON and verify metric cards render
  • Confirm Insights tabs load with available data
  • Test Cmd+K search returns results from multiple sources
  • Verify all existing tests pass (cd server && npm test)

Add Goal 10 (Health & Longevity) to GOALS.md covering MeatSpace health
tracking, mortality projections, and meatspace goal management. Define
M45 (Data Backup), M46 (Unified Search), M47 (Push Notifications) in
PLAN.md with full specs. Clean up completed milestones, stale tier
backlog items, and next actions queue.
Full update cycle for any app: git pull (rebase+autostash), npm install
per subdir, npm run setup if present, PM2 restart. Available via REST
endpoint and Socket.IO streaming for real-time progress feedback.
- genome.js: GENOME_DIR now uses PATHS.meatspace
- clinvar.js: GENOME_DIR now uses PATHS.meatspace
- epigenetic.js: DATA_DIR now uses PATHS.meatspace
- genome.js routes: all 18 comment references updated to /api/meatspace/genome
- IdentityTab: 'View Full Genome' button now navigates to /meatspace/genome
- docs/API.md: updated section header and all endpoints to /meatspace/genome
- docs/features/identity-system.md: updated genome.json path and route reference
- Created 01-01-SUMMARY.md documenting all tasks, commits, and deviations
- Updated STATE.md: plan 1/1 complete, metrics recorded, decisions logged
- Updated ROADMAP.md: Phase 1 marked complete, plan checked off
- Updated REQUIREMENTS.md: GEN-01, GEN-02, GEN-03 marked complete
- Created .changelog/v1.14.x.md with genome migration entries
- backup.js: runBackup, generateManifest, listSnapshots, restoreSnapshot, getState, saveState, getNextRunTime
- backupScheduler.js: startBackupScheduler, stopBackupScheduler with eventScheduler cron integration
- rsync-based incremental backup with exit code 24 tolerance
- SHA-256 manifest generation (streaming for files 512KB+)
- State persisted to data/backup/state.json
…nctions

- server/routes/backup.js: GET /status, POST /run, GET /snapshots, POST /restore
- server/lib/validation.js: backupConfigSchema and restoreRequestSchema Zod schemas
- server/index.js: mount /api/backup routes and startBackupScheduler() at startup
- client/src/services/api.js: getBackupStatus, triggerBackup, getBackupSnapshots, restoreBackup
…equirements

- 02-01-SUMMARY.md: full execution summary with deviation notes and commit log
- STATE.md: advanced to phase 2, plan 1 complete, decisions recorded
- ROADMAP.md: phase 2 plan progress updated (1/2 summaries)
- REQUIREMENTS.md: BAK-01, BAK-02, BAK-03 marked complete
- changelog: backup engine entries added to v1.14.x
- Create BackupWidget component with health status (green/yellow/red), manual backup trigger, snapshot list, and inline restore panel
- Health computed by backup age: <25h=healthy, 25-49h=warning, >49h=critical
- Manual trigger calls api.triggerBackup() with toast feedback (no window.alert)
- Snapshot list expandable; each snapshot has inline restore panel
- Restore panel supports optional subdirFilter, dry-run preview before confirming
- Import BackupWidget alphabetically in Dashboard, render in Row 4 beside SystemHealthWidget
- Update changelog v1.14.x.md with BackupWidget entry
- Add 02-02-SUMMARY.md with full task documentation
- Update STATE.md with session position and key decisions
- Update ROADMAP.md: Phase 2 plans 2/2 in SUMMARY count
- Mark requirements BAK-04, BAK-05, BAK-06, BAK-07 complete
- Task 2 checkpoint approved: BackupWidget visible, health indicator green,
  Backup Now triggers toast, snapshots list populated, dry-run preview works
- Updated SUMMARY.md to reflect full plan completion (2/2 tasks)
- Advanced STATE.md: Phase 2 complete, current focus now Phase 3
- ROADMAP.md updated: Phase 2 shows 2/2 plans complete
- Renamed server/routes/health.js to server/routes/systemHealth.js via git mv
- Changed /health/system route to /health/details inside systemHealth.js
- Updated index.js: import systemHealthRoutes, mount at /api/system
- Updated client api.js: checkHealth uses /system/health, getSystemHealth uses /system/health/details
- Updated health.test.js to import systemHealth.js and use /api/system/health path
- Added PATHS.health to fileUtils.js (data/health directory)
- Created appleHealthValidation.js with healthDataPointSchema, healthMetricSchema, healthIngestSchema (passthrough for future fields)
- Created appleHealthIngest.js: extractDateStr (substring, no tz conversion), dedupKey, readDayFile, writeDayFile, mergeIntoDay (dedup by date string), ingestHealthData (main entry)
…nt API functions

- Created appleHealthQuery.js: listDayFiles, getAvailableDateRange, getMetrics, getMetricSummary, getDailyAggregates (per-metric aggregation strategies), getCorrelationData (merges HRV+alcohol+steps+blood)
- Created appleHealth.js routes: POST /ingest, GET /metrics/:name, GET /metrics/:name/daily, GET /range, GET /correlation
- Registered appleHealthRoutes at /api/health in index.js
- Added 5 client API functions to api.js: ingestAppleHealth, getAppleHealthMetrics, getAppleHealthSummary, getAppleHealthRange, getAppleHealthCorrelation
- extractSnippet helper for ~100-char keyword-centered excerpts
- searchBrain: fans out to inbox/people/projects/ideas/links via Promise.allSettled
- searchMemory: BM25 keyword search with memory index summary lookup
- searchApps: case-insensitive match on name/description
- searchHistory: matches targetName and action fields
- searchHealth: static metric keyword matching with display aliases
- fanOutSearch: Promise.allSettled across all 5 adapters, omits empty sources
… API function

- searchQuerySchema added to validation.js (min 2, max 200, trimmed)
- GET /api/search route handler using asyncHandler + validateRequest pattern
- Registered app.use('/api/search', searchRoutes) in server/index.js
- search(q) function added to client api.js using encodeURIComponent
- 05-01-SUMMARY.md created with task commits, decisions, and dependency graph
- STATE.md advanced to phase 5 plan 1 complete, progress 90%
- ROADMAP.md updated with phase 5 plan progress (1/2 complete)
- REQUIREMENTS.md marks SRC-02 and SRC-03 complete
- Changelog entry added for unified search API and client function
- useCmdKSearch.js: global Cmd+K / Ctrl+K toggle hook with keydown listener
- CmdKSearch.jsx: Spotlight-style portal overlay with debounced search-as-you-type,
  results grouped by source with Highlight component for keyword matching,
  arrow-key + Enter + Escape keyboard navigation, body scroll lock, auto-focus
- Import and render <CmdKSearch /> as sibling to main content area
- Component renders via portal so placement doesn't affect layout
- Available on every page via the layout shell
- Client build verified clean
- Create 05-02-SUMMARY.md: Spotlight-style Cmd+K overlay with grouped results, keyboard nav, and portal rendering
- Update STATE.md: phase 5 plan 2 complete, all 5 phases done, milestone 100% progress
- Update REQUIREMENTS.md: mark SRC-01 and SRC-04 complete
- Update 04-02-SUMMARY.md: task count correction (3 tasks, checkpoint approved)
…proved

- Updated 04-02-SUMMARY.md: Task 3 human-verify approved, tasks_completed=3
- Removed duplicate Phase 04 decisions from STATE.md
- Cleaned up stray metrics table row in STATE.md
- Marked 04-01 and 04-02 plans as complete ([x]) in ROADMAP.md
Overview summary cards checked data.available to detect generated content,
but getThemeAnalysis() and getCrossDomainNarrative() only returned the raw
cached object on hit. Adds available:true spread to both cache-hit paths.
Phase 4 (Cross-Domain Insights Engine) verified and marked complete.
11/11 must-haves passed. Gap fix applied for overview available field.
5 phases, 10 plans, 24/24 requirements shipped.
Archives roadmap and requirements to milestones/.
Evolves PROJECT.md with validated requirements.
GSD milestone label was using arbitrary "v1.0" instead of the actual
package.json version (1.14.0). Renamed archive files, updated all
planning references, and removed the incorrect v1.0 git tag.
…aming

Wire ImportTab into MeatSpace routing/nav/tabs at /meatspace/import.
Add Health Auto Export JSON upload section with dedup-aware results display.
Fix heart_rate_variability → heart_rate_variability_sdnn naming mismatch
between JSON ingest and query service so HRV data renders on /meatspace/health.
Copilot AI review requested due to automatic review settings February 27, 2026 01:44
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements milestone v1.14 "Next Actions Batch" which delivers 5 major features across 10 execution plans:

Changes:

  • Phase 1: Genome migration cleanup - Fixed stale digital-twin path references
  • Phase 2: Data backup & recovery - Rsync-based incremental backup with scheduler and dashboard widget
  • Phase 3: Apple Health integration - JSON/XML import pipeline with health metric cards and correlations
  • Phase 4: Cross-domain insights engine - Genome-health correlations and LLM-powered identity themes
  • Phase 5: Unified search - Cmd+K overlay with server-side fan-out across all data sources

Reviewed changes

Copilot reviewed 116 out of 119 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
server/services/genome.js, clinvar.js, epigenetic.js Updated data paths from PATHS.digitalTwin to PATHS.meatspace
server/routes/genome.js Updated 19 route comments to reference /api/meatspace/genome
server/services/backup.js, backupScheduler.js New rsync backup engine with SHA-256 manifests and cron scheduling
server/services/appleHealthIngest.js, appleHealthQuery.js, appleHealthXml.js Complete Apple Health import pipeline with deduplication and day-partitioned storage
server/services/insightsService.js Cross-domain insights with genome-health correlations and LLM theme analysis
server/services/search.js Fan-out search engine across 5 data sources using Promise.allSettled
server/services/appUpdater.js New app update workflow (git pull, npm install, PM2 restart)
server/routes/*.js New routes for backup, appleHealth, insights, search; relocated systemHealth
client/src/components/* New UI components: BackupWidget, Health cards, Insights tabs, CmdKSearch overlay
package.json, server/package.json, client/package.json Added sax, multer, react-diff-viewer-continued dependencies
Files not reviewed (2)
  • client/package-lock.json: Language not supported
  • server/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@atomantic atomantic requested a review from Copilot February 27, 2026 01:50
Prevents "cannot be launched inside another Claude Code session" error
when PortOS server is running inside a Claude Code terminal. Strips the
CLAUDECODE nesting marker from child process env in all 6 spawn sites.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 121 out of 124 changed files in this pull request and generated 7 comments.

Files not reviewed (2)
  • client/package-lock.json: Language not supported
  • server/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

… keyboard nav, validate XML numerics, add stream error handler, fix alcohol 0-as-falsy
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 121 out of 124 changed files in this pull request and generated 15 comments.

Files not reviewed (2)
  • client/package-lock.json: Language not supported
  • server/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…colors, heart rate aggregation, key repeat guard, unused var
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 121 out of 124 changed files in this pull request and generated 1 comment.

Files not reviewed (2)
  • client/package-lock.json: Language not supported
  • server/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@atomantic atomantic merged commit 6d85841 into main Feb 27, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants