diff --git a/.changelog/README.md b/.changelog/README.md new file mode 100644 index 0000000..536e414 --- /dev/null +++ b/.changelog/README.md @@ -0,0 +1,196 @@ +# Release Changelogs + +This directory contains **all** release notes for PortOS. Unlike traditional projects that maintain a root `CHANGELOG.md` file, we use version-specific files that evolve with development and automatically archive on release. + +**No root CHANGELOG.md needed** - all changelog content lives in this directory. + +## Structure + +Each minor version series has its own markdown file following the naming convention: + +``` +v{major}.{minor}.x.md +``` + +The "x" is a literal character, not a placeholder - it represents the entire minor version series (e.g., all 0.10.x releases share `v0.10.x.md`). + +Examples: +- `v0.9.x.md` - Used for releases 0.9.1, 0.9.2, 0.9.3, etc. +- `v0.10.x.md` - Used for releases 0.10.1, 0.10.2, 0.10.3, etc. +- `v1.0.x.md` - Used for releases 1.0.1, 1.0.2, 1.0.3, etc. + +Alternatively, you can create a specific version file (e.g., `v0.10.5.md`) which takes precedence over the pattern file. + +## Format + +Each changelog file should follow this structure: + +```markdown +# Release v{major}.{minor}.x - {Descriptive Title} + +Released: YYYY-MM-DD + +## Overview + +A brief summary of the release, highlighting the main theme or most important changes. + +## ๐ŸŽ‰ New Features + +### Feature Category 1 +- Feature description with technical details +- Another feature in this category + +### Feature Category 2 +- More features... + +## ๐Ÿ› Bug Fixes + +### Fix Category +- Description of what was fixed +- Impact and technical details + +## ๐Ÿ”ง Improvements + +### Improvement Category +- What was improved +- Why it matters + +## ๐Ÿ—‘๏ธ Removed + +### Deprecated Features +- What was removed +- Why it was removed + +## ๐Ÿ“ฆ Installation + +\`\`\`bash +git clone https://github.com/atomantic/PortOS.git +cd PortOS +npm run install:all +pm2 start ecosystem.config.cjs +\`\`\` + +## ๐Ÿ”— Full Changelog + +**Full Diff**: https://github.com/atomantic/PortOS/compare/v{prev}...v{major}.{minor}.x +``` + +## Workflow Integration + +The GitHub Actions release workflow (`.github/workflows/release.yml`) automatically: + +### During Release (on main) +1. Checks for an exact version changelog file (e.g., `v0.10.5.md`) +2. If not found, checks for a minor version pattern file (e.g., `v0.10.x.md`) +3. If found, replaces version placeholders with actual release version +4. Creates the GitHub release with the substituted changelog +5. If no changelog found, falls back to generating from git commits + +### After Release (dev branch prep) +1. Checks out dev branch +2. If pattern file exists (e.g., `v0.10.x.md`): + - Renames it to versioned file (e.g., `v0.10.5.md`) using `git mv` to preserve history + - Replaces version placeholders in the renamed file + - Commits the renamed file to dev with `[skip ci]` + - Cherry-picks that commit to main with `[skip ci]` +3. Bumps dev branch to next minor version (e.g., 0.11.0) + +This means: +- You maintain one `v0.10.x.md` file throughout development +- On release, it's renamed to the actual version (preserving git history) +- Dev and main both have the historical record matching the tag +- Git history shows the file evolution from `v0.10.x.md` โ†’ `v0.10.5.md` + +## Creating a New Changelog + +When working on a new minor version series (e.g., starting 0.10.x development): + +1. **Start of Minor Version**: Create the changelog file when dev branch is bumped: + ```bash + # After main release bumps dev to 0.10.0, create the changelog + cp .changelog/v0.9.x.md .changelog/v0.10.x.md + + # Edit the new file with template structure + # Keep the version as "v0.10.x" throughout development + ``` + +2. **During Development**: Update `.changelog/v0.10.x.md` **every time** you add features and fixes + - This is your only changelog - no separate CHANGELOG.md file + - Add entries under appropriate emoji sections (๐ŸŽ‰ Features, ๐Ÿ› Fixes, ๐Ÿ”ง Improvements) + - Keep the version in the file as `v0.10.x` (literal x) + - Don't worry about the final patch number - it will be substituted automatically + +3. **Before Merging to Main**: Final review and updates: + - Ensure all changes are documented in `.changelog/v0.10.x.md` + - Add release date (update "YYYY-MM-DD" to actual date) + - Review and polish the content + - Commit the changelog file + +4. **On Release**: The GitHub Actions workflow will: + - Read `.changelog/v0.10.x.md` + - Replace all instances of `0.10.x` with the actual version (e.g., `0.10.5`) + - Create the GitHub release with the substituted changelog + - Create `.changelog/v0.10.5.md` in dev branch (archived copy) + - Merge that archived file back to main with `[skip ci]` + +5. **After Release**: + - The pattern file `v0.10.x.md` is renamed to `v0.10.5.md` in dev branch + - Main branch receives the same file via cherry-pick + - Both branches now have the versioned file matching the tag + - You'll need to create a new `v0.11.x.md` for the next minor version + +## Best Practices + +### โœ… Do: +- Update the changelog file **as you work** (not just before release) +- Use clear, descriptive section headings +- Group related changes together +- Include technical details where helpful +- Explain the "why" not just the "what" +- Use emoji section headers for visual organization (๐ŸŽ‰ โœจ ๐Ÿ› ๐Ÿ”ง ๐Ÿ—‘๏ธ ๐Ÿ“ฆ) +- Link to relevant documentation or issues +- Include upgrade instructions for breaking changes +- Highlight security improvements +- Keep all changelog content in `.changelog/` directory only + +### โŒ Don't: +- Create a root `CHANGELOG.md` file (all changelogs live in `.changelog/` directory) +- Use vague descriptions like "various improvements" +- Include internal implementation details users don't care about +- Repeat the same information in multiple sections +- Use raw commit messages without context +- Forget to update the release date before merging to main +- Leave placeholder or TODO content +- Change the version from `v0.10.x` to specific patch numbers during development + +## Maintenance + +### Updating Past Releases + +If you need to update a past release's changelog: + +1. Edit the `.changelog/v{version}.md` file +2. Update the GitHub release manually: + ```bash + gh release edit v{version} --notes-file .changelog/v{version}.md + ``` + +### Consistency Check + +Periodically verify that: +- All tagged releases have corresponding changelog files +- Root `CHANGELOG.md` is in sync with `.changelog/` directory +- Release dates match git tag dates +- Links to full diffs are correct + +## Tools + +### View Release on GitHub +```bash +gh release view v{version} +``` + +### Edit Release Notes +```bash +gh release edit v{version} --notes-file .changelog/v{version}.md +``` diff --git a/.changelog/v0.10.x.md b/.changelog/v0.10.x.md new file mode 100644 index 0000000..ac97ef5 --- /dev/null +++ b/.changelog/v0.10.x.md @@ -0,0 +1,250 @@ +# Release v0.10.x - Changelog Documentation System + +Released: 2026-01-XX + +## Overview + +This release introduces a structured changelog documentation system for creating rich, detailed release notes. The system follows best practices from our other projects and integrates seamlessly with GitHub Actions to automatically create professional release descriptions. + +## ๐ŸŽ‰ New Features + +### Changelog Documentation System +- **Structured Release Notes**: Added `.changelog/` directory with `v{major}.{minor}.x.md` naming pattern +- **Comprehensive README**: Created detailed documentation for changelog format and best practices +- **GitHub Actions Integration**: Updated release workflow to automatically use changelog files with version substitution +- **Fallback Support**: Maintains automatic commit-based changelogs when manual changelog is not provided +- **Automatic File Rename**: Pattern files (e.g., `v0.10.x.md`) automatically renamed to versioned files (e.g., `v0.10.5.md`) on release, preserving git history + +### Documentation Updates +- **CLAUDE.md Updates**: Added detailed release changelog process instructions +- **Best Practices Guide**: Documented when and how to create changelog files +- **Format Templates**: Provided clear structure with emoji-based sections for visual organization + +### Chief of Staff - ETA Countdown +- **Agent ETA Display**: Running agents now show remaining time (e.g., "~2m 30s left") based on historical task completion times +- **Overtime Indicator**: When tasks exceed their typical duration, shows "+Xm Xs over estimate" in yellow with a pulse animation +- **Progress Bar Enhancement**: Improved progress bar with clearer "X% complete" and prominent ETA display +- **Visual Polish**: Slightly thicker progress bars and better typography for the countdown display + +### Dashboard - Activity Streak +- **Current Streak Display**: Shows consecutive days of activity with dynamic emoji (fire for 7+ days, lightning for 3+, sparkles for less) +- **Longest Streak Tracking**: Displays your personal best streak alongside current streak +- **Personal Best Badge**: Green "Personal best!" badge appears when you match or beat your longest streak +- **7-Day Activity Bar**: Visual mini-chart showing activity for the past week with green/gray bars +- **Gamification**: Motivates consistent usage through streak tracking and progress visualization + +### PortOS Stack Template - AI Provider Integration +- **AI Toolkit Integration**: New projects created with the PortOS Stack template now include `@portos/ai-toolkit` by default +- **Provider Management UI**: Scaffolded projects include an AI Providers page (`/providers`) for adding, editing, testing, and deleting providers +- **Navigation Integration**: New apps have icons in the sidebar with active state highlighting for Home, AI Providers, and About routes +- **Provider Management API**: `/api/providers` endpoints for managing AI providers (CLI or API-based) +- **Run Tracking**: `/api/runs` endpoints for executing and tracking AI runs +- **Prompt Templates**: `/api/prompts` endpoints for managing reusable prompt templates +- **Automatic Symlink**: Template creates symlinks to PortOS's ai-toolkit package in root, server, and client node_modules + +### Create App Improvements +- **Default Directory Path**: The Create App page (`/apps/create`) now defaults to the PortOS parent directory +- **Folder Browser**: Added a folder picker button to browse and select project directories visually +- **Templates Auto-Default**: Templates page now automatically sets the target directory without requiring folder selection + +### Digital Twin - Identity System +- **Digital Twin Documents**: New `/digital-twin` page for managing identity scaffold documents that define persona, values, and cognitive style +- **Document Management**: View, edit, create, and delete digital twin markdown documents organized by category (core, audio, behavioral, enrichment) +- **Multi-Model Behavioral Testing**: Run 14 behavioral alignment tests against multiple AI providers/models simultaneously with side-by-side comparison +- **Enrichment System**: Guided questionnaire across 14 categories (memories, favorites, values, routines, non-negotiables, decision heuristics, error intolerance, personality assessments, etc.) that generates document content from answers +- **Export Formats**: Export digital twin for use in external LLMs as System Prompt, CLAUDE.md, JSON, or individual files +- **CoS Integration**: Digital twin context automatically injected into Chief of Staff agent prompts when enabled +- **Identity Navigation Group**: Brain and Digital Twin now grouped under "Identity" section in navigation + +### Digital Twin Enhancement: Reliable Identity Scaffolds +- **Completeness Validator**: New API endpoint and UI showing percentage of required identity sections covered (identity, values, communication, decision making, non-negotiables, error intolerance) with actionable suggestions for missing sections +- **Contradiction Detector**: AI-powered analysis to detect inconsistencies and conflicting statements across digital twin documents with severity levels and resolution suggestions +- **Dynamic Test Generation**: Generate behavioral tests automatically based on digital twin content - tests values in action, communication style, non-negotiables, and decision patterns +- **Writing Sample Analysis**: Paste writing samples to extract your authentic voice patterns (sentence structure, vocabulary, formality, tone) and automatically generate a WRITING_STYLE.md document +- **Creation Wizard**: 5-step guided wizard for new users covering identity basics, core values, communication style, decision making, and non-negotiables +- **Document Weighting**: Priority slider (1-10) on each document - higher weighted documents are preserved first when context limits force truncation +- **New Enrichment Categories**: Added non_negotiables, decision_heuristics, error_intolerance, and personality_assessments categories with 3 preset questions each +- **Best Practices Documentation**: Added DIGITAL_TWIN_GUIDE.md with comprehensive guidance on creating effective digital twin documents + +### Digital Twin - Mobile Responsive Design +- **Mobile-First Layout**: Complete responsive redesign of the Digital Twin page and all tabs for mobile devices +- **Adaptive Document Editor**: Documents list and editor use stacked layout on mobile with back navigation +- **Touch-Friendly Controls**: All buttons and interactive elements have 40px+ minimum touch targets for accessibility +- **Responsive Grids**: Category grids, test provider selections, and export options adapt from multi-column to single-column on small screens +- **Horizontal Tab Scrolling**: Tab navigation scrolls horizontally on narrow screens with hidden scrollbar +- **Mobile-Optimized Forms**: Enrichment questionnaire, wizard, and all modals display properly on mobile with appropriate spacing + +### Digital Twin - Personality Assessments +- **New Enrichment Category**: Added `personality_assessments` enrichment category for capturing Myers-Briggs, Big Five (OCEAN), Enneagram, DISC, StrengthsFinder, and other personality type results +- **Preset Questions**: Three guided questions covering MBTI type variations, Big Five trait scores, and other assessment results +- **PERSONALITY.md Target**: Answers enriched into a dedicated PERSONALITY.md core document +- **Support for Type Variance**: Questions acknowledge that people may test differently at different times (e.g., ENTJ/ENTP/INTJ/INTP for those near the middle on E/I and J/P dimensions) + +### Digital Twin Rebranding +- **Soul โ†’ Digital Twin**: Renamed the entire "Soul" feature to "Digital Twin" for clearer terminology +- **Route Changes**: `/soul` routes now at `/digital-twin` +- **API Endpoints**: `/api/soul` endpoints now at `/api/digital-twin` +- **Data Directory**: `data/soul/` renamed to `data/digital-twin/` +- **Backwards Compatibility**: Legacy function names and imports aliased for backwards compatibility + +### Digital Twin - Quantitative Personality Modeling (M34 Phase 1 & 2) +- **Big Five Trait Extraction**: AI-powered analysis extracts OCEAN personality scores (0-1 scale) from digital twin documents with supporting evidence notes +- **Values Hierarchy**: Automatically extracts and ranks core values with conflict detection between values +- **Communication Profile**: Quantifies writing style including formality (1-10), verbosity (1-10), emoji usage, preferred tone, and distinctive markers +- **Confidence Scoring**: Calculates per-dimension confidence scores based on evidence quantity, specificity, and consistency +- **Gap Recommendations**: Identifies low-confidence dimensions and suggests specific enrichment questions to fill gaps +- **PersonalityMap Component**: New radar chart visualization showing Big Five traits with confidence-based coloring (green/yellow/red) +- **ConfidenceGauge Component**: Visual gauge showing overall twin confidence with dimension breakdown and trend indicators +- **GapRecommendations Component**: Prioritized list of suggested enrichments with urgency labels and specific questions +- **Prompt Templates**: New `twin-trait-extractor.md` and `twin-confidence-analyzer.md` prompt templates for AI-powered analysis +- **API Endpoints**: New `/api/digital-twin/traits`, `/api/digital-twin/traits/analyze`, `/api/digital-twin/confidence`, `/api/digital-twin/confidence/calculate`, and `/api/digital-twin/gaps` endpoints +- **Local Fallback**: Heuristic-based confidence calculation when AI provider is unavailable +- **Overview Tab Integration**: New components integrated into Digital Twin Overview tab for at-a-glance personality insights + +### Digital Twin - External Data Integration (M34 Phase 4) +- **Import Tab**: New dedicated tab for importing personal data from external services +- **Goodreads CSV Import**: Parse Goodreads library exports to extract reading history, favorites, and shelf data for personality analysis +- **Spotify JSON Import**: Import Spotify extended streaming history and library data to capture music preferences and listening patterns +- **Letterboxd CSV Import**: Parse Letterboxd exports for film ratings, reviews, and watchlists +- **iCal Import**: Parse calendar files (.ics) to understand routines, priorities, and time allocation patterns +- **AI-Powered Analysis**: Uses `twin-import-analyzer.md` prompt template to extract personality insights from imported data +- **Document Generation**: Analyzed imports can be saved directly as digital twin documents with appropriate categorization +- **Provider Selection**: Choose AI provider/model for analysis with multi-tier support +- **Validation Schemas**: New Zod schemas for import sources and analysis results +- **API Endpoints**: New `/api/digital-twin/import/sources`, `/api/digital-twin/import/analyze`, and `/api/digital-twin/import/save` endpoints + +### AI Provider Usage Limit Handling & Fallback +- **Usage Limit Detection**: Agents now detect provider usage limit errors (e.g., "You've hit your usage limit. Upgrade to Pro or try again in 1 day 1 hour 33 minutes") and mark providers as temporarily unavailable +- **Provider Status Tracking**: New `providerStatus.js` service tracks provider availability, usage limits, and estimated recovery times with persistence across server restarts +- **Automatic Fallback**: When a provider hits usage limits, tasks automatically use configured fallback providers - no manual intervention needed +- **Fallback Provider Configuration**: New "Fallback Provider" field in provider settings allows configuring which provider to use when the primary is unavailable +- **Task-Level Fallback Override**: Tasks can specify `metadata.fallbackProvider` to override provider-level and system defaults - gives fine-grained control per task type +- **Provider Status UI**: Health tab now shows provider availability status with real-time updates via WebSocket - displays unavailable providers, reason, and estimated recovery time +- **Manual Recovery**: "Recover" button allows manually marking a provider as available when you know the limit has reset +- **Socket.IO Events**: Real-time `provider:status:changed` events notify UI of provider status changes +- **API Endpoints**: New `/api/providers/status`, `/api/providers/:id/status`, and `/api/providers/:id/status/recover` endpoints for status management +- **AI Toolkit Integration**: Core rate limit detection, error categorization, and provider status tracking now built into `portos-ai-toolkit` v0.2.0 - PortOS wraps the toolkit service for backwards compatibility + +### CoS Learning Data Reset +- **Reset Skipped Task Types**: Added ability to reset learning data for auto-paused task types that have been stuck due to historical failures (e.g., task types with 5% success after 200+ attempts from a now-fixed bug) +- **Clean Metric Subtraction**: Resetting a task type correctly subtracts its metrics from totals and cleans up associated error patterns, giving it a fresh start +- **UI Reset Button**: Each skipped task type in the Learning tab now shows a "Reset" button with toast notification feedback +- **API Endpoint**: New `POST /api/cos/learning/reset/:taskType` endpoint for programmatic access +- **Test Coverage**: 9 new unit tests covering reset logic, metric subtraction, error pattern cleanup, and edge cases + +### CoS Auto-Rehabilitation for Skipped Task Types +- **Automatic Retry of Fixed Tasks**: Task types that were auto-paused due to poor performance (<30% success rate after 5+ attempts) are now automatically given a fresh chance after a configurable grace period (default: 7 days) +- **Grace Period Configuration**: New `rehabilitationGracePeriodDays` config option allows tuning how long skipped tasks wait before being retried +- **Periodic Rehabilitation Check**: During task evaluation, CoS checks every ~100 evaluations (~2 hours) for task types eligible for rehabilitation +- **Learning Data Reset**: When a task type is rehabilitated, its historical learning data is reset, giving it a clean slate to prove the underlying issue was fixed +- **Status Visibility**: New `getSkippedTaskTypesWithStatus()` function provides detailed info about skipped tasks including days until rehabilitation eligibility + +### CoS Task Prompt Enhancement +- **AI-Powered Prompt Enhancement**: New "Enhance with AI" checkbox in the Add Task form that uses AI to expand brief task descriptions into comprehensive, detailed prompts +- **Actionable Prompts**: Enhanced prompts include specific steps, relevant context, success criteria, and edge cases to watch for +- **Prompt Stage Configuration**: Enhancement now uses the `cos-task-enhance` prompt stage for provider/model selection - configure via Prompt Manager instead of hardcoded defaults +- **Visual Feedback**: Loading spinner and "Enhancing..." state during prompt processing, with clear success/error toasts +- **API Endpoint**: New `POST /api/cos/tasks/enhance` endpoint for programmatic access to prompt enhancement + +## ๐Ÿ”ง Improvements + +### Prompt Manager UX +- **Alphabetical Sorting**: Stages and variables lists are now sorted alphabetically for easier navigation in large prompt collections + +### Release Process +- **Rich Release Descriptions**: GitHub releases now support detailed, categorized changelogs +- **Flexible Workflow**: Developers can choose between detailed changelogs or auto-generated commit logs +- **Consistency**: Standardized release note format across all versions +- **Version Substitution**: Workflow automatically replaces version placeholders with actual release version +- **Git History Preservation**: Pattern files renamed using `git mv` to maintain full file history + +### Developer Experience +- **Clear Guidelines**: Comprehensive documentation in `.changelog/README.md` +- **Easy Maintenance**: Simple process for creating and updating release notes +- **Tool Integration**: Instructions for using `gh` CLI to manage releases + +## ๐Ÿ› Fixes + +### Media Page Debug Logs Cleanup +- **Removed Debug Console Logs**: Cleaned up 10 `console.log` statements from `client/src/pages/Media.jsx` that were left over from AudioContext debugging - improves production code quality + +### CoS Agent Provider/Model Validation +- **Provider Override Support**: Tasks specifying a `provider` in metadata now correctly use that provider instead of always using the active provider +- **Model Compatibility Validation**: When spawning agents, models are now validated against the target provider's model list - prevents invalid combinations like using Claude models with the Codex CLI provider +- **Graceful Fallback**: Invalid model/provider combinations fall back to the appropriate tier model for the actual provider with a warning log + +### CoS Learning-Based Model Selection +- **Adaptive Model Upgrades**: Model selection now consults historical task performance data - task types with <60% success rate are automatically upgraded to heavier models (Opus) +- **Smarter Resource Allocation**: CoS learns from past failures and allocates more capable models to struggling task types, improving overall success rates +- **Transparent Logging**: Model selection logs now include learning-based reasons (e.g., "learning-suggested - task type has 45% success rate") + +### System Task Deletion +- **Task Type Detection**: Deleting system tasks (prefixed with `sys-`) now correctly passes `type=internal` to the API, fixing "Task not found" errors when deleting auto-generated investigation tasks + +## ๐Ÿ—‘๏ธ Removed + +### Dead Code Cleanup +- **Legacy Service Files**: Removed three unused `.old.js` files from `server/services/`: + - `providers.old.js` - superseded by current `providers.js` + - `promptService.old.js` - superseded by current `promptService.js` + - `runner.old.js` - superseded by AI Toolkit's runner service +- These files were unreferenced throughout the codebase and represented ~1,220 lines of dead code + +### DRY Refactoring: Shared File Utilities +- **New `server/lib/fileUtils.js`**: Created shared utility module for directory and path operations + - `ensureDir(path)` - Ensures directory exists, creating recursively if needed + - `ensureDirs(paths)` - Ensures multiple directories exist + - `dataPath(...segments)` - Get path relative to data directory + - `rootPath(...segments)` - Get path relative to project root + - `PATHS` - Centralized path constants (data, cos, brain, digitalTwin, runs, memory, agents, scripts, reports) +- **Reduced Code Duplication**: Refactored 4 services (`apps.js`, `history.js`, `usage.js`, `notifications.js`) to use shared utility, eliminating identical `ensureDir` implementations and reducing boilerplate +- **Pattern for Future Refactoring**: Establishes pattern for remaining 7+ services with similar duplicated directory logic + +### Chief of Staff - Navigation Improvements +- **Sidebar Sub-Navigation**: Chief of Staff sub-pages (Tasks, Agents, Scripts, Schedule, Digest, Learning, Memory, Health, Config) now appear as children in the sidebar navigation under Chief of Staff, making all tabs accessible without scrolling the page +- **Collapsible Agent Panel**: On mobile, the COS Agent avatar panel is now collapsible with a tap toggle header showing state and running status +- **Touch-Friendly Tabs**: Tab navigation buttons have 40px minimum height for better mobile accessibility +- **Scrollable Tab Bar**: Tab bar uses horizontal scroll with `shrink-0` to prevent tab overflow issues +- **Badge Support on Collapsible Sections**: Notification badges now display correctly on collapsible nav sections (both collapsed and expanded states) + +## ๐Ÿ› Bug Fixes + +### Task Enhancer Directory Not Found Error +- **Fixed ENOENT error in task prompt enhancement**: The `taskEnhancer.js` was creating its own runId (`enhance-xxx`) and passing it to `createRun`, but the AI Toolkit's `createRun` ignores passed IDs and generates UUIDs. When `executeApiRun` was then called with the original `enhance-xxx` ID, the directory didn't exist. +- **Solution**: Now uses the `runId` returned from `createRun({ ... })` instead of generating a separate ID + +### Fixed Multi-Line Context Loss in Agent Resume +- **Root Cause**: When resuming agent tasks, the "Additional Instructions" and "Previous Agent Context" provided in the resume form were truncated to just the first line (e.g., only "## Additional Instructions" was saved) +- **The Fix**: Task metadata values containing newlines are now properly escaped (`\n` โ†’ `\\n`) when saving to the TASKS.md file and unescaped when parsing back +- **Impact**: Resume context now correctly passes full multi-line instructions to agents, including previous agent output, error context, and user refinements +- **Test Coverage**: Added new round-trip test to verify multi-line metadata preservation through save/parse cycle + +### Brain Memory Delete Handler +- **Fixed Delete Success Detection**: The MemoryTab delete handler was checking `result !== null` to determine success, but successful API responses might not return a value. Changed to track failures explicitly with a boolean flag, fixing false-positive "Deleted" toasts when delete operations failed + +### Critical: Fixed "Unexpected end of JSON input" Errors +- **Safe JSON Parsing**: Added comprehensive JSON parsing utilities to `server/lib/fileUtils.js`: + - `isValidJSON(str)` - Pre-validates JSON structure before parsing (supports both objects and arrays) + - `safeJSONParse(str, defaultValue)` - Safely parses JSON with fallback on empty/corrupted content + - `readJSONFile(filePath, defaultValue)` - Combines file reading with safe JSON parsing + - `safeJSONLParse(content)` - Safely parses JSONL files, skipping malformed lines + - `readJSONLFile(filePath)` - Reads and parses JSONL files safely +- **Fixed 15+ Services**: Applied safe JSON parsing to all file-reading operations across: + - `history.js`, `memory.js`, `notifications.js`, `usage.js` + - `weeklyDigest.js`, `taskLearning.js`, `appActivity.js`, `taskSchedule.js` + - `brainStorage.js`, `subAgentSpawner.js`, `digital-twin.js`, `streamingDetect.js` + - `routes/scaffold.js`, `routes/detect.js` +- **Root Cause**: Empty or corrupted JSON files would cause `JSON.parse()` to throw "Unexpected end of JSON input", crashing services. Now all JSON file reads use safe parsing with sensible defaults + +## ๐Ÿ“ฆ Installation + +```bash +git clone https://github.com/atomantic/PortOS.git +cd PortOS +npm run install:all +pm2 start ecosystem.config.cjs +``` + +## ๐Ÿ”— Full Changelog + +**Full Diff**: https://github.com/atomantic/PortOS/compare/v0.9.9...v0.10.x diff --git a/.changelog/v0.9.x.md b/.changelog/v0.9.x.md new file mode 100644 index 0000000..fe5764e --- /dev/null +++ b/.changelog/v0.9.x.md @@ -0,0 +1,161 @@ +# Release Notes - v0.9.x + +**Release Date:** YYYY-MM-DD + +## ๐ŸŽ‰ Features + +### AI Toolkit Package (`@portos/ai-toolkit`) +- **Created shared library for AI provider patterns**: New npm workspace package at `packages/ai-toolkit/` + - Server services: Provider management, run execution, prompt templating + - Express routes: REST API for providers, runs, and prompts + - React components: `ProviderDropdown` with theming support + - React hooks: `useProviders` and `useRuns` for state management + - API client: Configurable fetch-based client with error handling + - Zod validation schemas for type safety + - Comprehensive test suite with 6 passing tests + - Full documentation: README, INTEGRATION guide, and inline JSDoc + - Ready for extraction to standalone package or use in other PortOS-style apps + +- **Integrated toolkit into PortOS server**: Replaced local AI services with toolkit + - Created compatibility shims for existing service imports + - Extended toolkit routes with PortOS-specific vision testing endpoints + - Added prompts route wrapper to maintain PortOS API contract + - Added hooks for usage tracking and error event emission + - Backed up original files (`.old.js` suffix) for reference + - Server successfully restarted with toolkit integration + - All existing functionality maintained + +- **Integrated toolkit into FableLoom**: Used npm link to test cross-project compatibility + - Created TypeScript declarations for toolkit services + - TypeScript wrapper maintains type safety + - Both JavaScript (PortOS) and TypeScript (FableLoom) projects now using shared library + +- **Full CRUD for prompt stages**: Added create and delete operations to toolkit + - New `createStage()` and `deleteStage()` methods in prompts service + - `POST /api/prompts` endpoint for creating stages with config and template + - `DELETE /api/prompts/:stage` endpoint for removing stages + - Automatic cleanup of template files on deletion + - Updated TypeScript declarations + +### Prompt Management UI +- **Create/Delete Prompt Stages**: Added stage management UI to `/prompts` page + - "+" button to create new stages with modal form + - Delete button (trash icon) on each stage in the list + - Modal form includes: stage key, display name, description, model selection, returns JSON toggle, template editor + - Real-time validation and error handling + - Removed unused placeholder stages (code-analysis, command-suggestion) + +- **System Stage Protection**: Protect critical prompt stages from accidental deletion + - Visual "System" badge on 9 protected stages in the list + - `GET /api/prompts/:stage/usage` endpoint returns usage information + - Enhanced confirmation dialogs showing what features use each stage + - Backend protection requires `?force=true` query param to delete system stages + - Clear warning messages: "This is a SYSTEM stage used by [features]" + - Protected stages: CoS (4), Brain (3), Memory (1), App Detection (1) + +### Template Creation Enhancement +- **Directory Picker for Template Target Path**: Replaced freeform text input with interactive directory browser on `/templates` page + - Default directory now set to parent of PortOS project (`/Users/antic/github.com/atomantic`) + - Visual folder navigation with breadcrumb-style current path display + - Double-click folders to select or use "Select Current" button + - "Parent" and "Default" navigation buttons for easy directory traversal + +### Dashboard Activity Visualization +- **Hourly Activity Heatmap**: New visualization showing session distribution across all 24 hours + - GitHub-style contribution graph with 5-level intensity colors + - Identifies peak productivity hours with session counts + - Responsive design with condensed hour labels on mobile + - Fully accessible with ARIA labels and hover tooltips + - Displays total sessions tracked for context + +### Chief of Staff Enhancement (M35) +- **BM25 Hybrid Memory Search**: Added BM25 text search algorithm for hybrid memory retrieval + - `server/lib/bm25.js` - BM25 with IDF weighting, inverted index, stemming + - `server/services/memoryBM25.js` - Index management, persistence to `data/cos/memory/bm25-index.json` + - Hybrid search combines BM25 + vector with reciprocal rank fusion for better context retrieval + +- **Event-Driven Scheduling**: Replaced interval-based loops with cron-compatible event scheduler + - `server/services/eventScheduler.js` - Timeout-safe timers (clamps to 2^31-1 ms), cron expressions + - Converted `cos.js` from setInterval to scheduleEvent() calls + - Supports interval, cron, and one-shot event types + +- **Execution Lanes**: Lane-based concurrency control for agent spawning + - `server/services/executionLanes.js` - Critical (1 slot), Standard (2 slots), Background (3 slots) + - Lane acquisition with waitForLane() timeout and queue management + - Integrated into subAgentSpawner.js for controlled parallel execution + +- **Tool State Machine**: Comprehensive lifecycle tracking for tool executions + - `server/services/toolStateMachine.js` - IDLE โ†’ START โ†’ RUNNING โ†’ UPDATE โ†’ END โ†’ ERROR states + - Tracks timing, state history, and metadata per execution + - Enables better error recovery and execution analytics + +- **Thinking Levels**: Dynamic model selection based on task complexity + - `server/services/thinkingLevels.js` - off, minimal, low, medium, high, xhigh levels + - Hierarchy resolution: task โ†’ agent โ†’ provider for thinking level + - Integrated into selectModelForTask() for automatic model upgrades + +- **Error Recovery Strategies**: Structured error analysis and recovery + - `server/services/errorRecovery.js` - retry, escalate, fallback, decompose, defer, investigate strategies + - Analyzes error types (network, rate-limit, auth, validation, etc.) + - Returns actionable recovery recommendations + +- **Agent Run Cache**: TTL-based caching for agent outputs + - `server/services/agentRunCache.js` - 10-minute default TTL, key-based storage + - Memory-efficient with periodic cleanup + - Prevents redundant re-execution of recent tasks + +- **Missions Service**: Long-term goal tracking for proactive task generation + - `server/services/missions.js` - Mission CRUD, sub-tasks, progress tracking + - Generates proactive tasks when user queue is empty + - App-owned missions for autonomous improvement of managed apps + +- **Comprehensive Test Suite**: 18 new test files with 722 total tests + - Unit tests for all new services + - Integration tests for state machine and lane management + - Missions service test coverage for CRUD and task generation + +## ๐Ÿ”ง Improvements + +### Chief of Staff Task Management +- **Separate Active Tasks from Pending**: Task list now clearly distinguishes between pending, active (in_progress), blocked, and completed tasks + - Pending tasks shown in yellow section (waiting to be started) + - Active tasks shown in blue section with animated pulse indicator (currently running) + - Blocked tasks shown in red section (waiting for external factors) + - Completed tasks remain collapsible in green section + - Applies to both User Tasks (TASKS.md) and System Tasks (COS-TASKS.md) + - Makes it immediately clear which tasks are actively being worked on vs queued + +### Architecture +- **Monorepo with npm workspaces**: Converted root package.json to support workspaces + - Workspace packages: `packages/*`, `server`, `client` + - Enables sharing code between main app and reusable packages + - Foundation for extracting more shared libraries + +### API +- Added `GET /api/directories` endpoint for browsing file system directories + - Returns current path, parent path (if available), and list of subdirectories + - Filters out hidden directories (starting with `.`) + - Defaults to parent of PortOS project directory when no path specified + +### UI Components +- New `DirectoryPicker` component in `client/src/components/DirectoryPicker.jsx` + - Dropdown interface with folder icons and navigation + - Real-time directory browsing without typing paths + - Keyboard and mouse navigation support + +## ๐Ÿ› Bug Fixes + +### Chief of Staff - Model Selection +- **Fixed default model selection to respect provider's defaultModel**: Standard tasks now use the provider's configured default model instead of always using the medium tier model + - Changed `selectModelForTask()` to return `provider.defaultModel` for standard tasks instead of `provider.mediumModel` + - Updated fallback logic to handle 'default' tier in addition to 'heavy', 'medium', and 'light' + - This ensures that when a provider (like claude-code) has Opus set as the defaultModel, tasks without explicit model specifications will use Opus instead of Sonnet + - Task-specific model selection (heavy for complex tasks, light for documentation) continues to work as before + +### Chief of Staff - Multi-Provider CLI Support +- **Fixed Codex CLI provider support**: Agent spawner was hardcoded to use Claude CLI arguments regardless of selected provider + - Added `buildCliSpawnConfig()` function for provider-specific CLI invocation + - Codex CLI now correctly uses: `codex exec --model ` + - Claude CLI continues using: `claude --dangerously-skip-permissions --print --model ` + - Updated cos-runner to accept `cliCommand` and `cliArgs` parameters + - Maintains backwards compatibility with legacy `claudePath` parameter diff --git a/.claude/commands/replan.md b/.claude/commands/replan.md new file mode 100644 index 0000000..fb851ac --- /dev/null +++ b/.claude/commands/replan.md @@ -0,0 +1,205 @@ +# Replan Command + +You are tasked with reviewing and updating the PLAN.md file and ALL documentation to keep them clean, current, and action-oriented. You also generate new work items when the backlog is depleted. + +## Your Responsibilities + +### 1. Review PLAN.md Structure +- Read the entire PLAN.md file +- Identify completed milestones (marked with [x]) that have detailed documentation +- Identify sections that should be moved to permanent documentation +- Check if Next Actions section has actionable items remaining + +### 2. Extract Documentation from Completed Work +For each completed milestone with substantial documentation: +- Determine the appropriate docs file (ARCHITECTURE.md, API.md, PM2.md, TROUBLESHOOTING.md, etc.) +- Extract the detailed documentation sections +- Move them to the appropriate docs file with proper formatting +- If creating a new docs file, follow the existing docs structure + +**Files to consider:** +- `docs/ARCHITECTURE.md` - System design, data flow, services architecture +- `docs/API.md` - API endpoints, schemas, WebSocket events +- `docs/PM2.md` - PM2 patterns and process management +- `docs/PORTS.md` - Port allocation and conventions +- `docs/VERSIONING.md` - Version format, release process +- `docs/GITHUB_ACTIONS.md` - CI/CD workflow patterns +- `docs/CONTRIBUTING.md` - Code guidelines, git workflow +- `docs/TROUBLESHOOTING.md` - Common issues and solutions +- `docs/features/*.md` - Individual feature documentation + +### 3. Clean Up PLAN.md +After moving documentation: +- Replace detailed milestone documentation with a brief summary (1-3 sentences) +- Add a reference link to the docs file where details were moved +- Keep the milestone checklist status ([x] for completed) +- Remove redundant or outdated information +- Keep the Quick Reference section up to date + +**Example transformation:** +```markdown +Before: +- [x] M16: Memory System + +### Architecture +- **Memory Service** (`server/services/memory.js`): Core CRUD, search, and lifecycle operations +- **Embeddings Service** (`server/services/memoryEmbeddings.js`): LM Studio integration +[... 50 more lines of detailed docs ...] + +After: +- [x] M16: Memory System - Semantic memory with vector embeddings for CoS agent context. See [Memory System](./docs/features/memory-system.md) +``` + +### 4. Maintain All Documentation Files +Review and update ALL docs files to ensure accuracy: + +**Core Documentation:** +- `docs/ARCHITECTURE.md` - Verify system diagrams match current implementation +- `docs/API.md` - Ensure all endpoints are documented with current schemas +- `docs/PM2.md` - Check PM2 patterns match ecosystem.config.cjs +- `docs/PORTS.md` - Verify port allocations are current +- `docs/VERSIONING.md` - Confirm version process is accurate +- `docs/GITHUB_ACTIONS.md` - Check CI/CD workflows match .github/workflows/ +- `docs/CONTRIBUTING.md` - Verify code guidelines match CLAUDE.md +- `docs/TROUBLESHOOTING.md` - Add any new issues discovered + +**Feature Documentation (`docs/features/`):** +- Cross-reference each feature doc with its implementation +- Update API tables if endpoints changed +- Add new features that lack documentation +- Remove docs for features that were removed + +**Maintenance Tasks:** +1. Run `ls docs/` and `ls docs/features/` to see all doc files +2. For each doc, check if it references current file paths +3. Verify code examples still work +4. Update version numbers if mentioned +5. Ensure internal links between docs are valid + +### 5. Update Documentation Index +- Ensure the Documentation section in PLAN.md lists all docs files +- Add any new docs files you created +- Verify all links are correct + +### 6. Focus on Next Actions +At the end of PLAN.md: +- Add a "## Next Actions" section if it doesn't exist +- List 5-8 concrete next steps based on: + - Incomplete milestones + - Recent git commits + - Areas that need attention + - Documentation gaps + - Test coverage gaps +- Make these action items specific and actionable + +### 7. Generate New Work When Backlog is Depleted +If the Next Actions list is nearly empty (fewer than 3 items) or all items are completed: + +**Discovery Methods:** +1. **Analyze Recent Commits**: Run `git log --oneline -20` to see recent work and identify follow-up tasks +2. **Check Test Coverage**: Look for untested or under-tested code areas +3. **Review TODOs**: Run `grep -r "TODO\|FIXME\|HACK" server/ client/src/ --include="*.js" --include="*.jsx"` to find inline tasks +4. **Audit Dependencies**: Check for outdated packages with `npm outdated` +5. **Security Scan**: Identify potential security improvements +6. **Performance Opportunities**: Look for optimization targets +7. **UX Polish**: Identify UI/UX improvements needed +8. **Documentation Gaps**: Find undocumented features or outdated docs +9. **Code Quality**: Identify refactoring opportunities (duplication, complexity) +10. **Feature Roadmap**: Review COS-GOALS.md for mission-aligned features + +**New Work Categories to Consider:** +- **Technical Debt**: Code cleanup, refactoring, removing deprecated patterns +- **Testing**: Unit tests, integration tests, E2E tests +- **Performance**: Caching, query optimization, bundle size +- **Security**: Input validation, auth hardening, dependency updates +- **Accessibility**: ARIA labels, keyboard navigation, screen reader support +- **Documentation**: API docs, user guides, architecture diagrams +- **DevEx**: Better error messages, logging improvements, dev tooling +- **Features**: New functionality aligned with project goals + +**Output Format for New Items:** +```markdown +## Next Actions + +1. **[Category] Brief Title** - Detailed description of what needs to be done and why +2. **[Category] Brief Title** - Detailed description... +... +``` + +**Prioritization:** +- Security issues โ†’ Critical priority +- Broken functionality โ†’ High priority +- Technical debt โ†’ Medium priority +- Polish/nice-to-have โ†’ Low priority + +### 8. Commit Your Changes +After reorganizing: +- Use `/gitup` to commit changes with a clear message like: + ``` + docs: reorganize PLAN.md and update documentation + + - Moved M## documentation to docs/features/ + - Updated docs/API.md with new endpoints + - Updated PLAN.md to focus on next actions + - Generated new work items + ``` + +## Guidelines + +- **Be thorough**: Read all completed milestones and assess documentation value +- **Be surgical**: Only move substantial documentation (>20 lines), keep brief summaries in PLAN +- **Be organized**: Group related content in docs files with clear headings +- **Be consistent**: Match the style and format of existing docs files +- **Be helpful**: Make it easy to find information by adding clear references +- **Be proactive**: Always ensure there are 5+ actionable next items +- **Be comprehensive**: Review ALL docs, not just PLAN.md + +## Example Output Structure + +After running `/replan`, the PLAN.md should have: +```markdown +# Port OS - Implementation Plan + +## Quick Reference +[... existing quick reference ...] + +### Milestones +- [x] M0-M15: Core features complete - See [ARCHITECTURE.md](./docs/ARCHITECTURE.md) +- [x] M16: Memory System - See [Memory System](./docs/features/memory-system.md) +- [x] M17-M32: Advanced features - See respective docs +- [ ] M33: Next feature... + +### Documentation +- [Architecture Overview](./docs/ARCHITECTURE.md) +- [API Reference](./docs/API.md) +- [Memory System](./docs/features/memory-system.md) +- [Brain System](./docs/features/brain-system.md) +- [...more docs...] + +## Next Actions + +1. **[Feature] Complete M7 Templates** - Implement template management UI and app scaffolding +2. **[Testing] Add Soul Service Tests** - Cover edge cases in soul document management +3. **[Security] Dependency Audit** - Run npm audit and update vulnerable packages +4. **[Docs] Update API.md** - Add missing /api/soul/* endpoints +5. **[Performance] Optimize Memory Search** - Profile and improve BM25 index performance +6. **[DevEx] Improve Error Messages** - Add context to validation errors in routes +7. **[Feature] Memory Consolidation** - Implement automatic memory deduplication +8. **[Polish] Mobile Navigation** - Fix responsive layout issues on smaller screens +``` + +## Notes + +- Don't delete information - move it to appropriate docs files +- Keep API endpoint tables consolidated in API.md +- Keep architectural diagrams and data flow in ARCHITECTURE.md +- Create feature-specific docs in docs/features/ for complex systems +- Preserve all historical information but organize it better +- Update CLAUDE.md if any commands or conventions changed +- Always leave the project with 5+ actionable Next Actions items +- Review ALL docs files each time, not just those related to recent work +- Check for broken internal links between docs +- Ensure code examples in docs still match current implementation +- Update version references if they've changed +- Add new discoveries from TODOs/FIXMEs to Next Actions +- Cross-reference COS-GOALS.md for mission-aligned work generation diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 307b3b6..ff0d803 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -28,16 +28,8 @@ jobs: node-version: ${{ matrix.node-version }} cache: 'npm' - - name: Install root dependencies - run: npm ci - - - name: Install client dependencies - working-directory: ./client - run: npm ci - - - name: Install server dependencies - working-directory: ./server - run: npm ci + - name: Install all dependencies + run: npm run install:all - name: Run server tests working-directory: ./server diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 26ee0cc..0c8b36a 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -46,11 +46,42 @@ jobs: id: changelog if: steps.tag-check.outputs.exists == 'false' run: | - # Get the previous tag or use initial commit - PREV_TAG=$(git describe --tags --abbrev=0 2>/dev/null || git rev-list --max-parents=0 HEAD) + VERSION="${{ steps.package-version.outputs.version }}" + MAJOR_MINOR=$(echo $VERSION | cut -d. -f1-2) + + # Try exact version first (e.g., v0.10.5.md), then minor.x pattern (e.g., v0.10.x.md) + CHANGELOG_FILE_EXACT=".changelog/v${VERSION}.md" + CHANGELOG_FILE_PATTERN=".changelog/v${MAJOR_MINOR}.x.md" + + if [ -f "$CHANGELOG_FILE_EXACT" ]; then + # Use exact version changelog + CHANGELOG=$(cat "$CHANGELOG_FILE_EXACT") + elif [ -f "$CHANGELOG_FILE_PATTERN" ]; then + # Use minor version pattern changelog (e.g., v0.10.x.md) + CHANGELOG=$(cat "$CHANGELOG_FILE_PATTERN") + # Replace version placeholder with actual version + CHANGELOG=$(echo "$CHANGELOG" | sed "s/v${MAJOR_MINOR}.x/v${VERSION}/g" | sed "s/${MAJOR_MINOR}.x/${VERSION}/g") + else + # Fallback: Generate changelog from commits (exclude [skip ci] commits) + PREV_TAG=$(git describe --tags --abbrev=0 2>/dev/null || git rev-list --max-parents=0 HEAD) + COMMIT_LOG=$(git log $PREV_TAG..HEAD --pretty=format:"- %s" --no-merges | grep -v "\[skip ci\]" | head -50) + + # Create a basic changelog structure + CHANGELOG="# Release v${VERSION} + + ## Changes - # Generate changelog from commits (exclude [skip ci] commits) - CHANGELOG=$(git log $PREV_TAG..HEAD --pretty=format:"- %s" --no-merges | grep -v "\[skip ci\]" | head -50) + ${COMMIT_LOG} + + ## Installation + + \`\`\`bash + git clone https://github.com/atomantic/PortOS.git + cd PortOS + npm run install:all + pm2 start ecosystem.config.cjs + \`\`\`" + fi # Handle multiline output echo "changelog<> $GITHUB_OUTPUT @@ -63,19 +94,7 @@ jobs: with: tag_name: v${{ steps.package-version.outputs.version }} name: v${{ steps.package-version.outputs.version }} - body: | - ## Changes - - ${{ steps.changelog.outputs.changelog }} - - ## Installation - - ```bash - git clone https://github.com/atomantic/PortOS.git - cd PortOS - npm run install:all - pm2 start ecosystem.config.cjs - ``` + body: ${{ steps.changelog.outputs.changelog }} draft: false prerelease: false env: @@ -84,12 +103,11 @@ jobs: - name: Prep dev branch for next release if: steps.tag-check.outputs.exists == 'false' run: | - # Get current version + # Get current version (the one we just released) CURRENT_VERSION=${{ steps.package-version.outputs.version }} - - # Split into parts MAJOR=$(echo $CURRENT_VERSION | cut -d. -f1) MINOR=$(echo $CURRENT_VERSION | cut -d. -f2) + MAJOR_MINOR="$MAJOR.$MINOR" # Increment minor, reset patch to 0 NEW_MINOR=$((MINOR + 1)) @@ -99,12 +117,39 @@ jobs: git fetch origin dev git checkout dev - # Update package.json files + # Rename and version the changelog file + PATTERN_FILE=".changelog/v${MAJOR_MINOR}.x.md" + VERSIONED_FILE=".changelog/v${CURRENT_VERSION}.md" + + if [ -f "$PATTERN_FILE" ]; then + # Rename the file (preserves git history) + git mv "$PATTERN_FILE" "$VERSIONED_FILE" + + # Replace version placeholders in the renamed file + sed -i.bak "s/v${MAJOR_MINOR}\.x/v${CURRENT_VERSION}/g; s/${MAJOR_MINOR}\.x/${CURRENT_VERSION}/g" "$VERSIONED_FILE" + rm "${VERSIONED_FILE}.bak" + + # Commit the renamed and updated changelog + git add "$VERSIONED_FILE" + git commit -m "docs: archive changelog for v${CURRENT_VERSION} [skip ci]" + + # Capture the changelog commit hash before switching branches + CHANGELOG_COMMIT=$(git rev-parse HEAD) + git push origin dev + + # Merge changelog back to main (without triggering CI) + git checkout main + git cherry-pick "$CHANGELOG_COMMIT" + git push origin main + git checkout dev + fi + + # Update package.json files for next version npm version $NEW_VERSION --no-git-tag-version cd client && npm version $NEW_VERSION --no-git-tag-version && cd .. cd server && npm version $NEW_VERSION --no-git-tag-version && cd .. - # Commit and push + # Commit and push version bump git add package.json package-lock.json client/package.json server/package.json git commit -m "build: prep v$NEW_VERSION for next release [skip ci]" git push origin dev diff --git a/CLAUDE.md b/CLAUDE.md index 1e3a6f0..7911890 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -48,6 +48,17 @@ See `docs/PORTS.md` for the full port allocation guide. ### Data Flow Client โ†’ HTTP/WebSocket โ†’ Routes (validate) โ†’ Services (logic) โ†’ JSON files/PM2 +### AI Toolkit (`portos-ai-toolkit`) + +PortOS depends on `portos-ai-toolkit` as an npm module for AI provider management, run tracking, and prompt templates. The toolkit is a separate project located at `../portos-ai-toolkit` and published to npm. + +**Key points:** +- Provider configuration (models, tiers, fallbacks) is managed by the toolkit's `providers.js` +- PortOS extends toolkit routes in `server/routes/providers.js` for vision testing and provider status +- When adding new provider fields (e.g., `fallbackProvider`, `lightModel`), update the toolkit's `createProvider()` function +- The toolkit uses spread in `updateProvider()` so existing providers preserve custom fields, but `createProvider()` has an explicit field list +- After updating the toolkit, run `npm update portos-ai-toolkit` in PortOS to pull changes + ## Code Conventions - **No try/catch** - errors bubble to centralized middleware @@ -79,6 +90,57 @@ port-error: #ef4444 - **main**: Production releases only - PR `dev โ†’ main` creates tagged release and preps next version - **Use `/gitup` to push** - The dev branch receives auto version bump commits from CI. Always use `git pull --rebase --autostash && git push` (or `/gitup`) instead of plain `git push`. +- Update `.changelog/v{major}.{minor}.x.md` when making changes (see Release Changelog Process below) - commit code after each feature or bug fix See `docs/VERSIONING.md` for details. + +## Release Changelog Process + +All release notes are maintained in `.changelog/v{major}.{minor}.x.md` files (e.g., `.changelog/v0.10.x.md`). Each minor version series has a single changelog file that accumulates changes throughout development. + +### Starting a New Minor Version + +When `dev` is bumped to a new minor version (e.g., 0.10.0), create a new changelog file: + +```bash +# Copy previous version as template +cp .changelog/v0.9.x.md .changelog/v0.10.x.md +# Edit to clear content but keep structure +# Keep version as "v0.10.x" (literal x, not a placeholder) +``` + +### During Development + +**Always update `.changelog/v0.10.x.md`** when you make changes: +- Add entries under appropriate emoji sections (๐ŸŽ‰ Features, ๐Ÿ› Fixes, ๐Ÿ”ง Improvements, ๐Ÿ—‘๏ธ Removed) +- Keep the version as `v0.10.x` throughout development (don't change it to 0.10.1, 0.10.2, etc.) +- Group related changes together for clarity +- Explain the "why" not just the "what" + +### Before Releasing to Main + +Final review before merging `dev โ†’ main`: +- Ensure all changes are documented in `.changelog/v0.10.x.md` +- Add the release date (update "YYYY-MM-DD" to actual date) +- Polish descriptions for clarity +- Commit the changelog + +### On Release (Automated) + +When you merge to `main`, the GitHub Actions workflow automatically: +1. Reads `.changelog/v0.10.x.md` +2. Replaces all instances of `0.10.x` with actual version (e.g., `0.10.5`) +3. Creates the GitHub release with substituted changelog +4. Checks out dev branch +5. Renames `v0.10.x.md` โ†’ `v0.10.5.md` using `git mv` (preserves git history) +6. Commits the renamed file to dev: `"docs: archive changelog for v0.10.5 [skip ci]"` +7. Cherry-picks that commit to main +8. Bumps dev to next minor version (e.g., 0.11.0) + +**Result:** +- Both `dev` and `main` have `.changelog/v0.10.5.md` matching the tagged release +- Git history shows: `v0.10.x.md` โ†’ `v0.10.5.md` (rename) +- You create `.changelog/v0.11.x.md` to start the next development cycle + +See `.changelog/README.md` for detailed format and best practices. diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..7b82f23 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2025 Adam Eivy (@antic|@atomantic) + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/PLAN.md b/PLAN.md index 7be1d15..6788621 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1,7 +1,5 @@ # Port OS - Implementation Plan -See full plan at: `~/.claude/plans/mutable-inventing-eagle.md` - ## Quick Reference ### Tech Stack @@ -25,39 +23,55 @@ pm2 start ecosystem.config.cjs pm2 logs ``` -### Milestones -- [x] M0: Bootstrap (ports, basic UI shell) -- [x] M1: App registry + grid -- [x] M2: PM2 integration (start/stop/restart/status) -- [x] M3: Log viewer (Socket.IO streaming) -- [x] M4: App wizard (register existing + create new) -- [x] M5: AI providers + headless executor -- [x] M6: Dev tools (history, command runner) -- [ ] M7: App templates (create from templates, template management) -- [x] M8: Prompt manager (customizable AI prompts, variables, stages) -- [x] M9: Streaming import detection (websocket updates, progressive discovery) -- [x] M10: Enhanced DevTools (provider/model selection, screenshot upload, git status, usage metrics) -- [x] M11: AI Agents Page (process detection, kill ability, colorful UI, expandable details) -- [x] M12: History Improvements (expandable entries, runtime/output capture, full command display) -- [x] M13: Autofixer Integration (PM2 crash detection, Claude CLI auto-fix, session history UI) -- [x] M14: Chief of Staff (autonomous agent manager, TASKS.md, system health, self-improvement) -- [x] M15: Graceful Error Handling (error normalization, auto-fix integration, Socket.IO events) -- [~] M16: Memory System (semantic memory, LM Studio embeddings, auto-extraction, agent context injection) -- [x] M17: PM2 Ecosystem Config Enhancement (per-process port detection, CDP_PORT support, refresh button) -- [x] M18: PM2 Standardization (LLM-powered config refactoring, import integration, button trigger) -- [x] M19: CoS Agent Runner (isolated PM2 process for agent spawning, prevents orphaned processes) -- [x] M20: AI Provider Error Handling (error extraction, categorization, CoS investigation tasks) -- [x] M21: Usage Metrics Integration (usage tracking for all AI runs, mobile responsive design) -- [x] M22: Orphan Auto-Retry (automatic retry for orphaned agents, investigation task on max retries) -- [x] M23: Self-Improvement System (automated UI/security/code quality analysis with Playwright and Opus) -- [x] M24: Goal-Driven Proactive Mode (COS-GOALS.md mission file, always-working behavior, expanded task types) -- [x] M25: Task Learning System (completion tracking, success rate analysis, model effectiveness, recommendations) -- [x] M26: Scheduled Scripts (cron scheduling, agent triggers, command allowlist, run history) -- [x] M28: Weekly Digest UI (visual digest tab with insights, accomplishments, week-over-week comparisons) -- [x] M29: Comprehensive App Improvement (self-improvement operations extended to managed apps, 10 analysis types, rotation system) -- [x] M30: Configurable Task Intervals (per-task-type scheduling: daily, weekly, once, on-demand; schedule UI tab; execution history tracking) +--- + +## Milestones + +### Completed + +- [x] **M0-M3**: Bootstrap, app registry, PM2 integration, log viewer - Core infrastructure +- [x] **M4**: App Wizard - Register existing apps or create from templates. See [App Wizard](./docs/features/app-wizard.md) +- [x] **M5**: AI Providers - Multi-provider AI execution with headless Claude CLI +- [x] **M6**: Dev Tools - Command runner with history and execution tracking +- [x] **M8**: Prompt Manager - Customizable AI prompts with variables and stages. See [Prompt Manager](./docs/features/prompt-manager.md) +- [x] **M9**: Streaming Import - Real-time websocket updates during app detection +- [x] **M10**: Enhanced DevTools - Provider/model selection, screenshots, git status, usage metrics +- [x] **M11**: AI Agents Page - Process detection and management with colorful UI +- [x] **M12**: History Improvements - Expandable entries with runtime/output capture +- [x] **M13**: Autofixer - Autonomous crash detection and repair. See [Autofixer](./docs/features/autofixer.md) +- [x] **M14**: Chief of Staff - Autonomous agent manager with task orchestration. See [Chief of Staff](./docs/features/chief-of-staff.md) +- [x] **M15**: Error Handling - Graceful error handling with auto-fix. See [Error Handling](./docs/features/error-handling.md) +- [x] **M16**: Memory System - Semantic memory with LLM classification. See [Memory System](./docs/features/memory-system.md) +- [x] **M17**: PM2 Config Enhancement - Per-process port detection and CDP_PORT support +- [x] **M18**: PM2 Standardization - LLM-powered config refactoring +- [x] **M19**: CoS Agent Runner - Isolated PM2 process for agent spawning. See [CoS Agent Runner](./docs/features/cos-agent-runner.md) +- [x] **M20**: AI Error Handling - Enhanced error extraction and CoS integration +- [x] **M21**: Usage Metrics - Comprehensive AI usage tracking and mobile UI +- [x] **M22**: Orphan Auto-Retry - Automatic retry for orphaned agents +- [x] **M23**: Self-Improvement - Automated UI/security/code analysis with Playwright +- [x] **M24**: Goal-Driven Mode - COS-GOALS.md mission file and always-working behavior +- [x] **M25**: Task Learning - Completion tracking and success rate analysis +- [x] **M26**: Scheduled Scripts - Cron-based automation with agent triggering +- [x] **M27**: CoS Capability Enhancements - Dependency updates, performance tracking, learning insights +- [x] **M28**: Weekly Digest UI - Visual digest with insights and comparisons +- [x] **M29**: App Improvement - Comprehensive analysis extended to managed apps +- [x] **M30**: Configurable Intervals - Per-task-type scheduling (daily, weekly, once, on-demand) +- [x] **M31**: LLM Memory Classification - Intelligent memory extraction with quality filtering +- [x] **M32**: Brain System - Second-brain capture and classification. See [Brain System](./docs/features/brain-system.md) +- [x] **M33**: Soul System - Digital twin identity scaffold management. See [Soul System](./docs/features/soul-system.md) +- [x] **M34 P1-P2,P4**: Digital Twin - Quantitative personality modeling and confidence scoring. See [Digital Twin](./docs/features/digital-twin.md) +- [x] **M35**: Chief of Staff Enhancement - Proactive autonomous agent with hybrid memory, missions, LM Studio, thinking levels. See [CoS Enhancement](./docs/features/cos-enhancement.md) + +### Planned + +- [ ] **M7**: App Templates - Template management and app scaffolding from templates +- [ ] **M34 P3,P5-P7**: Digital Twin - Behavioral feedback loop, multi-modal capture, advanced testing, personas -### Documentation +--- + +## Documentation + +### Architecture & Guides - [Architecture Overview](./docs/ARCHITECTURE.md) - System design, data flow - [API Reference](./docs/API.md) - REST endpoints, WebSocket events - [Contributing Guide](./docs/CONTRIBUTING.md) - Code guidelines, git workflow @@ -67,1578 +81,97 @@ pm2 logs - [GitHub Actions](./docs/GITHUB_ACTIONS.md) - CI/CD workflow patterns - [Troubleshooting](./docs/TROUBLESHOOTING.md) - Common issues and solutions -### Error Handling +### Feature Documentation +- [App Wizard](./docs/features/app-wizard.md) - Register apps and create from templates +- [Autofixer](./docs/features/autofixer.md) - Autonomous crash detection and repair +- [Brain System](./docs/features/brain-system.md) - Second-brain capture and classification +- [Chief of Staff](./docs/features/chief-of-staff.md) - Autonomous agent orchestration +- [CoS Agent Runner](./docs/features/cos-agent-runner.md) - Isolated agent process management +- [CoS Enhancement](./docs/features/cos-enhancement.md) - M35 hybrid memory, missions, thinking levels +- [Digital Twin](./docs/features/digital-twin.md) - Quantitative personality modeling +- [Error Handling](./docs/features/error-handling.md) - Graceful error handling with auto-fix +- [Memory System](./docs/features/memory-system.md) - Semantic memory with LLM classification +- [Prompt Manager](./docs/features/prompt-manager.md) - Customizable AI prompts +- [Soul System](./docs/features/soul-system.md) - Digital twin identity scaffold + +--- + +## Next Actions + +Based on recent work and incomplete milestones: + +1. **Complete M7: App Templates** - Implement template management UI and app scaffolding from templates +2. **Digital Twin P3: Behavioral Feedback Loop** - Add "sounds like me" response validation and adaptive weighting +3. **Vision API Polish** - Continue refining LM Studio vision integration based on test results +4. **Memory Consolidation** - Implement automatic memory consolidation for similar memories +5. **Documentation Updates** - Keep ARCHITECTURE.md and API.md current with M35 features + +--- + +## Error Handling Summary + The server implements comprehensive error handling: - **asyncHandler**: All routes wrapped with error handler that catches uncaught errors - **ServerError**: Custom error class with status, code, severity, and context - **Socket.IO Events**: Errors broadcast to UI via `error:occurred` event - **Process Handlers**: Unhandled rejections and uncaught exceptions emit socket events - **Logging**: Errors logged with emoji prefixes, no server crashes -- See `server/lib/errorHandler.js` and `server/services/socket.js` +- See [Error Handling](./docs/features/error-handling.md) for details --- -## M4: App Wizard (Detailed) - -The wizard supports two modes: - -### Mode 1: Register Existing App -For apps already running on the system (any path, any user): - -**Steps:** -1. **Basic Info**: Name, description, icon -2. **Location**: Repo path (file picker or manual entry) -3. **Ports**: UI port, API port (can auto-detect from running processes) -4. **Process Config**: - - Start command(s) - - PM2 process name(s) - - Env file location -5. **Confirm & Register** +## Security Audit (2026-01-08) -**Features:** -- Detect running processes on specified ports -- Validate repo path exists -- Optional: import existing PM2 process into registry -- No scaffolding, no git operations +Comprehensive security audit performed by CoS Self-Improvement agent. -### Mode 2: Create New App -Scaffold a new project from template: +### Vulnerabilities Found and Fixed -**Steps:** -1. **Basic Info**: Name, description -2. **Template**: Select template (vite+express, node-server, static) -3. **Location**: Parent directory for new repo -4. **Ports**: Allocate from available range -5. **Git Setup**: - - Initialize git - - Create GitHub repo (optional, via `gh` CLI) -6. **Confirm & Create** +1. **Command Injection in Git Service** (CRITICAL - FIXED) + - File: `server/services/git.js` + - Fix: Replaced `exec()` with `spawn()` and `shell: false`, added path validation -**Actions on create:** -- Copy template files -- Configure .env with ports -- Run `npm install` -- Initialize git + first commit -- Create GitHub repo (if selected) -- Generate PM2 ecosystem config -- Register in PortOS -- Start with PM2 +2. **Path Traversal in Screenshots Route** (HIGH - FIXED) + - File: `server/routes/screenshots.js` + - Fix: Added `sanitizeFilename()` and path validation -### API Endpoints (M4) -| Route | Description | -|-------|-------------| -| POST /api/apps | Register existing app | -| POST /api/scaffold | Create new app from template | -| GET /api/templates | List available templates | -| POST /api/detect/ports | Detect process on port | -| POST /api/detect/repo | Validate repo path, detect type | +### Secure Patterns (No Issues Found) +- Command execution uses allowlist +- PM2 operations use spawn with shell: false +- Input validation with Zod schemas +- No dangerouslySetInnerHTML in React +- API keys stored server-side only +- JSON content type required for mutations --- -### All API Endpoints -| Route | Description | -|-------|-------------| -| GET /api/health | Health check | -| GET/POST /api/apps | List/register apps | -| GET/PUT/DELETE /api/apps/:id | App CRUD | -| POST /api/apps/:id/start | Start via PM2 | -| POST /api/apps/:id/stop | Stop via PM2 | -| POST /api/apps/:id/restart | Restart via PM2 | -| GET /api/apps/:id/status | PM2 status | -| GET /api/apps/:id/logs | Get logs | -| GET /api/ports/scan | Scan ports | -| GET /api/logs/processes | List PM2 processes | -| GET /api/logs/:name | Get logs (static) | -| Socket.IO logs:subscribe | Stream logs (WebSocket) | -| POST /api/scaffold | Create new app | -| GET /api/templates | List templates | -| POST /api/detect/ports | Detect process on port | -| POST /api/detect/repo | Validate repo path | -| GET /api/providers | List AI providers | -| POST /api/providers | Add provider | -| PUT /api/providers/:id | Update provider | -| DELETE /api/providers/:id | Delete provider | -| POST /api/providers/:id/test | Test provider connectivity | -| PUT /api/providers/active | Set active provider | -| GET /api/runs | List run history | -| POST /api/runs | Execute new run | -| GET /api/runs/:id | Get run details | -| GET /api/runs/:id/output | Get run output | -| POST /api/runs/:id/stop | Stop active run | -| DELETE /api/runs/:id | Delete run | -| GET /api/history | List action history | -| GET /api/history/stats | Get history statistics | -| DELETE /api/history | Clear history | -| POST /api/commands/execute | Execute shell command | -| POST /api/commands/:id/stop | Stop running command | -| GET /api/commands/allowed | List allowed commands | -| GET /api/commands/processes | List PM2 processes | -| POST /api/detect/ai | AI-powered app detection | -| GET /api/templates | List available templates | -| POST /api/templates/create | Create app from template | -| POST /api/templates | Add custom template | -| DELETE /api/templates/:id | Delete custom template | -| GET /api/agents | List running AI agent processes | -| GET /api/agents/:pid | Get agent process details | -| DELETE /api/agents/:pid | Kill agent process | +## Planned Feature Details ---- - -## M7: App Templates (Planned) +### M7: App Templates Templates allow creating new apps from pre-configured project structures. -### Built-in Template: PortOS Stack -The default template mirrors this application's architecture: +**Built-in Template: PortOS Stack** - Express.js API server - React + Vite frontend - Tailwind CSS styling -- Collapsible navigation layout - PM2 ecosystem configuration - GitHub Actions CI/CD workflows - Auto-versioning system -### Features -1. **Template Selection**: Browse available templates with feature descriptions -2. **App Creation**: Scaffold new project with chosen name and target directory -3. **Custom Templates**: Register additional templates from local paths -4. **Template Management**: View, edit, delete custom templates +**Features** +1. Template Selection - Browse available templates with feature descriptions +2. App Creation - Scaffold new project with chosen name and target directory +3. Custom Templates - Register additional templates from local paths +4. Template Management - View, edit, delete custom templates -### Pages +**Pages** - `/templates` - Template browser and app creation - `/templates/new` - Register custom template -### Template Structure -Templates are stored in `./data/templates/` with: -- `manifest.json` - Template metadata and feature list -- Source files to copy when creating new app - -### API Endpoints +**API Endpoints** | Route | Description | |-------|-------------| | GET /api/templates | List all templates | | POST /api/templates | Add custom template | | POST /api/templates/create | Create app from template | | DELETE /api/templates/:id | Remove custom template | - ---- - -## M8: Prompt Manager (Planned) - -Customizable AI prompts for all backend AI operations. Inspired by void-private's prompt management architecture. - -### Architecture -- **File-based prompts**: Prompts stored as `.md` files for easy editing and version control -- **Variable system**: Reusable variables with metadata, categories, and usage tracking -- **Stage configuration**: Each prompt stage defines provider, model type, and execution options -- **Template rendering**: Mustache-like syntax with conditionals, arrays, and nested data - -### Directory Structure -``` -./data/prompts/ -โ”œโ”€โ”€ stages/ # Individual prompt templates (.md files) -โ”‚ โ”œโ”€โ”€ app-detection.md -โ”‚ โ”œโ”€โ”€ code-analysis.md -โ”‚ โ””โ”€โ”€ ... -โ”œโ”€โ”€ variables.json # Reusable prompt variables -โ””โ”€โ”€ stage-config.json # Stage metadata and provider config -``` - -### Features -1. **Prompt Stages**: Define different prompts for different AI tasks (detection, analysis, etc.) -2. **Variables**: Reusable content blocks (personas, formats, constraints) -3. **Per-Stage Provider Config**: Each stage can use different AI providers/models -4. **Web UI**: Edit prompts, variables, and preview rendered output -5. **Template Syntax**: `{{variable}}`, `{{#condition}}...{{/condition}}`, arrays - -### UI Pages -- `/prompts` - Prompt Manager with tabs for Stages, Variables, Elements -- Live preview with test variables -- Insert variable references - -### API Endpoints -| Route | Description | -|-------|-------------| -| GET /api/prompts | List all prompt stages | -| GET /api/prompts/:stage | Get stage template | -| PUT /api/prompts/:stage | Update stage/template | -| POST /api/prompts/:stage/preview | Preview compiled prompt | -| GET /api/prompts/variables | List all variables | -| PUT /api/prompts/variables/:key | Update variable | -| POST /api/prompts/variables | Create variable | -| DELETE /api/prompts/variables/:key | Delete variable | - ---- - -## M9: Streaming Import Detection (Planned) - -Enhanced app import with real-time websocket updates as AI discovers project configuration. - -### Features -1. **Progressive Discovery**: Form fields update in real-time as AI analyzes the project -2. **Status Animations**: Visual indicators showing detection progress -3. **PM2 Detection**: Check if app is already running in PM2 -4. **Websocket Updates**: Server streams discoveries to UI as they happen - -### Discovery Steps (streamed to UI) -1. Validating directory path -2. Reading package.json -3. Detecting project type (React, Express, monorepo, etc.) -4. Finding configuration files -5. Extracting ports from configs -6. Detecting start commands -7. Checking PM2 process status -8. AI-powered analysis for name, description -9. Generating PM2 process names - -### UI/UX -- Progress indicator with current step -- Form fields animate in as values are discovered -- Already-running indicator if detected in PM2 -- Expandable "Detection Log" showing raw discovery output - -### API -| Route | Description | -|-------|-------------| -| WS /api/detect/stream | Websocket for streaming detection | -| GET /api/detect/pm2-status | Check if app running in PM2 | - ---- - -## M13: Autofixer Integration - -Autonomous crash detection and repair using Claude CLI. - -### Architecture -- **Daemon Process** (`autofixer/server.js`): Monitors PM2 for crashed processes registered in PortOS -- **UI Server** (`autofixer/ui.js`): Web interface for viewing logs and fix history on port 5560 -- **PM2 Integration**: Runs as `portos-autofixer` and `portos-autofixer-ui` processes - -### Features -1. **Crash Detection**: Polls PM2 every 15 minutes for `errored` status on registered apps -2. **Auto-Fix**: Invokes Claude CLI with crash context (error logs, app info) to diagnose and repair -3. **Session History**: Stores fix attempts with prompts, outputs, and success/failure status -4. **Cooldown**: 30-minute cooldown per process to prevent repeated fix loops -5. **Log Streaming**: Real-time SSE log streaming from any PM2 process -6. **Tailscale Compatible**: Dynamic hostname for remote access - -### Data Storage -``` -./data/autofixer/ -โ”œโ”€โ”€ index.json # Fix session index -โ””โ”€โ”€ sessions/ - โ””โ”€โ”€ {sessionId}/ - โ”œโ”€โ”€ prompt.txt # Prompt sent to Claude - โ”œโ”€โ”€ output.txt # Claude's response - โ””โ”€โ”€ metadata.json # Session details -``` - -### Autofixer UI (port 6000) -- Process sidebar with live status indicators -- SSE-powered log viewer with pause/clear controls -- History tab showing fix attempts with success/failure status -- Links back to PortOS Dashboard - -### Configuration -| Setting | Value | -|---------|-------| -| UI Port | 5560 | -| Check Interval | 15 minutes | -| Fix Cooldown | 30 minutes | -| Max History | 100 entries | - ---- - -## M14: Chief of Staff - -Autonomous agent manager that watches task files, spawns sub-agents, and maintains system health. - -### Architecture -- **Task Parser** (`server/lib/taskParser.js`): Parses TASKS.md and COS-TASKS.md formats -- **CoS Service** (`server/services/cos.js`): State management, health monitoring, task evaluation -- **Task Watcher** (`server/services/taskWatcher.js`): File watching with chokidar -- **Sub-Agent Spawner** (`server/services/subAgentSpawner.js`): Claude CLI execution with MCP -- **CoS Routes** (`server/routes/cos.js`): REST API endpoints -- **CoS UI** (`client/src/pages/ChiefOfStaff.jsx`): Tasks, Agents, Health, Config tabs - -### Features -1. **Dual Task Lists**: User tasks (TASKS.md) and system tasks (COS-TASKS.md) -2. **Autonomous Execution**: Auto-approved tasks run without user intervention -3. **Approval Workflow**: Tasks marked APPROVAL require user confirmation -4. **System Health Monitoring**: PM2 process checks, memory usage, error detection -5. **Sub-Agent Spawning**: Claude CLI with --dangerously-skip-permissions and MCP servers -6. **Self-Improvement**: Can analyze performance and suggest prompt/config improvements -7. **Script Generation**: Creates automation scripts for repetitive tasks -8. **Report Generation**: Daily summaries of completed work - -### Task File Format -```markdown -# Tasks -## Pending -- [ ] #task-001 | HIGH | Task description - - Context: Additional context - - App: app-name - -## In Progress -- [~] #task-002 | MEDIUM | Another task - - Agent: agent-id - - Started: 2024-01-15T10:30:00Z - -## Completed -- [x] #task-003 | LOW | Done task - - Completed: 2024-01-14T15:45:00Z -``` - -### System Task Format (with approval flags) -```markdown -- [ ] #sys-001 | HIGH | AUTO | Auto-approved task -- [ ] #sys-002 | MEDIUM | APPROVAL | Needs user approval -``` - -### Data Storage -``` -./data/cos/ -โ”œโ”€โ”€ state.json # Daemon state and config -โ”œโ”€โ”€ agents/{agentId}/ # Agent prompts and outputs -โ”œโ”€โ”€ reports/{date}.json # Daily reports -โ””โ”€โ”€ scripts/ # Generated automation scripts -``` - -### API Endpoints -| Route | Description | -|-------|-------------| -| GET /api/cos | Get CoS status | -| POST /api/cos/start | Start daemon | -| POST /api/cos/stop | Stop daemon | -| GET/PUT /api/cos/config | Configuration | -| GET /api/cos/tasks | Get all tasks | -| POST /api/cos/evaluate | Force evaluation | -| GET /api/cos/health | Health status | -| POST /api/cos/health/check | Run health check | -| GET /api/cos/agents | List agents | -| POST /api/cos/agents/:id/terminate | Terminate agent | -| GET /api/cos/reports | List reports | - -### Prompt Templates -| Template | Purpose | -|----------|---------| -| cos-agent-briefing | Brief sub-agent on task | -| cos-evaluate | Evaluate tasks and decide actions | -| cos-report-summary | Generate daily summary | -| cos-self-improvement | Analyze and suggest improvements | - -### Configuration -| Setting | Default | Description | -|---------|---------|-------------| -| evaluationIntervalMs | 60000 | Task evaluation interval | -| healthCheckIntervalMs | 900000 | Health check interval | -| maxConcurrentAgents | 3 | Max parallel agents | -| maxProcessMemoryMb | 2048 | Memory alert threshold | -| autoStart | false | Start on server boot | -| selfImprovementEnabled | true | Allow self-analysis | - -### Model Selection Rules -The `selectModelForTask` function in `subAgentSpawner.js` routes tasks to appropriate model tiers: - -| Tier | Trigger | Example Tasks | -|------|---------|---------------| -| **heavy** | Critical priority, visual analysis, complex reasoning | Architect, refactor, security audit, long context | -| **medium** | Standard development tasks, default | Most coding tasks, bug fixes, feature implementation | -| **light** | Documentation-only tasks | Update README, write docs, format text | - -**Important**: Light model (haiku) is NEVER used for coding tasks. Tasks containing keywords like `fix`, `bug`, `implement`, `test`, `feature`, `api`, `component`, etc. are automatically routed to medium tier or higher. - ---- - -## M15: Graceful Error Handling - -Enhanced error handling system with automatic recovery and UI notifications. - -### Architecture -- **Error Handler** (`server/lib/errorHandler.js`): Centralized error normalization and Socket.IO emission -- **Auto-Fixer** (`server/services/autoFixer.js`): Automatic agent spawning for critical errors -- **Socket.IO Integration**: Real-time error notifications to connected clients -- **Route Protection**: All routes use asyncHandler wrapper for consistent error handling - -### Features -1. **Graceful Error Handling**: Server never crashes, all errors caught and handled -2. **Socket.IO Error Events**: Real-time error notifications to UI with severity and context -3. **Auto-Fix Tasks**: Critical errors automatically create CoS tasks for agent resolution -4. **Error Recovery UI**: Client can request manual error recovery via Socket.IO -5. **Process Error Handlers**: Unhandled rejections and exceptions trigger auto-fix -6. **Error Deduplication**: Prevents duplicate auto-fix tasks within 1-minute window - -### Error Severity Levels -| Severity | Description | Auto-Fix | -|----------|-------------|----------| -| warning | Non-critical issues | No | -| error | Server errors, failures | No | -| critical | System-threatening errors | Yes | - -### Socket.IO Events -| Event | Direction | Payload | -|-------|-----------|---------| -| error:occurred | Server โ†’ Client | Error details with severity, code, timestamp | -| system:critical-error | Server โ†’ Client | Critical errors only | -| error:notified | Server โ†’ Subscribers | Error notification to subscribed clients | -| errors:subscribe | Client โ†’ Server | Subscribe to error events | -| errors:unsubscribe | Client โ†’ Server | Unsubscribe from error events | -| error:recover | Client โ†’ Server | Request manual error recovery | -| error:recover:requested | Server โ†’ Client | Recovery task created confirmation | - -### Auto-Fix Flow -1. Error occurs in route or service -2. `asyncHandler` catches and normalizes error -3. Error emitted to `errorEvents` EventEmitter -4. `autoFixer` checks if error should trigger auto-fix -5. If yes, creates CoS task with error context -6. Socket.IO broadcasts error to all connected clients -7. CoS evaluates and spawns agent to fix the error -8. Agent analyzes, fixes, and reports back - -### Implementation Files -| File | Purpose | -|------|---------| -| `server/lib/errorHandler.js` | Error classes, asyncHandler, middleware | -| `server/services/autoFixer.js` | Auto-fix task creation and deduplication | -| `server/services/socket.js` | Socket.IO error event forwarding | -| `server/routes/*.js` | All routes use asyncHandler wrapper | -| `client/src/hooks/useErrorNotifications.js` | Client-side error event handler with toast notifications | -| `client/src/components/Layout.jsx` | Mounts error notification hook for app-wide coverage | - -### Error Context -Errors include rich context for debugging: -- Error code and message -- HTTP status code -- Timestamp -- Stack trace (for 500+ errors) -- Custom context object -- Severity level -- Auto-fix flag - ---- - -## M16: Memory System - -Semantic memory system for the Chief of Staff that stores facts, learnings, observations, decisions, and user preferences with vector embeddings for intelligent retrieval. - -### Architecture -- **Memory Service** (`server/services/memory.js`): Core CRUD, search, and lifecycle operations -- **Embeddings Service** (`server/services/memoryEmbeddings.js`): LM Studio integration for vector generation -- **Memory Extractor** (`server/services/memoryExtractor.js`): Extract memories from agent output -- **Memory Retriever** (`server/services/memoryRetriever.js`): Context injection for agent prompts -- **Memory Routes** (`server/routes/memory.js`): REST API endpoints -- **Memory Tab** (`ChiefOfStaff.jsx`): UI with list, timeline, and graph views - -### Features -1. **Six Memory Types**: fact, learning, observation, decision, preference, context -2. **Semantic Search**: LM Studio embeddings for similarity-based retrieval -3. **Auto-Extraction**: Memories extracted from successful agent task completions -4. **Auto-Injection**: Relevant memories injected into agent prompts before execution -5. **Importance Decay**: Time-based decay with access-based boosts -6. **Memory Consolidation**: Merge similar memories automatically -7. **Real-time Updates**: WebSocket events for memory changes -8. **Graph Visualization**: D3.js relationship graph (planned) - -### Memory Schema -```javascript -{ - id: string, // UUID - type: 'fact' | 'learning' | 'observation' | 'decision' | 'preference' | 'context', - content: string, // Full memory content - summary: string, // Short summary - category: string, // e.g., 'codebase', 'workflow', 'tools' - tags: string[], // Auto-extracted and user-defined - relatedMemories: string[], // Linked memory IDs - sourceTaskId: string, // Origin task - sourceAgentId: string, // Origin agent - embedding: number[], // Vector (768 dims for nomic-embed) - confidence: number, // 0.0-1.0 - importance: number, // 0.0-1.0 (decays over time) - accessCount: number, - lastAccessed: string, - createdAt: string, - status: 'active' | 'archived' | 'expired' -} -``` - -### Data Storage -``` -./data/cos/memory/ -โ”œโ”€โ”€ index.json # Lightweight metadata for listing/filtering -โ”œโ”€โ”€ embeddings.json # Vector storage for semantic search -โ””โ”€โ”€ memories/ # Full memory content - โ””โ”€โ”€ {id}/ - โ””โ”€โ”€ memory.json -``` - -### API Endpoints -| Route | Description | -|-------|-------------| -| GET /api/memory | List memories with filters | -| GET /api/memory/:id | Get single memory | -| POST /api/memory | Create memory | -| PUT /api/memory/:id | Update memory | -| DELETE /api/memory/:id | Delete (soft) memory | -| POST /api/memory/search | Semantic search | -| GET /api/memory/categories | List categories | -| GET /api/memory/tags | List tags | -| GET /api/memory/timeline | Timeline view data | -| GET /api/memory/graph | Graph visualization data | -| GET /api/memory/stats | Memory statistics | -| POST /api/memory/link | Link two memories | -| POST /api/memory/consolidate | Merge similar memories | -| POST /api/memory/decay | Apply importance decay | -| DELETE /api/memory/expired | Clear expired memories | -| GET /api/memory/embeddings/status | LM Studio connection status | - -### WebSocket Events -| Event | Description | -|-------|-------------| -| cos:memory:created | New memory created | -| cos:memory:updated | Memory updated | -| cos:memory:deleted | Memory deleted | -| cos:memory:extracted | Memories extracted from agent | -| cos:memory:approval-needed | Medium-confidence memories pending approval | - -### Setup Requirements - -**LM Studio** must be running with an embedding model loaded for the memory system to work: - -1. Download and install [LM Studio](https://lmstudio.ai/) -2. Load an embedding model: `text-embedding-nomic-embed-text-v2-moe` (recommended) or `text-embedding-nomic-embed-text-v1.5` -3. Start the local server on port 1234 (default) -4. The memory system will automatically connect - -### LM Studio Configuration -```javascript -memory: { - enabled: true, - embeddingProvider: 'lmstudio', - embeddingEndpoint: 'http://localhost:1234/v1/embeddings', - embeddingModel: 'text-embedding-nomic-embed-text-v2-moe', - embeddingDimension: 768, - maxContextTokens: 2000, - minRelevanceThreshold: 0.7, - autoExtractEnabled: true -} -``` - -### Memory Extraction -Memories are extracted from agent output: -1. **Structured Blocks**: `content` -2. **Pattern Matching**: "I learned that...", "User prefers...", "I decided to..." -3. **High confidence (>0.8)**: Auto-saved -4. **Medium confidence (0.5-0.8)**: Queued for user approval - -### Memory Injection -Before agent task execution: -1. Generate embedding for task description -2. Find semantically similar memories (>0.7 relevance) -3. Include high-importance user preferences -4. Include relevant codebase facts -5. Format as markdown section in agent prompt - -### Implementation Files -| File | Purpose | -|------|---------| -| `server/lib/memoryValidation.js` | Zod schemas for memory operations | -| `server/lib/vectorMath.js` | Cosine similarity, clustering helpers | -| `server/services/memory.js` | Core CRUD, search, lifecycle | -| `server/services/memoryEmbeddings.js` | LM Studio embedding generation | -| `server/services/memoryExtractor.js` | Extract memories from agent output | -| `server/services/memoryRetriever.js` | Retrieve and format for injection | -| `server/routes/memory.js` | REST API endpoints | -| `client/src/pages/ChiefOfStaff.jsx` | MemoryTab, MemoryTimeline, MemoryGraph | -| `client/src/services/api.js` | Memory API client functions | - ---- - -## M17: PM2 Ecosystem Config Enhancement - -Enhanced app management to detect and track all PM2 processes with their ports. - -### Features -- **Per-process port detection**: Extracts ports for each PM2 process from ecosystem.config.js/cjs -- **CDP_PORT support**: Detects Chrome DevTools Protocol ports for browser processes -- **Constant resolution**: Handles variable references like `CDP_PORT: CDP_PORT` by parsing top-level constants -- **Comment handling**: Properly skips `//` and `/* */` comments when parsing config files -- **Refresh button**: UI button to re-scan ecosystem config and update app data - -### Schema Extension -```javascript -// Added to server/lib/validation.js -export const processSchema = z.object({ - name: z.string().min(1), - port: z.number().int().min(1).max(65535).nullable().optional(), - description: z.string().optional() -}); - -// Added to appSchema -processes: z.array(processSchema).optional() -``` - -### API Endpoint -``` -POST /api/apps/:id/refresh-config -``` -Re-parses the app's ecosystem config and updates the app with new process/port data. - -### Implementation Files -| File | Changes | -|------|---------| -| `server/lib/validation.js` | Added `processSchema` and `processes` field to `appSchema` | -| `server/services/streamingDetect.js` | Added `parseEcosystemConfig()` with port/constant/comment parsing | -| `server/routes/apps.js` | Added `POST /:id/refresh-config` endpoint | -| `client/src/services/api.js` | Added `refreshAppConfig()` function | -| `client/src/pages/Apps.jsx` | Added "Refresh Config" button with loading state | - ---- - -## M18: PM2 Standardization - -LLM-powered refactoring to standardize app configurations to follow PM2 best practices. - -### Features -1. **Standardize PM2 Button**: Available in each app's expanded details row -2. **LLM Analysis**: Uses configured AI provider to analyze project structure -3. **Auto-apply Changes**: Automatically modifies files with git backup -4. **Git Backup**: Creates backup branch before any modifications -5. **Port Consolidation**: Moves all ports to ecosystem.config.cjs env blocks -6. **Stray Port Removal**: Removes PORT from .env files, comments out port in vite.config - -### PM2 Standard -- All ports defined in `ecosystem.config.cjs` env blocks -- PM2 configured with watch for live-reload on server processes -- Vite processes use `npx vite --host --port XXXX` in args -- No stray port references in .env or vite.config - -### API Endpoints -| Route | Description | -|-------|-------------| -| POST /api/standardize/analyze | Analyze app and generate standardization plan | -| POST /api/standardize/apply | Apply changes with git backup | -| GET /api/standardize/template | Get standard PM2 template reference | -| POST /api/standardize/backup | Create git backup only | - -### Socket Events (for streaming mode) -| Event | Description | -|-------|-------------| -| standardize:start | Client requests standardization | -| standardize:step | Progress update (analyze, backup, apply) | -| standardize:analyzed | Analysis complete with plan | -| standardize:complete | Standardization finished | - -### Implementation Files -| File | Purpose | -|------|---------| -| `server/services/pm2Standardizer.js` | Core analysis and apply logic | -| `server/routes/standardize.js` | API endpoints | -| `server/services/socket.js` | Socket event handlers | -| `client/src/pages/Apps.jsx` | Standardize PM2 button | -| `client/src/services/api.js` | API functions | - ---- - -## M19: CoS Agent Runner - -Isolated PM2 process for spawning Claude CLI agents, preventing orphaned processes when portos-server restarts. - -### Problem -When multiple CoS agents are running and the main portos-server restarts (due to code changes, crashes, or manual restart), child processes spawned via `child_process.spawn()` become orphaned. The parent loses track of them because the `activeAgents` Map is in memory. - -### Solution -A separate `portos-cos` PM2 process that: -1. Runs independently from `portos-server` -2. Manages agent spawning via HTTP/Socket.IO bridge -3. Doesn't restart when `portos-server` restarts -4. Maintains its own state file for PID tracking - -### Architecture -``` -โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” HTTP/Socket.IO โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” -โ”‚ portos-server โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บ โ”‚ portos-cos โ”‚ -โ”‚ (5554) โ”‚ spawn/terminate โ”‚ (5558) โ”‚ -โ”‚ โ”‚ โ—„โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚ โ”‚ -โ”‚ subAgentSpawnerโ”‚ events/output โ”‚ cos-runner โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ - โ”‚ spawn - โ–ผ - โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” - โ”‚ Claude CLI โ”‚ - โ”‚ Processes โ”‚ - โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ -``` - -### Features -1. **Isolated Process**: portos-cos runs as separate PM2 app, unaffected by portos-server restarts -2. **Fallback Mode**: If portos-cos is unavailable, falls back to direct spawning -3. **Event Bridge**: Socket.IO connection forwards agent output/completion events -4. **PID Tracking**: Runner maintains state file with active PIDs for orphan cleanup -5. **Graceful Shutdown**: 30-second kill timeout to allow agents to complete - -### Configuration -| Setting | Value | Description | -|---------|-------|-------------| -| Port | 5558 | HTTP/Socket.IO API port | -| kill_timeout | 30s | Grace period for agent shutdown | -| max_restarts | 5 | Limit restarts to prevent loops | -| min_uptime | 30s | Minimum runtime before healthy | - -### API Endpoints (portos-cos) -| Route | Description | -|-------|-------------| -| GET /health | Health check with active agent count | -| GET /agents | List active agents | -| POST /spawn | Spawn a new agent | -| POST /terminate/:agentId | Terminate specific agent | -| POST /terminate-all | Terminate all agents | -| GET /agents/:agentId/output | Get agent output buffer | - -### Socket.IO Events -| Event | Direction | Payload | -|-------|-----------|---------| -| agent:output | Runner โ†’ Server | agentId, text | -| agent:completed | Runner โ†’ Server | agentId, taskId, exitCode, success, duration | -| agent:error | Runner โ†’ Server | agentId, error | - -### Implementation Files -| File | Purpose | -|------|---------| -| `server/cos-runner/index.js` | Standalone Express server for agent management | -| `server/services/cosRunnerClient.js` | Client library for portos-server to communicate with runner | -| `server/services/subAgentSpawner.js` | Updated to use runner when available | -| `ecosystem.config.cjs` | Added portos-cos PM2 configuration | - -### Data Storage -``` -./data/cos/ -โ”œโ”€โ”€ runner-state.json # Active agent PIDs and stats -โ””โ”€โ”€ agents/{agentId}/ # Agent prompts and outputs (shared) -``` - ---- - -## M20: AI Provider Error Handling - -Enhanced error extraction and display for AI provider execution failures, with automatic CoS investigation task creation. - -### Problem -When AI provider executions fail (e.g., invalid model, API errors), the devtools/runner UI only showed exit codes without meaningful error information, making debugging difficult. - -### Solution -1. **Error Extraction**: Extract meaningful error details from CLI output -2. **Error Categorization**: Classify errors by type (model_not_found, auth_error, rate_limit, etc.) -3. **Suggested Fixes**: Provide actionable suggestions for each error category -4. **CoS Integration**: Create investigation tasks for actionable failures -5. **UI Enhancement**: Display error details, category, and suggestions in history list - -### Error Categories -| Category | Description | Actionable | -|----------|-------------|------------| -| model_not_found | Model doesn't exist in provider | Yes | -| auth_error | API key invalid or unauthorized | Yes | -| api_error | Generic API request failure | Yes | -| rate_limit | Rate limiting triggered | No (transient) | -| quota_exceeded | Billing/quota issue | Yes | -| network_error | Connection refused/timeout | No (transient) | -| timeout | Process timeout (SIGTERM) | No | -| command_not_found | CLI tool not installed | Yes | -| permission_denied | File/directory access denied | Yes | -| unknown | Unrecognized error pattern | No | - -### Error Data Flow -1. **Extraction** (`runner.js::extractErrorDetails`): Parse CLI output for error messages -2. **Categorization** (`runner.js::categorizeError`): Match against known patterns -3. **Emission** (`runner.js::emitProviderError`): Emit to errorEvents EventEmitter -4. **Auto-fix** (`autoFixer.js::handleAIProviderError`): Create CoS investigation task -5. **Display** (`DevTools.jsx`): Show in history with error details, category, and fix suggestion - -### Metadata Schema Enhancement -Failed runs now include additional fields: -```javascript -{ - // Existing fields - exitCode: number, - success: boolean, - error: string, // Primary error message - - // New fields - errorDetails: string, // Additional error context - errorCategory: string, // e.g., 'model_not_found' - suggestedFix: string // Actionable suggestion -} -``` - -### CoS Investigation Tasks -When actionable errors occur, a CoS task is created: -```markdown -## Pending -- [ ] #task-XXX | HIGH | APPROVAL | [Auto] Investigate agent failure: Model not found - - Context: # AI Provider Execution Failure - - Provider: Claude Code CLI - - Model: codex - - Error: model: codex - - Suggested Fix: Update model configuration -``` - -### Implementation Files -| File | Changes | -|------|---------| -| `server/services/runner.js` | Added `extractErrorDetails`, `categorizeError`, `emitProviderError` | -| `server/services/autoFixer.js` | Added `handleAIProviderError`, `buildAIProviderErrorContext` | -| `server/services/subAgentSpawner.js` | Added `analyzeAgentFailure`, `createInvestigationTask` | -| `client/src/pages/DevTools.jsx` | Display `errorCategory`, `errorDetails`, `suggestedFix` | - -### UI Display -In the DevTools history, failed runs now show: -- **Error Category Badge**: Colored tag showing the error type -- **Error Message**: Extracted error details -- **Suggested Fix Panel**: Yellow-bordered box with actionable advice -- **Additional Details**: Expandable section with raw error context - ---- - -## M21: Usage Metrics Integration - -Enhanced usage tracking to capture all AI provider executions across CoS agents and DevTools runner. - -### Problem -The Usage page existed but displayed zero data because the tracking functions were never called when runs executed. - -### Solution -1. **Usage Service Integration**: Added `recordSession` and `recordMessages` calls to runner.js and subAgentSpawner.js -2. **Mobile Responsive Design**: Redesigned usage page with responsive Tailwind breakpoints -3. **Unified Tracking**: Both manual DevTools runs and CoS agent executions now record usage - -### Data Tracked -| Metric | Source | Description | -|--------|--------|-------------| -| Sessions | createRun, createAgentRun | Incremented when any AI execution starts | -| Messages | run completion | Recorded on successful run completion | -| Tokens | Output-based estimate | ~4 characters per token estimation | -| Provider Stats | Per provider | Sessions, messages, tokens by provider | -| Model Stats | Per model | Sessions, messages, tokens by model | -| Daily Activity | Per day | Sessions, messages, tokens by date | -| Hourly Activity | 24-hour array | Session counts by hour of day | - -### Mobile Responsive Grid -```css -/* Summary stats */ -grid-cols-2 sm:grid-cols-3 lg:grid-cols-5 - -/* Charts and tables */ -grid-cols-1 md:grid-cols-2 -``` - -### Implementation Files -| File | Changes | -|------|---------| -| `server/services/runner.js` | Import usage.js, call recordSession in createRun, recordMessages on completion | -| `server/services/subAgentSpawner.js` | Import usage.js, call recordSession in createAgentRun, recordMessages in completeAgentRun | -| `client/src/pages/DevTools.jsx` | Mobile-first responsive grid for UsagePage | - -### Data Flow -``` -createRun() / createAgentRun() - โ””โ”€โ–บ recordSession(providerId, providerName, model) - โ””โ”€โ–บ updates byProvider, byModel, dailyActivity, hourlyActivity - -executeCliRun() / executeApiRun() / completeAgentRun() - โ””โ”€โ–บ recordMessages(providerId, model, 1, estimatedTokens) - โ””โ”€โ–บ updates totalMessages, byProvider, byModel tokens -``` - ---- - -## M22: Orphan Auto-Retry - -Automatic retry for orphaned agents with investigation task creation after repeated failures. - -### Problem -When agents become orphaned (server restart, runner crash, etc.), the task remained stuck and the system didn't automatically retry or investigate the issue. - -### Solution -1. **Auto-Retry**: Reset orphaned tasks to pending for automatic retry (up to 3 attempts) -2. **Retry Tracking**: Track retry count in task metadata (orphanRetryCount) -3. **Investigation Task**: After max retries, create auto-approved investigation task -4. **Evaluation Trigger**: Trigger task evaluation immediately after orphan cleanup - -### Retry Flow -``` -Agent Orphaned - โ””โ”€โ–บ cleanupOrphanedAgents() marks agent as completed - โ””โ”€โ–บ handleOrphanedTask() checks retry count - โ”œโ”€โ–บ retryCount < 3: Reset task to pending, trigger evaluation - โ””โ”€โ–บ retryCount >= 3: Mark task blocked, create investigation task -``` - -### Task Metadata -```javascript -metadata: { - orphanRetryCount: number, // Incremented each orphan - lastOrphanedAt: string, // ISO timestamp - lastOrphanedAgentId: string, // Agent that was orphaned - blockedReason: string // Only if max retries exceeded -} -``` - -### Investigation Task (Auto-Approved) -When max retries (3) are exceeded, an investigation task is created: -- Priority: HIGH -- approvalRequired: false (auto-approved, will run immediately) -- Contains: original task description, retry count, last agent ID -- Instructions: Check logs, verify spawning, look for resource constraints - -### Implementation Files -| File | Changes | -|------|---------| -| `server/services/subAgentSpawner.js` | Added `handleOrphanedTask()`, updated `cleanupOrphanedAgents()` to reset tasks and trigger evaluation | -| `server/services/cos.js` | Added `getTaskById()` export, updated `updateTask()` to merge metadata properly | - -### Configuration -| Setting | Value | Description | -|---------|-------|-------------| -| MAX_ORPHAN_RETRIES | 3 | Retries before creating investigation task | - ---- - -## M23: Self-Improvement System - -Automated self-analysis and improvement system that uses Playwright and Opus to continuously improve PortOS. - -### Problem -CoS idle reviews only checked managed apps for simple fixes. The system wasn't proactively improving itself - checking its own UI for bugs, mobile responsiveness issues, security vulnerabilities, or code quality. - -### Solution -A comprehensive self-improvement system that rotates through 7 analysis types: - -1. **ui-bugs**: Navigate to all routes with Playwright, check console errors, fix bugs -2. **mobile-responsive**: Test at mobile/tablet/desktop viewports, fix layout issues -3. **security**: Audit server/client code for XSS, injection, path traversal -4. **code-quality**: Find DRY violations, large functions, dead code -5. **accessibility**: Check ARIA labels, color contrast, keyboard navigation -6. **console-errors**: Capture and fix all JavaScript errors -7. **performance**: Analyze re-renders, N+1 queries, bundle size - -### Key Features -- **Automatic rotation**: Cycles through analysis types when CoS is idle -- **Opus model**: All self-improvement tasks use claude-opus-4-5-20251101 for thorough analysis -- **Playwright integration**: Uses MCP browser tools to analyze live UI -- **Auto-approved**: Tasks run without requiring manual approval -- **Alternates with app reviews**: Balances self-improvement with managed app maintenance - -### Task Flow -``` -CoS Idle (no user/system tasks) - โ””โ”€โ–บ Check lastSelfImprovement vs lastIdleReview - โ”œโ”€โ–บ Self-improvement older: Generate self-improvement task - โ”‚ โ””โ”€โ–บ Rotate to next analysis type - โ”‚ โ””โ”€โ–บ Spawn Opus agent with Playwright instructions - โ””โ”€โ–บ App review older: Generate idle app review (existing behavior) -``` - -### Implementation Files -| File | Purpose | -|------|---------| -| `server/services/cos.js` | Added `generateSelfImprovementTask()`, updated `generateIdleReviewTask()` | -| `server/services/selfImprovement.js` | Constants, prompt templates, utility functions | - -### State Tracking -```javascript -stats: { - lastSelfImprovement: ISO timestamp, - lastSelfImprovementType: 'ui-bugs' | 'mobile-responsive' | ... , - lastIdleReview: ISO timestamp -} -``` - -### Example Task -``` -[Self-Improvement] UI Bug Analysis - -Use Playwright MCP to analyze PortOS UI at http://localhost:5555/ -- Navigate to each route -- Capture browser snapshots and console messages -- Fix any bugs found -- Commit changes - -Use model: claude-opus-4-5-20251101 -``` - ---- - -## M24: Goal-Driven Proactive Mode - -Makes CoS proactive and goal-driven, spending more time working and less time idle. - -### Problem -CoS was too passive - it would sit idle when apps were on cooldown instead of finding other productive work. The system lacked clear goals and mission guidance. - -### Solution - -#### 1. COS-GOALS.md Mission File -Created `data/COS-GOALS.md` with: -- Mission statement and core principles -- 5 active goals (codebase quality, self-improvement, documentation, user engagement, system health) -- Task generation priorities -- Behavior guidelines -- Metrics to track - -#### 2. Always-Working Behavior -Updated `generateIdleReviewTask()` to ALWAYS return work: -- First tries self-improvement if it's been longer since last run -- Then tries app reviews if any apps are off cooldown -- **Falls back to self-improvement** if apps are on cooldown (instead of returning null) - -#### 3. Expanded Task Types -Added 4 new self-improvement task types: -- `cos-enhancement`: Improve CoS capabilities and prompts -- `test-coverage`: Add missing tests -- `documentation`: Update PLAN.md, docs/, code comments -- `feature-ideas`: Brainstorm and implement small features - -Now 11 task types total, rotating through all goals. - -#### 4. Configuration Changes -| Setting | Old | New | -|---------|-----|-----| -| `evaluationIntervalMs` | 300000 (5m) | 60000 (1m) | -| `appReviewCooldownMs` | 3600000 (1h) | 1800000 (30m) | -| `idleReviewPriority` | LOW | MEDIUM | -| `proactiveMode` | (new) | true | -| `goalsFile` | (new) | data/COS-GOALS.md | - -### Task Rotation -``` -1. ui-bugs โ†’ Check UI for errors -2. mobile-responsive โ†’ Test viewport sizes -3. security โ†’ Audit for vulnerabilities -4. code-quality โ†’ Find DRY violations, dead code -5. console-errors โ†’ Fix JS errors -6. performance โ†’ Optimize re-renders, queries -7. cos-enhancement โ†’ Improve CoS itself -8. test-coverage โ†’ Add tests -9. documentation โ†’ Update docs -10. feature-ideas โ†’ Implement new features -11. accessibility โ†’ Check a11y issues -``` - -### Key Behavior Change -``` -Before: Apps on cooldown โ†’ Return null โ†’ Idle -After: Apps on cooldown โ†’ Run self-improvement โ†’ Always working -``` - -### Implementation Files -| File | Changes | -|------|---------| -| `data/COS-GOALS.md` | New mission and goals file | -| `server/services/cos.js` | Updated config defaults, expanded task types, always-working logic | - ---- - -## M25: Task Learning System - -Tracks patterns from completed tasks to improve future task execution and model selection. - -### Features -1. **Completion Tracking**: Records success/failure for every agent task completion -2. **Success Rate Analysis**: Calculates success rates by task type and model tier -3. **Duration Tracking**: Tracks average execution time per task type -4. **Error Pattern Analysis**: Identifies recurring error categories -5. **Model Effectiveness**: Compares performance across model tiers (light/medium/heavy) -6. **Recommendations**: Generates actionable insights based on historical data - -### Data Tracked -| Metric | Grouping | Description | -|--------|----------|-------------| -| Completion count | By task type | Total tasks completed per type | -| Success rate | By task type | Percentage of successful completions | -| Average duration | By task type | Mean execution time | -| Error patterns | By category | Recurring error types with affected tasks | -| Model effectiveness | By tier | Success rate and duration per model | - -### API Endpoints -| Route | Description | -|-------|-------------| -| GET /api/cos/learning | Get learning insights and recommendations | -| GET /api/cos/learning/durations | Get task duration estimates by type | -| POST /api/cos/learning/backfill | Backfill learning data from history | - -### Data Storage -``` -./data/cos/ -โ””โ”€โ”€ learning.json # Learning metrics and patterns -``` - -### Implementation Files -| File | Purpose | -|------|---------| -| `server/services/taskLearning.js` | Core learning service | -| `server/routes/cos.js` | Learning API endpoints | - ---- - -## M26: Scheduled Scripts - -Cron-based script scheduling with optional agent triggering for automated health checks and maintenance. - -### Features -1. **Schedule Presets**: Common intervals (5min, 15min, hourly, daily, weekly, on-demand) -2. **Custom Cron**: Full cron expression support for complex schedules -3. **Command Allowlist**: Security-first design using same allowlist as command runner -4. **Agent Triggering**: Scripts can spawn CoS agents when issues are detected -5. **Run History**: Track last execution, output, and exit codes -6. **Enable/Disable**: Toggle scripts without deleting them - -### Schedule Presets -| Preset | Cron Expression | -|--------|-----------------| -| every-5-min | `*/5 * * * *` | -| every-15-min | `*/15 * * * *` | -| every-30-min | `*/30 * * * *` | -| hourly | `0 * * * *` | -| every-6-hours | `0 */6 * * *` | -| daily | `0 9 * * *` | -| weekly | `0 9 * * 1` | -| on-demand | (manual only) | - -### Trigger Actions -| Action | Description | -|--------|-------------| -| log-only | Record output, no further action | -| spawn-agent | Create CoS task with output as context | -| create-task | Add task to COS-TASKS.md (not yet implemented) | - -### API Endpoints -| Route | Description | -|-------|-------------| -| GET /api/cos/scripts | List all scripts | -| POST /api/cos/scripts | Create a new script | -| GET /api/cos/scripts/presets | Get schedule presets | -| GET /api/cos/scripts/allowed-commands | Get allowed commands | -| GET /api/cos/scripts/jobs | Get scheduled job info | -| GET /api/cos/scripts/:id | Get specific script | -| PUT /api/cos/scripts/:id | Update a script | -| DELETE /api/cos/scripts/:id | Delete a script | -| POST /api/cos/scripts/:id/run | Execute immediately | -| GET /api/cos/scripts/:id/runs | Get run history | - -### Data Storage -``` -./data/cos/ -โ”œโ”€โ”€ scripts/ # Script output logs -โ””โ”€โ”€ scripts-state.json # Script configurations -``` - -### Socket Events -| Event | Description | -|-------|-------------| -| script:created | New script created | -| script:updated | Script configuration changed | -| script:deleted | Script removed | -| script:started | Execution started | -| script:completed | Execution finished | -| script:error | Execution error | - -### Implementation Files -| File | Purpose | -|------|---------| -| `server/services/scriptRunner.js` | Script execution and scheduling | -| `server/routes/scripts.js` | REST API endpoints | -| `client/src/components/cos/tabs/ScriptCard.jsx` | Script UI component | - ---- - -## Security Audit (2026-01-08) - -Comprehensive security audit performed by CoS Self-Improvement agent. - -### Vulnerabilities Found and Fixed - -#### 1. Command Injection in Git Service (CRITICAL - FIXED) -**File**: `server/services/git.js` - -**Problem**: The service used `exec()` with string concatenation for shell commands, allowing command injection attacks: -```javascript -// VULNERABLE - user input directly in shell command -await execAsync(`git add ${paths}`, { cwd: dir }); -await execAsync(`git commit -m "${message}"`, { cwd: dir }); -``` - -An attacker could inject shell commands via file paths like `; rm -rf /` or commit messages containing shell metacharacters. - -**Fix Applied**: -- Replaced `exec()` with `spawn()` and `shell: false` to prevent shell interpretation -- Created `execGit()` helper that uses argument arrays instead of string concatenation -- Added `validateFilePaths()` to reject paths with: - - Null bytes (`\0`) - - Shell metacharacters (`;`, `|`, `&`, `` ` ``, `$`) - - Path traversal (`..`) - - Absolute paths (`/`) - -#### 2. Path Traversal in Screenshots Route (HIGH - FIXED) -**File**: `server/routes/screenshots.js` - -**Problem**: User-provided filenames were used directly without sanitization: -```javascript -// VULNERABLE - path traversal possible -const filepath = join(SCREENSHOTS_DIR, filename); -``` - -An attacker could access arbitrary files via paths like `../../../etc/passwd`. - -**Fix Applied**: -- Added `sanitizeFilename()` to strip path components and special characters -- Use `resolve()` to normalize paths, then verify result stays within `SCREENSHOTS_DIR` -- Defense in depth: check both upload and retrieval endpoints - -### No Issues Found (Secure Patterns in Use) - -#### Command Execution -- **`server/services/commands.js`**: Uses allowlist (`ALLOWED_COMMANDS`) to restrict which commands can be executed. Only the base command is checked, but it uses `spawn('sh', ['-c', command])` which allows pipes and redirects for allowed commands only. - -#### PM2 Operations -- **`server/services/pm2.js`**: Uses `spawn('pm2', args, { shell: false })` correctly. - -#### Input Validation -- **Zod schemas**: Routes use Zod validation for structured input (`lib/validation.js`, `lib/memoryValidation.js`) -- **Type coercion**: Query parameters properly parsed with `parseInt()` with fallback defaults - -#### React XSS Protection -- No `dangerouslySetInnerHTML` usage found -- React's JSX escapes content by default -- API client properly uses `JSON.stringify()` for request bodies - -#### No Sensitive Data in Client -- API keys stored server-side only in `data/providers.json` -- No localStorage usage for sensitive data -- Client uses relative API paths, no hardcoded secrets - -#### CSRF Protection -- Same-origin deployment (client and server on same domain) -- JSON content type required for all POST/PUT requests - -### Command Allowlist Review -**File**: `server/services/commands.js` - -The allowlist is comprehensive for the intended use case: -```javascript -const ALLOWED_COMMANDS = new Set([ - 'npm', 'npx', 'pnpm', 'yarn', 'bun', // Package managers - 'node', 'deno', // Runtimes - 'git', 'gh', // Git operations - 'pm2', // Process manager - 'ls', 'cat', 'head', 'tail', 'grep', 'find', 'wc', // File inspection - 'pwd', 'which', 'echo', 'env', // System info - 'curl', 'wget', // Network - 'docker', 'docker-compose', // Containers - 'make', 'cargo', 'go', 'python', 'python3', 'pip', 'pip3', // Build tools - 'brew' // macOS package manager -]); -``` - -**Note**: While this prevents arbitrary command execution, the system allows shell pipes/redirects within allowed commands because it uses `spawn('sh', ['-c', command])`. This is intentional for DevTools functionality but could theoretically be abused (e.g., `npm run "$(malicious)"`). - -### Recommendations for Future - -1. **Rate Limiting**: Consider adding rate limiting to prevent DoS -2. **Audit Logging**: Log all command executions for forensic analysis -3. **Content Security Policy**: Add CSP headers to client responses -4. **Command Argument Validation**: Consider validating arguments to allowed commands, not just base commands - ---- - -## M27: CoS Capability Enhancements (2026-01-08) - -Enhancements to the Chief of Staff system for better task learning, smarter prioritization, and expanded self-improvement capabilities. - -### Features - -#### 1. New Self-Improvement Task Type: Dependency Updates -Added `dependency-updates` task type that: -- Runs `npm audit` in both server and client directories -- Checks for outdated packages with `npm outdated` -- Reviews CRITICAL and HIGH severity vulnerabilities -- Updates dependencies carefully (patch โ†’ minor โ†’ major) -- Runs tests and build to verify updates -- Commits with changelog of what was updated - -#### 2. Enhanced Performance Tracking -New `getPerformanceSummary()` function provides: -- Overall success rate across all task types -- Top performing task types (>80% success) -- Task types needing attention (<50% success) -- Task types being skipped (<30% success) -- Average duration statistics - -#### 3. Learning Insights System -New functions for recording and retrieving observations: -- `recordLearningInsight()` - Store observations about what works/doesn't -- `getRecentInsights()` - Retrieve recent learning insights -- Insights stored with timestamp, type, and context - -#### 4. Periodic Performance Logging -Every 10 evaluations, CoS logs: -- Overall success rate and total tasks completed -- Count of top performers and tasks needing attention -- Evaluation count tracking in state - -### New API Endpoints - -| Route | Description | -|-------|-------------| -| GET /api/cos/learning/performance | Get performance summary | -| GET /api/cos/learning/insights | Get recent learning insights | -| POST /api/cos/learning/insights | Record a learning insight | - -### Self-Improvement Task Types (12 total) - -1. **ui-bugs** - Check UI for JavaScript errors -2. **mobile-responsive** - Test viewport sizes -3. **security** - Audit for vulnerabilities -4. **code-quality** - Find DRY violations, dead code -5. **console-errors** - Fix JS errors -6. **performance** - Optimize re-renders, queries -7. **cos-enhancement** - Improve CoS itself -8. **test-coverage** - Add tests -9. **documentation** - Update docs -10. **feature-ideas** - Implement new features -11. **accessibility** - Check a11y issues -12. **dependency-updates** - Update npm packages - -### Implementation Files - -| File | Changes | -|------|---------| -| `server/services/cos.js` | Added dependency-updates type, performance logging | -| `server/services/taskLearning.js` | Added getPerformanceSummary, recordLearningInsight, getRecentInsights | -| `server/services/selfImprovement.js` | Added new analysis types to constants | -| `server/routes/cos.js` | Added 3 new API endpoints for learning insights | - ---- - -## M28: Weekly Digest UI (2026-01-08) - -Added a visual "Digest" tab to the Chief of Staff page that displays weekly activity summaries with insights, accomplishments, and week-over-week comparisons. - -### Features - -1. **Weekly Summary View**: Visual dashboard showing tasks completed, success rate, work time, and issue count -2. **Week-over-Week Comparison**: Shows percentage changes from previous week with trend indicators -3. **Live Week Progress**: Real-time view of current week progress with projected totals -4. **Top Accomplishments**: Lists most significant completed tasks sorted by duration -5. **Task Type Breakdown**: Table view of performance metrics by task type -6. **Error Patterns**: Highlights recurring errors that need attention -7. **Actionable Insights**: Auto-generated insights like "Star Performer", "Needs Attention", "Recurring Issue" -8. **Historical Navigation**: Dropdown to view digests from previous weeks -9. **Collapsible Sections**: Expandable/collapsible sections for better information density - -### UI Components - -The DigestTab displays: -- **Summary Cards**: 4 stat cards showing completed tasks, success rate, work time, and issues -- **Live Progress Panel**: Shows current week progress with running agents indicator -- **Insights Section**: Color-coded insight cards (success/warning/action/info) -- **Accomplishments List**: Top 10 accomplishments with task type and duration -- **Task Type Table**: Sortable table showing completion counts and success rates -- **Issues Panel**: Error patterns with occurrence counts and affected tasks - -### API Integration - -Uses existing Weekly Digest backend endpoints: -- `GET /api/cos/digest` - Current week's digest -- `GET /api/cos/digest/list` - List all available digests -- `GET /api/cos/digest/progress` - Live current week progress -- `POST /api/cos/digest/generate` - Force regenerate digest -- `GET /api/cos/digest/:weekId` - Get specific week's digest - -### Implementation Files - -| File | Purpose | -|------|---------| -| `client/src/components/cos/tabs/DigestTab.jsx` | New Digest tab component | -| `client/src/components/cos/index.js` | Added DigestTab export | -| `client/src/components/cos/constants.js` | Added digest tab to TABS array | -| `client/src/pages/ChiefOfStaff.jsx` | Import and render DigestTab | - ---- - -## M29: Comprehensive App Improvement (2026-01-09) - -Extended the CoS self-improvement system to apply the same comprehensive analysis to managed apps (not just PortOS itself). - -### Problem - -The CoS had a sophisticated self-improvement system with 12 different analysis types (security, code quality, performance, etc.) for PortOS itself, but managed apps only received simple idle reviews (formatting, dead code, typos). This meant managed apps weren't getting the same level of thorough analysis and improvement. - -### Solution - -Created an app-agnostic self-improvement system that applies comprehensive analysis to any managed app: - -1. **10 Analysis Types for Apps**: Security audit, code quality, test coverage, performance, accessibility, console errors, dependency updates, documentation, error handling, and TypeScript typing -2. **Rotation System**: Each app tracks its last improvement type and rotates through all 10 types -3. **App Activity Tracking**: Extended `appActivity.js` to track `lastImprovementType` per app -4. **Configuration Toggle**: New `comprehensiveAppImprovement` config option (default: true) -5. **Backward Compatible**: Can fall back to simple idle reviews by disabling the config - -### Key Features - -- **App-Specific Context**: Each task template includes the app's name and repo path -- **Opus Model**: All comprehensive improvements use claude-opus-4-5-20251101 for thorough analysis -- **Auto-Approved**: Tasks run without requiring manual approval -- **Cooldown Management**: Respects existing 30-minute cooldown between working on the same app - -### Analysis Types - -| Type | Description | -|------|-------------| -| security-audit | Command injection, XSS, path traversal, authentication | -| code-quality | DRY violations, dead code, magic numbers | -| test-coverage | Add missing tests for critical paths | -| performance | N+1 queries, unnecessary re-renders, bundle size | -| accessibility | ARIA labels, color contrast, keyboard navigation | -| console-errors | Fix JavaScript and server errors | -| dependency-updates | npm audit, update packages safely | -| documentation | README, JSDoc, inline comments | -| error-handling | Try-catch blocks, error logging | -| typing | TypeScript type improvements | - -### Configuration - -```javascript -comprehensiveAppImprovement: true // Enable comprehensive analysis for managed apps -``` - -### Implementation Files - -| File | Changes | -|------|---------| -| `server/services/cos.js` | Added `MANAGED_APP_IMPROVEMENT_TYPES` array, `generateManagedAppImprovementTask()` function, updated `generateIdleReviewTask()` | -| `server/services/appActivity.js` | Added `lastImprovementType` field to app activity tracking | - -### Task Flow - -``` -CoS Idle โ†’ No user/system tasks - โ””โ”€โ–บ Try PortOS self-improvement (if older than last app review) - โ””โ”€โ–บ Try managed app improvement - โ”œโ”€โ–บ Get next app off cooldown - โ”œโ”€โ–บ Check comprehensiveAppImprovement config - โ”‚ โ”œโ”€โ–บ true: Generate comprehensive improvement task (rotates through 10 types) - โ”‚ โ””โ”€โ–บ false: Generate simple idle review (formatting, dead code, typos) - โ””โ”€โ–บ Mark app as reviewed, start cooldown -``` - -### Example Task - -``` -[App Improvement: MyApp] Security Audit - -Analyze the MyApp codebase for security vulnerabilities: - -Repository: /Users/user/projects/myapp - -1. Review routes/controllers for: - - Command injection in exec/spawn calls - - Path traversal in file operations - - Missing input validation - ... - -Use model: claude-opus-4-5-20251101 -``` - ---- - -## M31: LLM-based Memory Classification (2026-01-09) - -Replaced pattern-based memory extraction with intelligent LLM-based evaluation using local LM Studio. - -### Problem - -The existing memory extraction system used simple regex patterns to extract memories from agent output. This produced low-quality memories like "Task 'fix button': ## Summary" which just echoed task descriptions without any useful information. - -### Solution - -Created an LLM-based memory classifier that uses LM Studio's gptoss-20b model to intelligently evaluate agent output and extract only genuinely useful memories. - -### Key Features - -1. **LLM Evaluation**: Uses local LM Studio to analyze agent output -2. **Quality Filtering**: Rejects task echoes, generic summaries, and incomplete content -3. **Configurable**: Model, endpoint, confidence thresholds all configurable -4. **Fallback Support**: Falls back to pattern-based extraction if LLM unavailable -5. **Reasoning Capture**: Stores LLM's reasoning for why each memory is useful - -### Memory Evaluation Criteria - -**Good memories:** -- Codebase facts: File locations, architecture patterns, dependencies -- User preferences: Coding style, tool preferences, workflow patterns -- Learnings: Discovered behaviors, gotchas, workarounds -- Decisions: Architectural choices with reasoning - -**Rejected memories:** -- Task echoes: Just restating what the task was -- Generic summaries: "The task was successful" -- Temporary info: Session-specific data, timestamps -- Truncated/incomplete content - -### Configuration - -```json -// data/memory-classifier-config.json -{ - "enabled": true, - "provider": "lmstudio", - "endpoint": "http://localhost:1234/v1/chat/completions", - "model": "gptoss-20b", - "timeout": 60000, - "maxOutputLength": 10000, - "minConfidence": 0.6, - "fallbackToPatterns": true -} -``` - -### Prompt Template - -The `memory-evaluate.md` prompt template guides the LLM to: -1. Analyze the task context and agent output -2. Identify genuinely reusable knowledge -3. Return structured JSON with memories and rejections -4. Include reasoning for each decision - -### Implementation Files - -| File | Purpose | -|------|---------| -| `server/services/memoryClassifier.js` | LLM-based classification service | -| `data/prompts/stages/memory-evaluate.md` | Prompt template for memory evaluation | -| `data/memory-classifier-config.json` | Classifier configuration | -| `server/services/memoryExtractor.js` | Updated to use classifier when available | -| `data/prompts/stage-config.json` | Added memory-evaluate stage | -| `data/providers.json` | Added gptoss-20b model to LM Studio provider | - -### API - -The classifier exports: -- `classifyMemories(task, agentOutput)` - Main classification function -- `isAvailable()` - Check if LM Studio is reachable -- `getConfig()` - Get current configuration -- `updateConfig(updates)` - Update configuration - -### Flow - -``` -Agent completes task - โ””โ”€โ–บ memoryExtractor.extractAndStoreMemories() - โ””โ”€โ–บ Check if LLM classifier available - โ”œโ”€โ–บ Yes: Call classifyMemories() with LM Studio - โ”‚ โ””โ”€โ–บ Parse JSON response, filter by confidence - โ””โ”€โ–บ No: Fall back to pattern-based extraction - โ””โ”€โ–บ Create memories with embeddings - โ””โ”€โ–บ Auto-approve high confidence (โ‰ฅ0.8) - โ””โ”€โ–บ Queue medium confidence for approval -``` diff --git a/README.md b/README.md index 20edaac..0a9fbac 100644 --- a/README.md +++ b/README.md @@ -13,8 +13,10 @@ Think Umbrel, but for your active git repos and dev environments. Access your de - **Real-time Logs** โ€” Stream PM2 logs via Socket.IO with tail length control - **Smart Import** โ€” Point to a directory and auto-detect project config from package.json, vite.config, and ecosystem.config - **Chief of Staff** โ€” Autonomous task manager that monitors TASKS.md and dispatches AI agents +- **Weekly Digest** โ€” Visual insights, accomplishments, and week-over-week comparisons - **Task Learning** โ€” Tracks task success rates and duration to improve model selection - **Scheduled Scripts** โ€” Cron-based automation with optional agent triggering +- **Brain** โ€” Second-brain system for capturing thoughts and auto-classifying to People/Projects/Ideas/Admin - **Dev Tools** โ€” Process monitor, AI agent tracker, action history, git status, and shell runner - **AI Runner** โ€” Execute prompts via Claude Code, Codex, Gemini CLI, Ollama, or LM Studio - **Mobile Ready** โ€” Responsive design with collapsible sidebar for on-the-go access diff --git a/client/package-lock.json b/client/package-lock.json index 0f4ae65..79a2216 100644 --- a/client/package-lock.json +++ b/client/package-lock.json @@ -1,12 +1,12 @@ { "name": "portos-client", - "version": "0.5.5", + "version": "0.9.3", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "portos-client", - "version": "0.5.5", + "version": "0.9.3", "dependencies": { "@dnd-kit/core": "^6.3.1", "@dnd-kit/sortable": "^10.0.0", @@ -2345,9 +2345,9 @@ } }, "node_modules/react-router": { - "version": "7.11.0", - "resolved": "https://registry.npmjs.org/react-router/-/react-router-7.11.0.tgz", - "integrity": "sha512-uI4JkMmjbWCZc01WVP2cH7ZfSzH91JAZUDd7/nIprDgWxBV1TkkmLToFh7EbMTcMak8URFRa2YoBL/W8GWnCTQ==", + "version": "7.12.0", + "resolved": "https://registry.npmjs.org/react-router/-/react-router-7.12.0.tgz", + "integrity": "sha512-kTPDYPFzDVGIIGNLS5VJykK0HfHLY5MF3b+xj0/tTyNYL1gF1qs7u67Z9jEhQk2sQ98SUaHxlG31g1JtF7IfVw==", "license": "MIT", "dependencies": { "cookie": "^1.0.1", @@ -2367,12 +2367,12 @@ } }, "node_modules/react-router-dom": { - "version": "7.11.0", - "resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-7.11.0.tgz", - "integrity": "sha512-e49Ir/kMGRzFOOrYQBdoitq3ULigw4lKbAyKusnvtDu2t4dBX4AGYPrzNvorXmVuOyeakai6FUPW5MmibvVG8g==", + "version": "7.12.0", + "resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-7.12.0.tgz", + "integrity": "sha512-pfO9fiBcpEfX4Tx+iTYKDtPbrSLLCbwJ5EqP+SPYQu1VYCXdy79GSj0wttR0U4cikVdlImZuEZ/9ZNCgoaxwBA==", "license": "MIT", "dependencies": { - "react-router": "7.11.0" + "react-router": "7.12.0" }, "engines": { "node": ">=20.0.0" diff --git a/client/package.json b/client/package.json index 8839ddb..1455d83 100644 --- a/client/package.json +++ b/client/package.json @@ -1,6 +1,6 @@ { "name": "portos-client", - "version": "0.8.10", + "version": "0.9.19", "private": true, "type": "module", "scripts": { diff --git a/client/src/App.jsx b/client/src/App.jsx index 539d4e8..862b92d 100644 --- a/client/src/App.jsx +++ b/client/src/App.jsx @@ -7,7 +7,9 @@ import CreateApp from './pages/CreateApp'; import Templates from './pages/Templates'; import PromptManager from './pages/PromptManager'; import ChiefOfStaff from './pages/ChiefOfStaff'; +import Brain from './pages/Brain'; import Media from './pages/Media'; +import DigitalTwin from './pages/DigitalTwin'; // Lazy load heavier pages for code splitting // DevTools pages are large (~2300 lines total) so lazy load them @@ -46,6 +48,10 @@ export default function App() { } /> } /> } /> + } /> + } /> + } /> + } /> } /> } /> } /> diff --git a/client/src/components/AppTile.jsx b/client/src/components/AppTile.jsx index 8a802c0..741a8d4 100644 --- a/client/src/components/AppTile.jsx +++ b/client/src/components/AppTile.jsx @@ -69,7 +69,7 @@ const AppTile = memo(function AppTile({ app, onUpdate }) { {app.repoPath}

- {/* Actions */} + {/* Actions - min-h-10 ensures 40px touch target on mobile */}
{/* Open UI */} {appUrl && isOnline && ( @@ -77,7 +77,7 @@ const AppTile = memo(function AppTile({ app, onUpdate }) { href={appUrl} target="_blank" rel="noopener noreferrer" - className="px-3 py-1.5 text-sm rounded-lg bg-port-accent hover:bg-port-accent/80 text-white transition-colors" + className="px-4 py-2 min-h-10 text-sm rounded-lg bg-port-accent hover:bg-port-accent/80 text-white transition-colors inline-flex items-center" aria-label={`Open ${app.name} UI in new tab`} > Open UI @@ -89,7 +89,7 @@ const AppTile = memo(function AppTile({ app, onUpdate }) { + + {/* Directory Browser Dropdown */} + {isOpen && ( +
+ {/* Current Path Header */} +
+ + {currentPath} +
+ + {/* Navigation Controls */} +
+ {parentPath && ( + + )} + + +
+ + {/* Directory List */} +
+ {loading ? ( +
+ +
+ ) : error ? ( +
{error}
+ ) : directories.length === 0 ? ( +
No subdirectories
+ ) : ( +
+ {directories.map(dir => ( + + ))} +
+ )} +
+ + {/* Footer Help */} +
+ Double-click a folder to select it, or use "Select Current" +
+
+ )} +
+ ); +} diff --git a/client/src/components/FolderPicker.jsx b/client/src/components/FolderPicker.jsx new file mode 100644 index 0000000..50a0edc --- /dev/null +++ b/client/src/components/FolderPicker.jsx @@ -0,0 +1,175 @@ +import { useState, useEffect, useRef } from 'react'; +import { Folder, FolderOpen, ChevronUp, X, Check } from 'lucide-react'; +import * as api from '../services/api'; + +export default function FolderPicker({ value, onChange }) { + const [isOpen, setIsOpen] = useState(false); + const [currentPath, setCurrentPath] = useState(''); + const [parentPath, setParentPath] = useState(null); + const [directories, setDirectories] = useState([]); + const [loading, setLoading] = useState(false); + const modalRef = useRef(null); + + // Load directory contents + const loadDirectory = async (path = null) => { + setLoading(true); + const result = await api.getDirectories(path).catch(() => null); + if (result) { + setCurrentPath(result.currentPath); + setParentPath(result.parentPath); + setDirectories(result.directories || []); + } + setLoading(false); + }; + + // Load initial directory when opened + useEffect(() => { + if (isOpen) { + loadDirectory(value || null); + } + }, [isOpen]); + + // Close on click outside + useEffect(() => { + const handleClickOutside = (e) => { + if (modalRef.current && !modalRef.current.contains(e.target)) { + setIsOpen(false); + } + }; + + if (isOpen) { + document.addEventListener('mousedown', handleClickOutside); + return () => document.removeEventListener('mousedown', handleClickOutside); + } + }, [isOpen]); + + // Close on Escape + useEffect(() => { + const handleEscape = (e) => { + if (e.key === 'Escape') setIsOpen(false); + }; + + if (isOpen) { + document.addEventListener('keydown', handleEscape); + return () => document.removeEventListener('keydown', handleEscape); + } + }, [isOpen]); + + const handleSelect = () => { + onChange(currentPath); + setIsOpen(false); + }; + + const handleNavigate = (path) => { + loadDirectory(path); + }; + + return ( +
+ + + {isOpen && ( +
+
+ {/* Header */} +
+

Select Folder

+ +
+ + {/* Current Path */} +
+

+ {currentPath} +

+
+ + {/* Directory List */} +
+ {loading ? ( +
+ Loading... +
+ ) : ( +
+ {/* Go Up */} + {parentPath && ( + + )} + + {/* Directories */} + {directories.map((dir) => ( + + ))} + + {directories.length === 0 && !parentPath && ( +
+ No subdirectories +
+ )} +
+ )} +
+ + {/* Footer */} +
+ + +
+
+
+ )} +
+ ); +} diff --git a/client/src/components/Layout.jsx b/client/src/components/Layout.jsx index ab6dce6..a352443 100644 --- a/client/src/components/Layout.jsx +++ b/client/src/components/Layout.jsx @@ -20,7 +20,14 @@ import { ExternalLink, Crown, Play, - Camera + Camera, + Brain, + Heart, + Fingerprint, + Clock, + Calendar, + GraduationCap, + Settings } from 'lucide-react'; import packageJson from '../../package.json'; import Logo from './Logo'; @@ -30,6 +37,7 @@ import NotificationDropdown from './NotificationDropdown'; const navItems = [ { to: '/', label: 'Dashboard', icon: Home, single: true }, + { separator: true }, { label: 'AI Config', icon: Bot, @@ -39,9 +47,31 @@ const navItems = [ ] }, { to: '/apps', label: 'Apps', icon: Package, single: true }, - { to: '/media', label: 'Media', icon: Camera, single: true }, { href: '//:5560', label: 'Autofixer', icon: Wrench, external: true, dynamicHost: true }, - { to: '/cos', label: 'Chief of Staff', icon: Crown, single: true, showBadge: true }, + { + label: 'Chief of Staff', + icon: Crown, + showBadge: true, + children: [ + { to: '/cos/tasks', label: 'Tasks', icon: FileText }, + { to: '/cos/agents', label: 'Agents', icon: Cpu }, + { to: '/cos/scripts', label: 'Scripts', icon: Terminal }, + { to: '/cos/schedule', label: 'Schedule', icon: Clock }, + { to: '/cos/digest', label: 'Digest', icon: Calendar }, + { to: '/cos/learning', label: 'Learning', icon: GraduationCap }, + { to: '/cos/memory', label: 'Memory', icon: Brain }, + { to: '/cos/health', label: 'Health', icon: Activity }, + { to: '/cos/config', label: 'Config', icon: Settings } + ] + }, + { + label: 'Identity', + icon: Fingerprint, + children: [ + { to: '/brain', label: 'Brain', icon: Brain }, + { to: '/digital-twin', label: 'Digital Twin', icon: Heart } + ] + }, { label: 'Dev Tools', icon: Terminal, @@ -54,7 +84,8 @@ const navItems = [ { to: '/devtools/processes', label: 'Processes', icon: Activity }, { to: '/devtools/usage', label: 'Usage', icon: BarChart3 } ] - } + }, + { to: '/media', label: 'Media', icon: Camera, single: true } ]; const SIDEBAR_KEY = 'portos-sidebar-collapsed'; @@ -126,7 +157,14 @@ export default function Layout() { return false; }; - const renderNavItem = (item) => { + const renderNavItem = (item, index) => { + // Separator + if (item.separator) { + return ( +
+ ); + } + const Icon = item.icon; // External link @@ -222,16 +260,32 @@ export default function Layout() { title={collapsed ? item.label : undefined} >
- +
+ + {/* Badge for collapsed state on collapsible sections */} + {item.showBadge && unreadCount > 0 && collapsed && ( + + {unreadCount > 9 ? '9+' : unreadCount} + + )} +
{item.label}
- {!collapsed && ( - expandedSections[item.label] - ? - : - )} +
+ {/* Badge for expanded state on collapsible sections */} + {item.showBadge && unreadCount > 0 && !collapsed && ( + + {unreadCount > 9 ? '9+' : unreadCount} + + )} + {!collapsed && ( + expandedSections[item.label] + ? + : + )} +
{/* Children items */} @@ -385,8 +439,8 @@ export default function Layout() { {/* Main content */} -
- {location.pathname.startsWith('/cos') ? ( +
+ {location.pathname.startsWith('/cos') || location.pathname.startsWith('/brain') || location.pathname.startsWith('/digital-twin') ? ( ) : (
diff --git a/client/src/components/brain/constants.js b/client/src/components/brain/constants.js new file mode 100644 index 0000000..9ded360 --- /dev/null +++ b/client/src/components/brain/constants.js @@ -0,0 +1,103 @@ +import { MessageSquare, Database, Calendar, Shield, Users, FolderKanban, Lightbulb, ClipboardList, Settings } from 'lucide-react'; + +// Main navigation tabs +export const TABS = [ + { id: 'inbox', label: 'Inbox', icon: MessageSquare }, + { id: 'memory', label: 'Memory', icon: Database }, + { id: 'digest', label: 'Digest', icon: Calendar }, + { id: 'trust', label: 'Trust', icon: Shield }, + { id: 'config', label: 'Config', icon: Settings } +]; + +// Memory sub-tabs for entity types +export const MEMORY_TABS = [ + { id: 'people', label: 'People', icon: Users }, + { id: 'projects', label: 'Projects', icon: FolderKanban }, + { id: 'ideas', label: 'Ideas', icon: Lightbulb }, + { id: 'admin', label: 'Admin', icon: ClipboardList } +]; + +// Destination display info +export const DESTINATIONS = { + people: { + label: 'People', + icon: Users, + color: 'bg-purple-500/20 text-purple-400 border-purple-500/30' + }, + projects: { + label: 'Projects', + icon: FolderKanban, + color: 'bg-blue-500/20 text-blue-400 border-blue-500/30' + }, + ideas: { + label: 'Ideas', + icon: Lightbulb, + color: 'bg-yellow-500/20 text-yellow-400 border-yellow-500/30' + }, + admin: { + label: 'Admin', + icon: ClipboardList, + color: 'bg-green-500/20 text-green-400 border-green-500/30' + }, + unknown: { + label: 'Unknown', + icon: MessageSquare, + color: 'bg-gray-500/20 text-gray-400 border-gray-500/30' + } +}; + +// Inbox status colors +export const STATUS_COLORS = { + filed: 'bg-port-success/20 text-port-success border-port-success/30', + needs_review: 'bg-port-warning/20 text-port-warning border-port-warning/30', + corrected: 'bg-blue-500/20 text-blue-400 border-blue-500/30', + error: 'bg-port-error/20 text-port-error border-port-error/30' +}; + +// Project status colors +export const PROJECT_STATUS_COLORS = { + active: 'bg-green-500/20 text-green-400 border-green-500/30', + waiting: 'bg-yellow-500/20 text-yellow-400 border-yellow-500/30', + blocked: 'bg-red-500/20 text-red-400 border-red-500/30', + someday: 'bg-purple-500/20 text-purple-400 border-purple-500/30', + done: 'bg-gray-500/20 text-gray-400 border-gray-500/30' +}; + +// Admin status colors +export const ADMIN_STATUS_COLORS = { + open: 'bg-blue-500/20 text-blue-400 border-blue-500/30', + waiting: 'bg-yellow-500/20 text-yellow-400 border-yellow-500/30', + done: 'bg-green-500/20 text-green-400 border-green-500/30' +}; + +// Confidence thresholds for display +export const CONFIDENCE_COLORS = { + high: 'text-green-400', // >= 0.8 + medium: 'text-yellow-400', // >= 0.6 + low: 'text-red-400' // < 0.6 +}; + +export function getConfidenceColor(confidence) { + if (confidence >= 0.8) return CONFIDENCE_COLORS.high; + if (confidence >= 0.6) return CONFIDENCE_COLORS.medium; + return CONFIDENCE_COLORS.low; +} + +// Format relative time +export function formatRelativeTime(dateString) { + if (!dateString) return 'Never'; + + const date = new Date(dateString); + const now = new Date(); + const diffMs = now - date; + const diffMins = Math.floor(diffMs / 60000); + const diffHours = Math.floor(diffMs / 3600000); + const diffDays = Math.floor(diffMs / 86400000); + + if (diffMins < 1) return 'Just now'; + if (diffMins < 60) return `${diffMins}m ago`; + if (diffHours < 24) return `${diffHours}h ago`; + if (diffDays < 7) return `${diffDays}d ago`; + + return date.toLocaleDateString(); +} diff --git a/client/src/components/brain/index.js b/client/src/components/brain/index.js new file mode 100644 index 0000000..8f424d3 --- /dev/null +++ b/client/src/components/brain/index.js @@ -0,0 +1,8 @@ +// Constants +export * from './constants'; + +// Tab Components +export { default as InboxTab } from './tabs/InboxTab'; +export { default as MemoryTab } from './tabs/MemoryTab'; +export { default as DigestTab } from './tabs/DigestTab'; +export { default as TrustTab } from './tabs/TrustTab'; diff --git a/client/src/components/brain/tabs/ConfigTab.jsx b/client/src/components/brain/tabs/ConfigTab.jsx new file mode 100644 index 0000000..fccb30c --- /dev/null +++ b/client/src/components/brain/tabs/ConfigTab.jsx @@ -0,0 +1,373 @@ +import { useState, useEffect } from 'react'; +import * as api from '../../../services/api'; +import { + Settings, + RefreshCw, + Save, + Zap, + Clock, + Calendar, + TrendingUp, + CheckCircle +} from 'lucide-react'; +import toast from 'react-hot-toast'; + +const DAYS_OF_WEEK = [ + 'monday', + 'tuesday', + 'wednesday', + 'thursday', + 'friday', + 'saturday', + 'sunday' +]; + +export default function ConfigTab({ onRefresh }) { + const [loading, setLoading] = useState(true); + const [saving, setSaving] = useState(false); + const [settings, setSettings] = useState(null); + const [providers, setProviders] = useState([]); + const [selectedProvider, setSelectedProvider] = useState(''); + const [selectedModel, setSelectedModel] = useState(''); + const [confidenceThreshold, setConfidenceThreshold] = useState(0.6); + const [dailyDigestTime, setDailyDigestTime] = useState('09:00'); + const [weeklyReviewDay, setWeeklyReviewDay] = useState('sunday'); + const [weeklyReviewTime, setWeeklyReviewTime] = useState('16:00'); + + useEffect(() => { + fetchData(); + }, []); + + const fetchData = async () => { + const [settingsData, providersData] = await Promise.all([ + api.getBrainSettings().catch(() => null), + api.getProviders().catch(() => ({ providers: [] })) + ]); + + if (settingsData) { + setSettings(settingsData); + setSelectedProvider(settingsData.defaultProvider || ''); + setSelectedModel(settingsData.defaultModel || ''); + setConfidenceThreshold(settingsData.confidenceThreshold || 0.6); + setDailyDigestTime(settingsData.dailyDigestTime || '09:00'); + setWeeklyReviewDay(settingsData.weeklyReviewDay || 'sunday'); + setWeeklyReviewTime(settingsData.weeklyReviewTime || '16:00'); + } + + if (providersData) { + setProviders(providersData.providers || []); + } + + setLoading(false); + }; + + const handleSave = async () => { + setSaving(true); + + const updatedSettings = { + defaultProvider: selectedProvider, + defaultModel: selectedModel, + confidenceThreshold: parseFloat(confidenceThreshold), + dailyDigestTime, + weeklyReviewDay, + weeklyReviewTime + }; + + const result = await api.updateBrainSettings(updatedSettings).catch(err => { + toast.error(err.message || 'Failed to save settings'); + return null; + }); + + setSaving(false); + + if (result) { + toast.success('Settings saved successfully'); + setSettings(result); + onRefresh?.(); + } + }; + + const getAvailableModels = () => { + if (!selectedProvider) return []; + const provider = providers.find(p => p.id === selectedProvider); + return provider?.models || []; + }; + + const hasChanges = () => { + if (!settings) return false; + return ( + selectedProvider !== settings.defaultProvider || + selectedModel !== settings.defaultModel || + parseFloat(confidenceThreshold) !== settings.confidenceThreshold || + dailyDigestTime !== settings.dailyDigestTime || + weeklyReviewDay !== settings.weeklyReviewDay || + weeklyReviewTime !== settings.weeklyReviewTime + ); + }; + + if (loading) { + return ( +
+ +
+ ); + } + + return ( +
+ {/* Header */} +
+
+ +

Configuration

+
+ {hasChanges() && ( + + )} +
+ + {/* AI Provider & Model Section */} +
+
+ +

AI Provider & Model

+
+ +
+ {/* Provider Selection */} +
+ + +

+ Used for thought classification, digests, and reviews +

+
+ + {/* Model Selection */} +
+ + +

+ Select the model for AI operations +

+
+
+
+ + {/* Classification Settings */} +
+
+ +

Classification Settings

+
+ +
+ +
+ setConfidenceThreshold(parseFloat(e.target.value))} + className="flex-1 h-2 bg-port-bg rounded-lg appearance-none cursor-pointer accent-port-accent" + /> + + {(confidenceThreshold * 100).toFixed(0)}% + +
+

+ Thoughts classified below this confidence level will require manual review +

+
+
+ + {/* Schedule Settings */} +
+
+ +

Schedule Settings

+
+ +
+ {/* Daily Digest Time */} +
+ + setDailyDigestTime(e.target.value)} + className="w-full px-3 py-2 bg-port-bg border border-port-border rounded-lg text-white focus:outline-none focus:ring-2 focus:ring-port-accent" + /> +

+ Time when daily digest will be generated automatically +

+
+ + {/* Weekly Review Day */} +
+ + +

+ Day of the week for weekly review generation +

+
+ + {/* Weekly Review Time */} +
+ + setWeeklyReviewTime(e.target.value)} + className="w-full px-3 py-2 bg-port-bg border border-port-border rounded-lg text-white focus:outline-none focus:ring-2 focus:ring-port-accent" + /> +

+ Time when weekly review will be generated automatically +

+
+
+
+ + {/* Current Schedule Summary */} + {settings && ( +
+
+ +

Schedule Summary

+
+
+
+ + Daily digest at {dailyDigestTime} +
+
+ + Weekly review on {weeklyReviewDay.charAt(0).toUpperCase() + weeklyReviewDay.slice(1)}s at {weeklyReviewTime} +
+ {settings.lastDailyDigest && ( +
+ + Last digest: {new Date(settings.lastDailyDigest).toLocaleString()} +
+ )} + {settings.lastWeeklyReview && ( +
+ + Last review: {new Date(settings.lastWeeklyReview).toLocaleString()} +
+ )} +
+
+ )} + + {/* Save button at bottom for mobile */} + {hasChanges() && ( +
+ +
+ )} +
+ ); +} diff --git a/client/src/components/brain/tabs/DigestTab.jsx b/client/src/components/brain/tabs/DigestTab.jsx new file mode 100644 index 0000000..8ae09e0 --- /dev/null +++ b/client/src/components/brain/tabs/DigestTab.jsx @@ -0,0 +1,299 @@ +import { useState, useEffect, useCallback } from 'react'; +import * as api from '../../../services/api'; +import { + RefreshCw, + Play, + Calendar, + ChevronDown, + ChevronRight, + AlertCircle, + CheckCircle, + Clock +} from 'lucide-react'; +import toast from 'react-hot-toast'; + +import { formatRelativeTime } from '../constants'; + +export default function DigestTab({ onRefresh }) { + const [latestDigest, setLatestDigest] = useState(null); + const [latestReview, setLatestReview] = useState(null); + const [digestHistory, setDigestHistory] = useState([]); + const [reviewHistory, setReviewHistory] = useState([]); + const [loading, setLoading] = useState(true); + const [runningDigest, setRunningDigest] = useState(false); + const [runningReview, setRunningReview] = useState(false); + const [showDigestHistory, setShowDigestHistory] = useState(false); + const [showReviewHistory, setShowReviewHistory] = useState(false); + + const fetchData = useCallback(async () => { + const [digest, review, digests, reviews] = await Promise.all([ + api.getBrainLatestDigest().catch(() => null), + api.getBrainLatestReview().catch(() => null), + api.getBrainDigests().catch(() => []), + api.getBrainReviews().catch(() => []) + ]); + + setLatestDigest(digest); + setLatestReview(review); + setDigestHistory(digests.slice(1)); // Exclude latest + setReviewHistory(reviews.slice(1)); // Exclude latest + setLoading(false); + }, []); + + useEffect(() => { + fetchData(); + }, [fetchData]); + + const handleRunDigest = async () => { + setRunningDigest(true); + const result = await api.runBrainDigest().catch(err => { + toast.error(err.message || 'Failed to generate digest'); + return null; + }); + setRunningDigest(false); + + if (result) { + toast.success('Daily digest generated'); + fetchData(); + onRefresh?.(); + } + }; + + const handleRunReview = async () => { + setRunningReview(true); + const result = await api.runBrainReview().catch(err => { + toast.error(err.message || 'Failed to generate review'); + return null; + }); + setRunningReview(false); + + if (result) { + toast.success('Weekly review generated'); + fetchData(); + onRefresh?.(); + } + }; + + if (loading) { + return ( +
+ +
+ ); + } + + return ( +
+ {/* Daily Digest Section */} +
+
+
+ +

Daily Digest

+
+ +
+ + {latestDigest ? ( +
+
+ + Generated {formatRelativeTime(latestDigest.generatedAt)} + +
+ +

{latestDigest.digestText}

+ +
+ {/* Top Actions */} +
+

+ + Top Actions +

+
    + {latestDigest.topActions?.map((action, i) => ( +
  • โ€ข {action}
    • + ))} +
+
+ + {/* Stuck Thing */} +
+

+ + Stuck +

+

{latestDigest.stuckThing || 'Nothing stuck'}

+
+ + {/* Small Win */} +
+

+ + Small Win +

+

{latestDigest.smallWin || 'Keep going!'}

+
+
+
+ ) : ( +
+ +

No daily digest yet.

+

Click "Generate Now" to create your first digest.

+
+ )} + + {/* Digest History */} + {digestHistory.length > 0 && ( +
+ + + {showDigestHistory && ( +
+ {digestHistory.map(digest => ( +
+
+ + {formatRelativeTime(digest.generatedAt)} + +
+

{digest.digestText}

+
+ ))} +
+ )} +
+ )} +
+ + {/* Weekly Review Section */} +
+
+
+ +

Weekly Review

+
+ +
+ + {latestReview ? ( +
+
+ + Generated {formatRelativeTime(latestReview.generatedAt)} + +
+ +

{latestReview.reviewText}

+ +
+ {/* What Happened */} +
+

What Happened

+
    + {latestReview.whatHappened?.map((item, i) => ( +
  • โ€ข {item}
    • + ))} +
+
+ + {/* Suggested Actions */} +
+

Actions Next Week

+
    + {latestReview.suggestedActionsNextWeek?.map((action, i) => ( +
  • โ€ข {action}
    • + ))} +
+
+
+ + {/* Open Loops */} + {latestReview.biggestOpenLoops?.length > 0 && ( +
+

Biggest Open Loops

+
    + {latestReview.biggestOpenLoops.map((loop, i) => ( +
  • โ€ข {loop}
    • + ))} +
+
+ )} + + {/* Recurring Theme */} + {latestReview.recurringTheme && ( +
+

Pattern Noticed

+

{latestReview.recurringTheme}

+
+ )} +
+ ) : ( +
+ +

No weekly review yet.

+

Click "Generate Now" to create your first review.

+
+ )} + + {/* Review History */} + {reviewHistory.length > 0 && ( +
+ + + {showReviewHistory && ( +
+ {reviewHistory.map(review => ( +
+
+ + {formatRelativeTime(review.generatedAt)} + +
+

{review.reviewText}

+
+ ))} +
+ )} +
+ )} +
+
+ ); +} diff --git a/client/src/components/brain/tabs/InboxTab.jsx b/client/src/components/brain/tabs/InboxTab.jsx new file mode 100644 index 0000000..40f9f85 --- /dev/null +++ b/client/src/components/brain/tabs/InboxTab.jsx @@ -0,0 +1,586 @@ +import { useState, useEffect, useCallback, useRef } from 'react'; +import * as api from '../../../services/api'; +import { + Send, + RefreshCw, + AlertCircle, + CheckCircle, + Edit2, + ChevronDown, + ChevronRight, + ExternalLink, + Trash2, + Save, + X +} from 'lucide-react'; +import toast from 'react-hot-toast'; + +import { + DESTINATIONS, + STATUS_COLORS, + getConfidenceColor, + formatRelativeTime +} from '../constants'; + +export default function InboxTab({ onRefresh, settings }) { + const [inputText, setInputText] = useState(''); + const [sending, setSending] = useState(false); + const [entries, setEntries] = useState([]); + const [counts, setCounts] = useState({}); + const [loading, setLoading] = useState(true); + const [showNeedsReview, setShowNeedsReview] = useState(true); + const [showFiled, setShowFiled] = useState(true); + const [fixingId, setFixingId] = useState(null); + const [fixDestination, setFixDestination] = useState(''); + const [editingId, setEditingId] = useState(null); + const [editText, setEditText] = useState(''); + const [confirmingDeleteId, setConfirmingDeleteId] = useState(null); + const inputRef = useRef(null); + + const fetchInbox = useCallback(async () => { + const data = await api.getBrainInbox().catch(() => ({ entries: [], counts: {} })); + setEntries(data.entries || []); + setCounts(data.counts || {}); + setLoading(false); + }, []); + + useEffect(() => { + fetchInbox(); + }, [fetchInbox]); + + const handleSubmit = async (e) => { + e.preventDefault(); + if (!inputText.trim() || sending) return; + + setSending(true); + const result = await api.captureBrainThought(inputText.trim()).catch(err => { + toast.error(err.message || 'Failed to capture thought'); + return null; + }); + setSending(false); + + if (result) { + toast.success(result.message || 'Thought captured'); + setInputText(''); + fetchInbox(); + onRefresh?.(); + } + }; + + const handleResolve = async (entryId, destination) => { + const result = await api.resolveBrainReview(entryId, destination).catch(err => { + toast.error(err.message || 'Failed to resolve'); + return null; + }); + + if (result) { + toast.success(`Filed to ${destination}`); + fetchInbox(); + onRefresh?.(); + } + }; + + const handleRetry = async (entryId) => { + const result = await api.retryBrainClassification(entryId).catch(err => { + toast.error(err.message || 'Failed to retry'); + return null; + }); + + if (result) { + toast.success(result.message || 'Reclassified'); + fetchInbox(); + onRefresh?.(); + } + }; + + const handleFix = async (entryId) => { + if (!fixDestination) { + toast.error('Select a destination'); + return; + } + + const result = await api.fixBrainClassification(entryId, fixDestination).catch(err => { + toast.error(err.message || 'Failed to fix'); + return null; + }); + + if (result) { + toast.success(`Moved to ${fixDestination}`); + setFixingId(null); + setFixDestination(''); + fetchInbox(); + onRefresh?.(); + } + }; + + const handleEdit = (entry) => { + setEditingId(entry.id); + setEditText(entry.capturedText); + }; + + const handleSaveEdit = async (entryId) => { + if (!editText.trim()) { + toast.error('Text cannot be empty'); + return; + } + + const result = await api.updateBrainInboxEntry(entryId, editText.trim()).catch(err => { + toast.error(err.message || 'Failed to update'); + return null; + }); + + if (result) { + toast.success('Entry updated'); + setEditingId(null); + setEditText(''); + fetchInbox(); + onRefresh?.(); + } + }; + + const handleCancelEdit = () => { + setEditingId(null); + setEditText(''); + }; + + const handleDelete = async (entryId) => { + await api.deleteBrainInboxEntry(entryId).catch(err => { + toast.error(err.message || 'Failed to delete'); + return null; + }); + + toast.success('Entry deleted'); + setConfirmingDeleteId(null); + fetchInbox(); + onRefresh?.(); + }; + + const needsReviewEntries = entries.filter(e => e.status === 'needs_review'); + const filedEntries = entries.filter(e => e.status === 'filed' || e.status === 'corrected'); + const errorEntries = entries.filter(e => e.status === 'error'); + + if (loading) { + return ( +
+ +
+ ); + } + + return ( +
+ {/* Capture input */} +
+
+ setInputText(e.target.value)} + placeholder="One thought at a time..." + className="flex-1 px-4 py-3 bg-port-card border border-port-border rounded-lg text-white placeholder-gray-500 focus:outline-none focus:border-port-accent" + disabled={sending} + /> + +
+

+ Capture a thought. AI will classify and route it automatically. + {settings?.confidenceThreshold && ( + Confidence threshold: {Math.round(settings.confidenceThreshold * 100)}% + )} +

+
+ + {/* Needs Review section */} + {needsReviewEntries.length > 0 && ( +
+ + + {showNeedsReview && ( +
+ {needsReviewEntries.map(entry => ( +
+
+ {editingId === entry.id ? ( +
+