Skip to content

fix(audit): code audit, health check, and performance improvements#66

Open
dimakis wants to merge 17 commits intomainfrom
fix/multi-bug-sprint
Open

fix(audit): code audit, health check, and performance improvements#66
dimakis wants to merge 17 commits intomainfrom
fix/multi-bug-sprint

Conversation

@dimakis
Copy link
Owner

@dimakis dimakis commented Mar 12, 2026

Summary

  • Feature 41 — Code Audit: Comprehensive review of 270+ Python files (28,500 LOC) identifying code smells, error handling issues, modularity problems, and performance bottlenecks
  • P0/P1 fixes implemented: Mutable default arg, N+1 query elimination, boilerplate extraction, narrowed error handling, fast-model orchestrator classification, concurrent discovery sync
  • Feature 42 spec: Agent Memory Layer specification

Changes

Bug Fixes

  • Fix mutable default body: dict = {} in proposals YAML update endpoint (security/correctness)
  • Fix N+1 query in list_proposals — replaced per-status loop with single list_recent() query
  • Fix orchestrator session management — use async with get_session() instead of manual factory

Performance

  • Orchestrator uses fast-tier model for intent classification instead of user's frontier model
  • Discovery sync: concurrent automation/script config fetches via asyncio.gather() with semaphore (10 concurrent)

Refactoring

  • Extracted _inherited_model_context() helper — eliminated 45 lines of 3× duplication
  • Narrowed except Exception in all 6 agent tools to specific exception types with proper logging
  • Removed redundant logging import and duplicate env var in sandbox runner

Documentation

  • Feature 41 code audit spec and implementation plan
  • Feature 42 Agent Memory Layer spec

Test plan

  • All 3,372 unit tests pass
  • Coverage at 80.02% (meets 80% threshold)
  • make ci-local passes (format, lint, mypy, bandit, tests)

Made with Cursor

dimakis added 17 commits March 12, 2026 04:39
…stics

- Fix InsightType enum mismatch: replace _enum_column(String) with
  postgresql.ENUM(create_type=False) so SQLAlchemy emits correct type
  casts for insighttype/insightstatus/insightimpact columns
- Energy tariff: add server-side period fallback using TARIFF_SCHEDULE
  when HA entity is missing; fall back to period rate when current is 0
- Energy sensors: filter out aggregate/accumulative sensors in
  ConsumptionSummary (by entity_id pattern and source attribute)
- Diagnostics error log: align summary shape (add errors/warnings from
  counts_by_level); map known_patterns to frontend-expected fields
- Diagnostics traces: enrich response with run_name, job_type, title
- Jobs: persist descriptive titles as MLflow tag ha.job_title; prefer
  tag over formatted run_name; fix filter-before-sort with overfetch
- Activity panel: add onClick navigation to /jobs on job rows
- Settings: sync AppSettingsResponse type; use setQueryData for
  immediate cache update; add inline error display for mutations

Made-with: Cursor
Defines the guided automation builder agent feature that wires
Feature 36's existing automation builder workflow into the system
as a dedicated agent with chat preset, orchestrator routing, and
step-by-step progress in the activity panel.

Made-with: Cursor
Cover connection/auth, subscribe, event dispatch, exponential
backoff reconnection (doubling, ceiling), backoff reset on
successful connect, start_task lifecycle, stop/cancel, and
non-event message filtering.

Made-with: Cursor
Cover event queuing, per-entity debounce, batch upsert data
shape, DB error recovery (restore pending), automation state
triggering proposal sync, stats tracking, and flush-on-stop.

Made-with: Cursor
Cover threshold filtering (all impact levels), quiet hours
(same-day, midnight-crossing), batch vs single notification,
disabled state, from_settings factory with DB and fallback,
send failure graceful return, and _parse_time helper.

Made-with: Cursor
Add NotificationForm with enabled toggle, min_impact dropdown,
quiet hours start/end time inputs. Widen SettingsValue type to
include string for notification preferences. Grid expanded to
4 columns to accommodate new tab.

Made-with: Cursor
Add optimization.jobHistory() client function with status/limit
params, useJobHistory() React Query hook, and optimization.history
query key for F38 job history UI.

Made-with: Cursor
Add expandable Job History section with status filter dropdown
and clickable rows showing status badge, truncated job ID, date,
and insight/suggestion counts. Selecting a row loads it as the
active job in the existing detail card.

Made-with: Cursor
Apply contextlib.suppress, sort imports, add type annotations
for context manager functions, and use cast for SimpleNamespace
duck-typed Insight objects.

Made-with: Cursor
The Insight entity declares insighttype, insightimpact, and
insightstatus with create_type=False (managed by Alembic).
Integration tests use Base.metadata.create_all which skips
enum creation, causing "type does not exist" errors. Create
the enum types explicitly before table creation.

Made-with: Cursor
pgvector-based persistent memory for cross-session recall of user
preferences, decisions, corrections, and device patterns. Explicit
tool-call storage (no background extraction), semantic retrieval
injected into agent context, user management via UI and API.

Made-with: Cursor
Replace flat vector-only memory with a hybrid approach: MemoryNode
table with pgvector embeddings for semantic search, plus MemoryEdge
table for typed relationships to other memories and domain entities
(devices, areas, automations, insights). Graph traversal via recursive
CTEs, bounded to 2 hops. No external graph DB — all within PostgreSQL.

Made-with: Cursor
Comprehensive code audit across 270+ Python files (28,500 LOC):
- 7 oversized files needing modularization (P2-P3)
- 33 bare except-Exception instances swallowing errors (P0)
- N+1 queries, sequential I/O, expensive orchestrator model (P1)
- Positive: zero f-string logging, no TODOs, good security posture

Prioritized plan with T1-T14 tasks across 4 phases.

Made-with: Cursor
T1: Replace `body: dict = {}` (B006) with `Body(default={})` in the
YAML update endpoint — mutable default is a correctness bug.

T3: Replace per-status loop (N separate DB queries) with a single
`ProposalRepository.list_recent()` query for unfiltered listing.

Made-with: Cursor
T2+T4: Replace 3× duplicated 15-line model_context boilerplate with
a single `_inherited_model_context()` context manager.

Narrow every `except Exception` in agent tools from a generic catch-all
to specific `httpx.HTTPError | TimeoutError | ConnectionError` for
network errors (with logging), with a final `except Exception` that
logs at ERROR and returns a generic safe message instead of leaking
internal details.

Made-with: Cursor
… mgmt

T5: Force a fast-tier model (e.g. gpt-4o-mini) for intent classification
regardless of the user's selected model. Classification is simple JSON
output — using frontier models adds latency and cost for no benefit.

T7: Replace manual session factory + try/finally with
`async with get_session()` context manager. Also use the existing
`list_routable()` DAL method instead of filtering in Python.

Made-with: Cursor
T6: Replace sequential automation and script config HTTP fetches with
asyncio.gather() bounded by a semaphore (max 10 concurrent). For a
typical HA instance with 50+ automations, this reduces sync time from
O(n × latency) to O(latency + n/10 × latency).

Also: remove redundant `import logging` in sandbox runner and fix
duplicate MPLCONFIGDIR env var.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant