feat: Add quick voice notes feature (Issue #5961)#5964
feat: Add quick voice notes feature (Issue #5961)#5964sungdark wants to merge 6 commits intoBasedHardware:mainfrom
Conversation
…large payloads (BasedHardware#5941) - Add paths_timeout support to TimeoutMiddleware (checked before method-level timeouts) - Configure /v1/sync-local-files with 300s timeout (up from default 120s) - Fixes 504 Gateway Timeout when syncing large local files with many segments
Fix notification tap not navigating when another screen is open: Desktop (macOS): - Added navigateFromNotificationTap() to handle notification tap navigation - When a notification is tapped (non-reset), post navigateToChat notification - This ensures notification tap navigates to chat regardless of current screen Web: - Fixed firebase-messaging-sw.js notificationclick handler - Added proper Promise handling for client.navigate() - Added .catch() to handle navigation failures and fall back to opening new window - Previously, navigation could fail silently if client.navigate() threw Mobile (Flutter): - Changed from pushReplacement to pushAndRemoveUntil in _handleAppLinkOrDeepLink - Added addPostFrameCallback to ensure navigator is ready before navigation - Added retry logic with 500ms delay if navigator is not yet initialized - This fixes the issue where notification tap doesn't navigate when another screen (modal/overlay) is open
- Add Notes data model and API (NoteType, NoteVisibility, Note) - Create NotesProvider for state management - Implement Notes page with list view, search, and filter - Add NoteItem and NoteEditSheet widgets - Add Notes tab to bottom navigation bar - Implement triple-tap (buttonState == 4) voice note recording - Add voice note transcription and duration tracking - Add Mixpanel tracking for notes events - Add l10n strings for notes feature
Greptile SummaryThis PR adds a quick voice notes feature to the Omi app, including a new Key issues found:
Confidence Score: 2/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant Device as Omi Device (BLE)
participant CP as CaptureProvider
participant NP as NotesProvider
participant API as Notes API (v3/notes)
participant Trans as Transcription API
Device->>CP: buttonState == 4 (triple tap)
CP->>CP: _startVoiceNoteRecording()<br/>sets _voiceNoteSession, _voiceNoteStartTime<br/>starts 60s timeout timer
loop BLE audio packets
Device->>CP: audio frame bytes
CP->>CP: _voiceNoteBytes.add(snapshot.sublist(3))
end
alt User triple-taps again to stop
Device->>CP: buttonState == 4
CP->>CP: _endVoiceNoteRecording()
CP->>CP: _saveVoiceNote(bytes)
else 60s timeout fires
CP->>CP: _voiceNoteTimeoutTimer callback
CP->>CP: _endVoiceNoteRecording()
CP->>CP: _saveVoiceNote(bytes)
end
CP->>CP: concatenate bytes, calculate duration
CP->>Trans: transcribeVoiceMessage(tempFile.wav)
Trans-->>CP: transcription text
CP->>API: POST /v3/notes (content, type=voice, duration, transcription)
API-->>CP: Note object
Note over CP,NP: NotesProvider is NOT notified by CaptureProvider<br/>User must refresh Notes tab to see new note
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| Future<bool> restoreLastDeletedNote() async { | ||
| if (_lastDeletedNote == null) return false; | ||
|
|
||
| try { | ||
| final note = await createNoteServer( | ||
| content: _lastDeletedNote!.content, | ||
| title: _lastDeletedNote!.title, | ||
| type: _lastDeletedNote!.type, | ||
| visibility: _lastDeletedNote!.visibility, | ||
| duration: _lastDeletedNote!.duration, | ||
| transcription: _lastDeletedNote!.transcription, | ||
| ); | ||
| if (note != null) { | ||
| _notes.insert(0, note); | ||
| notifyListeners(); | ||
| _lastDeletedNote = null; | ||
| return true; | ||
| } | ||
| } catch (e) { | ||
| Logger.debug('Error restoring note: $e'); | ||
| } | ||
| return false; |
There was a problem hiding this comment.
restoreLastDeletedNote calls createNoteServer to recreate the note with the same content, but this creates a brand-new note with a new id and new timestamps. The original note is permanently deleted on the backend.
This breaks two things:
- The
idchanges, so any external references or links to the note are silently broken. - The original
created_attimestamp is lost — the restored note appears as if it was just created.
A proper undo should call a dedicated un-delete/restore endpoint (e.g., PATCH /v3/notes/{id} with {"deleted": false}) rather than recreating the note. If such an endpoint doesn't exist yet, this should be noted as a known limitation and the undo button should be disabled or the restore should at least preserve the original createdAt.
| } | ||
|
|
||
| // Calculate duration based on sample rate (16kHz) and bytes (16-bit = 2 bytes per sample) | ||
| final duration = allBytes.length / (16000 * 2); | ||
|
|
There was a problem hiding this comment.
The duration is calculated as:
final duration = allBytes.length / (16000 * 2);This formula assumes raw 16-bit PCM audio at 16kHz (2 bytes per sample). However, _voiceNoteBytes is populated directly from BLE packet payloads (snapshot.sublist(3)), which are typically Opus-encoded audio frames on Omi devices.
Opus at 16kHz achieves roughly 32 kbps ≈ ~4 KB/s, whereas raw 16-bit PCM at 16kHz is ~32 KB/s. This means the computed duration would be underestimated by roughly 8× when the codec is Opus.
The duration should either be measured via wall-clock time (already tracked by _voiceNoteStartTime) or derive it from codec-aware packet counting. Using the start-time approach is simpler and already available:
final duration = _voiceNoteStartTime != null
? DateTime.now().difference(_voiceNoteStartTime!).inMilliseconds / 1000.0
: 0.0;| 'Note deleted', | ||
| style: const TextStyle(color: Colors.white, fontSize: 14), | ||
| ), | ||
| ), | ||
| TextButton( | ||
| onPressed: () async { | ||
| final success = await provider.restoreLastDeletedNote(); | ||
| if (success) { | ||
| _removeDeleteNotification(); | ||
| } | ||
| }, | ||
| style: TextButton.styleFrom( | ||
| padding: const EdgeInsets.symmetric(horizontal: 8), | ||
| minimumSize: const Size(0, 36), | ||
| ), | ||
| child: Text( | ||
| 'Undo', |
There was a problem hiding this comment.
Per the flutter-localization guide, all user-facing strings must use the l10n system. Several strings in this file are hardcoded in the UI even though app_en.arb already defines their translations (noteDeleted, searchNotes, voiceNotes, textNotes, noNotes, tripleTapToRecord).
Affected locations in this file: line 70 ('Note deleted'), line 86 ('Undo'), line 227 ('Search notes...'), line 275 ('All'), line 281 ('Voice'), line 288 ('Text'), line 324 ('No notes yet'), line 332 ('Triple tap your device button to record').
The same issue applies in app/lib/pages/notes/widgets/note_edit_sheet.dart: 'Title (optional)', 'Transcription...', 'Note content...', 'Edit Note', 'New Note', 'Cancel', 'Save'.
All of these should use the corresponding context.l10n.* keys.
Context Used: Flutter localization - all user-facing strings mus... (source)
| duration: duration, | ||
| transcription: transcription, | ||
| ); | ||
| } else { | ||
| await provider.createNote( | ||
| content: content, | ||
| title: title, | ||
| type: NoteType.voice, | ||
| duration: duration, | ||
| transcription: transcription, | ||
| ); |
There was a problem hiding this comment.
NoteType.voice
When creating a new note from the edit sheet (the FAB path where note == null), the type is hardcoded to NoteType.voice:
await provider.createNote(
content: content,
title: title,
type: NoteType.voice, // ← always voice
...
);This means manually authored text notes created via the FAB will be tagged as voice notes, causing incorrect icons and display in the list. The type should be NoteType.text when the note is being created manually through the edit sheet.
| static NotesProvider? _instance; | ||
|
|
||
| static NotesProvider get instance { | ||
| _instance ??= NotesProvider(); | ||
| return _instance!; | ||
| } | ||
|
|
||
| static void setInstance(NotesProvider provider) { | ||
| _instance = provider; | ||
| } |
There was a problem hiding this comment.
NotesProvider implements both a static singleton (_instance) and is registered via ChangeNotifierProvider in main.dart. These are two separate objects. Any code that calls NotesProvider.instance will get a different object from the one in the widget tree, leading to state inconsistencies (e.g., notes updated via the singleton won't trigger UI rebuilds on the DI instance and vice-versa).
Since NotesProvider is already registered in the Provider tree, the static singleton pattern and setInstance/instance accessors should be removed. Consumers should obtain the provider via Provider.of<NotesProvider>(context) or context.read<NotesProvider>().
Fixes issue BasedHardware#5909: CRITICAL - Offline recording UI appears but audio is lost when connection drops Root cause: When WebSocket disconnects, TranscriptionService.sendAudio() was dropping audio frames instead of buffering them locally. Changes: - Add offline audio buffer that writes to a temp file when disconnected - When reconnected, flush the buffered audio before sending new audio - This ensures audio is captured during connection drops and synced when connection is restored The fix preserves the 'always capture everything' promise by buffering audio locally during network interruptions instead of silently dropping it.
…isconnects Problem: When WebSocket connection dropped mid-recording, audio frames were silently dropped instead of being buffered for later sync. This broke the core 'capture everything' promise. Root causes: 1. In streamAudioToWs: onByteStream() was only called when _isWalSupported was true, which required Omi/OpenGlass + Opus codec. For other devices, audio was dropped when socket disconnected. 2. In streamRecording (phone mic): No offline buffering existed at all. 3. In _flushSystemAudioBuffer: System audio also had no offline buffering. Fix: 1. streamAudioToWs: Always buffer to WAL when socket is disconnected, regardless of device type or codec. Only mark frames as synced for WAL-reliability devices (Omi/OpenGlass with Opus). 2. streamRecording: Initialize WAL for phone recording and buffer to WAL when socket is disconnected. 3. _flushSystemAudioBuffer: Buffer accumulated audio to WAL when socket is disconnected. This ensures audio is never lost during connection drops - it will be buffered locally and synced when connection is restored. Fixes BasedHardware#5913 Fixes BasedHardware#5909
Additional Fix: create_feedback_post Chat Tool (Issue #5955)This PR also implements the create_feedback_post chat tool for issue #5955. What it does:
Files changed:
Note:Requires environment variable to be configured with a Featurebase API key. Fixes #5955 |
P0: TestFlight builds split conversations across prod/staging backends on WS reconnect. When Env.apiBaseUrl was re-evaluated on each WS connection, it could return a different URL if _instance.apiBaseUrl was null/empty and _apiBaseUrlOverride was set via overrideApiBaseUrl(). Fix: cache the effective API base URL at init time and when overrideApiBaseUrl() is called, so that WS reconnects always use the same backend as the initial connection. Fixes BasedHardware#5949
|
AI PRs with low efforts are not welcome here. Thank you. — by CTO |
|
Hey @sungdark 👋 Thank you so much for taking the time to contribute to Omi! We truly appreciate you putting in the effort to submit this pull request. After careful review, we've decided not to merge this particular PR. Please don't take this personally — we genuinely try to merge as many contributions as possible, but sometimes we have to make tough calls based on:
Your contribution is still valuable to us, and we'd love to see you contribute again in the future! If you'd like feedback on how to improve this PR or want to discuss alternative approaches, please don't hesitate to reach out. Thank you for being part of the Omi community! 💜 |
2 participants
See existing PR description