Skip to content

Add Golf Mode: AI caddie with GPS, scorecard, and club recommendations#16

Open
ivansene2025 wants to merge 35 commits intoIntent-Lab:mainfrom
ivansene2025:main
Open

Add Golf Mode: AI caddie with GPS, scorecard, and club recommendations#16
ivansene2025 wants to merge 35 commits intoIntent-Lab:mainfrom
ivansene2025:main

Conversation

@ivansene2025
Copy link

Summary

  • Adds Golf Mode to VisionClaw — Gemini AI acts as an intelligent caddie via Meta Ray-Ban glasses
  • Refactors isMeetingMode: BoolSessionMode enum (.normal, .meeting, .golf) across all files for clean multi-mode architecture
  • GPS coordinates injected to Gemini every 30s for course identification and local weather lookups
  • All golf intelligence flows through existing execute tool → OpenClaw (no new Gemini tools needed)
  • Full green-themed UI overlay: score-to-par badge, hole card HUD, pulsing mode pill

Changes

  • New: LocationManager.swift — CLLocationManager wrapper for GPS
  • New: GolfOverlayView.swift — Golf UI components (GolfOverlay, GolfModePill, ScoreToParBadge, GolfHoleCard, GolfModeButton)
  • Modified: GeminiConfig.swift — SessionMode enum + golf system instruction (AI caddie personality)
  • Modified: GeminiLiveService.swift — SessionMode support, sendTextContext() for GPS injection
  • Modified: GeminiSessionViewModel.swift — Golf lifecycle, GolfState, GPS timer, scorecard save
  • Modified: StreamView.swift — Golf overlay branch, GolfModeButton in controls, mode-aware audio-only display
  • Modified: NonStreamView.swift — Green "Golf Mode" button on home screen
  • Modified: GeminiOverlayView.swift — MeetingModeButton updated for SessionMode
  • Modified: Info.plist — NSLocationWhenInUseUsageDescription
  • Modified: project.pbxproj — New files registered
  • Also includes pending capture_and_send improvements (OpenClaw image upload, ToolCallRouter updates)

Test plan

  • Build succeeds with no errors
  • Normal AI mode works unchanged (tools, audio, transcription)
  • Meeting mode works unchanged (TEXT only, no tools, note-taking)
  • Tap "Golf Mode" → GPS permission prompt → Gemini connects with golf instruction
  • Voice: "what course am I at?" → AI calls execute with GPS → identifies course
  • Voice: "hole 1, par 4, I made 5" → AI updates scorecard via execute
  • Voice: "what club for 150 yards?" → AI calls execute for weather → recommends club
  • Stop session → scorecard saved to OpenClaw workspace

🤖 Generated with Claude Code

Kikinhochow and others added 30 commits February 27, 2026 12:48
…mendations

Refactors isMeetingMode bool to SessionMode enum (.normal, .meeting, .golf) across all files.
Golf mode uses Gemini + existing execute tool → OpenClaw for course lookup, weather, and scorecard
management. GPS coordinates injected every 30s via sendTextContext(). Includes full green-themed UI
overlay with score-to-par badge, hole card HUD, and pulsing mode pill.

New files: LocationManager.swift, GolfOverlayView.swift
Also includes pending capture_and_send improvements (OpenClaw image upload, ToolCallRouter updates).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Switch all modes to AUDIO modality (native audio model rejects TEXT).
  Meeting mode already suppresses playback; notes come via outputTranscription.
- Reset hasReceivedFirstFrame in stopAudioOnlySession() to prevent
  subsequent audio-only sessions showing a spinner instead of the UI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ode, WebRTC speed

Crash Fixes:
- Fix AVAudioEngine crash when audio format has sampleRate=0 after session reconfig
- Add format validation + retry with session reactivation before installTap
- Guard session start failures in golf/meeting/translation modes to prevent orphaned tasks
- Safer stopCapture() — check attachedNodes before detaching playerNode

Discord Session Notes:
- Add DiscordWebhookService with mode-specific embeds (meeting/golf/translation/normal)
- Snapshot session data before clearing state in stopSession()
- Show share prompt on session end, with OpenClaw fallback
- Add Discord webhook URL to Settings

Translation Mode:
- Rewrite system prompt: translate OTHER speakers only, ignore wearer's voice
- Fix audio routing: remove translation from backgroundMix path so audio plays through glasses
- Add TranslationOverlayView with subtitle-style display

WebRTC Live Streaming Speed:
- Start with STUN-only immediately, fetch TURN credentials in background
- Reduce TURN fetch timeout from 60s to 5s
- Pre-warm Fly.io signaling server on StreamView appear
- Reduce signaling WebSocket timeout from 30s to 10s

OpenClaw Connectivity:
- Add eager connection check on app appear (before session start)
- Fix discoverTunnelURL() — validate JSON response before parsing
- Add retry logic to ping()
- Show OpenClaw status pill when configured (even without active session)
- Add NetworkMonitor for WiFi/cellular detection

Other:
- Golf course API service for real course data
- Device status pill showing glasses connection state
- Location manager golf mode support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…aAccess/Settings/SettingsManager.swift,samples/CameraAccess/CameraAccess/WebRTC/SignalingClient.swift,samples/CameraAccess/CameraAccess/WebRTC/WebRTCSessionViewModel.swift (+2906 more)

Claude Code auto-commit at 2026-03-08 20:16:15

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…er.swift

Claude Code auto-commit at 2026-03-08 20:16:48

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…er.swift

Claude Code auto-commit at 2026-03-08 20:16:56

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…swift

Claude Code auto-commit at 2026-03-08 20:17:03

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…swift

Claude Code auto-commit at 2026-03-08 20:17:15

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…swift

Claude Code auto-commit at 2026-03-08 20:17:22

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…swift

Claude Code auto-commit at 2026-03-08 20:17:29

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…el.swift

Claude Code auto-commit at 2026-03-08 20:17:49

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ewModel.swift

Claude Code auto-commit at 2026-03-08 20:18:03

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ewModel.swift

Claude Code auto-commit at 2026-03-08 20:18:17

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ewModel.swift

Claude Code auto-commit at 2026-03-08 20:18:29

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ewModel.swift

Claude Code auto-commit at 2026-03-08 20:18:48

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:18:54

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:19:14

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:19:22

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…verlayView.swift

Claude Code auto-commit at 2026-03-08 20:20:19

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ewModel.swift

Claude Code auto-commit at 2026-03-08 20:20:27

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:20:53

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:21:00

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:21:08

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:21:15

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…r.swift

Claude Code auto-commit at 2026-03-08 20:24:24

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…er.swift

Claude Code auto-commit at 2026-03-08 20:24:48

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…swift

Claude Code auto-commit at 2026-03-08 20:25:24

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:25:37

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:25:47

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:25:57

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Kikinhochow and others added 5 commits March 8, 2026 20:26
Claude Code auto-commit at 2026-03-08 20:26:06

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:26:20

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:26:27

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:26:45

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code auto-commit at 2026-03-08 20:27:25

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant