Skip to content

Post Resolution search visualizations#658

Closed
ngoiyaeric wants to merge 4 commits into
mainfrom
post-resolution-visualizations-14462340195639869331
Closed

Post Resolution search visualizations#658
ngoiyaeric wants to merge 4 commits into
mainfrom
post-resolution-visualizations-14462340195639869331

Conversation

@ngoiyaeric

Copy link
Copy Markdown
Collaborator

This PR enhances the resolution search visualizations by making them more contextual to user needs and queries.

Key changes:

  • Added mapboxImageLabel, googleImageLabel, and analysisFocus fields to the resolutionSearchSchema in lib/schema/resolution-search.ts.
  • Updated the resolutionSearch agent's system prompt in lib/agents/resolution-search.tsx to generate these labels and explicitly focus on user-drawn features.
  • Modified ResolutionCarousel in components/resolution-carousel.tsx to accept and display these dynamic labels.
  • Updated app/actions.tsx to pass the analysis results from the agent to the UI component.

PR created automatically by Jules for task 14462340195639869331 started by @ngoiyaeric

Updated resolution search schema, agent prompt, and carousel component to support dynamic, contextual labels based on AI analysis and user-drawn features.

Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@qodo-code-review

Copy link
Copy Markdown
Contributor

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@vercel

vercel Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
qcx Error Error Jul 9, 2026 2:20pm

@coderabbitai

coderabbitai Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@ngoiyaeric, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 50 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 21e1e970-4ff5-489b-b26f-b23d0dc7fb01

📥 Commits

Reviewing files that changed from the base of the PR and between 9e7cf1a and 6b87794.

⛔ Files ignored due to path filters (1)
  • server.log is excluded by !**/*.log
📒 Files selected for processing (7)
  • app/actions.tsx
  • components/header-search-button.tsx
  • components/map/map-3d.tsx
  • components/map/map-data-context.tsx
  • components/resolution-carousel.tsx
  • lib/agents/resolution-search.tsx
  • lib/schema/resolution-search.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch post-resolution-visualizations-14462340195639869331

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@ngoiyaeric

Copy link
Copy Markdown
Collaborator Author
image You see the labeling and box drawn on the right side, its random. It should be contextual

- Added critical guidelines to prevent generic or random labels.
- Instructed AI to prioritize user-drawn features and avoid hallucinated GeoJSON.
- Enhanced contextual labeling requirements to be highly specific.

Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
@ngoiyaeric

Copy link
Copy Markdown
Collaborator Author
Screenshot_20260608_161328_Chrome @coderabbitai why doesn't it know what its pointing at. The blue box and red dot are out of context

- Updated `resolutionSearchSchema` to require high-confidence and relevant GeoJSON features.
- Enhanced `resolutionSearch` agent system prompt with critical guidelines to avoid generic/random labels and hallucinations.
- Added strict contextual alignment instructions for user-drawn features.

Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Look at the resolution search agent system prompt and how marker/pointer coordinates are passed
cat $(fd -e tsx -e ts "resolution-search" | head -5)

echo "---actions.tsx---"
cat $(fd -e tsx "actions" --max-depth 3 | grep "app/actions" | head -1)

echo "---resolution-carousel.tsx---"
cat $(fd -e tsx "resolution-carousel" | head -1)

Length of output: 44767


🏁 Script executed:

rg -n "Geographic Coordinates" lib/agents/resolution-search.tsx

Length of output: 154


🏁 Script executed:

rg -n "resolution_search\|cursorLat\|cursorLng\|append.*action\|resolution_search" components/ --type tsx -A5 -B2
fd -e tsx "header-search-button" | xargs cat
fd -e tsx "search-button" | xargs cat 2>/dev/null | head -200

Length of output: 14899


🏁 Script executed:

fd -e tsx -e ts "map-data-context" | xargs cat

Length of output: 1821


🏁 Script executed:

# Find map click event handlers
rg -n "on.*[Cc]lick\|\.on\('click'\|\.on\(\"click\"" --type-add 'tsx:*.tsx' --type tsx -A4 -B2
rg -n "on.*[Cc]lick\|\.on\('click'\|\.on\(\"click\"" --type-add 'ts:*.ts' --type ts -A4 -B2 | head -100

Length of output: 473


🏁 Script executed:

# Find map click handlers using glob patterns
rg -n "onClick\|\.on\('click'\|\.on\(\"click\"\|map\.on\|'click'" components/map/ -A4 -B2 | head -120

# Also list all files in the map directory
fd . components/map/ -e tsx -e ts

Length of output: 627


🏁 Script executed:

# Read the most likely files to contain click handlers
echo "=== mapbox-map.tsx ==="
cat components/map/mapbox-map.tsx

echo "=== google-map.tsx ==="
cat components/map/google-map.tsx

echo "=== map-3d.tsx (first 100 lines) ==="
head -100 components/map/map-3d.tsx

Length of output: 26788



@ngoiyaeric Now I have the full picture. Neither mapbox-map.tsx nor map-3d.tsx has any click handler to capture where the user points — there's no cursorLocation tracked anywhere. Here's the complete 6-file fix:

Complete Fix: Wire cursor location from map click → agent

  • Add cursorLocation to the shared map context
  • Capture it on map click (Mapbox + Google 3D)
  • Forward it through FormData → server action → agent
  • Make the agent focus on the specific pointed-at coordinate

🧠 Learnings used
Learnt from: ngoiyaeric
Repo: QueueLab/QCX PR: 0
File: :0-0
Timestamp: 2026-04-25T08:10:37.673Z
Learning: In the QCX repository (feat/ai-sdk-ui-migration branch and going forward), there is no token generation/counting tracking and no multimodal LLM features. Base64 image data sent via `append()` body fields in components like `header-search-button.tsx` and `resolution-carousel.tsx` is for server-side `resolution_search` processing only — it is NOT passed as multimodal message content parts to a vision model. Avoid flagging token tracking or multimodal LLM handling gaps as review concerns in this codebase.

The requested changes have been implemented and a pull request has been created: View PR

Copy link
Copy Markdown
Collaborator Author

Closing this pull request as part of a bulk cleanup of pending PRs.

@ngoiyaeric ngoiyaeric closed this Jul 9, 2026
@ngoiyaeric ngoiyaeric reopened this Jul 9, 2026
@qodo-code-review

Copy link
Copy Markdown
Contributor

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@charliecreates charliecreates Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the diff and current checks (GitGuardian/Vercel green; license/cla is still pending).

Blocking feedback

  1. Contextual labels are only wired through the rehydration path, so fresh resolution_search responses still show fallback labels until a reload — app/actions.tsx#L833-L839
Non-blocking feedback (1)
  1. Unrelated local runtime output was committed in server.logserver.log#L5-L69
    This file now includes local dev requests/errors (including /test-carousel failures), which makes the PR noisy and increases avoidable merge churn. Consider dropping this file from the PR unless log snapshots are intentionally versioned for this feature.

If you'd like me to push fixes, reply with the item number(s) you want addressed (for example: please fix 1).

Comment thread app/actions.tsx
mapboxImage={mapboxImage}
googleImage={googleImage}
initialImage={image}
mapboxImageLabel={mapboxImageLabel}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new label props are only applied in getUIStateFromAIState (rehydration). The first streamed resolution_search response in submit() still renders ResolutionCarousel without mapboxImageLabel / googleImageLabel / analysisFocus (around lines 227-231), so users keep seeing fallback labels (MAPBOX, GOOGLE SATELLITE, MAP CAPTURE) until they reload the chat.

Suggested fix: once analysisResult resolves, update the streamed carousel with these three fields (or make those labels streamable values so the initial response updates in-place).

… resolution search accuracy. Here is a summary of the changes:

- Added `cursorLocation` tracking to `MapDataContext`, `MapboxMap`, and `Map3D`.
- Forwarded the `cursorLocation` through the `HeaderSearchButton` and the search execution flow.
- Updated the `resolutionSearch` agent to prioritize user clicks and specific points for geocoding and analysis.
- Refined the `resolutionSearchSchema` and system prompt to ensure high-confidence, contextual overlays and labels.
- Suppressed generic or "random" GeoJSON features in favor of highly relevant landmarks.

Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants