Skip to content

feat(integrations): add Google Street View and Maps Platform integration - #7947

Open
shelbyjames24 wants to merge 2 commits into
google:mainfrom
shelbyjames24:feat/street-view-integration
Open

shelbyjames24 wants to merge 2 commits into
google:mainfrom
shelbyjames24:feat/street-view-integration

Conversation

@shelbyjames24

@shelbyjames24 shelbyjames24 commented Nov 2, 2026 •

Copy link
Copy Markdown

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

  • N/A (new integration)

2. Or, if no issue exists, describe the change:

Problem:
Multimodal ADK agents performing spatial and physical verification workflows (such as disputed proof-of-delivery audits, insurance claim triage, field-service inspections, and merchant storefront checks) face two recurring gaps:

  1. Cold-start visual baselines: Newly serviced addresses have zero historical ground-truth photos to compare against a submitted target photo.
  2. Property boundary ambiguity: Agents lack structured signal distinguishing SINGLE_FAMILY homes (tight porch/driveway boundaries) from MULTI_UNIT complexes (distributed drop-off locations across lobbies, mailrooms, and parcel lockers) or detecting omitted apartment/suite numbers.

Solution:
Adds a self-contained Google Street View and Maps Platform integration under src/google/adk/integrations/street_view/ (requiring zero new core dependencies in pyproject.toml):

  • StreetViewToolset (BaseToolset) & standalone ADK tools:
    • street_view_fallback_tool: Runs a 2-step Street View Static API pre-flight check (metadata endpoint with source="outdoor" first to avoid billing quota or downloading 404 placeholder tiles), computes forward compass bearing (heading) from the camera vehicle's GPS coordinates toward the target property, saves street_view_baseline.jpg as an ADK session artifact, and writes JSON-serializable metadata to tool_context.state.
    • property_classification_tool: Standardizes addresses via the Google Address Validation API (X-Goog-Api-Key header), classifies SINGLE_FAMILY vs. MULTI_UNIT, flags omitted sub-units (missingComponentTypes), and falls back gracefully to deterministic regex parsing on transient failures or unconfigured environments.
    • reverse_geocode_tool: Resolves raw GPS telemetry into physical street context with bounded LRU caching partitioned by a non-reversible key fingerprint.
  • GoogleMapsConfig: Resolves credentials from explicit api_key, MAPS_API_KEY, or lazy-imported Google Cloud Secret Manager (MAPS_API_KEY_SECRET_RESOURCE via google-adk[gcp]), with mTLS support via _mtls_utils and automatic API key redaction in diagnostic strings.
  • audit_photo_with_fallback: Two-stage Gemini Vision helper for synthesizing stable architectural baselines and scoring target imagery.
  • Documentation: Added integration README.md and Developer Unit Guide at docs/guides/integrations/street_view/index.md (indexed in docs/guides/README.md).

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.
$ pytest tests/unittests/integrations/street_view/
tests/unittests/integrations/street_view/test_street_view.py ................ [100%]
======================== 16 passed in 3.17s ========================

$ python3 scripts/compliance_checks.py src/google/adk/integrations/street_view/*.py tests/unittests/integrations/street_view/*.py
$ python3 scripts/check_docs.py
$ pyink --check src/google/adk/integrations/street_view/ tests/unittests/integrations/street_view/
All done! 8 files would be left unchanged.
$ isort --check src/google/adk/integrations/street_view/ tests/unittests/integrations/street_view/
$ mypy src/google/adk/integrations/street_view/
Success: no issues found in 6 source files

Manual End-to-End (E2E) Tests:

Enable Street View Static API, Address Validation API, and Geocoding API in a Google Cloud project, set MAPS_API_KEY and GOOGLE_API_KEY, and run:

import asyncio
from google.adk.agents import LlmAgent
from google.adk.integrations.street_view import StreetViewToolset
from google.adk.runners import InMemoryRunner


async def main():
  agent = LlmAgent(
      name="spatial_auditor",
      model="gemini-2.5-flash",
      instruction=(
          "Classify the address, reverse-geocode the coordinates, "
          "and capture a Street View baseline."
      ),
      tools=[StreetViewToolset()],
  )
  runner = InMemoryRunner(agent=agent, app_name="e2e_test")
  await runner.run_debug(
      "Audit 500 Terry A Francois Blvd, San Francisco, CA "
      "at GPS (37.7706, -122.3871)."
  )


asyncio.run(main())

Verify that property_classification_tool flags MULTI_UNIT with is_missing_subunit=True, reverse_geocode_tool populates state["reverse_geocoded_address"], and street_view_fallback_tool saves street_view_baseline.jpg as a session artifact with a valid Google Maps panorama deep link and computed heading.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

@shelbyjames24
shelbyjames24 force-pushed the feat/street-view-integration branch from 7d589eb to b881592 Compare November 4, 2026 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants