Bug #4458

fix-id-token-cloud-run-auth.md

@@ -0,0 +1,119 @@
+# Fix: ID Token support for Cloud Run MCP auth
+
+## Why
+
+Cloud Run (and Cloud Functions, IAP, etc.) with "Require authentication" expects
+an **OIDC ID Token**, not an OAuth2 access token. The
+`ServiceAccountCredentialExchanger` always fetched an access token via
+`credentials.token`, so calls to authenticated Cloud Run services failed with
+`401 Unauthorized`.
+
+There was no way for users to ask for an ID token — the only workaround was
+monkey-patching the exchanger at runtime.
+
+## What changed
+
+### `ServiceAccount` model (`src/google/adk/auth/auth_credential.py`)
+
+Two new optional fields:
+
+- **`use_id_token`** (`bool`, default `False`) — when `True`, fetch an ID token
+  instead of an access token.
+- **`audience`** (`str`) — the target audience for the ID token, typically the
+  service URL. Required when `use_id_token` is `True`.
+
+`scopes` also got a default (`[]`) so callers using ID tokens don't have to pass
+an empty list.
+
+### `ServiceAccountCredentialExchanger` (`src/google/adk/tools/openapi_tool/auth/credential_exchangers/service_account_exchanger.py`)
+
+Added a `_fetch_id_token` helper and a branch at the top of `exchange_credential`
+that calls it when `use_id_token` is set. The existing access-token path is
+untouched.
+
+For default credentials it uses `google.oauth2.id_token.fetch_id_token()`. For
+explicit service-account JSON keys it uses
+`service_account.IDTokenCredentials.from_service_account_info()`.
+
+### Tests
+
+Four new tests covering: default-credential ID token, explicit-SA ID token,
+missing audience validation, and fetch failure handling. All existing tests still
+pass.
+
+## Usage
+
+```python
+auth_credential = AuthCredential(
+    auth_type=AuthCredentialTypes.SERVICE_ACCOUNT,
+    service_account=ServiceAccount(
+        use_default_credential=True,
+        use_id_token=True,
+        audience="https://my-service-xyz.us-central1.run.app/mcp",
+    ),
+)
+
+mcp_toolset = McpToolset(
+    connection_params=StreamableHTTPConnectionParams(url=MCP_URL),
+    auth_scheme=HTTPBearer(bearerFormat="JWT"),
+    auth_credential=auth_credential,
+)
+```
+
+## PR Description (ready to paste)
+
+### Summary
+
+Cloud Run-protected MCP endpoints require an OIDC ID token, but service-account
+exchange always returned an OAuth access token. This change adds an opt-in ID
+token path for service account auth and keeps existing access-token behavior as
+the default.
+
+### Testing Plan
+
+- Run formatter (`autoformat.sh` equivalent on Windows shell):
+  - `.\.venv\Scripts\python.exe -m isort src\google\adk\auth\auth_credential.py src\google\adk\tools\openapi_tool\auth\credential_exchangers\service_account_exchanger.py tests\unittests\tools\openapi_tool\auth\credential_exchangers\test_service_account_exchanger.py`
+  - `.\.venv\Scripts\python.exe -m pyink --config pyproject.toml src\google\adk\auth\auth_credential.py src\google\adk\tools\openapi_tool\auth\credential_exchangers\service_account_exchanger.py tests\unittests\tools\openapi_tool\auth\credential_exchangers\test_service_account_exchanger.py`
+- Run focused unit tests:
+  - `.\.venv\Scripts\python.exe -m pytest tests\unittests\tools\openapi_tool\auth\credential_exchangers\test_service_account_exchanger.py -q`
+- Run related broader tests:
+  - `.\.venv\Scripts\python.exe -m pytest tests\unittests\tools\openapi_tool\auth\ tests\unittests\auth\test_credential_manager.py tests\unittests\tools\mcp_tool\test_mcp_tool.py -q`
+
+### Unit Test Evidence
+
+- Focused exchanger tests: **11 passed in 1.59s**
+- Broader auth + MCP tests: **126 passed, 336 warnings in 2.64s**
+
+### Manual E2E Evidence (MCP + Cloud Run auth)
+
+> Note: This local workspace currently has no Cloud Run endpoint configured
+> (`MCP_URL` and `GOOGLE_CLOUD_PROJECT` are empty), so this section is prepared
+> for final evidence capture in your Cloud environment.
+
+Please attach in the PR:
+
+1. A screenshot of `adk web` prompt/response where the MCP tool call succeeds.
+2. Console logs proving successful authenticated call (no 401).
+3. The exact agent config used (`use_id_token=True`, `audience=<cloud-run-url>`).
+
+Suggested log snippet to include:
+
+```text
+HTTP Request: POST https://<service>.run.app/mcp "HTTP/1.1 200 OK"
+... tool call result ...
+```
+
+### Docs Impact
+
+- This introduces user-facing auth fields (`use_id_token`, `audience`) for
+  service-account auth flow.
+- Recommended follow-up: open/update a docs PR in `google/adk-docs` to document
+  Cloud Run authenticated MCP setup with ID token usage.
+
+### Review Request
+
+- Request review from ADK auth/tooling maintainers.
+- Suggested focus areas:
+  - Backward compatibility of service-account access-token flow.
+  - Correctness of ID-token exchange for ADC and explicit service-account keys.
+  - Error messaging when `audience` is missing.

src/google/adk/auth/auth_credential.py

@@ -147,8 +147,13 @@ class ServiceAccount(BaseModelWithConfig):
   """Represents Google Service Account configuration."""
 
   service_account_credential: Optional[ServiceAccountCredential] = None
-  scopes: List[str]
+  scopes: List[str] = []
   use_default_credential: Optional[bool] = False
+  # Set use_id_token=True and provide audience to fetch an OIDC ID token
+  # instead of an access token.  Required for identity-aware services like
+  # Cloud Run, Cloud Functions, or IAP-protected resources.
+  use_id_token: Optional[bool] = False
+  audience: Optional[str] = None
 
 
 class AuthCredentialTypes(str, Enum):

src/google/adk/tools/openapi_tool/auth/credential_exchangers/service_account_exchanger.py

@@ -20,6 +20,7 @@
 
 import google.auth
 from google.auth.transport.requests import Request
+from google.oauth2 import id_token as google_id_token
 from google.oauth2 import service_account
 import google.oauth2.credentials
 
@@ -51,12 +52,17 @@ def exchange_credential(
     to fetch an access token. Otherwise, the default service credential will be
     used for fetching an access token.
 
+    When ``use_id_token`` is set on the service account config, an OIDC ID
+    token is fetched instead.  This is needed for identity-aware services
+    like Cloud Run, Cloud Functions, or IAP-protected resources.
+
     Args:
         auth_scheme: The auth scheme.
         auth_credential: The auth credential.
 
     Returns:
-        An AuthCredential in HTTPBearer format, containing the access token.
+        An AuthCredential in HTTPBearer format, containing the access token
+        (or ID token when ``use_id_token`` is set).
     """
     if (
         auth_credential is None
@@ -73,6 +79,11 @@ def exchange_credential(
       )
 
     try:
+      # When use_id_token is set, fetch an OIDC ID token instead of an
+      # access token.  Cloud Run and similar services need this.
+      if auth_credential.service_account.use_id_token:
+        return self._fetch_id_token(auth_credential.service_account)
+
       if auth_credential.service_account.use_default_credential:
         credentials, project_id = google.auth.default(
             scopes=["https://www.googleapis.com/auth/cloud-platform"],
@@ -103,7 +114,42 @@ def exchange_credential(
       )
       return updated_credential
 
+    except AuthCredentialMissingError:
+      raise
     except Exception as e:
       raise AuthCredentialMissingError(
           f"Failed to exchange service account token: {e}"
       ) from e
+
+  def _fetch_id_token(self, sa_config) -> AuthCredential:
+    """Fetches an OIDC ID token for identity-aware services."""
+    audience = sa_config.audience
+    if not audience:
+      raise AuthCredentialMissingError(
+          "The `audience` field is required on ServiceAccount when"
+          " `use_id_token=True`. Set it to the URL of the target service"
+          " (e.g. 'https://my-service-xyz.run.app')."
+      )
+
+    try:
+      if sa_config.use_default_credential:
+        token = google_id_token.fetch_id_token(Request(), audience)
+      else:
+        id_creds = service_account.IDTokenCredentials.from_service_account_info(
+            sa_config.service_account_credential.model_dump(),
+            target_audience=audience,
+        )
+        id_creds.refresh(Request())
+        token = id_creds.token
+
+      return AuthCredential(
+          auth_type=AuthCredentialTypes.HTTP,
+          http=HttpAuth(
+              scheme="bearer",
+              credentials=HttpCredentials(token=token),
+          ),
+      )
+    except Exception as e:
+      raise AuthCredentialMissingError(
+          f"Failed to fetch ID token for audience '{audience}': {e}"
+      ) from e

tests/unittests/tools/openapi_tool/auth/credential_exchangers/test_service_account_exchanger.py

@@ -218,3 +218,124 @@ def test_exchange_credential_exchange_failure(
     service_account_exchanger.exchange_credential(auth_scheme, auth_credential)
   assert "Failed to exchange service account token" in str(exc_info.value)
   mock_from_service_account_info.assert_called_once()
+
+
+def test_exchange_id_token_with_default_credential(
+    service_account_exchanger, auth_scheme, monkeypatch
+):
+  """Test fetching an ID token using application default credentials."""
+  monkeypatch.setattr(
+      "google.adk.tools.openapi_tool.auth.credential_exchangers."
+      "service_account_exchanger.google_id_token.fetch_id_token",
+      MagicMock(return_value="mock_id_token"),
+  )
+
+  auth_credential = AuthCredential(
+      auth_type=AuthCredentialTypes.SERVICE_ACCOUNT,
+      service_account=ServiceAccount(
+          use_default_credential=True,
+          use_id_token=True,
+          audience="https://my-service-xyz.run.app",
+      ),
+  )
+
+  result = service_account_exchanger.exchange_credential(
+      auth_scheme, auth_credential
+  )
+
+  assert result.auth_type == AuthCredentialTypes.HTTP
+  assert result.http.scheme == "bearer"
+  assert result.http.credentials.token == "mock_id_token"
+
+
+def test_exchange_id_token_with_explicit_service_account(
+    service_account_exchanger, auth_scheme, monkeypatch
+):
+  """Test fetching an ID token using explicit service account credentials."""
+  mock_id_creds = MagicMock()
+  mock_id_creds.token = "mock_sa_id_token"
+  mock_id_creds.refresh = MagicMock()
+
+  monkeypatch.setattr(
+      "google.adk.tools.openapi_tool.auth.credential_exchangers."
+      "service_account_exchanger.service_account."
+      "IDTokenCredentials.from_service_account_info",
+      MagicMock(return_value=mock_id_creds),
+  )
+
+  auth_credential = AuthCredential(
+      auth_type=AuthCredentialTypes.SERVICE_ACCOUNT,
+      service_account=ServiceAccount(
+          service_account_credential=ServiceAccountCredential(
+              type_="service_account",
+              project_id="your_project_id",
+              private_key_id="your_private_key_id",
+              private_key="-----BEGIN PRIVATE KEY-----...",
+              client_email="...@....iam.gserviceaccount.com",
+              client_id="your_client_id",
+              auth_uri="https://accounts.google.com/o/oauth2/auth",
+              token_uri="https://oauth2.googleapis.com/token",
+              auth_provider_x509_cert_url=(
+                  "https://www.googleapis.com/oauth2/v1/certs"
+              ),
+              client_x509_cert_url=(
+                  "https://www.googleapis.com/robot/v1/metadata/x509/..."
+              ),
+              universe_domain="googleapis.com",
+          ),
+          scopes=["https://www.googleapis.com/auth/cloud-platform"],
+          use_id_token=True,
+          audience="https://my-service-xyz.run.app",
+      ),
+  )
+
+  result = service_account_exchanger.exchange_credential(
+      auth_scheme, auth_credential
+  )
+
+  assert result.auth_type == AuthCredentialTypes.HTTP
+  assert result.http.scheme == "bearer"
+  assert result.http.credentials.token == "mock_sa_id_token"
+  mock_id_creds.refresh.assert_called_once()
+
+
+def test_exchange_id_token_missing_audience(
+    service_account_exchanger, auth_scheme
+):
+  """Test that missing audience raises an error when use_id_token is True."""
+  auth_credential = AuthCredential(
+      auth_type=AuthCredentialTypes.SERVICE_ACCOUNT,
+      service_account=ServiceAccount(
+          use_default_credential=True,
+          use_id_token=True,
+          # audience intentionally omitted
+      ),
+  )
+
+  with pytest.raises(AuthCredentialMissingError) as exc_info:
+    service_account_exchanger.exchange_credential(auth_scheme, auth_credential)
+  assert "audience" in str(exc_info.value).lower()
+
+
+def test_exchange_id_token_failure(
+    service_account_exchanger, auth_scheme, monkeypatch
+):
+  """Test error handling when ID token fetch fails."""
+  monkeypatch.setattr(
+      "google.adk.tools.openapi_tool.auth.credential_exchangers."
+      "service_account_exchanger.google_id_token.fetch_id_token",
+      MagicMock(side_effect=Exception("metadata server unavailable")),
+  )
+
+  auth_credential = AuthCredential(
+      auth_type=AuthCredentialTypes.SERVICE_ACCOUNT,
+      service_account=ServiceAccount(
+          use_default_credential=True,
+          use_id_token=True,
+          audience="https://my-service-xyz.run.app",
+      ),
+  )
+
+  with pytest.raises(AuthCredentialMissingError) as exc_info:
+    service_account_exchanger.exchange_credential(auth_scheme, auth_credential)
+  assert "Failed to fetch ID token" in str(exc_info.value)