Skip to content

Add hybrid function calling sample - #2855

Open
VinayGuthal wants to merge 2 commits into
masterfrom
function_calling
Open

VinayGuthal wants to merge 2 commits into
masterfrom
function_calling

Conversation

@VinayGuthal

Copy link
Copy Markdown
Contributor

Description

Adds a new sample screen to test function calling tools across on-device, hybrid (PREFER_ON_DEVICE), and cloud inference modes. This sample complements the function calling support added in firebase-android-sdk #8758.

Features

  • Tool Configurations:
    • Manual (FunctionDeclaration): Defines fetchWeather(city, state, date), inspects returned FunctionCallPart entries, executes the local tool, and sends FunctionResponsePart to resolve the multi-turn exchange.
    • Automatic (AutoFunctionDeclaration): Validates automatic tool invocation handled directly by the SDK.
    • Unsupported Tool (Tool.googleSearch()): Demonstrates validation behavior and automatic cloud fallback when non-function tools are used with hybrid inference.
    • No Tools: Baseline generation.
  • Inference Mode & Method Flexibility:
    • Supports all inference modes (ONLY_ON_DEVICE, PREFER_ON_DEVICE, ONLY_IN_CLOUD).
    • Supports unary (generateContent) and streaming (generateContentStream) generation methods.
  • UI & Diagnostics:
    • Displays on-device model status and download progress.
    • Inspects function calls with argument payloads and call IDs.
    • Highlights inference source (ON_DEVICE vs IN_CLOUD).
    • Detailed Logcat and diagnostic summary card.

Summary of Changes

  • gradle/libs.versions.toml:
    • Bumped firebase-ai to 18.2.0 and firebase-ai-ondevice to 16.0.0-beta08.
  • firebase-ai/app:
    • Added FunctionCallingViewModel.kt, FunctionCallingUiState.kt, and FunctionCallingScreen.kt under com.google.firebase.quickstart.ai.
    • Registered FUNCTION_CALLING in ScreenType (Sample.kt).
    • Added "Hybrid Function Calling" sample under HYBRID and TOOLS_FC categories in FirebaseAISamples.kt.
    • Added screen dispatch in MainActivity.kt.

Testing

  • Built and verified with ./gradlew :app:assembleDebug
  • Verified code style formatting with ./gradlew ktlintCheck

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new Hybrid Function Calling feature, adding the necessary ViewModel, UI screen, state representation, and navigation routing to support testing function calling tools across on-device, hybrid, and cloud inference modes. The feedback highlights opportunities to improve robustness and consistency, specifically by using safe casting to prevent potential exceptions during JSON parsing, ensuring the streaming generation path handles automatic function calling consistently with the unary path, and avoiding redundant model downloads in the UI when the selected option has not changed.

Comment on lines +285 to +302
if (toolOption == FunctionToolOption.MANUAL && detectedCalls.isNotEmpty()) {
val firstCall = detectedCalls.first()
val city = (firstCall.args["city"] as? JsonPrimitive)?.content ?: "Boston"
val state = (firstCall.args["state"] as? JsonPrimitive)?.content ?: "MA"
val date = (firstCall.args["date"] as? JsonPrimitive)?.content ?: "2024-10-17"
val weatherJson = WeatherRepository.fetchWeather(city, state, date)
executionResult = weatherJson.toString()

val followUp = model.generateContent(
content(role = "user") { text(promptText) },
content(role = "model") { part(firstCall) },
content(role = "user") {
part(FunctionResponsePart(firstCall.name, weatherJson, firstCall.id))
}
)
fullText = followUp.text.orEmpty()
source = followUp.inferenceSource
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In the STREAMING generation method, there is no handling for FunctionToolOption.AUTO to capture and set the executionResult when a function call is detected. This causes an inconsistency with the UNARY method where the execution result is captured and displayed in the UI. Adding an else if block for AUTO will resolve this.

                        if (toolOption == FunctionToolOption.MANUAL && detectedCalls.isNotEmpty()) {
                            val firstCall = detectedCalls.first()
                            val city = (firstCall.args["city"] as? JsonPrimitive)?.content ?: "Boston"
                            val state = (firstCall.args["state"] as? JsonPrimitive)?.content ?: "MA"
                            val date = (firstCall.args["date"] as? JsonPrimitive)?.content ?: "2024-10-17"
                            val weatherJson = WeatherRepository.fetchWeather(city, state, date)
                            executionResult = weatherJson.toString()

                            val followUp = model.generateContent(
                                content(role = "user") { text(promptText) },
                                content(role = "model") { part(firstCall) },
                                content(role = "user") {
                                    part(FunctionResponsePart(firstCall.name, weatherJson, firstCall.id))
                                }
                            )
                            fullText = followUp.text.orEmpty()
                            source = followUp.inferenceSource
                        } else if (toolOption == FunctionToolOption.AUTO && detectedCalls.isNotEmpty()) {
                            val firstCall = detectedCalls.first()
                            val city = (firstCall.args["city"] as? JsonPrimitive)?.content ?: "Boston"
                            val state = (firstCall.args["state"] as? JsonPrimitive)?.content ?: "MA"
                            val date = (firstCall.args["date"] as? JsonPrimitive)?.content ?: "2024-10-17"
                            executionResult = WeatherRepository.fetchWeather(city, state, date).toString()
                        }

Comment on lines +114 to +117
onClick = {
selectedModelOption = modelOpt
viewModel.checkAndDownloadModel(modelOpt.option)
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To avoid redundant network requests and job cancellations when the user taps an already selected model option, add a check to ensure checkAndDownloadModel is only called when the selection actually changes.

                    onClick = {
                        if (selectedModelOption != modelOpt) {
                            selectedModelOption = modelOpt
                            viewModel.checkAndDownloadModel(modelOpt.option)
                        }
                    },

@VinayGuthal
VinayGuthal requested a review from rlazo October 29, 2026 17:25
VinayGuthal added a commit to firebase/firebase-android-sdk that referenced this pull request Oct 30, 2026
Tested at firebase/quickstart-android#2855
### Description
Adds support for function calling when generating content using
on-device and hybrid (`PREFER_ON_DEVICE`) inference modes, following the
ML Kit GenAI Prompt APIs.

When function calling tools are supplied,
`OnDeviceGenerativeModelProvider` verifies that the underlying on-device
model supports function calling via `isFunctionCallingAvailable()`. If
function calling is not supported by the on-device model or if
unsupported non-function tools (such as Google Search or Code Execution)
are configured, a `FirebaseAIOnDeviceNotAvailableException` is raised,
triggering automatic fallback to cloud inference in `PREFER_ON_DEVICE`
mode.

### Summary of Changes

- **`gradle/libs.versions.toml`**:
- Updated `genaiPrompt` to `1.0.0-beta5` and `genaiSchema` to
`1.0.0-alpha2`.

- **`ai-logic/firebase-ai-ondevice-interop`**:
- Added `FunctionDeclaration` and `FunctionSchema` interop data
structures.
  - Added `FunctionCallPart` and `FunctionResponsePart` to `Part.kt`.
  - Added `functionCalls: List<FunctionCallPart>` to `Candidate`.
- Added `tools: List<FunctionDeclaration>?` to `GenerateContentRequest`.
- Added `suspend fun isFunctionCallingAvailable(): Boolean` to
`GenerativeModel`.
  - Updated `api.txt` and `CHANGELOG.md`.

- **`ai-logic/firebase-ai-ondevice`**:
- Implemented `GenerativeModelImpl.isFunctionCallingAvailable()`
delegating to ML Kit's
`getAvailableFeatures().contains(Feature.FUNCTION_CALLING)`.
- Added converters in `Converters.kt` for `FunctionCallPart`,
`FunctionResponsePart`, `FunctionDeclaration`, and `FunctionSchema`
to/from ML Kit equivalents.
- Mapped candidate function calls in
`GenerateContentResponse.toInterop()`.
  - Updated `api.txt` and `CHANGELOG.md`.
- Added unit test coverage in `GenerativeModelImplTest` and
`ConvertersTest`.

- **`ai-logic/firebase-ai`**:
- Added `tools: List<Tool>? = null` to `OnDeviceConfig` to allow
configuring tools for on-device inference, defaulting to model-level
tools when omitted.
- Forwarded tools configuration in `GenerativeModel` to
`OnDeviceGenerativeModelProvider`.
- Updated `Candidate.fromInterop` to convert interop function calls into
`FunctionCallPart` instances.
  - In `OnDeviceGenerativeModelProvider`:
- Checks `onDeviceModel.isFunctionCallingAvailable()` when function
declarations are provided.
- Throws `FirebaseAIOnDeviceNotAvailableException` if non-function tools
are passed to on-device inference or if function calling is unsupported.
- Converts `FunctionCallPart` and `FunctionResponsePart` in conversation
history.
- Maps function declarations and auto-function declarations into interop
schemas.
  - Updated `api.txt` and `CHANGELOG.md`.
- Added unit tests in `OnDeviceGenerativeModelProviderTests` covering
tool execution, candidate mapping, multi-turn history with function
responses, streaming support, and cloud fallback behavior.

### Testing
- `./gradlew :ai-logic:firebase-ai-ondevice-interop:check`
- `./gradlew :ai-logic:firebase-ai-ondevice:check`
- `./gradlew :ai-logic:firebase-ai:check`
- `./gradlew spotlessCheck`

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…i/feature/hybrid/FunctionCallingViewModel.kt

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant