Skip to content

feat: Add subquery support in pipeline#2323

Open
cherylEnkidu wants to merge 27 commits intomainfrom
cheryl/subquery
Open

feat: Add subquery support in pipeline#2323
cherylEnkidu wants to merge 27 commits intomainfrom
cheryl/subquery

Conversation

@cherylEnkidu
Copy link
Contributor

@cherylEnkidu cherylEnkidu commented Feb 24, 2026

feat: add support for Pipeline subqueries

This PR introduces support for Pipeline Subqueries, allowing users to perform complex data transformations and aggregations on subcollections within a pipeline.

Key Features

  • Subcollection Pipelines: Added Pipeline.subcollection(String path) to initiate a pipeline on a subcollection of the current document.
  • Variable Definition: Added Pipeline.define(...) to bind values to variables for reuse within the pipeline scope.
  • Expression Enhancements:
    • Expression.currentDocument(): Reference the current document in a pipeline stage.
    • Expression.variable(String name): Access defined variables.
    • Pipeline.toArrayExpression() and Pipeline.toScalarExpression(): Convert subquery results into array or scalar expressions for use in parent pipelines.

Testing

  • Added ITPipelineSubqueryTest.java containing integration tests for subquery functionality.
  • Verified compatibility with the Enterprise backend.

@cherylEnkidu cherylEnkidu requested review from a team as code owners February 24, 2026 19:09
@generated-files-bot
Copy link

generated-files-bot bot commented Feb 24, 2026

Warning: This pull request is touching the following templated files:

  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentProto.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/Value.java

@product-auto-label product-auto-label bot added size: l Pull request size is large. api: firestore Issues related to the googleapis/java-firestore API. labels Feb 24, 2026
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. size: l Pull request size is large. and removed size: l Pull request size is large. size: xl Pull request size is extra large. labels Feb 25, 2026
@cherylEnkidu cherylEnkidu marked this pull request as draft February 26, 2026 19:33
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Feb 26, 2026
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: xl Pull request size is extra large. labels Feb 26, 2026
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Feb 26, 2026
@cherylEnkidu cherylEnkidu marked this pull request as ready for review March 11, 2026 16:52
@cherylEnkidu cherylEnkidu changed the title Subquery feat: Add subquery support in pipeline Mar 24, 2026
Comment on lines +32 to +33
@InternalApi
public FunctionExpression(String name, List<? extends Expression> params) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we make this internal?

Comment on lines +82 to +90
bookDocs =
ImmutableMap.<String, Map<String, Object>>builder()
.put(
"book1",
ImmutableMap.<String, Object>builder()
.put("title", "The Hitchhiker's Guide to the Galaxy")
.put("author", "Douglas Adams")
.put("genre", "Science Fiction")
.put("published", 1979)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this is a copy of the bookDocs from the ITPipelineTest.java setup. Can you pull this setup code out from both of these places and put it into a function that we can reuse in both places? Maybe we could remove ~150 LOC from here.

Copy link
Contributor

@wu-hui wu-hui Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am skeptical about consolidating test setup here. While it serves to remove some duplication, we are talking about tests here, and having all test data ready in the same file has benefits to both human and agents.

I do want to see the multiple review collections we setup in the individual tests to be moved in here. That is, we setup both the books, and reviews or some other collections here. The number of documents can be shrinked, like 3,4 books and their reviews might be enough.

@@ -0,0 +1,36 @@
/*
* Copyright 2024 Google LLC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2026

@@ -0,0 +1,39 @@
/*
* Copyright 2025 Google LLC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2026

Comment on lines +82 to +90
bookDocs =
ImmutableMap.<String, Map<String, Object>>builder()
.put(
"book1",
ImmutableMap.<String, Object>builder()
.put("title", "The Hitchhiker's Guide to the Galaxy")
.put("author", "Douglas Adams")
.put("genre", "Science Fiction")
.put("published", 1979)
Copy link
Contributor

@wu-hui wu-hui Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am skeptical about consolidating test setup here. While it serves to remove some duplication, we are talking about tests here, and having all test data ready in the same file has benefits to both human and agents.

I do want to see the multiple review collections we setup in the individual tests to be moved in here. That is, we setup both the books, and reviews or some other collections here. The number of documents can be shrinked, like 3,4 books and their reviews might be enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: firestore Issues related to the googleapis/java-firestore API. size: xl Pull request size is extra large.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants