Skip to content

Support AggregateState in Parquet/Iceberg #2206

Description

@alex-zaitsev

We need to support AggregateState in Parquet/Iceberg. Since there is no such a datatype, it should be stored as a binary, but interpreted as AggregateState in ClickHouse.

We can test it as follows.

  1. Create AggregatingMergeTree with uniq() function.
  2. Insert some data
  3. Create a counterpart in Iceberg (ideally with CREATE TABLE, but external is also ok)
  4. Export PARTITION to Iceberg table
  5. Query uniq() from the source and Iceberg table, it should give the same results.

Same should work for SimpleAggregateFunction as well.

We should look how it is done in DataBricks, since it is for sure supports aggregate states

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions