Skip to content

[feature](agg-state) Add aggregate state combine combinator - #66942

Open
HappenLee wants to merge 1 commit into
apache:masterfrom
HappenLee:feature/agg-state-combine
Open

[feature](agg-state) Add aggregate state combine combinator#66942
HappenLee wants to merge 1 commit into
apache:masterfrom
HappenLee:feature/agg-state-combine

Conversation

@HappenLee

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

Existing <aggregate>_state plus <aggregate>_union constructs and serializes one aggregate state for every input row before merging those states. For queries with tens of millions of raw rows, this per-row state materialization can erase much of the pre-aggregation benefit.

This PR adds a generic <aggregate>_combine(...) combinator. It feeds raw arguments directly into the nested aggregate function and emits one compatible AggState per group. For example, avg_combine(x) is aggregation-equivalent to avg_union(avg_state(x)), while avoiding a serialized AVG state for every source row. The resulting state remains consumable by the existing _union and _merge combinators.

The BE wrapper preserves the nested aggregate state layout and delegates raw accumulation, partial serialization, merge/deserialization, nullable handling, and fixed-length serialized column creation. Nereids resolves and translates the generic combinator and supports MV roll-up through the matching _union function. Zero-argument aggregates, DISTINCT, and aggregate ORDER BY expressions are rejected for now.

Release note

Add the experimental <aggregate>_combine(...) combinator for directly producing reusable aggregate states.

Check List (For Author)

  • Test
    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:

Regression coverage: test_agg_state_avg, including nullable input, all-null input, grouped partial aggregation, and compatibility with avg_union(avg_state(...)).

Unit coverage: AggregateStateCombineTest and CombineCombinatorTest. Full BE and FE builds also passed; clang-tidy reported no warnings for the changed BE files.

  • Behavior changed:

    • No.
    • Yes. Adds a new aggregate-state combinator.
  • Does this need documentation?

    • No. The function is covered by the existing experimental AggState feature.
    • Yes.

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Existing <aggregate>_state plus <aggregate>_union constructs and serializes one aggregate state for every input row before merging those states. Add <aggregate>_combine to aggregate raw inputs directly with the nested aggregate function and emit one compatible AggState, preserving partial aggregation and avoiding per-row state materialization.

### Release note

Add the experimental <aggregate>_combine(...) combinator for directly producing reusable aggregate states.

### Check List (For Author)

- Test:
    - Regression test: test_agg_state_avg
    - Unit Test: AggregateStateCombineTest and CombineCombinatorTest
- Behavior changed: Yes. Adds a new aggregate-state combinator.
- Does this need documentation: No. The function is covered by the existing experimental AggState feature.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@HappenLee

Copy link
Copy Markdown
Contributor Author

/review

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: All Codex review accounts are usage-limited; earliest retry is 2026-08-20T03:35:00Z.
Workflow run: https://github.com/apache/doris/actions/runs/32230645299

Please trigger /review again after that time.

@HappenLee

Copy link
Copy Markdown
Contributor Author

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants