Repository navigation
fix: exact top-k answers keep the chosen series' labels - #794
Merged
Merged
Conversation
PromQL topk and bottomk answer whole input series. Stage 2's exact top-k implementation projected the limited rows to the logical [keys.., topk_k] columns and dropped the series identity, so `topk by (G) (k, ...)` (and its count and rate forms) answered with the G labels only. The Project now keeps $promql_series_identity after the logical columns when the ranked rows carry it: each answer has the chosen series' full label set, and readers of [keys.., topk_k] find those columns where they were. Tests: topk by (region) (2, sum_over_time(data[1m])) over five series answers the four chosen series with their full labels and hand-computed sums, equal to the query-time reference; every topk/bottomk form, grouped or not, over sums, counts and rates answers whole series. The example-1 fixture and its exact top-k shape test now include the column.
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
PromQL
topkandbottomkreturn whole input series.topk by (G) (k, …)and its count andrateforms returned the right values but only theGlabels. In the seed-862 workload-01 planq06-topk-sum.json, the root projection keeps onlyregion,serviceandtopk_4, so a runtime cannot tell which series each answer is.What
Exact top-k answers keep the chosen series' identity, so each answer carries that series' full label set. This holds for every
topkform, grouped or not.bottomkand generictopklower to a plain sort and limit with no projection, so they already kept it; the new test covers them as well.How
The column is dropped in Stage 2, not in the PromQL frontend. The frontend lowers these queries to an
Aggregate{[TopK]}whose logical schema is[keys.., topk_k]. Stage 2 implements exact top-k asLimit(Sort)and then adds aProjectback to[keys.., topk_k], and thatProjectdropped the series identity. It now also keeps$promql_series_identitywhenever the ranked rows carry it. The column goes after the logical columns, so a reader of[keys.., topk_k]finds them where they were. The executor reads an answer's labels from that identity.Key code interfaces added / modified
None. Only the exact top-k root's output schema gains a trailing
$promql_series_identitycolumn, which is visible in plan files, for exampleplanner-layering-example1.json.Usage (the test
topk_answers_carry_the_chosen_series_labels):Before this PR
The same query answered
{region=east} 60,{region=east} 30,{region=west} 9and{region=west} 3. The values were right, but the series labels were lost.After this PR
Each answer is the chosen input series with its value, as in Prometheus.
Verification
topk_answers_carry_the_chosen_series_labels(written first, failed onmain). It runs the query above end to end throughplan_installableand the executor and compares against the hand-computed answers andquery_time_reference. Onmainit failed withleft: {"region": "east"},right: {"instance": "e2", "region": "east", "service": "web"}.every_topk_form_answers_whole_series. It coverstopk(2, sum_over_time),topk by (region) (1, count_over_time),topk by (region) (1, rate),bottomk by (region) (1, sum_over_time)andbottomk(3, sum_over_time). Each answer must carry itsinstanceandservice, and the answers must equal the reference.tools/dag-viewer/examples/planner-layering-example1.jsonwas regenerated withstage_pipeline --example planner-layering-1. The shape teststage2_physical_asap_optimization_exact_topk_returns_job_and_topk_10now expects the trailing identity column.cargo test --workspace --locked --no-fail-fastwith the fixture already regenerated: 149 test binaries passed, and the one failure was that shape test. After updating it,planner_layering_example1passes (29 passed). Under the current speed-first policy,fmtandclippywere not run.SummaryEstimate{TopK}output is not executable yet (fix(ir): top-k sketch readouts return selected rows; Example 1 expectations follow planner output #579).