Skip to content

Wrong results: ORDER BY ... NULLS LAST on a running MAX window function puts the NULL row first #27267

Open

Description

@alamb

Describe the bug

A query that computes MAX(x) OVER (ORDER BY ts) and then sorts on that column with ASC NULLS LAST returns rows in the wrong order when x contains NULLs. The NULL row is returned first, where the query asked for it last.

The same wrong order appears for:

  • MIN(x) OVER (ORDER BY ts) sorted DESC NULLS LAST
  • MAX(x) OVER (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) sorted DESC NULLS FIRST

To Reproduce

set datafusion.execution.target_partitions = 1;

CREATE TABLE t(ts INT, x INT) AS VALUES (1, NULL), (2, 3), (3, 1), (4, 5);

SELECT ts, m
FROM (
  SELECT ts, MAX(x) OVER (ORDER BY ts) AS m
  FROM (SELECT * FROM t ORDER BY ts)
)
ORDER BY m ASC NULLS LAST;

Returns:

+----+------+
| ts | m    |
+----+------+
| 1  | NULL |     <-- NULL is first, not last!
| 2  | 3    |
| 3  | 3    |
| 4  | 5    |
+----+------+

Expected behavior

NULLS LAST means the NULL row comes after every non-NULL value:

+----+------+
| ts | m    |
+----+------+
| 2  | 3    |
| 3  | 3    |
| 4  | 5    |
| 1  | NULL |
+----+------+

Additional context

EXPLAIN shows the final sort on m is removed, so the planner believes the window output already satisfies m ASC NULLS LAST. Reproduced on main at 94486b1.

Found by @jayzhan211 while reviewing the EquivalenceProperties docs in #27250. The original report and repro are in this review comment: #27250 (comment)

Activity

  1. added
    bugSomething isn't working
    on Nov 9, 2026
  2. Surajsuthar commented on Nov 9, 2026

    @Surajsuthar
    Contributor

    take

  3. Surajsuthar commented on Nov 10, 2026

    @Surajsuthar
    Contributor

    untake

  4. itang06 commented on Nov 11, 2026

    @itang06
    Contributor

    take

  5. itang06 commented on Nov 11, 2026

    @itang06
    Contributor

    I reproduced all three cases and tried a fix that keeps the sort when the window result can be NULL. That fixes the wrong row order, but it also adds a sort to an existing test for a running SUM over a NOT NULL column.

    Is that trade-off okay for this fix, or would you prefer that I keep the optimization for cases where we can prove the result won’t be NULL?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions