Skip to content

[refactor](parser) Dispatch parser statements by first token - #66944

Open
morrySnow wants to merge 2 commits into
apache:masterfrom
morrySnow:codex/opt-antlr-statement-dispatch
Open

[refactor](parser) Dispatch parser statements by first token#66944
morrySnow wants to merge 2 commits into
apache:masterfrom
morrySnow:codex/opt-antlr-statement-dispatch

Conversation

@morrySnow

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: statementBase previously grouped unrelated statement families into one ANTLR adaptive prediction decision. This PR splits cross-prefix families and dispatches them through first-token-specific rules, changing generated statementBase() from one adaptivePredict call to a direct LA(1) switch. It also removes the redundant supported prefix from statement grammar rules and generated Context/visitor names; dispatcher rules use the *StatementDispatch suffix where the natural statement name is already occupied. All concrete grammar alternatives and FE behavior are preserved. Parent CST Context compatibility is intentionally out of scope because the standalone parser CST is not a public API.

Shared-host JMH rounds were rejected because they did not pass the predeclared noise gate, so this PR does not claim an unverified wall-clock improvement.

Release note

None

Check List (For Author)

  • Test
    • Unit Test
      • ./run-fe-ut.sh --run org.apache.doris.sqlparser.StatementBaseDispatchTest,org.apache.doris.sqlparser.DorisSqlParserTest,org.apache.doris.nereids.parser.NereidsParserTest
      • 163 tests passed: 73 dispatch, 7 parser, and 83 Nereids parser/visitor tests.
    • Manual test
      • mvn checkstyle:check -pl fe-core
      • DISABLE_BUILD_UI=ON ./build.sh --fe
      • FE reactor BUILD SUCCESS with 0 Checkstyle violations.
  • Behavior changed:
    • No.
  • Does this need documentation?
    • No.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: `statementBase` grouped unrelated statement families into one ANTLR adaptive prediction decision. Split cross-prefix families and dispatch them through first-token-specific rules so the generated `statementBase()` changes from one `adaptivePredict` call to a direct `LA(1)` switch. Preserve all concrete statement alternatives and FE visitor behavior; parent CST Context compatibility is intentionally out of scope because the standalone parser CST is not a public API. Shared-host JMH rounds were excluded because they did not pass the predeclared noise gate, so this change does not claim an unverified wall-clock improvement.

### Release note

None

### Check List (For Author)

- Test: Unit Test and Manual test
    - `./run-fe-ut.sh --run org.apache.doris.sqlparser.StatementBaseDispatchTest,org.apache.doris.sqlparser.DorisSqlParserTest,org.apache.doris.nereids.parser.NereidsParserTest`
    - `DISABLE_BUILD_UI=ON ./build.sh --fe`
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Rename statement grammar rules to remove the redundant `supported` prefix. Use `*StatementDispatch` for first-token family dispatchers where the natural statement name is occupied, and update generated-parser accessors, FE visitors, encryption handling, and README examples. SQL grammar alternatives and semantics are unchanged.

### Release note

None

### Check List (For Author)

- Test: Unit Test and Manual test
    - `./run-fe-ut.sh --run org.apache.doris.sqlparser.StatementBaseDispatchTest,org.apache.doris.sqlparser.DorisSqlParserTest,org.apache.doris.nereids.parser.NereidsParserTest`
    - `mvn checkstyle:check -pl fe-core`
    - `DISABLE_BUILD_UI=ON ./build.sh --fe`
- Behavior changed: No
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow morrySnow changed the title [refactor](fe) Dispatch parser statements by first token [refactor](parser) Dispatch parser statements by first token Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants