Skip to content

Spark: Parse typed numeric literal suffixes - #2912

Draft
andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:spark-numeric-literal-suffixes
Draft

andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:spark-numeric-literal-suffixes

Conversation

@andygrove

Copy link
Copy Markdown
Member

Spark SQL types a numeric literal by a case-insensitive suffix: Y, S and L after an integer, and F, D and BD after any number. SparkSqlDialect reads every suffix except an uppercase L as a column alias:

SELECT 1D, 1.5BD, 10Y, 1l
-- today: SELECT 1 AS D, 1.5 AS BD, 10 AS Y, 1 AS l

This adds Dialect::supports_numeric_literal_suffixes, enabled for Spark. The tokenizer then reads Y, S, F, D and BD into a new Token::SuffixedNumber, which parses to Value::SuffixedNumber(String, NumberSuffix). An L in either case still gives Value::Number(_, true). As in Spark, L, S and Y only follow an integer, and a suffix directly followed by an identifier character, as in 1Dx, is not one. Other dialects are unchanged.

Spark docs: https://spark.apache.org/docs/latest/sql-ref-literals.html#numeric-literal

This PR was prepared with AI assistance (Claude).

Dialects: Spark

Spark SQL types a numeric literal by a case-insensitive suffix: `Y`,
`S` and `L` after an integer, and `F`, `D` and `BD` after any number.
The tokenizer read `1D` as the number `1` followed by the alias `D`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant