Repository navigation
Escape backslashes in SQL string values for MySQL-like dialects - #1939
Conversation
PR Summary
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1939 +/- ##
============================================
+ Coverage 92.40% 92.41% +0.01%
- Complexity 3565 3570 +5
============================================
Files 348 348
Lines 7055 7065 +10
Branches 692 691 -1
============================================
+ Hits 6519 6529 +10
- Misses 370 371 +1
+ Partials 166 165 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
I got a feeling that this a "hack for a hack". |
|
Fair point. Right now the dialect only answers a yes/no question ( What do you think of moving it into the dialect itself, something like: // SqlDialect
public String escapeStringLiteral(String value) {
return switch (this) {
case BIGQUERY, CLICKHOUSE, INFOBRIGHT, MARIADB, MYSQL, SPARKSQL ->
value.replace("\\", "\\\\").replace("'", "\\'");
default -> value.replace("'", "''");
};
}Then If that sounds good I'll rework the PR that way. I can do the same for identifiers in #1940 ( |
|
Yes, this sounds good. |
|
Done. Each dialect now gets its escaping rule through the constructor ( |
asolntsev
left a comment
There was a problem hiding this comment.
Thanks for the PR! A few remarks inline.
One more place outside the diff: SqlTransformer.addValues() (around line 118) still wraps the SPARKSQL field prefix as quote + dialect.getFieldPrefix(...) + quote without escaping. A nested field named it's or a\b produces named_struct('it's', ...), which is invalid SQL. It should go through dialect.escapeStringLiteral(...) too.
MySQL, MariaDB, BigQuery, Spark SQL and the other dialects that escape a quote with a backslash also treat a lone backslash as an escape character. A value such as C:\temp\ was written as 'C:\temp\', where the last backslash escapes the closing quote and the statement is broken. Double the backslash in these dialects (and without a dialect, which uses the same backslash escaping). Dialects that double quotes are unchanged.
The backslash rule is now a constructor parameter of SqlDialect instead of a switch, and SqlTransformer asks the dialect to escape the value.
…ct behaviour, SNOWFLAKE backslash
fa0328b to
23c567d
Compare
|
Thanks, pushed 23c567d:
#1940 is rebased on top. |
asolntsev
left a comment
There was a problem hiding this comment.
Thanks, all points are addressed. Just two small things left inline.
|
Done, pushed: removed |
asolntsev
left a comment
There was a problem hiding this comment.
Thanks for addressing all the comments! LGTM.
Follow-up to #1936.
The dialects that escape a quote with a backslash (MySQL, MariaDB, BigQuery, Spark SQL, ClickHouse, Infobright) also treat a lone backslash as an escape character, but a backslash in a value was copied as is. A value like
C:\temp\came out aswhere the last backslash escapes the closing quote, so MySQL rejects the statement (and
\tturns into a tab).Now a backslash is doubled whenever the quote is escaped with a backslash, which gives
'C:\\temp\\'. That includes the no-dialect output, since it already uses\'. Dialects that double quotes (PostgreSQL, Oracle, SQL Server, H2, ...) don't treat backslash specially and are unchanged.Tests:
escapesBackslashInStringValues(per dialect) andescapesBackslashBeforeQuote(a\'bbecomes'a\\\'b'). Both fail without the change.SqlTestpasses (109 tests).