Fix MySQL :copyfrom into mixed-case tables; ArgumentError for unknown isolation:; doc gaps

README.md

@@ -134,7 +134,7 @@ end
   transaction is over. Other query objects (the parent `queries`, another generated class sharing the pool) used from
   another fiber or thread check out their own connection and run outside the transaction: pass `tx` instead.
 * `isolation:` (`pg`, `mysql2`) takes `:read_uncommitted`, `:read_committed`, `:repeatable_read`, `:serializable`;
-  `mode:` (`sqlite3`) takes `:deferred`, `:immediate`, `:exclusive`. Other values raise before anything runs, and so
+  `mode:` (`sqlite3`) takes `:deferred`, `:immediate`, `:exclusive`. Other values raise `ArgumentError` before anything runs, and so
   do `isolation:` / `mode:` on a nested `transaction` (a savepoint cannot change them).
 * PostgreSQL: if a statement fails and the block rescues the error, PostgreSQL still rolls the whole transaction
   back, so `transaction` raises `PG::InFailedSqlTransaction` instead of pretending it committed. In a nested
@@ -166,6 +166,8 @@ server; duplicate keys and invalid values raise `Mysql2::Error` instead of being
 `INSERT OR REPLACE` / `OR IGNORE` / `ON CONFLICT ... DO ...` on SQLite, `INSERT IGNORE` (duplicates skipped) and
 `REPLACE` on MySQL. Clauses `COPY` / `LOAD DATA` cannot express (`ON CONFLICT` / `RETURNING` on PostgreSQL,
 `ON DUPLICATE KEY UPDATE` on MySQL) are generation errors; use `:batchexec` for those.
+* MySQL: `LOAD DATA` names the table and columns as the `INSERT` writes them (`` `MixedCase` ``): sqlc reports MySQL
+  names in lowercase, but table names are case-sensitive on Linux.
 * MySQL: `LOAD DATA` commits an open transaction that was started with raw SQL (`BEGIN`); call `:copyfrom` inside
   `transaction` instead.
 
@@ -195,14 +197,17 @@ server; duplicate keys and invalid values raise `Mysql2::Error` instead of being
 * SQLite follows its [type affinity rules](https://www.sqlite.org/datatype3.html#determination_of_column_affinity)
   (`VARCHAR(255)` → `String`, `DECIMAL(10,5)` → `Float`, `UNSIGNED BIG INT` → `Integer`); `JSON` and `UUID` columns are `String`s,
   also when SQLite stored numeric-looking text as a number (`'42'` → `"42"`, in SQLite's spelling: `'1.50'` → `"1.5"`).
-  The generated SQLite code uses `Monitor`: if you run Steep, add `library "monitor"` to your `Steepfile`.
+  The generated SQLite code uses `Monitor`: if you run Steep, add `library "monitor"` to your `Steepfile`. Code
+  generated for a `rubyVersion` below 3.3 also uses `Timeout` (see [Transactions](#transactions)): add `library "timeout"` too.
 
 ### Names
 * Query methods and keyword arguments are snake_cased, with acronyms kept together (`MIN_ID` → `min_id`,
   `HTTPServer` → `http_server`); Ruby keywords get a trailing `_` (`end_:`). Row members use the column spelling
   reported by sqlc for the engine, except names that conflict with row methods, including the private ones `Data` / `Struct` call while building and copying rows (`class_`, `hash_`, `initialize_`):
   PostgreSQL and SQLite keep `"camelCase"` as `:camelCase`, while MySQL reports it as `:camelcase`. A column named
-  `end` is the row member `:end` but the argument `end_:`.
+  `end` is the row member `:end` but the argument `end_:`. `Struct` rows (`rubyVersion` below 3.2) have more methods
+  than `Data` rows, so more names get the `_`: a column `count` is `:count` from 3.2 but `:count_` below
+  (`Struct#count` would otherwise be `Enumerable#count`); check the generated `.rbs` when changing `rubyVersion`.
 * A parameter used twice is one keyword when named with `sqlc.arg`; repeated `?` / `$n` of one column become `id:`, `id_2:`.
 * Models may be named like Ruby or driver constants (tables `exceptions`, `data`, `threads`, `sets`, ...): the
   generated code and `.rbs` refer to those as `::Exception`, `::Data`, `::Thread`, ... so the models cannot shadow them.
@@ -212,6 +217,9 @@ server; duplicate keys and invalid values raise `Mysql2::Error` instead of being
 ### Limitations
 * Two result columns with the same name (e.g. `a.id, b.id`) are an error: alias one (`b.id AS book_id`) or use `sqlc.embed`.
 * SQLite's `NUMERIC` affinity can return an `Integer` where `Float` is declared.
+* SQLite: a statement takes at most 32,766 parameters (`SQLITE_MAX_VARIABLE_NUMBER`), so a `sqlc.slice` with more
+  elements raises `SQLite3::SQLException: too many SQL variables`; on MySQL the limit is the server's
+  `max_allowed_packet`. Load larger sets into a table (`:copyfrom`) and join it instead.
 * `:batchexec` / `:batchone` / `:batchmany` are not atomic on their own: wrap them in `transaction` for all-or-nothing.
 * SQLite: sqlc does not see parameters in `ON CONFLICT ... DO UPDATE SET` (it leaves `sqlc.arg(...)` in the SQL or
   reports fewer parameters than there are `?`), so such queries are generation errors. Use `excluded.<column>`

docs/Coverage.md

@@ -24,7 +24,7 @@ only when the behavior is rejected or decided during generation and generated co
 | Names: keywords, method collisions (also private row hooks such as `initialize`), acronyms, quoted mixed case, repeated `$n`, plural models | [`test_keyword_and_method_names_keep_distinct_row_and_argument_names`](../tests/end2end_pg.rb#test_keyword_and_method_names_keep_distinct_row_and_argument_names), [`test_repeated_column_parameters_get_distinct_names`](../tests/end2end_pg.rb#test_repeated_column_parameters_get_distinct_names) | [`test_keyword_and_method_names_keep_distinct_row_and_argument_names`](../tests/end2end_mysql2.rb#test_keyword_and_method_names_keep_distinct_row_and_argument_names), [`test_repeated_positional_parameters_get_distinct_names`](../tests/end2end_mysql2.rb#test_repeated_positional_parameters_get_distinct_names) | [`test_keyword_and_method_names_keep_distinct_row_and_argument_names`](../tests/end2end_sqlite3.rb#test_keyword_and_method_names_keep_distinct_row_and_argument_names) | [`test_snake_case_keeps_acronyms_together`](../test/features_test.rb#test_snake_case_keeps_acronyms_together), [`test_query_names_cannot_replace_generated_methods`](../test/features_test.rb#test_query_names_cannot_replace_generated_methods), [`test_tables_with_the_same_singular_get_distinct_models`](../test/features_test.rb#test_tables_with_the_same_singular_get_distinct_models), [`test_columns_named_like_row_methods_still_build_and_read_rows`](../test/features_test.rb#test_columns_named_like_row_methods_still_build_and_read_rows) |
 | Connections, injected pools, reconnect behavior and SQLite serialization | [`test_queries_can_share_an_injected_pool`](../tests/end2end_pg.rb#test_queries_can_share_an_injected_pool), [`test_a_connection_closed_by_the_server_is_reconnected`](../tests/end2end_pg.rb#test_a_connection_closed_by_the_server_is_reconnected), [`test_connection_lost_inside_transaction_is_not_reconnected_until_the_transaction_ends`](../tests/end2end_pg.rb#test_connection_lost_inside_transaction_is_not_reconnected_until_the_transaction_ends) | [`test_queries_can_share_an_injected_pool`](../tests/end2end_mysql2.rb#test_queries_can_share_an_injected_pool), [`test_a_connection_closed_by_the_server_is_replaced`](../tests/end2end_mysql2.rb#test_a_connection_closed_by_the_server_is_replaced), [`test_a_stopped_async_task_does_not_leave_a_closed_connection_in_the_pool`](../tests/end2end_mysql2.rb#test_a_stopped_async_task_does_not_leave_a_closed_connection_in_the_pool) | [`test_fibers_sharing_a_database_take_turns`](../tests/end2end_sqlite3.rb#test_fibers_sharing_a_database_take_turns), [`test_threads_sharing_a_database_take_turns_in_transactions`](../tests/end2end_sqlite3.rb#test_threads_sharing_a_database_take_turns_in_transactions), [`test_a_database_returning_hash_rows_still_builds_rows`](../tests/end2end_sqlite3.rb#test_a_database_returning_hash_rows_still_builds_rows) | [`test_engine_must_match_the_driver`](../test/features_test.rb#test_engine_must_match_the_driver) |
 | Transaction result, commit/rollback, nested savepoint, exit semantics and rescued PostgreSQL error | [`test_nested_savepoint_rolls_back_while_outer_transaction_commits`](../tests/end2end_pg.rb#test_nested_savepoint_rolls_back_while_outer_transaction_commits), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_pg.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version), [`test_transaction_whose_failed_statement_was_rescued_raises_instead_of_committing`](../tests/end2end_pg.rb#test_transaction_whose_failed_statement_was_rescued_raises_instead_of_committing) | [`test_nested_savepoint_rolls_back_while_outer_transaction_commits`](../tests/end2end_mysql2.rb#test_nested_savepoint_rolls_back_while_outer_transaction_commits), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_mysql2.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version) | [`test_nested_transaction_rolls_back_to_a_savepoint`](../tests/end2end_sqlite3.rb#test_nested_transaction_rolls_back_to_a_savepoint), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_sqlite3.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version) | [`test_nested_transactions_reject_isolation_and_mode`](../test/features_test.rb#test_nested_transactions_reject_isolation_and_mode) |
-| Isolation applied; invalid isolation/mode rejected before begin; SQLite locks | [`test_transaction_isolation_level`](../tests/end2end_pg.rb#test_transaction_isolation_level) | [`test_transaction_isolation_level_is_applied`](../tests/end2end_mysql2.rb#test_transaction_isolation_level_is_applied) | [`test_transaction_mode`](../tests/end2end_sqlite3.rb#test_transaction_mode), [`test_immediate_and_exclusive_transactions_block_a_second_connection`](../tests/end2end_sqlite3.rb#test_immediate_and_exclusive_transactions_block_a_second_connection), [`test_unknown_transaction_mode_is_rejected_before_beginning`](../tests/end2end_sqlite3.rb#test_unknown_transaction_mode_is_rejected_before_beginning) | [`test_nested_transactions_reject_isolation_and_mode`](../test/features_test.rb#test_nested_transactions_reject_isolation_and_mode) |
+| Isolation applied; invalid isolation/mode rejected with `ArgumentError` before begin; SQLite locks | [`test_transaction_isolation_level`](../tests/end2end_pg.rb#test_transaction_isolation_level) | [`test_transaction_isolation_level_is_applied`](../tests/end2end_mysql2.rb#test_transaction_isolation_level_is_applied) | [`test_transaction_mode`](../tests/end2end_sqlite3.rb#test_transaction_mode), [`test_immediate_and_exclusive_transactions_block_a_second_connection`](../tests/end2end_sqlite3.rb#test_immediate_and_exclusive_transactions_block_a_second_connection), [`test_unknown_transaction_mode_is_rejected_before_beginning`](../tests/end2end_sqlite3.rb#test_unknown_transaction_mode_is_rejected_before_beginning) | [`test_nested_transactions_reject_isolation_and_mode`](../test/features_test.rb#test_nested_transactions_reject_isolation_and_mode) |
 | Logger debug format, transaction logs, secret-free parameters | [`test_logger_receives_query_names_and_durations`](../tests/end2end_pg.rb#test_logger_receives_query_names_and_durations) | [`test_logger_receives_query_names_and_durations`](../tests/end2end_mysql2.rb#test_logger_receives_query_names_and_durations) | [`test_logger_receives_query_names_and_durations`](../tests/end2end_sqlite3.rb#test_logger_receives_query_names_and_durations) | [`test_logger_support`](../test/features_test.rb#test_logger_support) |
 | Fiber scheduler concurrency / MySQL no prepared-statement leaks | [`test_queries_run_concurrently_under_the_fiber_scheduler`](../tests/end2end_pg.rb#test_queries_run_concurrently_under_the_fiber_scheduler) | [`test_generated_queries_overlap_under_sync_including_transactions`](../tests/end2end_mysql2.rb#test_generated_queries_overlap_under_sync_including_transactions), [`test_generated_queries_do_not_create_server_side_prepared_statements`](../tests/end2end_mysql2.rb#test_generated_queries_do_not_create_server_side_prepared_statements) | [`test_fibers_sharing_a_database_take_turns`](../tests/end2end_sqlite3.rb#test_fibers_sharing_a_database_take_turns) | |
 | Duplicate result columns, unsupported options, invalid `:copyfrom` clauses and generation errors | E2E cannot reach generation-time rejection | E2E cannot reach generation-time rejection | E2E cannot reach generation-time rejection | [`test_duplicate_result_columns_are_rejected`](../test/features_test.rb#test_duplicate_result_columns_are_rejected), [`test_copyfrom_keeps_or_rejects_conflict_clauses`](../test/features_test.rb#test_copyfrom_keeps_or_rejects_conflict_clauses), [`test_unknown_options_are_rejected_with_a_hint`](../test/options_test.rb#test_unknown_options_are_rejected_with_a_hint), [`test_query_names_cannot_replace_generated_methods`](../test/features_test.rb#test_query_names_cannot_replace_generated_methods) |
@@ -43,6 +43,7 @@ CI runs `test/coverage_map_test.rb`, which checks every linked test method exist
 | Interrupts at `BEGIN` / `COMMIT` (`Thread#raise`, raised by the statement) roll back and leave no open transaction | [`test_interrupts_around_begin_and_commit_never_leave_the_connection_in_a_transaction`](../tests/end2end_pg.rb#test_interrupts_around_begin_and_commit_never_leave_the_connection_in_a_transaction) | [`test_interrupts_around_begin_and_commit_never_leave_the_connection_in_a_transaction`](../tests/end2end_mysql2.rb#test_interrupts_around_begin_and_commit_never_leave_the_connection_in_a_transaction) | [`test_interrupts_around_begin_and_commit_never_leave_the_database_in_a_transaction`](../tests/end2end_sqlite3.rb#test_interrupts_around_begin_and_commit_never_leave_the_database_in_a_transaction) | |
 | PostgreSQL statements dropped by `DEALLOCATE ALL` / `DISCARD ALL` are prepared again | [`test_statements_are_prepared_again_after_deallocate_all`](../tests/end2end_pg.rb#test_statements_are_prepared_again_after_deallocate_all) | not MySQL (no server-side prepared statements) | not SQLite | [`test_statements_are_named_after_the_module_and_query_class`](../test/features_test.rb#test_statements_are_named_after_the_module_and_query_class) |
 | `:execresult` with `RETURNING` runs the statement | covered by `:execresult` semantics | covered by `:execresult` semantics | [`test_execresult_with_returning_runs_the_statement`](../tests/end2end_sqlite3.rb#test_execresult_with_returning_runs_the_statement) | |
+| `:copyfrom` keeps the case of quoted / mixed-case table and column names (MySQL `LOAD DATA`) | [`test_copyfrom_keeps_the_case_of_table_and_column_names`](../tests/end2end_pg.rb#test_copyfrom_keeps_the_case_of_table_and_column_names) | [`test_copyfrom_keeps_the_case_of_table_and_column_names`](../tests/end2end_mysql2.rb#test_copyfrom_keeps_the_case_of_table_and_column_names) | [`test_copyfrom_keeps_the_case_of_table_and_column_names`](../tests/end2end_sqlite3.rb#test_copyfrom_keeps_the_case_of_table_and_column_names) | |
 | `:copyfrom` takes any `Enumerable` (empty, lazy, `Enumerator`) | [`test_copyfrom_takes_any_enumerable`](../tests/end2end_pg.rb#test_copyfrom_takes_any_enumerable) | [`test_copyfrom_takes_any_enumerable`](../tests/end2end_mysql2.rb#test_copyfrom_takes_any_enumerable) | [`test_copyfrom_takes_any_enumerable`](../tests/end2end_sqlite3.rb#test_copyfrom_takes_any_enumerable) | |
 | Query names like `HasAuthors` / `IsActive` keep their name and stay RuboCop-clean | [`test_predicate_like_query_names_are_kept`](../tests/end2end_pg.rb#test_predicate_like_query_names_are_kept) | [`test_predicate_like_query_names_are_kept`](../tests/end2end_mysql2.rb#test_predicate_like_query_names_are_kept) | [`test_predicate_like_query_names_are_kept`](../tests/end2end_sqlite3.rb#test_predicate_like_query_names_are_kept) | [`test_predicate_like_query_names_are_rubocop_clean`](../test/features_test.rb#test_predicate_like_query_names_are_rubocop_clean) |
 | SQLite `JSON` / `UUID` columns are `String`s, also numeric-looking text | not PostgreSQL | not MySQL | [`test_json_and_uuid_columns_return_strings_that_look_numeric`](../tests/end2end_sqlite3.rb#test_json_and_uuid_columns_return_strings_that_look_numeric) | |

examples/authors/mysql/query.sql

@@ -207,6 +207,19 @@ SELECT * FROM name_cases WHERE id = sqlc.arg(id);
 -- name: TruncateNameCases :exec
 TRUNCATE TABLE name_cases;
 
+-- name: CopyMixedCaseCopies :copyfrom
+-- MySQL table names are case-sensitive on Linux: LOAD DATA must use `MixedCaseCopies`, not sqlc's lowercased name.
+INSERT INTO `MixedCaseCopies` (`Id`, `SomeValue`) VALUES (?, ?);
+
+-- name: CopyMixedCaseCopiesUnquoted :copyfrom
+INSERT INTO MixedCaseCopies (SomeValue, Id) VALUES (?, ?);
+
+-- name: ListMixedCaseCopies :many
+SELECT * FROM `MixedCaseCopies` ORDER BY `Id`;
+
+-- name: TruncateMixedCaseCopies :exec
+TRUNCATE TABLE `MixedCaseCopies`;
+
 -- name: GetUnnamedColumns :one
 SELECT 1, 'two';
 

examples/authors/mysql/schema.sql

@@ -100,6 +100,8 @@ CREATE TABLE mysql_binding_values (
   json_unicode JSON
 );
 
+CREATE TABLE `MixedCaseCopies` (`Id` BIGINT NOT NULL PRIMARY KEY, `SomeValue` TEXT);
+
 CREATE TABLE name_cases (
   id BIGINT NOT NULL PRIMARY KEY,
   `class` TEXT NOT NULL,

examples/authors/postgresql/query.sql

@@ -243,6 +243,15 @@ SELECT * FROM name_cases WHERE id = sqlc.arg(id);
 -- name: TruncateNameCases :exec
 TRUNCATE TABLE name_cases;
 
+-- name: CopyMixedCaseCopies :copyfrom
+INSERT INTO "MixedCaseCopies" ("Id", "SomeValue") VALUES ($1, $2);
+
+-- name: ListMixedCaseCopies :many
+SELECT * FROM "MixedCaseCopies" ORDER BY "Id";
+
+-- name: TruncateMixedCaseCopies :exec
+TRUNCATE TABLE "MixedCaseCopies";
+
 -- name: GetUnnamedColumns :one
 SELECT 1, 'two';
 

examples/authors/postgresql/schema.sql

@@ -137,6 +137,8 @@ CREATE TABLE batch_types (
   stamps  timestamptz[]
 );
 
+CREATE TABLE "MixedCaseCopies" ("Id" BIGINT NOT NULL PRIMARY KEY, "SomeValue" TEXT);
+
 CREATE TABLE name_cases (
     id BIGINT PRIMARY KEY,
     "class" TEXT NOT NULL,

examples/authors/sqlite/query.sql

@@ -200,6 +200,12 @@ SELECT * FROM name_cases WHERE id = sqlc.arg(id);
 -- name: TruncateNameCases :exec
 DELETE FROM name_cases;
 
+-- name: CopyMixedCaseCopies :copyfrom
+INSERT INTO "MixedCaseCopies" ("Id", "SomeValue") VALUES (?, ?);
+
+-- name: ListMixedCaseCopies :many
+SELECT * FROM "MixedCaseCopies" ORDER BY "Id";
+
 -- name: GetUnnamedColumns :one
 SELECT 1, 'two';
 

examples/authors/sqlite/schema.sql

@@ -35,6 +35,8 @@ CREATE TABLE tags (
     uses INTEGER NOT NULL DEFAULT 0
 );
 
+CREATE TABLE "MixedCaseCopies" ("Id" INTEGER NOT NULL PRIMARY KEY, "SomeValue" TEXT);
+
 CREATE TABLE name_cases (
   id INTEGER PRIMARY KEY,
   "class" TEXT NOT NULL,

examples/mysql2/models.rb

@@ -117,6 +117,9 @@ class MysqlBindingValu < ::Data.define(
   end
   # rubocop:enable Naming/VariableNumber
 
+  class Mixedcasecopy < ::Data.define(:id, :somevalue)
+  end
+
   class NameCase < ::Data.define(
     :id,
     :class_,

examples/mysql2/models.rbs

@@ -193,6 +193,12 @@ module Mysql2Codegen
     ) -> void
   end
 
+  class Mixedcasecopy
+    attr_reader id: ::Integer
+    attr_reader somevalue: ::String?
+    def initialize: (  ::Integer id, ::String? somevalue) -> void
+  end
+
   class NameCase
     attr_reader id: ::Integer
     attr_reader class_: ::String