Raise instead of committing a MySQL transaction the server rolled back on a deadlock

README.md

@@ -144,6 +144,11 @@ end
 * PostgreSQL: if a statement fails and the block rescues the error, PostgreSQL still rolls the whole transaction
   back, so `transaction` raises `PG::InFailedSqlTransaction` instead of pretending it committed. In a nested
   `transaction` only the savepoint is rolled back: the outer block can rescue that error and go on.
+* MySQL: a deadlock (or a lock wait timeout with `innodb_rollback_on_timeout`) rolls the whole transaction back on
+  the server, its savepoints too. If the block rescues that error, every later query in the transaction and
+  `transaction` itself raise it again (a `Mysql2::Error` with the same `error_number`, its message prefixed with
+  `transaction rolled back:`) instead of running in autocommit mode and reporting a commit; retry the whole
+  `transaction`. A lock wait timeout without that setting fails only its statement: the block can rescue it and go on.
 * A connection the server closed (restart, failover, killed backend) is reconnected (`pg`) or replaced in the pool
   (`mysql2`) the next time it is checked out; the call that hit the dead connection still raises. Inside `transaction`
   a lost `pg` connection is not reconnected: every later call in the block and `transaction` itself raise

docs/Coverage.md

@@ -23,7 +23,7 @@ only when the behavior is rejected or decided during generation and generated co
 | SQLite `busy_handler_timeout=` waits for another connection without blocking other threads | not SQLite | not SQLite | [`test_busy_handler_timeout_waits_for_another_connection_without_blocking_threads`](../tests/end2end_sqlite3.rb#test_busy_handler_timeout_waits_for_another_connection_without_blocking_threads) | |
 | Names: keywords, method collisions (also private row hooks such as `initialize`), acronyms, quoted mixed case, repeated `$n`, plural models | [`test_keyword_and_method_names_keep_distinct_row_and_argument_names`](../tests/end2end_pg.rb#test_keyword_and_method_names_keep_distinct_row_and_argument_names), [`test_repeated_column_parameters_get_distinct_names`](../tests/end2end_pg.rb#test_repeated_column_parameters_get_distinct_names) | [`test_keyword_and_method_names_keep_distinct_row_and_argument_names`](../tests/end2end_mysql2.rb#test_keyword_and_method_names_keep_distinct_row_and_argument_names), [`test_repeated_positional_parameters_get_distinct_names`](../tests/end2end_mysql2.rb#test_repeated_positional_parameters_get_distinct_names) | [`test_keyword_and_method_names_keep_distinct_row_and_argument_names`](../tests/end2end_sqlite3.rb#test_keyword_and_method_names_keep_distinct_row_and_argument_names) | [`test_snake_case_keeps_acronyms_together`](../test/features_test.rb#test_snake_case_keeps_acronyms_together), [`test_query_names_cannot_replace_generated_methods`](../test/features_test.rb#test_query_names_cannot_replace_generated_methods), [`test_tables_with_the_same_singular_get_distinct_models`](../test/features_test.rb#test_tables_with_the_same_singular_get_distinct_models), [`test_columns_named_like_row_methods_still_build_and_read_rows`](../test/features_test.rb#test_columns_named_like_row_methods_still_build_and_read_rows) |
 | Connections, injected pools, reconnect behavior and SQLite serialization | [`test_queries_can_share_an_injected_pool`](../tests/end2end_pg.rb#test_queries_can_share_an_injected_pool), [`test_a_connection_closed_by_the_server_is_reconnected`](../tests/end2end_pg.rb#test_a_connection_closed_by_the_server_is_reconnected), [`test_connection_lost_inside_transaction_is_not_reconnected_until_the_transaction_ends`](../tests/end2end_pg.rb#test_connection_lost_inside_transaction_is_not_reconnected_until_the_transaction_ends) | [`test_queries_can_share_an_injected_pool`](../tests/end2end_mysql2.rb#test_queries_can_share_an_injected_pool), [`test_a_connection_closed_by_the_server_is_replaced`](../tests/end2end_mysql2.rb#test_a_connection_closed_by_the_server_is_replaced), [`test_a_stopped_async_task_does_not_leave_a_closed_connection_in_the_pool`](../tests/end2end_mysql2.rb#test_a_stopped_async_task_does_not_leave_a_closed_connection_in_the_pool) | [`test_fibers_sharing_a_database_take_turns`](../tests/end2end_sqlite3.rb#test_fibers_sharing_a_database_take_turns), [`test_threads_sharing_a_database_take_turns_in_transactions`](../tests/end2end_sqlite3.rb#test_threads_sharing_a_database_take_turns_in_transactions), [`test_a_database_returning_hash_rows_still_builds_rows`](../tests/end2end_sqlite3.rb#test_a_database_returning_hash_rows_still_builds_rows) | [`test_engine_must_match_the_driver`](../test/features_test.rb#test_engine_must_match_the_driver) |
-| Transaction result, commit/rollback, nested savepoint, exit semantics and rescued PostgreSQL error | [`test_nested_savepoint_rolls_back_while_outer_transaction_commits`](../tests/end2end_pg.rb#test_nested_savepoint_rolls_back_while_outer_transaction_commits), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_pg.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version), [`test_transaction_whose_failed_statement_was_rescued_raises_instead_of_committing`](../tests/end2end_pg.rb#test_transaction_whose_failed_statement_was_rescued_raises_instead_of_committing) | [`test_nested_savepoint_rolls_back_while_outer_transaction_commits`](../tests/end2end_mysql2.rb#test_nested_savepoint_rolls_back_while_outer_transaction_commits), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_mysql2.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version) | [`test_nested_transaction_rolls_back_to_a_savepoint`](../tests/end2end_sqlite3.rb#test_nested_transaction_rolls_back_to_a_savepoint), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_sqlite3.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version) | [`test_nested_transactions_reject_isolation_and_mode`](../test/features_test.rb#test_nested_transactions_reject_isolation_and_mode) |
+| Transaction result, commit/rollback, nested savepoint, exit semantics, rescued PostgreSQL error and rescued MySQL deadlock / lock wait timeout | [`test_nested_savepoint_rolls_back_while_outer_transaction_commits`](../tests/end2end_pg.rb#test_nested_savepoint_rolls_back_while_outer_transaction_commits), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_pg.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version), [`test_transaction_whose_failed_statement_was_rescued_raises_instead_of_committing`](../tests/end2end_pg.rb#test_transaction_whose_failed_statement_was_rescued_raises_instead_of_committing) | [`test_nested_savepoint_rolls_back_while_outer_transaction_commits`](../tests/end2end_mysql2.rb#test_nested_savepoint_rolls_back_while_outer_transaction_commits), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_mysql2.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version), [`test_rescued_deadlock_in_a_transaction_raises_instead_of_committing`](../tests/end2end_mysql2.rb#test_rescued_deadlock_in_a_transaction_raises_instead_of_committing), [`test_rescued_deadlock_in_a_nested_transaction_raises_instead_of_committing`](../tests/end2end_mysql2.rb#test_rescued_deadlock_in_a_nested_transaction_raises_instead_of_committing), [`test_rescued_lock_wait_timeout_rolls_back_only_its_statement`](../tests/end2end_mysql2.rb#test_rescued_lock_wait_timeout_rolls_back_only_its_statement) | [`test_nested_transaction_rolls_back_to_a_savepoint`](../tests/end2end_sqlite3.rb#test_nested_transaction_rolls_back_to_a_savepoint), [`test_transaction_nonlocal_exit_semantics_follow_timeout_version`](../tests/end2end_sqlite3.rb#test_transaction_nonlocal_exit_semantics_follow_timeout_version) | [`test_nested_transactions_reject_isolation_and_mode`](../test/features_test.rb#test_nested_transactions_reject_isolation_and_mode) |
 | Isolation applied; invalid isolation/mode rejected with `ArgumentError` before begin; SQLite locks | [`test_transaction_isolation_level`](../tests/end2end_pg.rb#test_transaction_isolation_level) | [`test_transaction_isolation_level_is_applied`](../tests/end2end_mysql2.rb#test_transaction_isolation_level_is_applied) | [`test_transaction_mode`](../tests/end2end_sqlite3.rb#test_transaction_mode), [`test_immediate_and_exclusive_transactions_block_a_second_connection`](../tests/end2end_sqlite3.rb#test_immediate_and_exclusive_transactions_block_a_second_connection), [`test_unknown_transaction_mode_is_rejected_before_beginning`](../tests/end2end_sqlite3.rb#test_unknown_transaction_mode_is_rejected_before_beginning) | [`test_nested_transactions_reject_isolation_and_mode`](../test/features_test.rb#test_nested_transactions_reject_isolation_and_mode) |
 | Logger debug format, transaction logs, secret-free parameters | [`test_logger_receives_query_names_and_durations`](../tests/end2end_pg.rb#test_logger_receives_query_names_and_durations) | [`test_logger_receives_query_names_and_durations`](../tests/end2end_mysql2.rb#test_logger_receives_query_names_and_durations) | [`test_logger_receives_query_names_and_durations`](../tests/end2end_sqlite3.rb#test_logger_receives_query_names_and_durations) | [`test_logger_support`](../test/features_test.rb#test_logger_support) |
 | Fiber scheduler concurrency / MySQL no prepared-statement leaks | [`test_queries_run_concurrently_under_the_fiber_scheduler`](../tests/end2end_pg.rb#test_queries_run_concurrently_under_the_fiber_scheduler) | [`test_generated_queries_overlap_under_sync_including_transactions`](../tests/end2end_mysql2.rb#test_generated_queries_overlap_under_sync_including_transactions), [`test_generated_queries_do_not_create_server_side_prepared_statements`](../tests/end2end_mysql2.rb#test_generated_queries_do_not_create_server_side_prepared_statements) | [`test_fibers_sharing_a_database_take_turns`](../tests/end2end_sqlite3.rb#test_fibers_sharing_a_database_take_turns) | |

examples/mysql2/connection.rb

@@ -6,19 +6,49 @@ module Mysql2Codegen
   # State kept on every Mysql2::Client the first time it is used.
   module ConnectionState
     attr_accessor :in_transaction
+    # The error with which the server rolled back the whole transaction (a deadlock, or a lock wait timeout with
+    # innodb_rollback_on_timeout): the transaction's later queries raise instead of running in autocommit mode.
+    attr_accessor :rolled_back_by
 
     # Stays true when a Thread#kill interrupts the query (it skips rescue): see #with_client.
     def query(...)
+      raise rolled_back_error if in_transaction && rolled_back_by
+
       @query_running = true
       result = super
       @query_running = false
       result
+    rescue ::Mysql2::Error => e
+      @query_running = false
+      self.rolled_back_by ||= e if in_transaction && ends_transaction?(e)
+      raise
     rescue ::Exception # rubocop:disable Lint/RescueException
       @query_running = false
       raise
     end
 
     def query_running? = @query_running == true
+
+    def rolled_back_error
+      error = rolled_back_by
+      error.class.new("transaction rolled back: #{error.message}", nil, error.error_number, error.sql_state)
+    end
+
+    # ER_LOCK_DEADLOCK always rolls back the whole transaction, ER_LOCK_WAIT_TIMEOUT only with
+    # innodb_rollback_on_timeout (otherwise just the statement).
+    def ends_transaction?(error)
+      return true if error.error_number == 1213
+      return false unless error.error_number == 1205
+
+      if @rollback_on_timeout.nil?
+        @rollback_on_timeout = begin
+          query('SELECT @@innodb_rollback_on_timeout', as: :array).first.first == 1
+        rescue ::Mysql2::Error
+          false
+        end
+      end
+      @rollback_on_timeout
+    end
   end
 
   # Raised when the query object yielded by #transaction is used after the transaction ended.

examples/mysql2/connection.rbs

@@ -2,8 +2,11 @@
 module Mysql2Codegen
   module ConnectionState
     attr_accessor in_transaction: bool
+    attr_accessor rolled_back_by: untyped
     def query: (*untyped) -> untyped
     def query_running?: () -> bool
+    def rolled_back_error: () -> untyped
+    def ends_transaction?: (untyped error) -> bool
   end
 
   class TransactionEnded < ::StandardError

examples/mysql2/query_sql.rb

@@ -572,7 +572,7 @@ def transaction(isolation: nil)
             failed = true
             bound&.close
             begin
-              client.query(rollback) if started || outermost
+              client.query(rollback) if (started || outermost) && !client.rolled_back_by
             rescue ::Mysql2::Error
               # a deadlock / lock wait timeout already rolled back the transaction: its savepoints are gone;
               # an interrupted query closed the connection: the server rolls back
@@ -587,6 +587,10 @@ def transaction(isolation: nil)
                   # Thread#kill interrupted a query: the client still waits for its result and cannot send
                   # ROLLBACK. Closing it makes the server roll back; #with_client discards it.
                   client.close
+                elsif client.rolled_back_by
+                  # the server already rolled the whole transaction back (its savepoints too): COMMIT would only
+                  # end the statements run since, so raise instead of reporting a commit
+                  raise client.rolled_back_error if ::Thread.current.status != 'aborting'
                 else
                   commit = outermost ? 'COMMIT' : "RELEASE SAVEPOINT #{savepoint}"
                   begin
@@ -604,7 +608,10 @@ def transaction(isolation: nil)
                 end
               end
             ensure
-              client.in_transaction = false if outermost
+              if outermost
+                client.in_transaction = false
+                client.rolled_back_by = nil
+              end
             end
           end
         end
@@ -1573,7 +1580,7 @@ def load_data(client, sql, strict: true)
       ensure
         if outermost
           client.query(done ? 'COMMIT' : 'ROLLBACK')
-        else
+        elsif !client.rolled_back_by # a deadlock rolled the whole transaction back, its savepoint too
           client.query(done ? 'RELEASE SAVEPOINT sqlc_copy' : 'ROLLBACK TO SAVEPOINT sqlc_copy')
         end
       end

lib/sqlc_gen_ruby/drivers/mysql2.rb

@@ -11,7 +11,9 @@ def name = 'Mysql2'
       def module_members
         [
           [CONNECTION_STATE, "module ConnectionState\n  attr_accessor in_transaction: bool\n" \
-                             "  def query: (*untyped) -> untyped\n  def query_running?: () -> bool\nend"],
+                             "  attr_accessor rolled_back_by: untyped\n" \
+                             "  def query: (*untyped) -> untyped\n  def query_running?: () -> bool\n" \
+                             "  def rolled_back_error: () -> untyped\n  def ends_transaction?: (untyped error) -> bool\nend"],
           [TRANSACTION_CONNECTION, TRANSACTION_CONNECTION_RBS],
           [ISOLATION_LEVELS, ISOLATION_RBS]
         ]
@@ -136,19 +138,49 @@ def helper_methods(queries)
         # State kept on every Mysql2::Client the first time it is used.
         module ConnectionState
           attr_accessor :in_transaction
+          # The error with which the server rolled back the whole transaction (a deadlock, or a lock wait timeout with
+          # innodb_rollback_on_timeout): the transaction's later queries raise instead of running in autocommit mode.
+          attr_accessor :rolled_back_by
 
           # Stays true when a Thread#kill interrupts the query (it skips rescue): see #with_client.
           def query(...)
+            raise rolled_back_error if in_transaction && rolled_back_by
+
             @query_running = true
             result = super
             @query_running = false
             result
+          rescue ::Mysql2::Error => e
+            @query_running = false
+            self.rolled_back_by ||= e if in_transaction && ends_transaction?(e)
+            raise
           rescue ::Exception # rubocop:disable Lint/RescueException
             @query_running = false
             raise
           end
 
           def query_running? = @query_running == true
+
+          def rolled_back_error
+            error = rolled_back_by
+            error.class.new("transaction rolled back: #{error.message}", nil, error.error_number, error.sql_state)
+          end
+
+          # ER_LOCK_DEADLOCK always rolls back the whole transaction, ER_LOCK_WAIT_TIMEOUT only with
+          # innodb_rollback_on_timeout (otherwise just the statement).
+          def ends_transaction?(error)
+            return true if error.error_number == 1213
+            return false unless error.error_number == 1205
+
+            if @rollback_on_timeout.nil?
+              @rollback_on_timeout = begin
+                query('SELECT @@innodb_rollback_on_timeout', as: :array).first.first == 1
+              rescue ::Mysql2::Error
+                false
+              end
+            end
+            @rollback_on_timeout
+          end
         end
       RUBY
 
@@ -206,7 +238,7 @@ def transaction(isolation: nil)
                 failed = true
                 bound&.close
                 begin
-                  client.query(rollback) if started || outermost
+                  client.query(rollback) if (started || outermost) && !client.rolled_back_by
                 rescue ::Mysql2::Error
                   # a deadlock / lock wait timeout already rolled back the transaction: its savepoints are gone;
                   # an interrupted query closed the connection: the server rolls back
@@ -221,6 +253,10 @@ def transaction(isolation: nil)
                       # Thread#kill interrupted a query: the client still waits for its result and cannot send
                       # ROLLBACK. Closing it makes the server roll back; #with_client discards it.
                       client.close
+                    elsif client.rolled_back_by
+                      # the server already rolled the whole transaction back (its savepoints too): COMMIT would only
+                      # end the statements run since, so raise instead of reporting a commit
+                      raise client.rolled_back_error if ::Thread.current.status != 'aborting'
                     else
                       commit = outermost ? 'COMMIT' : "RELEASE SAVEPOINT #{savepoint}"
                       begin
@@ -238,7 +274,10 @@ def transaction(isolation: nil)
                     end
                   end
                 ensure
-                  client.in_transaction = false if outermost
+                  if outermost
+                    client.in_transaction = false
+                    client.rolled_back_by = nil
+                  end
                 end
               end
             end
@@ -295,7 +334,7 @@ def load_data(client, sql, strict: true)
           ensure
             if outermost
               client.query(done ? 'COMMIT' : 'ROLLBACK')
-            else
+            elsif !client.rolled_back_by # a deadlock rolled the whole transaction back, its savepoint too
               client.query(done ? 'RELEASE SAVEPOINT sqlc_copy' : 'ROLLBACK TO SAVEPOINT sqlc_copy')
             end
           end