Repository navigation
Migrate several Flyway schemas in one container, start Flyway faster, and clean up only what a run created. - #37
Merged
gustavofreze merged 4 commits intoNov 14, 2026
Conversation
… those a failed creation leaves behind.
… in one Flyway container.
…he run-scoped cleanup.
gustavofreze
deleted the
feat/multi-schema-flyway-and-run-scoped-cleanup
branch
November 14, 2026 17:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A test bootstrap that migrates one schema per mutation thread started one Flyway container per schema: on 4 CPUs that is 5 Flyway runs in a row, each a fresh container, a fresh copy of the migrations, a fresh MySQL check, and a JVM that spent 5 to 46 seconds starting while Flyway reported under 1.3 seconds of SQL. On a busy host the run brushed the 120 second timeout. Cleanup also reached beyond the run that asked for it: the reaper and
remove()pruned every unused network carrying the library label on the host, including those of other processes, while a container whose process died before it started, or whose creation timed out after Docker registered it, was never removed.What changes
cleanAndMigrateSchemas(array $schemas)cleans and migrates a list of schemas in one Flyway container, onFlywayDockerContainerand onFlywayContainerGated. Flyway still runsclean migrateonce per schema, so each schema ends exactly aswithSchema($schema)->cleanAndMigrate()leaves it, with its own history table, while the container is created, the migrations are copied, and the MySQL source is checked once. The timeout of each schema is the one ofwithTimeout(), so the run waits up to that timeout times the number of schemas. An empty list raisesSchemaListEmpty.JAVA_ARGS=-XX:TieredStopAtLevel=1 -XX:+UseSerialGC, which cuts the CPU time of each Flyway run by about 45%. SettingJAVA_ARGSthroughwithEnvironmentVariable()replaces it. Disabling telemetry was measured too and saved nothing, so it is not part of this change.tiny-blocks.docker-container.run=<host name>-<process ID>. Inside Docker, the first creation of a process starts a reaper for that process, which removes, once the runner container exits, every container of the process in any state and its networks that nothing uses anymore.remove()prunes only the unused networks of its own process. A creation that fails after Docker registered the container removes the containers of the process carrying that name. No cleanup runs a host-wide prune anymore.Measured with the bootstrap of a service that migrates its main schema and one schema per thread (MySQL 8.4, Flyway 13.9.0, 4 threads, on a 4 CPU host shared with other work). The four variants ran interleaved, three rounds each, from a cold MySQL every time. Medians, with the gain against 6.0.0:
cleanAndMigrate()per schemacleanAndMigrateSchemas()for the threadscleanAndMigrateSchemas()for all five schemasFlyway CPU is the user and system time of the Flyway JVMs, sampled on the host from
/proc, and it is the measure that holds still while the load of the host moves: wall times of the same variant swung up to 3 times between rounds. Flyway wall runs from the first Flyway container creation to the last migration, and the bootstrap also includes the MySQL start, which none of these changes touch and which varied from 9 to 31 seconds. Every one of the 12 runs started 5 Flyway JVMs, every 6.1.0 JVM ran with-XX:TieredStopAtLevel=1 -XX:+UseSerialGCon its command line (the per schema loop included), and every schema of every run matched the schemas 6.0.0 produces, compared by columns, indexes, foreign keys, migration history, and rows.The gain in wall time depends on how busy the host is, while the gain in CPU does not. An earlier prototype, measured at a load of 8.6 to 15.4, went from 71.6 to 37.2 seconds (-48%) for the whole bootstrap and from 49.3 to 23.0 seconds (-53%) for Flyway with one container for the thread schemas and the tuned JVM, close to the -55% above at a similar load. A series at a load of 5 to 12 gave only -16% to -31% for Flyway, because on idle cores the default JVM spreads its startup over several compiler and collector threads, so it finishes nearly as fast as the tuned one while still burning about twice the CPU (7.9 to 8.6 against 3.9 to 4.8 seconds per run here, and a median of 8.8 against 4.6 in a single schema probe).
Public API
The change is additive, so the release is
6.1.0.Added:
FlywayDockerContainer::cleanAndMigrateSchemas(),FlywayContainerGated::cleanAndMigrateSchemas()andSchemaListEmpty.Changed: Flyway containers receive
JAVA_ARGSunless it is set. Containers and networks carry one more label. Inside Docker, a customDockerClientnow receives the reaperdocker runbefore every container creation, named after the process, instead of adocker psprobe followed by the reaper after a shared container started. A container another run created and this run only reuses no longer starts a reaper, since this run created nothing.remove()prunes the unused networks of its own process instead of every unused network with the library label.Related issue
None.
Checklist
make reviewpasses.make testspasses.