Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 54 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
- [Configuring migrations](#configuring-migrations)
- [Configuring Flyway options](#configuring-flyway-options)
- [Running Flyway commands](#running-flyway-commands)
- [Migrating several schemas](#migrating-several-schemas)
- [Running only when enabled](#running-only-when-enabled)
+ [Handling failures](#handling-failures)
+ [Using a custom Docker client](#using-a-custom-docker-client)
Expand Down Expand Up @@ -103,14 +104,18 @@ To wait after the container starts, register a wait strategy on the definition (
[Wait strategies](#wait-strategies)).

A container that fails to receive its files, to start, or to satisfy a wait strategy is removed before the failure is
raised, since nothing else holds it.
raised, since nothing else holds it. A creation that fails after Docker registered the container, for example when the
image pull outlasts the pull timeout, removes that container too.

### Running if not exists

Starts a container only if a container with the same name does not exist yet, and reuses it otherwise. The returned
`SharedContainer` is shared by every PHP process that asks for the same name (as in mutation testing), so it offers
neither `stopOnShutdown()` nor `remove()`: no single process owns it. When the code runs inside Docker, a reaper
container removes the shared container once the test runner container exits. The wait strategies registered with
neither `stopOnShutdown()` nor `remove()`: no single process owns it. When the code runs inside Docker, the first
container a process creates also starts a reaper container for that process. Once the test runner container exits,
the reaper removes every container the process created, in any state, and the networks it created that nothing uses
anymore. It finds them by the label of the process (see [Stopping on shutdown](#stopping-on-shutdown)), so the
containers and networks of other processes are never touched. The wait strategies registered with
`withWaitAfterStarted()` apply only when a new container starts.

Processes that ask for the same name at the same time are safe: Docker lets only one of them create the container,
Expand Down Expand Up @@ -162,8 +167,8 @@ $alreadyRunning = $started->wasReused();
### Setting network

Sets the Docker network the container should join. The network is created automatically when the container runs, if
it does not already exist. Networks created by the library are labeled with `tiny-blocks.docker-container=true` for
safe cleanup. When a cleanup running in parallel removes the network before the container starts, the network is
it does not already exist. Networks created by the library are labeled with `tiny-blocks.docker-container=true` and
with the label of the process that created them, for safe cleanup. When a cleanup running in parallel removes the network before the container starts, the network is
created again and the container is started once more.

```php
Expand Down Expand Up @@ -268,10 +273,12 @@ following cleanup is performed automatically:

- The container is killed and removed (`docker rm --force --volumes`).
- Anonymous volumes created by the container (e.g., MySQL's `/var/lib/mysql`) are removed.
- Unused networks created by the library are pruned.
- Unused networks created by this process are pruned.

Only resources labeled with `tiny-blocks.docker-container=true` are affected. Containers, volumes, and networks
from other environments are never touched.
Every container and network the library creates carries the label `tiny-blocks.docker-container=true` and the label of
the process that created it, `tiny-blocks.docker-container.run=<host name>-<process ID>`. Only resources carrying the
label of this process are affected. Containers, volumes, and networks of other processes and other environments are
never touched.

```php
$started = $container->run();
Expand Down Expand Up @@ -560,6 +567,13 @@ $flyway = $flyway->withMigrations(pathOnHost: '/path/to/migrations');
| `withMigrationNamingValidation` | | Rejects migrations whose file names break the naming convention. |
| `withEnvironmentVariable` | `$key`, `$value` | Sets any other Flyway setting through its `FLYWAY_*` variable. |

Flyway starts with a JVM tuned for short runs, `JAVA_ARGS=-XX:TieredStopAtLevel=1 -XX:+UseSerialGC`, which roughly
halves the processor time each run spends before Flyway reaches the database. Setting `JAVA_ARGS` replaces it:

```php
$flyway = $flyway->withEnvironmentVariable(key: 'JAVA_ARGS', value: '-Xmx512m');
```

#### Running Flyway commands

| Method | Flyway command | Description |
Expand Down Expand Up @@ -587,6 +601,19 @@ use TinyBlocks\Time\Duration;
$flyway->withTimeout(timeout: Duration::fromSeconds(seconds: 300))->cleanAndMigrate();
```

#### Migrating several schemas

`cleanAndMigrateSchemas()` cleans and migrates a list of schemas in a single Flyway container. Each schema ends exactly
as `withSchema($schema)->cleanAndMigrate()` leaves it, cleaned, migrated, and with its own history table, while the
container is created, the migrations are copied, and the MySQL source is checked once for all of them instead of once
per schema. The schemas are migrated in order, and the first one Flyway fails on stops the run with
`ContainerExitUnsuccessful`. The run waits up to the timeout of `withTimeout()` for each schema, so 120 seconds per
schema by default. An empty list raises `SchemaListEmpty`.

```php
$flyway->cleanAndMigrateSchemas(schemas: ['test_adm', 'test_adm_0', 'test_adm_1']);
```

#### Running only when enabled

`onlyWhenEnabled()` returns a `FlywayContainerGated`, whose commands run only while an environment
Expand All @@ -603,7 +630,7 @@ $execution = $flyway
->cleanAndMigrate(); # ExecutionCompleted when RUN_MIGRATIONS=1, null otherwise
```

The gated container offers the same four commands, with the same exceptions. A command returns an
The gated container offers the same five commands, with the same exceptions. A command returns an
`ExecutionCompleted` only when Flyway did run, and `null` tells the caller that nothing ran.

### Handling failures
Expand All @@ -616,6 +643,7 @@ The exceptions consumers may catch live in `TinyBlocks\DockerContainer\Exception
| `ContainerWaitTimeout` | A readiness condition does not hold, or a one-shot container does not exit, in time. |
| `ContainerExitUnsuccessful` | A one-shot container, such as Flyway, exits with a non-zero status. |
| `ImageNameEmpty` | A container is created from an empty image name. |
| `SchemaListEmpty` | `cleanAndMigrateSchemas()` is called with an empty list of schemas. |

`ContainerExitUnsuccessful` carries what the container wrote to its standard error, so a failed migration reports the
Flyway error itself:
Expand Down Expand Up @@ -739,6 +767,23 @@ FlywayDockerContainer::from(image: 'flyway/flyway:13.9.0-alpine')
->cleanAndMigrate();
```

When each process of the run works on a schema of its own, as mutation threads do, migrate the main schema and the
schema of every thread in one Flyway run instead of one run per schema:

```php
<?php

declare(strict_types=1);

$schemas = ['my_service_test'];

for ($thread = 0; $thread < (int)getenv('MUTATION_THREADS'); $thread++) {
$schemas[] = sprintf('my_service_test_%d', $thread);
}

$flyway->onlyWhenEnabled(variable: 'RUN_MIGRATIONS')->cleanAndMigrateSchemas(schemas: $schemas);
```

## FAQ

### 01. Why is the shared container name scoped to the test runner?
Expand Down
2 changes: 2 additions & 0 deletions phpstan.neon.dist
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ parameters:
- src/Internal/Containers/Definitions/ContainerDefinition.php
- src/Internal/Containers/Definitions/CopyInstruction.php
- src/Internal/Containers/Drivers/MySQL/JdbcUrl.php
- src/Internal/Containers/Drivers/Flyway/FlywaySchemas.php
- src/Internal/Containers/Drivers/Flyway/FlywaySettings.php
- src/Internal/Containers/Drivers/Flyway/FlywaySource.php
- src/Internal/Containers/Drivers/MySQL/MySQLCommands.php
Expand All @@ -42,6 +43,7 @@ parameters:
paths:
- src/Internal/Containers/ContainerLauncher.php
- src/Internal/Containers/ContainerWaits.php
- src/Internal/Containers/OwnedContainers.php
- src/Internal/Containers/Address/Ports.php
- src/Internal/Containers/Definitions/ContainerDefinition.php
# docker inspect json_decode and sprintf feed mixed values across the system boundary.
Expand Down
18 changes: 18 additions & 0 deletions src/Exceptions/SchemaListEmpty.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<?php

declare(strict_types=1);

namespace TinyBlocks\DockerContainer\Exceptions;

use InvalidArgumentException;

/**
* Raised when Flyway is asked to clean and migrate an empty list of schemas.
*/
final class SchemaListEmpty extends InvalidArgumentException
{
public function __construct()
{
parent::__construct(message: 'Schema list cannot be empty.');
}
}
21 changes: 21 additions & 0 deletions src/FlywayContainerGated.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
use TinyBlocks\DockerContainer\Exceptions\ContainerExitUnsuccessful;
use TinyBlocks\DockerContainer\Exceptions\ContainerWaitTimeout;
use TinyBlocks\DockerContainer\Exceptions\DockerCommandExecutionFailed;
use TinyBlocks\DockerContainer\Exceptions\SchemaListEmpty;
use TinyBlocks\DockerContainer\Internal\Containers\Environment\EnvironmentFlag;

/**
Expand Down Expand Up @@ -89,4 +90,24 @@ public function cleanAndMigrate(): ?ExecutionCompleted
{
return $this->flag->isEnabled() ? $this->flyway->cleanAndMigrate() : null;
}

/**
* Runs the Flyway clean command followed by migrate once for each schema, in a single Flyway container, when the
* variable is enabled, and blocks until Flyway exits.
*
* <p>See {@see FlywayDockerContainer::cleanAndMigrateSchemas()} for how the schemas are migrated and how long the
* run may take.</p>
*
* @param array<int, string> $schemas The schemas to clean and migrate, in order.
* @return ExecutionCompleted|null The successful outcome, or null when the variable is not enabled and nothing
* ran.
* @throws SchemaListEmpty If the variable is enabled and no schema is given.
* @throws DockerCommandExecutionFailed If a Docker command fails.
* @throws ContainerWaitTimeout If the MySQL source is not ready or Flyway does not exit within the timeout.
* @throws ContainerExitUnsuccessful If Flyway exits with a non-zero status, carrying the Flyway error output.
*/
public function cleanAndMigrateSchemas(array $schemas): ?ExecutionCompleted
{
return $this->flag->isEnabled() ? $this->flyway->cleanAndMigrateSchemas(schemas: $schemas) : null;
}
}
83 changes: 65 additions & 18 deletions src/FlywayDockerContainer.php
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,12 @@
use TinyBlocks\DockerContainer\Exceptions\ContainerWaitTimeout;
use TinyBlocks\DockerContainer\Exceptions\DockerCommandExecutionFailed;
use TinyBlocks\DockerContainer\Exceptions\ImageNameEmpty;
use TinyBlocks\DockerContainer\Exceptions\SchemaListEmpty;
use TinyBlocks\DockerContainer\Internal\Client\CommandLineDockerClient;
use TinyBlocks\DockerContainer\Internal\Containers\Definitions\ContainerDefinition;
use TinyBlocks\DockerContainer\Internal\Containers\Drivers\Flyway\FlywaySchemas;
use TinyBlocks\DockerContainer\Internal\Containers\Drivers\Flyway\FlywaySettings;
use TinyBlocks\DockerContainer\Internal\Containers\OneShotContainer;
use TinyBlocks\Time\Duration;

/**
Expand All @@ -25,9 +29,10 @@
private const int DEFAULT_TIMEOUT_IN_SECONDS = 120;

private function __construct(
private DockerClient $client,
private Duration $timeout,
private FlywaySettings $settings,
private GenericDockerContainer $container
private ContainerDefinition $definition
) {
}

Expand All @@ -45,9 +50,10 @@ public static function from(
DockerClient $client = new CommandLineDockerClient()
): FlywayDockerContainer {
return new FlywayDockerContainer(
client: $client,
timeout: Duration::fromSeconds(seconds: self::DEFAULT_TIMEOUT_IN_SECONDS),
settings: FlywaySettings::create(),
container: GenericDockerContainer::from(image: $image, client: $client)
definition: ContainerDefinition::create(image: $image)
);
}

Expand All @@ -61,9 +67,11 @@ public static function from(
*/
public function repair(): ExecutionCompleted
{
return $this->settings
->applyTo(container: $this->container)
->runToCompletion(timeout: $this->timeout, commands: ['repair']);
return new OneShotContainer(client: $this->client)->run(
timeout: $this->timeout,
commands: ['repair'],
definition: $this->settings->applyTo(definition: $this->definition)
);
}

/**
Expand All @@ -76,9 +84,11 @@ public function repair(): ExecutionCompleted
*/
public function migrate(): ExecutionCompleted
{
return $this->settings
->applyTo(container: $this->container)
->runToCompletion(timeout: $this->timeout, commands: ['migrate']);
return new OneShotContainer(client: $this->client)->run(
timeout: $this->timeout,
commands: ['migrate'],
definition: $this->settings->applyTo(definition: $this->definition)
);
}

/**
Expand All @@ -91,9 +101,11 @@ public function migrate(): ExecutionCompleted
*/
public function validate(): ExecutionCompleted
{
return $this->settings
->applyTo(container: $this->container)
->runToCompletion(timeout: $this->timeout, commands: ['validate']);
return new OneShotContainer(client: $this->client)->run(
timeout: $this->timeout,
commands: ['validate'],
definition: $this->settings->applyTo(definition: $this->definition)
);
}

/**
Expand Down Expand Up @@ -144,7 +156,7 @@ public function withSource(MySQLDockerContainer $container): FlywayDockerContain
*/
public function withNetwork(string $name): FlywayDockerContainer
{
return clone($this, ['container' => $this->container->withNetwork(name: $name)]);
return clone($this, ['definition' => $this->definition->withNetwork(name: $name)]);
}

/**
Expand Down Expand Up @@ -177,7 +189,7 @@ public function withMigrations(string $pathOnHost): FlywayDockerContainer
key: 'FLYWAY_LOCATIONS',
value: sprintf($template, self::MIGRATIONS_ON_CONTAINER)
),
'container' => $this->container->copyToContainer(
'definition' => $this->definition->withCopyInstruction(
pathOnHost: $pathOnHost,
pathOnContainer: self::MIGRATIONS_ON_CONTAINER
)
Expand All @@ -197,10 +209,13 @@ public function withMigrations(string $pathOnHost): FlywayDockerContainer
*/
public function cleanAndMigrate(): ExecutionCompleted
{
return $this->settings
->applyTo(container: $this->container)
->withEnvironmentVariable(key: 'FLYWAY_CLEAN_DISABLED', value: 'false')
->runToCompletion(timeout: $this->timeout, commands: ['clean', 'migrate']);
return new OneShotContainer(client: $this->client)->run(
timeout: $this->timeout,
commands: ['clean', 'migrate'],
definition: $this->settings
->applyTo(definition: $this->definition)
->withEnvironmentVariable(key: 'FLYWAY_CLEAN_DISABLED', value: 'false')
);
}

/**
Expand Down Expand Up @@ -245,7 +260,7 @@ public function withCredentials(string $password, string $username): FlywayDocke
*/
public function withPullTimeout(Duration $timeout): FlywayDockerContainer
{
return clone($this, ['container' => $this->container->withPullTimeout(timeout: $timeout)]);
return clone($this, ['definition' => $this->definition->withPullTimeout(timeout: $timeout)]);
}

/**
Expand All @@ -259,6 +274,38 @@ public function withConnectRetries(int $retries): FlywayDockerContainer
return $this->withEnvironmentVariable(key: 'FLYWAY_CONNECT_RETRIES', value: (string)$retries);
}

/**
* Runs the Flyway clean command followed by migrate once for each schema, in a single Flyway container, and blocks
* until Flyway exits.
*
* <p>Each schema ends exactly as <code>withSchema($schema)->cleanAndMigrate()</code> leaves it: cleaned, migrated,
* and with its own history table. The schemas are migrated in order, the migrations are copied and the MySQL
* source is checked once for all of them, and the first schema Flyway fails on stops the run. The timeout of
* each schema is the one set by withTimeout(), so the run waits up to that timeout times the number of
* schemas.</p>
*
* @param array<int, string> $schemas The schemas to clean and migrate, in order.
* @return ExecutionCompleted The successful outcome, carrying the Flyway output of every schema.
* @throws SchemaListEmpty If no schema is given.
* @throws DockerCommandExecutionFailed If a Docker command fails.
* @throws ContainerWaitTimeout If the MySQL source is not ready or Flyway does not exit within the timeout.
* @throws ContainerExitUnsuccessful If Flyway exits with a non-zero status, carrying the Flyway error output.
*/
public function cleanAndMigrateSchemas(array $schemas): ExecutionCompleted
{
$flywaySchemas = FlywaySchemas::from(schemas: $schemas);

return new OneShotContainer(client: $this->client)->run(
timeout: $flywaySchemas->timeoutFor(timeout: $this->timeout),
commands: $flywaySchemas->toCommands(),
definition: $flywaySchemas->applyTo(
definition: $this->settings
->applyTo(definition: $this->definition)
->withEnvironmentVariable(key: 'FLYWAY_CLEAN_DISABLED', value: 'false')
)
);
}

/**
* Returns a copy of the FlywayDockerContainer with an environment variable added.
*
Expand Down
3 changes: 3 additions & 0 deletions src/Internal/Commands/DockerCreate.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

use TinyBlocks\DockerContainer\DockerCommandWithTimeout;
use TinyBlocks\DockerContainer\Internal\Containers\Definitions\ContainerDefinition;
use TinyBlocks\DockerContainer\Internal\Containers\Run;
use TinyBlocks\Time\Duration;

final readonly class DockerCreate implements DockerCommandWithTimeout
Expand Down Expand Up @@ -41,6 +42,8 @@ public function toArguments(): array
$name,
'--label',
self::MANAGED_LABEL,
'--label',
Run::current()->label(),
...$this->definition->toArguments(),
$this->definition->image->name,
...$this->commands
Expand Down
Loading