Container lifecycle management as Maven and Gradle plugins

[lukaseder]

In jOOQ, one of the most re-occurring ideas is that jOOQ should tightly couple Testcontainers, Flyway/Liquibase and jOOQ's code generation. See e.g. jOOQ/jOOQ#6551

This coupling is obviously a bad idea, because the glue code is only a few lines of code, but to make it sufficiently configurable on behalf of all 3 involved products, we'd have to go through a lot of hassle. Much better if all 3 products could be configured completely independently in a CI/CD pipeline.

This is already possible for Flyway/Liquibase and jOOQ, both in Maven and Gradle, but I think it's not trivial to do for Testcontainers yet? What I'd love to see is to have Maven and Gradle plugins available that would allow for controlling the lifecycle of a test container instance in a build, for example (in pseudo-Maven):

<plugins>

  <!-- Start the database container -->
  <plugin>
    <artifactId>testcontainers-postgres-maven<artifactId>
    <phase>initialize</phase>
    <goal>start</goal>
  </plugin>

  <!-- Apply all schema migrations -->
  <plugin>
    <artifactId>flyway-maven<artifactId>
    <phase>initialize</phase>
  </plugin>

  <!-- Generate jOOQ sources -->
  <plugin>
    <artifactId>jooq-codegen-maven<artifactId>
    <phase>generate-sources</phase>
  </plugin>

  <!-- Stop the containers -->
  <plugin>
    <artifactId>testcontainers-postgres-maven<artifactId>
    <phase>post-integration-test</phase>
    <goal>stop</goal>
  </plugin>
</plugin>

I don't think Maven has an equivalent of Java's finally clause (i.e. a phase that runs at the end irrespective of how many phases were run), so if the post-integration-test phase isn't reached, then the containers won't stop, but that can be handled:

• E.g. they could stop at the latest when the build VM terminates (might be possible?)
• Gradle might not have this problem
• The start goal could reset the container first, if it is already running
• They could use Maven profiles for the whole thing and put all 4 plugin executions in there

There are many details to flesh out, but you get the idea. I think the sort of versatility this offers to highly generic, configurable builds would be yet another killer feature for testcontainers, not just for testing, but for CI/CD pipelines in general.

I've found an example project here, that does something like what I have in mind:
https://github.com/squark-io/testcontainers-maven-plugin

[sparsick]

I think your use case is already possible with the https://github.com/fabric8io/docker-maven-plugin. It has a start and stop goal.

           <plugin>
                <groupId>io.fabric8</groupId>
                <artifactId>docker-maven-plugin</artifactId>
                <version>0.38.1</version>
                <configuration>
                    <images>
                        <image>
                            <name>postgres:12.1</name>
                            <alias>database</alias>
                            <run>
                                <ports>
                                    <port>postgresql.port:5432</port>
                                </ports>
                                <wait>
                                    <log>.*database system is ready to accept connections.*</log>
                                    <time>10000</time>
                                </wait>
                            </run>
                        </image>
                    </images>
                </configuration>
                <executions>
                    <execution>
                        <id>start</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>3.0.0-M5</version>
                <configuration>
                    <systemProperties>
                        <property>
                            <name>postgresql.port</name>
                            <value>${postgresql.port}</value>
                        </property>
                        <property>
                            <name>docker.host</name>
                            <value>${docker.hostname}</value>
                        </property>
                    </systemProperties>
                </configuration>
            </plugin>

I used this idea as replacement for elder database tests that originally run against an embedded database. But maybe you can adopt it for your use case.

[julianladisch]

The docker-maven-plugin doesn't run the stop goal when some previous phase fails. Always stopping the container is the main reason why a testcontainers solution with Ryuk resource reaper or a similar cleanup method is needed.

[Jotschi]

I started work on a plugin which also covers the start/stop of a database container. It however goes a bit beyond that. It mainly focuses on providing throwaway databases for unit tests which a server is handing out to requesting tests.
This way no database container needs to restart, No flyway needs to setup the database again for each test, no truncate tables and setup of test fixtures need to run again. I have been using this process in the past to cut CI time from 8h down to 30min and the project is a rewrite of the solution I did a few years ago.

https://github.com/metaloom/testdatabase-provider
https://metaloom.github.io/testdatabase-provider/

The maven plugin may also be used without the provider part and only start a postgreSQL container. It also supports reuse to keep the container running even if the mvn process terminates.

It however still needs some work before I can release it to mvn central.

I have released an early version to mvn central.

[regulskimichal]

@lukaseder
For Gradle, please check: https://github.com/regulskimichal/testcontainers-gradle-plugin
Also, please read an article: https://www.michalregulski.pl/posts/why-i-built-testcontainers-gradle-plugin/