Dependent containers about testcontainers-scala

[dimafeng]

@LMnet
I've improved the MultipleContainers in this way:

val pgContainer = PostgreSQLContainer()
val appContainer = AppContainer(pgContainer.jdbcUrl, pgContainer.username, pgContainer.password)

val containers = MultipleContainers(LazyContainer(pgContainer), LazyContainer(appContainer))

I've added com.dimafeng.testcontainers.LazyContainer that prevents premature access to pgContainer's data unless it's started.

Does it cover your case? Any suggestions on this approach?

---

For Everyone's Information, I've also changed the type of MultipleContainers.containers to shapeless.HList so that, in order to access the particular nested container, you would need to do something like val c1 :: c2 :: HNil = containers.containers

[LMnet]

@dimafeng Maybe I'm missing something, but:

val pgContainer = PostgreSQLContainer()
val appContainer = AppContainer(pgContainer.jdbcUrl, pgContainer.username, pgContainer.password) // an error will be thrown on this line because pgContainer is not lazy here

val containers = MultipleContainers(LazyContainer(pgContainer), LazyContainer(appContainer))

Also, are you sure about shapeless? It is a pretty heavy dependency.

Here is my current full solution:

trait TestContainersSupport extends BeforeAndAfterAll { this: Suite =>

  import scala.collection.JavaConverters._

  lazy val pgContainer = PostgreSQLContainer()
  lazy val appContainer = AppContainer(
    dockerJdbcUrl(pgContainer),
    pgContainer.username,
    pgContainer.password
  )

  // hack to obtain container IP in the containers network
  private def dockerJdbcUrl(pgContainer: PostgreSQLContainer): String = {
    import org.testcontainers.containers.{PostgreSQLContainer => OTCPostgreSQLContainer}

    val ip = pgContainer.container.getContainerInfo.getNetworkSettings.getNetworks.asScala.values.head.getIpAddress
    s"jdbc:postgresql://$ip:${OTCPostgreSQLContainer.POSTGRESQL_PORT}/test"
  }


  override def beforeAll(): Unit = {
    super.beforeAll()
    pgContainer.starting()
    appContainer.starting()
  }

  override def afterAll(): Unit = {
    super.afterAll()
    appContainer.finished()
    pgContainer.finished()
  }
}

As you can see from the code, I faced another problem - pgContainer.jdbcUrl works only from the test code, not between containers. There is an issue about this in testcontainers-java.

I'm not really happy with the code above. But atm I don't know how to solve all this without significant improvements and/or refactorings in both scala and java testcontainers projects.

There is rnorth/containercore#1 and it looks really promising. I think we should direct our efforts to this because new suggested architecture with builders will make it impossible to get jdbcUrl before container's start.

[dimafeng]

Hi @LMnet,
The problem is that the Postgre container is not aware of the actual port that will be assigned outside the container thus it can't generate jdbcUrl - the Postgre container doesn't use fixed port bindings to let testcontainers start multiple instances of the container in parallel.

How to solve this:

1. Use DockerComposeCintainer and define both containers in the compose file so they are in the same network. In this case you'll be able to specify jdbcUrl using native Postgre port and Postgre container's host name.
2. I can add a scala wrapper for FixedHostPortGenericContainer so it gives the ability to run container with a predefined host port that can be used in your scenario without docker compose.

Let me know if you would like to submit a PR for p. 2 🙂

[LMnet]

Actually, I don't think that any of this 2 solutions is good enough:

1. docker-compose is, of course, a possible solution for some cases. But docker-compose doesn't support health checks or waiting strategies. There are some hacks, like wait-for-it.sh. But it hacks, not a proper solution.
2. Fixing a port or host is a solution too, but this solution deprives advantages of testcontainers. I like this automatic port binding, I don't want to think about it in my tests.

I think that the main problem is in ForAllTestContainer trait. The current implementation is ok only for the simplest cases when you have only one single container and you need to test it. But, do we really need testcontainers to test only one container? I believe that the main use case for testcontainers is testing some complex docker hierarchies when two or more containers somehow depend on each other. And current solution (MultipleContainers) couldn't help me with such installation.

At the moment I'm trying to solve the problem without ForAllTestContainer. My code right now looks like this:

class MyTest extends FreeSpec with BeforeAndAfterAll {

  lazy val pgContainer = PostgreSQLContainer()
  lazy val appContainer = AppContainer(pgContainer.jdbcUrl, pgContainer.username, pgContainer.password)

  override def beforeAll(): Unit = {
    pgContainer.starting()
    appContainer.starting()
  }

  override def afterAll(): Unit = {
    appContainer.finished()
    pgContainer.finished()
  }

  // tests here
}

This solution gives me full control on containers. Also, I don't need to do tuple calls with ._1, I can call containers by well-defined name. What do you think about this scheme?

[dimafeng]

Your example makes sense. I can try to come up with something lazy for MultipleContainers

[dimafeng]

I gave a wrong example - missed lazy. I meant this:

lazy val pgContainer = PostgreSQLContainer()
lazy val appContainer = AppContainer(pgContainer.jdbcUrl, pgContainer.username, pgContainer.password)

val containers = MultipleContainers(LazyContainer(pgContainer), LazyContainer(appContainer))

LazyContainer has the following signature

class LazyContainer[T <: Container](factory: => T) extends Container

so the example above should work fine.

You can take a look at tests:
https://github.com/testcontainers/testcontainers-scala/blob/b5eed02b7477bdb2331cadfc58265634d823ea03/src/test/scala/com/dimafeng/testcontainers/MultipleContainersSpec.scala

[dimafeng]

@LMnet Just to clarify - the example that you showed:

class MyTest extends FreeSpec with BeforeAndAfterAll {

  lazy val pgContainer = PostgreSQLContainer()
  lazy val appContainer = AppContainer(pgContainer.jdbcUrl, pgContainer.username, pgContainer.password)

  override def beforeAll(): Unit = {
    pgContainer.starting()
    appContainer.starting()
  }

  override def afterAll(): Unit = {
    appContainer.finished()
    pgContainer.finished()
  }

  // tests here
}

Does it work as expected?

[LMnet]

@dimafeng Conceptually it works. But later I sent example with TestContainersSupport - this is the full and workable code.

[ildac]

I had a similar issue as @LMnet, solved with some lazy val around the code, didn't know about the LazyContainer maybe I should try that!

[dimafeng]

Hi @ildac,
Unfortunately, LazyContainer is not released yet. I got stuck with migration from gradle to SBT. Hopefully, I'll release it within next few days.

[dimafeng]

LazyContainer has been released