Using Testcontainers without Docker Engine in Quarkus with Kotlin and LocalStackContainer

A comprehensive guide for integration tests without relying on Docker Desktop

podman container engine, localstack setup, cloud based runtime

Key Highlights

Docker Alternative: Using Podman or cloud-based container runtimes to run Testcontainers without Docker Engine.
Seamless Integration: Configuring Quarkus and Kotlin for integration tests with LocalStack to simulate AWS services.
Flexible Test Configuration: Detailed setup and adjustments to manage test resource lifecycles and environment configurations in various CI scenarios.

Overview

Integration testing in modern applications is a crucial aspect of ensuring that seemingly isolated components work together reliably. With the growing needs of microservice architectures and serverless applications, tools like Testcontainers have become indispensable. Testcontainers provides lightweight, disposable instances of services required for integration tests. When applied in a Quarkus environment using Kotlin, developers can simulate real-world AWS service interactions by incorporating LocalStackContainer.

Using Testcontainers without relying on a Docker Engine is particularly relevant in scenarios where Docker Desktop is either unavailable or undesired (e.g., CI/CD pipelines, local resource constraints, or security policies). This guide will detail how to set up and configure Testcontainers to work with Quarkus and Kotlin, while managing integration tests with a simulated AWS environment provided by LocalStackContainer.

Preparing Your Environment

Setting Up Your Project

Begin by creating a Quarkus project that uses Kotlin as the primary language. Ensure you have at least Java 11, Kotlin, and a build system such as Maven or Gradle ready. In your project, include the dependencies for Testcontainers, LocalStack, and any additional Quarkus or Kotlin libraries needed for testing.

Dependencies Configuration (Maven)

In your pom.xml, add the following dependencies:


    <!-- Testcontainers for integration tests -->
    <dependency>
      <groupId>org.testcontainers</groupId>
      <artifactId>junit-jupiter</artifactId>
      <version>1.18.0</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.testcontainers</groupId>
      <artifactId>localstack</artifactId>
      <version>1.18.0</version>
      <scope>test</scope>
    </dependency>
    <!-- Additional dependencies for Quarkus and Kotlin -->

Dependencies Configuration (Gradle)

If you use Gradle, add these configurations in your build.gradle.kts file:


    dependencies {
        testImplementation("org.testcontainers:junit-jupiter:1.18.0")
        testImplementation("org.testcontainers:localstack:1.18.0")
        // Add other dependencies for Quarkus and Kotlin
    }

This initial setup ensures that your project is ready to leverage Testcontainers and LocalStack for integration tests.

Configuring Testcontainers without Docker Engine

Leveraging Alternative Container Runtimes

Typically, Testcontainers relies on Docker to manage container lifecycles, but you can bypass the Docker Engine by using alternatives such as Podman or even Testcontainers Cloud. Podman is an open-source, daemonless container engine that mimics Docker’s functionalities but operates in a rootless mode and without a persistent daemon.

Using Podman

To use Podman as an alternative:

Install Podman on your development machine or CI environment.
Start Podman ensuring its socket is available—usually, this is located at unix:///run/user/1000/podman/podman.sock.
Set the environment variable TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE to Podman’s socket path. This tells Testcontainers to point to the Podman socket instead of the default Docker socket.

With these steps, Testcontainers will manage containers using Podman in the same effortless manner as it would with Docker.

Using Testcontainers Cloud

An alternative option is to utilize Testcontainers Cloud, which offloads container management to cloud-based runtimes. This solution is particularly useful for developers who wish to conserve local system resources. The configuration involves:

Installing Testcontainers Desktop if you are on macOS (or similar platforms) and selecting the cloud runtime mode.
Adjusting the settings in your Testcontainers configuration to work with remote container management.
Ensuring your CI/CD environment is configured to authenticate and interact with the cloud runtime platform.

Both alternatives allow you to run your integration tests without requiring a local Docker Engine.

Integrating LocalStackContainer for AWS Simulation

Purpose and Use Cases

LocalStack provides a fully functional local AWS cloud stack. It simulates various AWS services such as S3, SQS, and more. When using Testcontainers with LocalStackContainer, your tests can interact with emulated AWS endpoints, allowing for realistic testing of AWS-dependent functionalities without incurring costs or requiring an actual AWS environment.

Setting Up LocalStackContainer

Basic Configuration

Create a test class that uses the LocalStackContainer to simulate AWS services. Annotate your test class with @Testcontainers to enable Testcontainers integration.

In Kotlin, a typical test setup may look like:


    import org.junit.jupiter.api.Test
    import org.junit.jupiter.api.extension.ExtendWith
    import org.testcontainers.containers.LocalStackContainer
    import org.testcontainers.junit.jupiter.Container
    import org.testcontainers.junit.jupiter.Testcontainers
    import software.amazon.awssdk.auth.credentials.AwsBasicCredentials
    import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider
    import software.amazon.awssdk.regions.Region
    import software.amazon.awssdk.services.s3.S3Client
    import java.net.URI

    @Testcontainers
    @ExtendWith(YourQuarkusExtension::class)  // Ensure Quarkus test integration is applied.
    class AwsIntegrationTest {

        companion object {
            @Container
            val localStackContainer = LocalStackContainer("localstack/localstack:latest")
                .withServices(LocalStackContainer.Service.S3, LocalStackContainer.Service.SQS)
        }

        @Test
        fun testAwsS3Interaction() {
            val s3Client = S3Client.builder()
                .endpointOverride(localStackContainer.getEndpointOverride(LocalStackContainer.Service.S3))
                .region(Region.of(localStackContainer.region))
                .credentialsProvider(
                    StaticCredentialsProvider.create(
                        AwsBasicCredentials.create("test", "test")
                    )
                )
                .build()

            // Write your integration test logic here, e.g., creating buckets, uploading objects etc.
        }
    }

This sample demonstrates how to configure an AWS S3 client to point to LocalStack's S3 endpoint. The similar approach can be extended to other AWS services simulated by LocalStack, such as SQS, SNS, etc.

Managing the Container Lifecycle

For more robust testing, especially where multiple tests rely on the same container configuration, you can integrate a testing resource lifecycle manager. Implement a TestResourceLifecycleManager to start and stop LocalStackContainer efficiently.


    import io.quarkus.test.common.QuarkusTestResourceLifecycleManager
    import org.testcontainers.containers.LocalStackContainer

    class LocalStackTestResource : QuarkusTestResourceLifecycleManager {
      
        companion object {
            private val dockerImage = LocalStackContainer.DEFAULT_IMAGE_NAME.withTag("latest")
            val localStackContainer = LocalStackContainer(dockerImage)
                .withServices(LocalStackContainer.Service.S3)
        }

        override fun start(): Map<String, String> {
            localStackContainer.start()
            return mapOf(
                "quarkus.s3.aws.credentials.static-provider.secret-access-key" to localStackContainer.defaultCredentials.awsSecretKey,
                "quarkus.s3.aws.credentials.static-provider.access-key-id" to localStackContainer.defaultCredentials.awsAccessKey
            )
        }

        override fun stop() {
            localStackContainer.stop()
        }
    }

Including this resource lifecycle manager in your test configuration will ensure that LocalStack starts before your tests run and is disposed of properly afterward. Annotate your Quarkus test class with @QuarkusTestResource(LocalStackTestResource::class).

Disabling Docker-Dependent Features in Quarkus

Ensuring Compatibility and Resource Efficiency

Quarkus features, such as DevServices, often come configured to automatically spin up containerized services. However, when you opt for alternative runtime solutions or wish to avoid Docker dependencies, it is essential to disable these features. This prevents unwanted interference with your custom container runtime configuration.

Configuration Adjustments

Within your Quarkus application.properties file, add the following properties:


    # Disable Quarkus DevServices if not using Docker Engine
    quarkus.devservices.enabled=false
    quarkus.datasource.devservices.enabled=false

These settings ensure that Quarkus does not attempt to manage any containers on its own. Instead, it relies entirely on your Testcontainers configuration, managed by either Podman or the cloud-based solution.

Best Practices for Integration Testing

Optimizing Test Reliability

When setting up integration tests in an environment that does not use a Docker Engine, adhering to best practices is crucial. Below are some recommendations to maximize test stability and performance:

Container Management

Ensure that your alternative runtime (Podman or cloud-based container engine) is properly configured and that its socket or API endpoints are accessible in your test environment.
Regularly update the container images to leverage bug fixes and performance improvements, particularly for LocalStack which is actively maintained.
Monitor the lifecycle of containers. If tests tend to leave behind dangling resources, consider employing cleanup routines in your lifecycle manager.

AWS Client Configuration

When interacting with LocalStack, configuring your AWS SDK client properly is key:

Use the endpoint provided by LocalStackContainer; this guarantees that all AWS calls are rerouted to your local testing environment.
Set static credentials (“test” is often used) in a secure manner. In production-like tests, ensure that these are separated from real AWS credentials.
Validate responses from LocalStack to confirm that its behavior closely mimics production AWS services.

CI/CD Considerations

Running integration tests in CI/CD pipelines might involve additional nuances:

Ensure that your CI environment has Podman installed or is configured to utilize Testcontainers Cloud.
Set appropriate environment variables such as TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE.
Disable any Quarkus features that may conflict with your container runtime as described earlier.

A Comparative Table of Key Configurations

Aspect	Description	Configuration Example
Dependency Setup	Include Testcontainers and LocalStack dependencies for integration tests.	Maven: add dependencies for `junit-jupiter` and `localstack`.
Alternative Runtime	Use Podman or Testcontainers Cloud to bypass Docker Engine	Set `TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE` to Podman’s socket.
Container Lifecycle	Manage starting and stopping of LocalStack using a lifecycle manager.	Implement `QuarkusTestResourceLifecycleManager` in Kotlin.
Quarkus Config	Disable Docker-dependent DevServices.	`quarkus.devservices.enabled=false` in `application.properties`.

Advanced Customizations and Troubleshooting

Handling Specific Challenges

During integration testing, some challenges may be encountered, such as:

Ryuk Container Issues: Testcontainers uses a container called "Ryuk" for resource cleanup. If running without Docker Desktop, ensure that your alternative runtime supports this or consider manually controlling container shutdown.
Networking and Endpoint Overrides: For LocalStack, always verify that the AWS client is correctly configured to use the endpoint provided by the container. Misconfiguration here can lead to tests failing silently.
Temporary Directory Management: CI environments sometimes require custom configurations for temporary directories, which may affect how LocalStack writes logs and handles state.

Extending Test Coverage

By building additional integration tests, you can simulate multiple AWS services simultaneously. For example, if your application uses both S3 and SQS:


    @Testcontainers
    @QuarkusTest
    @QuarkusTestResource(LocalStackTestResource::class)
    class AwsMultiServiceTest {

        companion object {
          @Container
          val localStack = LocalStackContainer("localstack/localstack:latest")
            .withServices(
                LocalStackContainer.Service.S3,
                LocalStackContainer.Service.SQS
            )
        }

        @Test
        fun testServices() {
            // Setup S3 client
            val s3Client = S3Client.builder()
                .endpointOverride(localStack.getEndpointOverride(LocalStackContainer.Service.S3))
                .region(Region.of(localStack.region))
                .credentialsProvider(StaticCredentialsProvider.create(
                    AwsBasicCredentials.create("test", "test")
                ))
                .build()

            // Setup SQS client accordingly and conduct functional tests
        }
    }

These advanced configurations enable more robust integration tests, simulating real-world scenarios where multiple AWS services interact within your application.

Additional Considerations

Security and Credential Management

While LocalStack offers an environment for testing without actual AWS resources, always take caution in managing credentials. Use distinct and temporary credentials for these tests so that they do not accidentally compromise production environments.

Performance Considerations

Using a Docker alternative may introduce different performance characteristics. Regular benchmarking and monitoring are recommended to ensure that integration tests remain efficient, particularly in a CI/CD pipeline where test execution time is critical.