Seaworthy¶

Seaworthy is a test harness for Docker container images. It allows you to use Docker containers and other Docker resources as fixtures for tests written in Python.

Seaworthy supports Python 3.4 and newer. You can find more information in the documentation.

A demo repository is available with a set of Seaworthy tests for a simple Django application. Seaworthy is also introduced in our blog post on continuous integration with Docker on Travis CI.

For more background on the design and purpose of Seaworthy, see our PyConZA 2018 talk (slides).

Quick demo¶

First install Seaworthy along with pytest using pip:

pip install seaworthy[pytest]

Write some tests in a file, for example, test_echo_container.py:

from seaworthy.definitions import ContainerDefinition

container = ContainerDefinition(
    'echo', 'jmalloc/echo-server',
    wait_patterns=[r'Echo server listening on port 8080'],
    create_kwargs={'ports': {'8080': None}})
fixture = container.pytest_fixture('echo_container')


def test_echo(echo_container):
    r = echo_container.http_client().get('/foo')
    assert r.status_code == 200
    assert 'HTTP/1.1 GET /foo' in r.text

Run pytest:

pytest -v test_echo_container.py

Project status¶

Seaworthy should be considered alpha-level software. It is well-tested and works well for the first few things we have used it for, but we would like to use it for more of our Docker projects, which may require some parts of Seaworthy to evolve further. See the project issues for known issues/shortcomings.

The project was originally split out of the tests we wrote for our docker-django-bootstrap project. There are examples of Seaworthy in use there.

Getting started¶

Installation¶

Seaworthy can be installed using pip:

pip install seaworthy

The pytest and testtools integrations can be used if those libraries are installed, which can be done using extra requirements:

pip install seaworthy[pytest,testtools]

Defining containers for tests¶

Containers should be defined using subclasses of ContainerDefinition. For example:

from seaworthy.definitions import ContainerDefinition
from seaworthy.logs import output_lines


class CakeContainer(ContainerDefinition):
    IMAGE = 'acme-corp/cake-service:chocolate'
    WAIT_PATTERNS = (
        r'cake \w+ is baked',
        r'cake \w+ is served',
    )

    def __init__(self, name):
        super().__init__(name, self.IMAGE, self.WAIT_PATTERNS)

    # Utility methods can be added to the class to extend functionality
    def exec_cake(self, *params):
        return output_lines(self.inner().exec_run(['cake'] + params))

WAIT_PATTERNS is a list of regex patterns. Once these patterns have been seen in the container logs, the container is considered to have started and be ready for use. For more advanced readiness checks, the wait_for_start() method should be overridden.

This container can then be used as fixtures for tests in a number of ways, the easiest of which is with pytest:

import pytest

container = CakeContainer('test')
fixture = container.pytest_fixture('cake_container')

def test_type(cake_container):
    output = cake_container.exec_cake('type')
    assert output == ['chocolate']

A few things to note here:

The pytest_fixture() method returns a pytest fixture that ensures that the container is created and started before the test begins and that the container is stopped and removed after the test ends.
The scope of the fixture is important. By default, pytest fixtures have function scope, which means that for each test function the fixture is completely reinitialized. Creating and starting up a container can be a little slow, so you need to think carefully about what scope to use for your fixtures. See ContainerDefinition.clean for a way to avoid container setup/teardown overhead.

For simple cases, ContainerDefinition can be used directly, without subclassing:

container = ContainerDefinition(
    'test', 'acme-corp/soda-service:cola', [r'soda \w+ is fizzing'])
fixture = container.pytest_fixture('soda_container')

def test_refreshment(soda_container):
    assert 'Papor-Colla Corp' in soda_container.get_logs()

Note that pytest is not required to use Seaworthy and there are several other ways to use the container as a fixture. For more information see Test framework integration and Resource definitions & helpers.

Resource definitions & helpers¶

Two important abstractions in Seaworthy are resource definitions and helpers. These provide test-oriented interfaces to all of the basic (non-Swarm) Docker resource types.

Definitions¶

Resource definitions provide three main functions:

Make it possible to define resources before those resources are actually created in Docker. This is important for creating test fixtures—if we can define everything about a resource before it is created, then we can create the resource when it is needed as a fixture for a test.
Simplify the setup and teardown of resources before and after tests. For example, ContainerDefinition can be used to check that a container has produced certain log lines before it is used in a test.
Provide useful functionality to interact with and introspect resources. For example, the http_client() method can be used to get a simple HTTP client to make requests against a container.

Resource defintions can either be instantiated directly or subclassed to provide more specialised functionality.

For a simple volume, one could create an instance of VolumeDefinition:

from seaworthy.definitions import VolumeDefinition
from seaworthy.helpers import DockerHelper


docker_helper = DockerHelper()
volume = VolumeDefinition('persist', helper=docker_helper)

Using definitions in tests¶

Definitions can be used as fixtures for tests in a number of different ways.

As a context manager:

with VolumeDefinition('files', helper=docker_helper) as volume:
    assert volume.created

assert not volume.created

With the as_fixture decorator:

network = NetworkDefinition('lan_network', helper=docker_helper)

@network.as_fixture()
def test_network(lan_network):
    assert lan_network.created

When using pytest, it’s easy to create a fixture:

container = ContainerDefinition('nginx', 'nginx:alpine')
fixture = container.pytest_fixture('nginx_container')

def test_nginx(nginx_container):
    assert nginx_container.created

You can also use classic xunit-style setup/teardown:

import unittest


class EchoContainerTest(unittest.TestCase):
    def setUp(self):
        self.helper = DockerHelper()
        self.container = ContainerDefinition('echo', 'jmalloc/echo-server')
        self.container.setup(helper=self.helper)
        self.addCleanup(self.container.teardown)

    def test_container(self):
        self.assertTrue(self.container.created)

Relationship to helpers¶

Every resource definition instance needs to have a “helper” set before it is possible to actually create the Docker resource that the instance defines. Resource helpers are described in more detail later in this section, but for now, know that a helper needs to be provided to the definition in one of three ways:

Using the helper keyword argument in the constuctor:

helper = DockerHelper()
network = NetworkDefinition('net01', helper=helper)
network.setup()

Using the helper keyword argument in the setup() method:

helper = DockerHelper()
volume = VolumeDefinition('vol02')
volume.setup(helper=helper)

Directly, using the set_helper() method:

helper = DockerHelper()
container = ContainerDefinition('con03', 'nginx:alpine')
container.set_helper(helper)
container.setup()

This only needs to be done once for the lifetime of the definition.

For the most part, interaction with Docker should almost entirely occur via the definitions, but the definition objects need the helpers to actually interact with Docker.

Mapping to Docker SDK types¶

Each resource definition wraps a model from the Docker SDK for Python. The underlying model can be accessed via the inner() method, after the resource has been created. The mapping is as follows:

Seaworthy resource definition	Docker SDK model
`ContainerDefinition`	`docker.models.containers.Container`
`NetworkDefinition`	`docker.models.networks.Network`
`VolumeDefinition`	`docker.models.volumes.Volume`

Helpers¶

Resource helpers provide two main functions:

Namespacing of resources: by prefixing resource names, the resources are isolated from other Docker resources already present on the system.
Teardown (cleanup) of resources: when the tests end, the networks, volumes, and containers used in those tests are removed.

In addition, some of the behaviour around resource creation and removal is changed from the Docker defaults to be a better fit for a testing environment.

Accessing the various helpers is most easily done via the DockerHelper:

from seaworthy.helpers import DockerHelper


# Create a DockerHelper with the default namespace, 'test'
docker_helper = DockerHelper()

# Create a network using the NetworkHelper
network = docker_helper.networks.create('private')

# Create a volume using the VolumeHelper
volume = docker_helper.volumes.create('shared')

# Fetch (pull) an image using the ImageHelper
image = docker_helper.images.fetch('busybox')

# Create a container using the ContainerHelper
container = docker_helper.containers.create(
    'conny', image, network=network, volumes={volume: '/vol'})

The DockerHelper can be configured with a custom Docker API client. The default client can be configured using environment variables. See docker.client.from_env().

Mapping to Docker SDK types¶

Each resource helper wraps a “model collection” from the Docker SDK. The underlying collection can be accessed via the collection attribute. The mapping is as follows:

Seaworthy resource helper	Docker SDK model collection
`ContainerHelper`	`docker.models.containers.ContainerCollection`
`ImageHelper`	`docker.models.images.ImageCollection`
`NetworkHelper`	`docker.models.networks.NetworkCollection`
`VolumeHelper`	`docker.models.volumes.VolumeCollection`

Test framework integration¶

We have strong opinions about the testing tools we use, and we understand that other people may have equally strong opinions that differ from ours. For this reason, we have decided that none of Seaworthy’s core functionality will depend on pytest, testtools, or anything else that might get in the way of how people might wants to write their tests. On the other hand, we don’t want to reinvent a bunch of integration and helper code for all the third-party testing tools we like, so we also provide optional integration modules where it makes sense to do so.

pytest¶

Seaworthy is a pytest plugin and all the functions and fixtures in the seaworthy.pytest module will be available when Seaworthy is used with pytest.

`docker_helper` fixture¶

A fixture for a DockerHelper instance is defined by default.

This fixture creates DockerHelper instances with default parameters and has module-level scope. Since all other Docker resource fixtures typically depend on the docker_helper fixture, resources must have a scope smaller than or equal to the docker_helper’s scope.

The defaults for this fixture can be overridden by defining a new docker_helper fixture using the docker_helper_fixture() fixture factory. For example:

from seaworthy.pytest.fixtures import docker_helper_fixture

docker_helper = docker_helper_fixture(scope='session', namespace='seaworthy')

…would change the scope of the docker_helper fixture to the session-level and change the namespace of created Docker resources to seaworthy.

`dockertest` decorator¶

The dockertest() decorator can be used to mark tests that require Docker to run. These tests will be skipped if Docker is not available. It’s possible that some tests in your test suite may not require Docker and you may want to still be able to run your tests in an environment that does not have Docker available. The decorator can be used as follows:

@dockertest()
def test_docker_thing(cake_container):
    assert cake_container.exec_cake('variant') == ['gateau']

Fixture factories¶

A few functions are provided in the seaworthy.pytest.fixtures module that are factories for fixtures. The most important two are:

resource_fixture(definition, name, scope='function', dependencies=())[source]

Create a fixture for a resource.

Note

This function returns a fixture function. It is important to keep a reference to the returned function within the scope of the tests that use the fixture.

fixture = resource_fixture(PostgreSQLContainer(), 'postgresql')

def test_container(postgresql):
    """Test something about the PostgreSQL container..."""

Parameters:	definition – A resource definition, one of those defined in the `seaworthy.definitions` module. name – The fixture name. scope – The scope of the fixture. dependencies – A sequence of names of other pytest fixtures that this fixture depends on. These fixtures will be requested from pytest and so will be setup, but nothing is done with the actual fixture values.
Returns:	The fixture function.

clean_container_fixtures(container, name, scope='class', dependencies=())[source]

Creates a fixture for a container that can be “cleaned”. When a code block is marked with @pytest.mark.clean_<fixture name> then the clean method will be called on the container object before it is passed as an argument to the test function.

Note

This function returns two fixture functions. It is important to keep references to the returned functions within the scope of the tests that use the fixtures.

f1, f2 = clean_container_fixtures(PostgreSQLContainer(), 'postgresql')

class TestPostgresqlContainer
    @pytest.mark.clean_postgresql
    def test_clean_container(self, web_container, postgresql):
        """
        Test something about the container that requires it to have a
        clean state (e.g. database table creation).
        """

    def test_dirty_container(self, web_container, postgresql):
        """
        Test something about the container that doesn't require it to
        have a clean state (e.g. testing something about a dependent
        container).
        """

Parameters:	container – A “container” object that is a subclass of `ContainerDefinition`. name – The fixture name. scope – The scope of the fixture. dependencies – A sequence of names of other pytest fixtures that this fixture depends on. These fixtures will be requested from pytest and so will be setup, but nothing is done with the actual fixture values.
Returns:	A tuple of two fixture functions.

testtools¶

We primarily use testtools when matching against complex data structures and don’t use any of its test runner functionality. Currently, testtools matchers are only used for matching PsTree objects. See the API documentation for the seaworthy.ps module.

Testing our integrations¶

To make sure that none of the optional dependencies accidentally creep into the core modules (or other optional modules), we have several sets of tests that run in different environments:

tests-core: This is a set of core tests that cover basic functionality. tox -e py36-core will run just these tests in an environment without any optional or extra dependencies installed.
tests-pytest, etc.: These are tests for the optional pytest integration modules. tox -e py36-testtools will run just the seaworthy.pytest modules’ tests in an environment with only the necessary dependencies installed.
tests-testtools, etc.: These are tests for the optional testtools integration module. tox -e py36-testtools will run just the seaworthy.testtools module’s tests.
tests: These are general tests that are hard or annoying to write with only the minimal dependencies, so we don’t have any tooling restrictions here. tox -e py36-full will run these, as well as all the other test sets mentioned above, in an environment with all optional dependencies (and potentially some additional test-only dependencies) installed.

Frequently asked questions¶

What about TestContainers?¶

Seaworthy’s goals have some overlap with TestContainers, but our current primary use case is testing the behaviour of Docker images, rather than providing a way to use Docker containers to test other software. Also, Seaworthy is written in Python rather than Java.

What are the similarities between Seaworthy and docker-compose?¶

Seaworthy does try to reuse some of the default behaviour that docker-compose implements in order to make it easier and faster to start running containers.

All Docker resources (networks, volumes, containers) are namespaced by prefixing the resource name, e.g. a container called cake-service could be namespaced to have the name test_cake-service.
A new bridge network is created by default for containers where no network is specified.
Containers are given network aliases with their names, making it easier to connect one container to another.

Both Seaworthy and docker-compose are built using the official Docker SDK for Python.

…what are the differences?¶

Seaworthy is fundamentally designed for a different purpose. docker-compose uses YAML files to define Docker resources—it does not have an API for this. With Seaworthy, all Docker resources are created programmatically, typically as fixtures for tests.

Seaworthy includes functionality specific to its purpose:

Predictable setup/teardown processes for all resources.
Various utilities for inspecting running containers, e.g. for matching log output, for listing running processes, or for making HTTP requests against containers.
Integrations with popular Python testing libraries (pytest and testtools).

Seaworthy currently lacks some of the functionality of docker-compose:

The ability to build images for containers
Any sort of Docker Swarm functionality
Any concept of multiple instances of containers
Probably other things…

What about building images?¶

Seaworthy doesn’t currently implement an interface for building images. In most cases, we expect users to build their images in a previous step of their continuous integration process and then use Seaworthy to test that image. However, there may be cases where having Seaworthy build Docker images would make sense, such as if an image is built purely to be used in tests.

API Reference¶

`seaworthy`	seaworthy
`seaworthy.checks`	Checks and test decorators for skipping tests that require Docker to be present.
`seaworthy.client`	A requests-based HTTP client for interacting with containers that have forwarded ports.
`seaworthy.containers.nginx`
`seaworthy.containers.postgresql`	PostgreSQL container definition.
`seaworthy.containers.rabbitmq`	RabbitMQ container definition.
`seaworthy.containers.redis`	Redis container definition.
`seaworthy.definitions`	Wrappers over Docker resource types to aid in setup/teardown of and interaction with Docker resources.
`seaworthy.helpers`	Classes that track resource creation and removal to ensure that all resources are namespaced and cleaned up after use.
`seaworthy.ps`	Tools for asserting on processes running in containers using `ps`.
`seaworthy.pytest`	Some (optional) utilities for use with pytest.
`seaworthy.pytest.checks`	pytest mark to skip tests that require Docker.
`seaworthy.pytest.fixtures`	A number of pytest fixtures or factories for fixtures.
`seaworthy.stream`
`seaworthy.stream.logs`
`seaworthy.stream.matchers`
`seaworthy.testtools`	Some (optional) utilities for use with testtools.
`seaworthy.utils`

Seaworthy¶

Quick demo¶

Project status¶

Getting started¶

Installation¶

Defining containers for tests¶

Resource definitions & helpers¶

Definitions¶

Using definitions in tests¶

Relationship to helpers¶

Mapping to Docker SDK types¶

Helpers¶

Mapping to Docker SDK types¶

Test framework integration¶

pytest¶

docker_helper fixture¶

dockertest decorator¶

Fixture factories¶

testtools¶

Testing our integrations¶

Frequently asked questions¶

What about TestContainers?¶

What are the similarities between Seaworthy and docker-compose?¶

…what are the differences?¶

What about building images?¶

API Reference¶

Indices and tables¶

`docker_helper` fixture¶

`dockertest` decorator¶