exteon/docker-recipes

A library to aggregate a multistage Dockerfile from multiple Dockerfiles and add reusable templates

3.0.0 2022-07-09 18:54 UTC

Requires

• php: ~8.0.0 || ~8.1.1
• exteon/file-helper: ^1.6.2
• symfony/yaml: ^5.3

Apache-2.0

• Dinu Marina

README

Library that allows merging Dockerfile and docker-compose.yml recipes, adding modularity and reusability to docker recipes

Concepts

Templates

In the context of docker-recipes, a template is a fragment of Dockerfile, using the same syntax, that is used to piece together a final Dockerfile.

A template can require multiple other templates.

Dockerfile

With docker-recipes, a source dockerfile is a regular Dockerfile, with added syntax to include any number of templates.

docker-compose.yml

docker-recipes provides a mechanism for merging multiple docker-compose.ymls into a destination docker-compose.yml. At the same time, referenced templated Dockerfiles in docker-compose.ymls are managed to point to the generated Dockerfiles in the generated merged docker-compose.yml

AppEnv's

Many times, an app has multiple build environments it can run in. For example, it can have a live environment, with minimal footprint, and a dev environment where a number of debugging tools are added to containers (such as xdebug).

Since v3.0, docker-recipes provides a mechanism for specifying app-env's for Dockerfile templates as well as for docker-compose files. Each Dockerfile and docker-compose file has one app-env associated with it, determined based on the directory structure (for more details see Locators).

When compiling Dockerfiles and docker-compose files, an app-env stack can be provided as an array:

• Dockerfiles are then looked up using the order of the provided app-envs, (with index 0 having top priority).
• docker-compose files are stacked and merged in the reverse order of the provided app-envs (with index 0 on top).

Dockerfile composition and syntax

In both Dockerfiles and templates, the syntax to require another template is:

#TEMPLATE[ template-name ]

-or-

#TEMPLATE[ app-env/template-name ]

That is, a comment starting at the beginning of the line, with a single comment hash and followed, with no spaces, by TEMPLATE[. Between the square brackets, the template name is provided, possibly preceded by the app-env and a slash. The mechanism of locating templates is described in Locators.

The purpose of the library is, for every source Dockerfile, to generate a target Dockerfile with all templates compiled in.

When a template is required multiple times, (presumably by multiple different other templates), it will be deduplicated and included only once in the compiled Dockerfile; the concept of functional (invokable/parameterisable) templates is not implemented but is considered for future versions.

In the Dockerfile templates/images, when you have for instance a COPY instruction, you need the path to the template dir. For this, you need to use the $TEMPLATE_DIR variable which will be compiled into the proper path, i.e.:

COPY $TEMPLATE_DIR/supervisord/supervisord.conf /etc/supervisord.conf

(see this Example template)

docker-compose.yml composition and syntax

A final docker-compose.yml will be generated by merging the docker-compose.yml file returned by each locator. There is no special syntax, except when you need to reference the path to the final context the compose will run in. This is also known as "the project path" and is sent as $projectRoot to the DockerComposeCompiler constructor (see docker-compose compiling).

To reference that directory, use the ${PROJECT_DIR} environment variable in the source docker-compose.yml files.

(see this example)

Locators

Images and templates are referenced by name. In order to locate and map them to a source file, one or more locators are used. A standard locator implementation is provided, with classes StdDockerfileLocator and StdDockerComposeLocator that receives a root directory as constructor parameter; every root directory contains a set of templates and a set of images, and, in case of StdDockerComposeLocator, zero or one docker-compose.yml file.

The file structure is as follows:

<root_dir>
  ├ templates
  │  ├ template_name_1
  │  │  └ Dockerfile
  │  └ some_other_template
  │     └ Dockerfile
  ├ image_name_1
  │  └ Dockerfile
  ├ some_other_image
  │  └ Dockerfile
  └ docker-compose.yml

The difference between the two locators is that:

• StdDockerfileLocator is used by DockerfileCompiler and will only process images and templates
• StdDockerComposeLocator is used by DockerComposeCompiler and will process, in addition to images and templates, also the docker-compose.yml file.

If you need a different directory structure, you can create custom locators implementing DockerComposeLocator and DockerfileLocator interfaces.

AppEnv's directory structure

The directory structure described above is valid for the default '' (blank) app-env. For using multiple app-envs, the directory structure changes as such:

<root_dir>
  ├ common
  │  ├ templates
  │  │  ├ template_name_1
  │  │  │  └ Dockerfile
  │  │  └ some_other_template
  │  │     └ Dockerfile
  │  ├ image_name_1
  │  │ └ Dockerfile
  │  ├ some_other_image
  │  │  └ Dockerfile
  │  └ docker-compose.yml
  ├ live
  │  ├ templates
  │  │  └ template_name_1
  │  │     └ Dockerfile
  │  └ docker-compose.yml
  └ dev
     └ docker-compose.yml

Then when a DockerComposeCompiler is instanced with ['live', 'common'] as app-env chain for instance, then:

• When referenced in a #TEMPALTE[ ] tag without an app-env specified, or when referenced in a docker-compose file, templates and images will be looked up first in the live then in the common dir
• Both common/docker-compose.yml, live/docker-compose.yml will be merged, in this order.

Dockerfile compiling

Compilation of images from templates is done by DockerfileCompiler, which is instantiated as follows:

/**
 * @param DockerfileLocator[] $locators
 * @param string $targetDir
 * @param string[] $appEnv
 */
public function __construct(
   array $locators,
   string $targetDir,
   array $appEnv = ['']
)

The $targetDir is a target directory where compiled image files will be written.

To compile the images, you use:

/** @var \Exteon\DockerRecipes\DockerfileCompiler $compiler */
$compiler->compile();

This will produce a target directory that for the example source directory above, will produce:

<target_dir>
  ├ image_name_1
  │  └ Dockerfile
  └ some_other_image
     └ Dockerfile

So for every source image, there will be a target image compiled from the templates.

docker-compose compiling

Compiling recipes for docker-compose is done using DockerComposeCompiler, which compiles both images (the same as DockerfileCompiler, and a docker-compose.yml) file from the locators' docker-compose.yml files. It is instanced as:

/**
 * @param DockerComposeLocator[] $locators
 * @param string $dockerfilesTargetDir
 * @param string $composeFileTargetPath
 * @param string $projectRoot
 * @param string[] $appEnv
 */
public function __construct(
    array $locators,
    string $dockerfilesTargetDir,
    string $composeFileTargetPath,
    string $projectRoot,
    array $appEnv = ['']
)

The $dockerfilesTargetDir is the target directory for the compiled images. For more info, see Dockerfile compiling.

The $composeFileTargetPath is the target path for the docker-compose.yml file to be compiled.

The $projectRoot is the path to a directory that will be used as ${PROJECT_DIR} substitution in docker-compose.yml files.

The $appEnv is the app-env stack order used to compose the files.

To compile the images / compose files, you use:

/** @var \Exteon\DockerRecipes\DockerComposeCompiler $compiler */
$compiler->compile();

When compiling docker-compose.yml from multiple sources (multiple locators that provide a docker-compose.yml), the target file is merged from all the source files using the following algorithm: recursively, for all string keys in the yml file, values are merged (or overridden if scalar). For all sequential arrays, the contents are appended.

Example

Since an example is worth a thousand words, you can take a look at the example dir to see an example of compiling a CentOS8 image with a webserver, using templates that can be reused for building different other images (such as the supervisor-rhel8 template).

To run the example, just run php example.php. The centos8-web image recipe will be generated in example/target and the compose file at example/docker-compose.yml. To build and run the container, run composer up -d in example