How to Use Docker Buildx Bake to Create Complex Image Build Pipelines

Estimated read time 6 min read


Graphic showing the Docker logo

The docker buildx command group uses BuildKit to expose advanced image build capabilities. Baked builds are a high-level feature that can be used to define automated build pipelines. They lets you produce multiple images from a single build operation.

Baked workflows are helpful when you want to publish different variants of your images or build several linked projects in parallel. In this article we’ll cover the key features of docker buildx bake and how you can use them to streamline complex builds.

Getting Started

The docker buildx bake command executes multiple build “targets” that each produce a container image. Targets run in parallel where possible to maximize performance. Targets may also directly reference predecessors to create sequential pipelines.

Build targets can be defined using several different mechanisms including existing Docker Compose files. Buildx will automatically build all the images identified in the file.

More advanced features are exposed when you list build targets in JSON or HCL files. These support variables, functions, and value interpolation to customize your builds.

The buildx bake command looks for the following files in order:

  • docker-compose.yml
  • docker-compose.yaml
  • docker-bake.json
  • docker-bake.override.json
  • docker-bake.hcl
  • docker-bake.override.hcl

You can specify a different file with the -f command flag.

Build Targets

Build targets encapsulate all the configuration related to your build. They include details such as

  • the path to the Dockerfile to build
  • build context paths, defining the content available within your Dockerfile
  • tags and labels to attach to the output images
  • the platforms to produce images for.

A complete list of supported config fields is available in the documentation. Previously you may have supplied these settings as command-line flags to docker buildx build (or even plain docker build), forcing you to remember the correct values each time. With buildx bake you can reliably use the same values by defining them in your version-controlled baked file.

Here’s a simple example of a docker-bake.hcl command that defines a single build target:

target "default" {
    dockerfile = "app/Dockerfile"
    contexts = {
        app = "app/src"
        shared = "shared-components/src"
    }
    tags = ["my-app:latest", "docker.io/my-org/my-app:latest"]
}

Running docker buildx bake with this bake file will load the app/Dockerfile Dockerfile from your working directory. It’ll have access to the app/src and shared-components/src directories as build contexts. The image that’s produced will be assigned two tags.

The default target is built automatically when you run docker buildx bake. You can also define named targets that can be built on-demand:

target "app" {
    // ...
}
$ docker buildx bake app

Using Multiple Targets

You can build another image simultaneously by defining it as a new target inside your bake file:

group "default" {
    targets = ["app", "api"]
}

target "app" {
    dockerfile = "app/Dockerfile"
    contexts = {
        app = "app/src"
        shared = "shared-components/src"
    }
    tags = ["my-app:latest", "docker.io/my-org/my-app:latest"]
}

target "api" {
    dockerfile = "api/Dockerfile"
    contexts = {
        src = "https://www.howtogeek.com/devops/how-to-use-docker-buildx-bake-to-create-complex-image-build-pipelines/api/src"
    }
    tags = ["my-api:latest", "docker.io/my-org/my-api:latest"]
}

These images can be built simultaneously because they’re nested into a group. The api and app images will be built in parallel each time you run the docker buildx bake command as the default group is automatically selected. You can use named groups similarly to the named targets example above.

Build Target Inheritance

Build targets can inherit from each other to reuse configuration. One scenario where this can be useful concerns images that need to be customized for different environments. You might want to add extra config files to image variants intended for development use. Here’s a docker-bake.hcl that demonstrates this model:

group "default" {
    targets = ["backend", "backend-dev"]
}

target "backend" {
    dockerfile = "backend/Dockerfile"
    contexts = {
        src = "https://www.howtogeek.com/devops/how-to-use-docker-buildx-bake-to-create-complex-image-build-pipelines/api/src"
        config = "api/config"
    }
    tags = ["backend:latest"]
}

target "backend-dev" {
    inherits = ["backend"]
    contexts = {
        config = "api/config-dev"
    }
    tags = ["backend:dev"]
}

The backend-dev target inherits all the properties of the backend target but overrides the config context and applies a different tag.

You can preview the merged file structure by running the bake command with the --print flag:

$ docker buildx bake --print
...
    "backend-dev": {
      "context": ".",
      "contexts": {
        "config": "api/config-dev",
        "src": "https://www.howtogeek.com/devops/how-to-use-docker-buildx-bake-to-create-complex-image-build-pipelines/api/src"
      },
      "dockerfile": "backend/Dockerfile",
      "tags": [
        "backend:dev"
      ]
    }
...

Using a Previous Target as a Base Image

Sometimes you might want a build target to use the image created by a previous target as its own base. This is an alternative to multi-stage builds that can be used when your Dockerfiles depend on each other but can’t be merged together, perhaps because they exist in different projects.

group "default" {
    targets = ["org-base-image", "api"]
}

target "org-base-image" {
    dockerfile = "docker-base/Dockerfile"
    tags = ["org-base-image:latest"]
}

target "api" {
    dockerfile = "api/Dockerfile"
    contexts = {
        base = "target:org-base-image"
    }
    tags = ["api:latest"]
}

The example first builds the org-base-image target. This could contain some utilities that are common to your organization’s containerized workloads. The api target is then built with the output from the org-base-image target accessible as the base build-context. The API Dockerfile can now reference content inside the base image:

COPY --from=base /utilities/example /usr/bin/example-utility

This is a powerful pattern that lets you create dependency links between images while maintaining separate Dockerfiles.

Overriding Properties of Targets at Build Time

The docker buildx bake command lets you override properties of your targets when you run your build:

$ docker buildx bake --set api.dockerfile="api/Dockerfile-dev"

This example changes the Dockerfile of the api target. The * wildcard is supported when identifying the target to change. * on its own selects every target while api* will modify all the targets that begin with api.

Setting Variables

HCL files can define variables that you can reference in your build targets. use a variable block to set them up:

variable "TAG" {
    default = "latest"
}

group "default" {
    targets = ["app"]
}

target "app" {
    dockerfile = "src/Dockerfile"
    tags = ["my-app:${TAG}"]
}

Running docker buildx bake with this configuration will tag the app target as my-app:latest. You can change the value of the TAG variable by setting an environment variable before you execute the command:

$ TAG=v1 docker buildx bake

You can use all the variable interpolation and comparison capabilities of the HCL language to make your build targets reusable. Functions are available too for parsing and transforming your values.

Summary

Baked Buildx builds let you encapsulate image build configuration as “targets” defined in a file. When you run buildx bake, images for all the referenced targets are built in parallel.

Targets can inherit from and depend on each other. You can also use variables and functions to create highly complex and configurable build pipelines.

The docker buildx bake command is a high-level operation that’s not necessary in every workflow. You don’t need to use it when you’re creating simple images with no cross-project dependencies. Using docker compose build is a better alternative for most use cases that keeps build configuration in your docker-compose.yml file. Switching to baked builds should be considered when you’re building many images simultaneously using different variables, platforms, build contexts, and config overrides.





Source link

You May Also Like

More From Author