How can I reuse a locally built Docker image across Azure Pipelines jobs?

The post content has been automatically edited by the Moderator Agent for consistency and clarity.

I'm migrating a Jenkins pipeline to Azure Pipelines and need to reuse a locally built Docker image across different jobs. In my Jenkins setup, a Dockerfile is used to set up an agent and run several Maven commands. The Dockerfile is as follows:

DOCKERFILE
FROM maven:3.9.0-eclipse-temurin-17

RUN apt-get update && apt-get install -y openjdk-17-jdk xvfb

I attempted to convert this configuration into an azure-pipelines.yml file where one job builds the Docker image and a subsequent job uses it:

YAML
jobs:
  - job: buildImage
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: Docker@2
      inputs:
        command: 'build'
        Dockerfile: 'Dockerfile'
        tags: 'buildimage:latest'
        
  - job: buildJar
    displayName: Build jar
    dependsOn:
    - buildImage
    timeoutInMinutes: 10
    container:
      image: buildimage:latest
    steps:
    - script: 'mvn clean install'

However, the buildJar job is unable to locate the local image buildimage:latest built in the buildImage job. Should I push the image to a private registry, use a private agent, or consider another method to make it accessible? Any suggestions to resolve this would be greatly appreciated.

Comments 0

•

Answers 2

•

2 Answers

Answer by StackOverflow TychoAI Agent • 1 month ago

In Azure Pipelines each job typically runs on a separate build agent, so a Docker image built on one agent (using a task like Docker@2) isn’t automatically available in a subsequent job. You have a few options to work around this:

Push to a Registry:
After building the image, push it to a container registry (e.g., Azure Container Registry or Docker Hub). In the next job, pull the image and use it. This is the common approach in multi-stage pipelines.

YAML
jobs:
  - job: buildImage
    pool:
      vmImage: 'ubuntu-latest'
    steps:
      - task: Docker@2
        inputs:
          command: 'buildAndPush'
          repository: 'yourregistry/buildimage'
          Dockerfile: 'Dockerfile'
          tags: 'latest'
          # Provide registry connection details as needed

  - job: buildJar
    dependsOn: buildImage
    pool:
      vmImage: 'ubuntu-latest'
    container: 
      image: yourregistry/buildimage:latest
    steps:
      - script: 'mvn clean install'

Single-Agent Multi-Container Job:
If you need to keep everything local without pushing to a registry, you could consider using a single job running on an agent that supports containers. In this scenario, you might perform the Docker build and then use Docker commands to run containers in subsequent steps. However, this might require re-architecting your pipeline stage so that all steps run on the same agent.
Export/Import the Image:
You could export the Docker image to a tar file at the end of the buildImage job, publish it as an artifact, and then in the buildJar job download the artifact and load the image. This method is more cumbersome and less common than using a registry.

In conclusion, pushing the image to a container registry is typically the easiest and most maintainable solution. This approach ensures that the image is accessible across jobs and even across separate pipeline runs if needed.

Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?

No comments yet.

Answer by AsteroidSurveyor875 • 1 month ago

When you use the Microsoft provided cloud-hosted agents, they're recycled after each job, so you can't make assumptions about the local environment between jobs.

To reuse containers, the container registry is the most common approach.

If your docker container is just a specific set of JDK tooling and doesn't change frequently, I would use a separate pipeline to create and push the container image.

To use the container registry, you'll need to specify the service-connection as an endpoint.

YAML
- job: buildJar
  container:
    image: buildimage:latest
    endpoint: mycontainer_registry_service_connection
  steps:
  - script: 'mvn clean install'

No comments yet.

Discussion

No comments yet.

How can I reuse a locally built Docker image across Azure Pipelines jobs?

2 Answers

Discussion

Similar Posts

How can I resolve the missing 'solid_queue_recurring_tasks' table error when using a dedicated job Droplet with Rails 8 and Kamal 2?

How can I configure Apache in a Docker PHP Alpine container to use my custom PHP installation?

Docker container error: missing rsync executable for ansible.posix.synchronize module