Use Concourse to Build a Docker Image

This blog will describe how to use Concourse to build a Docker Image through a simple example.

The easiest and cheapest way to get a Concourse instance going is to use a pre-built Vagrant box. We will use Vagrant with VirtualBox, since it is built-in to Vagrant and free. You can easily install them by following the instructions in the following links.

• Vagrant
• VirtualBox

Spin up Concourse with Vagrant and Install Fly

To spin up Concourse with Vagrant, run the following command in any directory.

$ vagrant init concourse/lite
-> It creates ./Vagrantfile.

$ vagrant up
-> It downloads the box and spins up the VM.

The web server will be running at 192.168.100.4:8080.It does not have any pipeline configured now. We will write a pipeline which builds a Docker Image for Go Running/Testing Environment in the next section.

Write the Dockerfile and Pipelines

It is a good idea to always follow the 12-factor design patterns when we work on different projects from tiny ones such as the example in this blog to huge projects involving multiple sub-projects. Hence the first step is setting up a git repo for the project, then we will create a Dockerfile which will be built in the pipeline to create a Docker Image and a YAML file for pipeline configure.

• 12-Factor Apps in Plain English

Set up a git repo

Inside the directory we spin up Concourse, run the following commands to set up a git repo locally.

$ mkdir concourse-build-docker-image

$ cd concourse-build-docker-image

$ git init
-> Creates a .git subdirectory in the project root: concourse-build-docker-image.

$ echo "# Concourse-Build-Docker-Image" >> README.md
-> It creates a readme file and add title to it.

$ git add README.md

$ git commit -m "Initial commit"

Next we will push this local repo to Github. In your github account, create a repo, let’s name it Concourse-Build-Docker-Image. At the top of your GitHub repository’s Quick Setup page, click to copy the remote repository URL.

$ git remote add origin remote repository URL
-> Sets up the new remote.

$ git remote -v
-> Verifies the new remote URL.

$ git push -u origin master
-> Push local master to remote master branch.

We can use the normal git workflow from now on.

Before we move on to the next step, let’s create a directory called ci, then we will create the Dockerfile and pipeline.yml inside the ci folder.

Create the Dockerfile

In this example, we create a Docker Image which supports Running/Testing a Go App. Depending on what packages you are using, you may not need to install all the packages in the following Dockerfile.
Please check Dockerfile Reference for details about how to create a Dockerfile.

FROM concourse/concourse-ci
#https://github.com/concourse/concourse/blob/master/ci/dockerfiles/concourse-ci/Dockerfile
RUN apt-get update && apt-get install curl wget bzr -y
ADD http://stedolan.github.io/jq/download/linux64/jq /usr/bin/
RUN chmod 775 /usr/bin/jq
# Install Go
# mkdir -p /goroot will create goroot dir under the root in the docker
# curl url will output to stand out, with | to direct it to tar, which will extract the pre-compiled
# golang to /goroot
RUN \
  mkdir -p /goroot /gopath && \
  curl https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz | \
  tar xvzf - -C /goroot --strip-components=1
# Set environment variables.
ENV GOROOT /goroot
ENV GOPATH /gopath
ENV PATH $GOROOT/bin:$GOPATH/bin:$PATH
RUN go get golang.org/x/tools/cmd/vet
RUN go get golang.org/x/tools/cmd/cover
RUN go get github.com/golang/lint/golint
RUN go get github.com/tools/godep
RUN go get github.com/laher/goxc

Write the YAML file

We first create the yml file by running the following command: touch pipeline.yml.
A simple yml file looks like this:

---
jobs:
- name: build-docker-image
  public: true
  serial: true
  plan:
  - get: git-concourse-build-docker-image
    trigger: true
  - put: docker-image
    params:
      build: git-concourse-build-docker-image/ci
resources:
- name: git-concourse-build-docker-image
  type: git
  source:
    uri: git repo uri
    branch: master
    private_key: {{github-private-key}}
- name: docker-image
  type: docker-image
  source:
    email: {{docker-hub-email}}
    username: {{docker-hub-username}}
    password: {{docker-hub-password}}
    repository: {{docker-create-concourse-go-image}}

Put all the credentials in a separate file, if it is in the same repo, please remember to add the credentials file to .gitignore. NEVER NEVER commit your credentials to the public! It seems like a beautiful lady naked walking in the public, that is VERY dangerous!

Next git commit and push our change to the remote, run the fly command in the repo root directory to configure the pipeline.

fly configure concourse-build-docker-image --vars-from credentials.yml -c ci/pipeline.yml

This will configure a pipeline named councourse-build-docker-image using the configuration in ci/pipeline.yml.Now if you refresh 192.168.100.4:8080, you will see councourse-build-docker-image.

Newly configured pipelines are paused by default, to unpause it you can either reveal the pipelines sidebar via the hamburger icon in the top left and press "play", or run:

fly configure concourse-build-docker-image --vars-from credentials.yml -c ci/pipeline.yml --paused=false

When you see the pipeline become green, congratulations! Go to your Docker Hub account, you should be able to see the Docker Image you built with Concourse!