Part 3: Bolting on a real Sidecar in our Buildpack

Sidecars allow additional processes that share filesystem and socks with a Cloud Foundry application. We investigate separating these concerns from app developers by distributing and installing them with Buildpacks.

This article is almost Part 3 in a 5-part series. Part 1 starts with Tim Downey’s How to Push an App to Cloud Foundry with Sidecars. In Part 2: Installing Cloud Foundry sidecars with Buildpacks, I demonstrated the idea of distributing a Sidecar within a Buildpack. In this article, Part 3, we will finish the job of installing a working Sidecar executable into a Buildpack.

In Part 4 we will refactor the Buildpack to allow it to be installed as a system buildpack for all users to use, and to include cached assets so it can be installed into air-gapped/internet-starved Cloud Foundry foundations.

Finally, in Part 5 we will look at integration testing our buildpack within applications running on Cloud Foundry to ensure they work.

---

At the end of Part 2 our sidecar was tiny, and silly. It did nothing. Now let’s replace it with a real application that our main application can interact over TCP to get some secret configuration from the sidecar. Our example app has /config endpoint to display the config it fetched over TCP from the sidecar.

$ curl -k https://app-using-config-server.dev.cfdev.shHi, I'm an app with a sidecar!
$ curl -k https://app-using-config-server.dev.cfdev.sh/config
{"Scope":"some-service.admin","Password":"not-a-real-p4$$w0rd"}

In the new “Part 3” repository https://github.com/starkandwayne/part3-sidecar-buildpack we see some changes from Part 2 project.

Our repository now contains the buildpack (bin/supply), the source code for the installed sidecar, and our sample/test application. The source code for the sidecar could be in a separate repository. Sample applications are best inside the buildpack repository.

Our sample application will attempt to communicate with its sidecar over a TCP port. So that both applications know which port to communicate, they look at the shared environment variable CONFIG_SERVER_PORT.

  get '/config' do
    response = Typhoeus.get("localhost:#{ENV['CONFIG_SERVER_PORT']}/config/")
    if response.body.size > 0
      response.body
    else
      {"error": "error connecting to local config-server"}.to_json
    end
  end

No clients outside the application container can reach the sidecar TCP port.

Application manifest

To deploy our sample application, with the buildpack and its sidecar, and run the sidecar:

cf v3-create-app app-using-config-server
cf v3-apply-manifest -f fixtures/rubyapp/manifest.yml
cf v3-push app-using-config-server -p fixtures/rubyapp

In our example we will set the shared CONFIG_SERVER_PORT environment variable in the application’s manifest.

applications:
- name: app-using-config-server
  env:
    CONFIG_SERVER_PORT: 8082
  buildpacks:
  - https://github.com/starkandwayne/part3-sidecar-buildpack
  - ruby_buildpack
  sidecars:
  - name: config-server
    process_types: [web]
    command: './config-server

The role of the application manifest above is to describe how to run the application and its sidecars.

The role of the supply buildpack is to install a working binary and any dependencies. We need the buildpack’s bin/supply to install ./config-server.

Our sidecar application is a Golang application. One option for implementing our buildpack could be to install Golang and compile our source code to ./config-server.

Instead, our buildpack – and most buildpacks that you’ll encounter or write for yourself – assumes that the binary has already been compiled, and is available for download into Cloud Foundry.

bin/supply

Our implementation of bin/supply fetches the binary the Internet, places it at ./config-server, and marks it as executable.

curl -sSf $(cat .downloadurl) -o $BUILD_DIR/config-server
chmod +x $BUILD_DIR/config-server

When Cloud Foundry runs our bin/supply script – during the staging process – the first argument provided to the script is the “build directory”. That is, the root folder of our application. Our script renames this first argument as $BUILD_DIR, downloads the binary from the contents of the .downloadurl file, and places it within $BUILD_DIR/config-server. During the runtime of our application, it will therefore be available as ./config-server which is how we reference it within the manifest’s sidecars: section.

Your bin/supply scripts need to put files within $BUILD_DIR or one of the other directories provided, else they will be ignored.

Pre-compiled assets for download

At the time of writing, the .downloadurl file contains a public URL to a file on AWS S3 https://s3.amazonaws.com/sample1-sidecar-buildpack/config-server-sidecar/config-server-v0.0.1.

How did our source code get compiled, placed within this S3 bucket, and be publicly available for download?

Your buildpack bin/supply script can download pre-compiled assets from anywhere it wants. If your Cloud Foundry is air gapped – does not have access to the public Internet – then you will need your assets to be stored somewhere private that your Cloud Foundry staging containers can access. We will consider this problem in our next article.

For my example buildpack I’ve authored a build_and_upload.sh script to compile the source code, and upload it to an AWS S3 bucket.

This script is run locally to my machine, or within my CI environment. It is not run within Cloud Foundry. Since I’m uploading to AWS S3 it assumes I have the aws CLI and appropriate credentials. Since I’m building a Golang application, it assumes I have go CLI.

The script finishes by updating the local .downloadurl file. I need to git commit and git push the change to this file so that subsequent uses of the buildpack get the new asset.

Interact with sidecar

A sidecar’s TCP ports, or sockets, are not available outside the application container.

Therefore we can only directly interact with our sidecar process from within the application container. For debugging we can use cf ssh.

# cf ssh app-using-config-server

Once inside the container:

$ echo $CONFIG_SERVER_PORT
8082
$ curl localhost:$CONFIG_SERVER_PORT/config/
{"Scope":"some-service.admin","Password":"not-a-real-p4$$w0rd"}
$ ps axwf
    PID COMMAND
     39 ./config-server
     24 /home/vcap/deps/1/vendor_bundle/ruby/2.4.0/bin/rackup config.ru -p 8080
     ...

Next: Cached buildpacks

In the next blog post, Part 4, we will rewrite our buildpack to use the https://github.com/cloudfoundry/libbuildpack project and its buildpack-packager CLI to create and curate our buildpack. This library and CLI provides several benefits to our bespoke bin/supply and build_and_upload.sh scripts.

We will be able to create and install a system buildpack for all CF users to use, without referencing the Git repository.

applications:
- name: app-using-config-server
  buildpacks:
  - my_sidecar_buildpack
  - ruby_buildpack
...

An we will be able to create cached buildpacks. The buildpack will include all assets (our config-server binary) rather than reach out to the Internet for them. This is great for internet-starved Cloud Foundry environments, and essential for air-gapped Cloud Foundry environments.