Private buildpacks for Air-Gapped Cloud Foundry

Chad asked how to he could get FreeTDS into his Cloud Foundry application so he could use the RubyGem tiny_tds. The answer for adding system-level libraries or CLIs is to use a new supply buildpack, and so I made Chad a new buildpack https://github.com/starkandwayne/freetds-buildpack.

Chad asked a good question – “but how do I use it if our firewalls do not allow access to GitHub, and if I cannot install new system buildpacks?”

Fortunately, there is an answer in three parts:

1. Use cached buildpacks
2. Upload cached buildpacks to a private blobstore/bucket
3. Use explicit URL to the buildpack during cf push:

Use cached buildpacks

For freetds-buildpack I am publishing cached buildpacks with each GitHub release https://github.com/starkandwayne/freetds-buildpack/releases, so you only need to download the latest one, and bring it into your air gapped environment.

I also publish cached buildpacks for the new Pancake buildpack used to flatten $VCAP_SERVICES into many small environment variables.

On the other hand, the Cloud Foundry Core team’s buildpacks are not published as cached buildpacks, rather they are empty uncached buildpacks – which require all assets to be downloaded from the Internet during cf push or cf restage. For example, a Ruby buildpack release:

Never fear, you can recreate your own cached buildpacks from the buildpack project:

git clone https://github.com/cloudfoundry/ruby-buildpackcd ruby-buildpack
source .envrc
go install github.com/cloudfoundry/libbuildpack/packager/buildpack-packager
buildpack-packager build --cached --stack cflinuxfs3

This process will download many 100Mbs of dependencies to your local machine and create a large ruby-buildpack-cached-cflinuxfs3-v1.7.39.zip.

Upload cached buildpacks to a private blobstore/bucket

Let’s skip a step and see what the end result will be. To use your cached buildpack, your air gapped Cloud Foundry needs to be able to download it during push/staging.

cf push myapp \
  -b https://blobstore.internal/freetds_buildpack-cached-cflinuxfs3-v1.1.6.zip \
  -b ruby_buildpack

Unfortunately we cannot upload the buildpack during cf push; it must be available via an HTTP URI. That is, it must be available on a blobstore/bucket that is within your air gapped environment.

You will need to discover what internal blobstore/bucket you have access to within your organization, upload the file to it, and finally make the bucket or each uploaded item “public”.

Your Cloud Foundry needs to be able to access the HTTP URI without additional authentication credentials.

The blobstore is “private” with respect to the wider world, but it must be public to your Cloud Foundry.

Use explicit URL to the buildpack

I’ve already shared this twice before, but for chronological completeness, here are two ways to use use your newly uploaded air gapped buildpack.

First, as above, you can provide the URL to the cf push -b flag.

cf push myapp \
  -b https://blobstore.internal/freetds_buildpack-cached-cflinuxfs3-v1.1.6.zip \
  -b ruby_buildpack

Second, you can deploy your application with a manifest that explicitly describes your buildpacks:

---
applications:
- name: myapp
  buildpacks:
  - https://blobstore.internal/freetds_buildpack-cached-cflinuxfs3-v1.1.6.zip
  - ruby_buildpack

Learn more about Air-gapped Cloud Foundry

Our own Mike Ferris gave a great talk on air-gapped Cloud Foundry earlier this year at CF Summit in Philadelphia, together with our friend Vincent White, from Agile Defense. Please watch it to learn more about running air-gapped Cloud Foundry.