Is using BUCC a better way to bootstrap BOSH?

BUCC is the convention over configuration tool to make it easy to deploy the best, securest, most backupable BOSH/Concourse on a single VM.

The Problem

The bosh-deployment is the fantastic tool for deploying a BOSH VM to any infrastructure. A vanilla BOSH or one with UAA/CredHub. A BOSH that can deploy normal infrastructure VMs or a BOSH that deploys to local Garden containers (aka BOSH Lite).

Unfortunately, the flexibility means that using bosh create-env involves passing a long set of files and flags and it means knowing which set of flags and files to use.

bosh create-env bosh-deployment/bosh.yml \
    -o bosh-deployment/aws/cpi.yml \
    --state=state.json \
    --vars-store=creds.yml \
    -v director_name=bosh-1 \
    -v internal_cidr=10.0.0.0/24 \
    -v internal_gw=10.0.0.1 \
    -v internal_ip=10.0.0.6 \
    -v access_key_id=AKI... \
    -v secret_access_key=wfh28... \
    -v region=us-east-1 \
    -v az=us-east-1a \
    -v default_key_name=bosh \
    -v default_security_groups=[bosh] \
    --var-file private_key=~/Downloads/bosh.pem \
    -v subnet_id=subnet-ait8g34t

After the first run, two files state.json and creds.yml are created, and they are then reused on subsequent updates/re-runs.

• state.json containing references to the IaaS assets, notably the persistent disk where your data is stored
• creds.yml containing generates secrets and private keys

You’ll need to keep hold of those files for all future changes, upgrades, and resizing of your BOSH VM. One suggestion is a private Git repo that allows you to keep an audit log of: when, who, and why a change was made to the BOSH VM. You’ll probably also keep a submodule of https://github.com/cloudfoundry/bosh-deployment to track the specific versions of the base bosh.yml and various operator files you’re using.

Over times you’ll want to add more subsystems to your BOSH VM. In the example above, we have a bare bones BOSH with bosh create-env bosh-deployment/bosh.yml -o -o bosh-deployment/aws/cpi.yml.

If you want to SSH into the BOSH VM, or want use it as a gateway for SSH connections to your deployment VMs, then you’ll want to add the jumpbox-user.yml operator file.

If you want the magic of secrets with CredHub then you’ll need to add uaa.yml and credhub.yml.

You need to remember to use all the same flags and files every time you do every update. You and your team won’t just remember. It’s not possible. So instead you’ll want to write a wrapper shell script. You’ll store that shell script next to your state.json since they go together.

If you have any bespoke operator files, you’ll store them there too. For example, if you want to save 85% on your AWS bill, try out Spot Instances with this operator file:

- type: replace
  path: /resource_pools/name=vms/cloud_properties/spot_bid_price?
  value: ((spot_bid_price))

Once you’re BOSH/UAA/CredHub is running, you will then need to extract the generated credentials and private keys from creds.yml and login to each subsystem. The documentation proposes the following commands:

# Configure local alias
$ bosh alias-env bosh-1 -e 10.0.0.6 --ca-cert <(bosh int ./creds.yml --path /director_ssl/ca)
# Log in to the Director
$ export BOSH_CLIENT=admin
$ export BOSH_CLIENT_SECRET=`bosh int ./creds.yml --path /admin_password`
# Query the Director for more info
$ bosh -e bosh-1 env

To target the UAA and CredHub you will need to install their CLIs, to discover the secrets from creds.yml, target, and login. You’ll end up writing wrapper shell scripts to make this simpler.

The Solution with BUCC

BUCC simplifies all these decisions, throws in Concourse so you can start using pipelines immediately, includes BOSH Backup & Restore (BBR), and provides you a single bucc CLI with all the functionality you need to deploy/upgrade and to authenticate with BOSH/UAA/CredHub/Concourse/BBR.

Easy Deployment

The first time you run BUCC up it will create the initial folder structure where state & creds will be stored. It will create a documented vars.yml where you provide your IaaS/networking/VM/disk information.

bucc up is the wrapper script you wish you’d created. It composes the bosh create-env command for your CPI.

BUCC also solves tricky questions for you:

• How do I include Concourse in my BOSH VM so I can immediately start using pipelines to deploy my subsystems?
• How do I integrate BBR so I can immediately start backing up my BOSH VM and all BOSH deployments?

Easy Authentication

Once your BUCC VM is running you will want to target and authenticate with your BOSH, CredHub, or Concourse.

BUCC has simple subcommands to do this for you.

Without BUCC you will write shell scripts to extract private keys and store them in local files, to extract secrets, and to target the CLIs and authenticate for you.

With BUCC it’s very simple.

To target your BOSH CLI:

source <(bucc env)
bosh env

You now have a normal, empty BOSH director. You will now upload a stemcell, update your cloud config, and start deploying subsystems like Cloud Foundry or Kubernetes.

To target your CredHub CLI:

bucc credhub
credhub find

To target your Concourse fly CLI:

bucc fly
fly -t bucc pipelines
fly -t bucc workers

BUCC will even download the latest CLIs so they are ready to use.

Easy Backup and Restore

Finally, you will want to use BBR to backup and restore your BOSH and all its deployments.

This is built in to BUCC from day one:

cd path/to/store/backups
bucc bbr backup

Summary

I initially migrated my BOSH environments to BUCC because I was tired of curating my own bosh create-env command with all the right flags and files. I stayed for the authentication subcommands and backups.