Jun 08, 2016 Using Genesis to Deploy Cloud Foundry
In this post, we're going to use Genesis to deploy Cloud Foundry. We will make use of some of Genesis's cool features to generate unique credentials for each deployment, and Vault to keep the credentials out of the saved manifests. We will do this on BOSH-Lite, but templates exist to easily deploy to AWS with a nearly identical process (AWS will require a couple more parameters to be defined).
These instructions assume that you have a working installation of BOSH-Lite, and Vault. It also assumes you have set the VAULT_ADDR environment variable, and successfully authenticated to an unsealed Vault. The vault-boshrelease has some good instructions on getting this up and running quickly. Additionally, the Spruce, Safe, and certstrap command line utilities are required.
First, we will use Genesis to create a new repo for managing our Cloud Foundry deployments:
genesis new deployment --template cf
This will create a git repository in the current working directory called
cf-deployments. This repo will be what you use to manage all of your Cloud Foundry deployments, using the Genesis templates. An
upstream remote is created automatically to track changes to the original templates, allowing you to easily pull in updates that are made to it over time.
Secondly, we will create a new site for BOSH-Lite:
$ cd cf-deployments $ genesis new site --template bosh-lite macbook
This creates a new site directory in the
cf-deployments repo, called
macbook, based on the
bosh-lite site templates. As of the time this post was written, both
aws site templates were provided, but more may now exist. If not, you can create your own, and hopefully submitting an upstream pull request, so others can make use of your work.
Third, create a new environment in our
genesis new environment macbook sandbox
This creates a
sandbox directory representing the environment inside the
cf-deployments/macbook site directory. It also does a couple nifty things under the hood that are not immediately apparent:
- It runs any scripts in
cf-deployments/.env_hooks. In this case, there are scripts which will connect to your Vault installation and generate unique secrets/certificates/keys for your Cloud Foundry deployment. Each time you create a new environment, these credentials will be unique, decreasing the risks of environments accidentally communicating with one-another.
- It will generate a logical name for your bosh deployment, based on the type of deployment being done (cf), site (macbook), and environment (sandbox):
- It grabs the UUID of the currently targeted BOSH director, and inserts that to
director.yml, so you don't have to.
Next, we will attempt to deploy, to see if there are any parameters need to be filled out to get our environment deployed:
$ cd macbook/sandbox $ make deploy Refreshing site definitions for macbook/sandbox Refreshing global definitions for macbook/sandbox 4 error(s) detected: - $.networks.cf.subnets: Specify the subnet to use for CF - $.properties.cc.security_group_definitions.load_balancer.rules: Specify the rules for allowing access for CF apps to talk to the CF Load Balancer External IPs - $.properties.cc.security_group_definitions.services.rules: Specify the rules for allowing access to CF services subnets - $.properties.cc.security_group_definitions.user_bosh_deployments.rules: Specify the rules for additional BOSH user services that apps will need to talk to Failed to merge templates; bailing... make: *** [manifest] Error 5
It looks like we need to provide some configuration for the network setup this Cloud Foundry will run on. Let's use this, to match what the traditional Cloud Foundry on BOSH-Lite will look like:
$ cat <<EOF > networking.yml networks: - name: cf subnets: - gateway: 10.244.0.1 range: 10.244.0.0/24 static: - 10.244.0.2 - 10.244.0.100 properties: cc: security_group_definitions: - name: services rules: - destination: 10.244.0.0 - 10.244.1.255 protocol: all - name: load_balancer rules: - destination: 10.244.0.34 protocol: all - name: user_bosh_deployments rules: - destination: 10.244.0.0 - 10.244.255.255 protocol: all EOF
Now that that's taken care of, let's try again:
Refreshing site definitions for macbook/sandbox Refreshing global definitions for macbook/sandbox <output omitted for brevity - it's deploying a full CF to bosh-lite, after all>
Since there were no errors generating the manifest, Genesis moves on to deploying Cloud Foundry. Now take a look at the generated manifest in
cf-deployments/macbook/sandbox/manifests/manifest.yml, and look for the certs and passwords. You will see that they're all
REDACTED. Genesis tells Spruce to redact credentials from Vault except in the manifest that is actively being deployed via BOSH (which is cleaned up immediately after). This greatly minimizes the risk that sensitive credentials might be committed into your repo, and accidentally leaked to unauthorized eyes.
To find the credentials that were generated for the deployment, you can use
safe tree secret/macbook/sandbox/cf to see where the creds are stored and
safe get <path> to retrieve them for use with
cf login or
Finally, shout out to
@starkandwayne with the
#genesis hashtag to let us know you love genesis!
The Long Term Scenario
When living with this in a real world scenario, you will inevitably need to make modifications and upgrades over time and across a variety of environments. Genesis makes this really easy to manage, since the global and site templates are shared across all your deployments. Many changes you make will go from
write-once-per-deployment-hoping-you-applied-all-the-changes-correctly, deploy-many to
The workflow goes something like this:
- Edit the applicable files in
make refresh deployin the testing environment
- Validate that all went well
- Lather, rinse repeat steps 2-3 as needed for the remaining sites to be updated. If you changed environment-level YAML files in step 1, those will also need to be replicated in each environment).
- Marvel at how much time you're saving in both updating YAML, fretting over whether you added all the properties correctly for each environment, and fixing/redeploying from any mistakes made along the way.