At Stark & Wayne, we love Concourse pipelines! We use them for testing/releasing CLI utilities, deploying Cloud Foundry apps, building docker images, creating and testing BOSH releases, and vetting changes to BOSH deployments in an automated fashion starting in sandbox environments all the way to production.
Uh-oh! credentials.yml file got committed?
One of the most common challenges we’ve run into both internally and with our clients is securing credentials to ensure people don’t accidentally commit sensitive data to the our repos. Initially, we tried adding a .gitignore on credentials.yml (a file that we would reference via fly set-pipeline --load-vars-from). This was mediocre at best. It left creds on-disk for long periods of time, required people to remember the .gitignore on newly pipelined repos, and didn’t scale well when multiple people collaborated on the same project.
Our current solution addresses all of these issues. Leveraging Spruce and its (( vault )) operator, a script called repipe creates the pipeline config, updates via fly, and deletes the generated config file on completion/error so that it doesn’t live extended periods of time on-disk. Our entire ci configuration can now be committed to public repos, since there is no sensitive data stored in it. As long as collaborators of the project have access to the Vault, any one of them can make changes to the pipeline.
How to Get Started
To get started using this with your current project, take a look at our pipeline-templates repo. It has instructions for setting up your project with one of the currently provided pipeline templates. Even if your pipeline isn’t an exact match to the templates currently provided, you can still make use of the repipe script to generate and update your pipeline configs using Vault. You just need to ensure that your project follows the basic file structure described in the README (ci/repipe, ci/pipeline.yml, and ci/settings.yml).
For an example of this in the wild, feel free to browse through Spruce’s CI configurations.
Spread the word
