This is a hands on introduction to habitat.sh. It is a relatively new tool in the world of config management and aims to simplify many aspects of packaging / deploying and operating any kind of distributed systems regardless of the intended deploy target.
Some of the features that I have found particularly useful are:
- Simple / sane and reproducible approach to packaging.
- Supervision and handling lifecycles of processes.
- Service discovery including propagation of configuration to consuming services.
In this brief tour we will see some basic commands needed to build and run services packaged with habitat, without going into a lot of detail.
Setup
Lets start by cloning Stark & Wayne’s habitat-plans repo.
$ git clone https://github.com/starkandwayne/habitat-plans.git
$ cd habitat-plans
$ ls
LICENSE README.md ci mysql postgresql redis scripts shield shield-agent wordpress wordpress-proxy
Here we can see the list of software that S&W engineers are working on to create ‘Enterprise Ready’ habitat plans.
We will use the hab cli to build and run these services so now would be a good time to install habitat.
$ hab --version
hab 0.21.0/20170421000356
Since we are building the plans under the starkandwayne origin we will need to generate an origin key for starkandwayne. Every package in Habitat belongs to an origin, and is cryptographically signed with that origin’s private key.
$ hab origin key generate starkandwayne
» Generating origin key for starkandwayne
★ Generated origin key pair starkandwayne-20170424121434.
Since I will be demonstrating the workflow in a Mac OS environment you will also need Docker for Mac installed.
Building a package
Once your environment is set up you can enter the habitat studio (a shell environment setup to quickly iterate on your habitat plans). On a Mac this will download and start a docker container. On Linux this will simply chroot
$ HAB_ORIGIN=starkandwayne hab studio enter
(...)
[0][default:/src:0]#
A habitat package is defined via a plan.sh file. Lets take a look at the one for postgresql.
[0][default:/src:0]# ls postgresql
config default.toml hooks plan.sh tests
[1][default:/src:0]# less postgresql/plan.sh
(...)
You will see a bunch of variables:
pkg_name=postgresql
pkg_version=9.6.1
pkg_origin=starkandwayne
pkg_description="PostgreSQL is a powerful, open source object-relational database system."
pkg_upstream_url="https://www.postgresql.org/"
pkg_license=('PostgreSQL')
pkg_source=https://ftp.postgresql.org/pub/source/v${pkg_version}/${pkg_name}-${pkg_version}.tar.bz2
(...)
And further down some hooks that start with do_*
do_build() {
# ld manpage: "If -rpath is not used when linking an ELF
# executable, the contents of the environment variable LD_RUN_PATH
# will be used if it is defined"
./configure --disable-rpath \
--with-openssl \
--prefix="$pkg_prefix" \
--with-uuid=ossp \
--with-includes="$LD_INCLUDE_PATH" \
--with-libraries="$LD_LIBRARY_PATH" \
--sysconfdir="$pkg_svc_config_path" \
--localstatedir="$pkg_svc_var_path"
make world
}
do_install() {
make install-world
}
These functions and variables instruct habitat on how to create the starkandwayne/postgresql package. Let’s build it now:
[2][default:/src:0]# build postgresql
This will take a while. The Postgresql source is downloaded and compiled according to the plan.sh. Eventually, you will see in the output that the build has completed:
(...)
postgresql: hab-plan-build cleanup
postgresql:
postgresql: Source Path: /hab/cache/src/postgresql-9.6.1
postgresql: Installed Path: /hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553
postgresql: Artifact: /src/results/starkandwayne-postgresql-9.6.1-20170424121553-x86_64-linux.hart
postgresql: Build Report: /src/results/last_build.env
postgresql: SHA256 Checksum: 6187c711bd1f4471722053754b068f8f6de048612ca2b9a00aaec08f2ce6eb17
postgresql: Blake2b Checksum: ae297ffaa007478eea6fba9676c2170e0af6dbb025dfdb61cf9512178a636fb2
postgresql:
postgresql: I love it when a plan.sh comes together.
postgresql:
postgresql: Build time: 5m23s
The resulting *.hart file (which is the resulting artifact) will reside under the results/ directory. This directory will also be present on your host machine because the directory you were in got mounted under /src when you typed hab studio enter.
[3][default:/src:0]# ls results
last_build.env starkandwayne-postgresql-9.6.1-20170424121553-x86_64-linux.hart
Habitat has also already installed your built plan into the studio under /hab/pkgs/...
[4][default:/src:0]# find /hab/pkgs/starkandwayne -type d -maxdepth 4
/hab/pkgs/starkandwayne
/hab/pkgs/starkandwayne/postgresql
/hab/pkgs/starkandwayne/postgresql/9.6.1
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/share
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/lib
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/config
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/bin
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/include
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/hooks
Running your built service
To start your built package in the habitat studio you can simply run:
[5][default:/src:0]# hab start starkandwayne/postgresql
hab-sup(MR): Butterfly Member ID 67642576ef154e76b8160c83c605b475
hab-sup(SR): Adding starkandwayne/postgresql/9.6.1/20170424122104
hab-sup(MR): Starting butterfly on 0.0.0.0:9638
hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
postgresql.default(SR): Initializing
postgresql.default(SV): Starting process as user=hab, group=hab
postgresql.default(O): Starting PostgreSQL
postgresql.default(O): 2017-04-24 12:45:35.587 GMT: FATAL: could not open shared memory segment "/PostgreSQL.1733059837": Permission denied
postgresql.default(O): 2017-04-24 12:45:35.597 GMT: LOG: database system is shut down
hab-sup(SV): postgresql.default - process 15914 died with exit code 1
hab-sup(SV): postgresql.default - Service exited
Due to the constraints of the studio context this package actually won’t start the way we want it (Permission denied). But that doesn’t really matter! Since I am introducing you to a possible mac workflow you will need to export your packages to docker anyway.
[6][default:/src:0]# hab pkg export docker starkandwayne/postgresql
(...)
Once we have exported the package we can exit the studio and run it in docker on our host machine:
[7][default:/src:0]# exit
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
starkandwayne/postgresql 9.6.1-20170424122104 938e89b8286b About a minute ago 429 MB
starkandwayne/postgresql latest 938e89b8286b About a minute ago 429 MB
$
$ docker run -p 5432:5432 starkandwayne/postgresql
hab-sup(MR): Butterfly Member ID eb68d2c1489a4c8084ac824311702750
hab-sup(SR): Adding starkandwayne/postgresql/9.6.1/20170424122104
hab-sup(MR): Starting butterfly on 0.0.0.0:9638
hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
hab-sup(SC): Updated pwfile 30087c89f3dfc2b2179a2b567a67f094cf1cd8b1c6e3cc1473aef95fe65de533
hab-sup(SC): Updated postgresql.conf 3ad4f841f942d1ae606ad89a54cb0ebe8a4313a1af4f1a8958c50b3aedaeb6ef
hab-sup(SC): Updated pg_hba.conf cc1489fff1c05e7b547fac8018dbe22d0327a8ea6a288a6604a1b6c8232619e1
postgresql.default(SR): Initializing
(...)
You can test connecting to postgres from another terminal:
$ cat postgresql/default.toml | egrep '(name|password)'
name = 'admin'
password = 'admin'
$ PGPASSWORD=admin psql -h localhost -U admin postgres
psql (9.6.2, server 9.6.1)
Type "help" for help.
postgres=# \q
Conclusion
This was a very quick introduction of how to build, export and run a PostgreSQL service packaged via habitat.
To learn more about habitat please go through the tutorial on the habitat.sh website. It does a good job of explaining the basic concepts while walking you through the creation of your own plan.
The docs are also an excellent point of reference.
Also since Stark & Wayne have begun contributing plans to the habitat community you can keep following our blog for more habitat related content including workflow tips / best practices and other topics.
Spread the word
