This is the second in a collection of articles as I figure out what’s what with Knative for Kubernetes. The full set of articles are:
- Deploying 12-factor apps to Knative
- Building and deploying applications to Knative
- Adding public traffic to Knative on Google Kubernetes Engine
- Adding a custom hostname domain for Knative services
- Build Docker images inside your Kubernetes with Knative Build
- Binding secrets to Knative services
Knative has three high level subsystems: Serving to coordinate autoscaling of a service and its pods, and its routing; Build to provide a toolchain to convert bespoke application code into an intermediate container image; and Eventing to encourage loosely coupled application services through an abstraction of pub/sub events.
In the first article we got started with Knative Serving to deploy an application already built into a container image.
In this article we use allow Knative Build to convert our application into a container image during the deployment process. We’ll use both a Dockerfile and a Cloud Foundry buildpack. We’ll also look at deploying from our local filesystem as well as from a remote Git repository.
It has been simple so far. It turns out that installing Knative into any Kubernetes environment, running a pre-built container image, and interacting with it via curl was very simple. Abbreviated from the previous article:
knctl install [--node-ports] [--exclude-monitoring]
knctl deploy --service <service-name> --image <image-name>
knctl curl --service <service-name>
The latter two steps – deploying an image into a running Kubernetes pod, versioned across each deployment, and interacting with it via HTTP routing – was a demonstration of the Knative Serving subsystem.
Knative also supports a flexible subsystem for building container images, which will then be run as a Kubernetes pod.
The Knative Build subsystem is generously flexible. I’ll look at some specific use cases:
- upload a local directory and build from an explicit Dockerfile
- upload a local directory and build using a Buildpack
- source comes from a remote Git repository, building using either of the two build sequences above
Namespaces
In the first article I explicitly included --namespace helloworld on every knctl command. The explicitness appeals to me. For others, you might want to set a default namespace and keep your commands shorter.
You can also configure knctl to have a "current namespace".
kubectl create ns my-simple-app
export KNCTL_NAMESPACE=my-simple-app
All knctl commands will assume this namespace.
$ knctl service list
Services in namespace 'my-simple-app'
Name Domain Annotations Age
0 services
We can reuse $KNCTL_NAMESPACE in our low-level kubectl commands:
kubectl get pods -n $KNCTL_NAMESPACE
Upload a local directory with a Dockerfile
In all Knative Build examples the byproduct of a Build sequence is a container image (the secret code word for “Docker image”). These images have to live somewhere, such as Docker Hub, GCP Container Registry, Azure Container Registry, or an on-premise/DIY registry like Harbor.
We need to configure Knative with our container registry location and secrets using knctl basic-auth-secret create within each applicable Kubernetes namespace.
For Docker Hub, use --docker-hub flag:
knctl basic-auth-secret create -s registry --docker-hub -u <username> -p <password>
For GCP Container Registry, use --gcr flag:
knctl basic-auth-secret create -s registry --gcr -u <username> -p <password>
For any other container registry that doesn’t have a convenience flag, use the flags --type and --url:
knctl basic-auth-secret create -s registry --type docker --url https://registry.domain.com/ -u <username> -p <password>
Next, map the container registry secrets into a Kubernetes service account, which will provide the information above into the pods used by Knative Build.
knctl service-account create --service-account build -s registry
This maps down to a Kuberneters service account:
$ kubectl get serviceaccount -n $KNCTL_NAMESPACE
NAME SECRETS AGE
build 2 37s
default 1 3h
Now we are ready to authorize Knative Build subsystem to create new container images.
Clone a sample Go application and deploy it from its local directory to a Docker Hub image name:
git clone https://github.com/cppforlife/simple-app
cd simple-app
DOCKER_IMAGE=index.docker.io/<your hub.docker.com org or user>/knative-simple-app
knctl deploy \
--service simple-app \
--directory=$PWD \
--service-account build \
--image ${DOCKER_IMAGE:?required} \
--env SIMPLE_MSG="Built from local directory using Dockerfile"
Thought: the explicitly named container image is an intermediate byproduct of the Build -> Serve sequence and I really don’t have strong opinions of its name; but am required to explicitly provide it.
The output of the knctl deploy in its entirity might look similar to:
Name simple-app
Waiting for new revision to be created...
Tagging new revision 'simple-app-00001' as 'latest'
Tagging new revision 'simple-app-00001' as 'previous'
[2018-10-15T13:18:31+10:00] Uploading source code...
[2018-10-15T13:19:59+10:00] Finished uploading source code...
Watching build logs...
build-step-build-and-push | INFO[0000] Downloading base image golang:1.10.1
build-step-build-and-push | ERROR: logging before flag.Parse: E1015 03:20:01.547607 1 metadata.go:142] while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
build-step-build-and-push | ERROR: logging before flag.Parse: E1015 03:20:01.550268 1 metadata.go:159] while reading 'google-dockercfg-url' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg-url
build-step-build-and-push | INFO[0001] Executing 0 build triggers
build-step-build-and-push | INFO[0001] Extracting layer 0
build-step-build-and-push | INFO[0003] Extracting layer 1
build-step-build-and-push | INFO[0004] Extracting layer 2
build-step-build-and-push | INFO[0004] Extracting layer 3
build-step-build-and-push | INFO[0007] Extracting layer 4
build-step-build-and-push | INFO[0010] Extracting layer 5
build-step-build-and-push | INFO[0015] Extracting layer 6
build-step-build-and-push | INFO[0015] Taking snapshot of full filesystem...
build-step-build-and-push | INFO[0027] WORKDIR /go/src/github.com/mchmarny/simple-app/
build-step-build-and-push | INFO[0027] cmd: workdir
build-step-build-and-push | INFO[0027] Changed working directory to /go/src/github.com/mchmarny/simple-app/
build-step-build-and-push | INFO[0027] Creating directory /go/src/github.com/mchmarny/simple-app/
build-step-build-and-push | INFO[0027] COPY . .
build-step-build-and-push | INFO[0027] RUN CGO_ENABLED=0 GOOS=linux go build -v -o app
build-step-build-and-push | INFO[0027] cmd: /bin/sh
build-step-build-and-push | INFO[0027] args: [-c CGO_ENABLED=0 GOOS=linux go build -v -o app]
build-step-build-and-push | net
build-step-build-and-push | vendor/golang_org/x/net/lex/httplex
build-step-build-and-push | vendor/golang_org/x/net/proxy
build-step-build-and-push | net/textproto
build-step-build-and-push | crypto/x509
build-step-build-and-push | crypto/tls
build-step-build-and-push | net/http/httptrace
build-step-build-and-push | net/http
build-step-build-and-push | github.com/mchmarny/simple-app
build-step-build-and-push | INFO[0030] Taking snapshot of full filesystem...
build-step-build-and-push | INFO[0034] Storing source image from stage 0 at path /kaniko/stages/0
build-step-build-and-push | INFO[0038] trying to extract to /kaniko/0
build-step-build-and-push | INFO[0038] Extracting layer 0
build-step-build-and-push | INFO[0040] Extracting layer 1
build-step-build-and-push | INFO[0041] Extracting layer 2
build-step-build-and-push | INFO[0041] Extracting layer 3
build-step-build-and-push | INFO[0043] Extracting layer 4
build-step-build-and-push | INFO[0046] Extracting layer 5
build-step-build-and-push | INFO[0051] Extracting layer 6
build-step-build-and-push | INFO[0051] Extracting layer 7
build-step-build-and-push | INFO[0051] Deleting filesystem...
build-step-build-and-push | INFO[0053] No base image, nothing to extract
build-step-build-and-push | INFO[0053] Taking snapshot of full filesystem...
build-step-build-and-push | INFO[0062] COPY --from=0 /go/src/github.com/mchmarny/simple-app/app .
build-step-build-and-push | INFO[0063] Taking snapshot of files...
build-step-build-and-push | INFO[0063] EXPOSE 8080
build-step-build-and-push | INFO[0063] cmd: EXPOSE
build-step-build-and-push | INFO[0063] Adding exposed port: 8080/tcp
build-step-build-and-push | INFO[0063] ENTRYPOINT ["/app"]
build-step-build-and-push | ERROR: logging before flag.Parse: E1015 03:21:04.751338 1 metadata.go:142] while reading 'google-dockercfg' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg
build-step-build-and-push | ERROR: logging before flag.Parse: E1015 03:21:04.753927 1 metadata.go:159] while reading 'google-dockercfg-url' metadata: http status code: 404 while fetching url http://metadata.google.internal./computeMetadata/v1/instance/attributes/google-dockercfg-url
build-step-build-and-push | 2018/10/15 03:21:06 pushed blob sha256:72a682eea3309941d5e8e6f993a07ae4d33a413b8b7fa2762f8e969310b5996a
build-step-build-and-push | 2018/10/15 03:21:07 pushed blob sha256:9c24aa788ba416c5e1e631d8af3e3115519ad7ca0f659ac10f40682524c6d9cd
build-step-build-and-push | 2018/10/15 03:21:07 index.docker.io/drnic/knative-simple-app:latest: digest: sha256:b5823ead77d9544998b5bc844f049d1a7dfb0aefe7461b74b3e4f67fb5481fa1 size: 428
nop | Nothing to push
Succeeded
Debugging Knative Build
Currently knctl deploy does not show any internal errors or warnings from Knative Build subsystem. You can find yourself just sitting there watching Waiting for new revision to be created... and nothing more.
One option for debugging is to use kail to stream the logs from the Knative Build subsystem:
kail -n knative-build
Then you need to stare deep into the mess of logs and look for errors, such as: "msg":"Failed the resource specific validation{error 25 0 serviceaccounts \"build\" not found}"
Build using Buildpacks
The Cloud Foundry and Heroku approaches to building container images is personally very satisfying, and fortunately for us all it is supported by Knative Build using a custom build template.
First, register the build template with the name “buildpack” into your active namespace:
kubectl -n $KNCTL_NAMESPACE apply -f https://raw.githubusercontent.com/knative/build-templates/master/buildpack/buildpack.yaml
To use the custom build template, add the --template buildpack flag. Any additional environment variables used by the build template (or the buildpack sequence in this case) can be passed with --template-env NAME=value.
For example, the Cloud Foundry Go Buildpack requires $GOPACKNAME (see docs):
knctl deploy \
--service simple-app \
--directory=$PWD \
--service-account build \
--image ${DOCKER_IMAGE:?required} \
--env SIMPLE_MSG="Built from local directory using Buildpack template" \
--template buildpack \
--template-env GOPACKAGENAME=main
The output shows the same output you’d see from a Cloud Foundry buildpack:
Name simple-app
Waiting for new revision (after revision 'simple-app-00001') to be created...
Tagging new revision 'simple-app-00002' as 'latest'
Tagging older revision 'simple-app-00001' as 'previous'
[2018-10-15T13:40:41+10:00] Uploading source code...
[2018-10-15T13:42:08+10:00] Finished uploading source code...
Watching build logs...
build-step-build | -----> Go Buildpack version 1.8.26
build-step-build | -----> Installing godep 80
build-step-build | Download [https://buildpacks.cloudfoundry.org/dependencies/godep/godep-v80-linux-x64-cflinuxfs2-06cdb761.tgz]
build-step-build | -----> Installing glide 0.13.1
build-step-build | Download [https://buildpacks.cloudfoundry.org/dependencies/glide/glide-v0.13.1-linux-x64-cflinuxfs2-aab48c6b.tgz]
build-step-build | -----> Installing dep 0.5.0
build-step-build | Download [https://buildpacks.cloudfoundry.org/dependencies/dep/dep-v0.5.0-linux-x64-cflinuxfs2-52c14116.tgz]
build-step-build | -----> Installing go 1.8.7
build-step-build | Download [https://buildpacks.cloudfoundry.org/dependencies/go/go1.8.7.linux-amd64-cflinuxfs2-fff10274.tar.gz]
build-step-build | **WARNING** Installing package '.' (default)
build-step-build | -----> Running: go install -tags cloudfoundry -buildmode pie .
build-step-export | 2018/10/15 03:47:58 mounted blob: sha256:1124eb40dd68654b8ca8f5d9ec7e439988a4be752a58c8f4e06d60ab1589abdb
build-step-export | 2018/10/15 03:47:58 mounted blob: sha256:6be38da025345ffb57d1ddfcdc5a2bc052be5b9491825f648b49913d51e41acb
build-step-export | 2018/10/15 03:47:58 mounted blob: sha256:a5733e6358eec8957e81b1eb93d48ef94d649d65c69a6b1ac49f616a34a74ac1
build-step-export | 2018/10/15 03:47:58 mounted blob: sha256:21324a9f04e76c93078f3a782e3198d2dded46e4ec77958ddd64f701aecb69c0
build-step-export | 2018/10/15 03:47:59 pushed blob sha256:efa2d34b82bc07588a1a8fd4526322257408109547ee089a792b3f51c383f8e6
build-step-export | 2018/10/15 03:47:59 pushed blob sha256:d495696b33936c79216ec8178726b9fbe915fafbffdd0911a7fdabce4297d9a4
build-step-export | 2018/10/15 03:48:00 index.docker.io/drnic/knative-simple-app:latest: digest: sha256:e5ef1d4d255b4bcbb38d4b43bb6302423c33e6eeabd0e20d5fda4e5ce4c46668 size: 1082
nop | Nothing to push
We can now see that the application has been redeployed:
$ knctl curl -s simple-app
<h1>Built from local directory using Buildpack template</h1>
Private Git Secret
In the two sections above we uploaded the source code for our application from our local machine, and then built an intermediate Docker image (either from a Dockerfile or a Cloud Foundry buildpack) before running the application.
Knative can also fetch the source code from a git repository. (Technically speaking, Knative Build currently only supports fetching source code from a git repository. The "upload a local directory" feature is implemented only in Dmitriy’s knctl CLI.)
To ask Knative Build to fetch a git repository, replace the flag --directory=$PWD with flags --git-url and --git-revision.
If your git repository is private, then you also need to include git ssh credentials in the serviceaccount (named build in our example above). The knctl ssh-auth-secret create is a helper to create a kubernetes.io/ssh-auth secret.
$ knctl ssh-auth-secret create --secret git --github --private-key "$(cat ~/.ssh/id_rsa)"
Name git
Type kubernetes.io/ssh-auth
$ kubectl get secrets -n $KNCTL_NAMESPACE
NAME TYPE DATA AGE
...
git kubernetes.io/ssh-auth 1 5m
registry kubernetes.io/basic-auth 2 3h
We now need to add git secret to our build service account.
At the time of writing, knctl did not provide a knctl serviceaccounts update command or similar. Instead we can delete and recreate:
kubectl delete serviceaccounts -n $KNCTL_NAMESPACE build
knctl service-account create --service-account build -s registry -s git
Deploy from Git
To deploy from Git we remove --directory and add --git-url and --git-revision:
knctl deploy \
--service simple-app \
--git-url [email protected]:cppforlife/simple-app.git \
--git-revision master \
--service-account build \
--image ${DOCKER_IMAGE:?required} \
--env SIMPLE_MSG="Built from Git repo using Buildpack template" \
--template buildpack \
--template-env GOPACKAGENAME=main
Summary
The knctl deploy command provides a nice experience atop of Knative to create new container images prior to deploying them. It allows building from local source directories or from a git repository, Dockerfiles or Cloud Foundry buildpacks, and support for different intermediate Docker registries.
Spread the word
