Adding a custom hostname domain for Knative services

This is the fourth in a collection of articles as I figure out what’s what with Knative for Kubernetes. The full set of articles are:

• Deploying 12-factor apps to Knative
• Building and deploying applications to Knative
• Adding public traffic to Knative on Google Kubernetes Engine
• Adding a custom hostname domain for Knative services
• Build Docker images inside your Kubernetes with Knative Build
• Binding secrets to Knative services

In the first article we deployed an application service and used its internal routing to test. In the previous third article we setup a static IP to our ingress gateway, and set up some wildcard DNS. This allowed Knative to generate hostnames for each application service, in the functional format “appname.namespace.basedomain.whatever”. We can agree that we’re not going public with these generated hostnames. Not as many people would be addicted to a site called moneymaker.production.facebook.com.

We want pretty hostnames – starkandwayne.com – crisp and efficient.

---

In the previous article we set up a common wildcard custom domain (knative.starkandwayne.com in my example) to a long-lived static IP that was bound to the ingress gateway. But the generated route for our helloworld application — hello.helloworld.knative.starkandwayne.com wasn’t exactly how we’d like it to be for public consumption. We want hello.starkandwayne.com or maybe use a whole new domain like sayhellowithknative.com.

In this article we’ll add a custom route to our Knative application service. To do this we will break out from the high-level knctl CLI and interact with Knative/Istio using kubectl and some poeticly aesthetic YAML.

A Note on Terminology

OMG, I can be confused about when/how to use the terms hostname, route, and domain within Knative/knctl/Kubernetes/DNS. You can be confused too for now until I too have their usage clear in my mind. I’m sorry.

Generated routes

From the previous article, we have configured a custom base domain for all routes:

$ knctl domain list
Domains
Name                       Default
knative.starkandwayne.com  true

Therefore, Knative generates a route hostname for each service based on its service name and namespace within Kubernetes.

$ knctl routes list -n helloworld
Routes in namespace 'helloworld'
Name   Traffic         Ready  Domain                                      Age
hello  100% -> hello:  true   hello.helloworld.knative.starkandwayne.com  2m

All generate routes are guaranteed to be unique by Knative. All routes are guaranteed to be ugly by me.

We will set up some prettier custom routes in this article.

Internal routes

In addition to the public routes above, Knative maintains internal routes for each service.

These are recorded by /status/domainInternal in a routes CRD:

$ kubectl get routes.serving.knative.dev -n helloworld hello -o yaml
...
status:
  domain: hello.helloworld.knative.starkandwayne.com
  domainInternal: hello.helloworld.svc.cluster.local
  ...

Regardless of any changes to public routes, the domainInternal will be a consistent internal hostname.

Our solution for custom routes will be to create a Knative virtual service that rewrites requests from our pretty custom domains, into the domainInternal internal route for our service.

Custom route to internal route

Let’s configure two routes – hello.knative.starkandwayne.com and hello.starkandwayne.com.

The former will "just work" because we already have wildcard DNS set up for *.knative.starkandwayne.com.

The latter hello.starkandwayne.com will require additional DNS configuration. This is the same for any new domains like hellothere.com.

Create a file hello-entry-route.yaml:

---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hello-entry-route
  namespace: helloworld
spec:
  gateways:
  - knative-shared-gateway.knative-serving.svc.cluster.local
  hosts:
  - hello.knative.starkandwayne.com
  - hello.starkandwayne.com
  http:
  - match:
    - uri:
        prefix: "/"
    rewrite:
      authority: hello.helloworld.svc.cluster.local
    route:
    - destination:
        host: knative-ingressgateway.istio-system.svc.cluster.local
      weight: 100

Apply it to create or update:

kubectl apply -f hello-entry-route.yaml

Give it a moment and try out the already-enabled wildcard hostname:

$ curl hello.knative.starkandwayne.com
Hello World: Rev1!

Configure custom DNS

Before we can try out hello.starkandwayne.com or any other custom hostname we need to add a CNAME DNS entry to our ingress gateway:

Now that we have both the external DNS entry and the internal Knative/Istio routing, we can try out our custom route:

$ curl hello.starkandwayne.com
Hello World: Rev1!

Routing Paths To Additional Services

Our current routing will rewrite any requests to hello.knative.starkandwayne.com and hello.starkandwayne.com and forward them to the internal route hello.helloworld.svc.cluster.local. The internal service will receive the same path, headers, data, etc.

We can also split the incoming traffic to multiple application services. For example, we could route any traffic to hello.starkandwayne.com/blog or its children to a blog application.

Deploy An Additional Service

kubectl create ns blog
knctl deploy \
      --namespace blog \
      --service hello-blog \
      --image gcr.io/knative-samples/helloworld-go \
      --env TARGET="Blog Readers"

We are impatient operators who want to know, "Is it done yet?" Fortunately, we can watch Kubernetes events to see the action unfold.

$ kubectl get events -n blog --watch-only
NAMESPACE   LAST SEEN   TYPE     REASON    KIND   MESSAGE
blog        1s          Normal   Created   Pod    Created container
blog   1s    Normal   Started   Pod   Started container
blog   1s    Normal   Pulling   Pod   pulling image "gcr.io/knative-releases/github.com/knative/serving/cmd/[email protected]:65b4824c5f21134a1b8373cbc2be8d431744f06caf118a309f94b6514cd6f40c"
blog   2s    Normal   Pulled   Pod   Successfully pulled image "gcr.io/knative-releases/github.com/knative/serving/cmd/[email protected]:65b4824c5f21134a1b8373cbc2be8d431744f06caf118a309f94b6514cd6f40c"
blog   1s    Normal   Created   Pod   Created container
blog   1s    Normal   Started   Pod   Started container
blog   1s    Normal   Pulled   Pod   Container image "docker.io/istio/proxyv2:1.0.0" already present on machine
blog   1s    Normal   Created   Pod   Created container
blog   1s    Normal   Started   Pod   Started container
blog   0s    Warning   Unhealthy   Pod   Readiness probe failed: Get http://10.16.1.13:8022/health: dial tcp 10.16.1.13:8022: getsockopt: connection refused
blog   0s    Normal   RevisionReady   Revision   Revision becomes ready upon endpoint "hello-blog-00001-service" becoming ready
blog   0s    Normal   ConfigurationReady   Configuration   Configuration becomes ready
blog   0s    Normal   LatestReadyUpdate   Configuration   LatestReadyRevisionName updated to "hello-blog-00001"
blog   0s    Normal   RevisionReady   Revision   Revision becomes ready upon endpoint "hello-blog-00001-service" becoming ready
blog   0s    Normal   RevisionReady   Revision   Revision becomes ready upon endpoint "hello-blog-00001-service" becoming ready
blog   0s    Normal   Created   Route   Created VirtualService "hello-blog"
blog   1s    Normal   Updated   Route   Updated status for route "hello-blog"
blog   0s    Normal   RevisionReady   Revision   Revision becomes ready upon endpoint "hello-blog-00001-service" becoming ready
blog   0s    Normal   RevisionReady   Revision   Revision becomes ready upon endpoint "hello-blog-00001-service" becoming ready
blog   1s    Normal   RevisionReady   Revision   Revision becomes ready upon endpoint "hello-blog-00001-service" becoming ready

I’d prefer an event that said "everything is ready; go go go!", but the above events will have to do for now.

$ kubectl get events -n blog --watch-only
Running: curl '-sS' '-H' 'Host: hello-blog.blog.knative.starkandwayne.com' 'http://35.233.215.214:80'
Hello World: Blog Readers!
$ curl hello-blog.blog.knative.starkandwayne.com
Hello World: Blog Readers!

Our new application will also have an internal route hello-blog.blog.svc.cluster.local.

Splitting Incoming Traffic

We can now update our hello-entry-route route (an Istio VirtualService custom resource) to split /blog traffic to our new application service.

Update tmp/hello-entry-route.yaml and insert a new /spec/http to match a /uri/prefix: /blog:

---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hello-entry-route
  namespace: helloworld
spec:
  gateways:
  - knative-shared-gateway.knative-serving.svc.cluster.local
  hosts:
  - hello.knative.starkandwayne.com
  - hello.starkandwayne.com
  http:
  - match:
    - uri:
        prefix: /blog
    rewrite:
      authority: hello-blog.blog.svc.cluster.local
    route:
    - destination:
        host: knative-ingressgateway.istio-system.svc.cluster.local
      weight: 100
  - match:
    - uri:
        prefix: /
    rewrite:
      authority: hello.helloworld.svc.cluster.local
    route:
    - destination:
        host: knative-ingressgateway.istio-system.svc.cluster.local
      weight: 100

Apply it to update:

kubectl apply -f hello-entry-route.yaml

For a demonstration of the Istio routing in action:

$ curl hello.starkandwayne.com
Hello World: Rev1!
$ curl hello.starkandwayne.com/blog
Hello World: Blog Readers!

Summary

After four articles, we’ve come a long way. We can now deploy any source code, build using Dockerfile or Cloud Foundry buildpack (or any build template you want), from local source code or Git repository, into an autoscaled Kubernetes deployment, with a beautiful custom hostname of our own choosing.