I ran across a nifty trick today to deploy a local DNS for Cloud Foundry. We all deserve nice things so I figured I would share.
We have a Cloud Foundry environment which is being migrated across infrastructures and need to test different scenarios keeping system and run urls the same as the old environment. We are not ready to expose the new environment yet. For now we need local DNS to resolve in the new environment.
Warning: Here be Dragons
This is a quick/dirty solution for temporary use. For a longer term solution consider using local-dns.yml opsfile for bosh2 deployments. BOSH + apt-get anything makes operators sad in the long run.
We start by adding one more job to the CF deployment manifest:
jobs: - name: dns instances: 1 networks: - name: default static_ips: - 10.120.2.12 resource_pool: medium_z1 templates: 
This will add basically an empty vm which we can use to configure DNS.
bosh ssh onto the server and install
apt-get update apt-get install dnsmasq -y
Edit the config file at
/etc/dnsmasq.conf and add the dns mappsings you would like:
# Points to our CF HAProxy address=/system.pr.starkandwayne.com/10.130.16.15 address=/run.pr.starkandwayne.com/10.130.16.15
To enable this DNS entry restart the
service dnsmasq restart
Now to get all of the vms in the Cloud Foundry deployment to use this DNS value, modify the
networks: block and add the ip address of your new DNS server and redeploy:
networks: - name: default subnets: - cloud_properties: dns: - 10.120.2.12
Now you can dig
system.pr.starkandwayne.com from any server with the new DNS entry and get the ip address of the HAProxy server in CF:
; <<>> DiG 9.10.3-P4-Ubuntu <<>> system.pr.starkandwayne.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2181 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;system.pr.starkandwayne.com. IN A ;; ANSWER SECTION: system.pr.starkandwayne.com. 0 IN A 10.130.16.15 ;; Query time: 0 msec ;; SERVER: 10.120.2.12#53(10.130.16.15) ;; WHEN: Wed Oct 25 00:37:16 UTC 2017 ;; MSG SIZE rcvd: 65
If you run
dig from a server which does not have this dns server entry you'll get the old environment (ie: the customer only sees the old environment):
; <<>> DiG 9.10.3-P4-Ubuntu <<>> system.pr.starkandwayne.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14649 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;system.pr.starkandwayne.com. IN A ;; ANSWER SECTION: system.pr.starkandwayne.com. 60 IN CNAME CF-System-7168675309.us-east-1.elb.amazonaws.com. CF-System-7168675309.us-east-1.elb.amazonaws.com. 60 IN A 220.127.116.11 CF-System-7168675309.us-east-1.elb.amazonaws.com. 60 IN A 18.104.22.168 ;; Query time: 28 msec ;; SERVER: 10.13.6.28#53(10.30.22.82) ;; WHEN: Wed Oct 25 00:42:38 UTC 2017 ;; MSG SIZE rcvd: 171
Once we are ready to expose this new environment to the world we simply remove the dns entry
10.120.2.12 from the
default network, redeploy and register public DNS to our CF HAProxy node.
I'm sure there are a dozen ways this could have been done, this one just happend to work for our use case. Alternate suggestions are welcome in the comments!