When the time on BOSH director and CF VMs such as cells are off, it may throw off some of your applications with unusual or unexpected errors. One cause of time drift can occur when you update the NTP servers on your BOSH director without recreating the CF VMs that are already deployed.
If you see time drift between your BOSH director and CF VMs, there are some steps you can follow to fix it. The basic idea is that first making sure the correct NTP servers are configured in the manifest for BOSH director and deploy BOSH successfully, then make sure the NTP servers successfully sync up to the CF VMs that deployed by BOSH.
More specifically, first make sure the right NTP servers are configured in the manifest for BOSH director itself and redeploy BOSH director if you made a change to the manifest. When you ssh to the BOSH director, take a look at /var/vcap/bosh/etc/ntpserver, you should see the same NTP servers listed there, check /var/vcap/bosh/settings.json, you should also find a block with the correct NTP information.
Next, bosh ssh to the CF VMs that are affected by time drift, take a look at /var/vcap/bosh/etc/ntpserver, you probably see the old or default NTP servers, also check /var/vcap/bosh/settings.json, you probably will also find a ntp block with the old or default NTP servers.
In this case, you can bosh recreate the CF VMs that have time drift. After recreation, you will see the NTP servers on the CF VMs have been updated to the same ones configured on the BOSH director and time drift issue should now be fixed. You can run bosh recreate with a --max-in-flight= flag to control how many VMs you want to update at the same time.
Spread the word
