Ask PAT: how long does it take to push apps?

In the experiments above, it took on average 43.64s to push a small Ruby app to Cloud Foundry.

Introducing PAT: Performance Acceptance Tests!

PAT is actively developed by IBMers julz, simonleung8, and others and recently inducted into the Cloud Foundry Incubator.

In the graph above, I pointed PAT at Pivotal Web Services. Some other comparisons are below.

Setup

The README includes a number of ways to get the pat executable: https://github.com/cloudfoundry-incubator/pat#download-pats-binary

To confirm it is available from within the unpacked project folder:

$ ./pat --help

Next, and I don’t know why – but PAT needs gcf CLI rather than cf. The CF CLI was renamed many many months ago.

To make PAT happy, copy the CLI to gcf.

$ which cf
/usr/local/bin/cf
$ cp /usr/local/bin/cf /usr/local/bin/gcf

Finally, before you run pat (in CLI or Server mode), target and login to a Cloud Foundry:

$ cf api https://api.run.pivotal.io
$ cf login

Ask PAT

Once you have targeted a Cloud Foundry API, organization and space:

./pat -workload=gcf:push -iterations=10 -concurrency=3

When targeting Pivotal Web Services from my local laptop to Pivotal’s AWS us-east-1 deployment:

I saw similar push times – approx 40s – to TryCF. This was also pushing from my local laptop to us-east-1.

But I have a performance regression on a client’s AWS-based Cloud Foundry. It recorded average deploy times of 2 minutes. More odd was that pat was running within another AWS VM in the same region (also us-east-1).

Ask PAT for graphs

In the image at the top of this post, I was running PAT in server mode:

$ ./pat -server

Open PAT locally in your browser: http://localhost:8080

Now click the Experiment button and launch an set of cf push experiements.

Wait patiently now for the first bars to appear.

Click the bar graph tab to see bar graphs of each experiment as it completes.

The experiments are also documented at the bottom of the page. Scroll to the bottom to see the most recent result, which includes the most recent "average" of all experiments.

Cleanup

pat does not cleanup the applications and routes it creates. Run the following commands to delete them all:

cf apps | grep pats | awk '{ print $1 }' | xargs -L 1 cf delete -f
domain=$(cf domains | grep shared | awk '{print $1}')
cf routes | grep pats | awk '{print $1}' | xargs -L1 cf delete-route $domain -f -n