One of the most requested features for Stark & Wayne kStreams (powered by Apache Kafka) was, "How do we share our Kafka topics between different Cloud Foundry spaces?" Great question, but it wasn’t in the domain of a service broker to break the fundamental isolation tenants of Cloud Foundry. Fortunately, the Cloud Foundry Services API team, with the CAPI and CLI teams, has just shipped this very feature!
UPDATE: I mistakenly attributed this work solely to the CAPI & CLI teams initially. Thanks to Denise Yu for correcting me.
Our Pivotal Cloud Foundry and open source Cloud Foundry customers will now be asking, "Can we now share service instances?"
The answer is, "yes, but…"
Open Source Cloud Foundry
Yes, this feature is available in open source Cloud Foundry. As an admin you will need to upgrade your Cloud Foundry and your developer/users will need to upgrade to the latest CLI to v6.35. As an admin you will need to enable the beta feature:
cf enable-feature-flag service_instance_sharing
But, none of your existing service offerings will support sharing. At the time of writing, the current implementation of shareable service instances requires every service broker to explicitly offer sharing, even though they don’t have to implement any new behavior. The reasoning is explained by Denise Yu on the cf-dev mailing list:
"The decision to have service authors opt in was to account for the fact that some services may not be shareable out-of-the-box, primarily due to security considerations. Some brokers may currently be designed to only issue global read+write permissions, but authors may want to change their service permissions model if shareability is now on the cards, for example, read+write for SpaceDevs in the original space, but read-only for spaces that received the instance via sharing."
If you wrote your own service brokers, you need to tweak their /v2/catalog output to include metadata: {shareable: true} in each service offering.
If you’re using third party service brokers, you’ll need to request that they read https://docs.cloudfoundry.org/services/enable-sharing.html and make the same change.
An example change to /v2/catalog can be found at https://github.com/cloudfoundry-community/kafka-service-broker-boshrelease/blob/d57b6375ce1547965cdc7d8008a06047b77018d1/jobs/kafka-service-broker/templates/config/catalog.json#L18
Pivotal Cloud Foundry
The joy for PCF customers who are aware of new innovations in open source Cloud Foundry is awareness of new features coming down the pipes. But it also means those features aren’t simultaneously available to you. Fingers crossed that "Shareable Service Instances" is included in PCF PAS 2.1 coming later in March/April (PAS is the new name for the old Elastic Runtime/ERT product).
But, even if it is included in PCF PAS 2.1, you will need your tiles to have the metadata: {shareable: true} attribute set. Your third-party tile providers might not be aware of this requirement, so I highly recommend you start now to ask them to include it in their next tile version.
Stark & Wayne kStreams
We have now upgraded the upstream Kafka Service Broker BOSH release that feeds into the Stark & Wayne kStreams tile for PCF. The next version of the tile will support Shareable Service Instances in preparation for future PCF PAS support.
https://github.com/cloudfoundry-community/kafka-service-broker-boshrelease/releases/tag/v2.2.0
If you would like help adding Apache Kafka to your Cloud Foundry, or want Stark & Wayne expertise to upgrade and operate your Cloud Foundry or Pivotal Cloud Foundry installations, please contact us https://starkandwayne.com or on twitter @starkandwayne.
Spread the word
