Using Puppet as a node source for Rundeck

We’ve recently been asked to explore introducing Rundeck into an organization that heavily invested in Puppet for their IT automation.

What the heck is Rundeck?

You can follow the link above for more details, but basically Rundeck is an API and user interface for performing operations tasks. It provides access controls, job scheduling and the ability to create complex workflows across multiple nodes in a single run.

We are going to assume that because you are reading this blog, you probably know what Puppet is already.

Let’s get started!

In our initial research we found several great projects, including a Rundeck Puppet module here. There is also a small handful of blogs and presentations about getting the two platforms to play nice. These were great for getting the base Puppet manifest together and getting Rundeck to a usable state.

The ultimate problem

Unfortunately we were still missing something. In order to do anything useful with Rundeck it needs a list of nodes on which it can operate, provided by a nodes.xml file containing the basic details needed to connect to a node in your environment. Puppet already knows all of this information so of course we would like to get at it without too much hassle.

Passing Puppet node data to Rundeck seems like it would be simple enough task – just query the Puppet DB via the nodes API; however, in our case we had a small hiccup. In our environment the Puppet master does not expose the Puppet API remotely so we can only access the nodes endpoint on localhost. This meant that any solution we came up with had to run directly on the Puppet master.

The penultimate solution

Fortunately there is a gem available that can import Puppet nodes into Rundeck here. It’s a fine solution for the right situation but it did not quite fit ours. The problem we ran into is that it creates another service API to install and maintain and it requires remote access to your Puppet master. Neither of these were an option in our case.

Conjunction junction! What’s your function?

Eventually we realized that there is actually a simple solution. It turns out that custom Puppet functions run on the Puppet master during manifest compilation. This means that we can use a custom function to gather data from the Puppet nodes endpoint and pass the content back to the agent.

If you haven’t worked with custom functions before in Puppet, setting them up is simple and is explained in the Puppet documentation here.

The solution.

Here is a an example function that can get the job done. We extracted it from a larger source and modified it for simplicity and of course names have been changed to protect the innocent. Note that it assumes you have a function called get_fact that will return fact data for a given hostname formatted however you prefer. There are several ways to accomplish this that have been left off for simplicity.

require "net/http"
require "json/pure"
module Puppet::Parser::Functions
  newfunction(:nodes_to_rundeck, :type => :rvalue) do |args|
    NODES_URL  = "http://localhost:8080/v3/nodes/"
    FACTS_PATH = "/facts/"
    #get the list of nodes from the nodes api
    node_data  = Net::HTTP.get_response(URI.parse(NODES_URL))
    nodes      = JSON.parse([node_data.body])
    xml_output = ""
    #loop over each node and compile the necessary fact data.
    nodes.each do |node|
      hostname  = node['name']
      ip_value   = function_get_fact([hostname, 'ipaddress'])
      role_value = function_get_fact([hostname, 'configured_role'])
      arch_value= function_get_fact([hostname, 'architecture'])
      os_value   = function_get_fact([hostname, 'osfamily'])
      #xml added inline for simplicity, you may want this in a template
      xml_output +=  <<-XML
      <node name="#{ hostname }"
        type="Node"
        description="Node Sourced from Puppet Master"
        hostname="#{ip_value }"
        osArch="#{arch_value }"
        osFamily="#{ os_value }"
        tags="#{role_value }"
        username="rundeck" />
      XML
    end
    #return the xml output 'r-value' back to the calling manifest
    xml_output
  end
end

Below is how you would use it to create a nodes.xml file in your Puppet manifest.

  file { ${rundeck_nodes_path}/nodes.xml":
    ensure => 'present',
    content => nodes_to_rundeck(),
  }

Are we there yet?

Almost! Once you have a nodes.xml file created locally, you can pass it to the Rundeck Puppet module to let Rundeck know that it exists.

class { 'rundeck' :
    rdeck_home            => $rundeck_base,
    package_ensure        => 'present',
    projects_organization => 'your_org',
  }
  $node_sources = {
    '#{rundeck_nodes_path}' =>  {
      source_type => 'file',
      resource_format => 'resourcexml'
    }
  }
  rundeck::config::project { $rundeck_project_name :
    resource_sources => $node_sources
  }