Using the wash gem to write your external plugins

Note: This blog post’s been updated with the meta attribute => partial metadata change in Wash 0.18.0 and wash-ruby 0.3.0.

Wash ships with plugins that let you interact with your AWS, GCP, Docker, and Kubernetes resources. You can also write your own plugins via the external plugin interface. An external plugin consists of a plugin script. Wash shells out to this script whenever it needs to invoke an entry’s supported action (like list), or if it needs to query something about the entry (like its metadata). In a more general sense, Wash shells out to the plugin script whenever it needs to invoke an entry’s supported method. All plugin script invocations adopt the following calling convention

<plugin_script> <method> <path> <state> <args...>

where

Now let <entry> = <path> <state>. Then the plugin script’s usage becomes

<plugin_script> <method> <entry> <args...>

From this usage, we see that <path> and <state> are two different representations of an entry. <path> is useful for simple plugins where reconstructing the entry is easy. <state> is useful for more complicated plugins where entries could be represented as classes.

If you’re a Ruby developer, then the wash gem makes external plugin development easy by letting you model your entries as classes. We’ll show you what this looks like by walking you through some of the puppetwash plugin’s implementation, specifically the puppetwash.rb file.

To facilitate the following discussion, here’s what the puppetwash plugin looks like

wash . ❯ stree puppet
puppet
└── [pe_instance]
    └── nodes_dir
        └── [node]
            ├── catalog
            ├── facts_dir
            │   └── [fact]
            └── reports_dir
                └── [report]

If you’ve used Puppet before, then much of the above output should look familiar (and you should definitely try out the plugin!) If not, then don’t worry – this blog post is written for a general audience so you should still be able to extract some useful information from it.

Each label that you see in stree’s output corresponds to a class. For example, the puppet label (plugin root) corresponds to the Puppetwash class. The pe_instance label (a PE instance) corresponds to the PEInstance class. Similarly, the report label (a node’s report) corresponds to the Report class.

Let’s take a look at each class’s implementations. Note that we’re omitting some code for brevity.

class Puppetwash < Wash::Entry
  label 'puppet'
  is_singleton
  parent_of 'PEInstance'

  def init(_wash_config)
  end

  def list
    config.keys.map do |name|
       PEInstance.new(name)
    end
  end
end

class PEInstance < Wash::Entry
  label 'pe_instance'
  parent_of 'NodesDir'

  def initialize(name)
    @name = name
  end

  def list
    [NodesDir.new('nodes', name)]
  end
end

...

# Report relies on end_time and hash. The others are included as useful metadata.
METADATA_FIELDS = {
  'end_time': 'string',
  'environment': 'string',
  'status': 'string',
  'noop': 'boolean',
  'puppet_version': 'string',
  'producer': 'string',
  'hash': 'string'
}

class ReportsDir < Wash::Entry
  label 'reports_dir'
  is_singleton
  parent_of 'Report'
  state :node_name, :pe_name

  def initialize(name, node_name, pe_name)
    @name = name
    @node_name = node_name
    @pe_name = pe_name
  end

  def list
    response = client(@pe_name).request(
      'reports',
      [:extract,
        METADATA_FIELDS.keys,
        [:'=', :certname, @node_name]]
    )
    response.data.map do |report|
      Report.new(report, @node_name, @pe_name)
    end
  end
end

class Report < Wash::Entry
  label 'report'
  attributes :mtime
  partial_metadata_schema(
      type: 'object',
      properties: METADATA_FIELDS.map { |k, v| [k, { type: v }] }.to_h
  )
  state :node_name, :pe_name, :hash

  def initialize(report, node_name, pe_name)
    @name = report['end_time']
    @node_name = node_name
    @pe_name = pe_name
    @hash = report['hash']
    @partial_metadata = report
    @mtime = Time.parse(report['end_time'])
  end

  def read
    response = client(@pe_name).request(
      'reports',
      [:and, [:'=', :certname, @node_name], [:'=', :hash, @hash]]
    )
    make_readable(response.data)
  end
end

Notice that

The Wash::Entry class makes all of this possible. It also comes with even more helpers that let you do other things like specify cache TTLs (see Wash::Entry#cache_ttls) or any prefetched methods (see Wash::Entry#prefetch).

Recall that plugin script invocations adopt the following calling convention

<plugin_script> <method> <entry> <args...>

where <entry> = <path> <state>.

All plugin script invocations are handled by the Wash.run function. Here’s how Wash.run is used in puppetwash.rb

Wash.run(Puppetwash, ARGV)

And here’s Wash.run’s implementation. At a high-level, Wash.run

Thus, we see that the wash gem frees you from having to do all of this plumbing yourself. All you have to do is define a few Ruby classes, implement some methods, then pass everything over to Wash.run to handle the rest. Try it out and let us know what you think!