Intro

If you’ve never heard of a Salt “formula” it’s probably because the term is somewhat ambiguous. It’s analogous to a Chef Recipe or an Ansible Playbook. In those tools, a specific directory structure is dictated by the product - your variables are defined here, your desired states are declared here, your file templates go there, etc. - and it must be followed for the tool to work.

With Salt, there are no rules about how you structure your configuration management code. This provides a lot of flexibility, and that flexibility can come at a price. One consequence of all that flexibility is that the notion of a “formula” is more of a convention than an actual product feature. The Salt team have recognized the need for structured, reusable formulas, and there is even an collection of them online with hundreds of pre-built solutions, but that does not mean there is a single standard for what a formula looks like.

So why did I feel the need to create a new framework for formulas? Simply put, the existing solutions have never worked for me. Here’s why:

Salt users will often use a formula if one already exists, and don’t follow any standard at all if one does not. This leads to a very inconsistent code base where SLS folders can vary wildly in the features they support.
Formulas are seen as big and complicated - the current formula template contains 150 files! Salt users seem to like them if someone else is maintaining them, but they’re considered to be too much trouble to write for your own code.
I want a standard structure for all of my configuration management code. I need something lightweight enough to use everywhere that also provides the features I need most.
The online collection of formulas do not follow the same standards. They are not curated, vetted, or tested by a single organization, so it can be a management challenge to support them in your own environment.
Because there are no standards enforced, code is duplicated, often with slight differences, in each formula directory.

The existing formula system is just too big and cumbersome to manage successfully. Surely we can come up with something a little easier to use…

Pinch: A Framework for Salt Formulas

https://gitlab.com/paragonsginc/pinch

Here are some of the goals of this new framework:

Make formulas easy to use. The Pinch formula template contains just 2 files, so you can standardize all your configuration management code as Pinch formulas.
Avoid hard-coded references within the formula to itself. These break any time code is moved, copied, or renamed. Pinch provides a simple solution for formulas to reference themselves.
Minimize the use of conditional statements in state files. These make your code hard to read and maintain. Pinch includes a simple system to override configuration values for different operating systems, or by grains and pillars.
Provide a mechanism to create unique state identifiers. One problem when managing a large configuration management code base is there is no way to know if your state identifier is already in use. Pinch provides a standard mechanism to apply namespaces to state identifiers.
Avoid duplicate code. Pinch shares the same Jinja code for all formulas, while permitting overrides for each formula if some custom behavior is required.
Eliminate unneeded files in formulas. Pinch requires each formula to contain, at a minimum, an empty defaults.yaml file and a single line of Jinja in your SLS files. That’s all it takes to benefit from the Pinch framework.

Getting Started

The only files needed to make all of this work are formula.jinja and builtins.jinja from the shared directory. These files can be in the formula directory or any of its ancestor directories. My recommendation is to put those files at the root of your states directory. That way all formulas share the same Pinch framework code. If you need another copy for some reason (e.g. debugging, custom behavior, etc.), you can always place it in the formula directory and it will override the one in the higher directory for that particular formula.

Once those two Jinja files are in-place, you are ready to create Pinch formulas. You can use cookiecutter, copy one of the sample directories, or just start modifying your existing SLS files. There are only two things required for Pinch to treat a directory as a formula:

Each formula must contain a defaults.yaml file (it can even be empty). This file is how Pinch recognizes a directory as the root of a formula.
Each SLS file must contain a Jinja import statement, which imports the shared Pinch framework code and injects some variables into the Jinja namespace.

There are also ways to make formula variables available to template files, and to import the namespace of another formula. See the README for details.

Benefits

If needed, Pinch also includes a mechanism to define configuration variables, and override them depending on the operating system. Values can be overridden using grains or pillar data. This provides an extremely flexible tool for creating formulas that work on a variety of infrastructure.

This is done by defining a default value in defaults.yaml and defining OS-specific overrides in either osfamilymap.yaml or osmap.yaml. This is similar to how many existing Salt formulas work today. One difference is that Pinch allows Jinja code in your YAML files.

Pasting one line of Jinja (the import statement mentioned above) into your SLS files produces quite a few benefits:

Detect the path of the current formula in the state.
Merge the YAML configuration files, according to the minion operating system.
Merge Grains and Pillar data, if present, to override the YAML configuration files.
Yield a formula dictionary containing the appropriate configuration values for the minion operating system.
Define the following built-in keys/values in the formula dictionary:
- formula.path The full formula path, separated by slashes.
- formula.namespace The full formula path, separated by dots.
- formula.url The URL to the formula in the state tree.
- formula.name The last folder name in the formula path.

Example

Any values needed by the formula can be defined in defaults.yaml. Those values can be overridden by other YAML files, Grains, or Pillars. Notice that we can even include Jinja templating in our YAML files. An import statement is required to access Pinch built-in variables, as in the following example.

{% from salt["slsutil.findup"](sls, "builtins.jinja") import formula with context -%}

package: sample
service: sample
configfile:
  path: /etc/sample.cfg
  source: {{ formula.url ~ "files/sample.cfg" }}
  user: root
  group: root
  mode: "0644"

The variables defined in defaults.yaml are combined with the Pinch built-in variables (path, namespace, url, and name) to produce a single dictionary containing all configuration values for this formula. By default, this dictionary is called formula, but another name can be assigned if needed. Using the values in the formula dictionary, we can create reusable state files like this:

{% from salt["slsutil.findup"](tplfile, "formula.jinja") import formula with context -%}

# Install the package
{{ formula.namespace }}.install:
  pkg.installed:
    - name: {{ formula.package }}

# Manage the config file
{{ formula.namespace }}.config:
  file.managed:
    - name: {{ formula.configfile.path }}
    - source: {{ formula.configfile.source }}
    - template: jinja
    - user: {{ formula.configfile.user | yaml_encode }}
    - group: {{ formula.configfile.group | yaml_encode }}
    - require:
      - pkg: {{ formula.namespace }}.install

# Start the service
{{ formula.namespace }}.start:
  service.running:
    - name: {{ formula.service }}
    - enable: true
    - require:
      - pkg: {{ formula.namespace }}.install
    - watch:
      - file: {{ formula.namespace }}.config

Notice how this SLS file now achieves all the goals laid out above:

Make formulas easy to use for all Salt states.
Avoid hard-coded references within the formula to itself.
Separate configuration values from code.
Minimize the use of conditional statements in state files.
Define default configuration values and ways to override them.
Provide a mechanism to create unique state identifiers.
Allow a formula to be renamed or moved without breaking references.

Try It!

Pinch is free and open-source. Give it a try today to supercharge your Salt states.