Puppet 3.2 Introduces an Experimental Parser and New Iteration Features

Puppet 3.2.1 landed today. Though it's a "patch" release, it's the first public release of the Puppet 3.2 series, and it includes a taste of the Puppet DSL's future in the form of an experimental parser that introduces some new features you'd expect to find in traditional programming languages.

I spoke to Puppet product owner Eric Sorenson about the Puppet 3.2 series, and he called out the new parser as the headline item:

"It's called 'Future Parser,'" he said, "because that's the command line argument you have to pass to puppet in order to turn it on. It's really a ground-up reimplementation of the Puppet language, using an expression- instead of statement-based grammar, which allows both a lot more power and flexibility with what you can do inside the language."

Eric said the new parser was a response to the tough decision made with Puppet 3.1 to deprecate the Puppet Ruby DSL, a pure Ruby implementation of the Puppet DSL.

"What we did instead was double down on the Puppet DSL by implementing the things that people who were using the Ruby DSL were actually trying to do. We asked, 'what is that capability? What is it that made you feel the Puppet DSL wasn't meeting your needs and caused you to turn to the Ruby DSL?'"

According to Eric, "people wanted the ability to iterate. They wanted loops. Puppet looks enough like a programming language that people who are used to programming languages say, 'I should be able to do all the things I do in Ruby or another language.'"

In the experimental parser, Puppet has picked Ruby-like "each" loops, along with many of the same methods one could call on an enumerable object in Ruby: inject, collect, select and reject.

Eric noted that the new parser isn't enabled by default, "because there are a couple of things we want to work out, and we really want to hear feedback on."

For instance, "there are two implementations of loop structures. There's one where each looks like a function similar to existing Puppet parser functions like include or template. The other one is a dot-suffix operator, like that found in Ruby."

The goal of including two ways to express the same thing is all about community feedback:

"We're interested in learning which people think is more readable and more in line with their expectations. We're going to do some user testing from the UX side and solicit feedback from the community."

You can give the new parser a shot by updating to Puppet 3.2 and enabling it in one of two ways:

Setting parser = future in your puppet.conf file
Adding the command line switch --parser=future

Eric said the experimental parser will remain so until the Puppet developer and UX team can gather community feedback, and learn how it interacts with all the Puppet manifests already out in the wild:

"We're curious not just about how the parser works specifically with new constructs, but how it works with the existing body of Puppet code."

You can read more about the new parser, including how to use the new iteration functions, in some documentation prepared by Puppet developer Henrik Lindberg on the Puppet docs site.

External CA Support

Puppet 3.2 also includes better support for external certificate authorities, contributed by an unusual visit from Mozilla's Dustin Mitchell, who flew out to the Puppet Labs offices to work directly with the Puppet development team.

"We've never had anybody from outside the organization come in and sit with us and actually do community contributor stuff in real-time," said Eric.

Dustin's contribution involved providing Mozilla's own Puppet use as a blueprint to development efforts around external CA support:

"Because SSL is hard and PKI is hard, we didn't want to go in shotgun and make everything under the sun work. We wanted to really specifically focus on what Mozilla's use case was and use it as a spec to write to."

The result of that approach is a working reference for other Puppet users who want to use an external certificate authority:

"If they follow the Mozilla model, then we can guarantee that will be supported in Puppet. [Puppet Labs developer] Jeff McCune did some great documentation work around that, describing what's supported and not supported."

Wi-Fi Puppet With OpenWRT

Also new in Puppet 3.2 is a collection of updates that address compatibility with OpenWRT, a Linux distribution popular on embedded systems like Wi-Fi routers. OpenWRT makes it possible to turn simple consumer routers into full-fledged Linux systems, and now Puppet can manage them.

Some of the support is predicated on the recent Facter 1.7 release:

"There were some assumptions in Facter that were not correct for non-x86 architecture," Eric said. Facter addresses that problem by working better with ARM chipsets.

In Puppet, "the support is specifically for OpenWRT. There's a new package provider that supports the OpenWRT OPKG packaging package manager," and there's also support for managing services on OpenWRT.

"Facter 1.7 enables the detection of OpenWRT, and Puppet 3.2 guarantees we can do something useful," said Eric.

And Everything Else

Besides a new, experimental parser, better support for external CAs, and support for a whole new class of device, Puppet 3.2 includes a few other new features:

Puppet's "splay" setting, which was meant to mitigate the problems caused by a thundering herd of agents hitting their puppet master at the same time, has been improved.
Puppet officially supports Ruby 2.
Puppet module tool now works on Windows

… and even more. As always, the complete list of bug fixes and new features can be found in the official release notes for Puppet 3.2.