Chef Client 12.1.0 Multipackage Installs

This is a guest post by Phil Dibowitz, Production Engineer at Facebook

At the Chef Community Summit this year, as with all Summits, there was a Hack Day. My hack project was to extend Chef’s `package` resource to be able to handle multiple packages at once. Why? There are two primary use-cases:

* When provisioning a new system, installing your base set of packages one at a time is very time consuming. It means setting up an HTTP transaction to download the package, writing the package to disk, ending the HTTP transaction, starting a yum/apt/etc. transaction, unpacking the package to disk, closing/committing the transaction, and then rinse and repeat for each package. Being able to download all of those packages in one HTTP transaction saves a ton of time and installing them all at once saves some more. Even a minimal attempt at grouping one set of packages shaved 4 minutes off of provisioning time for us.
* Sometimes you have two or more packages that must be updated in a single transaction… perhaps because a file moved between them (or because upstream packagers set up poor dependencies). Chef was entirely incapable of handling this (without, for example, using an `execute` resource to call out to yum/apt/etc. directly)

It was a fairly large undertaking (26 files changed, 955 insertions, 266 deletions), and I had a lot of help from Lamont Granquist, but on February 4th, it was finally merged and will be available in Chef 12.1.0. The result – in addition to some cleaned-up code (and some uglied-up code) – is you can now do this:

This will not only keep the whole set of packages up-to-date, it’ll only upgrade the necessary subset (if any) and tell you which packages were upgraded (if any).

Note that this can make for some ugly logs – particularly on long lists of packages, so we recommend this instead:

Using this format has the advantage that any notifications or subscriptions that point to to this will not need to be updated if you add/remove a package from the list.

But we don’t just support `:upgrade`, you can also do:

Again, this will figure out all the changes that need to happen to converge the state of these packages and do it all as one single yum/apt transaction.

And of course the same with `:remove` and `:purge`. Multipackage rules are supported on platforms that use the `yum` or `apt` providers. While the base `package` provider fully supports it, the underlying subclasses must also support it, and only `yum` and `apt` have had the appropriate surgery.

In addition, Lamont wrote the Multipackage cookbook which you can use to build a list of packages throughout your run that will be installed in a single multipackage call. On Chef versions < 12.1.0 it will fall back to a loop of single-package rules for backward compatibility. I hope this helps you cook up even more awesome!