working on Boxstarter v3

Sun, Nov 28, 2021 9-minute read

What is Boxstarter?

Big Kudos to Matt Wrock, the original creator of Boxstarter.

Boxstarter enables repeatable, reboot resilient windows environment installations made easy using Chocolatey packages. When its time to repave either bare metal or virtualized instances, locally or on a remote machine, Boxstarter can automate both trivial and highly complex installations. Compatible with all Windows versions from Windows 7/2008 R2 forward.

I like to put it this way: Boxstarter makes Chocolatey even sweeter to use, as it provides a very efficient, yet simple way to fully automate complex software deployments on both local as well as remote environments. If you’ve got a fresh OS, you can use Boxstarter to set it up just the way you want it - all you need is a simple script/configuration file that can be stored in a text file / gist / whatever - just tell Boxstarter to do its work and enjoy a coffee until it is done.

I’ve been using Boxstarter since late 2016 and still remember how I immediately fell in love with how easy it allowed me to automate complex setups that required multiple reboots, remote scenarios as well as automating installing Windows updates.

I got totally hyped when Channel 9 announced that Boxstarter was handed over to Chocolatey, and even more so: Microsoft itself uses Boxstarter to setup dev boxes.

Is Boxstarter still a thing in 2021?

Boxstarter development unfortunately took quite a hit some time ago, one major problem being that all current v2.x releases depend on Chocolatey version 0.9.8. Due to the nature how Boxstarter connects with Chocolatey, no licensed extension/features could be used and thus basically put off all C4B customers. Naturally, features and bugfixes that came with newer choco versions never made it to Boxstarter. As a consequence, a lot of issues were raised on Github that were long gone in Chocolatey - those have partially been fixed in Boxstarter or were put aside because of the dependency on “old Chocolatey”.

All in all, not a very pleasant situation.

BUT NO, BOXSTARTER IS NOT DEAD

Big thanks to Paul and David, who took their time and sat through a couple of meetings with me earlier this year, drafting the possible future of Boxstarter and most important things to get done right away.

So what’s going on?

One of the tasks I’ve been working on in 2021 has been rethinking how Boxstarter interacts with Chocolatey and see how it can be changed in order to enable compatibility with newer Chocolatey releases (and therefore all new features and bugfixes since choco 0.9.8).

Since this is a major change in how Boxstarter works, we’ve decided also to give this a major version number change - meaning Boxstarter v3 is actively being worked on and will ship with a current version of Chocolatey (0.11.3 at the time of writing this).

Boxstarter v3

Boxstarter’s value comes from extending Chocolatey with its own CMDLets, such as it’s WinConfig features, Reboot resiliency as well as the ability to run remotely.

Boxstarter x Chocolatey - the hidden complexity

The way Boxstarter < 3.0 extends Chocolatey requires a deeper understanding of how choco itself is wired up and connects to its internal PowerShell host. The basic idea was to modify the way Autofac wires up internal components and provide an alternative implementation for specific parts. To put a finger on it, Boxstarter directly calls Lets.GetChocolatey().RegisterContainerComponent in order to inject a custom IPowerShellService that all Chocolatey commands will run in.

“Welcome to the sausage factory” – © Rob

It took me more time than I like to admit to grasp the implications of this design decision and even more to figure out if using the regular choco.exe was an option at all.

Chocolatey extensions to the rescue

Extensions allow you to package up PowerShell functions that you may reuse across packages as a package that other packages can use and depend on. This allows you to use those same functions as if they were part of Chocolatey itself. Chocolatey loads these PowerShell modules up as part of the regular module import load that it does for built-in PowerShell modules.

This is just what we need for Boxstarter CMDLets to be available in Chocolatey packages. Also, creating a extension module for Boxstarter allows us to check for pending reboots before and after each package installation as well as perform neccesary tasks for reboot resiliency.

Nevertheless, it is NOT an option to turn Boxstarter solely into a Chocolatey extension - it probably would work to some degree, yet a “Boxstarter extension” fundamentally changes the way Chocolatey behaves - and this is not what Boxstarter aims to do.

Boxstarter should empower users to do complex tasks easily and utilize Chocolatey during its operation.

Starting in Boxstarter version 3, a dynamicaly generated Chocolatey extension named boxstarter-choco will be installed when utilizing choco commands in a Boxstarter script. The extension module is removed in the cleanup phase of Boxstarter, therefore Chocolatey itself is not affected by using Boxstarter or having the Chocolatey Boxstarter packages installed on a machine.

This “new design” untangles some of the current complexity in Boxstarter and hopefully will pave the way for further improvements.

Breaking changes

Changing how Boxstarter interacts with Chocolatey might not come as a “seamless drop-in replacement” for all users of Boxstarter.

Many currently existing bugs will be fixed with Boxstarter v3, but you know what they say about other users workflows:

EVERY CHANGE BREAKS SOMEONE’S WORKFLOW

Chocolatey.dll no more

Boxstarter does not depend on chocolatey.dll anymore. In fact, it only comes with a nupkg version of Chocolatey in case it has to install Chocolatey. Since Boxstarter does not come with the library version of choco, it has to ensure “the actual thing” is present on the target machine before any Chocolatey related commands can be executed.

This change leads to all of the following points/changes:

Boxstarter installs Chocolatey

Up to v3, it was possible to use a “portable version” of Boxstarter, utilizing Boxstarter.bat from a once-installed, then packaged Boxstarter installation. (you needed to install Boxstarter once, after that you could run the whole thing from $BoxstarterBaseDir - if you copied this folder, you effectively created a portable version of Boxstarter)

Without chocolatey.dll at hand, there is no possibility of installing Chocolatey packages. To come around this problem, a nupkg-distributable of Chocolatey is included in Boxstarter, just as the chocolatey.dll has been before. If Boxstarter doesn’t recon choco is already present on target machine, it’ll utilize the included nupgk to install Chocolatey before doing anything else.

Dedicated PowerShell sessions

Due to the nature of how Boxstarter hijacked the PowerShell host of Chocolatey, all choco install / choco <whatever> commands ran in the same PowerShell session. This lead to a couple of problems, i.e. when installing multiple packages that loaded the same assembly using Add-Type. (Issues you would not see with regular Chocolatey, as each choco.exe has it’s very own PowerShell host and therefore clean environment).

Starting in Boxstarter v3. each call to choco.exe is done via Start-Process, yielding a clean environment and hopefully the same behavior as using a couple of choco commands from the command line / a script.

Ability to use C4B features and commands

Up to Boxstarter v3 loading of the Chocolatey extension simply failed an you got (multiple) messages like this

Error when registering components for 'chocolatey.licensed.infrastructure.app.registration.ContainerBinding':
 Exception has been thrown by the target of an invocation.

whenever using Boxstarter to install a package.

This behavior was caused by the way the Chocolatey licensed extension is being loaded and effectively could not instantiate required components because Boxstarter directly accessed RegisterContainerComponent and messed with choco’s internal state.

Future plans

As mentioned in the beginning, there have been a couple of meetings in which we’ve been discussion on how we’d like Boxstarter to evolve and fix some of the most urgent pain-points. As soon as Boxstarter v3 is released, there will be further changes, we’ve got a lot of things to do, but following are the points I care most about.

CI integration, workflow enhancements

During the preparation of Boxstarter v3, James configured AppVeyor (which is currently used to build Boxstarter when doing PRs) to keep the binaries, so they can be downloaded and tested. Apart from that, we currently do not automatically run any tests, do code analysis or status badges. Implementing those features, in combination with a higher (beta-) release frequency, will make it easier to get new people on board and let the community members step in to fix bugs and implement features with confidence.

PowerShell 7+

Boxstarter v3 still aims to be compatible with PowerShell v2, but the cost of maintaining this compatibility is something we constantly have to deal with. There already have been requests and inquiries if Boxstarter could run in a pwsh host and at some point we will be moving away from PowerShell version 2 towards version 4, including PowerShell v7.

Also, the ability to run on a v7 host could possibly lead to being able to use Boxstarter in a CI system, orchestrating other (remote) hosts - independent of the platform Boxstarter is running on.

Ok, the second part may take a while and also is probably more of a personal side-project as I vastly prefer working on my Linux boxes… but hey, priorities might change 😄

my role in all of this

I’m a maintainer. I care about the Boxstarter project just as I care about Chocolatey, as the combination of these tools basically marked my “step into the DevOps world” way back in 2016. Being a software dev for a living, father of two and happy dog-owner I naturally got little time to spare for side projects. I used to spend about 4 hours a week doing package moderation on the Chocolatey Community Repository. Over that last couple of months, I basically stepped down from doing any moderation and used this time to work on Boxstarter instead. With Boxstarter version 3 being around the corner, I hopefully will find some time again to work on the community repository, but still watch out for things that need to be done on the Boxstarter project.

Are you a current user of Boxstarter / do you know some PowerShell / are you interested in fixing some issues in a OSS project? It’d be awesome if you could get in touch - I’m certain we’ll find something for you to work on that’ll benefit not only you, but all of the Boxstarter/Chocolatey community 😃