Category: SoftwareAutomation

Getting Started With Chocolatey 4 Business & Jenkins CI

I’ve had an argument with a colleague quite recently about the usage of open-source automation tools, especially Chocolatey in business environments. A keypoint of this argument was the integration of new/open tools in long-existing, mostly commercial software based workflows.

One of the main reasons to use Chocolatey in corporate environments is its ability to integrate seamlessly with already existent automation infrastructure. I’ve been able to integrate Chocolatey successfully with Jenkins CI, Finalbuilder, corporate internal windows services as well as simple ‘installer-batch-files’, a friend and coworker (flurl) is deploying a corporate internal management solution with Chocolatey.

The day I replaced the multi-100 line content of a legacy batch installer with a single line of

choco install {package} -y

was a really good day πŸ™‚

In this blogpost I’m going to give you an idea of how you could get started with Chocolatey at your organization concerning two parts: a possible way to roll-out Chocolatey to your coworkers and a small collection of things to get started integrating Chocolatey with a CI-server (Jenkins, but probably applicable to anything else that does support PowerShell).

Deploying Chocolatey At Your Organization

You’ll want to make it as easy as possible, hence putting a .bat-file on a network share that’s accessible from everywhere in your org is probably a good idea.
If you’re just getting started with Choco I’d also suggest putting your first packages on such a share.
Let’s assume following share-layout:

  • \\your-server\Choco\install
  • \\your-server\Choco\packages

Go to https://chocolatey.org/packages and fetch the latest Chocolatey.nupkg file. Put that file together with this install.ps1 and this install.bat onto the Choco/install share.

You need to adapt the URLs and UNC-paths to the scripts inside the files to make them work in your environment.
Now you’re good to go, just double-click on install.bat from any machine and Chocolatey will be ready to use.
Notice that the community repository is removed during the installation progress and another (internal) repository is set up. For details on how this is done, check the contents of install.ps1.

Your First Packages

Doing a

choco list
now looks pretty boring, because there are no packages at your feed yet (the community repository has already been removed for corporate usage), so go ahead and download some .nupkg-Files from https://chocolatey.org/packages and put them into your Β prepared network share.

Now you should be able to list the available packages with ‘choco list’, search for packages with ‘choco search’ and install packages with ‘choco install’.

Of course you can also create your own packages, just type

choco new {package name}
and see what happens.

So far, you’ve only been using OSS features, and you can use all this without limitation in a corporate environment, but there are some things you may need to consider: the packages you’ve pulled in from https://chocolatey.org probably rely on downloading external resources like .exe, .MSI or .zip files. You don’t want to do that in an organization, as you’ll want to have complete reliability in your solution, and downloading from the internet could have reliability issues (not to mention trust). There’s two options: manually download the required resources, changing the source code of the packages install.ps1 and repackage every single one of the required packages, or you could simple use package internalizer.

Package Internalizer

The best of the pack. If you’re using Chocolatey in a corporate environment, you’ll really dig this. Just type

choco download {packagename} --internalize
and let the magic happen. This will automatically download all needed assets and put them into the generated package, so you’ll achieve the maximum availability.

Just keep in mind that you’ll have to license all hosts that are consumers of packages you manage via internalization. (Chocolatey for Business starts at just $600 USD for up to 35 machines, and is $16/machine/year with volume discounts thereafter – see https://chocolatey.org/pricing)

Choco + Jenkins

Let’s automate the automation!

To be able to use the following jobs you’ll need to know the basics of Jenkins job configurations. You’ll need to set-up parameterized jobs with the PowerShell plugin, also Chocolatey should already be installed on the corresponding Jenkins nodes.

This job just updates all the Chocolatey packages on the server.

choco upgrade all -y

Yes, it’s as easy as that πŸ™‚
This simple Jenkins-job allows you to internalize any package from the community-Feed with the click of a button:

# PowerShell script to internalize chocolatey packages from the community feed to an internal server using the business edition 'internalize' feature. This script is designed to be run from jenkins, P_* variables are defined in jenkins job!
# section CREDS
	$pkgs = $env:P_PKGLIST
	$uncshare = $env:P_UNC_SHARE
    $targetfolder = $env:P_DST_FOLDER
	$targetserver = $env:P_DST_SRV
	$apikey = $env:P_API_KEY

	$envtmp = $env:temp
	$tmpdir = "$envtmp\chocointernalize"
	$basefeed = "https://chocolatey.org/api/v2/"
# endsection

function InternalizePkg($pkg) {
	Push-Location $tmpdir
	choco download --internalize $pkg --resources-location="$uncshare\$pkg" --source="$basefeed" --no-progress
	$genpkg = ((Get-ChildItem *.nupkg -recurse).FullName | Select-String -Pattern $pkg)
    if ($targetfolder) {
        Write-Output "> copying package '$genpkg' to '$targetfolder'"
        if (-Not (Test-Path $targetfolder)) {
            New-Item $targetfolder -ItemType Directory -Force -Verbose            
        }
        Copy-Item $genpkg $targetfolder -Verbose -ErrorAction "Stop"
    } else {
        Write-Output "> pushing package '$genpkg' to '$targetserver'"
        choco push $genpkg --source="$targetserver" --api-key="$apikey" -Verbose
    }
    Write-Output "------------------------------------------------------------------------"
    Write-Output ""
	Pop-Location
}

if ((Test-Path $tmpdir)) {
	Remove-Item $tmpdir -Recurse -Force -Verbose
}
New-Item $tmpdir -ItemType Directory -Force -Verbose

$pkgs | ForEach-Object {
	InternalizePkg $_
}

Remove-Item $tmpdir -Recurse -Force -Verbose

 

This job reports what packages are outdated. (A newer version of that package is available at the community feed)


$mrecipient = $env:P_MAIL_REC
$msender = $env:P_MAIL_SEND
$msmtp = $env:P_SMTP_SERVER
$availPkgs = @()

#$chocoout = $(choco outdated)
$chocoout | ForEach-Object {
    $up = $_.Split("|")
    if ($up -And ($up[3] -ne "false")) {
			if ($up[1] -eq $up[2]) {
				$availPkgs += "$($up[0]): $($up[1]) -> $($up[2])"
			}
		} 
	
}

$res = $availPkgs | Format-Table | Out-String
if ($res) {
	Send-MailMessage -To $mrecipient -Subject "Chocolatey Packages Outdated!" -Body $res -Verbose -ErrorAction "Stop" -From $msender -SmtpServer $msmtp
}

Mind that this setup and these scripts are really minimalistic. When you’re working in an environment with a couple of users you may want additional feeds to create some logical package separation or access control. There are unimaginably many use-cases on how to interop with a command-line utility such as Chocolatey, but I just wanted to give you a quick dive-in how my first contact with Chocolatey looked like. (or how I wished it looked like to be honest ;-)).

Happy hacking!

Links:

Using Chocolatey In Corporate Environments (Part #1)

It’s been about a year now that I’ve started pulling in Chocolatey as a tool for software deployment for the development teams at my workplace.
As of today, the setup of development machines, buildservers as well as specific terminal-hosts have been fully transformed from manually following many-pages of wiki install instructions to a single ‘meta-package’ that takes the hosts from 0 to 100 in ‘no-time’ (really, just like an hour compared to a full work-day previously!). And yes, all machines receive their very-own configuration, licensing for different software packages etc. etc. hyped? so am I!

Even though this is pretty obvious, it doesn’t hurt to recollect:
Having the install-procedure as well as the configuration in code (PowerShell in this case) makes it repeatable and less error prone! In addition to that, you’ll want to put your code under version control, so you’ve got the history covered!

Package Separation

For the sake of convenience I’ve decided to split up the packages that actually install software and the configuration of this software into separate packages.

vs2010
vs2010-settings-teamX
vs2010-settings-teamY
...

Internalize Everything!

In a business scenario, you probably don’t want to rely on public package source repositories (for multiple reasons) – so downloading the wanted packages, doing a virus-check and putting the artifacts on an internal-only server is basically what you want 99% of the time.
This can be a challenging task when done manually, trust me – I’ve been doing this and trying to automate it for about a month, but wait – this task already IS fully automated via the ‘internalize’ feature!
This feature is only available in the ‘business’ licensing model of Chocolatey, if you’re planning on using Chocolatey for work – go get a trial and let the magic work for you!

It is literally as easy as this:

choco download --internalize $pkg --source=https://repo...
choco push $pkg*.nupkg --source=https://myorg-choco... --api-key=$apikey

If you’re keen on going a step further, I’ve put up some sample ‘internalize’ script that I use to fetch packages via Jenkins CI. [internalize.ps1 on GitHub]

Completely Offline Chocolatey Installation

.\corporate_install.ps1 -dont -bug -me -with -external -resources

If you want to get into a lot of detail here, go check out the manual at chocolatey.org.

One thing that is currently missing (afaik) is a completely-offline Chocolatey installer.
You can always take the install.ps1 from chocolatey.org and modify it to your needs – but when you look at chocolateysetup.psm1 you’ll see that even a offline copy of chocolatey.nupkg could try to download a .Net Framework from a Microsoft mirror.

To overcome this rather annoying circumstance, I’ve decided to modify the base install.ps1 a little further to install a .Net Framework, if missing before installing Chocolatey – so no public internet connection is needed at any point.
I’ve put my efforts down into a script called corporate_install.ps1.
The script can be parametrized, so you should be able to fit it into your organization by just passing the right parameters!

You’ll also notice I’ve pulled Boxstarter, a great project adding reboot-resiliency (among many other cool features), into my base setup – this is because I use it all the time!

MIND! The first thing you’ll want to do when using chocolatey in a corporate environment is to remove the public community repository. (this is automatically done in my ‘corporate_install.ps1’ ;-))

choco source remove --name chocolatey

Artifact Server

Currently I’m using ProGet as artifact server, for solely two reasons:
1. It’s extremely easy to set-up and use
2. It’s got a feature called ‘package cache’

The package-cache feature pulls the wanted package from the HQ to the local package cache when first requested.
With this feature we set-up ‘location-based’ servers for our remote-offices. Now all employees are pulling packages from their ‘closest’ server – so no extra bandwith to HQ is needed.

Configuration Deployment & Software Licensing

This is probably the most ‘tricky’ bit from the currently existing setup and therefore will get an extra blogpost asap. Just note: we’ve created a simple ‘configuration service’ containing information about ‘software’ and ‘licensees’ (‘software’ matches the package-id and a ‘licensee’ currently is either a hostname or Domain\Username). This information can be pulled from the chocolatey*.ps1 scripts via inline C# … … …

Multiple Chocolatey Package Source Repositories

If something turns out to be working great, you’ll want it to scale with your needs. In order to be able to scale ‘company-wide’ it seemed to be a good idea to split-up our Chocolatey packages into multiple package repositories for the following reasons:

  • packages could be grouped logically
  • (office packages, developer packages, testing packages, …)

  • package visibility could be controlled by the activated package source repositories
  • (office hosts don’t even get the chance to install i.e. 20GB of VisualStudio)

  • favor package versions based on different feed-priorities for different teams/ user groups
  • (team ‘X’ is on a ‘slow’-feed that only contains VS2015 update I, not II because a certain VC compiler settings would be unwanted for Product X, team ‘Y’ is on a ‘fast’-feed with all the VisualStudio versions available)

We’ve came up with a ‘hierarchy’ that can be described as follows:

(technology)-[(department|team)*]-[(project)*]

I will devote a whole blogpost to this, so stay tuned πŸ™‚

Chocolatey Support

What should I say? I’ve had a couple of talks with Rob Reynolds, Gary Evan Park, Rich, Kim, Pascal,… (yup, basically the whole Chocolatey team). If you need help or have any question – just get in contact via gitter, they’ll be happy to help.
I will also be hanging around there sometimes πŸ˜‰

Why Working On Windows Is Fun Again

Hint: it’s because of PowerShell and Chocolatey! πŸ™‚

I’ve just needed to setup IntelliJ with Scala-Plugin and JDK on a new machine:

choco install jdk8 -y
Chocolatey v0.10.3 Business
Installing the following packages:
jdk8 v8.0.121 [Approved]
Chocolatey installed 1/1 packages. 0 packages failed.
choco install intellijidea-community -y
Installing the following packages:
intellijidea-community v2017.1 [Approved]
Chocolatey installed 1/1 packages. 0 packages failed.

Automating Software Automation

Yet another Chocolatey/PowerShell post!
Sorry, I rarely have time to put my thoughts to words – so I take any chance – and lately I’ve been thinking about software automation on Windows a lot! The possibilities seem quite overwhelming, and I like this alot! πŸ˜‰

Continue reading “Automating Software Automation”

It’s All About The Choco(latey?)

Things chocolate is good for:

  • eating chocolate – the darker the better
  • chocolate fondue
  • chocolate brownies
  • triple choco cookies
  • installing software

> installing software . . .Β (WAIT WHAT ?!?!?!)

The last one isn’ quite right – but, no matter how sweet chocolate is, chocolatey might be even more delicious!

Ok, clarification: Chocolatey (short: “choco”) is a package manger for Windows that actually works really well, has loads of packages in the public repo – and you can create packages yourself pretty easily!

– if you are not hyped right now – you may as well leave πŸ™‚ –

Continue reading “It’s All About The Choco(latey?)”