Using Chocolatey In Corporate Environments (Part #1)

It’s been about a year now that I’ve started pulling in Chocolatey as a tool for software deployment for the development teams at my workplace.
As of today, the setup of development machines, buildservers as well as specific terminal-hosts have been fully transformed from manually following many-pages of wiki install instructions to a single ‘meta-package’ that takes the hosts from 0 to 100 in ‘no-time’ (really, just like an hour compared to a full work-day previously!). And yes, all machines receive their very-own configuration, licensing for different software packages etc. etc. hyped? so am I!

Even though this is pretty obvious, it doesn’t hurt to recollect:
Having the install-procedure as well as the configuration in code (PowerShell in this case) makes it repeatable and less error prone! In addition to that, you’ll want to put your code under version control, so you’ve got the history covered!

Package Separation

For the sake of convenience I’ve decided to split up the packages that actually install software and the configuration of this software into separate packages.

vs2010
vs2010-settings-teamX
vs2010-settings-teamY
...

Internalize Everything!

In a business scenario, you probably don’t want to rely on public package source repositories (for multiple reasons) – so downloading the wanted packages, doing a virus-check and putting the artifacts on an internal-only server is basically what you want 99% of the time.
This can be a challenging task when done manually, trust me – I’ve been doing this and trying to automate it for about a month, but wait – this task already IS fully automated via the ‘internalize’ feature!
This feature is only available in the ‘business’ licensing model of Chocolatey, if you’re planning on using Chocolatey for work – go get a trial and let the magic work for you!

It is literally as easy as this:

choco download --internalize $pkg --source=https://repo...
choco push $pkg*.nupkg --source=https://myorg-choco... --api-key=$apikey

If you’re keen on going a step further, I’ve put up some sample ‘internalize’ script that I use to fetch packages via Jenkins CI. [internalize.ps1 on GitHub]

Completely Offline Chocolatey Installation

.\corporate_install.ps1 -dont -bug -me -with -external -resources

If you want to get into a lot of detail here, go check out the manual at chocolatey.org.

One thing that is currently missing (afaik) is a completely-offline Chocolatey installer.
You can always take the install.ps1 from chocolatey.org and modify it to your needs – but when you look at chocolateysetup.psm1 you’ll see that even a offline copy of chocolatey.nupkg could try to download a .Net Framework from a Microsoft mirror.

To overcome this rather annoying circumstance, I’ve decided to modify the base install.ps1 a little further to install a .Net Framework, if missing before installing Chocolatey – so no public internet connection is needed at any point.
I’ve put my efforts down into a script called corporate_install.ps1.
The script can be parametrized, so you should be able to fit it into your organization by just passing the right parameters!

You’ll also notice I’ve pulled Boxstarter, a great project adding reboot-resiliency (among many other cool features), into my base setup – this is because I use it all the time!

MIND! The first thing you’ll want to do when using chocolatey in a corporate environment is to remove the public community repository. (this is automatically done in my ‘corporate_install.ps1’ ;-))

choco source remove --name chocolatey

Artifact Server

Currently I’m using ProGet as artifact server, for solely two reasons:
1. It’s extremely easy to set-up and use
2. It’s got a feature called ‘package cache’

The package-cache feature pulls the wanted package from the HQ to the local package cache when first requested.
With this feature we set-up ‘location-based’ servers for our remote-offices. Now all employees are pulling packages from their ‘closest’ server – so no extra bandwith to HQ is needed.

Configuration Deployment & Software Licensing

This is probably the most ‘tricky’ bit from the currently existing setup and therefore will get an extra blogpost asap. Just note: we’ve created a simple ‘configuration service’ containing information about ‘software’ and ‘licensees’ (‘software’ matches the package-id and a ‘licensee’ currently is either a hostname or Domain\Username). This information can be pulled from the chocolatey*.ps1 scripts via inline C# … … …

Multiple Chocolatey Package Source Repositories

If something turns out to be working great, you’ll want it to scale with your needs. In order to be able to scale ‘company-wide’ it seemed to be a good idea to split-up our Chocolatey packages into multiple package repositories for the following reasons:

  • packages could be grouped logically
  • (office packages, developer packages, testing packages, …)

  • package visibility could be controlled by the activated package source repositories
  • (office hosts don’t even get the chance to install i.e. 20GB of VisualStudio)

  • favor package versions based on different feed-priorities for different teams/ user groups
  • (team ‘X’ is on a ‘slow’-feed that only contains VS2015 update I, not II because a certain VC compiler settings would be unwanted for Product X, team ‘Y’ is on a ‘fast’-feed with all the VisualStudio versions available)

We’ve came up with a ‘hierarchy’ that can be described as follows:

(technology)-[(department|team)*]-[(project)*]

I will devote a whole blogpost to this, so stay tuned 🙂

Chocolatey Support

What should I say? I’ve had a couple of talks with Rob Reynolds, Gary Evan Park, Rich, Kim, Pascal,… (yup, basically the whole Chocolatey team). If you need help or have any question – just get in contact via gitter, they’ll be happy to help.
I will also be hanging around there sometimes 😉

Grazer Linuxtage 2017

Just like almost every year, my friend Martin and I attended the “Grazer Linuxtage”, a free conference that annually takes place at FH Joanneum Graz. (#glt17)

We’ve more or less set-up the track for single board computers, consisting of a workshop on Friday, as well as an info-stand and a talk on Saturday.

Thanks to the 200+ people that visited our talk, hope you enjoyed it!

 

Our slides are available for download, feel free to contact us if you’ve got any questions concerning the discussed topics! [talk at Linuxtage website]

I’m building a PhotoBox

Finally I put up some time and started building a “PhotoBox” / “PhotoBooth”. Many thanks to my brother-in-law who’s skilled enough in handcraft to make this project possible without falling apart right away 🙂

The main goal of this project is to come up with an actual use case for one of my old Nikon D2x-cameras, that otherwise wouldn’t leave the shelf at all. Everything is build around the amazing SLR Booth android app – I’ve added a little Raspberry Pi spice (people using the booth should be able to download their pictures to their mobile phones for ~ 5 minutes after they’ve taken a picture).

Once I’m done with it and everything works as I expect it to, I’ll be sharing building instructions and the source-code for the RPi extension 🙂

What is the PhotoBox made of?

  • Wood
  • Screws
  • Metal Coupler Pieces
  • Nikon D2x
  • Pixel C
  • Raspberry Pi
  • Canon Selphy Printer
  • Cables

Why Working On Windows Is Fun Again

Hint: it’s because of PowerShell and Chocolatey! 🙂

I’ve just needed to setup IntelliJ with Scala-Plugin and JDK on a new machine:

choco install jdk8 -y
Chocolatey v0.10.3 Business
Installing the following packages:
jdk8 v8.0.121 [Approved]
Chocolatey installed 1/1 packages. 0 packages failed.
choco install intellijidea-community -y
Installing the following packages:
intellijidea-community v2017.1 [Approved]
Chocolatey installed 1/1 packages. 0 packages failed.

Merging XML with XSLT and PowerShell? – OK!

Combining XMl files from PowerShell – well, that’s pretty easy once you figured out how to work with , but doing a correct & automatic merge turns out to be a quite challenging task.

Luckily there’s this: merge.xslt by (LGPL) by Oliver Becker – a XSL transformation ready accomplish this task in no time!

Let’s assume we’ve got two XML files.

FileA.xml:

<?xml version="1.0" encoding="utf-8"?>
<dict>
	<awesome id="21">Saxon</awesome>
	<awesome id="42">Chocolatey</awesome>
	
	<someweirdtag>
		I want candy.
	</someweirdtag>
</dict>
FileB.xml:

<?xml version="1.0" encoding="utf-8"?>
<dict>
	<awesome id="21">Coffee</awesome>
	<awesome id="42">Chocolatey</awesome>
	
	<someweirdtag>
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

What we want to accomplish is a merge of these two files – and there are several different possible outcomes:
If we just combine the two files, we’d want a result like this:

<?xml version="1.0" encoding="utf-8"?><dict>
	<awesome id="21">SaxonCoffee</awesome>
	<awesome id="42">Chocolatey</awesome>
	
	<someweirdtag>
		I want candy.
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

In the scenario I’ve been facing I needed to combine two files, favoring the second one – so all existing elements from FileA would be overridden by the elements of FileB.

<?xml version="1.0" encoding="utf-8"?><dict>
	<awesome id="21">Coffee</awesome>
	<awesome id="42">Chocolatey</awesome>
	
	<someweirdtag>
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

Adding the missing PowerShell script:

param(
[Parameter(Mandatory = $True)][string]$file1,
[Parameter(Mandatory = $True)][string]$file2,
[Parameter(Mandatory = $True)][string]$path
)

# using only abs paths .. just to be safe
$file1 = Join-Path $(Get-Location) $file1
$file2 = Join-Path $(Get-Location) $file2
$path = Join-Path $(Get-Location) $path

# awesome xsl stylesheet from Oliver Becker
# http://web.archive.org/web/20160502194427/http://www2.informatik.hu-berlin.de/~obecker/XSLT/merge/merge.xslt
$xsltfile = Join-Path $(Get-Location) "merge.xslt"

$XsltSettings = New-Object System.Xml.Xsl.XsltSettings
$XsltSettings.EnableDocumentFunction = 1

$xslt = New-Object System.Xml.Xsl.XslCompiledTransform;
$xslt.Load($xsltfile , $XsltSettings, $(New-Object System.Xml.XmlUrlResolver))

[System.Xml.Xsl.XsltArgumentList]$args = [System.Xml.Xsl.XsltArgumentList]::new()
$args.AddParam("with", "", $file2)
$args.AddParam("replace", "", "true")

[System.Xml.XmlWriter]$xmlwriter = [System.Xml.XmlWriter]::Create($path)
$xslt.Transform($file1, $args, $xmlwriter)

The eagle-eyed viewer spotted a caveat: yes, this does not run on Linux, there’s no

System.Xml.Xsl

in DotNetCore/PowerShell so far, but hopefully this will change! In the meantime, just swap the .NETish XSL code with our all-time-favorite Saxon!

java -jar saxon9he.jar .\FileA.xml .\merge.xslt with=FileB.xml replace=true

~ happy hacking!

Links:

Longterm Internet Connection Speedtest In Linux … With PowerShell ??

Something weird just happened: I’ve actually created the first PowerShell script that now regularly runs on my main Linux machine … the start of something new? …

I’ve been using PowerShell, well, A LOT at work lately – and the syntax is just so easy and straight forward – you gotta love it, even as a dedicated Linux user 😉

Script Speedtest.ps1

$outfile = "speedtest.csv"

[regex]$regexDl="Download\:\s(?'speed'\d+\.\d+)\s"
[regex]$regexUl="Upload\:\s(?'speed'\d+\.\d+)\s"

"Date;Time;Download;Upload;" | Out-File $outfile -Encoding utf8

while (1 -eq 1) {
    Write-Host -ForegroundColor Green "doing speedtest..."
    $meas = $(speedtest)
    $meas
    $download = $regexDl.Match($meas).Groups[1].Value.Replace(".",",")
    $upload = $regexUl.Match($meas).Groups[1].Value.Replace(".",",")

    "$(Get-Date -Format "yyyy-mm-dd;HH:mm:ss");$download;$upload;" | Out-File $outfile -Encoding utf8 -Append
    Start-Sleep -Seconds 300
}

The results will be shared in a couple of days when I’ve collected a significant amount of data!

Update: first results – upload speed pretty constant, download unstable!

Raspberry Pi Zero Wireless

So that happened … Can’t say that I was looking forward to a replacement of the “old” Rapsberry Pi Zero, as it really was a neat device for what it cost, but hey, WiFi + Bluetooth will make it even a better IoT device! 

I’ve just placed an order for both, a(nother) old Pi Zero as well as the new version from https://www.kiwi-electronics.nl – as they seem to be the only place that  still has the old version in stock at a reasonable price. (go fast if you want one!)

My first projects with the “Pi Zero Wireless” will probably be home-automation related … maybe a little weather station or room-temperature regulation – something like that 🙂

If you’re interested in that area, my friend Martin has a great collection of sample projects and materials over at his site http://strohmayers.com – in addition to that, we’re going to do a workshop at Grazer Linuxtage where you can get hands-on RPi tinkering and programming!

my pre-holiday Schdraggel #1

I’ve just been looking on what to do and where to go this summer on the Azores … and it seems that 18 days on the islands won’t be enough to visit all the awesome places on this atlantic archipel! 🙁