Category: Programming

PowerShell Logging Made Easy (thinking RAII)

Just a quick something I want to share :-)!
Make your life easier using a scope-based action approach when writing PowerShell.
The idea is simple: group tasks together in blocks that are passed into a function which logs start and end time of that block, as well as doing some logging and error reporting if something goes wrong during exection.

Let’s elaborate:

$start = Get-Date
Do-Something
Do-SomethingElse
$end = Get-Date
$duration = $end-$start
Write-FancyLog $duration "Done something"

This code doesn’t look nice. If you need to measure execution time of different blocks, do logging based on results, catch and evaluate errors, it would be much cleaner to do something like this:

FancyLog -Description "Do Something" {
  Do-Something
  Do-SomethingElse
}

Yes, that looks much better and is easier to grasp.

Without further detours, here’s a basic ‘logging monitor’ that nicely integrates into any PowerShell environment 🙂

function Monitor {
param(
  [Parameter(Mandatory = $True, ValueFromPipeline = $True)]
  [scriptblock]$scriptblk,
  [Parameter(Mandatory = $True)]
  [string]$Description,
  [Parameter(Mandatory = $False)]
  [scriptblock]$ResultReceiver,
  [Parameter(Mandatory = $False)]
  [switch]$CanFail
)
  $_start = Get-Date
  $_ex = $null
  $_res = 0
  try {
    & $scriptblk
    $res = $lastexitcode
  }
  catch {
    $_ex = $_
    $res = $lastexitcode
  }
  finally {
    $_resobj = @{
      Exception   = $_ex
      Result      = $_res
      Description = $Description
      Duration    = ($(Get-Date) - $_start)
		}
		if ($ResultReceiver) {
			$ResultReceiver.Invoke($_resobj) 
		}
    if (-Not $CanFail -And $_ex) {
      throw $_ex
    }
  }
}

Sample usage with dummy-logging:

function PseudoLog {
param(
  [Parameter(Mandatory = $True)]
  [PSCustomObject]$logobj
)
	Write-Host "> $($logobj.Description)"
	Write-Host $($logobj | ConvertTo-Json)
}

The following code continnues after reporting and errror:

Monitor -Description 'This block errs but continues!' -ResultReceiver $function:PseudoLog -CanFail {
	.\filesdoesnotexist
}

The following code breaks on the exception that is thrown, logs correctly and then rethrows the exception:

Monitor -Description 'This block will probably fail!' -ResultReceiver $function:PseudoLog {
	echo 'hello'
	throw 'foobar'
	echo 'cruel world'
}

Hope you enjoy this snippet, happy hacking!

Using Chocolatey In Corporate Environments (Part #1)

It’s been about a year now that I’ve started pulling in Chocolatey as a tool for software deployment for the development teams at my workplace.
As of today, the setup of development machines, buildservers as well as specific terminal-hosts have been fully transformed from manually following many-pages of wiki install instructions to a single ‘meta-package’ that takes the hosts from 0 to 100 in ‘no-time’ (really, just like an hour compared to a full work-day previously!). And yes, all machines receive their very-own configuration, licensing for different software packages etc. etc. hyped? so am I!

Even though this is pretty obvious, it doesn’t hurt to recollect:
Having the install-procedure as well as the configuration in code (PowerShell in this case) makes it repeatable and less error prone! In addition to that, you’ll want to put your code under version control, so you’ve got the history covered!

Package Separation

For the sake of convenience I’ve decided to split up the packages that actually install software and the configuration of this software into separate packages.

vs2010
vs2010-settings-teamX
vs2010-settings-teamY
...

Internalize Everything!

In a business scenario, you probably don’t want to rely on public package source repositories (for multiple reasons) – so downloading the wanted packages, doing a virus-check and putting the artifacts on an internal-only server is basically what you want 99% of the time.
This can be a challenging task when done manually, trust me – I’ve been doing this and trying to automate it for about a month, but wait – this task already IS fully automated via the ‘internalize’ feature!
This feature is only available in the ‘business’ licensing model of Chocolatey, if you’re planning on using Chocolatey for work – go get a trial and let the magic work for you!

It is literally as easy as this:

choco download --internalize $pkg --source=https://repo...
choco push $pkg*.nupkg --source=https://myorg-choco... --api-key=$apikey

If you’re keen on going a step further, I’ve put up some sample ‘internalize’ script that I use to fetch packages via Jenkins CI. [internalize.ps1 on GitHub]

Completely Offline Chocolatey Installation

.\corporate_install.ps1 -dont -bug -me -with -external -resources

If you want to get into a lot of detail here, go check out the manual at chocolatey.org.

One thing that is currently missing (afaik) is a completely-offline Chocolatey installer.
You can always take the install.ps1 from chocolatey.org and modify it to your needs – but when you look at chocolateysetup.psm1 you’ll see that even a offline copy of chocolatey.nupkg could try to download a .Net Framework from a Microsoft mirror.

To overcome this rather annoying circumstance, I’ve decided to modify the base install.ps1 a little further to install a .Net Framework, if missing before installing Chocolatey – so no public internet connection is needed at any point.
I’ve put my efforts down into a script called corporate_install.ps1.
The script can be parametrized, so you should be able to fit it into your organization by just passing the right parameters!

You’ll also notice I’ve pulled Boxstarter, a great project adding reboot-resiliency (among many other cool features), into my base setup – this is because I use it all the time!

MIND! The first thing you’ll want to do when using chocolatey in a corporate environment is to remove the public community repository. (this is automatically done in my ‘corporate_install.ps1’ ;-))

choco source remove --name chocolatey

Artifact Server

Currently I’m using ProGet as artifact server, for solely two reasons:
1. It’s extremely easy to set-up and use
2. It’s got a feature called ‘package cache’

The package-cache feature pulls the wanted package from the HQ to the local package cache when first requested.
With this feature we set-up ‘location-based’ servers for our remote-offices. Now all employees are pulling packages from their ‘closest’ server – so no extra bandwith to HQ is needed.

Configuration Deployment & Software Licensing

This is probably the most ‘tricky’ bit from the currently existing setup and therefore will get an extra blogpost asap. Just note: we’ve created a simple ‘configuration service’ containing information about ‘software’ and ‘licensees’ (‘software’ matches the package-id and a ‘licensee’ currently is either a hostname or Domain\Username). This information can be pulled from the chocolatey*.ps1 scripts via inline C# … … …

Multiple Chocolatey Package Source Repositories

If something turns out to be working great, you’ll want it to scale with your needs. In order to be able to scale ‘company-wide’ it seemed to be a good idea to split-up our Chocolatey packages into multiple package repositories for the following reasons:

  • packages could be grouped logically
  • (office packages, developer packages, testing packages, …)

  • package visibility could be controlled by the activated package source repositories
  • (office hosts don’t even get the chance to install i.e. 20GB of VisualStudio)

  • favor package versions based on different feed-priorities for different teams/ user groups
  • (team ‘X’ is on a ‘slow’-feed that only contains VS2015 update I, not II because a certain VC compiler settings would be unwanted for Product X, team ‘Y’ is on a ‘fast’-feed with all the VisualStudio versions available)

We’ve came up with a ‘hierarchy’ that can be described as follows:

(technology)-[(department|team)*]-[(project)*]

I will devote a whole blogpost to this, so stay tuned 🙂

Chocolatey Support

What should I say? I’ve had a couple of talks with Rob Reynolds, Gary Evan Park, Rich, Kim, Pascal,… (yup, basically the whole Chocolatey team). If you need help or have any question – just get in contact via gitter, they’ll be happy to help.
I will also be hanging around there sometimes 😉

Why Working On Windows Is Fun Again

Hint: it’s because of PowerShell and Chocolatey! 🙂

I’ve just needed to setup IntelliJ with Scala-Plugin and JDK on a new machine:

choco install jdk8 -y
Chocolatey v0.10.3 Business
Installing the following packages:
jdk8 v8.0.121 [Approved]
Chocolatey installed 1/1 packages. 0 packages failed.
choco install intellijidea-community -y
Installing the following packages:
intellijidea-community v2017.1 [Approved]
Chocolatey installed 1/1 packages. 0 packages failed.

Merging XML with XSLT and PowerShell? – OK!

Combining XMl files from PowerShell – well, that’s pretty easy once you figured out how to work with , but doing a correct & automatic merge turns out to be a quite challenging task.

Luckily there’s this: merge.xslt by (LGPL) by Oliver Becker – a XSL transformation ready accomplish this task in no time!

Let’s assume we’ve got two XML files.

FileA.xml:

<?xml version="1.0" encoding="utf-8"?>
<dict>
	<awesome id="21">Saxon</awesome>
	<awesome id="42">Chocolatey</awesome>
	
	<someweirdtag>
		I want candy.
	</someweirdtag>
</dict>
FileB.xml:

<?xml version="1.0" encoding="utf-8"?>
<dict>
	<awesome id="21">Coffee</awesome>
	<awesome id="42">Chocolatey</awesome>
	
	<someweirdtag>
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

What we want to accomplish is a merge of these two files – and there are several different possible outcomes:
If we just combine the two files, we’d want a result like this:

<?xml version="1.0" encoding="utf-8"?><dict>
	<awesome id="21">SaxonCoffee</awesome>
	<awesome id="42">Chocolatey</awesome>
	
	<someweirdtag>
		I want candy.
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

In the scenario I’ve been facing I needed to combine two files, favoring the second one – so all existing elements from FileA would be overridden by the elements of FileB.

<?xml version="1.0" encoding="utf-8"?><dict>
	<awesome id="21">Coffee</awesome>
	<awesome id="42">Chocolatey</awesome>
	
	<someweirdtag>
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

Adding the missing PowerShell script:

param(
[Parameter(Mandatory = $True)][string]$file1,
[Parameter(Mandatory = $True)][string]$file2,
[Parameter(Mandatory = $True)][string]$path
)

# using only abs paths .. just to be safe
$file1 = Join-Path $(Get-Location) $file1
$file2 = Join-Path $(Get-Location) $file2
$path = Join-Path $(Get-Location) $path

# awesome xsl stylesheet from Oliver Becker
# http://web.archive.org/web/20160502194427/http://www2.informatik.hu-berlin.de/~obecker/XSLT/merge/merge.xslt
$xsltfile = Join-Path $(Get-Location) "merge.xslt"

$XsltSettings = New-Object System.Xml.Xsl.XsltSettings
$XsltSettings.EnableDocumentFunction = 1

$xslt = New-Object System.Xml.Xsl.XslCompiledTransform;
$xslt.Load($xsltfile , $XsltSettings, $(New-Object System.Xml.XmlUrlResolver))

[System.Xml.Xsl.XsltArgumentList]$args = [System.Xml.Xsl.XsltArgumentList]::new()
$args.AddParam("with", "", $file2)
$args.AddParam("replace", "", "true")

[System.Xml.XmlWriter]$xmlwriter = [System.Xml.XmlWriter]::Create($path)
$xslt.Transform($file1, $args, $xmlwriter)

The eagle-eyed viewer spotted a caveat: yes, this does not run on Linux, there’s no

System.Xml.Xsl

in DotNetCore/PowerShell so far, but hopefully this will change! In the meantime, just swap the .NETish XSL code with our all-time-favorite Saxon!

java -jar saxon9he.jar .\FileA.xml .\merge.xslt with=FileB.xml replace=true

~ happy hacking!

Links:

Longterm Internet Connection Speedtest In Linux … With PowerShell ??

Something weird just happened: I’ve actually created the first PowerShell script that now regularly runs on my main Linux machine … the start of something new? …

I’ve been using PowerShell, well, A LOT at work lately – and the syntax is just so easy and straight forward – you gotta love it, even as a dedicated Linux user 😉

Script Speedtest.ps1

$outfile = "speedtest.csv"

[regex]$regexDl="Download\:\s(?'speed'\d+\.\d+)\s"
[regex]$regexUl="Upload\:\s(?'speed'\d+\.\d+)\s"

"Date;Time;Download;Upload;" | Out-File $outfile -Encoding utf8

while (1 -eq 1) {
    Write-Host -ForegroundColor Green "doing speedtest..."
    $meas = $(speedtest)
    $meas
    $download = $regexDl.Match($meas).Groups[1].Value.Replace(".",",")
    $upload = $regexUl.Match($meas).Groups[1].Value.Replace(".",",")

    "$(Get-Date -Format "yyyy-mm-dd;HH:mm:ss");$download;$upload;" | Out-File $outfile -Encoding utf8 -Append
    Start-Sleep -Seconds 300
}

The results will be shared in a couple of days when I’ve collected a significant amount of data!

Update: first results – upload speed pretty constant, download unstable!

Automating Software Automation

Yet another Chocolatey/PowerShell post!
Sorry, I rarely have time to put my thoughts to words – so I take any chance – and lately I’ve been thinking about software automation on Windows a lot! The possibilities seem quite overwhelming, and I like this alot! 😉

Continue reading “Automating Software Automation”

VisualStudioCode Is Awesome!

I often complained that Notepad++ was the only usable text editor on Windows.. until I came in touch with VisualStudioCode in summer last year. It rocks!

It comes with a ton of cool features and is easily extensible with plugins. Just take a look at how great git diffs are visualized. And its FREE.

 

Thanks Microsoft, you really did us a great favor with this awesome tool!

 

Links

It’s All About The Choco(latey?)

Things chocolate is good for:

  • eating chocolate – the darker the better
  • chocolate fondue
  • chocolate brownies
  • triple choco cookies
  • installing software

> installing software . . . (WAIT WHAT ?!?!?!)

The last one isn’ quite right – but, no matter how sweet chocolate is, chocolatey might be even more delicious!

Ok, clarification: Chocolatey (short: “choco”) is a package manger for Windows that actually works really well, has loads of packages in the public repo – and you can create packages yourself pretty easily!

– if you are not hyped right now – you may as well leave 🙂 –

Continue reading “It’s All About The Choco(latey?)”