Category: Programming

WinOps 2018 Recap

I’ve had the chance to re-visit WinOps London this year, meet a couple of old friends and make new ones – after all, a great event!  I really enjoyed all of the talks and workshops I attended, but following topics somehow really stood out for me. (I am honestly looking forward for the videos being available on YouTube, so I can watch those talks for second time!)

SRE with Steven Murawski and Trond Hindenes

Negating releases is not a solution to face site reliability issues / incidents.

Steven Murawski
If it’s scary. You are not doing it right. Build robust systems: TRUST but VERIFY.

Trond Hindenes
Systems Reliability Engineering” really did feel like a “hipster buzzword” to me (sorry) .. UNTIL I attended those two talks, one from Steven Murawski and one from Trond Hindenes. Steven really gave some important insights in the why, Trond into the how of their experience with SRE in 2017/2018 and they presented some incredible results and numbers (achieved uptime, basically).

Docker with Elton Stoneman

Tearing down monolithic applications does not imply we have to do code changes. Elton Stoneman
To be honest, I never really thought about the possibility of splitting a monilithic app into pieces without having to modify the existing code base. Although I think nobody will ever be on par with Elton in concern of “containerization speed / efficiency”, seeing how a DotNet 3.5 app is split up into a bunch of containers whilist no code changes are being carried out really is an experience. 

Chocolatey with Rob Reynolds

You can’t spell system automation without package management. Rob Reynolds
In the last two years I’ve been lucky enough  see a couple of Rob’s talks and watch how Chocolatey matured and grew both as a tool, and as a company. The fluffy boss unicorn and dancing coworkers in Robs presentations have just been beat by the CCM (Chocolatey Central Management) live demo this time. I can’t wait for this feature to get out of it’s beta phase.

PowerShell with Chris Gardner

Pester is your friend. Chris Gardner
Watching Chris talk about how you should be developing your PowerShell modules really reminded me of a couple of things I tend to forget when I’m down coding.
  • when you’ve got a bug, write testcase for that bug
  • AFTERWARDS, fix that bug!
  • create docs for each public, preferably also for each private function
  • make the docs available!

So that’s my writeup for WinOps LDN 2018. Despite revisiting some topics I kind of forgot about I was lucky enough to get some insights in how folks at other companies tackle problems I’m still facing in my daily work – got a couple of ideas for improvements too, so I’d say – a win 🙂

“Killing Golden Images” at ChocolateyFest

After my wife and I [visited Hawaii], we took a short detour to San Francisco, where I was given the opportunity to give a talk at ChocolateyFest.

After attending WinOps conf in London last year, I was really looking forward to this event, being able to exchange my thoughts on Windows infrastructure automation with experts and other folks working in this area. (Every single talk at ChocoFest was somehow related to the work I’ve been doing for the last two years or so.)

“Killing Golden Images In Your Infrastructure Setups”
Being my first international talk, I must admit that I’ve been quite nervous, and you can definitely see this in the first 3-5 minutes watching the recording of the talk.

I’ve prepared supporting material for the talk on my GitHub page (you can also grab my slides there)

I really enjoyed listening to awesome, informative talks at the event, the evening workshops and coffee-break-chats with Dan, Stefan, Paul, Gary, … and of course, Willi Wonka himself 🙂

Rob Reynolds as Willi Wonka after the event

Looking forward to meeting a lot of you folks at WinOps London 2018 in less than 3 weeks from now!

automatically delete files that exist in another folder

Instead of removing items from one Google Photos gallery I decided to add them to another folder – dang, a 190 picture selection lost.

As I needed to download the images anyway, I got the idea: just download both albums and throw in a little PowerShell – to “re-do” the removal of the unwanted images for me!

$filesToRemove = (gci .\folderB\).Name
gci .\folderA\ | % { if ($filesToRemove -contains $_) { Remove-Item $_.FullName } else { echo “nope” }}

And done!
Writing this picece of PowerShell plus this blogpost took less than 5 minutes – about half the time it’d need me to re-select and remove the images manually – thanks PowerShell!

PowerShell Logging Made Easy (thinking RAII)

Just a quick something I want to share :-)!
Make your life easier using a scope-based action approach when writing PowerShell.
The idea is simple: group tasks together in blocks that are passed into a function which logs start and end time of that block, as well as doing some logging and error reporting if something goes wrong during exection. Let’s elaborate:

$start = Get-Date
Do-Something
Do-SomethingElse
$end = Get-Date
$duration = $end-$start
Write-FancyLog $duration "Done something"

This code doesn’t look nice. If you need to measure execution time of different blocks, do logging based on results, catch and evaluate errors, it would be much cleaner to do something like this:

FancyLog -Description "Do Something" {
  Do-Something
  Do-SomethingElse
}

Yes, that looks much better and is easier to grasp. Without further detours, here’s a basic ‘logging monitor’ that nicely integrates into any PowerShell environment 🙂
function Monitor {
param(
  [Parameter(Mandatory = $True, ValueFromPipeline = $True)]
  [scriptblock]$scriptblk,
  [Parameter(Mandatory = $True)]
  [string]$Description,
  [Parameter(Mandatory = $False)]
  [scriptblock]$ResultReceiver,
  [Parameter(Mandatory = $False)]
  [switch]$CanFail
)
  $_start = Get-Date
  $_ex = $null
  $_res = 0
  try {
    & $scriptblk
    $res = $lastexitcode
  }
  catch {
    $_ex = $_
    $res = $lastexitcode
  }
  finally {
    $_resobj = @{
      Exception   = $_ex
      Result      = $_res
      Description = $Description
      Duration    = ($(Get-Date) - $_start)
        }
        if ($ResultReceiver) {
            $ResultReceiver.Invoke($_resobj)
        }
    if (-Not $CanFail -And $_ex) {
      throw $_ex
    }
  }
}
The following code breaks on the exception that is thrown, logs correctly and then rethrows the exception:

Monitor -Description 'This block will probably fail!' -ResultReceiver $function:PseudoLog {
    echo 'hello'
    throw 'foobar'
    echo 'cruel world'
}

Hope you enjoy this snippet, happy hacking!

Merging XML with XSLT and PowerShell? – OK!

Combining XMl files from PowerShell – well, that’s pretty easy once you figured out how to work with


, but doing a correct & automatic merge turns out to be a quite challenging task.

Luckily there’s this: merge.xslt by (LGPL) by Oliver Becker – a XSL transformation ready accomplish this task in no time!

Let’s assume we’ve got two XML files.

FileA.xml:

<?xml version="1.0" encoding="utf-8"?>
<dict>
	<awesome id="21">Saxon</awesome>
	<awesome id="42">Chocolatey</awesome>

	<someweirdtag>
		I want candy.
	</someweirdtag>
</dict>
FileB.xml:

<?xml version="1.0" encoding="utf-8"?>
<dict>
	<awesome id="21">Coffee</awesome>
	<awesome id="42">Chocolatey</awesome>

	<someweirdtag>
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

What we want to accomplish is a merge of these two files – and there are several different possible outcomes:
If we just combine the two files, we’d want a result like this:

<?xml version="1.0" encoding="utf-8"?><dict>
	<awesome id="21">SaxonCoffee</awesome>
	<awesome id="42">Chocolatey</awesome>

	<someweirdtag>
		I want candy.
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

In the scenario I’ve been facing I needed to combine two files, favoring the second one – so all existing elements from FileA would be overridden by the elements of FileB.

<?xml version="1.0" encoding="utf-8"?><dict>
	<awesome id="21">Coffee</awesome>
	<awesome id="42">Chocolatey</awesome>

	<someweirdtag>
		I'm afraid of catfish.
	</someweirdtag>
	<IhaveNoMemoryOfThisPlace/>
</dict>

Adding the missing PowerShell script:

param(
[Parameter(Mandatory = $True)][string]$file1,
[Parameter(Mandatory = $True)][string]$file2,
[Parameter(Mandatory = $True)][string]$path
)

# using only abs paths .. just to be safe
$file1 = Join-Path $(Get-Location) $file1
$file2 = Join-Path $(Get-Location) $file2
$path = Join-Path $(Get-Location) $path

# awesome xsl stylesheet from Oliver Becker
# http://web.archive.org/web/20160502194427/http://www2.informatik.hu-berlin.de/~obecker/XSLT/merge/merge.xslt
$xsltfile = Join-Path $(Get-Location) "merge.xslt"

$XsltSettings = New-Object System.Xml.Xsl.XsltSettings
$XsltSettings.EnableDocumentFunction = 1

$xslt = New-Object System.Xml.Xsl.XslCompiledTransform;
$xslt.Load($xsltfile , $XsltSettings, $(New-Object System.Xml.XmlUrlResolver))

[System.Xml.Xsl.XsltArgumentList]$al = [System.Xml.Xsl.XsltArgumentList]::new()
$al.AddParam("with", "", $file2)
$al.AddParam("replace", "", "true")

[System.Xml.XmlWriter]$xmlwriter = [System.Xml.XmlWriter]::Create($path)
$xslt.Transform($file1, $al, $xmlwriter)

The eagle-eyed viewer spotted a caveat: yes, this does not run on Linux, there’s no

System.Xml.Xsl

in DotNetCore/PowerShell so far, but hopefully this will change!
– this seems to no longer be true, thank’s Brian!

If you don’t have System.Xml.Xsl or PowerShell for any reason, just swap the .NETish XSL code with our all-time-favorite Saxon!

java -jar saxon9he.jar .\FileA.xml .\merge.xslt with=FileB.xml replace=true

~ happy hacking!

Links:

Update: replaced $args in Code because of PowerShell 5.1 (thanks https://outofmemoryexception.wordpress.com/2016/08/05/powershell-5-1-14393/ )

Longterm Internet Connection Speedtest In Linux … With PowerShell ??

Something weird just happened: I’ve actually created the first PowerShell script that now regularly runs on my main Linux machine … the start of something new? …

I’ve been using PowerShell, well, A LOT at work lately – and the syntax is just so easy and straight forward – you gotta love it, even as a dedicated Linux user 😉

Script Speedtest.ps1

$outfile = "speedtest.csv"

[regex]$regexDl="Download\:\s(?'speed'\d+\.\d+)\s"
[regex]$regexUl="Upload\:\s(?'speed'\d+\.\d+)\s"

"Date;Time;Download;Upload;" | Out-File $outfile -Encoding utf8

while (1 -eq 1) {
    Write-Host -ForegroundColor Green "doing speedtest..."
    $meas = $(speedtest)
    $meas
    $download = $regexDl.Match($meas).Groups[1].Value.Replace(".",",")
    $upload = $regexUl.Match($meas).Groups[1].Value.Replace(".",",")

    "$(Get-Date -Format "yyyy-mm-dd;HH:mm:ss");$download;$upload;" | Out-File $outfile -Encoding utf8 -Append
    Start-Sleep -Seconds 300
}

The results will be shared in a couple of days when I’ve collected a significant amount of data!

Update: first results – upload speed pretty constant, download unstable!

VisualStudioCode Is Awesome!

I often complained that Notepad++ was the only usable text editor on Windows.. until I came in touch with VisualStudioCode in summer last year. It rocks!

It comes with a ton of cool features and is easily extensible with plugins. Just take a look at how great git diffs are visualized. And its FREE.

 

Thanks Microsoft, you really did us a great favor with this awesome tool!

 

Links

Wrapping A PowerShell-Script Call In A Batchfile

Yet another opinion on this topic, YAY!

As I wasn’t satisfied with the most common solutions out there (first 5 google hits) – I’ve decided to create a .ps1->.bat wrapper on my own:

>> WrapPowerShellScript.bat on GitHub <<

Why did I create another version of the very same thing that should be well known already? – because I wanted to pass all arguments/parameters that are passed to the .bat-File to the wrapped PowerShell script:

Update: of course this also works with ‘BoundParameters’ 🙂

 

If you don’t like it – have a look at those alternatives:
* howtogeek – how-to-use-a-batch-file-to-make-powershell-scripts-easieto-run
* stackoverflow how-to-run-a-powershell-script-file-from-within-a-bat-file-and-pass-multiple-par