Progress On ‘Invoke-Remote’

Yup, it’s been a while since I had time to work on my little side-project “Invoke-Remote” – it’s a small collection of scripts that aims to enhance my everyday workflow with remote Windows environments.
Today I finally managed to add a missing piece: Get-RemoteFileContent.ps1 (ir-get for the lazy)
This script utilizes IO.FileSystemWatcher in order to make it possible to WAIT until a file is being created on a remote host! 🙂
(comes in handy when waiting for a certain log, lock or sync file)

param($folder, $file)
$fullpath = Join-Path $folder $file
if (-Not (Test-Path $fullpath)) {
  $env:RF_FILE_CREATED_INDICATOR = $false
  $fsw = New-Object IO.FileSystemWatcher $folder, $file -Property @{IncludeSubdirectories = $false; NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'};
  $j = Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action {
  $env:RF_FILE_CREATED_INDICATOR = $true
};
while ($env:RF_FILE_CREATED_INDICATOR -eq $false) {
  Start-Sleep -Milliseconds 1000;
};
  $j = $fsw.Dispose();
  $j = Unregister-Event FileCreated
}
Get-Content $fullpath

Links:

A Quick Trip To Innervillgraten

From Wikipedia: “Innervillgraten is a municipality in the district of Lienz in the Austrian state of Tyrol. The region is one of the most remote regions in Austria.” (https://en.wikipedia.org/wiki/Innervillgraten)

The thing is: it’s not only one of the most remote, it’s also one of the most beautiful and original places Austria has to offer 🙂

find it on Google maps: https://www.google.com/maps/place/Innervillgraten

Innervillgraten’s website: http://www.innervillgraten.at/

Today I Learned … PowerShell | Elevation | Credential

I’ve been working on this far longer than I’m proud of, but this is it:
IT IS POSSIBLE to execute a powershell scriptblock from a non-admin session that does elevation and also makes use of user credentials 🙂

Elevation (the easy part)

param(
    [Parameter(Mandatory = $True, ValueFromPipeline = $True)]
    [ScriptBlock]$scriptBlock
)

$scriptBlockString = $scriptBlock.ToString()
$captureElevatedScriptBlock = @"
try {
    `$ErrorActionPreference = 'Stop'
    & { $scriptBlockString }
} catch {
    Write-Output `$_
    if (-Not `$lastexitcode) {
        `$lastexitcode = 1
    }
} finally {
    if (-Not `$lastexitcode) {
        `$lastexitcode = 0
    }
    exit `$lastexitcode
}
"@

$myWd = $(Get-Location).Path
$startElevatedProcessArgs = @{
    FilePath     = "PowerShell.exe"
    PassThru     = $true
    WorkingDir   = "$myWd"
    Verb         = "runas"
    ArgumentList = @("-Command", "& { Set-Location '$myWd'; $captureElevatedScriptBlock }")
}

$p = Start-Process @startElevatedProcessArgs -Wait
$p.WaitForExit()
$res = @{
    ExitCode = $p.ExitCode
}

This snippet just runs arbitrary commands (in a script-block) with elevation (if the current user is eligible for elevation). But wait, there’s no ouput of the started process, also output redirect doesn’t seem to be working when coming from a non-elevated session…

Using ‘Tee-Object’ to obtain the output of an elevated process

...
$myRandom = $([System.Guid]::NewGuid())
$myTmpPath = $([System.IO.Path]::GetTempPath())
$elevatedTextOutputFile = $(Join-Path $myTmpPath "$myRandom.elevated.log")
 
$captureElevatedScriptBlock = @"
try {
    `$ErrorActionPreference = 'Stop'
    & { $scriptBlockString } | Tee-Object $elevatedTextOutputFile
}
...
$res = @{
    ExitCode = $p.ExitCode
    Output   = Get-Content $elevatedTextOutputFile
}
Remove-Item $elevatedTextOutputFile

This already looks promising 🙂 – we now can run any command with elevation and also get it’s exit code (partially) and console output!
So what’s missing? – In my use case I’m always running certain scripts as a non-administrative user, and I need to to use [PSCredentials] (which are known) and then also do elevation.
You can’t use both “-verb runas” and “-Credential” in one call (or at least I didn’t manage to in a couple of hours!” – so I’ve come up with this solution:

We’ll make use of two wrappers:
1. already known – get output of elevated process
2. new: start the first wrapper from a new powershell.exe process that is started with “-Credentials”

Running a scriptblock with credentials

$startWithCredentialsArgs = @{
    FilePath               = "PowerShell.exe"
    PassThru               = $true
    WorkingDir             = $myWd
    ArgumentList           = @("-ExecutionPolicy bypass", "-noprofile", "-noninteractive", "-Command", "& { Set-Location '$myWd'; $someScriptBlock }")
    Credential             = $Credential
    RedirectStandardOutput = $wrapperScriptLog
}

$p = Start-Process @startWithCredentialsArgs -Wait
$p.WaitForExit()

Putting it all together

param(
    [Parameter(Mandatory = $True, ValueFromPipeline = $True)]
    [ScriptBlock]$scriptBlock,
    
    [Parameter(Mandatory = $False)]
    [pscredential]$Credential
)

$myRandom = $([System.Guid]::NewGuid())
$myTmpPath = $([System.IO.Path]::GetTempPath())
$elevatedTextOutputFile = $(Join-Path $myTmpPath "$myRandom.elevated.log")
  
$scriptBlockString = $scriptBlock.ToString()
$captureElevatedScriptBlock = @"
try {
    `$ErrorActionPreference = 'Stop'
    & { $scriptBlockString } | Tee-Object $elevatedTextOutputFile
} catch {
    Write-Output `$_
    if (-Not `$lastexitcode) {
        `$lastexitcode = 1
    }
} finally {
    if (-Not `$lastexitcode) {
        `$lastexitcode = 0
    }
    exit `$lastexitcode
}
"@

$myWd = $(Get-Location).Path
$startElevatedProcessArgs = @{
    FilePath     = "PowerShell.exe"
    PassThru     = $true
    WorkingDir   = "$myWd"
    Verb         = "runas"
    ArgumentList = @("-Command", "& { Set-Location '$myWd'; $captureElevatedScriptBlock }")
}

if (-Not $Credential) {
    # easy: run with the current user context
    $p = Start-Process @startElevatedProcessArgs -Wait
    $p.WaitForExit()
    $res = @{
        ExitCode = $p.ExitCode
        Output   = Get-Content $elevatedTextOutputFile
    }
}
else {
    # tricky: need to switch to a new session with given credentials ...
    # this session then needs to run the above code actually calling the elevated session
    $wrapperScript = Join-Path $myTmpPath "$myRandom.magic.ps1"
    $wrapperScriptLog = "$wrapperScript.log"

    @"
    `$startElevatedProcessArgs = @{
        FilePath     = `"PowerShell.exe"
        PassThru     = `$true
        WorkingDir   = '$myWd'
        Verb         = "runas"
        ArgumentList = @(`"-noexit`",`"-Command`", `"& { Set-Location '$myWd'; $($captureElevatedScriptBlock.Replace("`$","```$")) }")
    }
    `$p = Start-Process `@startElevatedProcessArgs -Wait
    `$p.WaitForExit()
    exit `$p.ExitCode
"@ | Out-File $wrapperScript -Encoding default

    $startWithCredentialsArgs = @{
        FilePath               = "PowerShell.exe"
        PassThru               = $true
        WorkingDir             = $myWd
        ArgumentList           = @("-ExecutionPolicy bypass", "-noprofile", "-noninteractive", "-Command", "& { Set-Location '$myWd'; $wrapperScript }")
        Credential             = $Credential
        RedirectStandardOutput = $wrapperScriptLog
    }

    $p = Start-Process @startWithCredentialsArgs -Wait
    $p.WaitForExit()
    Remove-Item $wrapperScript

    $res = @{
        ExitCode      = $p.ExitCode
        Output        = Get-Content $elevatedTextOutputFile
        WrapperOutput = Get-Content $wrapperScriptLog
    }
    Remove-Item $wrapperScriptLog
}

Remove-Item $elevatedTextOutputFile
$res

Of course, this source code is also available on GitHub!

very, very simple PowerShell performance monitoring + ElasitcSearch

Some time ago I’ve promised a friend to post a little “how-to” concerning PowerShell and ElasticSearch.
So here we go: the following code-snipped can be split into two parts: the “elasticsearch api” and a “naive performance monitor” 🙂

Please note that as always, I write my PoSh-Scripts with PowerShell 5 or 5.1 especially in this case I’m using class-objects, so this won’t run on anything below version 5!

param(
    [string]$es_uri = "http://my-elastic-uri:9200",
    [string]$es_perf_index = "perfdata",
    [string]$es_perf_mapping = "base",
    [int]$SecondsDelay = 10,
    [switch]$Verbose
)

$script:beVerbose = $Verbose

##########################################################################################
## basic elasticsearch powershell api
##########################################################################################
class Elastic {

    hidden [string]$es_uri

    hidden [PSCustomObject] _JSONIFY($obj) {
        return ($obj | ConvertTo-Json -Depth 10)
    }
    
    Elastic($ElasitcServerURI) {
        $this.es_uri = $ElasitcServerURI
    }

    [PSCustomObject] __call ($verb, $params, $body) {
        $uri = $this.es_uri
        if ($script:beVerbose) {
            Write-Host "`nCalling [$uri/$params]" -f Green
            if ($body) {
                if ($body) {
                    Write-Host "BODY`n--------------------------------------------`n$body`n--------------------------------------------`n" -f Yellow
                }
            }
        }
        $response = Invoke-WebRequest -Uri "$uri/$params" -method $verb -ContentType 'application/json' -Body $body
        return $response.Content
    }
  
    [PSCustomObject] _get ($params) {
        $params += "?format=json"    
        return ($this.__call("Get", $params, $null) | ConvertFrom-JSon)
    }
    [PSCustomObject] _delete ($params) {
        return $this.__call("Delete", $params, $null)
    }
    [PSCustomObject] _put ($params, $obj) {
        $obj = $this._JSONIFY($obj)
        return $this.__call("Put", $params, $obj)
    }
    [PSCustomObject] _post ($params, $obj) {
        $obj = $this._JSONIFY($obj)
        return $this.__call("Post", $params, $obj)
    }
    [PSCustomObject] catalogs() {
        return $this._get("_cat/indices")
    }
}

##########################################################################################
## naive performance monitor
##########################################################################################
function Get-CPULoad {
    return $(Get-WmiObject win32_processor | Measure-Object -property LoadPercentage -Average | Select-Object Average)
}

function Get-MEMLoad {
    $os = Get-WmiObject Win32_OperatingSystem
    return $([math]::Round(100 - ($os.FreePhysicalMemory / $os.TotalVisibleMemorySize) * 100, 2))
} 

while ($true) {
    $entry = @{
        "Time"    = $(Get-Date);
        "CPULoad" = $(Get-CPULoad);
        "MEMLoad" = $(Get-MEMLoad);
    }
    
    [Elastic]::new($es_uri)._post("$es_perf_index/$es_perf_mapping", $entry) > $null
    if ($script:beVerbose) {
        Write-Host "... going to sleep for $SecondsDelay seconds!"
    }
    Start-Sleep -Seconds $SecondsDelay
}

Of course you can also find my spin at ElasticSearch->PowerShell integration at GitHub
happy hacking!
– if you’ve got any questions concerning this sample, feel free to reach out to me any time!

A (Winter) Break In Scotland & New Year’s In Stonehaven

Scotland 2017 - Sand And Tide

In December 2017 my girlfriend and I decided to take a short-trip to Scotland. We earned a couple of high eyebrows and weird looks for this plan 🙂

Scotland in winter…
– “you’ll be happy if it’s just raining!”
– “It’ll sure be freezing cold all day!”
– “you won’t get much sun up there!”
– “you’ll stay bored in the hotel.”

Luckily, all of that didn’t turn out to be true 🙂
+ It’s been snowing half of the time.
+ Temperatures were between -2 and +7 °C
+ The sun rose shortly after 9am and went down at approx. 3pm
+ We’ve been out all day.

Continue reading “A (Winter) Break In Scotland & New Year’s In Stonehaven”

Getting Started With Chocolatey 4 Business & Jenkins CI

I’ve had an argument with a colleague quite recently about the usage of open-source automation tools, especially Chocolatey in business environments. A keypoint of this argument was the integration of new/open tools in long-existing, mostly commercial software based workflows.

One of the main reasons to use Chocolatey in corporate environments is its ability to integrate seamlessly with already existent automation infrastructure. I’ve been able to integrate Chocolatey successfully with Jenkins CI, Finalbuilder, corporate internal windows services as well as simple ‘installer-batch-files’, a friend and coworker (flurl) is deploying a corporate internal management solution with Chocolatey.

The day I replaced the multi-100 line content of a legacy batch installer with a single line of

choco install {package} -y

was a really good day 🙂

In this blogpost I’m going to give you an idea of how you could get started with Chocolatey at your organization concerning two parts: a possible way to roll-out Chocolatey to your coworkers and a small collection of things to get started integrating Chocolatey with a CI-server (Jenkins, but probably applicable to anything else that does support PowerShell).

Deploying Chocolatey At Your Organization

You’ll want to make it as easy as possible, hence putting a .bat-file on a network share that’s accessible from everywhere in your org is probably a good idea.
If you’re just getting started with Choco I’d also suggest putting your first packages on such a share.
Let’s assume following share-layout:

  • \\your-server\Choco\install
  • \\your-server\Choco\packages

Go to https://chocolatey.org/packages and fetch the latest Chocolatey.nupkg file. Put that file together with this install.ps1 and this install.bat onto the Choco/install share.

You need to adapt the URLs and UNC-paths to the scripts inside the files to make them work in your environment.
Now you’re good to go, just double-click on install.bat from any machine and Chocolatey will be ready to use.
Notice that the community repository is removed during the installation progress and another (internal) repository is set up. For details on how this is done, check the contents of install.ps1.

Your First Packages

Doing a

choco list
now looks pretty boring, because there are no packages at your feed yet (the community repository has already been removed for corporate usage), so go ahead and download some .nupkg-Files from https://chocolatey.org/packages and put them into your  prepared network share.

Now you should be able to list the available packages with ‘choco list’, search for packages with ‘choco search’ and install packages with ‘choco install’.

Of course you can also create your own packages, just type

choco new {package name}
and see what happens.

So far, you’ve only been using OSS features, and you can use all this without limitation in a corporate environment, but there are some things you may need to consider: the packages you’ve pulled in from https://chocolatey.org probably rely on downloading external resources like .exe, .MSI or .zip files. You don’t want to do that in an organization, as you’ll want to have complete reliability in your solution, and downloading from the internet could have reliability issues (not to mention trust). There’s two options: manually download the required resources, changing the source code of the packages install.ps1 and repackage every single one of the required packages, or you could simple use package internalizer.

Package Internalizer

The best of the pack. If you’re using Chocolatey in a corporate environment, you’ll really dig this. Just type

choco download {packagename} --internalize
and let the magic happen. This will automatically download all needed assets and put them into the generated package, so you’ll achieve the maximum availability.

Just keep in mind that you’ll have to license all hosts that are consumers of packages you manage via internalization. (Chocolatey for Business starts at just $600 USD for up to 35 machines, and is $16/machine/year with volume discounts thereafter – see https://chocolatey.org/pricing)

Choco + Jenkins

Let’s automate the automation!

To be able to use the following jobs you’ll need to know the basics of Jenkins job configurations. You’ll need to set-up parameterized jobs with the PowerShell plugin, also Chocolatey should already be installed on the corresponding Jenkins nodes.

This job just updates all the Chocolatey packages on the server.

choco upgrade all -y

Yes, it’s as easy as that 🙂
This simple Jenkins-job allows you to internalize any package from the community-Feed with the click of a button:

# PowerShell script to internalize chocolatey packages from the community feed to an internal server using the business edition 'internalize' feature. This script is designed to be run from jenkins, P_* variables are defined in jenkins job!
# section CREDS
	$pkgs = $env:P_PKGLIST
	$uncshare = $env:P_UNC_SHARE
    $targetfolder = $env:P_DST_FOLDER
	$targetserver = $env:P_DST_SRV
	$apikey = $env:P_API_KEY

	$envtmp = $env:temp
	$tmpdir = "$envtmp\chocointernalize"
	$basefeed = "https://chocolatey.org/api/v2/"
# endsection

function InternalizePkg($pkg) {
	Push-Location $tmpdir
	choco download --internalize $pkg --resources-location="$uncshare\$pkg" --source="$basefeed" --no-progress
	$genpkg = ((Get-ChildItem *.nupkg -recurse).FullName | Select-String -Pattern $pkg)
    if ($targetfolder) {
        Write-Output "> copying package '$genpkg' to '$targetfolder'"
        if (-Not (Test-Path $targetfolder)) {
            New-Item $targetfolder -ItemType Directory -Force -Verbose            
        }
        Copy-Item $genpkg $targetfolder -Verbose -ErrorAction "Stop"
    } else {
        Write-Output "> pushing package '$genpkg' to '$targetserver'"
        choco push $genpkg --source="$targetserver" --api-key="$apikey" -Verbose
    }
    Write-Output "------------------------------------------------------------------------"
    Write-Output ""
	Pop-Location
}

if ((Test-Path $tmpdir)) {
	Remove-Item $tmpdir -Recurse -Force -Verbose
}
New-Item $tmpdir -ItemType Directory -Force -Verbose

$pkgs | ForEach-Object {
	InternalizePkg $_
}

Remove-Item $tmpdir -Recurse -Force -Verbose

 

This job reports what packages are outdated. (A newer version of that package is available at the community feed)


$mrecipient = $env:P_MAIL_REC
$msender = $env:P_MAIL_SEND
$msmtp = $env:P_SMTP_SERVER
$availPkgs = @()

#$chocoout = $(choco outdated)
$chocoout | ForEach-Object {
    $up = $_.Split("|")
    if ($up -And ($up[3] -ne "false")) {
			if ($up[1] -eq $up[2]) {
				$availPkgs += "$($up[0]): $($up[1]) -> $($up[2])"
			}
		} 
	
}

$res = $availPkgs | Format-Table | Out-String
if ($res) {
	Send-MailMessage -To $mrecipient -Subject "Chocolatey Packages Outdated!" -Body $res -Verbose -ErrorAction "Stop" -From $msender -SmtpServer $msmtp
}

Mind that this setup and these scripts are really minimalistic. When you’re working in an environment with a couple of users you may want additional feeds to create some logical package separation or access control. There are unimaginably many use-cases on how to interop with a command-line utility such as Chocolatey, but I just wanted to give you a quick dive-in how my first contact with Chocolatey looked like. (or how I wished it looked like to be honest ;-)).

Happy hacking!

Links:

PowerShell Logging Made Easy (thinking RAII)

Just a quick something I want to share :-)!
Make your life easier using a scope-based action approach when writing PowerShell.
The idea is simple: group tasks together in blocks that are passed into a function which logs start and end time of that block, as well as doing some logging and error reporting if something goes wrong during exection.

Let’s elaborate:

$start = Get-Date
Do-Something
Do-SomethingElse
$end = Get-Date
$duration = $end-$start
Write-FancyLog $duration "Done something"

This code doesn’t look nice. If you need to measure execution time of different blocks, do logging based on results, catch and evaluate errors, it would be much cleaner to do something like this:

FancyLog -Description "Do Something" {
  Do-Something
  Do-SomethingElse
}

Yes, that looks much better and is easier to grasp.

Without further detours, here’s a basic ‘logging monitor’ that nicely integrates into any PowerShell environment 🙂

function Monitor {
param(
  [Parameter(Mandatory = $True, ValueFromPipeline = $True)]
  [scriptblock]$scriptblk,
  [Parameter(Mandatory = $True)]
  [string]$Description,
  [Parameter(Mandatory = $False)]
  [scriptblock]$ResultReceiver,
  [Parameter(Mandatory = $False)]
  [switch]$CanFail
)
  $_start = Get-Date
  $_ex = $null
  $_res = 0
  try {
    & $scriptblk
    $res = $lastexitcode
  }
  catch {
    $_ex = $_
    $res = $lastexitcode
  }
  finally {
    $_resobj = @{
      Exception   = $_ex
      Result      = $_res
      Description = $Description
      Duration    = ($(Get-Date) - $_start)
		}
		if ($ResultReceiver) {
			$ResultReceiver.Invoke($_resobj) 
		}
    if (-Not $CanFail -And $_ex) {
      throw $_ex
    }
  }
}

Sample usage with dummy-logging:

function PseudoLog {
param(
  [Parameter(Mandatory = $True)]
  [PSCustomObject]$logobj
)
	Write-Host "> $($logobj.Description)"
	Write-Host $($logobj | ConvertTo-Json)
}

The following code continnues after reporting and errror:

Monitor -Description 'This block errs but continues!' -ResultReceiver $function:PseudoLog -CanFail {
	.\filesdoesnotexist
}

The following code breaks on the exception that is thrown, logs correctly and then rethrows the exception:

Monitor -Description 'This block will probably fail!' -ResultReceiver $function:PseudoLog {
	echo 'hello'
	throw 'foobar'
	echo 'cruel world'
}

Hope you enjoy this snippet, happy hacking!

Autumn In Austria – With My New Camera

Yup, I’ve finally put up the money to buy a brand new Nikon D810 – and I totally love it! The detail you get from that 36MP sensor just makes my trusty D3s look old and ‘dusty’.

I’ve started taking a couple of landscape shots with the 14-24 Nikkor lens I bought last year – a combination that is worth having.

Looking forward to as many as possibly ‘golden’, rainy and colorful autumn days in the next couple of months.

Komberg - Fields

 

Styrian Sunset