If you want to run PowerShell commands in parallel you can use Powershell background jobs. In this post I’m going to have a look at waiting for jobs to finish, handling failures, timeouts and some other bits I came across when I started using background jobs.
Start-Job
To start a background job use Start-Job and pass in the script block you want to execute, PowerShell will immediately return to the console.
The job will execute in it’s own scope and won’t be able to use variables and functions declared in the script starting the job. You have to pass in all the functions required for the script block to run using the InitializationScript parameter when you call Start-Job. You can store the contents of the script block in a variable to enable reuse.
$functions =
{
...
functions
...
}If you have multiple jobs running you will need a way to know all of them completed execution. Complete in this case means it ran through the script block you passed it, it could have failed, or thrown exceptions or got disconnected from the remote machine or executed without errors. You can use Wait-Job and or Receive-Job for this.
Wait-Job
Wait-Job suppresses the console output until background jobs are completed and then it prints only the job state. Note the HasMoreData property that is true, this means there is something in the job output that can be retrieved. By default it will wait for all the jobs in the current session but you can pass in filters to wait for specific jobs. Only problem with Wait-Job is it doesn’t show the output from jobs, you can always log to a file but seeing what is happening is handy.
Receive-Job
Receive-Job’s actual purpose is to get the output from jobs at that point in time but by passing in the Wait parameter it will wait for the jobs to complete first. By default it will wait for all the jobs in the current session but you can pass in filters to wait for specific jobs. To me it looks like Receive-Job was not specifically designed as a mechanism to wait for job completion it is more a side effect when using the Wait parameter and it doesn’t have a timeout parameter.
You can also pass in the WriteEvents parameter that will print to the console as your jobs change state.
Handling Timeouts
Wait-Job takes a timeout parameter.
Wait-Job -Job $jobs -Timeout 60
If your jobs are not completed and the timeout is reached it will return from the Wait-Job call but it won’t show an error message. You’ll have to retrieve the jobs at this point and check which ones didn’t complete, you can use Get-Job for this.
Handling Errors
Errors that occur inside the script blocks running as a job won’t bubble up back to your script. If you use Receive-Job it will print to the console and if you use Wait-Job you won’t see anything, you’ll have to retrieve the jobs using Get-Job to see the failed status.
You can also store the errors that occur in each script block by using the ErrorVariable parameter, it is not a jobs specific feature it is one of the PowerShell CommonParameters, errors are stored in $Error automatic variable but by using ErrorVariable you can append multiple errors to the same variable and you can specify a different variable for each job to get the errors just for that job.
You can use Wait-Job and Receive-Job together by piping the result of Wait-Job to Receive-Job but that means you will only see the output from your jobs when they are done.
[Updated] Pass Parameters To Start-Job
As I mentioned earlier the job won’t have access to variables in your script, you can pass parameters to Start-Job using the Argumentlist parameter. It takes an array of arguments for example:
$param1 = @{"name" = "value" ; "name2" = "value2";}
$param2 = "value2"
$job = Start-Job -Name "job1" -ScriptBlock {
param($param1, $param2)
...
} -ArgumentList @($param1, $param2)
