Monday, September 30, 2013

When the Shell has too much power

We had an incident last Friday. Our production print queues went down for about 20 minutes

Our post incident report has this as the root cause

The following script was run with the variable $server set to PROD-PRINT-SERVER;
$printers = Get-WmiObject -class win32_printer -computername $server | where {$ -like "*Toshiba*"}
foreach ($printer in $printers)

This removed all Toshiba print queues on the server in question (the production print server). The $server variable should have been TEST-PRINT-SERVER.

Yup! That'll do it. It was all resolved pretty quickly, just restored the print server from the latest snapshot.

So when rolling out powershell to enthusiastic admins make sure to put the breaks on in the right places.

Our fix for this will be to change the ACLs for production print queues so that no-one has the rights to delete them. That way when the time comes to delete a queue we'll have the additional step of changing the ACL but that's better that being able to blow away production print queues with a goofed WMI query.

Anyone have any other thoughts on what could have prevented this situation? Leave us a comment!

No comments:

Post a Comment