When we are setting up new servers and services we often set them up in the rack in the office first. It has a fibre to the data centre so is well connected (in fact we can test on the data centre LAN so no config change when we move). We then move to the data centre later.
The new call server is in that stage, and we were going to move it. We did not move it last week. We figured "The data centre has had as many power outages as the office so no made rush just yet". Bad call! Sorry...
Well, a certain son of mine, who shall remain nameless, felt it was a good idea to plug in half an air-con unit that had "THIS IS DEAD" in huge red letters on it, and then wondered why it all went dark suddenly. It was not even a whole air-con unit, and yes it should have been disposed of already.
So, I suspect we'll be moving the call server to the data centre soon. Ideally a Sunday morning. We'll also be disposing (properly) with the dead air-con unit and looking at "procedures" for removing the plugs of anything like that in future!
Obviously, the call server is being deployed as a pair of servers. We have to try and address how we handle seamless failover on this as some phones get upset with calls from an unexpected IP address. This is all part of what we are trying to make work now.
The good news is calls in progress should have been fine, and also not charged for.
What with our favorite telco being a pain, this is not a good day.