This started one Sunday afternoon when I was reading the paper. My pager went off, with my boss' home number. I called the boss back, and he told me that the server for a major client site in Dallas (I'm in Chicago) is down.
This server handles the database for a distributed security system for a data center. While the security was not comprised, since the equipment runs independent of the server, the client couldn't grant new access, or, more important, couldn't revoke access either.The boss told me he'd tried EVERYTHING. He said to call the site and see what I could do. So I got the head of security on the phone and had him check the basics:
- Me: "Is the VT Terminal on?"
- Him: "Yes...the light is on."
- Me: "Is the MicroVAX on?"
- Him: "I think so...how would I know?"
- Me: "Is the green light on in the front?"
- Him: "Yes."
- Me: "Ok. Put your hand behind it on the right side. Do you feel the fan blowing air out?"
- Him: "Ah...hold on. Yah."
- Me: "Ok. "Is the remote dial in modem on?"
- Him: "Yah."
- Me: "Ok, turn it off and turn it on again."
- Him: "Ok, done."
At this point I tried to dial into the system. The modem answered, but after connecting there was no response from TTA1. Not good.
- Me: "Go behind the console and on the MicroVAX. You'll see a VERY small square button. Push and hold that button for 10 seconds. Then tell me what you see on the terminal connected to OPA0."
- Him: "Ah...ok...ok...ah...pushed it, but nothing happened on the screen. "
- Me: "Ok, turn the MicroVAX off, then on again."
- Him: "Nothing."
- Me: "Hm. Well, I'll give my boss a call and let him know that it's still down. We'll have someone there tomorrow to see what can be done."
So I called the boss and told him what went on. He said to bring a change of clothes to work the next day, as I might be taking a trip.
Later, at 7:30pm CST, I was standing in front of the site. I walked in and went to the OPA0 terminal. It was on, but there was no response from the server. I wasn't expecting there to be one, but I had to check. So I walked around to the back of the console to hit the HALT switch.Hmm...wait...something missing...ah...why isn't the power supply fan running?Why isn't the green light on?
I checked the power switch, and that was ok. The cable was plugged into the power strip. The power strip was plugged into...nothing. One inch from the outlet! Gah!
I plugged the strip back in...AH HA! And we have LIFE in that old MicroVAX!I filled out my paper work and stated to the Head of Security that the call would indeed be billable, plane fare and all. But they had an emergency service contract, he said! Yes, but it doesn't cover user error.
When I got back, my boss told me the boss of the Head of Security wanted to speak with me.
- Client's Boss: "The head of security says that the power strip wasn't unplugged until you got here."
- Me: "Really? Not sure what to say to that, why would I unplug it?"
- Client's Boss: "He says because you didn't want to tell him what was REALLY wrong."
- Me: "Interesting...well, since I was there less than five minutes, and that's all I did do to bring the server back up, I really don't know what else to tell you. You can check the video tape and see that's all I did. I'll even bet you could review them and see who DID unplug it."
Two months later, I found out that the company that supplied the security personnel was let go. It seemed the security server had been down for twelve hours before anyone noticed that a janitor had unplugged the power strip to allow his vacuum to run in the next room.
