Happy Monday!

Monday’s are wonderful, aren’t they?  I mean, it’s awesome to walk in first thing in the morning and find things are already broken.  The gremlins were working overtime this weekend, I see.

I get in and I’ve not even logged into my PC yet and one of the desktop techs approaches me about a possible user port being down.  So he waits patiently at my door, chatting with other folks outside my office as I finish logging in and trying to get into the switch remotely to see what’s going on.  Unfortunately, I cannot get into the switch.  I can ping it, but telnet and web access weren’t working. 

Ugh.

So I take a look at SolarWinds Orion to see what the system statistics look like.  Immediately I see that the CPU is pegged at 97%.  “Well, there’s your problem!” to quote Adam Savage of Mythbusters.

Unfortunately, that most likely means I’ll need to reboot the switch.  Meanwhile, more calls are coming in that they can’t get access on that switch.  But before I can deal with that, I get a call from a tech out at one of our remote offices.  She tells me that one of the satellite offices got struck by lightning, a laptop is burnt out and no one else can access anything on the network.  So I pass the reigns for the malfunctioning switch to my couterpart and begin looking into this new issue.

Turns out I can access the router and switch at that location just fine, but cannot connect to the domain controller, which is also the DHCP server that distributes IP addresses for the site.  If that’s down, ain’t no one getting on the network.  So I ask  the tech to drive to this office, which is about 30 minutes from her typical office and check on everything to see what else might have gotten fried, including a reboot of the domain controller.

After dispatching her, I head up to the malfunctioning switch to see how things were going.  Turns out that since the memory is maxed out, it won’t let us connect on the console locally, either.

UGH!

This definitely means a reboot and downtime for the users that were still working on the flailing switch.  We get permission for the reboot and my counterpart takes care of it while I follow up with the other issue.  Apparently the reboot of the domain controller there worked and everyone’s back up.

Thankfully the switch reboot worked, too. 

Two hours, two fires extinguished.

Monday’s sure keep me on my toes.