I lost the month of February.
Give me enough time and I might be able to piece the individual parts together and eventually tell you most of the things that happened during the month. Maybe.

The only things I clearly remember are the things that broke at work, starting with this, on our storage array on one of our sequel clusters:

That failure wouldn’t’ve been so bad if not for 2 things:
a) a 4 hour replacement contract doesn’t mean shit in the middle of a snowstorm
b) it would’ve been entirely avoided if my coworkers had bothered to google that error message when it first appeared

I had gotten the bluray of Amelie imported from Australia that evening, so I spent a lot of the night watching it (while waiting for DHL next-flight-out’s shipment of our replacement controller (DHL still exists in some fashion?!)). I remember being glad I wasn’t one of the people who were busy clearing the parking lot all night in said snowstorm, which means I wasn’t that bad off at the beginning of the month.

Unfortunately, at the end of that week vendors who came out to fix our flaky microwave connection to one of our buildings up the street killed the hell out of all of it. We managed to frankenstein something together that same weekend, but it never worked well.

When you give people an imperfect solution, they just see the faults in the solution. They don’t really understand what the alternatives would’ve been. It didn’t matter that we’d put in 20+ hours of work during the weekend so they’d have connectivity of some sort come Monday, it just mattered that their shit was slow.  Granted it really was pretty bad (apparently the window coating or something interfered with the signal during the day, when we tested at night it was really decent), but the alternative was complete work stoppage.

All I really remember after that is many many hours of standing on roofs in the cold.

Sometimes you find interesting things on roofs:

I’m pretty sure these warning are about the equipment I was working on.  I’m also pretty sure I’m not trained on safety using this equipment nor do I have a personal RF monitor.  I hope the ‘just don’t stand in front of the dish while its on’ advice I got is actually good enough.

Being low on the totem-pole sucks especially when you sit around and watch your advice get ignored and then have to spend hours and days trying to fix what shouldn’t be broken in the first place.  (The good side is, of course, that my responsibility in fixing any of this is less than it otherwise would be.  This period has sucked for the other members of my team at times more than it has for me.)  More than a month later things aren’t satisfactorily resolved still. I did spend a few hours at the end of last week on the top of a very tall ladder on top of a very tall building removing our old equipment so I could overnight it to TX to get it fixed (which, just in case you were wondering, was expensive as shit, they better reimburse me).  So hopefully it will be more-or-less permanently resolved soon.  I don’t know at what point I went from being comfortable on the edge of any cliff to being uncomfortable with heights, but I really didn’t enjoy this much at all.  (You can see the location I’m taking these pictures from in the first roof-top picture if you look for the little dish at the top of the wall over the big dish.)

The thing that sucks about IT jobs is that when something major is broken, everything else doesn’t necessarily stop and wait for it to be fixed, or wait for its turn to break.  This is especially true as apparently the departments who rely on this link have reneged on their pact with the Devil (it is the accounting and billing offices amongst others, so that’s likely).  Everything else related to this link has managed to break along the way.

For a good illustration of this, we can examine the first week of this month:

Monday 1st, work from midnight-6am upgrading our core switches, the downtime was scheduled to end at 4, but it took twice as long to do half as much.  By the way, our server room is really freaking cold, especially if you’re there for hours at at time:

I’m the guy on call for the hospital so they call me with stuff until 8am, at which point I sleep.  They call me at 10:30am because of issues with the above mentioned connection, so I go in and work until 3:30PM.  I go home and fall asleep around 4PM.  At 5PM, one of my coworkers calls me because he just woke up and noticed all of the unsuccessful attempts to reach him that morning.  At that point I can’t fall back sleep until 2AM.

Here’s my attempt to make coffee that morning, I’m glad I caught it this time before running through the cycle, it’s really hard to grind after you run it through the machine whole.

Dinner that night consisted of looking at things in my cupboards and finding the idea of eating completely incomprehensible.

Tuesday 2nd wasn’t so bad except I had to show up early for meetings, but I did get to follow some security dude from our main office around all week and see what it is that he does, which is more fun than what I normally do

Wednesday 3rd, had to show up early again, ended up moving a server to the other side of our crappy link, which wasn’t too problematic but another piece of that system broke while rebooting it during that move so I was there until 9 rebuilding it

Thursday 4th, called at 6am with a voice-mail that wasn’t sufficient to establish that a) there was actually a problem and b) they understood I wasn’t the one on call (my only week this year not being so in some fashion).  Called at 7am by someone else who managed to communicate that the above-mentioned connection was entirely down.  The switch that the wireless bridge was plugged in to died…

I can’t even remember Friday.

And last week we had an analog line magically go dead after decades of working for that same system.

The point of all of this isn’t really for me to bitch about how much my job sucks, or how underpaid I am for it, but is instead to document it in some fashion so that I can remember it later.
While these things are somewhat worse than things normally are at my job, it certainly is not that uncommon for my job to completely crap all over my life.  I normally forget about these periods fairly quickly, because I both tend to filter out bad memories, and additionally, when one gets this tired, it gets very hard to properly encode these memories so as to remember anything but a blur.  I don’t think I would’ve really noticed what was happening to me this time if not for the distinct contrast between my productivity on projects early this year, and my completely inability to accomplish, or even want to accomplish, anything by the end of last month.  I’ve come to realize that it is a bad sign if I start playing video games, which I was doing at the end of last month (Mass Effect 1 & 2 are pretty alright), but other than that, if I didn’t have so much stuff I was trying to get done, I would probably just drift for a couple of months until I recovered enough to feel like accomplishing something.

I’ve always casually made the remark that I’m trading my health and wellbeing  for money at this job, but I somehow never clearly understood just how much it was preventing me from accomplishing my many other projects until now.  Understanding that made it a lot easier for me to make decisions that might change this, hopefully I’ll have good news soon that I can look forward to a much more sane balance between my job and the rest of my life.  Keep your fingers crossed for me.

By the way, my new resume kicks ass and I feel marketable for the first time in my life, its weird as hell.