An archive of my old LiveJournal
Still trying to repair fallout from yesterday’s power outage.
I won’t go into the full litany, but after the garden-variety power-back-up issues, it mostly comes down to 2 servers and a bizarre network problem. 1 box fixed yesterday afternoon. Network problem popped up & fixed this morning. Still working on the other box.
Oh, and also things breaking that (in theory) weren’t affected. WTF?
*grumble*
Ended up missing my company’s Christmas party this year because we had tickets to see Tori Amos at the Grove on Saturday. Good concert, but our feet were pretty much dead by the end of the standing show. Sometime I’ll have to post the experience of trying to eat at El Torito Grill beforehand.
Meanwhile, everyone at my office hopped on a bus last Friday at noon to go out to Las Vegas. Apparently they did a murder mystery dinner for the actual party. It was a bit eerie afterward.
Still scrambling to finish a website at work. Also trying to finish some articles that I’m contributing to a book about the Flash. And Christmas shopping? Hah!
5 days before Christmas is a bad time to find out that you’re almost out of scotch tape, but I suppose it’s better than finding out at 8pm on December 24.
Truth in cartooning. We have a customer who, whenever email is down, sends me an email.
Naturally, I don’t receive this critical piece of information until after we’ve fixed the problem.
Annoyingly, I probably would have caught it before he did today, except I left my phone at home. So the handy message from the monitoring box went… elsewhere.
Got some nighttime photos yesterday of the orange moon (to the right) and the glow coming from the hills in front of Saddleback.
The smoke is back again, like Monday. It’s thinner, so at first I figured it would be better, but I feel much worse. Maybe because I’ve been breathing the crap for the last 4 days, maybe because without the wind, the smoke and ash are settling at ground level.
Meanwhile I’ve been putting out virtual fires with our network since about 3:00 yesterday. Yesterday my boss and I spent most of the afternoon trying to troubleshoot a computer that was running incredibly slow, but only when the network was plugged in. I was there until almost 6:30 trying to troubleshoot problems that started affecting the few other people who were still in the office at that hour.
This morning, I got to troubleshoot the connection issues that started cascading around the office. Finally I discovered a cable on the back of the stack of hubs was plugged into the wrong port. We’d temporarily disconnected it yesterday afternoon for testing, and someone (presumably my boss) plugged it back into the wrong connector.
And of course, anything that’s already flaky has taken the opportunity to stop working, leading to follow-up “Hey, X isn’t working” visits that have led, so far, to rebooting a server, replacing a network card in a desktop, and resetting a hub. Did I mention my boss isn’t in yet, and no one’s really sure if he’s going to be in today?
So here I am, sitting at my desk, writing this in bits and pieces when I have time, drinking hot lukewarm chocolate, wishing I’d remembered to grab that bottle of Vizine or Clear Eyes or whatever it is from home. I really don’t want to go anywhere at lunch today, between having to leave early and wanting to stay indoors as much as possible.
It’s amazing how the human mind can hold contradictory ideas in its head at the same time. For instance, I somehow got it into my head this morning that today was Microsoft Patch Tuesday. Despite it being Thursday. Despite me knowing it was Thursday, and planning things accordingly. Eventually I got to the point where I thought, “Hey, wait a minute! Today isn’t the second Tuesday of the month, it’s the first.. Oh, wait… it’s not even Tuesday.”
That simplifies my lunch hour immensely.
Heat wave + 3-day weekend = way too much hot air trying to sneak into the server room. Extra measures are being taken to ensure that the server AC can keep things cool, but I’m probably going to spend much of the weekend worrying about getting a “Server X down” SMS.
Brownout during lunchtime. I was at the Spectrum food court. All the lights flickered off and back on, and the muzak went silent for just a second. Twice. Here at work, most of the desktop machines rebooted. (Servers, of course, are on UPS.)
Coffee Crisis! Coffee maker at work shut off this morning just a few cups in. Determined that the power outlet had gone dead. Tried plugging it into another outlet, via extension cord + power strip. Power strip started buzzing, then circuit breaker tripped. Further analysis of coffee maker’s plug identified melted, cracked plastic. The company we lease it from can’t send out a replacement ’til Tuesday.
So, only 3 cups of coffee for the entire office. No gladiatorial combat, oddly enough. Though I did joke that it must be a fiendish plot by the cafe downstairs to sell more coffee.
Bust of the Century: Yesterday afternoon, got onto the 405 and immediately got stuck in traffic. Turned out to all be rubbernecking from something going on by the side of the road, where police were searching a stopped car. Lots of them. At least 5 squad cars, from at least 2 different cities, and a sheriff’s car.
The middle of the day was mostly spent trying to recover from a server crash. In theory, the fix was simple: Boot to the recovery console, then run chkdsk. Of course, that meant getting the right CD to run the recovery console for that version of Windows, waiting for it to boot, finding chkdsk (or rather a utility that chkdsk depends on, which the console couldn’t find on its own)… and then waiting for it to examine the entire RAID array (a process which took up most of the time).
All together, that “simple fix” took about 3.5 hours, during which we set up the higher-profile sites on a backup server (a process that also took longer than it should have, for other reasons).
Periodically I would walk out of the server room to answer email, run downstairs to grab lunch, or run back to my desk to eat a few bites of that lunch.
I just took my travel mug into the lunch room to wash it out. It was oddly heavy. Now I know why I felt so sluggish this morning. It wasn’t just a crash following the fix-the-most-heavily-used-server-now adrenaline high. I’d only drunk 1/3 my usual amount of coffee!
This morning, I put the release dates for new versions of Firefox and Internet Explorer on my calendar.
Though I think I lose a few points for doing it in pen instead of using Google Calendar or something.
Friday: Goo Goo Dolls/ Counting Crows concert at it’s-still-Irvine-Meadows-’cause-“Verizon-Wireless-Amphitheater”-just-sounds-dumb. Nifty view of the central Orange County valley before the sun set.
Saturday. Spent the day lying on the couch with a fan running, catching up on the Babylon 5 script book commentaries. So far I’ve only read about 1/4 of one script, but the introductions are a fascinating insight into the process of making the show. Given how much was out in the open (since JMS spent several hours a night online during production), some of the stuff that wasn’t revealed back then is mighty interesting. (Example: we all thought season 4 was compressed because they weren’t sure they’d get a season 5, and wanted to get as much done as they could. It turns out that at the beginning of the season, they were told in no uncertain terms that there would not be a season 5. So getting that fifth season on TNT was a real surprise.)
Dinner at CPK, then Superman Returns Enjoyed it, but no interest in seeing it again. It already felt like deja vu since there were so many references to the first two movies with Christopher Reeve. There were some great moments, but overall it was just kind of okay.
Sunday: Finished Bad Twin. Decent book, but not exactly a major insight into the secrets of Lost. Waited several hours until checking my email, at which point I discovered I really should’ve checked earlier… so I could bring a server back online. Went down to the office, brought the server up. alenxa tagged along and we had lunch at the crepe place at the Spectrum. Happened to glance over at one of the wall alcoves, which held a vinegar bottle… labeled Temeraire. After some shopping for food for upcoming family get-togethers, went home and played Heroes V for a few hours. Fortunately I was still awake when the same server went down again at 1AM. So I drove back, brought it online, figured out a way to make sure that it would automatically start up again next time, then came home. Still trying to diagnose the cause, but whatever it is does seem to like holiday weekends…
Monday: Traditionally, the 4th of July consists of hanging around at home followed by looking for fireworks. This year, both our families wanted to get together. So on Monday we went to visit my grandparents, carpooling with my parents and my brother. My aunt and her husband brought their dogs, which meant I spent most of the afternoon in a Benadryl-induced funk, despite drinking way too much Dr. Pepper to offset it.
Tuesday: I squeezed in an hour or two of Heroes in the morning, then it was off to visit Katie’s family, with more BBQ. Dinner was relatively early, so we cleared out around 7:00 so we could make it back to Laguna Beach for the fireworks display there. They launch the fireworks off one of the bluffs, out over the ocean, and the viewing is pretty good from the beach. We caught the tail end of it a couple of years ago, while driving around looking for displays. Two years ago we got there too late and missed the entire show while we walked from the car out to the beach. (Hey, have you ever tried finding parking in Laguna Beach?) Last year I was sick, so we just walked out to where we could see the local display. This year we made it with plenty of time to spare, parking up on Forest (roughly in line with the Laguna Playhouse, and maybe a few hundred feet from the entrance to Tivoli Terrace, where we had our wedding), and snagged a spot on the beach. The wind blew the smoke in our direction, so a lot of the fireworks were half-obscured, but they lit up the smoke a lot more than I can remember seeing anywhere else. After the show, we did our semi-traditional stop at Diedrich across the street.
Today: Back to work. I’ve been stress-testing the server, trying to get it to crash under controlled circumstances so I can get some information about what’s causing the problem… and it just won’t crash!