The blogging nightmare of the apocalypse

If only Dante were here to help document this….

I get back from Hawaii, throw down my last two posts in a jet-lag induced afternoon binge, do a little sleeping in, work from home on Friday, enjoy the weekend, roll back to the office on Monday the 16th to find… two urgent voice mails from adminstrative folks in the CS department alerting me that my blog site has been compromised. OK, what does that mean exactly?

Well, first of all it means we’re unbelievably stupid (by “we” I mean first and foremost me, and secondarily everyone in my lab that had anything to do with helping to set up the blog site that you see before you). Well, okay, so we’re not so much stupid as naive to yet another in a ridiculously long list of Internet abuses available for heapage upon innocent netizens like us. Excuse us for acting like this is a safe neighborhood.

What happened was we got up close and personal with splog. Here’s the approximate play-by-play.

The first big problem was that we had installed the multi-user version of WordPress. Why did we do that? I teach a class called Computers and Society, and I have students deliver their thoughts and reactions as short posts on actual blogs in the actual blogosphere. It’s an interesting experience for students to submit their homework to the world where the instructor and TA are two of a potentially larger number of random readers (including the entire class). Strangely it tends to generate higher quality work.

We’ve tried different approaches in the past, but this Fall I was determined that we should host blogs on our server for any students that didn’t already have one, and that we would make the process for them to set up a blog very easy (courtesy of multi-user WordPress). What we failed to grasp was that this was very much like going into a really bad neighborhood, leaving your front door wide open, the keys in the ignition of your car, and a sign on the front lawn reading, “FOOD IN THE FRIDGE!”

What happened next was that someone received an email from a stranger in the blogosphere suggesting that maybe our site had been compromised. Painfully obvious after the fact what had happened. Splog bots had stumbled on my blog, saw that WordPress was powering our world, checked to see if there was an easy way for a stranger to just launch a blog here. Sure enough. Paycheck loans? No problem, we’ll host that blog. Viagra? We’ll take two. Other… um… stuff… Sure why not?

By the time we became aware that things had become extremely dumb, we were hosting more than 300 blogs on our server on every topic imagineable (or unimaginable, as the case may be). We thought there were protections on automatic creation of blogs, like admin approvals, or at least email notifications, but nobody had seen any notification. The system hadn’t alerted us and we hadn’t noticed.

The next steps were pretty obvious. Take down the server to be sure it wasn’t compromised. Then bring it up except for Apache while we figure out just how big the problem was on the blog front and explore possible solutions. Ultimately we migrated the blog over to a single-user version of WordPress, gutted all the skanky content living on our server, and brought Apache (and the blog) back up.

The next realization was really shocking. In an attempt to determine the broader damage, I did a search for our server domain and found 84,000 Google hits (and for the most part they weren’t pretty)! Ordinarily Google power rankings are a desirable thing. But when the good names of your server, your department, and your university are being dragged through 84,000 mud puddles, it’s a really really bad thing. (As an interesting side note, MSN only had around 400 hits for the same domain, and Yahoo! had only 18! Not sure what that implies about the relative effectiveness of the search engines.)

We made a request of Google that the contents of our server domain be purged from their cache, which they quickly responded to. Apparently it’s a common enough request that they have an automated system for doing it. That dropped the directly hosted garbage from Google’s cache, but it didn’t do anything for every other splog or spammed blog on the planet that pointed to our server with a promise of replica watches or worse. After a week, those links have begun to weed out, and we’re now down to around 69,000 Google hits for our domain. Er, our former domain.

Because of our sense of the damage to the reputation of our server’s name (not to mention the unfortunate association of numerous inappropriate topics to our university domain name) we changed the server name to sequoia (after the new software engineering lab name). That doesn’t help the fact that there are still thousands of sites indirectly associating BYU with all this garbage, but we didn’t want live searches for my blog to turn up the garbage still in the cache.

We’ve since been fighting redirection to try and get traffic to the right places when people look for this blog. Some of it works right, some still doesn’t. You may see in the URL, or you may see okoboji still. We’ll sort it out eventually.

Meanwhile, my next move is probably to grab an entirely new domain (yet to be determined) and move the blog there for permanent safe keeping with a well-secured single-user version of WordPress, and a fresh reputation.

As President Bush said… “Fool me once… Shame on… shame on you. Fool me… can’t get fooled again.”

Why must my phone be in the off position?!

Got back from Hawaii this morning. Two flights out last week, two flights back last night and this morning. Four doses of airline-speak.

I have boarded, deplaned, stowed my belongings, found the card in the seat-pocket in front of me, and have been careful because items in the overhead bins do tend to shift during flight.

I have watched others pre-board, and pondered the unlikely event of a water landing.

I have returned my seat and tray table to their full, upright, and locked position.

I have been reminded that it is a federal offense to tamper with, disable, or destroy any lavatory smoke detector.

And I have turned my cell phone to the off position. Two questions: 1) Who in the world came up with this phraseology? 2) What precisely does it mean?

It’s bad enough that flight attendants do tend to overuse certain words that they do say repeatedly because they apparently do think that it sounds more officious and they do realize that we do have a choice of airlines and they do appreciate us choosing whatever airline this is. They really do…

But, I mean, what is this “off position”? Switches can be in an off position, but many electronic devices don’t have switches. They have some magic button that you hold until the device becomes dark and lifeless. Is said device now in the “off position”? Even more frightening, the average cell phone user has no idea that while their phone is “off” (meaning screen dark? or maybe silent ring?) it is still waking up periodically to check the availability of nearby cell towers, in case someone wants to call in. And with all due respect to the enormous amount of radio noise my Bose headphones must be generating, it can’t be within a couple orders of magnitude of what half the cell phones in the cabin are probably doing in the “off position.”

Now if they just did something like this… “Ladies and gentlemen, your cell phones generate radio signals even when they may appear to you to be off. Will you please do whatever magic incantation you have to do to your phone to make it so that the phone cannot receive incoming calls? When you have done that, your phone will be in radio silence, and will not interfere with any of the radio or telecommunication instruments in this big bird. Oh yes… and we do hope that you do forgive us if we put airline-speak in the off position.”

Blogging from the beach in Hawaii… without a laptop…

Didn’t work… AAARRRRrrrrggghghh!!! Tried, failed. It was going to be one of those uber-cool posts in which I prove my obsessive techno-tendencies beyond any reasonable doubt by sitting in a beach chair, on the Hukilau beach in Oahu, using nothing but my Treo, and blog about blogging on the beach because… um… ah… my wife will explain it to you.

Possessing a modicum of common sense, I made the attempt to login to my blog server on my Treo within the reasonably cool confines of our temporary beachside domicile. I could read the blog from the beach using the Treo web browser, but for an as-yet-undebugged reason, I could never successfully login to the server on the Treo. I thought it might have something to do with cookies, but that didn’t seem to be it after all. Scanned the meager offering of menu options for the Treo’s web browser to no avail. Tried resetting the password remotely (requiring a multi-step process involving a successful email adventure on said Treo), still no worky.

Finally my phone contorted itself into a state in which everytime I go to the web browser, the Treo reboots itself before doing anything else. Classic. I try to do something techno-cool (like blogging on the beach without a laptop) and by the time I get done trying, I’m in a worse state than I would have been had I never tried the uber-cool manuever in the first place. I hate that. And how do I fix my web browser when I can’t get to its menu options because it reboots my phone every time I try?! Sheesh.

Fortunately, the trip to Hawaii was not a total disaster. Weather was absolutely beautiful. Waves were great. Sand was sandy. Food was great. Polynesian Cultural Center was amazing. My visit to BYU-Hawaii in Laie was really fun. Pearl Harbor was moving. Wakiki was picture-perfect.

Oh yeah, and I was able to check my email a couple times a day on my Treo to help mitigate a potentially serious case of technical separation anxiety…

I think I need professional help.