Hacker Attack Post-Mortem

I've been pretty quiet about this for the past 6 weeks, but now that I've had time to recover almost everything that I lost I can now confirm that yes, my server did get hacked in early February. It appears that this group of hackers in Brazil who call themselves "SPYKIDS" used the now-commonly known exploit in older versions of awstats. It also appears that they got Russell Beattie and Jeremy Zawodny, both of who are far better sysadmins than I am so that makes me feel better. Looks like they also got Juju.org.

As Russell reports, this group apparently has an M.O. I discovered the mass defacement of every site on my server about 4 hours after it happened. Assuming it was just some mindless script kiddie I replaced each index page of each site with an Under Construction note, planning on restoring everything later in the day. It was terrible timing since that was the same day I was moving apartments from Manhattan to Brooklyn. Later that day I logged into the box and found that they had deleted the entire /log directory to cover their tracks and there was some weird telnet session open, likely a bot connected to an IRC server. Realizing that they had somehow acquired root access I knew the box was beyond repair and shut it down. The next day a friend and I went to the co-lo facility and rebuilt the OS, re-installed all the software and recreated the few accounts needed for the box.

I also learned that it's very important to run regular backups so that if your box gets compromised again it's easier to shutdown, rebuild and restore without losing any data. I guess I learned the hard way. I'm fortunate that I don't host any of my clients' web sites on my server, so all I have to lose is my development projects, some personal web sites, and a few sites I host for friends.