Anatomy of a Denial of Service Attack

Running one of the largest websites on the internet with about 5 million unique sites hosted exposes you to all sorts of issues.  There are constant events to deal with, some internal, some external.  This morning, one of the more common external events, a Distributed Denial of Service Attack occurred.  We experience these types of attacks rather frequently, but most are easily mitigated and have no user impact.  One this morning, however, was rather large and thus impacted some users.

Here is a timeline and description of this morning’s events:

9:40 AM EST — Our internal monitoring systems alerted us to unusual activity in one of the four geographically diverse datacenters which serve traffic.  Here is what that anomaly looks like in graphical terms:

10:00 AM EST — The target of the attack was identified and removed from our network.  The attack, however continued.  This is because the attacker had hijacked tens of thousands of computers (probably by installing a virus which was spread via email) and these computers had no idea the site was no longer there.  A small log sample shows over 8 million requests for this one site from over 10,000 unique IP addresses.

10:20 AM EST — Since we have servers in multiple data centers throughout the United States which serve traffic for all the time, we were able to route all legitimate traffic out of the affected data center, and let the single affected data center deal with the attack.   

11:30 AM EST — The IPs targeted in the attack were null routed at this point which allowed us to bring all datacenters back online to serve normal traffic.

We keep hourly traffic metrics and based on those numbers, it looks like during the attack there was about a 5% decrease in overall pageviews during the 40 minutes before traffic was re-routed.  All things considered, not a bad outcome for an attack this size.  Looking at bandwidth graphs, this attack was in the 500Mbit – 750Mbit/sec range.  

Author: Barry

To be written by someone famous...

27 thoughts on “Anatomy of a Denial of Service Attack”

  1. Excellent work Barry! That is a pretty sizable attack for sure! It always impresses me at how fast traffic can be rerouted with a well planned out setup like you guys have in place!

  2. Wow! Thanks for sharing that 🙂 It’s good to be able to see just how ably WordPress keeps itself protected, and how easily you can manouver around DoS attacks! Nothing like this to instill ever more faith in the great service you guys offer! Thanks 🙂

  3. Thanks for sharing this info! Although smaller in scale obviously, we’re running into similar problems too.

    Could you share in a new post what kind of tools you use to prevent ddos?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: