Statistically Indistinguishable from Perfect

Here’s a summary of why the Amazon S3 service went down last week. The title of this post comes from a statement at the end of their goals for their own service level. I really like that turn of a phrase. The takeaway lessons from this are 1) engineering services at this scale is always an adventure 2) failures of this magnitude tend to come from the places you would never think to look and 3) cloud computing as a model has a set of risks associated with it that tend to be glossed over when people talk about the ease of setup and cost of the service.