Comments on: Amazon’s S3 Web Service, our #1 cause of failure http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/ Thu, 11 Mar 2010 07:12:39 +0000 http://wordpress.org/?v=2.3.3 By: daven http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-2302 daven Wed, 10 Feb 2010 12:25:33 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-2302 I don't know if anybody's reported this before but the browser I got does not show this article rightly... It appears that a significant portion of it is not showing and the layout of your page doesn't seem right. Can you confirm that this post has been tested with Google Chrome? I don’t know if anybody’s reported this before but the browser I got does not show this article rightly… It appears that a significant portion of it is not showing and the layout of your page doesn’t seem right. Can you confirm that this post has been tested with Google Chrome?

]]> By: Usdating http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-2093 Usdating Sun, 10 Jan 2010 21:33:49 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-2093 There's some great tips there, just blogged about it too! There’s some great tips there, just blogged about it too!

]]> By: Святослав Маминов http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-1429 Святослав Маминов Sat, 22 Aug 2009 22:03:21 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-1429 Интересная тема, продолжайте. Иногда нахожу ответы, которые получить самому просто реально не хватает времени. Благодарю! Интересная тема, продолжайте. Иногда нахожу ответы, которые получить самому просто реально не хватает времени. Благодарю!

]]> By: Артем http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-1418 Артем Tue, 18 Aug 2009 23:56:49 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-1418 Вот это да... Куча комментов и практически ни одного по теме :) Вот это да… Куча комментов и практически ни одного по теме :)

]]> By: codediva http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-399 codediva Fri, 25 Jul 2008 02:48:44 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-399 I understand your frustration, however, anytime you used hosted services (even those from a trusted provider) you should plan on implementing some redundancy. I know this can be costly for small orgs, but it could have minimized the impact of this occurance. I understand your frustration, however, anytime you used hosted services (even those from a trusted provider) you should plan on implementing some redundancy. I know this can be costly for small orgs, but it could have minimized the impact of this occurance.

]]> By: LusciousPainGeek http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-395 LusciousPainGeek Thu, 24 Jul 2008 08:48:32 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-395 Isn't this one of the risks you take in exchange for the convenience of using a centralized system like S3? Unfortunate, but bound to happen and something you need to look into when performing risk assessment. Isn’t this one of the risks you take in exchange for the convenience of using a centralized system like S3? Unfortunate, but bound to happen and something you need to look into when performing risk assessment.

]]> By: Ted Z http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-394 Ted Z Tue, 22 Jul 2008 19:15:35 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-394 Look into Akamai, they'll do this correctly (disclaimer: I've worked for them). Or implement their system (quickly deteriorating DNS endpoint lookups) yourself, it's not hard if your scope is limited. You can just have S3 be your primary and when it fails, switch DNS over to your backup. Look into Akamai, they’ll do this correctly (disclaimer: I’ve worked
for them). Or implement their system (quickly deteriorating DNS
endpoint lookups) yourself, it’s not hard if your scope is limited.
You can just have S3 be your primary and when it fails, switch DNS
over to your backup.

]]> By: Ryan Haney http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-393 Ryan Haney Tue, 22 Jul 2008 15:14:43 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-393 Have you considered using S3 as a backup provider with a local disk as cache? I can envision software that emulates a hard disk with a caching mechanism that copies to and from S3 in the background, as files are requested / uploaded. I do think though that you haven't load tested your own storage solution. I have a friend that works for Seagate and hard drives fail ALL THE TIME. It's just nature of the beast. Have you considered using S3 as a backup provider with a local disk as cache? I can envision software that emulates a hard disk with a caching mechanism that copies to and from S3 in the background, as files are requested / uploaded.

I do think though that you haven’t load tested your own storage solution. I have a friend that works for Seagate and hard drives fail ALL THE TIME. It’s just nature of the beast.

]]> By: Joe http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-392 Joe Tue, 22 Jul 2008 03:05:39 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-392 Get a grip !!! It's not like you're running a stock brokerage system on Wall St ! My experience is that it's not hosting computers that are the reliability issue, but the ISP who provides the connection. Try getting any level of service guarantee is next to impossible. When I build "five nines" systems for clients, we end up putting data centers in multiple cities using multiple carrier connections. It costs a heck of a lot more than what Amazon charge for S3. It's like you paid for a cheap car, so don't whinge about the rattles. Get a grip !!! It’s not like you’re running a stock brokerage system on Wall St !

My experience is that it’s not hosting computers that are the reliability issue, but the ISP who provides the connection. Try getting any level of service guarantee is next to impossible. When I build “five nines” systems for clients, we end up putting data centers in multiple cities using multiple carrier connections. It costs a heck of a lot more than what Amazon charge for S3. It’s like you paid for a cheap car, so don’t whinge about the rattles.

]]> By: Striddy http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-391 Striddy Tue, 22 Jul 2008 02:06:19 +0000 http://blog.crowdflower.com/2008/07/amazons-s3-web-service-our-1-cause-of-failure/#comment-391 Should've been: Also, I don’t get the sense that FaceStat’s load is so heavy as to ... stress your other hosting service to the point of failure much. Should’ve been:
Also, I don’t get the sense that FaceStat’s load is so heavy as to … stress your other hosting service to the point of failure much.

]]>