HE is looking into the issue, will update when I know more...

Hurricane Electric had a switch failure, and they're moving our uplink to another switch. No definite ETA yet; could be 10 minutes.

Problem Resolved.

I've taken this, and the other problems we've had, up with Linode.com's sales rep at Hurricane Electric. Hopefully, the string of deficiencies will improve.

Thanks,
-Chris

Guys, I use my linode to keep my family and friends up to date. My chosen profession makes that kind of important. The growing frequency of outages is getting quite concerning. Please fix whatever the issue is.

Anthony

Chris,

Fortunately, my Linode is a hobby project for me, so this kind of downtime is minor for me, but hinestly, if I ran my network the way HE runs theirs, I'd be out of a job!

Knocking power cables during installs is just inexusable laziness on the part of the techs. Losing a UPS shouln't happens, because presumably these servers have redundant PS's on separate UPS's and so a single failure should matter. Same with the network upinks....why aren't there redundant uplinks to their switches - its really not that hard to do!

_________________
--Matt Cowger

I'm sorry, I've never ever once had a real complaint about linode, but this HE service is real BS.

I can understand a blunder or two, being that we are human and we sure aren't prefect. But this constant downtime really makes one wonder.

Bill Clinton

Indeed, the downtime lately has been atrocious and it feels like it's getting worse and more frequent.

Not sure how to respond to this statement. You calling me a liar? I suppose I could be feeding you guys whatever you want to hear, but I accept responsibility when something is my fault.

Over the past 45 days, only two or so of the network related issues have been related to DDoS attacks, which, I suppose you could consider "my fault". All the other times, it's been an HE problem: the power failure, a router problem late last month, another BGP problem a week or so ago, and then their switch that died on us tonight. Believe me when I tell you I'm more upset about this than you are.



Hurricane Electric touts redundancy as a main reason to choose them over someone else. For months their service was exceptional, but over the past 45 days or so they've been cursed with problems. I've taken it up with our sales rep and will let you guys know what the deal is. Not sure what else I can do. If you have any suggestions, please let me know.

-Chris

I've ask for redundant uplinks to different switches to our rep, I should have an answer tomorrow.

-Chris

Chris,
I edited prior to your posting, as I hadn't quite cooled down appropriately before posting. I appologize if it seemed like I was calling you a liar. I understand redudancy, so there is logic there, and I can accept that. I stand by my final assertion, but I appologize again for the tone I originally voiced it.

Anthony

caker,

Thanks for updating us on the situation.

No worries. You should have heard me on the phone with the HE techs!

Ultimately, all of this is my responsibility, and I still stand by my decision to use Hurricane Electric. The best thing we can do now is make HE aware of these problems and make sure they don't happen again.

As for the DDoS attacks, I've been spending a lot of time researching configurations and hardware to help mitigate the effects of attacks. Just want everyone to know that I'm working on it.

Thanks,
-Chris

If anyone thinks that tonight's downtime was bad or had unacceptable downtime within the last 45 days, think about how long the downtime would be if caker moved the servers to another ISP. Even then, the same problems could occur, if not worse.

How does the saying go? "The enemy you know is better than the one you don't."

I think caker did an excellent job sending out a notification about tonight's outage.

-John

If Chris ever did have to move stuff to another ISP then he'd have to use a swing server and do it gradually; install a server (of the highest spec matching what's in HE) in the new place, "swing" the linodes on one host over to the new; this free's up one host in HE which can then be moved to the new location etc etc etc.

Having taken part in a lot of datacenter moves recently (decomissioning one datacenter, leaving another building) for work I can tell you this is a painful business

Heh, maybe Chris could get a partial refund from HE for their outages and extend everyone's service by a week

_________________
Rgds
Stephen
(Linux user since kernel version 0.11)

The swing server would certainly be the way to do it, and I'm sure caker could make it happen with minimal disruption. (It would involve everyone's IPs changing though.)

But it would be a Major Pain for him, and us if we depended on our IPs. Probably we're suffering from a "grass is greener" syndrome. HE will probably get its act together.