Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
 Post subject: Re: Fremont RAM upgrade
PostPosted: Thu May 09, 2013 4:29 pm 
Offline
Junior Member

Joined: Tue Jan 31, 2012 5:17 am
Posts: 42
Website: https://www.serverfruit.com/
Location: Denmark
Valley wrote:
lakridserne wrote:
Solution: Move your Linode to another DC for a few weeks more while they get the last quirks worked out.

Many of us are geographically tied to Fremont.

Sigh.

Remember when Linode used to be the cool nerdy VPS provider that was transparent about this sort of thing?

Starting to make me uncomfortable / wonder what it is, exactly, that Linode doesn't wan't us to know.

1) It's not that much more time it takes when we're talking about the internet
2) They're the same cool and nerdy VPS provider that are transparent about things. They have never, to my knowlegde, preannounced things.
3) They will announce it as soon as possible.

_________________
lakridserne
Serverfruit - shared and managed VPS hosting, SSL certificates and domains
Awesome servers rented from Linode!


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Thu May 09, 2013 10:12 pm 
Offline
Senior Member
User avatar

Joined: Sat Aug 30, 2008 1:55 pm
Posts: 1739
Location: Rochester, New York
Strictly speaking, they did kinda pre-announce that Fremont would be getting the upgrades soon, and are not saying anything about why it hasn't happened yet. So, I would say that this isn't a great example of either transparency nor of avoiding premature announcements.

_________________
Code:
/* TODO: need to add signature to posts */


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Thu May 09, 2013 11:16 pm 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3112
Website: http://www.linode.com/
Location: Galloway, NJ
Here is the straight dope: We were getting everything ready for a move to Hurricane Electric's FMT2 facility, which is like two miles down the road from FMT1. This was all to be ready about a week after the upgrade announcement. FMT2 is a modern build-out, which we consider to be a great improvement over FMT1's aging power and space restrictions. The strategy involved us getting a 10G wave on fiber they have between buildings, which places our two distinct networks on the same layer2 network. Meaning no IP swaps, no disruption in IP failover support, private networking working as expected, etc. Essentially this has created one large LAN despite hardware and Linodes existing in both facilities. Over the course of 3 months or so, we would perform a few phases of migration queues + physically moving and upgrading hardware from FMT1 into FMT2, and then continuing with the next phase, rinse, repeat. Doing it in phases was largely a host-resource game.

This was all staged and ready to go, but we discovered this link wasn't as redundant as we had believed. We asked questions, and got answers from HE. There are scenarios where the wave between the two facilities could break, and that news put a stop to everything while we investigated how to implement absolute redundancy between the two facilities in the event the link failed. We have a solution, but want to greatly shorten the time we're relying on this bridged network, which means altering the FMT1 to FMT2 transition timeline. This is where we stand now.

Things could change, but we're now thinking this will be: a few weeks heads-up during which the upgrade-button would simultaneously upgrade you AND migrate you to FMT2. Those who don't perform the upgrade by the deadline(s) would have their host shut down, the host de-racked and physically moved to FMT2 and then powered back on. Of course we would have everything prepared in FMT2 to receive these machines and essentially it would be: a shutdown, unplug host, transport it to FMT2, rack it and plug it into already-ran cables, and then power it on. Logistics so far indicate the downtime for that scenario is minimal, 60-120 minutes, performed in the middle of the west-coast night. Over the course of a few days we'd be out of FMT1 completely.

We've already deployed an impressive amount of capacity in FMT2, however this shortened timeline and elimination of phases means upgrade+migrate-to-fmt2 availability will be on a first-come-first-served basis. We've also been working on redirecting host builds from other facilities to fmt2 to augment fmt2's host resources. However, we can't guarantee everyone that wants to move/upgrade early will be able to, under this scenario.

There are still a few loose ends, and of course this could all change, but that's where we are. The logistics of this type of endeavor are quite complicated as you can imagine, and we want to do this with the least amount of stress, downtime, and risk as possible. We hope to nail down the timeline and enable the upgrade button very soon.

-Chris


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 12:52 am 
Offline
Junior Member

Joined: Mon Jun 06, 2011 1:33 am
Posts: 40
Thank you so much for the update.

I have hated being on Fremont (FMT1) all this time, and I cannot wait until I get to FMT2. It's helpful to know what's going on. You're the best!

Thanks!
Bruce


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 2:00 am 
Offline
Newbie

Joined: Sat Dec 31, 2011 1:54 am
Posts: 3
Thanks Chris,

This is exactly the kind of answer I was hoping we'd get (it's obvious there was some sort of delay) so knowing what's happening and having appropriate expectations is the next best thing. Also, your response shows that you/the Linode team truly are concerned about making this a smooth transition for us, although I know that the world wouldn't come to an end if my very small $20/mo Linode went down, it's nice to know that minimizing downtime and inconvenience is a priority.


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 1:07 pm 
Offline
Senior Member
User avatar

Joined: Tue May 26, 2009 3:29 pm
Posts: 1691
Location: Montreal, QC
I really wish that you guys at Linode would post this sort of thing more often. Not necessarily all the time, or in that much detail, but it's nice to see. It's kind of like UI feedback; for a task that takes a given number of seconds, compare the user experience to having a progress bar versus just locking up the UI. The user who gets the progress bar is going to feel like the machine is faster and more response, while the user who just gets a frozen screen for the duration is going to feel the machine is slow and unresponsive.

It's like that for this; getting some insight into the technical stuff behind the scenes may not be something that we customers need, but it's definitely appreciated and reinforces that Linode (the company) is doing something. It also tends to make customers more tolerant of stuff too.

So, whenever you guys can afford to share some insight into goings-on, it's appreciated.


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 2:22 pm 
Offline
Senior Member

Joined: Mon Aug 31, 2009 2:33 pm
Posts: 78
Location: The OC
Thank you for the update, Chris, as well as your apparent attention to the details. It's great to see the move to FMT2. Does the current plan still mean no IP swaps?


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 2:30 pm 
Offline
Senior Member

Joined: Fri Nov 02, 2012 4:20 pm
Posts: 60
Thanks for the update! Would love to see more detail like this in the future as has been in the past.


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 4:44 pm 
Offline

Joined: Fri May 10, 2013 4:42 pm
Posts: 1
If we create a new linode at Fremont, which facility is the linode being created in?


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 5:13 pm 
Offline
Linode Staff
User avatar

Joined: Tue Apr 15, 2003 6:24 pm
Posts: 3112
Website: http://www.linode.com/
Location: Galloway, NJ
FMT1, until we enable the upgrade button - then all new Linodes will go to FMT2.

-Chris


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 5:20 pm 
Offline
Newbie

Joined: Wed May 01, 2013 9:00 am
Posts: 4
Thank you SO MUCH, Chris. :D

That's the kinda thing I would have been happy to hear earlier on -- as I am now. Good luck with the move, glad to hear about servers there already.

That all makes perfect sense. FMT2 is awesome. I like their "dark room" -- it's like putting an amphitheater over a small town then shutting off all the lights.


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 5:26 pm 
Offline
Newbie

Joined: Wed May 01, 2013 9:00 am
Posts: 4
lakridserne wrote:
Valley wrote:
lakridserne wrote:
Solution: Move your Linode to another DC for a few weeks more while they get the last quirks worked out.

Many of us are geographically tied to Fremont.

1) It's not that much more time it takes when we're talking about the internet


Yes, we are taking about the Internet -- and not the web, which seems to be your perspective since you do shared hosting. For other applications where latency is critical (let's use a game server as an example), a difference of even 10ms ping could make all the difference.


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Fri May 10, 2013 9:29 pm 
Offline
Senior Member

Joined: Fri Jan 09, 2009 4:32 pm
Posts: 666
Guspaz wrote:
I really wish that you guys at Linode would post this sort of thing more often. Not necessarily all the time, or in that much detail, but it's nice to see. It's kind of like UI feedback; for a task that takes a given number of seconds, compare the user experience to having a progress bar versus just locking up the UI. The user who gets the progress bar is going to feel like the machine is faster and more response, while the user who just gets a frozen screen for the duration is going to feel the machine is slow and unresponsive.


The problem is that situations like this are often fluid. There's a tough balance to strike between enough information and too much. Had Chris been putting out the updates along the way, I could easily see people misinterpreting pieces of it and getting unrealistic expectations or simply not understanding pieces of it and jumping to conclusions. There's a reason that linode usually doesn't communicate things out until they're done, because then you don't have to deal with threads like this where people are saying "it's been two weeks, where is it!?!?". My day job has plenty of this kind of stuff, so I'm very sympathetic to Chris' position (of course, my linodes are in NJ, so it helps that I'm not personally affected as well).


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Sat May 11, 2013 1:43 pm 
Offline

Joined: Sat May 11, 2013 1:35 pm
Posts: 1
Location: Ladera Ranch, California
I'm pleased to have some info on the Fremont RAM upgrade. I will admit to a little frustration at being in the dark for about a month. AS my company's CEO likes to say, "Our customers can handle good news. They can handle bad news. What they can't handle is no news." I kind of felt like that.

Thanks for the forthcoming upgrade and the news.


Top
   
 Post subject: Re: Fremont RAM upgrade
PostPosted: Sun May 12, 2013 6:57 pm 
Offline
Newbie

Joined: Fri Oct 08, 2010 7:59 am
Posts: 4
Thanks for the update, it's really very much appreciated.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group