what about this?

https://vpsboard.com/topic/3282-oh-no-l ... get-raped/

I believe there's a lot of uncertainty at this point. Some are saying it's really old data.

Either way, it would be great if an employee could clear this up and let us know what it's about. It took quite a long time last time before we got any clarification.

Indeed. There was no clarification on what happened at all last time and it caused great upset.

This looks like someone pulling old data off a Linode owned server with an old, and leaked, password. Surely that can't be what actually happened here?

My gut feeling is to end my 10-year relationship with linode.

I'm really beginning to think linode just don't grok security at all. After the last hack they took security seriously (caker says as much in the blog post), but it seems they don't actually _get_ it. You can spend a million dollars and not close any security holes if you don't get the problem. See the TSA as a perfect example of this.

One is happenstance.
Twice is incompetence.

or

Fool me once, shame on you
Fool me twice, shame on me

This may just be an "oh God, again?" feeling and I'll change my mind tomorrow.

The linode infrastructure and product offering is world class. My happiness quotient with the service is around the 95% level (spam and HE datacenter failures bringing it down). But c'mon guys; security!

_________________
Rgds
Stephen
(Linux user since kernel version 0.11)

Linode - Clarification needed, us old timers are getting tetchy.

Is this perhaps what you wanted?

It was posted earlier today.

https://blog.linode.com/2014/01/19/an-o ... swat-team/

Yes ... and no. I wanted reassurance from their announcement but I did not get it.

The SWAT thing is a red herring.

That doesn't fully answer my concerns. Why was Linode running an instance of mysql that was accessible from the Internet? Good practice is to tell mysql to only bind internal network addresses, block the mysql port with iptables from non-local IPs, and block non-local IPs with a hardware firewall if possible.

If this database wasn't part of their infrastructure why was it hosting an old copy of authentication data? This data may have leaked already but that's no reason to serve it up to anyone who didn't get a copy the first time around so that script kiddies can troll though it looking for password reuse on other sites.

Why was this database still using a password that could be found by a google search for 'linode hack'? Why was a 7 char password considered sufficient for any Internet facing system anyway? I use WAY longer passwords on things that are considerably less important. Why was any authentication allowed over the Internet in clear text? If you pass it in clear text over the Internet you should consider it public information.

Security isn't easy but this wasn't a zero day exploit, or anything advanced or unexpected. This was carelessness.

As caker points out in the blog post, the system was not part of Linode infrastructure, nor did it really have much in the way of private data. As it's not part of Linode infrastructure, it can, for all intents and purposes, be considered a personal system. The phpBB restore was from 2009 or 2010, and anybody who hasn't logged into the Linode forums in that time probably won't be logging in for a long time anyway. Yes, from a security standpoint, it was perhaps a bit careless, but at the same time, everybody was focusing on securing Linode infrastructure, and protecting your real personal information and private data. I don't see why you're making such a big deal out of something that poses no real threat to your security or that of Linode.

-Doug

Disclaimer: I am no longer employed by Linode; opinions are my own alone.

I noticed you haven't used your garden shed for several months, so I figured it was ok if I just took some of the stuff in there - right?

If the data was useless, why was it stored at all. And saying that it was old and not part of Linodes infrastructure means it should have been destroyed, not put on a non-secure box.

With all the knuckle-headed stuff that Linode does, it might be time to start taking EVERYTHING way more seriously - or people are going to get fed up and start leaving in droves.

If you can't figure out security, at least hire a freaking PR person to spin crap like this so that you don't look like bumbling dolts to the people who pay to keep you in business.

Lack of transparency, unclear and untimely responses, and the whole C'est la vie attitude is cute in some basement startup run by high school kids, it's not a brand image we want or expect from a professional hosting service.

_________________
Either provide enough details for people to help, or sit back and listen to the crickets chirp.
Security thru obscurity is a myth - and really really annoying.

Any system that contains customer data - any customer data, including "just forum accounts" - cannot be considered a "personal" system and should either be protected securely or wiped clean of customer data.

And you know that leak poses no real security threat to me how? Did you crack my password hash and try it on every Internet site out there? Did you do that for every listed user? No you didn't.

I don't reuse passwords but I wasn't always so careful, and I don't remember every single site I ever created a login on. Lots of people do reuse passwords and lots of people never change them, or dumbly change just one number at the end.

That server was under either Caker's personal, or Linode's control. Here is the proof:

;; AUTHORITY SECTION:
theshore.net.           10789   IN      SOA     ns1.linode.com. caker.theshore.net. 2014012050 14400 14400 1209600 86400

Oh look, the DNS got changed today. Presumably to hack newnova.theshore.net out of existence. This whole thing was not 'perhaps a bit careless', it was 'careless'.


EDIT: Da' fuq is this? http://web.archive.org/web/20140110234644/http://theshore.net/. It looks like that should have been retired 10 years ago, not served up 11 days ago. Frontpage '98, seriously?

And that's the problem; it _should_ have been considered part of the infrastructure. That's where all this new super-duper security caker mentioned falls down; it was incorrectly scoped. Every machine owned by linode or containing linode data should be considered in scope and evaluated for risk.

Except for user passwords... oops!

_________________
Rgds

Stephen

(Linux user since kernel version 0.11)

Yeah, how dare the founder of Linode keep sentimental history, the predecessor to what you know as Linode, online and functional as a memento. The hell is the matter with him?

With respect, reading your messages in this thread is tiring. I'm sure that every reader understands that you're upset, so you've successfully communicated that perspective. Now drain the emotion from the pool. When you start resorting to knocking Frontpage on a piece of history from the last millennium you've crossed from "has a good point" to "please step away and drink some tea" territory.

I used to fly off the handle about poo poo like this too. Then I realized, I can throw my hands up, bitch, moan, shout a bunch and get angry at people on the phone -- as I used to do on behalf of your tickets with upstream providers (and as Chris watched me do once with the office's Internet provider, a memory I regret) -- or I can approach the situation calm, reasonably, and get something done. The latter has proven far more successful in my career and earned me respect from unexpected places. I've watched people who remind me of the kind of person I was when I worked at Linode end their careers very quickly by shouting invective at the wrong people. All over computers breaking.

At the end of the day, you are dealing with security concerns at a company with which you do business. What, do you think, is the best way to handle that? Hint: it's not angrily arguing. Save the Tums for spicy food, not online drama.

Based on casual conversations with various folks that I've had since my departure, Linode has moved leaps and bounds toward better security. Undoubtedly, they still have a long way to go, but don't forget that in our line of work you're never really finished. If you want to bail, nobody is stopping you. Knowing management, every single message is read, as well; you've gotten your message across and while I remember a culture of dismissing many customer opinions, I genuinely believe that has gotten far better as well. I am not paid for that opinion, and it should say something that even though I moved on to greener pastures in terms of my career I still have many positive opinions about Linode as a company and service provider.

As an additional point, DigitalOcean and Linode have both struggled with security concerns and it has been interesting to contrast the reaction from each.

_________________
Disclaimer: I am no longer employed by Linode; opinions are my own alone.

Thanks for your input jed but ignoring the message and complaining about the way it's communicated is nothing but distraction.

My point restated as clearly as possible:

Leaking that forum data again was not 'a' misconfiguration but a whole string of them caused by an absence of security in layers. Only an internal address should have been bound by mysql, iptables should have been blocking that port from unexpected addresses, the database should not have been using a password that had already leaked, that database should not have been on that machine anyway. It would have only taken one one of those measures and this data would not have leaked. Claiming that database is not part of Linode's infrastructure is no excuse when it's a Linode owned server containing data Linode is responsible for.

I'm not the only one disappointed by this; sweh, Main Street James, and vonskippy are too.