Hi all,

I will soon be expanding with a fair amount of servers at Linode, and I would like to look into configuration management so I can easily expand and (re)-deploy servers. I currently set up my configuration with a stackscript, however if possible I would like to make the change to a configuration management system. I haven't used them yet so regardless of the choice I would have to learn them.

I have seen Puppet, Chef, Ansible, Salt and CFEngine. Currently I have a slight preference for salt because I could use Python scripts there and I know some python. Ansible is not for me since it is limited to 10 servers in the free edition. Puppet seems like the most popular one.

I have the following wishes:

- Push changes to multiple servers (and/or certain server-"groups")
- Rollback changes if something went wrong
- Able to easily set up a new, fresh server with certain variables, install some packages, etc... I believe this is the most basic functionality so this shouldn't be a problem.
- Easy to use (web) interface to see the status of nodes and deploying a new one. Bonuspoints for a link to (or options to built a) link to the Linode API so it can automatically create a new server, log in to it and set it up. (to do this, I believe the "master" must be able to use purely SSH, and I believe Salt has this option (instead of having to install a client on the server first)).

Nothing too special I'd say wishes-wise, but I'm still a bit torn up for what the best option would be. I'm leaning towards Salt closely followed by Puppet simply because it seems to be the standard for this kind of stuff.

Thoughts on this would be awesome!

EDIT: It seems that Ansible is in fact Free - but the use of the web interface (Ansible Tower) requires a license when used with 10+ servers. The pricing is out of my budget though - $2000/yr with up to 50 servers.

When I set up multiple Ubuntu instances, I use these tools, in fact I run them in cron every night for my accounts:

# first set up the accounts to match

dpkg --get-selections > pkgs.txt
update-alternatives --get-selections > pkgalts.txt
crontab -l > userid-cron.txt

Then just reverse them to set up each new instance and rsync the hom dir.

Or some version of the above.

On Linode it would be easier, just boot the target instance using their recovery ISO, mount the target dir say in /mnt then rsync the whole drive, edit IPs in the Apache config and edit /hosts, boot the target and done.

I have experience with both Chef and Puppet. At my last job, we decided to transition from manually managing our ~5 colocated servers to using Chef. It took longer than we expected to write all the necessary recipes for our software stack, but once we finished it made our lives much easier as we added servers and recovered from the occasional hardware failure. I had very little exposure to Ruby when I began working on Chef (I was mainly a PHP/Python guy), but I picked up all I needed to know within a couple weeks. We didn't use Chef for deploying our web app, since it's not well suited for that (we used Jenkins for deployment). Also, we didn't bother with a web UI for monitoring Chef, since we didn't have enough servers to justify that. Overall, Chef worked very well for us, but our needs were pretty basic.

At my current job, I was tasked to introduce a configuration management system for our ~4 manually-managed dedicated severs, and transition to a new host at the same time. I eventually decided on Linode for the new host and Puppet for the configuration management system. There were basically two reasons I went with Puppet: one, I wanted something that's easy to learn for myself and the other people on my team, none of whom had any experience with configuration management. Two, I was hoping that Puppet's relative maturity meant I could reuse modules other people wrote for most of what I needed, and spend less time writing my own code. It's been ~7 months since then, and I think I was wrong on both counts:

• Puppet requires you to write everything in a declarative DSL. It's a very elegant language, but it forces a lot of constraints on you, and takes a long time for non-sysadmins to really grok. Chef uses Ruby for everything and is easier for programmers to understand, since it lets you write things in an imperative style. I spent much longer trying to wrestle Puppet into doing what I wanted than I did with Chef, but the end result with Puppet was usually cleaner and simpler. If I was part of a big team of sysadmins, then that tradeoff might be worth it, but I'm not.
• There's tons of Puppet modules available on https://forge.puppetlabs.com/, but in my experience the average quality isn't great, and most of them require tweaking to do what you want. I've filed about 6 pull requests on Github to fix various bugs I've encountered.

We're using Capistrano for deploying our web app. Like Chef, Puppet isn't well suited for software deployment. For monitoring, we use Puppet Enterprise, which is free for up to 10 nodes. It's very slick, but I kinda wish I had gone with The Foreman instead. Anyway, in retrospect I would've stuck with Chef, but I don't think Puppet was a bad choice. They're both work great pieces of software that can handle pretty much any server configuration you can throw at it.

That's not config management, that's "What packages do I have installed, and what's my crontab?" I suppose that works if your configuration is just the distro-provided defaults, but that's almost never the case. A proper config management setup would allow you to kill the machine (zero the disks, remove the vm, etc.) and delete all the backups of the machine, and still be able to replace it with a functional copy, without needing any manual setup of any kind (except maybe installing the agent for the config management system you're using, but even that can be automated with the OS install).



Salt can do all of that, including creating new Linodes and deploying the minion onto them. I'm almost positive you could tell it to do the minion install from a StackScript, so that you don't even have to deal with SSH at all (minion keys may be an issue in this case; it's worth looking into). A web interface to Salt is in the works, but it's nowhere near finished yet. All of the components of Salt are open source on GitHub, so feel free to contribute improvements .



I've heard a lot of horror stories about writing Puppet manifests, so this must be where they come from. The large setup requirements for a useful Puppet installation is also big turn-off. Personally, I'm not a fan of ruby, so for me, that throws out both Chef and Puppet.

My main complaint with Ansible is that it chiefly uses SSH, which doesn't scale very well with large numbers of servers. Salt seems to solve all of my major requirements, and easily scales to tens to hundreds of thousands of servers without issue. There's a lot of really intelligent design decisions that go into Salt every day, and they give you more than enough rope to hang yourself with, which is a good thing.

At the end of the day, you should probably try them all, and see which one works best for your needs.

_________________
Disclaimer: I am no longer employed by Linode; opinions are my own alone.

dwfreed:

>> That's not config management, that's "What packages do I have installed, and what's my crontab?" I suppose that works if your configuration is just the distro-provided defaults, but that's almost never the case.

1. I looked at the configuration management tools. They suck.
2. We agree, Ruby sucks.
3. You're here to boost your Salt efforts, you trash all the other tools (and so do I because they suck) then say [but here try this one, it's great because I work on it]. Bleh.

Sorry, but "I think the config management tools suck" does not mean "Lets pretend some big file lists are config management". Also, if personal experience isn't a valid basis for giving recommendations, what are any of us doing here?

- Les

The free donuts?

_________________
Either provide enough details for people to help, or sit back and listen to the crickets chirp.
Security thru obscurity is a myth - and really really annoying.

We get doughnuts? Since when? Can I claim a back pay on doughnuts?!

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

Care to provide substantiation for this? It's one thing to say something sucks, but it's another thing entirely to prove why it sucks.



Um, no. I have 2 commits to all of Salt, and they're both really small things (PRs: https://github.com/saltstack/salt/pull/10676 and https://github.com/saltstack/salt/pull/10672). I suggest Salt because I've looked at it, and it really does solve all of my problems. I'm not an employee of SaltStack, nor am I a large contributor to Salt. It's a good project, and I like it, so I'd like to see it grow, and I recommend it to those where it'd be useful to them. Salt may not work out for Nuvini, but that's for him to determine, not me. I'm just offering it as a suggestion, and indicating which of his requirements it can fulfill.

_________________
Disclaimer: I am no longer employed by Linode; opinions are my own alone.

Have you looked at Ansible?

I had become very frustrated with both Puppet and Chef as I built up our CM environment. Then I found Ansible, and it's been an absolute joy to work with. It's not without its problems, but it is far easier to learn, far easier to write manifests for, and much more reasonable, not forcing you into a strictly declarative language. Ansible leaves ample room for ad-hoc tasks as well.

It's a relative new kid on the config management block, but it has a very large, engaged, and supportive community, and an ever-growing library of modules and publicly-shared roles that anyone can use.

Anyway, for anyone frustrated with the "legacy" CM players (CFEngine, Puppet, Chef, etc.), or for anyone just getting into CM, I'd highly recommend taking a look at Ansible.

_________________

I don't think I have tried it yet, thanks. Might give it a go here in the next few days.

http://www.ansible.com/how-ansible-works

This I like:

"MANAGE YOUR INVENTORY IN SIMPLE TEXT FILES"

Also, the fact that it doesn't require *anything* on the client side other than sshd and a python interpreter. No more puppet agent processes taking up valuable RAM and CPU cycles.

_________________

Thanks for all the responses. Right now it seems that the best option for me would be Ansible or Salt. Using ruby is a little bit off-putting for me so syntax wise I would probably be more comfortable with Ansible or Salt. I like the interface Salt has to offer (Halite) even though it does still seem to be very much in development. I will be using more than 10 servers which will give me some issues at Ansible.

I'll try both of them out, and hopefully one of them suits my needs (even if with CLI-only ). If not I'll try Chef/Puppet out as well, but I'm hoping that won't be needed.

It's good to know rollbacks (At least with Salt) won't be an issue either.

I won't be able to fully automate it though with the Linode deployment itself (Creating new Linodes with stackscripts to install the minion) though, since I will be using failovers, so I have to wait for the failover IPs to be assigned first

Either way, thanks for the great responses, looking forward to diving into configuration management