Agile sysadmin and the art of infrastructure automation
I recently gave a “Devops’-flavoured talk at the London Ruby Users Group on agile sysadmin practices, agile web ops, configuration languages and infrastructure automation. The slides are below, and you can also listen to an audio recording.
The idea of the talk was to look at modern sysadmin practices such as version control, configuration management and automation, and to show developers that with languages like Puppet, infrastructure is just code. We’re all developers now.
Version control
The benefits of version control with tools like CVS, Git and Subversion are well-known in software development:
You have a complete history of your project
Separate teams and developers can have their own working copies of the code, and then re-integrate them for testing
You can use branches to develop speculative or experimental features without affecting the main trunk
You can use branches to maintain and patch old releases
You can roll back to known good versions when things don’t work
All changes are logged with metadata of who made the change, and most importantly, why
What’s becoming increasingly clear thanks to devops conferences such as DevOpsDays is that these benefits apply equally to the systems and infrastructure development that sysadmins do. In the bad old days, sysadmins would make changes directly on the server, which were often undocumented and irreversible. Now with properly managed version control and central repositories, sysadmins can exercise the same degree of control over their ‘code’ as the development team.
Configuration management
The idea of configuration management is that all the information that makes a server useful for some specific task - being a web server, for example, or a working database - is stored in some central place and automatically distributed to the server.
For example, all the work that a sysadmin might do manually to build up a newly provisioned Linux server, such as installing Apache, adding virtual host configuration, setting up firewall rules, installing PHP with a special config to support some application, and so on - all this can be brought under automatic management.
Puppet’s client-server architecture is ideal for managing configuration across multiple servers - with no theoretical upper limit to the number of machines that can be controlled this way. There are sites managing thousands of servers with Puppet!
Infrastructure automation
Once you have version control and configuration management in place, it is possible to automate most of the tedious, repetitive tasks which consume sysadmins’ time and energy. For example, building new servers, application deployment, continuous integration, clustering, monitoring and backups can all be automated and managed centrally. Agile web ops means using automation to reduce the cost of changes to your existing infrastructure.
This has many benefits:
Operations staff are happier
You can provide resources more quickly to developers and business
Your servers are more reliable and robust
You have a change control process, which improves security and accountability
You can scale to tens or hundreds of servers which would be impossible to set up and maintain by hand
The most important benefit for the sysadmin is that she can devote much more time to her other important job roles:
Network architect
Budget manager
Performance engineer
Security analyst
Development liaison
Teacher and team leader
Finally, because sysadmins are now developers, they are working alongside the other developers in the organisation on an equal footing, and with an equal interest in making development and deployment smooth, easy, reliable and fun. Programming skills and good engineering practices can spread throughout the technical team, and developers and sysadmins can come together with an attitude of respect, sharing and co-operation.
That’s good news for everybody!
Find out more
Puppet books
I’ve written a comprehensive Puppet Beginner’s Guide book to get you started with Puppet, and for more advanced topics there’s also the Puppet Cookbook.
Other tools and resources
Agile sysadmin and consultant Patrick Debois has a very comprehensive review of the current state of the art in automated provisioning and configuration management. This is required reading for anyone interested in getting started with infrastructure automation:
