I like to build three layers when I automate something for other people:
Layer 1. A program that does the basic task.
Layer 2. A program that the user will run, with sudo, that collects her input, validates it, makes sure she isn't trying to do anything fishy, and then calls the first program.
Layer 3. A more user-friendly way to access these previous layers, such as a web interface or menu program.
For example, at one company, we had a process for pushing a new version of the company web site to the world. It involved three different web servers (actually they were virtual servers on two different machines, but those details aren't important).
www-draft.example.com
The next release of our web site was developed here.www-qa.example.com
The draft site would be copied here for QA to check over. Once the copy was made, the files were immediately made read-only. If QA approved this site, we needed to be able to verify that these exact bits were copied to the live site.www.example.com
This was the live site that external people would see.
The web designers would ask the system administrators to copy their draft to www-qa.example.com. When the QA group approved the site, they would tell the system administrators to make the site go live.
Each of these two functions was automated:
readyforqa
Copied the draft site to the QA site.golive
Copied the QA site to the live site.
Marketing demanded a way to make emergency updates when the QA department wasn't available. We created this command:
emergency-draft-to-live
Copied the draft site directly to the live site after asking "Are you sure?" a few times.
These three scripts comprised Layer 2, which I mentioned earlier. Layer 1 was a script that did the actual copying of one site's data to another site, making a backup along the way, and setting files to read-only (changing the ownership of the files, too). Layer 1 had to be done as root because it was changing ownership of files and accessing machines via secured channels.
sudo was programmed as described in Table 13-1.
Table 13-1. Web update permission table
Web developers
QA
Marketing
Readyforqa
X
X
Golive
X
Emergency-draft-to-live
X
We actually went through the effort of having management sign off on this chart, with real signatures, to make sure they understood that they were agreeing to what they thought they were agreeing to. The political process to get this approved was the difficult part. It took weeks. Presenting the information to management in the chart form made it a lot easier for a decision to be made. They could understand and update the chart themselves until they were happy with it. Translating the final chart into a sudo configuration file was the easy part.
Per Layer 3, we decided to make an easier way for people to access these commands. We considered a web interface, but, in this case, the users were satisfied with a menu program that presented them with a list of options that ran the appropriate command.
The menu ran without any additional privileges (i.e., not under sudo), but called the Layer 2 programs using sudo as needed .
Summary
Automation is great because it saves you time. It also permits you to push work to other, less-technical people.
There are four types of problems that SAs typically deal with:
Simple things done once
Hard things done once
Simple things done often
Hard things done often
"Hard things done once" and "Simple things done often" are the right things to try to automate. "Hard things done often," while tempting, is usually better served by off-the-shelf packages (commercial or free).
To automate a process, first be sure you can do the steps manually. Then document each step, and make sure that you can automate each step. Then bring all of the steps together.
You can save a lot of typing time by making aliases. This is true for command-line systems as well as for applications, such as SSH. Set the alias as close to the actual application as possible. For example, setting the alias in the SSH configuration file means all systems that leverage SSH will use the alias.
The Unix/Linux make command is extremely powerful. It is not just for programmers. You can use it to automate system administration tasks. On Unix/Linux systems, especially servers, standardize on having a Makefile in /etc that automates common tasks such as reindexing aliases, cloning data, and so on.
Bash and /bin/sh shell languages are more sophisticated and powerful than you may realize. The examples in this chapter reveal how to parse command-line options and even how to write a small malware detector!
When writing a long command line, test each part as you write it.
When writing code for other people, the user interface becomes more important. There are tricks and techniques to creating useful user interfaces. You can avoid the issue by shifting all data entry to a program like MS Excel or by providing a menu system or web interface that lets people access higher-privileged systems.
When writing code to let users do privileged operations, be extremely careful. Build on security tools that already exist and have good credibility, such as sudo. Use a permission table to explain to management who will have access to what. It is their job to manage risk and your job to help them understand the issues. Get approval before you deploy the system.
As you move through your career, you will find yourself automating more and more tasks. It is a good idea to learn a programming language suited for system administration functions, such as Perl, Python, Ruby, or Shell, as well as operating-specific techniques like those featured in the O'Reilly Cookbook series mentioned previously in this book.
Appendix A. Epilogue
Congratulations. You've made it all the way to the end of the book. So now what?
First, I recommend you reread the book. We learn through repetition. If you reread (or skim) the book while the topics are fresh in your mind, it will have a strong impact.