Seibeclass="underline" Another thing that came out of working on TeX, which you described in “The Errors of TeX,” was a log of every error that you found in the program. Folks like the Software Engineering Institute people say that part of a mature software-engineering process is keeping track of all your bugs and learning how to prevent the same kind of errors in the future. But you said that having kept this log, it doesn’t help you prevent future errors.
Knuth: Yeah. Though it’s hard to say that I wouldn’t have been even worse without the log.
Seibeclass="underline" But you didn’t feel like, “Ah, now that I’ve seen this I won’t do it again.”
Knuth: I just got to recognize my sins. People keep coming back for absolution, if you know theological terms.
Seibeclass="underline" So you find yourself now making bugs in your programs and then saying, “Oh, I’ve done it again, that same kind of bug.”
Knuth: Yeah.
Seibeclass="underline" So why is that? Is there something about the nature of the mistakes that makes it hard to distill a lesson that will prevent making them again?
Knuth: I think it’s probably more that I’ll try harder things. I always try things that are at my limit. If I had to go back and write those kinds of programs again, the easier ones, I wouldn’t make so many mistakes. But now that I know some more, I’m trying to write harder stuff. So I make mistakes because I’m always operating at my limit. If I only stay in comfortable territory all the time, that’s not so much fun.
Seibeclass="underline" So if you just kept writing typesetting systems for the rest of your life?
Knuth: Yeah, I would get those pretty good. But we keep raising the bar and then we stumble on it. We’re dealing with—as we said earlier—things that are on the edge of what human beings can handle and more complicated than have been done before.
If we restrict ourselves to the things that are really easy, then that’s not satisfactory because our appetite is always to push the boundary and go until it gets to something we can barely do. And once we’ve got to there, then we’re going to want to push that boundary and so on.
So inevitably we’re going to have bugs unless we decide we’re never going to write anything that stretches our capabilities. So how are we going to do it better? Every three years there’ll be another buzz word as to something that’s going to solve all these problems and make it really work. Extreme programming was one the last two or three years. Before that there was something else. Somebody will come up with another supposedly silver bullet and there’ll be a lot of people jumping on that bandwagon and then they’ll find, “Oh, it’s still hard.”
Seibeclass="underline" Has the kind of person who can be a good programmer changed over time?
Knuth: Pretty much a constant in my experience, over a long period of years, is that every time I’m exposed to 100 people from some population or other, except majors in computer science, 2 of them are programmers in the sense that they really resonate with the machine. Wasilla, Alaska, has 10,000 people, so it’s probably got 200 programmers.
Seibeclass="underline" So has programming changed enough that the kind of person who falls in that two percent has changed? Or is it still really the same?
Knuth: I don’t know—you can use the word programming in different senses. We’re always making tools that are intended to make more of a match between people’s brains and getting something done in a computer. I’m mostly talking about the way a machine really works when the machine is being pushed to the envelope rather than just getting an answer out.
We’ve got machines that are so powerful now that people who aren’t really good at programming, in my esoteric sense, are able to get answers out of these machines that would have taken a huge expert to do on old machines. But with the new machines, the people that I’m talking about are going to be doing the problems that couldn’t be handled by the old machines.
So there’s that change and then there’s the change that I’m really worried about: that the way a lot of programming goes today isn’t any fun because it’s just plugging in magic incantations—combine somebody else’s software and start it up. It doesn’t have much creativity. I’m worried that it’s becoming too boring because you don’t have a chance to do anything much new. Your kick comes out of seeing fun results coming out of the machine, but not the kind of kick that I always got by creating something new. The kick now is after you’ve done your boring work then all of the sudden you get a great image. But the work didn’t used to be boring.
Seibeclass="underline" But you still find the kind of programming you do interesting?
Knuth: Oh my God, yes. I’ve got this need to program. I wake up in the morning with sentences of a literate program. Before breakfast—I’m sure poets must feel this—I have to go to the computer and write this paragraph and then I can eat and I’m happy. It’s a compulsion; that I have to admit.
OK, let me show you the program I wrote yesterday. I’m multiplying huge integers that are way bigger than the universe—they’re special integers that you can compress the representation down, and so I can deal with them even though I couldn’t represent them in an ordinary notation, and I’ve been multiplying these integers that are inconceivably large and I’ve been squaring them and finding out how they look after squaring them. I’m very puzzled about what’s going on, but this is exciting to me.
Seibeclass="underline" You’re an academic but also have worked on big systems and have done some work in industry. How do you see the relation between academic computer science and industrial practice?
Knuth: It’s gone in waves. In the ’60s the academics were way ahead of the industry and the programs that were produced in industry, except for maybe airline-reservation systems, were laughable to everybody in universities.
By 1980 the situation had pretty much reversed and the programs that were being written by people in universities were laughed at by the people in industry because the universities had gone into theological mode and you weren’t allowed to use goto statements. I’m exaggerating to simplify, but basically there were no-nos in university programs that were keeping people’s hands tied, and the people in industry didn’t have to worry about that.
But then in universities people came up with some better ideas about networking and dealing with large pieces of data and so on, and got ahead. So it goes back and forth. But the trend in a lot of the algorithm and datastructure community has not been to my liking when they have lots of data structures that are just… baroque is the only word I can think of. They’re intricate and clever and you have to admire them for the intellectual challenge, but I find them sterile. They don’t connect with life; they’re working in another world. It’s an OK world and it’s got its structure, and they’re friendly and nice people, but it doesn’t appeal to me personally and it doesn’t really relate to practice.
I don’t know why it’s important to me if something relates to practice or not. There are mathematicians who never think about anything finite, and they hardly ever come down to countably infinite—they publish terrific papers just talking about kinds of infinity that are mind-boggling and they’re able to make sense out of it and that gives them satisfaction. And there are similar things like that in algorithms. But for me I’m turned on much more by the ideas that I would be able to use in my machine.
Seibeclass="underline" In 1974 you said that by 1984 we would have “Utopia 84,” the sort of perfect programming language, and it would supplant COBOL and Fortran, and you said then that there were indications that such language is very slowly taking shape. It’s now a couple of decades since ’84 and it doesn’t seem like that’s happened.