Knuth: No.
Seibeclass="underline" Was that just youthful optimism?
Knuth: I was thinking about Simula and trends in object-oriented programming when I wrote that, clearly. I think what happens is that every time a new language comes out it cleans up what’s understood about the old languages and then adds something new, experimental and so on, and nobody has ever come to the point where they have a new language and then they want to stop at what’s understood. They’re always wanting to push further.
Maybe someday somebody will say, “No, I’m not going to be innovative; I’m just going to be clean and simple, and I’m going to stick to it.” Pascal was started with that philosophy but then didn’t continue. Maybe we’ll get to a time when somebody will say, “Let’s set our sights lower and really try to make something that’s going to be stable.” It might be a good idea.
Seibeclass="underline" Isn’t part of the problem that while there are misfeatures, there are also missing features and if a thing is missing you’ve got to make up something to fill that gap.
Knuth: Yeah, that’s right. It’s got to be extensible somehow. Java didn’t make itself extensible in a good way.
Seibeclass="underline" You’ve designed some languages yourself—probably the most widely used of which is TeX.
Knuth: So TeX is a programming language but I had to put in those features kicking and screaming. Guy Steele, Terry Winograd, Leslie Lamport, and different people needed things when they were using TeX as a front end for their material. I think Terry Winograd was writing a book on the syntax of natural languages, so he had some really powerful macros that he wanted to write in order to make the diagrams in his book. That pushed TeX a lot towards becoming a programming language in the earliest days.
Seibeclass="underline" Do you ever wish you had focused more on the design of the language, as a language?
Knuth: I don’t know. I guess so. In a way I resent having every language be universal because they’ll be universal in a different way. It’s a little bit like Unix having 30 definitions of regular expressions under one roof—depending on which part of Unix you’re using you’ve got a slightly different flavor of regular expressions. If every tool that you have includes a Turing machine inside, is this really the way to go? I was really thinking of TeX as something that the more programming it had in it, the less it was doing its real mission of typesetting.
When I put in the calculation of prime numbers into the TeX manual I was not thinking of this as the way to use TeX. I was thinking, “Oh, by the way, look at this: dogs can stand on their hind legs and TeX can calculate prime numbers.”
Seibeclass="underline" But people use the fact that it’s a Turing-complete programming language to do typesetting-related computations. If it wasn’t Turing-complete they would be unable to do those things.
Knuth: Yeah, that’s right. I wrote a programming language for simulation in the ’60s that I had to work hard to kill because it had a lot of users, but then when Simula came out I liked Simula better and I told people to stop using my SOL language. Mostly I don’t consider that I have great talent for language design.
With TeX I was interacting with hundreds of years of human history and I didn’t want to throw out all of the things that book designers have learned over centuries and start anew and say, “Well, forget that guys; you know, we’re going to be logical now.” In this case, the name of the game was mostly to take an enormously complicated problem and find a fairly small set of primitives that would support it. Instead of having 1,000 primitives, I have 100 primitives or something like that. But going down to 50 primitives, 10 primitives—which we would do if we wanted to be mathematically clean—I believe wouldn’t work. The problem of making books goes too much into the complexity of the world, which just doesn’t want to be simplified.
Seibeclass="underline" I haven’t really done a study, but it seems like the vast majority of mathematical and scientific papers are typeset with TeX these days. There must have been things you’ve seen typeset in TeX that made you think, “Wow, my program played a part in this.”
Knuth: Well, the proof of Fermat’s Last Theorem was one of those. It’s one of the most famous mathematical papers. And it happens all the time that I see books that I know wouldn’t have been written if the authors had had to go through channels the way they used to. It’s again a little bit of the black-box thing.
It used to be, you would have to type something up and that would go into a compositor and it would come back in galley proofs, and so on. You’re going through all kinds of levels of people who aren’t mathematicians and then coming out with the product at the end. So you don’t dare do anything that would confuse any of the people in that line.
But if you can see yourself what it’s going to look like, and you can make up a notation that isn’t in somebody’s style sheet because it just happens to be the right one for your problem, then you’re encouraged to do a much better job.
So this brings me great satisfaction all the time when I know that people have been able to cut through this and their creativity goes directly to the reader.
Seibeclass="underline" Do you feel like programmers and computer scientists are aware enough of the history of our field? It is, after all, a pretty short history.
Knuth: There aren’t too many that are scholars. Even when I started writing my books in 1963, I didn’t think people knew what had happened in 1959. I was reading in American Scientist last week about people who had rediscovered an algorithm that Boyer and Moore had discovered in 1980. It happens all the time that people don’t realize the glorious history that we have. The idea that people knew a thing or two in the ’70s is strange to a lot of young programmers.
It’s inevitable that in such a complicated field that people will be missing stuff. Hopefully with things like Wikipedia, achievements don’t get forgotten the way they were before. But I wish I could also instill in more people the love that I have for reading original sources. Not just knowing that so-and-so gets credit for doing something, but looking back and seeing what that person said in his own words. I think it’s a tremendous way to improve your own skills.
It’s very important to be able to get inside of somebody else’s way of thinking, to decode their vocabulary, their notation. If you can understand something about the way they thought and the way they made a discovery, then that helps you make your own discoveries. I often read source materials of what brilliant people have said about this stuff in the past. It’ll be expressed in unusual ways by today’s conventions, but it’s worth it to me to penetrate their notation and to try to get into their idea.
For example I spent a good deal of time trying to look at Babylonian manuscripts of how they described algorithms 4,000 years ago, and what did they think about? Did they have while loops and stuff like this? How would they describe it? And to me this was very worthwhile for understanding about how the brain works, but also about how they discovered things.
A couple of years ago I found an old Sanskrit document from the 13th century that was about combinatorial math. Almost nobody the author knew would have had the foggiest idea what he was talking about. But I found a translation of this document and it was speaking to me. I had done similar kinds of thinking when I was beginning in computer programming.
And so to me reading source materials is great enrichment for my own life and creativity.
I was unable to pass that on to any of my students. There are people alive now in computer science who are doing this well—a few. But I could count on the fingers of one hand the people who love source materials the way I do.