(defun test-+ ()
(and
(= (+ 1 2) 3)
(= (+ 1 2 3) 6)
(= (+ -1 -3) -4)))
Whenever you want to run this set of test cases, you can call test-+.
CL-USER> (test-+)
T
As long as it returns T, you know the test cases are passing. This way of organizing tests is also pleasantly concise—you don't have to write a bunch of test bookkeeping code. However, as you'll discover the first time a test case fails, the result reporting leaves something to be desired. When test-+ returns NIL, you'll know something failed, but you'll have no idea which test case it was.
So let's try another simple—even simpleminded—approach. To find out what happens to each test case, you could write something like this:
(defun test-+ ()
(format t "~:[FAIL~;pass~] ... ~a~%" (= (+ 1 2) 3) '(= (+ 1 2) 3))
(format t "~:[FAIL~;pass~] ... ~a~%" (= (+ 1 2 3) 6) '(= (+ 1 2 3) 6))
(format t "~:[FAIL~;pass~] ... ~a~%" (= (+ -1 -3) -4) '(= (+ -1 -3) -4)))
Now each test case will be reported individually. The ~:[FAIL~;pass~] part of the FORMAT directive causes FORMAT to print "FAIL" if the first format argument is false and "pass" otherwise.[102] Then you label the result with the test expression itself. Now running test-+ shows you exactly what's going on.
CL-USER> (test-+)
pass ... (= (+ 1 2) 3)
pass ... (= (+ 1 2 3) 6)
pass ... (= (+ -1 -3) -4)
NIL
This time the result reporting is more like what you want, but the code itself is pretty gross. The repeated calls to FORMAT as well as the tedious duplication of the test expression cry out to be refactored. The duplication of the test expression is particularly grating because if you mistype it, the test results will be mislabeled.
Another problem is that you don't get a single indicator whether all the test cases passed. It's easy enough, with only three test cases, to scan the output looking for "FAIL"; however, when you have hundreds of test cases, it'll be more of a hassle.
Refactoring
What you'd really like is a way to write test functions as streamlined as the first test-+ that return a single T or NIL value but that also report on the results of individual test cases like the second version. Since the second version is close to what you want in terms of functionality, your best bet is to see if you can factor out some of the annoying duplication.
The simplest way to get rid of the repeated similar calls to FORMAT is to create a new function.
(defun report-result (result form)
(format t "~:[FAIL~;pass~] ... ~a~%" result form))
Now you can write test-+ with calls to report-result instead of FORMAT. It's not a huge improvement, but at least now if you decide to change the way you report results, there's only one place you have to change.
(defun test-+ ()
(report-result (= (+ 1 2) 3) '(= (+ 1 2) 3))
(report-result (= (+ 1 2 3) 6) '(= (+ 1 2 3) 6))
(report-result (= (+ -1 -3) -4) '(= (+ -1 -3) -4)))
Next you need to get rid of the duplication of the test case expression, with its attendant risk of mislabeling of results. What you'd really like is to be able to treat the expression as both code (to get the result) and data (to use as the label). Whenever you want to treat code as data, that's a sure sign you need a macro. Or, to look at it another way, what you need is a way to automate writing the error-prone report-result calls. You'd like to be able to say something like this:
(check (= (+ 1 2) 3))
and have it mean the following:
(report-result (= (+ 1 2) 3) '(= (+ 1 2) 3))
Writing a macro to do this translation is trivial.
(defmacro check (form)
`(report-result ,form ',form))
Now you can change test-+ to use check.
(defun test-+ ()
(check (= (+ 1 2) 3))
(check (= (+ 1 2 3) 6))
(check (= (+ -1 -3) -4)))
Since you're on the hunt for duplication, why not get rid of those repeated calls to check? You can define check to take an arbitrary number of forms and wrap them each in a call to report-result.
(defmacro check (&body forms)
`(progn
,@(loop for f in forms collect `(report-result ,f ',f))))
This definition uses a common macro idiom of wrapping a PROGN around a series of forms in order to turn them into a single form. Notice also how you can use ,@ to splice in the result of an expression that returns a list of expressions that are themselves generated with a backquote template.
With the new version of check you can write a new version of test-+ like this:
(defun test-+ ()
(check
(= (+ 1 2) 3)
(= (+ 1 2 3) 6)
(= (+ -1 -3) -4)))
that is equivalent to the following code:
(defun test-+ ()
(progn
(report-result (= (+ 1 2) 3) '(= (+ 1 2) 3))
(report-result (= (+ 1 2 3) 6) '(= (+ 1 2 3) 6))
(report-result (= (+ -1 -3) -4) '(= (+ -1 -3) -4))))
Thanks to check, this version is as concise as the first version of test-+ but expands into code that does the same thing as the second version. And now any changes you want to make to how test-+ behaves, you can make by changing check.
Fixing the Return Value
You can start with fixing test-+ so its return value indicates whether all the test cases passed. Since check is responsible for generating the code that ultimately runs the test cases, you just need to change it to generate code that also keeps track of the results.
As a first step, you can make a small change to report-result so it returns the result of the test case it's reporting.
(defun report-result (result form)
(format t "~:[FAIL~;pass~] ... ~a~%" result form)
result)
Now that report-result returns the result of its test case, it might seem you could just change the PROGN to an AND to combine the results. Unfortunately, AND doesn't do quite what you want in this case because of its short-circuiting behavior: as soon as one test case fails, AND will skip the rest. On the other hand, if you had a construct that worked like AND without the short-circuiting, you could use it in the place of PROGN, and you'd be done. Common Lisp doesn't provide such a construct, but that's no reason you can't use it: it's a trivial matter to write a macro to provide it yourself.
Leaving test cases aside for a moment, what you want is a macro—let's call it combine-results—that will let you say this: