Practical Common Lisp

:printer (make-instance 'indenting-printer :out *html-output*))))

With this function, you can emit HTML to *html-output*. Rather than expose the variable *html-output* as part of FOO's public API, you should define a macro, with-html-output, that takes care of binding the stream for you. It also lets you specify whether you want pretty HTML output, defaulting to the value of the variable *pretty*.

(defmacro with-html-output ((stream &key (pretty *pretty*)) &body body)

`(let* ((*html-output* ,stream)

(*pretty* ,pretty))

,@body))

So, if you wanted to use emit-html to generate HTML to a file, you could write the following:

(with-open-file (out "foo.html" :direction output)

(with-html-output (out :pretty t)

(emit-html *some-foo-expression*)))

What's Next?

In the next chapter, you'll look at how to implement a macro that compiles FOO expressions into Common Lisp so you can embed HTML generation code directly into your Lisp programs. You'll also extend the FOO language to make it a bit more expressive by adding its own flavor of special operators and macros.

31. Practicaclass="underline" An HTML Generation Library, the Compiler

Now you're ready to look at how the FOO compiler works. The main difference between a compiler and an interpreter is that an interpreter processes a program and directly generates some behavior—generating HTML in the case of a FOO interpreter—but a compiler processes the same program and generates code in some other language that will exhibit the same behavior. In FOO, the compiler is a Common Lisp macro that translates FOO into Common Lisp so it can be embedded in a Common Lisp program. Compilers, in general, have the advantage over interpreters that, because compilation happens in advance, they can spend a bit of time optimizing the code they generate to make it more efficient. The FOO compiler does that, merging literal text as much as possible in order to emit the same HTML with a smaller number of writes than the interpreter uses. When the compiler is a Common Lisp macro, you also have the advantage that it's easy for the language understood by the compiler to contain embedded Common Lisp—the compiler just has to recognize it and embed it in the right place in the generated code. The FOO compiler will take advantage of this capability.

The Compiler

The basic architecture of the compiler consists of three layers. First you'll implement a class html-compiler that has one slot that holds an adjustable vector that's used to accumulate ops representing the calls made to the generic functions in the backend interface during the execution of process.

You'll then implement methods on the generic functions in the backend interface that will store the sequence of actions in the vector. Each op is represented by a list consisting of a keyword naming the operation and the arguments passed to the function that generated the op. The function sexp->ops implements the first phase of the compiler, compiling a list of FOO forms by calling process on each form with an instance of html-compiler.

This vector of ops stored by the compiler is then passed to a function that optimizes it, merging consecutive raw-string ops into a single op that emits the combined string in one go. The optimization function can also, optionally, strip out ops that are needed only for pretty printing, which is mostly important because it allows you to merge more raw-string ops.

Finally, the optimized ops vector is passed to a third function, generate-code, that returns a list of Common Lisp expressions that will actually output the HTML. When *pretty* is true, generate-code generates code that uses the methods specialized on html-pretty-printer to output pretty HTML. When *pretty* is NIL, it generates code that writes directly to the stream *html-output*.

The macro html actually generates a body that contains two expansions, one generated with *pretty* bound to T and one with *pretty* bound to NIL. Which expansion is used is determined by the runtime value of *pretty*. Thus, every function that contains a call to html will contain code to generate both pretty and compact output.

The other significant difference between the compiler and the interpreter is that the compiler can embed Lisp forms in the code it generates. To take advantage of that, you need to modify the process function so it calls the embed-code and embed-value functions when asked to process an expression that's not a FOO form. Since all self-evaluating objects are valid FOO forms, the only forms that won't be passed to process-sexp-html are lists that don't match the syntax for FOO cons forms and non-keyword symbols, the only atoms that aren't self-evaluating. You can assume that any non-FOO cons is code to be run inline and all symbols are variables whose value you should embed.

(defun process (processor form)

(cond

((sexp-html-p form) (process-sexp-html processor form))

((consp form) (embed-code processor form))

(t (embed-value processor form))))

Now let's look at the compiler code. First you should define two functions that slightly abstract the vector you'll use to save ops in the first two phases of compilation.

(defun make-op-buffer () (make-array 10 :adjustable t :fill-pointer 0))

(defun push-op (op ops-buffer) (vector-push-extend op ops-buffer))

Next you can define the html-compiler class and the methods specialized on it to implement the backend interface.

(defclass html-compiler ()

((ops :accessor ops :initform (make-op-buffer))))

(defmethod raw-string ((compiler html-compiler) string &optional newlines-p)

(push-op `(:raw-string ,string ,newlines-p) (ops compiler)))

(defmethod newline ((compiler html-compiler))

(push-op '(:newline) (ops compiler)))

(defmethod freshline ((compiler html-compiler))

(push-op '(:freshline) (ops compiler)))

(defmethod indent ((compiler html-compiler))

(push-op `(:indent) (ops compiler)))

(defmethod unindent ((compiler html-compiler))

(push-op `(:unindent) (ops compiler)))

(defmethod toggle-indenting ((compiler html-compiler))

(push-op `(:toggle-indenting) (ops compiler)))

(defmethod embed-value ((compiler html-compiler) value)

(push-op `(:embed-value ,value ,*escapes*) (ops compiler)))

(defmethod embed-code ((compiler html-compiler) code)

(push-op `(:embed-code ,code) (ops compiler)))

With those methods defined, you can implement the first phase of the compiler, sexp->ops.

(defun sexp->ops (body)

(loop with compiler = (make-instance 'html-compiler)

for form in body do (process compiler form)

finally (return (ops compiler))))

During this phase you don't need to worry about the value of *pretty*: just record all the functions called by process. Here's what sexp->ops makes of a simple FOO form: