Allegro CL's CLOS and Domain-Specific Languages

January 1997

---

One of the key strengths of Common Lisp/CLOS is its extensibility, enabling the language to be "custom-tailored" to domain-specific application areas.

For example, see GBB by Blackboard Technology. This company's highly successful blackboard application development environment, GBB (Generic Blackboard Builder), is essentially an extension of Common Lisp/CLOS specifically designed for the blackboard systems domain. Other examples include game development languages created "on top" of Common Lisp/CLOS, such as Naughty Dog Inc.'s GOOL (Game Object Oriented Lisp) and Nichimen Graphics' N-World system.

With this article we take a look at this important characteristic of Common Lisp/CLOS, and investigate how it can be leveraged to create highly flexible, domain-specific applications with all of the powerful Dynamic Objects capabilities inherent in Common Lisp and the Common Lisp Object System.

A "Programmable Programming Language"

When John Foderaro, key software engineer on the Allegro CL team, described the Lisp language as a "programmable programming language," he was referring to Lisp's unique extensibility. You can add your own functions, control constructs and data types to the language, turning it into the kind of language you need for your particular project. As Paul Graham has written "If you're writing a text-editor, you can turn Lisp into a language for writing text-editors. If you're writing a CAD program, you can turn Lisp into a language for writing CAD programs. And if you're not sure yet what kind of program you're writing, it's a safe bet to write it in Lisp." (Paul Graham, On Lisp, page 2).

This kind of extensibility is becoming more important as software grows in complexity. Giving sophisticated users software that performs all their needs right out of the box is becoming harder and harder to do. However, it's quite possible with Lisp to give them extensible software that can change dynamically to meet their particular requirements, and that can evolve to reflect their changing requirements over time.

To CLOS users, the domain-specific extension can be (and normally is) embedded directly in standard CLOS code as a seamless extension. This extensibility results in very compact source code. To put it another way, a domain-specific language encapsulates common paradigms in a domain, resulting in source code which is very compact, readable and understandable to the domain.

In fact, most standard CLOS programmers invent a domain-specific language to describe the internal workings of each module in a larger application. Each module is designed for a specific function (e.g., one module might do natural language parsing) and benefits from having the language extensions which best describe that type of functionality.

If you measure functionality in terms of executable image size (i.e. how much work the CPU has to do per line of source code), CLOS programs are very high in functionality. To give some measures of functionality per line of source code: Assembler is about half that of C; C is about half that of C++; C++ is about one eighth that of CLOS. Figures like this can be achieved for particular domain-specific languages; what is remarkable here is that CLOS is a general purpose language.

The ability to express a large amount of functionality in a very compact way is one of the key features that makes CLOS highly suitable to large, complex applications. For example, the compact nature of CLOS results in large productivity gains. If you can compress eight times as much functionality into one line of source code, then maybe you can build an application with eight times fewer programmers! Or, if you are an individual, you can be eight times faster than your peers. CLOS users commonly quote figures of 5x to 10x productivity gains.

As an example of this, take a look at our customer profile, discussing Blackboard Technology's GBB application at work on the Canadian Space Agency's RADARSAT-1 project.

Extending CLOS for Domain-Specific Languages

In addition to allowing you to define your own functions, CLOS permits you to build code templates or "macros". A template is generally a form where you can fill in the blanks; in CLOS, you can fill in those blanks with other code, code that may be calculated. These code templates or "macros" are unlike anything available in other programming languages, and they can be used to make your programs significantly smaller and cleaner. We'll try to give you an idea of how they work.

In CLOS, you define a class by specifying its name, the classes it inherits from, its slot names (in C++ these are known as members), the slot data types, and many other possible components. So a CLOS class might look like:

(defclass foo (foo-superclass)

  ((slot-1 :type integer :initform 0)

   ... other slots ...))

In a mechanical CAD application, you might decide that parts are best modeled as CLOS objects, but you'd prefer that slots be components and that they be initialized by a database lookup:

(define-part foo (foo-superclass)

  ((component-1 :initform (flywheel))

   ... other components ...))

The above definition is really a template which expands into a "defclass" and the macro "flywheel" might expand into a piece of code:

(lookup-database :database-name 'flywheel :element-type 'default)

(which looks up the default type in a database).

Notice that semantics such as...

"Initialize the components of a 'part' will always occur by looking up the component type in the appropriate type database."

...are directly encoded into the new template. This design decision in the application no longer has to be understood by everyone attempting to extend or maintain it. Other people extending, maintaining or continuing to build this application will find it much easier concentrate on the changes that they want to make rather than having to figure out a vast mass of extraneous detail.

A Game Development Language Programmed in CLOS

Naughty Dog Inc., developers of the popular Crash Bandicoot game application for the Sony Playstation^tm, developed a domain-specific language called GOOL (Game Object-Oriented Lisp) to solve domain-specific problems.

Andrew Gavin, lead developer of GOOL and Crash Bandicoot, explains that the power of Allegro Common Lisp/CLOS is that it provides a feature-packed object-oriented general language that can be easily customized to meet specific requirements, a combination hard to find with any other programming language.

"C, for example, is just very awkward for a project like this," Gavin says, "Lisp, on the other hand, is ideal. GOOL borrows its syntax and basic forms from Common Lisp and has all of the object-oriented advantages of CLOS. At the same time, GOOL is equipped with 60 primitives and 415 macros which support its advanced flow of control and game-specific operations."

"Overall," Gavin continues, "having a custom language whose primitives and constructs both lend themselves to the general task (object programming), and are customizable to the specific task makes it much easier to write clean descriptive code very quickly."

In Summary

Working with CLOS, you can easily:

• Extend CLOS and Common Lisp by adding your own functions to the language
• Build powerful abstractions with "macros" which customize functionality
• Use this extensibility to develop domain-specific "embedded" languages on top of Common Lisp and CLOS

Experienced CLOS programmers build the language up toward their programs, as well as writing their programs down toward the language. The result is that the program can look as if the language had been designed expressly for it (and it often is!). The bottom-line benefits of this feature of Common Lisp/CLOS? Faster time-to-market, greater flexibility to handle complexity, and - of course - application extensibility.