OK, what is
language-oriented programming, really?
Racket’s more than just a batteries-included Lisp. It’s about solving problems by making languages. Language-oriented programming is the name of the game.
The idea has an immediate intuitive appeal. Perhaps it’s what drove you to Racket in the first place. And if it wasn’t, chances are good, if you stick around the community for a while, that you’ll find yourself thinking about languages.
But what in the world are we really talking about when we talk about
Looking around the Racket community & among its fellow travelers, one finds that
make your own language has a variety of senses. Although some ideas are shared, they get expressed in different ways. At times, one wonders: Are we all talking about the same thing?
To help foster the growth of language-oriented programming in general—and Racket in particular as the best environment for solving problems in that way—it’s important to understand the diversity of meanings of
language-oriented programming and kindred expressions.
A good place to help develop an understanding of language-oriented programming is the Racket community itself. See below for a sampler of quotes.
Language-oriented Programming with Racket: A Cultural Anthropology is an ebook (482 pages) that contains the result of a survey I’ve taken of a wide spectrum of Racket programmers about what
make your own language means to each of them.
Foreword by Martin Ward, who, in 1994, coined the term
(Go here to get your copy of the ebook.)
Here are snippets from each of the interviews. (A PDF sampler of these quotes is available here.)
That kind of gradual development from analmost identical to Racketlanguage to one that has all of the features that you need, and behaves differently (possibly very differently) from Racket—that’s where most of the benefits of LOP come in, and that’s where Racket’s approach shines. — Eli Barzilay
I could write an external preprocessor, but this is a separate language that generates code. I could try to write a plugin, but this requires stepping outside the language. If I were working in Racket, I would write a few macros to expand all of those things. — William Bowman
Racket provides a parser generator library but what makes Racket different as a language building material is two features: (i) its macro system, and (ii) the possibility to override what function application means. — Jörgen Brandt
I really had no exposure to the idea of DSLs. Rather, I wanted a certain result, and crept up on DSLs accidentally (it just seemed like the most direct way to do what I was after). — Matthew Butterick
Certain vocabulary emerges out of rudimentary Racket functions. It is like the Racket language is enriched in the direction of game playing, population matching, regenerating, etc. They carry significant meaning outside the defined range of original Racket. — Nguyen Linh Chi
I’ve always been fascinated with different programming languages and even with the idea of creating my own, but I always saw it, and much of the literature and the public discourse presented it, as some impossibly complicated task not meant for mere mortals. — Annaia Danvers
If the programming model affects the structure of program fragments significantly then a new library is a sub-optimal solution as it introduces layers of encodings. A new language can hide all this and result in concise representations. — Christos Dimoulas
I saw that I had an upcoming need to write lots of variations of the same kind of program, and wanted to make those programs as human-legible as possible, which means abstracting away overhead tasks and having logical names for things. — Joel Dueck
The instructor emphasized that it was commonplace in the Lisp world to program in two steps: first create the language that you want, then implement your solution in that language. That idea stuck with me, and I later applied it even when I wasn't programming in Lisp. — Eric Eide
For me the realaha!moments didn’t occur when I realized I could use language-oriented programming to solve some problem. The bigaha!moment was when I realized that LOP-heavy projects aren’t always worth doing. — Leandro Facchinetti
I want [my students] to see that there arelanguageseverywhere. Sports teams have languages for planning plays; recipes are languages; medical protocols are languages. — Kathi Fisler
After you use higher-order functions for a while, you’re not willing to go back. Macros are like that, so I think it’s just a matter of time for enough programmers to catch on. — Matthew Flatt
I find making languages to be the same as normal programming: sometimes is easy, sometimes its hard. And often when its hard its because you picked the wrong design at the start! — Spencer Florence
I was kind of doing LOP anyway when I create new libraries for people. When I discovered Racket and began researching it, I realized I could create more than just functions for people. I realized I could create new syntax and basically make theshapeof the code be anything I wanted it to be. — Stephen Foster
Language-oriented programming, to me, means tweaking, augmenting, or changing an underlying computational model—or coming up with a completely new computational model. — Tony Garnock-Jones
I understand the termlanguage-oriented programmingas an approach to problem solving where, before actually solving the problem, one focuses on expressing this problem in an optimal way. — Panicz Godek
I’d already gone fromthinking about problems in terms of data structurestothinking about problems in terms of typesand so when I heard Racket was good forthinking about problems in terms of a tailor-made languagethat just sounded good. — Ben Greenman
Having a common language meant the ops team was free to tweak the model and draft patterns while I bolted a CSV file parser onto a finite-state machine. Without explicit formalism, we solved a complicated problem elegantly by developing a pattern language together. — Eric Griffis
I think it’s the case that anyone engaged inbottom-upprogramming is actively doinglanguage-oriented programming. As a result, you can look at virtually any library out there and see a language in it. The types, and functions exported are the primitives used to build up layers of abstraction to form more complicated ideas. — Andrew Gwozdziewycz
I was always interested in programming languages and liked to learn about new ones, but the idea of realistically making and using potentially many languages never seemed practical until I learned about embedded DSLs made with macros. — William Hatch
[In Racket] it is relatively seamless to go between the two: to take alibraryand add somelanguagebehavior to it, or to take alanguage-enriched libraryand just drop thelanguageparts. — Shriram Krishnamurthi
My language design is almost entirely oriented around some sort of optimization that I want to do. I need to restrict the language so the optimization applies then I need to build a language that enforces that restriction. — Jay McCarthy
A lot are put off by Lisp'slack of syntax. Macros may be viewed as a black-art. If people tried them out and began to see that macros are an abstraction layer, they would discover that they are eminently usable without having to sacrifice (many) goats. — Ross McKinlay
I think there are two ways of language-oriented programming. There are the macros you write to make your own code easier, and there are the languages you write for others to use. I’ve used Racket for both. — Pavel Panchekha
I understood from a long way back that rather than simplycodingin a given target language there was virtue in bringing the descriptive language to the problem. This can begin with functional abstraction and a ruthless desire to eliminate duplication and boilerplate. — Daniel Prager
The correct justification [ofhomoiconicity] is that parentheses facilitate careful metaprogramming, thecompile-timecreation of code. And metaprogramming is at the heart of what makes Racket great. — Prabhakar Ragde
I don’t thinkmaking a languageis a yes-or-no decision: You create abstractions, at some point you abstract over syntax and maybe later down the path you discover that just doing require is awkward or repetitive, which is when you do a language. — Michael Sperber
The interface was originally provided strictly as a library, but composing using it felt like shoehorning the patterns into a program: quoting, boilerplate, etc. So I built a
#langinterface instead. — Vincent St-Amour
At some level, nothing in Sexptex couldn't have been done with TeX macros instead. But the surface syntax of Racket is so much easier on the eyes than backslashes and curly braces, which I think is an underrated reason to do metaprogramming. — Paul Stansifer
When I'm writing in the Anatomy language, it almost doesn't feel like programming, because all I have to think about is the structure of dinosaur skeletons, which is ultimately the only thing that feels relevant to the problem I'm trying to solve. — Elle Sullivan
At the moment the project I am working on is a website where students can solve math problems. Such a project consists of many small parts, but the language oriented approach pops up repeatedly. Inspired by scribble and pollen I made
#langmathscrible. — Jens Axel Søgaard
Originally, I thought language-oriented programming was just about using macros to build interesting dialects of Racket, such as Typed Racketor Lazy (Racket). But it’s not really about writing dialects necessarily, since the languages people make are often hidden as a tool used to implement some other program. — Asumu Takikawa
I find Racket’s ecosystem a good proof-by-construction of the feasibility of [language-oriented programming], considering the impressive tower of mini-languages that are used in concert. — Éric Tanter
The most frequent sticking point is the question of when something is a language versus a library. I adopt a liberal definition of a language that blurs the distinction between the two, as I find that this makes the concept of making one’s own language more approachable. — Emina Torlak
What’s nice for me about the Racket approach is that I only have to deal with the details of those aspects of the language implementation that I want. If I don’t want to deal with lexing and parsing, for example, I can use the reader. — Jesse Tov
I started out building a library of functions for building compilers in Racket. It was a simple API built using Racket's amazing C-FFI. While I was using this API to build a compiler I slowly built a set of macros which made working with these crude functions much easier. These set of macros slowly evolved and became an embedded language. — Rajan Walia
The particular domain of language construction might be especially amenable to domain-specific languages. (There’s a reasonyaccstands foryet another compiler compiler. And yacc is old.) — Jon Zeppieri
#langs under discussion
A lot of Racket
#langs and non-Racket programming languages are discussed:
Go here to get your copy of the ebook (PDF format). It comes out to 482 pages; there’s a ton of insight to be had.