What is lan­guage-ori­ent­ed pro­gram­ming, real­ly?

Rack­et’s more than just a bat­ter­ies-in­clud­ed Lisp. It’s about solv­ing prob­lems by mak­ing lan­guages. Lan­guage-ori­ent­ed pro­gram­ming is the name of the game.

The idea has an im­me­di­ate in­tu­itive ap­peal. Per­haps it’s what drove you to Rack­et in the first place. And if it wasn’t, chances are good, if you stick around the com­mu­ni­ty for a while, that you’ll find your­self think­ing about lan­guages.

But what in the world are we real­ly talk­ing about when we talk about lan­guage-ori­ent­ed pro­gram­ming?

The term has a va­ri­ety of mean­ings

Look­ing around the Rack­et com­mu­ni­ty & among its fel­low trav­el­ers, one finds that “make your own lan­guage” has a va­ri­ety of sens­es. Al­though some ideas are shared, they get ex­pressed in dif­fer­ent ways. At times, one won­ders: Are we all talk­ing about the same thing?

To help fos­ter the growth of lan­guage-ori­ent­ed pro­gram­ming in gen­er­al—and Rack­et in par­tic­u­lar as the best en­vi­ron­ment for solv­ing prob­lems in that way—it’s im­por­tant to un­der­stand the di­ver­si­ty of mean­ings of “lan­guage-ori­ent­ed pro­gram­ming” and kin­dred ex­pres­sions.

So who can say what it means?

A good place to help de­vel­op an un­der­stand­ing of lan­guage-ori­ent­ed pro­gram­ming is the Rack­et com­mu­ni­ty it­self. See be­low for a sam­pler of quotes.

Lan­guage-ori­ent­ed Pro­gram­ming with Rack­et: A Cul­tur­al An­thro­pol­o­gy is an ebook (314 pages) that con­tains the re­sult of a sur­vey I’ve tak­en of a wide spec­trum of Rack­et pro­gram­mers about what make your own lan­guage means to each of them.

Fore­word by Mar­tin Ward, who, in 1994, coined the term lan­guage-ori­ent­ed pro­gram­ming.

Get your copy here.

Meet the lan­guage mak­ers

Here are snip­pets from each of the in­ter­views. A PDF sam­pler of these quotes is avail­able here.

That kind of grad­ual de­vel­op­ment from an al­most iden­ti­cal to Rack­et lan­guage to one that has all of the fea­tures that you need, and be­haves dif­fer­ent­ly (pos­si­bly very dif­fer­ent­ly) from Rack­et—that’s where most of the ben­e­fits of LOP come in, and that’s where Rack­et’s ap­proach shines.

Eli Barzi­lay

I’ve al­ways been fas­ci­nat­ed with dif­fer­ent pro­gram­ming lan­guages and even with the idea of cre­at­ing my own, but I al­ways saw it, and much of the lit­er­a­ture and the pub­lic dis­course pre­sent­ed it, as some im­pos­si­bly com­pli­cat­ed task not meant for mere mor­tals.

An­na­ia Berry

Rack­et pro­vides a pars­er gen­er­a­tor li­brary but what makes Rack­et dif­fer­ent as a lan­guage build­ing ma­te­r­i­al is two fea­tures: (i) its macro sys­tem, and (ii) the pos­si­bil­i­ty to over­ride what func­tion ap­pli­ca­tion means.

Jör­gen Brandt

I real­ly had no ex­po­sure to the idea of DSLs. Rather, I want­ed a cer­tain re­sult, and crept up on DSLs ac­ci­den­tal­ly (it just seemed like the most di­rect way to do what I was af­ter).

Matthew But­t­er­ick

Cer­tain vo­cab­u­lary emerges out of rudi­men­ta­ry Rack­et func­tions. It is like the Rack­et lan­guage is en­riched in the di­rec­tion of game play­ing, pop­u­la­tion match­ing, re­gen­er­at­ing, etc. They car­ry sig­nif­i­cant mean­ing out­side the de­fined range of orig­i­nal Rack­et.

Nguyen Linh Chi

If the pro­gram­ming mod­el af­fects the struc­ture of pro­gram frag­ments sig­nif­i­cant­ly then a new li­brary is a sub-op­ti­mal so­lu­tion as it in­tro­duces lay­ers of en­cod­ings. A new lan­guage can hide all this and re­sult in con­cise rep­re­sen­ta­tions.

Chris­tos Di­moulas

I saw that I had an up­com­ing need to write lots of vari­a­tions of the same kind of pro­gram, and want­ed to make those pro­grams as hu­man-leg­i­ble as pos­si­ble, which means ab­stract­ing away over­head tasks and hav­ing log­i­cal names for things.

Joel Dueck

I want [my stu­dents] to see that there are lan­guages every­where. Sports teams have lan­guages for plan­ning plays; recipes are lan­guages; med­ical pro­to­cols are lan­guages.

Kathi Fisler

Af­ter you use high­er-or­der func­tions for a while, you’re not will­ing to go back. Macros are like that, so I think it’s just a mat­ter of time for enough pro­gram­mers to catch on.

Matthew Flatt

I find mak­ing lan­guages to be the same as nor­mal pro­gram­ming: some­times is easy, some­times its hard. And of­ten when its hard its be­cause you picked the wrong de­sign at the start!

Spencer Flo­rence

I was kind of do­ing LOP any­way when I cre­ate new li­braries for peo­ple. When I dis­cov­ered Rack­et and be­gan re­search­ing it, I re­al­ized I could cre­ate more than just func­tions for peo­ple. I re­al­ized I could cre­ate new syn­tax and ba­si­cal­ly make the ◊q{shape} of the code be any­thing I want­ed it to be.

Stephen Fos­ter

Lan­guage-ori­ent­ed pro­gram­ming, to me, means tweak­ing, aug­ment­ing, or chang­ing an un­der­ly­ing com­pu­ta­tion­al mod­el—or com­ing up with a com­plete­ly new com­pu­ta­tion­al mod­el.

Tony Gar­nock-Jones

I un­der­stand the term lan­guage-ori­ent­ed pro­gram­ming as an ap­proach to prob­lem solv­ing where, be­fore ac­tu­al­ly solv­ing the prob­lem, one fo­cus­es on ex­press­ing this prob­lem in an op­ti­mal way.

Pan­icz Godek

I’d al­ready gone from think­ing about prob­lems in terms of data struc­tures to think­ing about prob­lems in terms of types and so when I heard Rack­et was good for think­ing about prob­lems in terms of a tai­lor-made lan­guage that just sound­ed good.

Ben Green­man

Hav­ing a com­mon lan­guage meant the ops team was free to tweak the mod­el and draft pat­terns while I bolt­ed a CSV file pars­er onto a fi­nite-state ma­chine. With­out ex­plic­it for­mal­ism, we solved a com­pli­cat­ed prob­lem el­e­gant­ly by de­vel­op­ing a pat­tern lan­guage to­geth­er.

Eric Griff­is

I think it’s the case that any­one en­gaged in bot­tom-up pro­gram­ming is ac­tive­ly do­ing lan­guage-ori­ent­ed pro­gram­ming. As a re­sult, you can look at vir­tu­al­ly any li­brary out there and see a lan­guage in it. The types, and func­tions ex­port­ed are the prim­i­tives used to build up lay­ers of ab­strac­tion to form more com­pli­cat­ed ideas.

An­drew Gwozdziewycz

I was al­ways in­ter­est­ed in pro­gram­ming lan­guages and liked to learn about new ones, but the idea of re­al­is­ti­cal­ly mak­ing and us­ing po­ten­tial­ly many lan­guages nev­er seemed prac­ti­cal un­til I learned about em­bed­ded DSLs made with macros.

William Hatch

[In Rack­et] it is rel­a­tive­ly seam­less to go be­tween the two: to take a li­brary and add some lan­guage be­hav­ior to it, or to take a lan­guage-en­riched li­brary and just drop the lan­guage parts.

Shri­ram Kr­ish­na­murthi

My lan­guage de­sign is al­most en­tire­ly ori­ent­ed around some sort of op­ti­miza­tion that I want to do. I need to re­strict the lan­guage so the op­ti­miza­tion ap­plies then I need to build a lan­guage that en­forces that re­stric­tion.

Jay Mc­Carthy

Think about the count­less hours poured into writ­ing tran­spilers for Java­Script. All that ef­fort to side­step in­ter­preters when a DSL or macro sys­tem could have achieved the same. Give me a ful­ly-fea­tured and pro­duc­tion ready Rack­et­Script any day.

Dar­ren New­ton

I think there are two ways of lan­guage-ori­ent­ed pro­gram­ming. There are the macros you write to make your own code eas­i­er, and there are the lan­guages you write for oth­ers to use. I’ve used Rack­et for both.

Pavel Panchekha

I un­der­stood from a long way back that rather than sim­ply cod­ing in a giv­en tar­get lan­guage there was virtue in bring­ing the de­scrip­tive lan­guage to the prob­lem. This can be­gin with func­tion­al ab­strac­tion and a ruth­less de­sire to elim­i­nate du­pli­ca­tion and boil­er­plate.

Daniel Prager

The cor­rect jus­ti­fi­ca­tion [of ho­moiconic­i­ty] is that paren­the­ses fa­cil­i­tate care­ful metapro­gram­ming, the com­pile-time cre­ation of code. And metapro­gram­ming is at the heart of what makes Rack­et great.

Prab­hakar Ragde

I don’t think mak­ing a lan­guage is a yes-or-no de­ci­sion: You cre­ate ab­strac­tions, at some point you ab­stract over syn­tax and maybe lat­er down the path you dis­cov­er that just do­ing re­quire is awk­ward or repet­i­tive, which is when you do a lan­guage.

Michael Sper­ber

The in­ter­face was orig­i­nal­ly pro­vid­ed strict­ly as a li­brary, but com­pos­ing us­ing it felt like shoe­horn­ing the pat­terns into a pro­gram: quot­ing, boil­er­plate, etc. So I built a #lang in­ter­face in­stead.

Vin­cent St-Amour

Orig­i­nal­ly, I thought lan­guage-ori­ent­ed pro­gram­ming was just about us­ing macros to build in­ter­est­ing di­alects of Rack­et, such as Typed Rack­et or Lazy Rack­et. But it’s not real­ly about writ­ing di­alects nec­es­sar­i­ly, since the lan­guages peo­ple make are of­ten hid­den as a tool used to im­ple­ment some oth­er pro­gram.

Asumu Takikawa

I find Rack­et’s ecosys­tem a good proof-by-con­struc­tion of the fea­si­bil­i­ty of [lan­guage-ori­ent­ed pro­gram­ming], con­sid­er­ing the im­pres­sive tow­er of mini-lan­guages that are used in con­cert.

Éric Tan­ter

The most fre­quent stick­ing point is the ques­tion of when some­thing is a lan­guage ver­sus a li­brary. I adopt a lib­er­al de­f­i­n­i­tion of a lan­guage that blurs the dis­tinc­tion be­tween the two, as I find that this makes the con­cept of mak­ing one’s own lan­guage more ap­proach­able.

Em­i­na Tor­lak

What’s nice for me about the Rack­et ap­proach is that I only have to deal with the de­tails of those as­pects of the lan­guage im­ple­men­ta­tion that I want. If I don’t want to deal with lex­ing and pars­ing, for ex­am­ple, I can use the read­er.

Jesse Tov

The par­tic­u­lar do­main of lan­guage con­struc­tion might be es­pe­cial­ly amenable to do­main-spe­cif­ic lan­guages. (There’s a rea­son yacc stands for yet an­oth­er com­pil­er com­pil­er. And yacc is old.)

Jon Zep­pieri

Get­ting a copy

Go here to get your copy of the ebook (PDF for­mat). It comes out to more than 300 pages; there’s a ton of in­sight to be had.