Is Rails a DSL? What is a DSL, and is it possible in Python? 9

Posted by ben Thu, 08 Dec 2005 20:17:40 GMT

I keep seeing blogs and blog comments pop up with some very odd notions. Many of these are summed up in this comment from a recent post by Tim on Ruby books

“It’s doubtful that the Python folks can come up with anything as compelling (or elegant) as Rails. Why? Because Ruby is so good at creating Domain Specific Langauges (DSLs). Ruby’s anonymous code blocks are a big part of what enables DSLs to be written so easily in the language. Python doesn’t have them. Python’s lambda’s (and closures in general) are crippled as well, which also doesn’t help Python’s cause.”

What is a Domain Specific Language (DSL)?

I always like to carefully define my terms before actually talking about them. So first I looked up several common definitions of a DSL. We have Martin Fowler’s entry on a DSL which describes it as:

“The basic idea of a domain specific language (DSL) is a computer language that’s targeted to a particular kind of problem, rather than a general purpose language that’s aimed at any kind of software problem.”

He further clarifies between two types of DSL:

“The most common unix-style approach is to define a language syntax and then either use code generation to a general purpose language, or write an interpreter for the DSL. Unix has many tools that make this easier. I use the term External DSL to refer to this kind of DSL.”

“The lisp and smalltalk communities also have a strong tradition of DSLs, but they tend to do it another way. Rather than define a new language, they morph the general purpose language into the DSL. (Paul Graham describes this well in Programming Bottom-Up.) This Internal DSL (also referred to as an embedded DSL) uses the constructs of the programming language itself to define to DSL.”

To summarize, an Internal DSL satisfies one of the following:

  • Uses a general purpose language, then morphs it into a language to fit the domain
  • Adds functionality to a general purpose language such that solving a specific domains problem is easier

An External DSL:

  • Completely new language syntax
  • Typically requires code generation or interpreter for DSL

Martin Fowler espouses on how very dynamic languages like Lisp, Smalltalk, and Ruby are more conducive to internal DSLs as the meta programming capabilities and dynamic nature make it easy to extend and customize the functionality for your domain. I completely agree with his contention that an Internal DSL is limited by the syntax and structure of the language.

DSL’s in Python and Ruby

While the anonymous code blocks in Ruby are useful and syntactically nice, similar functionality can be done in Python through the use of generators and iterators (more powerful generator functionality similar to Ruby’s anonymous blocks are planned for Python 2.5).

Ruby also lets you extend built-in types, which is utilized by Rails to add a few helper functions to core Ruby data types. Python and Ruby both have closures that operate slightly differently, Ruby’s closure is definitely more “full featured”. However this typically isn’t a problem because of list comprehension’s, generator expressions, and generators/iterators.

Zope is a very clear example that its quite possible to build an internal DSL in Python. Whether they built it in a clear, easy to use way is up for someone else to debate. SQLObject makes dealing with databases as easy as ActiveRecord, and there’s many other examples of internal DSL’s built in Python (Take a look at twill, a web application testing language implemented in Python).

Is Rails a DSL?

First, it should be obvious that if Rails is a DSL, its an Internal DSL. The most noticeable strides towards being a web programming DSL in Rails:

  • Over a dozen core Ruby classes are extended/modified
  • Dozens of helper functions
  • New classes (syntactic sugar) defining database access and web paradigms

Given how loose the definition of internal DSL is, I think its clear that Rails qualifies, so does Django, TurboGears, Aquarium, etc.

If you prefer to be more strict about an internet DSL, and argue that a lay programmer needs to be able to do a reasonable amount of ‘work’ without knowing the base language implementing it, Rails starts to move apart. You can do a reasonably large amount of work in Rails without knowing Ruby. A lot can also be done in Zope without knowing Python.

In either case, you can’t really start to make advanced applications in any of the frameworks without knowing the general purpose language its built on. DSL’s can and have been built in Python, and I believe if you go out and start counting them up, you’ll see more DSL’s implemented in Python than in Ruby.

Anyways, hopefully for people that run around re-iterating something they heard somewhere about DSL’s and Python vs. Ruby, this clarified something. It’s easy to create DSL’s in Python, lots of people do, and there’s lots of them. Python and Ruby are both great for making them, outside of Rails I don’t know any other DSL’s in Ruby. Perhaps someone can compile a list of all the internal DSL’s written in both Ruby and Python?

Comments

Leave a comment

  1. Avatar
    Dave Kirby about 2 hours later:

    Have you taken a look at Logix (http://www.livelogix.net/logix/)? This is a lisp-like metalanguage built on top of Python. In other words a DSL whose domain is writing other DSLs. Languages written in Logix compile down to Python bytecode so can be intermixed with Python and with each other.

  2. Avatar
    Ben Bangert about 3 hours later:

    Dave: Yep, it looks very cool.

    Of course, we can also mention PyPy which is an implementation of Python itself… in Python! Very cool stuff over all.

  3. Avatar
    Ian Bicking about 3 hours later:

    Of course, just about every templating language is a DSL to some degree. Rails’ templating being on the not-so-DSLish side. Cheetah and Kid being slightly-but-not-much-more DSLish. Django template lying out at the end of quite-DSLish. Python forces you into DSLs faster than some languages (like Ruby) because of the whitespace issue; but once you are writing one Python seems as good a base as anything else I’ve seen. But to the degree these are DSLs, they are all external.

    Internal DSLs are possible in Python. SQLObject class syntax is a little DSLish. The SQLObject sqlbuilder syntax is entirely a DSL, and is something that ActiveRecord notably does not have. Python is reasonably workable for that DSL—not perfect to be sure, but workable. I expect Ruby could implement the same thing very similarly. But ActiveRecord hasn’t.

    I was noticing this guys URL parsing as an interesting idea, and it is also definitely a DSL. PyParsing also would qualify, or stan

    As for pieces of Rails, I’m not sure. Is ActiveRecord a DSL? I think you need to interpret DSL pretty loosely to get there. All declarative programming isn’t DSL; ActiveRecord is just making use of Ruby’s particular techniques for declaratively defining a class. I’d call that “metaprogramming”, but all metaprogramming is not a language.

    Maybe this misinterpretation has happened because to do declarative programming in something like Java you need a DSL (in XML or otherwise). That just isn’t the case in Python or Ruby or Lisp.

  4. Avatar
    Anders Pearson about 8 hours later:

    rake is one of the oft-mentioned ruby DSLs.

    Numpy also has a real DSL feel to it. i’ve also used it to build my own cellular automata DSL. eg, Conway’s Came of Life looks like:

    alive = (c == 1) &  ((m_cnt == 2) | (m_cnt == 3))
born  = (m_cnt == 3) & (c == 0)
c   = alive | born
  5. Avatar
    Florian about 14 hours later:

    Yet another example of an internal DSL implementation: PLY

    Of course it’s unfair to pound on the ruby guys for:

    • not knowing enough python to make more qualified statements about it
    • not having large enough a community to create DSLs by the bucketfull.
  6. Avatar
    Florian about 14 hours later:

    ( I want to mention that the preview comment button here does not correlate much with the reality afterwards, wtf are those pesky borders around the list items?! )

  7. Avatar
    Ben Bangert about 20 hours later:

    Florian: Yea, those borders are annoying. I’m going to fix the CSS so they don’t pop-up.

    I think its absolutely fair to give the Ruby people a little nudge that maybe they should learn some Python so they can make qualified statements about it.

    They’ve created quite a few DSL’s as well, though obviously the larger Python community has resulted in many more.

  8. Avatar
    Hey, ho. about 23 hours later:

    Creating GUI with a Ruby library of mine:

    1. Requires Cross-GUI fast GUI writing require ‘cgr/cgr’
    1. Choose GTK as the back-end start(:GTK){|w|
      1. sets the title of the window w.title = ‘Hello World’
      1. creates a button which calls
      2. quit to terminate the app w.hpack button(‘Quit’){ quit }

    }

    It’s not a public library yet (may be in the future), but it’s real in the sense that the code above should work for several back-ends (currently GTK+, Fox-Toolkit, wxWidgets, CGI, Wee, Hiki-Wiki). Actually, if I don’t set :GTK for the start method, it will choose randomly among the available options. If, instead, I set :DESKTOP, it will choose one of the desktop available GUIs. It goes beyond simple GUIs, but not as beyong as it should, because this is part of a big personal project, which I’m not close to finishing it yet.

    Now, I consider the above code the “front-end” for application programming. Behind the scenes I use full object orientation with polymorphism when I can. It means that the hard part is hidden from the mere user. It really helps in cleaning up the code for lesser developers.

    I like Ruby because my programs and libraries can grow organically, without needing to reinvent the wheel outside of pure Ruby. It’s really like Python in many aspects, but I prefer the standard coding that I can do with Ruby. Once I tried to rewrite a Ruby program of mine in Python and it didn’t feel as pretty as the Ruby version, I didn’t even finish it.

    Until now, I haven’t tried to understand List Comprehension in Python. But I guarantee you that it’s one part of Python code that’s not as easy to understand and read, so it doesn’t follow the “easy to read” philosophy of Python. Also, those extra “self”s, compared to Lisp Comprehension, are just extra verbosity which too don’t help reading Python code any more.

    From my experience, I prefer most code in the vertical like in most Ruby code than in horizontal like in some Python code.

    That said, one of the apps that I’m currently using is Bazaar-NG (SCM), which is written in Python, and I have had the need to fix some issues in it, but it’s understandable because Bazaar-NG is under heavy development. So, like Bazaar-NG, many apps are written in Python, so I’m going to see a lot of Python in the future, and given that, I consider Python my “second language”, though I prefer Ruby a lot more and use it every day.

    He who uses Python is well served.

  9. Avatar
    Hey, ho. about 23 hours later:

    Sorry about that. I didn’t expect that the code would get messed up.

    I’ve pasted it at:

    http://rafb.net/paste/results/SF9sVQ96.html