Best of breed Controllers for MVC web frameworks
I’ve been doing a lot of comparisons and research into various Python web frameworks lately, mainly focused on MVC oriented frameworks. Some of them have templating languages they come with, some not so much. The thing I have started to notice is how similar the Controller in them all is starting to look. First though, lets see what they all can do when it comes to the Controller.
For comparison’s sake, I’m bringing out some examples using CherryPy, Myghty, Bricks, Aquarium, Ruby on Rails, and Django.
Please note I’m not including frameworks built using any of the ones mentioned as their Controller looks pretty much the same. I’m also not an expert in all these frameworks as I only have so much time in the day, so if I seem vague on something its because I don’t know for sure. Feel free to comment (or correct) and clear up any vagueness I express.
Dispatch and Responder Styles
All the frameworks I’ve mentioned implement a style of object publishing (sometimes referred to as dispatch-style) that maps to the URL coming in. If you have another name you want to call this, go for it, I haven’t seen anything consistent on what to call this stuff.
Three different styles seem to exist that make sure an object responds to the URL:
- User creates a regular expression to explicitly map a URL to an object (*Explicit Mapping*)
(r'^polls/(?P<poll_id>\d+)/vote/$', 'myproject.apps.polls.views.polls.vote')(from Django tutorial)
- Object is mapped out according to the URL (*Implicit Mapping*)
- /admin/login/hello/ —> admin/login.py/hello() (Myghty)
- Programmatic rule set determines object that responds (*Programmatic Mapping*)
- map.connect ':controller/:action/:id’ (Rails)
- m.connect(’:controller/:action/:id’) (Routes)
In the Python world, three different styles also exist for how that object (or function) is defined. The first one is most popular, and is similar to how Rails sets up the Controller. They are:
- Create a class inheriting from a Controller object, methods of the class instance are called to respond (*Class-method*)
- Create a callable object that responds (*Callable*)
- Create a function thats directly called (*Function*)
Note that the last 2 are functionally equivalent for the most part, however a callable object has the typical advantages (or disadvantages) of persistence in some web frameworks, as such if a web framework only shows one of those styles I won’t mention the other. Curious what frameworks use what styles from the above two sets?
For this listing, I’ll refer to the mapping scenario as dispatch-style and the method for defining the response as the responder-style, also please note that some frameworks support multiple dispatch-styles and responder-styles.
Aquarium:
- Dispatch-Style Mapping 1:
- Explicit, Implicit, Programmatic
- Responder-Styles:
- Callable (Though it looks almost exactly like the Class-method style)
Bricks:
- Dispatch-Style Mapping:
- Implicit
- Responder-Styles:
- Class-method
CherryPy:
- Dispatch-Style Mapping:
- Explicit (Not with regexp’s though, each class is attached off the root to determine what URL leads to it)
- Responder-Styles:
- Class-method (Must be explicitly exposed with @cpg.expose)
Django:
- Dispatch-Style Mapping:
- Explicit
- Responder-Styles:
- Function (Packaged in the same module if they’re related)
Myghty:
- Dispatch-Style Mapping 1:
- Explicit, Implicit, (Programmatic with Routes in upcoming 0.99)
- Responder-Styles:
- Class-method, Callable, Function
Rails:
- Dispatch-Style Mapping 1:
- Programmatic
- Responder-Styles:
- Class-method
1: User can create custom dispatchers to determine responder object
Who did it first?
If we throw in two more frameworks that utilize CherryPy, like Subway, and TurboGear, we can start to see that using a responder-style of Class-method is pretty popular. What’s even more interesting is that all the Class-method Controllers look remarkably similar, and I don’t think its because they’re copying each other.
The Aquarium project appears to be approximately 4 years old now, but from looking at the mailing list had little or no users until only 2-3 years ago (One poster from 2001 describes his aquarium plants turning brown…). From the sourceforge changelogs, the Controller stuff was added with 1.4 release in 2004. CherryPy has been around for 3+ years and I’m presuming it had the Controller stuff in it from the get-go as it seems a pretty core function of CherryPy. Rails has been out for a year or so, and was used in a form internally for BaseCamp for awhile predating that.
If you really want to trace back this style of Controller, you’d most likely find yourself looking at Java Python code (See Phillip J. Eby’s comment below). I’m really just trying to point out here, that even if one of these projects saw something they liked from the other, none of them were the originators. They all took a good concept, and implemented it differently, though they’ve all started to look remarkably similar.
A Controller Line-Up
So lets take a look at some of the examples on the pages of each respective website. Django is the only one that really stands out, as they’re not using a Class to contain the methods, rather they’re grouping them as plain functions inside a module.
Aquarium
class BaseController(Controller):
def call(self, callNext, *args, **kargs):
“”“Save a reference to myself in ctx.controller.”“”
self._ctx.controller = self
callNext(*args, **kargs)
def doMessageAddAction(self):
“”“Add a new message.
...does a bunch of stuff…
I mentioned that it was Callable but functioned in a way like a Class-method. This appears to be because Aquarium puts a context and maybe other initialization stuff per request in the __call__ bit, then has it actually calls a different method to respond.
Bricks
from bricks.controller.crud import CRUD
from bricks import user
class Test(CRUD):
# Override the default delete() method
def delete(self, rowid):
return CRUD.delete(self, rowid)
delete.expose = user(level=1, app='app’)
controller = Test('todo’)
Bricks adds an interesting ability here by attaching function attributes to indicate what authorization is needed to execute the function called. Some of the other frameworks shown here do similar things through function decorators.
CherryPy
from cherrypy import cpg
class HelloWorld:
def index(self):
return “Hello world!”
index.exposed = True
cpg.root = HelloWorld()
Notice that the class instance is attached to the location off cpg.root thats desired. To map a URL to a class instance farther down, say under /booga instead of /, you’d do that last line like so:cpg.root.booga = HelloWorld()
Pretty handy I think, and it beats writing regular expressions (though doesn’t appear as flexible). One of the reasons I prefer the Programmatic dispatch-style. Moving on..
Django
from django.core.extensions import get_object_or_404, render_to_response
from django.models.polls import choices, polls
from django.utils.httpwrappers import HttpResponseRedirect
def vote(request, poll_id):
p = get_object_or_404(polls, pk=poll_id)
... does a bunch more stuff to use Django-supplied functionality…
If you imagined the functions inside a class, instead of a module, it’d look rather similar to the others….
Myghty
class HelloWorld:
def doit(self, m, **params):
m.write(“hello world!”)
helloworld = HelloWorld()
At this point, the similarity between Bricks, CherryPy, and Myghty should be pretty obvious. Aquarium is close as well, while Django is only a wee farther from what’s appearing to be the norm.
Rails
class HelloController < ApplicationController
def index
return render :text => "hello world!"
end
end
Doesn’t look too different from some of our Python MVC frameworks…
What it All Means
I have no idea. Really, I just thought it was fascinating that these different frameworks started looking so similar. Some of them have been incredibly similar before others, but obviously these approaches catch on and find themselves adopted into other frameworks.
I do start to wonder at what point they’ll be similar enough that the barrier to switching web frameworks is narrowed down to only a few dozen lines of code to change here and there. Sure, going to/from Rails is going to be tough as that also means a language change, but for Python programmers tempted by a feature a different web framework offers I won’t be surprised to see more people hopping back and forth.
With regard to the Python web frameworks, none of them appear to agree on a template language. Several of them use Cheetah, some use Myghty’s template abilities, one uses Kid, and several of them have their very own template language.
It’s also interesting to consider whether or not having more possible dispatch/responder styles is beneficial or detrimental to the popularity of a specific framework. Rails creator David Hansson purposely includes only one style for each as he emphasizes with a concept known as convention over configuration. He provides defaults for pretty much everything, and following conventions makes it easier to write code as it enforces a consistent programming and naming style. I can’t help but notice that of the frameworks I mentioned above, the ones with less (apparent) options and better defaults tend to have more users (With the exception of Bricks as its still in alpha).
You might have noticed I designated Rails as being capable of user-programmed dispatching. While Rails uses its routes system by default, the thing people miss is that they’re just defaults. You can get in there and totally alter the way it works if you want, Ruby doesn’t completely close classes so you could just re-open them and override any aspect of its dispatch system you like.
I think this a valuable lesson that some of the more “verbose” examples above might want to consider. If there’s lines of code in a controller that you have to put in there every-time, find a way to make it the default case so that boiler-plate isn’t needed. It only needs to be a default, an advanced programmer will know how to override it should they need to. There’s obvious limitations to this since things work differently in Python, but there’s still Python ways to do it in a way that feels “right”.
For the Python Web Frameworks I missed
I know, I didn’t cover SkunkWeb, Spyce, Snaklets, Webware and more. Please feel free to add the relevant information regarding your framework in the comments. It’d be helpful if you could indicate what Dispatch-Styles and -Responder-Styles your framework supports.
If I failed to properly cover one of the above frameworks, bringing that to my attention is most appreciated.
The comment system supports the Textile format.
“”“If you really want to trace back this style of Controller, you’d most likely find yourself looking at Java code.”“”
I seriously doubt that. I have in front of me the March/April 1997 issue of “Web Apps Magazine”, which has an outside headline “Publishing Objects Without Using CGI...”
Inside, on page 50, is an article entitled “Using the Python Object Publisher”, by Paul Everitt, whom you may now know as the former president of Zope Corporation, then called Digital Creations. The article described “Bobo”, the great-grandpa of what eventually became Zope.
One of the reasons I still have this magazine is because that article was the reason I’m now a Python developer. At the time I read the article, I seized on the concept as being just what I needed for developing web applications, and so I tried implementing something like Bobo in Perl, and was beset by obstacle after obstacle. Not long after, Python became my language of choice.
At the time, I don’t believe Java had much introspective capability, though I could be wrong. However, the idea that it would have been mature enough at that time for somebody to come up with the idea for Java? Seems pretty darn unlikely to me. Note also that a March ’97 issue of a magazine would’ve had the articles written months ahead of time, so that means that this type of controller has been out there since 1996, using docstrings instead of 'expose’ attributes to mark methods suitable for publishing. The article mentions a lot of features that would’ve taken some time to develop, and the package being open source, and used with a variety of back-ends like FastCGI, so it’s likely that development was over a good part of 1996.
The article, by the way, either gives examples of, or explains how to do, pretty much all the styles of dispatch and response that you’ve listed here. If you’re going to give props, they rightfully belong to the Zope Pope, Jim Fulton, who singlehandedly invented this entire breed of dynamic language-based, RESTful web MVC, back before “REST” was even coined as a term, back when version 4 browsers were the latest-and-greatest.
Legend has it, by the way, that Jim was pressed into service to give a talk on CGI at the IPC (International Python Conference), and therefore had to learn CGI on the plane in order to give the talk. He was so horrified by what he learned that he invented a completely different paradigm on the flight back. It’s taken this many years for his radical invention to now be considered mainstream and even “cutting edge”.
Anyway, my point is that the frameworks you mention are similar because it’s a natural design, if you aren’t trapped by the CGI mindset. Jim Fulton just happened to do it first, not by a couple months or years, but by over half a decade ahead of even the oldest of the frameworks you mention.
And I’m only here as a Python programmer because of it. Before I tried duplicating Bobo in Perl, I was (believe it or not) perfectly happy with Perl. But once I saw what Bobo could do, I knew it was the right tool for the job. I never could have dreamed that it would take almost a decade for industry to suddenly “discover” this, and still be lagging behind it. No wonder Zope does so well with people who aren’t programmers; they don’t have as much to unlearn! (I personally hadn’t been doing CGI long when I read the article, not enough to stunt my brain anyway. :)
If I knew then what I knew now, Bobo would’ve had a screencast as soon as I could’ve figured out how to make one. There was no blogosphere and no Google, then, but I’d have found some way to spread the word. Bobo could have been for Python what Rails is to Ruby – the killer app you change languages for. It certainly did the trick for me.
Phillip J. Eby: Wow, that’s awesome. Such a rich history I didn’t know existed. Nice to know I don’t have to trace it back to Java, the reason I mentioned that was because several changelogs for various web frameworks mentioned Java stuff as inspiration.
Thanks again for the details, and for anyone interested, I dug up a link to the article PJE refers to, The Python Object Publisher, and here’s the page announcing the talk that occured in 1996. At least, I’m fairly sure thats the article, as its not dated on that site.
That’s not actually the article; the one I have in print is by Paul Everitt, not by Jim, and it goes into a lot more “how-to” than the page you found does.
The date when I “view page info” in Firefox shows Jim’s paper as last modified in August of 1996, and Paul Everitt’s talk at IPC 5 is shown for November 1996, so it’s probably a good bet that the infamous plane ride where Jim invented REST-MVC took place in June 1996
The agenda lists him as doing a class on “Python and Internet Programming”, so I guess that’s the class he got pressed into doing. If so, it’s pretty cool what they managed to turn Bobo into by the time the article I read got written, presumably a mere 6 months later.
Amusing note: on that same conference schedule is Guido’s keynote: “Python 1.4 and beyond”. Probably he announced cool new features like packages. :) (Actually, I think packages might not have officially arrived until Python 1.5, but I could be wrong.)
Interesting roundup! Thanks for writing this up!
This was an interesting comment:
I’ve switched from Quixote to Nevow to CherryPy, and the traversal mechanisms were all similar enough between the three of those that, particularly given my unit tests, it was not a difficult switch.
I wouldn’t personally put the dispatch style mechanisms of CherryPy, Quixote and Nevow in the same category as Django, because there is no additional configuration beyond the actual code that handles the request. That’s a different flavor to me.
Also, CherryPy does allow for computed traversal as well:
@turbogears.expose()
def default(self, other, parts, of path):
...do stuff…Overall, an interesting comparison.
A few minutes more Googling, and I found a paper that looks substantially similar to the WebApps Magazine article: Using the Python Object Publisher
The tone is fairly laid back, a sort of, “isn’t that a neat trick?”, but not the hype of a Rails, despite the fact that it demos what might be considered a “20-minute web app”. If only they’d added a few statements like, “This is the way of the future and the One Obvious Way You Should Do It”, and made a screencast or two. :)
The classic hello world web app, in Quixote, in a few lines of code. Other demos here and you can navigate back from there to get code, etc. Incidentally following those links takes to you python objects published via Quixote…
Quixote borrowed the very good idea from Zope – simple object publishing and none of the other stuff.
Its easy to understand, very powerful, quite fast (light and simple) and runs on a wide variety of systems and environments.
If using an object database is your bag, you might care to spend a day or two looking at the troika of Quixote (object publishing), Durus (object database patterned after ZODB but much simpler and therefore easy to read through and understand, and Dulcinea, an add-on with a number of pre-written components and some very useful development and site management utilities.
Oh, I have a demo of Toboso, which is a tiny application designed to show how parts from Dulcinea are brought together in a web app. It only shows off on the surface one little bit of Dulcinea, but in it you’ll find fairly complete user and permissions management system.
If intrigued, I suggest first get a Quixote running first, play with the demo apps and then tackle Dulcinea/Durus.
Thanks Phillip, thats a great article. For some reason I keep bursting into laughter as I read it.
I agree, if only they had a slick website, the power of blogs, and a screencast!
Interesting write-up!
Small correction: Note that Django controllers can be callables, too — they don’t necessarily have to be functions.
A couple more things…
A historic note for anybody who’s interested: Django also used to support the class-method responder style, but we removed it because, in practice, we never used it.
Also, in the interest of fairness, your examples should probably all perform the same function — either displaying “Hello world,” or whatever. The Aquarium, Bricks and Django examples do extra stuff, which could be deceiving for people who give this article a cursory glance and compare the frameworks by looking at the number of lines of code.
Adrian, very true. I went with 'Hello World’ controllers when I could find them in the docs. The Django docs typically make use of much more power, so I tried to indicate that.
From what I’ve looked at, there’s only an extra line or two in the Aquarium example as the object is called, then expected to call its appropriate function.
As I was trying to show that all the frameworks are getting rather comfy with each other, hopefully no one will actually consider using one framework or the other solely on these comparisons. :)
Great work! Thanks for doing it.
FWIW, controllers in CherryPy don’t have to be class methods either; they can be any callable that allows an “exposed” attribute to be set on it. They tend to be class methods because that’s the easiest way to hook them into the cherrypy.root handler tree.
I’ll also note that CherryPy doesn’t officially advocate any templating language (not even CherryTemplate), although some tools built on top of CherryPy do so.
About the history of web application frameworks:
What’s about WebObjects by NeXT?
WebObjects 1.0 was released early 1996 and had a lot of publicity. I’d say that they brought the idea of object publishing via webpages to the general public.
Thanks, Phillip, for the kind words and digging up some delightful
prehistoric artifacts. [wink] You’re right that Jim deserves credit
for object publishing, and that we missed the boat in not explaining
it more widely.
You’re also right that IPC 5
was where Jim was pressed into service. That page shows that I was
scheduled to give the presentation. I wanted, though, to go to France
with my wife. [wink] So I asked Jim, who had been with our company for
around 2 weeks, to give my presentation. Your recounting is accurate:
he hadn’t done CGI before and he wrote the essentials of the object
publisher on the plane ride back.
There’s also some interested history before that. At IPC 3, I gave a
CGI tutorial. There were some guys from a company called eShop in the
tutorial. One of them was this neat guy named Greg Stein. I showed
something called the ILU Requestor (props for this go to Rob Head and
Brian Lloyd), which eShop subsequently added to their e-commerce
product. A product which subsequently became Microsoft’s SiteServer
when eShop was bought. (I wrote a W3C tech note
afterwards on the ILU Requestor.) When Jim arrived and immediately
produced the idea of transparently traversing and calling Python
objects, we immediately dropped the ILU Requestor and became devoteees
of “object publishing”.
I remember thinking last year, after all this time, most packages
haven’t caught up to what Jim had on his laptop after the plane ride
back. So many systems seem more like the “Gateway” in CGI, than
systems that adopt the object model of the web. I suppose the rise of
dynamic languages in the mainstream has now made this idea more
realistic.
Stephan Diehl brings up an interesting point regarding WebObjects. I’ve found some stuff covering the 1.0 that he’s referring to including their original WebObjects 1.0 press release.
As I never actually ran it, I don’t know if they were using object publishing at that time. From the press release, it seems that the first 2 versions of it used normal scripting techniques while the Pro version allowed the use of Objective-C. No idea if that means it actually did object publishing though.
If anyone used WebObjects 1.0 and would like to speak up… :)
Mr. Fulton does indeed deserve credit as a visionary, but let’s get real: lack of a screencast isn’t the only reason—or even the primary one—that Zope missed the boat.
Making the implementation of any code beyond the trivial UTTERLY MADDENING compared to plain Python has probably dampened Zope’s success more than all other factors combined. I can’t even begin to count the number of hours I wasted on bizarre error messages, acquisition spaghetti, and excessive security restrictions (e.g., couldn’t access members of mx.DateTime objects in Python Methods) before I finally gave up on Zope.
Although Zope 3 claims to fix all of Zope 2’s most prominent flaws, the sad fact is that Zope 3 may be doomed by association. I have the impression that there’s a large pool of embittered ex-Zopeheads who will reflexively shy away from any system bearing that name.
That’s not necessarily a fair or rational stance, but consider how the perception of a technology is irreversibly forged by its initial characteristics, however unjustified that perception may have since become:
- “Java is slow.”
- “Lisp is some weird A.I. language that nobody writes practical code in.”
- “Functional programming is an academic oddity that’s useless in the real world.”
- “Zope makes the first 30% of an app a breeze, and the last 70% an agonizing hell of spaghettified complexity.”
Don’t get me wrong: I’m not condemning Zope 3, just expressing my belief that “Zope” will need to expunged from its name if it is to have any shot at popularity among programmers (as opposed to less technical people, who rightly love Zope 2).
Ben – Regarding WebObjects, yeah, the first versions had Objective C only at the higher levels.
As for object publishing, WebObjects did it, but in a different way. If you go to a WebObjects application like The Apple Store you’ll notice that the URLs are very ugly. What WebObjects does well, and has always done well, is very impressive state management. I believe that WebObjects basically targeted itself primarily towards OpenStep developers who used the Enterprise Objects Framework as another kind of user interface to put on their EO Models.
Going back to the mid eighties, NeXTStep (and its descendents, all the way through Cocoa) heavily promoted MVC development and provided tools that made it easy to develop that way. So with WebObjects, you could take the M and put a web based VC on it – effectively publishing the objects in your model onto the web. Features were added in later years like 'Direct To Web’ and 'Direct To Java’ which are basically like Scaffolding in Rails – a dynamic user interface made by introspecting the model.
WebObjects ugly URLs are managing state in a way that the developer of the application doesn’t have to think about like in most other apps. Cookies, sessions, request variables, are all basically transparently managed, leaving a web UI that is nearly as solid as a regular desktop application UI. How the dispatching and responders work out in this situation is a lot more complicated (under the hood) than most of the app frameworks and servers mentioned in the post.
So WebObjects was about publishing objects on the web. But it did it (and still does it) in a way that’s quite different than most other solutions. For years, it was the only other web technology besides Zope that I respected. Remember – WebObjects and Bobo/Principia/Zope had this object oriented web publishing mentality 8-10 years ago.
(By the way, you can publish “prettier” URLs with WebObjects using the 'DirectAction’ way of publishing. This is a method thats more akin to how many current web developers think, has less overhead than the full WebObjects way, and is well suited for serving up content whereas the full WebObjects suite is well suited for serving up rich applications).
> “It’s also interesting to consider whether or not having more possible dispatch/responder styles is beneficial or detrimental to the popularity of a specific framework.”
I’m really interested in this question in a more general way, because the “convention over configuration” aspect of Rails seems to have won a lot of people over.
Specifically with dispatch/responder styles, it seems hard to justify the trouble of learning multiple styles in one framework (not to mention the added responsibility of maintining them in the source), simply because setting these dispatch rules is both very important and very infrequently done. In building an app, you might edit these things a couple of times, and you want them to be correct immediately. It’s not an area of the app where personal preference or coding style matters much. If we were talking about user interfaces, I might compare it to when “wizards” are appropriate: when the user is doing something tricky, which she does infrequently, and for which the result had better be correct.
Now, in other areas of these frameworks, things like coding style or personal preference matter a lot. If you don’t like the way Rails’ ORM library works, well, you’re going to be pretty unhappy every minute you spend in it. “Tuning” the defaults in those kinds of areas is pretty crucial to making people want to use the framework. I think that’s where the really interesting questions are.
Maybe you forgot Nevow…
I consider Karrigell, is it possible to add it here for benchmark as well?