Groovie

So tonight I took the plunge, and converted the blog to Typo. It’s totally slick, and I’m digging it. Plus when I want to tweak stuff, I don’t have to mess around with Perl.

It’s currently lacking a feature I had grown accustomed to, Technorati auto-pinging. I’m thinking this would be a good thing to contribute, so I’ll start digging into the code and see what needs to be updated.

Migration was also a bit of a hassle, as it appears the MovableType 3 migration script only is happy if you use MySQL for your old and new blog. Getting it migrated to Typo required dumping the MovableType database using pg_dump, and loading it into the Typo database. Then the mt3.rb script needed to be edited to remove all the table prefixes. After this, the script happily imported all my prior posts, comments, and trackbacks.

I’ve begun porting the Routes system from Rails over the last weekend. It’s definitely been a good brain workout, as the one Nicholas Seckar wrote that drives Rails is quite a brain-fuck. His version actually does quite a bit of code generation based on the routes one sets up. At first I tried to do a fairly direct port of the Ruby code behind this to Python, but my brain just couldn’t wrap itself around what he was doing (He’s told me that quite a few others have said the same).

This left me kind of discouraged, and I left the project (more of a thought at that point) alone and proceed to replicate most of Rails ActionMailer in Python instead. Partly because I liked how ActionMailer assembled mail, and partly because after hearing a talk by Alex Martelli on Metaclasses and other Python “black magic” I just had to find some excuse to use a metaclass.

Why not just use Rails?

Now, before anyone starts yelling “Just use Rails!” at me, I should explain something. I am using Rails, and I rather like it. However, most companies wouldn’t be too thrilled with their employee’s changing the entire codebase of the company every year when the latest, greatest, way to write webapps comes along. Plus, since I already use Python extensively, a lot of stuff that makes life pleasant in Rails-land can be fairly easily adopted to Python-land.

Do I really want to re-write 5k or more lines of Python libraries in Ruby just so I can use Rails? Should the entire web team learn a new language and framework just because its a bit better than the current way? The time spent doing all this wouldn’t be paid off for at least a year or more, if that (and maybe some other language+framework will be the new thing by then). When it comes down to normal Python libraries, I don’t save any lines of code by moving to Ruby. So in the end, it makes sense to adopt part of Rails that make my life easier.

In Case You Didn’t Know

The Routes system from Rails is used by Rails in a lot of functions. Virtually all functions that require a URL to be written (url_for, link_to, form_tag, etc) all use the Routes system to figure out what URL to put in place.

In the Python world, as far as I know (and I’ve looked quite a bit), there is absolutely no web framework that will convert from controller/action -> URL.

Update (9/26/05): Phillip J. Eby rightfully pointed out that other Python web frameworks can do this. I should’ve been more specific in that none of them operate in this manner. Since putting word out about Routes I also found out Aquarium can designate routes for dispatch/generation in this manner, however Aquarium cannot do automatic route lookup to find the shortest URL. This ends up hard-coding the template URL to the route.

Plenty of web frameworks are happy to convert a regular expression into a dispatch command and let you tweak it (Myghty, Django, etc.) but they are all helpless if you want to generate a URL from the commands they used to dispatch.

Progress

Currently, I have URL generation (controller/action -> URL) and URL recognition (URL -> controller/action) working and passing all the tests that I’ve ported over from the Rails route testing. Since my version of Routes is independent of the framework, its my hope that multiple Python web frameworks will be able to incorporate the finished Route system I’m building. The URL -> controller/action results in a dict, that the web framework can choose to deal with as it pleases.

The first framework I plan on integrating this with is Myghty as the resolver system is fully user-customizable, no hacking on the framework source is needed to plug it in. The dispatch model will then follow Rails and make the same decisions for the user (controller goes in appdir/controllers/, name_controller.py, etc.) that Rails does.

Once I add a few more tests, and try and clean up my code some, I’ll put it out there. I’m considering a BSD license right now, in the meantime, here’s a sample:

>>> from routes import Mapper
>>> m = Mapper()
>>> m.connect('page/:id', controller='content', action='show_page', id =1)
>>> m.connect(':controller/:action/:id')
>>>
>>> m.generate(controller='content',action='index')
'/content'
>>> m.generate(controller='content', action='show_page')
'/page'
>>> m.generate(controller='content',action='show_page',id=4)
'/page/4'

Speed

My implementation choice was to rely heavily on the speed of set operations using the built-in set/frozenset functions in Python 2.4. Under Python 2.3 the sets module provides an ImmutableSet which I’ve also tested my port with. I was actually quite impressed with how fast the sets module is in 2.3, but there is a three-fold speed increase using Python 2.4 built-in sets.

Even under 2.3 the speed should be more than sufficient for most applications, plus, this is a first revision of the code. I’d hardly expect it to be the peak of efficiency right now.

What’s Missing

I currently haven’t implemented memory of the match that got to the current page. Rails does this so if you’re inside a template that was matched by ’/post/show/20’ and you call link_to :id => 30 it would give you a url of ’/post/show/30’. It does this by remembering the match that got you there. Since this match would be setup inside the web framework, I’m considering how to implement it without tying one to a specific framework.

The other noticeable feature missing is Named Routes which act as a sort of macro on top of the normal url generation calls. This shouldn’t be too tough to add at a later point.

Parting Thoughts (ala Jerry Springer)

Before you submit some comment saying I’m an idiot for porting this and should drink more Ruby kool-aid, don’t, please. I’m already drinking it, and it tastes quite good. But just because I like Ruby, doesn’t mean my taste for other languages has changed. Languages have their strong points, and they have things they lack (Yes, they both lack some things). When I’m writing Python, my web framework of choice is so very close to perfection for me, Routes is one of those things that would make it that much sweeter.

If you’re interested in adapting this to your Python web framework and want to start playing with the code I have so far, send me an email and remember boys and girls, be safe.

Please Note: Reading the last post would help greatly for this one, and this post does end up comparing aspects of Django with Rails in terms of making decisions rather than actual usage.

As I mentioned in my last post, Rails has been greatly helped by the decisions it makes for you. It decides the layout of your application directory, the directories you put stuff in, where you put what parts, etc. These decisions thankfully were done by someone familiar with good application design and had most likely even read Martin Fowler’s Refactoring to Patterns.

What results is a clean, well thought out application that you didn’t even need to think out very much. Before you know it, you fall into these useful and powerful design patterns without even thinking about it (I’d hope you would at least, they’re good patterns for a reason).

A lot of web frameworks make the assumption you are familiar with these patterns already, will setup your application to follow them, and use them in a way that makes sense. Part of the reason for this is because some web framework developers are rather appalled at the thought of making these decisions for the framework user. To be making those decisions for the framework user is looked at as holding their hand, and typically the programmers who write frameworks don’t need any hand holding and don’t expect their users to either (I’m assuming this based on the frameworks I’ve tried).

For example, during Ian Bicking’s test run with Rails he noted that having such a directory hierarchy setup for him put him off a little and mentions people arguing about file layout. I’ve seen very little, if anyone else, even comment on the directory structure that Rails lays out for you. The other irony of this particular post is that a lot of the stuff Ian finds “wrong” about Rails is actually Ruby’s “fault” (Implicit passing of names, mix-in’s, etc.). But moving on….

Ian clearly doesn’t want to lay down the law for users of his software, neither do other Python web framework developers. Having choice in how you lay out your application, with no clear direction what goes where, is considered a good thing. I don’t think there’s anything wrong with this approach, if the users you’re aiming your framework at are expected to be experienced, professional, programmers that are quite familiar with web applications and what works best for them. This does mean the audience will not be that large, and I think thats clearly reflected in the small communities most Python web frameworks have. While their communities are small, I’ve found them to be highly skilled and you get great answers to questions on their respective mailing lists.

The one Python web framework that has almost overnight gone from hardly any users, to at least a few dozen (probably more, but even a few dozen is quite good for a Python web framework), is Django. Django makes a lot of decisions for the user, more so even than Rails by a pretty good amount. While Rails was designed as a web application toolkit, and extracted form a large webapp, Django is designed more as a CMS web application toolkit as its been extracted from newspaper-style web sites. Thus, many of the decisions Django makes for you assume you will want users, user-access to the database objects, etc.

The question is, if someone else is making decisions for you, how important is it that they be good decisions?

One of the decisions the Django team made originally irritated me quite a bit. The Django framework is probably about 95% MVC in nature. It has database files that clearly are acting as Models (which it calls models), logic oriented files traditionally called Controllers (that it calls views), and template files that display the results to the user in a typical View role (though Django calls them templates).

The FAQ on the Django site notes that if you squint, sure enough the observations I just made do pop out, however because of the 5% non-MVC I mentioned they decided the names didn’t work. Rather than calling them by what they obviously were (or closest to), they swapped around names in a way that I think is bound to confuse Django programmers when they go into a real MVC environment and tell someone they edited the view when in fact it was the controller. If they had come up with completely different names that didn’t relate to the MVC paradigm at all, I think that would help to avoid later confusion on the part of a Django user.

Despite the naming of these parts, the decision made for the user is clearly for a MVC-style (or influenced) environment which I think is great ( please change the naming! ;) ). So what is the point of this whole thing?

I think its a good thing for developers of toolkits and especially frameworks to make some more decisions for the end-programmer of their work. This makes it easier for not so experienced programmers to get started quickly, and if you’re using a good design pattern even the experienced dev’s wont have anything to complain about. The two frameworks that currently are getting a lot of attention make a lot of decisions for the programmer using them, and this definitely increases the user-base (assuming you want a large user-base). Even though I know good practices on laying out stuff, it makes my life that much easier when its done for me.

I’ve been reading a lot of various blogs lately, quite a bit of loud thinking and some 37 signals vs noise as well. A mantra I see over and over regards simplicity, and focusing on doing a few things extremely well. This is reflected in the 37 Signals products, Basecamp, and Backpack. It also bubbles over into the web framework these are all built on, Ruby on Rails.

The Rails framework has a lot of appeal, in large part because of how easy and simple it makes the vast majority of mundane web programming. The sacrifice is completely intentional, and summed up with “Convention over configuration”. This mantra is in the Agile Development with Rails book, and has been uttered on Loud Thinking as well.

Ironically, the people I know that dislike Rails do so because of the “lack of flexibility”. Many decisions have definitely been made for you when you create a Rails application, though I see no harm in this because every decision follows what most regard as the best practice and pattern for that context.

While Rails and Ruby are skills I’m currently learning, I spend a lot of time working with Python. The Python programmers I’ve talked to generally regard lack of flexibility as very unappealing, and would happily sacrifice simplicity or making any decisions for the programmer (I’m referring to Python toolkits/frameworks intended for use by programmers). The assumption is that the programmer will read how to use the tool, see that there’s a half dozen ways to use it, and plug it in as he best sees fit.

Having more ways to use it is a good thing, with no clear direction on any sort of best practice or pattern provided typically. Strangely enough the majority of toolkits and web frameworks for Python generally have rather small groups of users. Is this a coincidence?

My favorite framework, Myghty recently got the very cool ability to call module components implicitly . This added two more mechanisms of dispatch control to the resolver, very powerful stuff. It’s also rather confusing, not because how it works is actually difficult, but because now the way you should use it has become even less certain with the added choices.

The fact that there’s so many ways to use it has definitely hampered the ability of people to understand it and use it effectively. The powerful flexibility has made it powerful though, for totally different paradigms to be built on top of it. For example, you could replicate the pipelining methodology that AxKit uses by configuring the resolver to pass the different extensions through recursively until an “end” is signaled. Or you could configure Myghty to dispatch control to a Controller exactly how Rails does (which I’ve worked on doing).

In the end though, why should I waste my time replicating what another framework already has? I’ve got to admit, even though I know how to pick up the tools and plug them in, I’m still attracted to simplicity.