Posted by ben
Wed, 29 Aug 2007 17:04:24 GMT
Routes has been a wonderfully successful project of mine as its used not just in Pylons but quite a few other small WSGI apps. It’s even been integrated as a CherryPy URL dispatching option for those using TurboGears 1.x or plain CherryPy. It’s come a long way since 1.0 and has diverged on quite a few fronts from the Rails version that it was originally duplicating in functionality, which has me thinking that perhaps its time to consider 2.0 and what that will mean.
First, I think there’s quite a few behaviors that don’t make sense and probably never have. The Route minimizing functionality sounds neat and fulfills the Rails ideal of ‘keeping urls pretty’ yet suffers from a fundamental flaw… they result in multiple valid URL’s for the same page. James Tauber covers some of the implications of this in addition to the issue with relative URL’s not working right either. The example he cites is even worse for Routes since Routes can easily result in multiple URL’s mapping to the same controller action.
Routes 2.0
To solve the multiple URL issue, and also address some Named Routes confusion that recently hit the Pylons mailing list, Routes 2.0 will do a few things differently than Routes 1.X. Many things will be significantly more explicit and more predictable as a result. In addition, I want to add more functionality so that Routes can be the end-all of URL generation in addition to URL dispatching, even for use purely generating links conditionally that aren’t even used for URL dispatching.
Some of the key changes planned for Routes 2.0:
- Routes recognition and generation will always be explicit
Currently, there’s an explicit option that removes the keyword controller/action/id defaults, 2.0 will not have these implicit defaults.
- Defaults don’t cause minimization
Defaults just mean they’ll be used, they won’t result in Route minimization which increases the amount of URL’s that end up matching to a single controller action.
- Trailing slashes shouldn’t be an issue
Without routes being minimized, a route such as ’/home/index’ will always be ’/home/index’ instead of being minimized to /home/. This will resolve the trailing slash issue.
- Named Routes will always use the route named
A named route currently just means that the defaults for that route will be used during route generation. This currently can cause confusion because people believe that the named route actually forces that route to be used during generation.
- Generation-only routes
A new option, which will result in routes that are used purely for generation. This option will likely be used primarily for static resources which may be on other servers, or may need the domain rotated so that the browser can do parallel resource loading. For example, one would be able to provide a list of domains to be used, and the generated links will rotate as desired on the page to split page resources over multiple domains.
- Redirect routes
Sometimes, (especially when replacing legacy apps), its desirable to slowly migrate URL scheme’s from the old to the new rather. While URL’s should never change, sometimes the system being replaced has horrid URL’s that violate all URL recommendations. Being able to provide a smooth migration path from the old URL’s to the new ones is handy, and permanent redirects are respected by many search crawlers as well.
Migration and Compatibility
Routes 1.8 will include options to turn on behavior that will be the default in Routes 2.0, and if you like how Routes 1.X works there’s no need to worry, it will still be maintained for the foreseeable future. It’s currently extremely stable, and has a massive unit test suite to ensure it operates as designed.
Add Your Desired Feature Here
What other features are Routes users currently looking for?
Posted in Python, Routes | 13 comments
Posted by ben
Mon, 27 Aug 2007 02:08:46 GMT
I’ve been seeing more and more movies I wouldn’t mind actually buying… except that I have a 56” HD TV set. It looks amazing, pretty good with DVD’s, but really amazing with HD so if I’m going to buy a movie I sure as heck want it in HD.
Unfortunately thanks to the tech companies refusing to come up with a single standard, I can either buy a $200 HD-DVD add-on for my Xbox 360, or a PlayStation 3 which I’ve heard is the cheapest Blu-Ray DVD player available. And of course, I really need to buy both if I want HD movies since some studios are only putting their movies out on one or the other format.
The end result? I’m not buying HD movies for HD-DVD or Blu-ray because its pathetic to have to have 2 players around (no, I don’t care about universal players). In addition, I’m not buying DVD’s anymore since I know that in a year or two I’ll want to re-buy it in whatever HD format wins (please let one of them win in a year or two!!!).
What are other people doing in this situation? Buying one or the other? Still buying DVD’s? Or not buying any movies at all until they get it straight?
Posted in Rants, Thoughts | 10 comments
Posted by ben
Tue, 21 Aug 2007 02:59:15 GMT
Finding out whats going on in an application is always a key point when trying to figure out what’s happening when things don’t go the way you expect. In these kind of annoying situations, following the log of how your request mapped in and what was going on can save a significant amount of time. TurboGears has had great logging throughout it, and with 0.9.6, Pylons add’s the same thorough logging.
But once the entire request is mapped out, it can be a real chore to track through them, toggle modules you want and don’t want logging for, etc. Chainsaw has had excellent graphical support for navigating logging output, and with the help of Phil Jenvey’s XMLLayout package its easy to output logging statements in a format for use with it.
With the recently updated Chainsaw section in the Pylons logging docs, you too can be lumberjacking with Chainsaw.
And of course, a shot of Chainsaw in action:
Posted in Pylons, Python | 2 comments | no trackbacks
Posted by ben
Sun, 19 Aug 2007 04:38:51 GMT
Sometimes, it amazes me whats possible fully utilizing WSGI middleware in an application stack. While it likely isn’t something totally unique to the framework, the relative ease with which it can be done still sometimes gets me to grin.
Tonight, a Pylons user on the IRC channel (irc.freenode.net, #pylons) asked if it was possible to get a URL laid out so that /s/SOMETHING would map into their ’s’ controller, with the second part passed in as a variable. That alone is pretty easy, however the additional requirement was that the controller action would change depending on the user’s “type”.
There’s two ways to deal with this, the first of which is the only possible way in many frameworks. Have every request to the URL map to a single function, and in that function load up the session and call the appropriate function to handle the request based on their user type. This way works fine in Pylons too, but thanks to Routes and WSGI middleware we have another option.
Routes has a lot of capabilities to it, there’s been numerous additions to the Python implementation that the Rails version is not capable of. One of them is the ability to alter the resulting URL match dictionary in various conditional functions. To toggle the controller action used, we’ll be using the ability to pass in a function to Routes conditions that can alter the resulting match.
This condition checking function has full access to the WSGI environ so if you wanted to restrict a specific controller/action combination to people referred from Slashdot, no problem! You can carefully fine-tune the conditions required for dispatch at the same place you define your URL resolution.
Since Pylons uses Beaker for session handling via WSGI middleware, the session object will already be available when our Pylons app gets called. Beaker loads the user session into environ['beaker.session']. Given this knowledge, we can write a conditional function for use with Routes like so:
def check_user(environ, result):
session = environ['beaker.session']
user_type = session.get('type')
if not user_type:
result['action'] = 'index'
elif user_type == 'admin':
result['action'] = 'view_action'
else:
result['action'] = 'not_logged_in'
return True
map.connect('s/:domain', controller='s', conditions=dict(function=check_user))
Viola! Now Routes will run the function provided to see if it returns True before accepting that as a valid match. In the process, the action used will be set as desired. I’ve always thought a good sign something is well designed is when people can use it in ways you didn’t originally anticipate. If that’s the criteria, I think Routes succeeds and then some.
Disclaimer: Yes, I wrote Routes, and a good chunk of Beaker and Pylons, so I might be biased and tooting my own horn. :)
Posted in Python, Routes | no comments
Posted by ben
Sun, 19 Aug 2007 04:23:21 GMT
In light of Phillip Eby’s recent post concerning WSGI Middleware as harmful, I’ve had more than a few thoughts on the issue. None of them are all that new, but given the post I think its useful to get some of them out there.
First, I agree 100% with PJE’s post. The issue it raises results in two lines of thought. Without a doubt these objects using the WSGI specification should not be called WSGI Middleware or WSGI Applications. This means that either:
1) People should stop using the WSGI spec for non WSGI application/middleware objects.
Or…
2) WSGI needs new terminology for this application of the specification, and should not be muddling up the WSGI middleware/application definitions and environ namespace with meta-framework API’s.
To answer either of these possibilities it helps to evaluate why things are developing like this right now, and almost all of it comes down to one thing… tool developers are incredibly picky and opinionated.
To avoid further muddling up WSGI definitions, I’ll be using the following term:
WSGI Framework Component (WFC) – A WSGI specification based component that acts possibly as either a WSGI application or WSGI middleware, or some mix of both. Example, a WFC that ensures users are logged in before accessing your WSGI application (thus acting as WSGI middleware), but will render its own form and go through its own login procedure should they need to login (thus acting like a WSGI application). This is referred to as a WFC because using WSGI is seen as a way to avoid binding it to a specific framework, while its clear that an application using it actually requires it to be there to operate (thus its not WSGI middleware).
Dealing with Disagreement the WSGI Way
WSGI makes it a lot easier to disagree, yet still harness the code and development efforts of those that disagreed with you. WFC’s allow re-usable code that isn’t utterly dependent on your framework of choice as long as its WSGI compatible. Thus the fact that many frameworks are WSGI compatible at various levels makes it very enticing to build re-usable components at the WSGI level instead of using framework-bound API’s.
A thought that started cropping up, and hitting the Web-SIG mail list, and which I believe one target of Phillip Eby’s post, regards putting ‘standard’ keys into the WSGI environ for applications to utilize. This would to an extent allow you to swap WFC’s that do similar things, but in different ways. Maybe you want to swap two resolving middleware, so you use the wsgi.org routing spec to determine how the URL was resolved then dispatch appropriately. You can now swap WFC’s that do routing to an extent since there’s a further specification in place.
There are other wsgi.org specifications underway, and lots of various WFC’s being developed. If I’m using Pylons, and someone using CleverHarold or some other WSGI type application makes a WFC that does something cool, I can use it as is without having to agree with the design of their application of WSGI based framework.
Compare that to a CherryPy2 filter, CherryPy3 tool, or Django middleware. To use any of those, I need to use the whole framework. For CherryPy, this may not be the case in the future should it allow a CherryPy tool to act in the middle like middleware. Robert Brewer has said in the past he wants CherryPy to be the end-point and not continue dispatching elsewhere which would rule out its use as a library for a WFC. Thus, I’m labeling CherryPy as a framework in the context of WFC creation, while Paste and Yaro are libraries usable both in WSGI apps/middleware and in WFC’s (Note that CherryPy3 is almost capable of being used as a WFC, except it can only dispatch to non-CP3 WSGI apps).
Going Overboard with WFC’s and WSGI
The other aspect to these new WFC’s that I think Phillip hit on the head, is that there’s quite a few being pushed into this layer that really don’t belong there. No one has put out a solid checklist to know when something should be in a library, a plugin API (possibly using setuptools entrypoints like the TG Template Plugin API), actual WSGI middleware, or a WFC. As a result, there are WFC’s that do very little, and in some cases have no reason to be operating at the WSGI layer.
So, I’m going to propose some guidelines, a rough draft as I’m sure there’ll be plenty of useful feedback, on when something should be considered for a WFC and when it should be a library. It’s also useful to note that libraries can operate on things from WSGI, vs WFC’s which get plugged into a framework/app as if it was WSGI middleware.
The guidelines for WFC’s should roughly follow the same guidelines you’d want for any WSGI middleware. There’s some conditions that make it more obvious than others on where some functionality belongs and of course there’s always exceptions to the rules.
Signs some code would be a good candidate for a WFC (It’s assumed that if you’re thinking of making a WFC, you will be wrapping your actual ‘application’ with it):
- A set of operations needs to always occur before and after the application is called, and requires knowledge of the incoming and outgoing headers
- Modifications are done to the HTTP headers and/or content being returned to the client (cookies, HTTP caching, content transformation)
- The application may not be called at all (authentication, authorization, conditional dispatching)
Note that the first condition doesn’t apply to functionality that merely requires something to setup. It’s overkill using WSGI just to run a function at the start of every request - even if it needs environ - there’s no reason you couldn’t just put the function call in your app, call it every request, and put the function in its own module/package (thus easy to re-use).
A lot of the Paste functions operate like this, and many of them just take the environ as their call giving you a nice
API without requiring a
WFC (which Phillip Eby advocates as well):
request = paste.wsgiwrapper.WSGIRequest(environ)
print request.cookies, request.path_info
There’s no reason a variety of
WFC’s I see
on the WSGI middleware and utils list couldn’t operate like this as well. Take
wsgiakismet for example, which parses the form submission and screens it against Akismet. The example as a
WFC actually looks more involved than I could see a library based version looking:
# theoretical library version of wsgiakismet
from akismetverify import verify_akismet
def app(environ, start_response):
# Wordpress API Key and website name are required arguments
usersub = verify_akismet(key='3489012ab121', site='http://blog.example.com/', environ)
start_response('200 OK', [('Content-type', 'text/plain')])
return ['Comment is %s' % usersub['comment'][0]]
Note that using it like this as a function that takes environ and the other 2 keys actually makes it easier to use than the original sample requiring you to import cgi and re-parse the form vars.
So some good ways to know you might be on the wrong track with a WFC:
- Only a few things are being done on setup, and stuffed into environ
- Some environ keys are manipulated
- Your code never alters or does anything with the status codes, headers, or content
- ... or none of the conditions to know when it should be a WFC exist
I’m sure there’s more criteria I’ve missed, and it’d be great to have a page possibly on wsgi.org regarding design decisions to hopefully avoid having anymore functionality pushed into the WSGI layer when there’s no good reason for it.
Posted in Python, Thoughts | no comments
