Groovie

In a recent Wired article regarding One Login, reference is made to a new social style network called GoingOn. The article spends most of its time focusing on one site that hopes to aggregate functionality that currently is split between Blogger, Flickr, Friendster, and Bloglines (for the most part). However, the thing it misses is what I previously discussed regarding the lack of a working distributed identity system.

After looking around more, I’m happy to say there are indeed working identity systems out there. Unfortunately the most promised of them, the Liberty Alliance doesn’t seem to have much oomph behind it, but two others that I previously didn’t know about are now out there.

The first is from the folks at Microsoft, which they’ve called an Identity Meta-System (or something like that), which is described over at vnunet. It seems to be rather tied (or at least integrated heavily) to Microsoft technology (go figure!), and will be included in Indigo and other various Micrsoft technologies. As a mainly open-source coder, this has little appeal to me, nor am I about to start using Microsoft API’s to write my websites and web code. The standards utilized by Microsoft for their Federated Identity are generally known as WS-* for some reason I’m too lazy to investigate.

The second is much more appealing (to interested users and web developers), and has actually been around for a very long time in a primitive form (2000 is ancient by web standards). The home site appears to be the identity commons, and the current sole Identity Broker is 2idi, the organization behind the standards is XDI. They’ve made the entire code-base they run the Identity Broker on, open-source under the Affero General Public License to ensure that users are never locked into just one Identity Broker (Yea!).

If you’re curious how the Microsoft and Liberty Alliance methodology differs, idcommons has a useful FAQ addressing the differences.

The most exciting aspect for me, is that all the technology behind the XDI approach is completely open-source, and geared towards maximum user flexibility and empowerment. The user gets to move data between Identity Brokers, and every care has been made to ensure the user is never locked into a single Identity Broker. Actually, the most exciting part, is that it works right now. :)

They’re currently preparing to switch to a SAML-2.0 backed code-base, however the code they have only works from PHP, Java, and Perl. If you want to try it out, here’s how to get an i-Name, and you can try it out on those two sites. Also, a developer made a ISSO (I-name Single Sign-On) authentication system for WordPress which is pretty cool.

So what’s stopping ISSO from being used on more websites? It’s free, its open-source, its standards based, its not controlled by a commercial corporation….

It needs Python libraries!

I should mention, when I first wrote this as far as I knew, there was no Ruby version. There still isn’t a public one, but Victor Grey is fairly close to a Ruby version with a full Rails rig to go with it which I’m rather looking forward to.

Anyone want to help? I’m tired of remembering a zillion usernames and passwords, and with ISSO on the horizon I shouldn’t need to, all the Python web frameworks will be a bit better (at least the sites that use usernames/passwords) with an easy way to use ISSO.

By the way, for a useful overview of SAML, there’s a very detailed write-up of SAML2 on xml.com.

Two of my favorite and most often-used Python packges are formencode and sqlobject. Using these packages together is done fairly often, but I’ve rarely seen any documentation describing a full form display, validation, and insert into a database of new data from said form.

In this example, I’ve tried to wean down the code as much as possible to get a very concise snippet (only 12 lines of web page Controller code) that will:

1. Display a new form to the user
2. Handle form submission
3. Validate the form
4. Coerce form values into the proper types for the database
5. Insert the form data to the database, or
6. Display the form with error messages, and maintain their values

The nice thing about using formencode, is that 3 of the 6 steps I listed above are handled by it in a fairly automatic way. Formencode will check the data given a form schema, coerce the values into the Python types you ask for, fill in errors should they occur, and maintain the existing values.

I’ll be using Myghty for this, but since all I’m really pulling from it is the request args, it should be pretty obvious what to change for whatever web framework makes you happy.

formencode

First, lets take a look at our basic form:

# myform.myt
<html>
<head><title>basic form</title></head>
<body>
<form action="/mypage" method="post">
Username: <input type="text" name="username" size="26" />
            <form:error name="username">
Age: <input type="text" name="age" size="3" />
            <form:error name="age">
<input type="submit" value="Send it" />
</form></body></html>

To validate this, we’ll setup a formencode form schema to run this through. I should note at this point, that the formencode web documentation kind of sucks. However, the doc strings are plentiful, and extremely useful for figuring out which validator to use in addition to having examples of each. To keep things organized, I usually place related form schema classes under the same module and import it as needed.

The other thing you might notice about the form, is that it has form:error fields. These are used by the formencode parser to put in the error message that the validation triggers. This lets us put the error messages right under the boxes they occured in. The drawback is that we have to process the form before first displaying it to strip out the form:error fields.

Here’s our simple schema to validate the above form:

from formencode import schema, validators
class UserInfoSchema(schema.Schema):
    allow_extra_fields = True
    filter_extra_fields = True

    username = validators.String(not_empty = True, max = 50)
    age = validators.Int(not_empty = True)

Hopefully the above should look pretty obvious. The allow_extra_fields bit is needed so that we can pass the entire request argument dict into formencode without it tripping up if there’s “extra” keys it didn’t expect (like the submit button). Since we’re going to be passing the dict we get back from formencode directly to sqlobject, we include filter_extra_fields to remove anything that our sqlobject isn’t going to like.

The form schema needs to include all the fields the database is going to take, since we’re stripping off anything it doesn’t mention. The Int validator not only ensures that the value is an int, but will change it into a Python integer in the process.

sqlobject

Now that we’ve handled validation and value coercion, lets have a look at the sqlobject class. I’m going to “cheat”, and assume your database for this was created like so (in Postgresql):

create table user_info (
    id serial primary key,
    username varchar(50),
    age int(2)
);

Since I’m feeling lazy, we’ll rely on SQLObject to pull the table info from the database giving us a SQLObject class like so:

from sqlobject import *
class UserInfo(SQLObject):
    class sqlmeta:
        fromDatabase = True

Personally, I think if you just leave the whole thing empty and put pass in for the body, it should assume you want it populated from the database…. but the above will do the trick. Please note I’m using the sqlmeta class to define this, which is used in the recent svn builds of sqlobject. If you use the release on the site, you could replace those two lines with _fromDatabase = True instead.

Putting It All Together

Now that our form, validation, and sqlobject schema is all done its time for the meat of the matter… the web page controller. Getting this function called will vary depending on your web framework, so I’ll just assume you can figure out how to get it called, here’s what it looks like in Myghty using implicit module components:

from formencode import htmlform
from ourschema import UserInfoSchema
from oursqlstuff import UserInfo

def mypage(m):
    html = m.scomp('/myform.myt')   # load the form into a string
    form = htmlform.HTMLForm(html, UserInfoSchema())
    if m.request_args:
        form_result, errors = form.validate(m.request_args)
        if errors:
            errorForm = form.render(m.request_args, errors)
            m.write(errorForm)
        else:
            UserInfo(**form_result)  # database insert
            m.subexec('/thankyou.myt')
    else:
        m.write(form.render())

And there you have it. In a brief 12 lines, we handle displaying a new form to a user, and handle form submission, validation, and database insertion while ensuring that the string values are coerced as needed before database insertion. This task is done quite often in web sites, so making this task as painless as possible is a real time saver.

Hopefully this will help out anyone out there, who was wondering about quicker and easier ways to handle cases like this. If you have any thoughts/suggestions on how to streamline this further, be sure to leave a comment.

I’ve been thinking a lot lately about web programmers and the web frameworks they choose, or don’t choose, and why. I’m mainly going to talk about Python Web Frameworks as the majority of them have small communites, and possible reasons this could be.

I only started using Python for web development about a year ago, and it took me about a month to settle down on a web framework. In that time, I looked over at least a dozen different frameworks. There’s so many python web frameworks, quite a few people have actually setup entire pages and sections of their site just to covering them all.

I think part of the reason for the proliferation of frameworks is because of the nature of many Python programmers, as I briefly mentioned in a prior post on Making Decisions for Others.

The recent appearence of Django on the Python web framework scene I’m sure has quite a few other Python web framework developers wondering, “Why isn’t the web framework I made getting this much attention and use?”

A Common Base

Many of these same people would like to blame it on hype and good marketing. While that will certaainly boost initial usage, I don’t believe it will create a lasting user base. I think a huge driving factor behind Rails and Django, besides for the hype and marketing, is the fact that both of them make a lot of decisions for you. These decisions start the users all off at a common base of understanding.

The linear progression from:

1. Never used the framework
2. Wrote the tutorial app
3. Wrote their own basic webapp
4. Wrote an advanced web application

Makes it easy for people a step or two up, to help other new users join them. Because the steps they all take are the same steps to achieve greater understanding of the web framework, they can easily help new users get to where they are. Most, if not all the other Python web frameworks I’ve seen are so flexible its hard to have a common base of understanding amongst new users. The process looks more like this:

1. Never used the framework
2. Researched the frameworks options and choices to find a possible starting point
3. Wrote a basic web application using method X
4. Wrote an advanced web app using method X

The flexibility of the web framework becomes an obstacle to a strong user-base in this case, as it fragments the users by the methodology they’re using to build their webapp. It also reduces the common re-usable components available, since different users will utilize different options of the framework and have possibly very different starting points.

Have a Tutorial Application

Also lacking from many Python web frameworks is a clear and obvious Tutorial application. Ideally the front page of a Python web framework should be an obvious path to become an experienced user of said framework. Such as:

1. Install the framework
2. Write a basic tutorial application
3. Look here/there for instruction as need to write your own more complex application

A good tutorial should leave a user feeling confident that they know how to install and start with a common base for writing their own web applications. It’s also amazing how many problems people can have just getting a framework installed and running in a minimal configuration. Having a tutorial that leaves them with a functioning web application gives them a big leap forward.

Since many users will do the first tutorial web application, other new users can give help to even newer users that run into a problem. This is where the common base effect really provides some power.

Methods of Fragmentation

The Python frameworks I’ve tried and used have fragmented their starting points and users in various ways. All of them as a result of their “flexibility and power”. Here are a few common trends of fragmentation I’ve seen:

• Let the user choose various template language schemes (Use ZPT, or Cheetah, or…)
• Let the user choose from web paradigm (MVC, page-driven, pipelined…)
• No base or example configuration for a fully working webapp (So everyone sets up their first application slightly differently)

The last one I listed, is probably the easiest to solve, especially with useful web framework template creators like Python Paste. Obviously, removing the first two will be seen by many Python web framework developers as undesirable. I think it’d really help the users though, as it gives them more in common with each other. If they all use the same paradigm, and the same template language with your framework, their ability to help each other increases and they feel confident they made the “right” choice as well.

Assumptions

I’ve assumed for the purpose of this post, that Python web framework makers are interested in having a large user-base. This isn’t always the case, I’m sure some just want a small, very experienced user-base that isn’t going to be asking basic questions like, “I can’t connect to my database like you show in the tutorial”.

I can understand that, but for the other Python web framework makers out there, try and consider some of the things I mentioned. There are a lot of Python coders out there, and a lot of them can live without having 4 template language choices and 2 different design paradigms. So when adding that feature that’d let people get so much “power and flexibility”, will it fragment your user-base?

Designing web pages is a pretty annoying task for most web programmers I’ve met, including myself. I’ll be first to admit it, I’m not too hot when it comes to web design. I haven’t even bothered changing the default theme for the blog engine I’m using.

What would be really slick is a sort of Web Design for Programmers book, ala Joel’s User Interface Design for Programmers writings and book. I’ve read quite a few books on web design that try to go over basics, but in the end for some reason it always feels like as a programmer, I’m missing those creative juices that result in an unique and interesting website.

There’s a fairly nice series with this exact title over at PeachPit.com that goes over some of the fundamentals. What I think would be ideal, is a website entirely devoted to this topic, with examples and layouts that are good starting points for your own web projects.

I searched around on Google quite a bit, and only came up with the PeachPit articles I cited above. A List Apart is a rather nice site to learn specific web design techniques, but doesn’t really focus on application of the techniques as related to the website on the whole. This part is left up to the designer, which doesn’t help a programmer much.

Having a beautiful website, or even just an aesthetically pleasing one can make a big difference even to other programmers, whether they want to acknowledge it or not. One of the clearest examples of this I’ve seen is with web frameworks and toolkits. A great looking website can get people into a toolkit/framework that might’ve otherwise moved onto a site that looked more “interesting”.

I’ve read quite a bit of Paul Graham’s articles regarding Lisp, how awesome Lisp is, how much of a dufus one might be for using a language programmed for dufuses.

Now, before I start, I should mention that these posts are quite old, and I don’t really want to start a flame fest over this again. However, I couldn’t help but notice today with a problem I had in my code, how easy Python made the solution (Zach points out the Lisp solution is quite succinct as well).

Without a doubt, Lisp excels at recursion, function/code generation, and closures. This frequently leaves Lisp looking like a god when you see how many lines of code other languages take to replicate the examples Paul chooses to compare (which always revolve around the things Lisp excels at as Paul Prescod points out)

Today, I came across a fairly common case, where I had a function taking keyword arguments and collecting them all. That would look like this in Python:

def somefunc(**kargs):

Now, if I want to take two of those keyword being passed in, and set some defaults so they’re not required but will always have something set in the function body, its rather easy:

def somefunc(keyone='default', keytwo='anotherdefault', **kargs):

So in one line, I have now added two defaults that will be available in my function body for use. How succinct is this in Lisp? What about in Ruby?

I cite two languages that came off very well in the Accumulator Generator shoot-off. They Ruby doesn’t do quite as well in this case, which I’ve actually encountered far more than the code generation cases Graham is apt to cite. (I actually like Ruby and am now using it quite a bit, I’ll be quite happy when it has keyword args)

What’s even worse is where in this thread that I referenced above, Graham at the end says he has no clue how one would create a basic class to handle accumulation in Lisp. I find that rather disturbing that something so easy in Python has Graham saying, “God only knows.” how to do it in Common Lisp Richard points out below that this is because PaulG is rather keen on macros, and not so big on OO. That makes complete sense to me as Lisp did not start out with OO features, those were added later when it was all the rage. When working in an elegant functional language like Lisp I can see why one would never have to consider OO.

That alone should indicate that many technical comparisons between languages can be easily skewed towards a language by using examples that heavily favor built-in abilities of the language one chooses to boast about.

In the end, I’m left with the belief that different languages have different applications. Claiming one language is the be-all, and is always better for any task is about as false as claiming that a language has no problems or issues.

Within certain realms it does make sense to compare languages, scripting vs scripting, functional vs functional, etc. But leaping to a comparison of functional/dynamic-typed vs non-functional/static-typed is typically going to result in some strange claims.

Anyways, if you feel like commenting, try and come up with an example of where Language X (that you use) has a very succinct solution compared to Language Z (all the others). It’d be great to compare some examples and see areas in which different languages fall flat on their face when it comes to succinctness. (Ie, in most dynamic languages, you’d have to add several lines of code to ensure variables are the type you want. A feature/annoyance of static-typed languages)

Update: An anonymous user kindly informs me that there’s no foundation for my claim that some languages are better in certain realms than others, unfortunately the anonymous user fails to say why.

Ruby has no keyword arguments currently (Ruby 2 will have them and keyword collectors ** as Python does). To even approximate my Python example in Ruby, you’d first need to declare the argument as optional which has the side effect of packaging it into an Array. Wheras in Python **kargs packages up the rest of the key/vals under a dictionary. If someone would like to write out the full translation in Ruby, I’d be happy to put it up here, but I doubt its going to be pretty (until Ruby 2).

Zach was helpful and provided an example showing that the specific task I cited is fairly short in Lisp as well, looking like this:

(defun somefunc (&rest kwargs
                 &key (keyone "default") (keytwo "anotherdefault")
                 &allow-other-keys)
                 ; ...
                 )

I would like to make it very clear that my point is not that Python is better, but that technical comparisons can be warped to favor certain languages. This is the same point Paul Prescod makes, and what I’d actually like to see is more technical comparisons that make this point obvious.