Misapplying book terms, Pylons, and the 'end-user' 14
On the Internet, I frequently see terms misapplied to other contexts. Whether its a basic definition misapplied to a different realm, or an analogy grossly misapplied to a context that is not in fact similar. Since this particular term ‘conceptual integrity’ however was used as a beating stick on Pylons, I felt it’s something worth discussing.
First, its very useful to actually understand the context of the book the term comes from, The Mythical Man-Month. The Mythical Man-Month is about Fred Brooks observations as a manager on the OS/360 project at IBM, and the problems encountered when developing an OS for end-users. Before getting into the question of whether an OS end-user is equivalent to a web framework end-user, its useful to look at the term being referred to, ‘conceptual integrity’.
Wikipedia has a fairly good summary of the term:
Conceptual Integrity
To make a user-friendly system, the system must have conceptual integrity, which can only be achieved by separating architecture from implementation. A single chief architect (or a small number of architects), acting on the user’s behalf, decides what goes in the system and what stays out. A “super cool” idea by someone may not be included if it does not fit seamlessly with the overall system design. In fact, to ensure a user-friendly system, a system may deliberately provide fewer features than it is capable of. The point is that if a system is too complicated to use, then many of its features will go unused because no one has the time to learn how to use them.
Is there Conceptual Integrity?
The Pylons developers, including myself, spend quite a bit of time deciding what goes into Pylons, and what does not. Pylons is in fact very very small at its core due to our refusal to let ‘stuff’ into Pylons that is outside its strictly defined scope. In fact, a great many comments I get are usually about features people want, but that aren’t in the scope of Pylons to provide. I would argue that this does show thought has gone into ‘conceptual integrity’, in the explicit decisions made about what is and isn’t provided.
The scope of Pylons is to provide a small, concise, WSGI-driven, lightweight framework, that provides the flexibility for a web developer to use the best tools for the job at hand for building their web application. I would argue based on the definition, the goal of Pylons, and what Pylons has in it now, that it fully meets the design goal of having conceptual integrity with regards to feature scope.
While the original blog post I cited above starts with conceptual integrity, it delves deeply into framework design issues like, ‘Should a framework pick the ORM for you?’. This is slightly odd in the context of Pylons, as the default ORM, and template language are chosen, Pylons merely emphasizes that the choice is yours should you wish to change it (as does Django).
In another misapplied analogy, a parallel is drawn up between having an opinion on a writing style, to having an opinion on a library. Consider how odd this analogy is, that the fairly minor effect of adding a trailing ‘S’, should be compared to whether a particular ORM library is capable of meeting your demands. Should the framework authors decide whether the ORM meets your demands? Should the framework lose a significant amount of capability the second its stock ORM fails to handle your needs? These are interesting questions, well beyond an attempt to draw a parallel to adding a trailing ‘S’ to a word.
Since its clear that Pylons doesn’t encroach on conceptual integrity regarding features that are unused, the next aspect to look at is the concept of a ‘user-friendly system’. This becomes a little more difficult to analyze, because it brings into question the expectations one has of the ‘user-base’. It also shows how the original term of conceptual integrity was misapplied as another parallel is drawn, this time one that I think all web developers should be somewhat terrified by.
Website end-user == Website Programmer?
The last parallel, is to compare the programming expertise of a web developer, to the internet naivety of the website end-user. (Does that offend any other web developers?) The latter of which is well documented in various web usability books, such as the “Don’t Make Me Think” book. The underlying concept being:
- Website users have short attention spans
- Website users won’t use a website that requires thinking
- Website users have no patience
The thesis of this parallel is that a framework author is somehow in the superior position of knowing the appropriate library that will work for your task, and should therefore decide the tools for you. The parallel applied looks like:
- Website programmers have short attention spans (ok, I somewhat buy this one :)
- Website programmers won’t think about whether the tools at hand will accomplish their task
- Website programmers have no patience (Given the languages you have to learn to do anything, I seriously doubt this)
Now, there are quite a few website programmers that don’t think about whether the tools they use, work best for their task. I have seen many websites that are slow, inefficient, or just not well designed because of a lack of thinking regarding architecture, and tool choices. I would not consider these to be website programmers I would want to work with, nor are they the target audience or ‘end-user’ of Pylons. The door is purposely left open in Pylons, and lit up, with the choice available in libraries to use with Pylons.
Regarding making decisions, Pylons has choices for all the same components as Django has decided on: ORM: SQLAlchemy Templating: Mako URL’s: Routes Sessions: Beaker Caching: Beaker
The main problem apparently being, Pylons has emphasized that there is choice, that the framework is built so that they could be swapped, should one of these not be the best tool for your web application.
The Pilot System
When designing a new kind of system, a team will design a throw-away system (whether it intends to or not). This system acts as a pilot plant that reveals techniques that will subsequently cause a complete redesign of the system. This second smarter system should be the one delivered to the customer, since delivery of the pilot system would cause nothing but agony to the customer, and possibly ruin the system’s reputation and maybe even the company’s.
In response to the original post regarding conceptual integrity, I submit the issue of the pilot system. Django and other frameworks that start off with libraries you didn’t analyze in regards to your particular web application, have more than a few times led to pilot systems (I usually hear about the second incarnation, which was then done on Pylons).
Where you realize later down the road, “wow, this ORM just can’t handle the data model I have”, which then leads to, “well, since the ORM was hooked into the XML serializer, the admin UI, the form generator, etc. there’s now no competitive advantage over Pylons… which didn’t have those extras.” Systems with heavy and deep integration of libraries that may or may not meet your demands, where large parts of the system become relatively useless should you stop using one part of it, are extremely prone to being a pilot system.
When your pilot system is done, I invite you to check out Pylons, and think about what is going to work for your website application.
Notes
Please note that I take deliberate care in referring to Pylons for use with developing web applications, its my belief that Django, and similar systems in other languages with an ‘app’ centric approach like Joomla, Drupal, etc. are excellent for making web sites. While you can easily make websites with Pylons apps, this is definitely not the primary use-case emphasized and designed for, so it is harder and less obvious in many respects than Django.
The other thing everyone should’ve learned from these posts, is that The Mythical Man-Month is a rather good book worth reading. ;)
Regardless of what the actual terminology or their interpretation Adam brings up numerous good points,
Pylons is almost unusable for someone who just wants to get stuff done right away, without having to read up an all kinds of swappable components, the docs suck and are misleading
You cannot use pylons unless you are well versed in a lot of concepts that you should not need to know about (ie what WSGI is, or how you hook things into it)
The analogy between web applications and web sites is good one. It’s also one of experience, if you’ve build a number of python web apps, you probably know what WSGI is, how you hook things into it, and most importantly when and why you’d want to use it. If you don’t know the when and the when then it just seems like a lot of “extra noise”. (for example it usually takes quite a bit of experience to recognize when and why you need something such as a transaction manager )
It’s no coincidence that the Django and Rails authors both came from a PHP background, the language that so strongly sings to the ideas of “just get stuff done right away” and the “don’t make me think”. Django and Rails in many respects remind me of Zope during the dot com era. All frameworks catered more to the beginner/intermediate developer, all frameworks had a lot of users who felt it was “the one true framework”, and all frameworks users tend to produce monolithic systems. Of course Zope had a wildly different architecture, and this may sound like I’m knocking Django, but as someone who cut my teeth on early Zope, it reminds me Unreal Tournaments reminder, “we were all n00bs once” (and all n00bs produce monolithic systems). I have great respect for frameworks that cater to the developer new to Python – Django certainly does a good job at introducing people to Python and teaches a lot of good practices early on to the aspiring developer without overwhelming them. It’s just that sometimes a brashness of tone that hurts it as it tends to induce users not to look beyond the “Django wall” at things produced by other Python developers who they see as “just making things hard” that is a harmful attitude to carry.
@somebody,
“everybody” (whoever that might be) is not looking for instant gratification.
A lot of times, considering the code as investment which will reap benefits in long run, will make the case stronger for frameworks like Zope3 and Pylons.
Kevin, two things:
1. Would you be so kind as to point out to me a specific example, either in the Django documentation or in a public statement by the Django team, in which this “brash tone” which discourages looking beyond the “Django wall” is evident?
2. It may sound like I’m knocking you, but you remind me of a Penny Arcade comic I once saw about Unreal Tournament. It had a picture of a chalkboard in it, if you’d like to go look it up, and once you do I have some suggestions about what you can do with your “we were all n00bs once” condescension.
Constructive criticism is great and I encourage it wholeheartedly. But people like you, Kevin, I can do without.
On the Internet, I frequently see people “refuting” articles they don’t understand. This is a quite remarkable example of not getting a single thing; reminds me of certain ID proponents.
@James
Sorry if I sounded condescending, I chose some of my words poorly and hastily.
I was responding primarily in response to comments that have been made by Django users – or before I cheese off all Django users as well – I was responding in response to any user who’s new to web development, builds a rather poor system and then says, “framework X sucks!” My characterization of Django users as such was meant to be taken as a complement to Django in that anyone who is very new to web development tends to make brash comments, and Django probably has more such people than many frameworks simply from the fact that the Django authors are doing a good job at creating an easy to learn framework, that draws people newer to web development to the framework. However, my words and tone didn’t come across that way – especially in that they conveyed this too much as a blanket statement over the Django authors and Django core community, who have generally been very good about promoting good attitudes.
As for a specific example on the Django site that is a bit brash is “this section”: http://www.djangoproject.com/documentation/faq/#why-did-you-write-all-of-django-from-scratch-instead-of-using-other-python-libraries . I understand that Django filled a much needed gap in the Python web framework space that hadn’t been met at the time, but phrases such as “We’re picky. You might even call us perfectionists.” and “existing frameworks/tools inevitably had some sort of fundamental, fatal flaw that made us squeamish. No tool fit our philosophies 100%.” isn’t the best way of saying, “the existing frameworks had different goals and priorities than us”.
There is a tension between creating frameworks that are presented as a cohesive whole by not reusing parts from different projects – and how that aspect of Django is one of the attributes that makes it an easier to learn framework – and how new developers can look at this design philosophy in the Django framework and miscarrying this philosophy by applying the idea of “I’m not going to use any external libraries in my project as it will loose cohesiveness”.
I still don’t buy it. You might as well call Ben “brash” for having written Pylons despite the existence of other Python frameworks.
And if somebody looks at Django and concludes they should never use any external libraries, point me at that person so I can mock him/her mercilessly. I don’t get much opportunity to do that these days, but such people would be deserving of it (especially since Django’s docs contain pointers to various non-Django modules for solving certain common problems).
“Django and Rails in many respects remind me of Zope during the dot com era.”
Except, you know, developers actually enjoy working with them.
It’s all about the learning curve, and if it’s too time-consuming, stressful and frustrating to get rolling with a Pylons site, the ultimate payoff just isn’t relevant.
If you “seriously doubt” that “Website programmers have no patience”, you have a distorted view of web developers and you won’t be able to connect with the vast majority of them.
Replying to “somebody,” I’m not sure what’s sucky or misleading about, say, the QuickWiki tutorial, among the other Pylons docs. I found Pylons quite easy to get moving with.
@observer: It’s easy to make baseless claims such as yours, I’ve gone to some effort to provide a premises and reasoning to back my assertion here. Try doing the same for yours, and shed your anonymity if your opinion should have some weight to it.
With general regards to ease of learning, I’ve had people say that the Pylons docs made it simple and obvious, and people say they were confused and had no idea what was going on. This usually reflects a different level of expertise on behalf of the programmer, and the docs definitely don’t do as good of a job helping along newer users as Django’s docs.
@Joe: I should rephrase that…. some website programmers have no patience. Django gets a lot of flack for not providing AJAX helpers because website programmers have little patience to go and learn Javascript in addition to everything else they’ve learned.
The question is, is Pylons aimed at those website programmers with no patience? (Even if they’re the majority of website programmers) I’d say no, Pylons is not for those in such a rush to throw any old thing up with little thought to whether the tools will work for the job. However, I don’t think one needs nearly as much patience to learn Pylons, as to learn a system like Zope which goes beyond basic Python to require a fairly complete understanding of concepts like interfaces.
This discussion is so decontextualized that I find it inpenetrable (and lacking conceptual integrity in itself).
I think this kind of debate about conceptual integrity, that does not involve parading out just what the conceptual integrity is, and for whom, is pointless and counter-productive.
Starting out, there is a terrible misdirection about Brooks use of conceptual integrity. Brooks may have unwittingly contributed to this misunderstanding.
OS/360 was in no sense an end-user system. The conceptual integrity was around architectural and feature structures that were seen by programmers and IT organizations. About the only thing that end users encountered were printed output (from line printers) and the closest thing to end-user programming was RPG (Report Program Generator), a scheme that preceded System 360 and was very much a legacy (and designed around a punched-card records and groups conceptual model).
When the primary data-entry device was a keypunch for making punched cards, you can see that today’s notion of user has no place in the OS/360 pantheon. Except for time-sharing systems (a limited case and not the OS/360 design point), use of keyboard terminals (like teletypes but nicer, often based on the IBM Selectric typewriter) and heads-down use of alphanumeric display devices was the next round of “user access” to these OS/360-based mainframe systems.
I recommend all of Chapter 4 of The Mythical Man Month. It should be clear that users in this case are programmers and operators. I quote, “By the architecture of a system, I mean the completed and detailed specification of the user interface. For a computer this is the programming manual. For a compiler it is the language manual. For a control program [such as OS/360 job-control] it is the the manuals for the language or languages used to invoke its functions. For the entire system it is the union of the manuals the user must consult to do his entire job.” (I realize that I disagree with Books about what architecture is versus where it might be revealed, but that doesn’t matter for discussion of conceptual integrity.) Brooks then affirms that “Architecture must be carefully distinguished from implementation,” putting him on safer ground, I think. Clearly, he means “user” in a way different than what we mean more than 30 years later.
In “The Mythical Man Month after 20 Years” Brooks pays homage to an example of conceptual integrity via a metaphor: “The WIMP [Windows, Icons, Menus, Pointing interface] is a superb example of a user interface that has conceptual integrity, achieved by adoption of a familiar mental model, the desktop metaphor, and its careful consistent extension to exploit a computer graphics implementation.” He is referring to the original Macintosh realization of WIMP, in this case.
An important question to address with regard to conceptual integrity is agreement on the level of abstraction at which the conceptual formulation/architecture applies. Be careful.
@orcmid: Thanks for the clarification, I indeed misread that as well. My primary intent in this response was to attempt to provide a counter to what I saw as people blindly grabbing a phrase that someone else had used in an attempt to put some label on why they didn’t like Pylons.
If the final conclusion of this is that the term is ambiguous, or difficult enough to apply that people are not running around saying, “oh it lacks conceptual integrity”, thats fine for me.
As you mentioned, its a very different scene 20 years later from OS/360. The main reason I went for this definition of ‘end-user’ in this context was in part due to the later comparison in Adam G’s blog with regards to the “Don’t Make Me Think” book where the term ‘user’ is very clearly applied in a modern context.
The original blog post made few if any declarative examples of how Pylons fails to have “conceptual integrity” besides for bashing the font used for the docs, and un-related projects like ToscaWidgets. I was hoping to at least provide an example of some conceptual integrity by illustrating the scope of Pylons, and how the tools are used in a minimal and consistent style, but its quite likely that didn’t come out as clearly as I desired.
Thanks for the clarification and comment!
All you have to do to find the “brash” is compare the usual tone of the Django developers on blog comment threads with that of the Pylons or TurboGears developers. Let’s call a spade a spade, the Django developers have done the “Django Rocks, Everyone Else Sucks” thing to death, and many people are sick of it. It is symptomatic of the underlying team philosophy and drives a lot more users away than you know.