 |  |  |
| Main | |
Building the Machine That Will Build the Machine
The first car in history was not built on an assembly line. In fact, the first car of any newly designed model is still not built on an assembly line. Well that goes without saying right? Why then do the leads on large projects (team of 50+) try to build the assembly line before they produce their first car? In fact, why do they try to use an assembly line at all when every car is going to be different?
Well okay, may be it is not exactly an assembly line (although some have been known to use the term), they are just trying to define the architectural framework, i.e. the way that modules will work together and re-use common parts. But, to borrow Tim Bray's words, this attempt to design the framework beforehand fails in its "overreaching completism, trying to be, in Gall's terms, 'a complex system designed from scratch.'" He goes on to characterize one of the tenets of agile development this way:
Until you've implemented and deployed a feature, you don't really understand it. So don't try to build multiple layers of features on things you don't really understand.
I subcontracted on a large government contract that demonstrated how to go about it the wrong way. As I discussed in The Large Software Contract Problem, large IT projects are not like highway construction where a much clearer critical path can be drawn to schedule tasks that depend on each other. That certainly presents a challenge but it slowly crystallized for me that the reason this large government contract was on a death march was the fault of its leaders: they were stubbornly spinning their wheels trying to build the architectural framework that would eventually be used to build the end product.
So much was being implemented under the pretense that it would be part of generating and ensuring the consistency and efficiency of the end solution that they never even approached a satisfactory end solution. After 3 years they had absolutely nothing tangible to show the customer!
Of all the most preposterous absurdities, they chose Oracle Developer 2000 to "help" lay the foundation for this project. Developer 2000 was essentially a framework that never really arrived, even by 2002. But in 1999 they hadn't even started trying to use it before they were figuring out how to work-around certain fundamental characteristics such as some sort of repository that did not support their schema design. Rather than seeing the folly in this, they also had a subcontractor hire a lot of people to pursue the development of a new database framework as well.
When you commit to a framework, whether it is a database development framework or a web community portal or a distributed network component architecture, a significant effort is shifted to installation, configuration and training for that framework. It would be one thing if that framework was the end solution such as a web community that needs no alteration, but then that is not a large project. Actually the problem is often more than a matter of alteration, it involves an attempt to overcome certain shortfalls in the framework, sometimes via a horrid attempt to marry it with yet another framework!
This industry sees wave after wave of application frameworks that come and go. The desire to have a framework is so strong that developers constantly keep turning to the idea of a framework rather than getting on with it! Can you imagine what they might have accomplished in 3 years on the contract mentioned above if instead of chasing frameworks they put all that effort into working closely with selected customers to build a few small tight user interfaces that greatly improved upon existing legacy systems? The problem was that instead of just doing the thing they know how to do, they are hung up on creating this big complex thing that they feel they need.
Early on in the story "Why I Hate Frameworks" it sounds kind of plausible that rather than building the actual tool, they will build the factory that can easily produce the specialized tool.
"According to our research, what people really needed wasn't a Universal Hammer after all. It's always better to have the right kind of hammer for the job. So, we started selling hammer factories, capable of producing whatever kind of hammers you might be interested in using."
But the absurdity becomes evident as the same logic continues to find value in abstracting the task one level further out:
"As it turns out, people don't want to build an entire factory just to manufacture a couple of hammers. Leave the factory-building up to the factory-building experts, that's what I always say!! ... So we stopped selling those schematics and started selling hammer-factory-building factories."
I used to read books about software development. Now I look more to market dynamics and read stories in business magazines like Baseline about how inspired people strike the right balance of passion, action, and design.
In large projects you desperately need a strong technical leader to plot a pragmatic course keeping all eyes on the prize. But too often I find that people who focus on software development principles have an affliction that I call "building the machine that will build the machine."
Ben Bernanke, who succeeded Alan Greenspan as chairman of the Federal Reserve, explains the difficult nature of economics by comparing it to "learning how to repair a car while it is still running". That is one of the necessities of dealing with something as large and complex as the economy or (I would argue) IT solutions in major institutions. Starting over from scratch with a fresh new framework is simply not a viable option.
see discussion at Joel's forum:
http://discuss.joelonsoftware.com/default.asp?joel.3.290075
note by original poster Ben Bryant, 18 Jan 2006 22:32:00 -0500
| Main | |
