By Jack Ganssle

Schedule Masturbation

Published 4/21/2003

Columnist Robert Martin replied by (properly) belittling the waterfall method, where developers create a complete design and then a schedule. He noted that it generally does not work in the real world. Unstable requirements and unknown issues conspire to destroy any schedule derived from a process of up-front design.

This is a basic tenant of most of the agile methods, one that's demonstrably true for most big projects. We're just not smart enough to completely specify a project till we jump in and start building it.

Martin suggests creating the schedule using XP. Engage in a number of two-week cycles to prototype feature subsets. Measure productivity, and repeat. After a number of these cycles it then becomes possible to create a valid schedule and cost estimate. This is a fantastic way to create schedules. It reduces risk, and gets a real handle on the software issues.

But it's fantasy. The boss wants a schedule by Friday, not in 4 months. Even the most complex government projects have a 30-60 day proposal window, hardly enough time to crank through enough 2-week cycles to get any meaningful information. In the commercial world most of us get a day, or at most a week, to come up with a schedule estimate.

The XP folks are right when they claim big up-front design is too inflexible for the real world of constantly changing requirements. They're spot-on when pointing out the impossibility of creating a schedule without doing a detailed design. But the XP solution - start building the system and sooner or later the ship date will become apparent - is utterly impractical.

The boss wants a delivery date now. Though scheduling a big project without understanding its scope and complexity might be absurd, real business considerations drive the boss's need for a date. Will the engineering be so expensive the project will be a fiscal bust? Will we meet the market window? What promises can we make to investors? Developers shudder when hearing these sorts of questions, but they are central to business planning.

Once I hoped that we'd breed a new generation of bosses who understood the difficulties of software development, who would buy into an evolutionary approach where the ship date is nebulous at first, but becomes clear by the time we're halfway or three-quarters of the way complete. But it ain't gonna happen. Our boss responds to his boss, who needs a real date to sequence other activities - marketing, promises to shareholders, competitive positioning, acquisition of funding.

Scheduling is an inherently flawed process because there's no correlation between corporate needs and development time. Until someone figures out how to tightly couple them, schedules will be the product of guesses and intuition, projects will run late, and our heroics will be the only "solution" to the software crisis.