Who could have suspected that the best practices for making good software also have something to say about the therapeutic process?
Over the Waterfall
As of 2011, we’ve developed some pretty big expectations for software. We expect our computers to start fast and work flawlessly for days at a time. When a user finds a major bug, it’s reason for upset if not outright rage. So an outsider might be led to believe that creating software is a well-understood, well-managed endeavor. Having worked for several years in software development, I can tell you with confidence that nothing could be further from the truth.
Many software project managers come from engineering backgrounds and even more are steeped in the engineering mindset. When engineers set out to build a bridge or a dam, they do so under the assumption that they have all the information needed before the first load of cement is poured. They can make a full and complete set of blueprints, accurate down to the last rivet. From this, it is possible to estimate time, materials, effort, and overall cost with confidence.
With this level of certainty, projects start to look like a straightforward, step-by-step process. The names and numbers of the steps may vary, but the process is lockstep: start step 1, finish step 1, start step 2, finish step 2, and so on. One common description of the steps in the ‘waterfall model’ includes:
- Gather requirements
- Design system
- Construct system
- Test system
- Maintain system
In this model of the world it is never appropriate to go ‘backwards’ to an earlier step, nor is it proper to work on two steps at the same time. Many people have likened this model to a series of waterfalls where each pool between them represents another stage in the process. Swimming ‘upstream’ is against the rules. No ‘salmon’ are permitted on these waterfalls.
The strength of the waterfall mindset is certainty. When you do waterfall project management, you can expect to say with precision which step you’re on, how long it should take, how many steps are left to go, and how long they should take. If you’re reporting to your higher-ups who are concerned with time, materials, and cost, a waterfall presentation makes you look like you’re on top of everything, and it gives them the impression of a very definite and precise accounting of what’s been done and what’s to come.
However, when the engineering-centric waterfall model was applied to software, it failed miserably. The early stages, concerned with gathering requirements (what should the software do?) and design (how will the software fulfill the requirements?) became longer and longer. Because these phases ran long, the middle or construction phases (actually writing the code) became compressed since an overall timetable and schedule had been agreed to by all at the outset. Meanwhile, developers were discovering things during “construction” phases that weren’t known during earlier phases, but there was little they could do with this knowledge since the one true path to success had already been dictated. In the testing and verification phases, developers often discovered that they hadn’t actually created what the requirements called for, or that they had, but that was “not what the end-user meant” when they wrote the requirement. Often, this forced the “maintenance” phase to contain a greater or lesser redesign and reimplementation of what went wrong during the waterfall process. Statistics vary greatly, but they all agree that a large percentage of software projects fail outright and are abandoned while many more drag on long beyond the deadline and over budget. In my personal experience, the only time I’ve ever seen a software project delivered on time, on budget, and working properly was one I did by myself.
The dirty secret of software is that trying to manage a software project like a construction project is doomed to failure. Because the waterfall model creates the illusion of certainty, it is still widely practiced, even by those who know its weaknesses. However an alternate approach called “agile development” has been gaining mind share in the software development world over the last few years.
I wish to apologize in advance to those in the agile software community. My description of the agile method will necessarily be incomplete and imprecise. For our purposes, agile can be seen as undermining the fundamental assumptions of the waterfall model. Agile assumes we don’t know exactly what sort of software we’re going to build, and indeed we can’t know. We can’t know because the act of construction will yield new knowledge, skills, and insights that fundamentally change the terms of the discussion. Agile replaces the steps of the waterfall with a continuous, rapid cycling through the tasks within the waterfall model many times in a project. Where the waterfall model starts at zero and ends with a ‘completed’ software system, agile also starts with nothing but rolls out a working, incomplete prototype early on in the project. This prototype is presented to end users and mined for insights on how it can be improved incrementally throughout the project.
To live with the agile method is to let go of the illusion of certainty and precision, exchanging them for early progress and continuous reflection and refinement. I am convinced that the waterfall model endures mostly due to the psychological needs it pretends to meet rather than any objective advantage over agile.
Therapy, too, can look a lot like the waterfall model. When a new client comes to therapy, they proceed through “intake”, where they fill out forms and talk with a therapist about their history, problem, and what they hope to gain from therapy. For all practical purposes, this is just like the “requirements gathering” phase of the waterfall model. Then the therapist, in collaboration with the client, develops a “treatment plan” that has a lot in common with the design phase of the waterfall model. In the working phase of therapy, the client and therapist work together to achieve those treatment goals by following the treatment plan. This is closely akin to the “construction” phase of the waterfall. Presuming (and that’s a big presumption) all goes to plan, the termination phase of therapy allows client and therapist to review progress and plan for maintenance of the change.
I can’t say that the therapeutic process described above never happens, but it’s the exception rather than the rule. Far more often the client is conflicted and unclear about what they would consider success. Treatment plans are tried, adapted, abandoned, replaced, and sometimes re-tried. A client’s level of motivation and goal-orientation naturally wax and wane over the space of weeks. Life circumstances change, often abruptly, dramatically, and without warning. Sometimes what seemed to be the clear goal early in therapy ends up being a dead end. That’s why I try to style my therapy on the agile model. Instead of following a structured plan, I’m considering all the feedback I can to guide my client towards the next right action. Next week, the “next right action” might be something completely different.
I believe the agile mindset has something important to say to everyone, not just programmers and therapists. If we’re being honest with ourselves, we have to admit that we don’t know ourselves, at least not fully, let alone our future selves or how our world will look in the months and years to come. So are we hurting or helping ourselves when we design grand plans and commit ourselves to having it all come out just so? Having a vision is great, as long as we remember it’s just a vision.
I used to think that adults asking children “what do you want to be when you grow up?” was an innocent question, but now I’m not sure. If we inculcate the mindset that our lives are predictable and plan-able far in excess of reality, then we’re setting ourselves up for the same upset and frustration that every manager experiences when their carefully-planned project goes badly over budget and behind schedule.
All clinical material on this site is peer reviewed by one or more clinical psychologists or other qualified mental health professionals. This specific article was originally published by Dr Greg Mulhauser, Managing Editor on .on and was last reviewed or updated by