Agile & Waterfall are two sides of the exact same Coin, and are equally good or bad.
Some Agile enthusiasts enjoy lambasting waterfall while espousing “Agile”, however the two techniques have many similarities — they both represent extreme ends of a continuum.
“Waterfall” is basically a Strategic method of attaining goals, and “Agile” is basically a Tactical method for achieving goals.
Both are equally limited; strategy without tactics is ineffective, and tactics without strategy are especially ineffective, especially over a longer term.
Some happy medium (Iterative Traditional) would be best; either extreme is equally good or equally bad depending on the point of view.
However trading in one extreme (“Waterfall/Strategy”) for another extreme (“Agile Programming/Tactics”) is certainly no great improvement, in fact it is merely a change — in this case, swapping one extreme for another.
The solution is not to be found at the extremes, but in the middle.
This has been known for centuries, but has been lost in what Sun Tzu rightfully decries as “Noise.”
Strategy without tactics is
the slowest route to victory.
Tactics without strategy is
the noise before defeat.
— Sun Tzu
Traditionalists might stand to improve from a little more iterative and incremental cycle, and Agilists could probably stand to improve with a little more documentation, process and architecture.
Any successful campaign must be waged using both Strategy and Tactics, not merely one or the other.
Examples of wars/battles failed by using exclusively one or the other are: The Maginot line was clearly a failure, with far too much emphasis on strategy and little on tactics. The current gulf war is a failure because it was purely tactical in nature (removing “Saddam”) with little though if any given to the strategy of a post Saddam Iraq.
Hopefully this shows that neither extreme is particularly ideal, and modern planners at all levels should be mindful of both Strategy and Tactics when waging campaigns.
Post Agilist 
Hi PostAgilist,
Thought provoking post.
Why do you assert that Agile optimizes for short term, tactical goals? Also, are you saying that all waterfall projects optimize for the long term?
I would agree with you that the basic building blocks of both Agile and Waterfall are the same. In the end, you have to have things to do, decide which to do now, decide how to do them, do them, and make sure that the software works.
Here’s a related post that you may find interesting. It may also help to show that not all Agile advocates think the same way about Agile: http://damonpoole.blogspot.com/2007/05/agile-development-waterfall.html
I don’t see the connection between the timeframe of the goals and the development methodology. I think those are more connected to the strategy and tactics of the company. If your company thinks tactically, then it may very well decide to pack 100 tactical features into a waterfall release. And if your company things strategically, it may very well develop something strategic using an Agile methodology and take a year before they release externally.
Cheers!
Damon
P.S. – you may be interested to know that some of your posts have been appearing on reddit.com/r/agile . There are additional comments there as well.
@Damon
Well, as I mentioned, I see Agile and Waterfall as more or less points on a continuum.
And those points that are staked out are basically tactical/short term versus strategic/long term.
It is not the time frame of the goals that leads to the difference; it is the time frame of developers in terms of what they are thinking of.
It is similar to driving a car; you can just look out at what is directly in front of the car, or you can look farther down the road.
People who look too far down the road may not see the child jumping out at them, which is what agilists complain about, that long term thinking does not allow for planning of last second (Tactical) events (the kid jumping out).
Or, one could drive like an agilist and constantly be looking 3 feet in front of the car and never keep track of your actual destination and the constant turns may not be necessary if the route had been planned better.
I think that even the founders of Agile would agree that it is tactical in nature.
My point, is that, in agreement with Sun-Tzu, I believe that a combination approach is best; one extreme or the other is lacking when compared to a holistic, middle approach.
PostAgilist
Perhaps this will make things more clear: here is a definition from Encarta
Strategy
1. military planning of war: the science or art of planning and conducting a war or a military campaign
2. planning in any field: a carefully devised plan of action to achieve a goal, or the art of developing or carrying out such a plan
Tactics
1. military direction of forces in battle: the science of organizing and maneuvering forces in battle to achieve a limited or immediate aim
2. finding means to end: the art of finding and implementing means to achieve particular immediate or short-term aims
Agile’s emphasis on YAGNI, emergent design, Scrums, etc, all point to it’s tactical leanings. In fact, all the losers of WW2 essentially employed tactics that they hoped would be sufficient to overcome their lack of resources to carry out a sustained, strategic campaign. E.G. Pearl Harbor was a tactical gambit to make up for the fact that Japan could not endure a long war. It was an agile move, but with no strategic plan to save them when the gambit failed, the final result was known to both parties for quite some time.
One could argue (wrongly) that Pearl Harbor was a strategy and not a tactic because the “strategy” was to disable the fleet. However, since they had only one tactic to achieve said “strategy”, it was merely a tactic and not a strategy. If they had organized 5 or 6 tactics that could have achieved their strategic goals then they would have in fact had a strategy.
More contemporaneously, in “Star Wars”, Tactics are referred to as “Aggressive Negotiations”. It’s what they do when they don’t have a plan (and rightly so).
Meanwhile Darth Sidious is the BUFD Strategic King that really hands it to the more tactical and relatively naive Jedi who never saw through all the layers of his slowly unfolding strategic masterpiece.
Once again Strategy vs Tactics is a continuum, but generally, more Up Front Planning=Strategy and Less Up Front Planning= Tactics.
Also, in War and Business, wealthier organizations can tend to afford more strategy than their less funded brethren. Perhaps this is a reason for agile popularity. But even small militaries can benefit from incorporating strategy into their Operations Plan.
PostAgilist
Hi PostAgilist,
I’d be interested in examples of Agile folk claiming that Agile is tactical in nature.
Also, my question was not “what is the difference between tactics and strategy.” I understand the difference and agree with your examples. But you based your post on the assertion that ‘“Waterfall” is basically optimizing for the long term, Strategic goals, and “Agile” is basically optimizing for the near term, Tactical goals.’
My question is, what is this assertion based on?
It would seem that one conclusion from your assertion is that Waterfall projects by definition are strategic. But I’ve seen plenty of waterfall projects that were mostly tactical in nature. I’ve also seen plenty of Agile projects that were strategic in nature.
I think there is a much stronger connection between the business goals and the outcome than between the methodology and the outcome. Can you explain why short iterations makes strategic thinking impossible or why waterfall encourages strategic thinking?
I see feature creep as a good example of how tactical thinking is fairly common in waterfall projects. While of course it is true that “that shouldn’t happen” the facts on the ground are that it does.
Enjoying the dialog,
Damon
@Damon
Thanks for the comments….
I never said they “had claimed” that it was tactical in Nature, I said they probably would agree that it is.
As far as why I view Agile as being tactical oriented, as I said, all of the YAGNI, Short Iterations, Daily Integrations, “lowering the cost of change”, lack of significant documentation, is all well suited for changing tactical situations on the ground.
If the longest horizon an Agile team looks at is 30 days or less, how can that be considered Strategic? Therefore it must be tactical, right?
YAGNI is the best example of the tactical leanings; under YAGNI, the US would never have spent 4+ years developing Nuclear Weapons because “we wouldn’t need them.” Obviously, WW2 planners were not relying solely on tactical success or the YAGNI principal to solve all of the problems.
It is just self evident, as in the duck test. If something looks like an SUV, it probably is an SUV. If something looks like a tactical fighter plane, it probably is.
As far as your other questions, I never said that Waterfall was designed to address Strategic goals; I said it was a Strategic oriented method.
One can address a Strategic Goal using a Tactical Method (as in Pearl Harbor), or a Strategic Goal using a Strategic Method (Nuclear Weapons during WW2).
The reverse is also true; I think in the above post you were confusing goals with methodology…
Using the Star Wars example, Star Wars 4-6 are about using Tactics to solve Strategic goals — which is to use Sidious’ creation of Luke & Leia in a judo move against Sidious (“You’re our only hope.”). But they had no strategy if the tactics failed (aside from fielding Leia….)
Star Wars 1-3 are about using Strategy to solve Strategic Goals.
Goals != Method… 🙂
Additionally, re tactical bias, one can easily see the various tactics used to promulgate agile that are used by the agile proselytizers. This once again reinforces the tactical mindset of the agile crowd. The “denigrate the enemy” tactical pattern has been often used by agile supporters — “You’re backward! You’re waterfall!”
If the goal is to win the election, a great tactic is to (attempt to) kneecap the opponent.
It’s tactics through and through..
Hope this helps clarify,
PostAgilist
Hi PostAgilist,
I guess we’ll have to agree to disagree. If you are interested, I’ve combined my comments above, added some additional thoughts and distilled it all into a new blog post here: http://damonpoole.blogspot.com/2008/06/strategy-vs-tactics-in-agile-projects.html
Damon
“one could drive like an agilist and constantly be looking 3 feet in front of the car and never keep track of your actual destination and the constant turns may not be necessary if the route had been planned better.”
But that’s absolutely *not* what we agilists are doing! In fact I would argue that we know our current location much better than waterfallists, and of course we also know our actual destination.
What we do, though, is to steer our vehicle all the time, thereby being very responsive to changes in the route to the destination, and even to changes in the destination itself.
“If the longest horizon an Agile team looks at is 30 days or less, how can that be considered Strategic?”
Yes, if that were true, your reasoning would make much more sense. It isn’t.
@Ilja
Argue based on what? Do you have any supporting arguments/examples/practices to go with your generalizations ? 🙂
PostAgilist
I was gonna weigh in with a longer post until I saw a tangent into Star Wars tactics. Not the best way to solidify the argument (even if I am a fan) with mainstream readers.
Agile vs. Waterfall is a useless debate. Agile is a philosophy that should be laid on top of a waterfall, RUP, or any process. That’s the best solution. Any attempt to productize agile, box it, and sell it… leads to a risk that it is perceived as a methodology and a single solution in agile is better than a single solution in Waterfall.
Cockburn’s Crystal approaches could really support this argument well and would make some great points in this debate… but Cockburn himself said at Agile 2007 that businesses seek structured recipe’s like XP and Scrum to implement. It’s hard to pitch an evolving/adpating unstructured methodology (like Crystal). Thus, organizations create a 200-page methodology that attempts to solve all problems and don’t recognize that nothing can solve all problems and that it’s better to recognize change and constantly think and adapt.
In this debate, I believe you are discussing the productized versions of Agile and Waterfall both of which will eventually fail. I advocate that we look at the problems at hand that block productivity and positive culture for a software team, apply the appropriate philosophy at that time and then apply a matching methodology.
Good thread… it made me think.
@Kevin
Well, my post was not to be an “Agile vs Waterfall” debate. It was to show that there are limitations in both methods, and some combo would be best, something you seem to agree with.
The only reason some people saw it as a “debate” is that it posits that Agile is not perfect on it’s own, leading to some push back from the Agile crowd, but it was not a versus post.
Re Star Wars, the original version of my post didn’t have any such references, obviously, but one concrete example is worth a thousand unapplied abstractions.
One of my difficulties with this post is that most people don’t know the difference between strategy and tactics, and many people don’t care about real military history or have much background in it.
Therefore I added the Star Wars examples in order to broaden the readership that it would make sense to….
Thanks for stopping by 🙂
PostAgilist
Sure I have examples – thanks for asking! 🙂
Agile teams knowing their actual destination: Scrum, for example, has a product backlog that contains “all functionality desired in the product” (http://www.mountaingoatsoftware.com/product_backlog). Extreme Programming has a Release Planning practice. A typical time span to look at in Release Planning is around 6 months.
Agile teams knowing their current location better than Waterfall teams: Agile teams measure their progress in terms of Running Tested Features. They get regular feedback on whether they are building the right system, ideally based on users actually using implemented features. They know their velocity and are very good at predicting how long implementing remaining features will take, based on historical data about full implementation of previous features. It’s much harder to predict how long testing of all features will take based on how long implementation took. In fact, you don’t know whether you are actually *done* implementing features until you are done testing.
Does that answer your question?
As an aside, I have no problem at all with Agile not being perfect. I do feel an obligation to chime in, though, when I see statements about Agile being made based on what I perceive as wrong assumptions.
@Ilja
As far as “Scrum has a product backlog…” etc… By “destination” I’m not really referring to overall long range destinations like the list of features etc, I’m talking about shorter range destinations like how to implement features, implement interfaces, etc.
Agile tends to do this process in an “ad hoc” fashion — so called emergent design, compared to other methodologies. I’m not saying here that they don’t have a list of features, I’m saying they don’t (generally) have a plan on how to implement those features, they “wing it” using TDD and emergent design, YAGNI and other approaches that lead to much (needless)
refactoringrewriting.In other words, Agilists may know the destinations on the trip ( the “backlog” ) but they don’t do much route planning as far as how to hit those destinations, leading to more U-turns, backtracking, and, according to Jim Coplien among others, an Architectural Meltdown around Iteration Three.
As far as knowing their location — the same can be said about “Waterfall” teams — once they start implementing they can also look at how long it took on certain tasks to refine their estimates…. what is unique about agile in that regard?
As far as “wrong assumptions” — what are my wrong assumptions — I’d love to know? Are you saying Agile doesn’t have a tactical bent?
Interestingly, in this post I have bashed “Waterfall” as hard as “Agile” and yet only the Agile folks seem upset. Why is this?
My thesis is that an approach that encompasses Strategy and Tactics, Agile and Waterfall is the best overall approach. Eg, Have Documents, but be willing to refine them. Have an Architecture, but be willing to refine it.
Additionally the notion that Documents can not be refined, and that a Traditional process cannot respond to changes is ludicrous. The whole notion of “Waterfall” as a fixed an inflexible process is merely a parody, a straw man built up by Agilists in a desperate attempt to make their views seem unique and pertinent when in fact they are not new.
It’s Tactics versus Strategy — except it’s called Agile versus Waterfall.
I think one problem agilists have with understanding this is they think that “winging it” is a Strategy. It isn’t. It’s just a glorification of the tactical.
If one sees that Agilists favor “winging it” over “planning it” — and this is obvious from the literature — then it naturally follows that they favor Tactics over Strategy.
PostAgilist
Pingback: What Sun Tzu and Eisenhower share (but we’re deaf to) « Clarification
@PostAgilist, have you actually worked on a significant number of Agile projects? It doesn’t look to me as if you have.
There is, as with all methods (see below) a disconnect between how Agile is described for pedagogical purposes in the books and what actually happens on the ground.
I very much doubt that Cedric has seen the techniques he criticises used in practice, and Cope is well known for emitting strong opinions based on misinterpretations of something he read once. Architectural melt-down in iteration 3? How on earth would he know?
Meanwhile you wonder why the Waterfallists aren’t rushing to defend that method against your criticisms. Maybe there just aren’t any. Maybe that argument was actually lost a long time ago. A very long time ago have you read Royce’s paper that introduces the Waterfall (“Managing the Development of Large Software Systems”, IEEE WESTCON 1970) ?
On pages 1 and 2 he introduces what we would now call the Waterfall process. And then he says
“I believe in this concept, but the implementation described above is risky and invites failure. The problem is [that the] testing phase which occurs at the end of the development cycle is the first event for which [system behaviour is] experienced as distinguished from analyzed. These phenomena are not precisely analyzable. […] Yet if these phenomena fail to satisfy the various external constraints, then invariably a major redesign is required. […] The required design changes are likely to be so disruptive that the software requirements upon which the design is based and which provides the rationale for everything are violated. Either the requirements must be modified, or a substantial change in the design is required. In effect the development process has returned to the origin and one can expect up to a l00-percent overrun in schedule and/or costs.”
Whoops! The “Waterfall” process (analyze->design->build->test) was a straw-man from day one. Unfortunately, authors of software engineering textbooks seem never to have got past page 2. In the rest of the paper Royce tells us what we should do instead. We should start testing before we finish coding, which we should start before we finish design, which we should start before we finish analysis. And all these activities should be mingled together. He recommends that we build the software more than once in order to learn form the experience and with the intention of throwing the earlier stuff away. And so on. The seeds are there.
One specific comment he makes is that “During the early phase of software development the documentation is the specification and is the design. Until coding begins these three nouns (documentation, specification, design) denote a single thing.” Despite what your prejudices tell you, this is exactly what Agile teams aim do with TDD: they write specification and design documents before coding—documents that have the signal advantage of being automatically checkable against the implementation. One of the things we aim for in Agile development is to have the acceptance/functional/user level tests written far in advance of development of those features. If you think that this can be done without “strategic” thinking about the solution, I’d like to know how.
@Keith
You have one good paragraph here so let’s start there:
“The “Waterfall” process (analyze->design->build->test) was a straw-man from day one…. We should start testing before we finish coding, which we should start before we finish design, which we should start before we finish analysis. And all these activities should be mingled together. He recommends that we build the software more than once in order to learn form the experience and with the intention of throwing the earlier stuff away. And so on. ”
Exactly – the version of “Waterfall” that you refer to is a very dead straw horse and yet agilists crowd continue to beat that dead straw horse as if that is the horse that “non Agilists” actually ride when they are doing a project?
Waterfall is of course a straw man — and noone uses the straw version of it. Everyone “adapts” the waterfall process to be more incremental and iterative, whether it’s through pilot projects, testing and coding through the design phase, incremental releases, partitioning, etc.
It gets back to the central part of my thesis which is that Traditional Methods (so called “Waterfall” ) and “New Methods” (so called Agile) are really points on a continuum and not discrete.
Additionally your usage of documentation clearly separates you from the XP camp and shows that there is a wide continuum in the agile space…some of which might even intersect the more traditional and iterative models.
Some of the agile rhetoric along these lines is interesting and goes like this: Are you agile? No? Then you must be Waterfall? And thus that means you test last! Q.E.D.
Or…how about… All projects that aren’t Agile are Waterfall! And all Waterfall projects fail! Therefore there was no successful software developed prior to ~2000! (or 1995, pick a date).
The facts on the ground rather overtly contradict the popularly promulgated fallacies in this area.
PostAgilist
@PostAgilist
I’m glad to see these points being made. It has always bugged me that both Waterfall and Agile, as commonly defined by the opposite party, don’t typically happen. In practice, Waterfall iterates and Agile plans ahead. I’m sure both sides have their fanatics, but they hardly represent the norm.
I recently had breakfast with one of the engineering chairs at UC Berkeley, and he was very clear about the fact his super-huge government projects are not completely pre-planned and micromanaged. They iterate and evolve. This was confirmed by personnel in the administration office who oversee the contracts and spending. These are the projects that are often cited as examples of big bad Waterfall, yet they don’t work that way.
Likewise, the Agile folks I’ve spoken to freely admit to some up-front planning. I believe even Kent Beck himself once said that the first few iterations of a typical XP project tend to be research-oriented.
In my opinion, it all comes down to the kind of risks you’re facing. Bridge builders are not likely to encounter a sudden mid-construction shift from a suspension design to a cable-stayed design, but the impact to the project would be massive if it did occur. Low likelihood, high consequence, plan ahead. Your typical software project, on the other hand, is likely to need a sudden integration with an outside service or a switch to a new database vendor, the impact of which should be relatively isolated in a well-designed application. High likelihood, low consequence, less planning required.
It’s important that the above not be taken as gospel. A change in bridge design is only high-consequence because we humans presently lack the ability to just push a few buttons and *BAM* there’s a bridge. That may someday become possible, and if it does you can be sure civil engineers will be all over it. Similarly, not all software projects are necessarily low-consequence — just ask the ESA about the Ariane 5.
Understanding the risk is key. If you are able to plan less and change later with little consequence, go for it. If not, don’t.
Pingback: Middle Of The Road Software Architecture Design « John Dulleck's Industrial Commentary
I disagree a bit with your explanation.
I would say this:
Goal: To deliver a product, that will function in “this” platform, in X amount of Days.
Strategy:
The way that i will deal with Requirements is that I will use Waterfall instead of Agile, which means that i gather requirements as much as i can at the start, instead of gathering as each requirement is fulfilled.
Tactics: for each requirement i will use JIRA instead of Excell sheet.
Looking foward for comments =)