Death by a thousand cuts, Agile/Scrum style — an example

TerraT posted an interesting comment on this blog post.

I thoght I’d repost it here since it is illustrative of the slippery slope a project can get on when doing a “by the book” agile adoption and what the ramifications can be.

Emphasis added below is mine. This shows how things can go from agile to fragile all to quickly without overall higher level planning.

TerraT

Reading this has made my day.  I had got to the point of feeling isolated out of development completely purely down to the fact that even mentioning the short comings of Agile (as it has been applied) labels me as an unprofessional lunatic. Truefully Scrum and other “Agile” pseudo project management methodologies have the same problem that waterfall had; unrealistic expectations. Waterfall fell down due to an unrealistic expectation that scope wouldn’t slip and that design on paper would translate into good working software.  Agile/Scrum fell down due to the unrealistic expectation that you didn’t need a global technical design, that businesses could afford the weight of testing and release (automated or not) and that business stakeholders would actually want to be continually involved.

Agile has become the monster that Waterfall could only have dreamed of. As a consultant I get a peek into many dev teams and overall productivity has evaporated and the focus seems to be fulfilling the requirements of the process rather than on producing the best possible software.  What is worse the whole process has become so ingrained that developers can’t even see how mired in it they have become. Consider the following thought process that is remarkably common :

Agile is a must right? Hmm, we can’t manually test all this every 4 weeks. So you will need 100% unit testing coverage of course. Ah then we’ll need DI/IOC/Mocking on pretty much every class. Ok with that much decoupling our automated integration testing will need to be 100% too. Hmm, all that is hard to retrofit so we’ll have to use TDD. Ah, that doesn’t cover UI so lets dust off selenium and watin etc. Oh wait I have lots of JScript and Ajax calls and these JS testing and mocking frameworks are still not completely reliable. Ooops I have purely visual defects on dynamic content pages how am I going to deal with that? Hmm, the business stakeholder is telling me my automated testing isn’t a strong enough guarentee for the business as bugs are still hitting production; we’ll have to manually test it as well. Dang, my code is an abomination to DRY principles and the consistency is awful I am going to need a deep refactoring. Double dang, my tests all broke as they only covered refactoring where the interfaces stayed the same and not normalisations across classes and modules. Phew, finally got my tests all working but we had to descope half the stories from this sprint to give us the time. The product backlog seems to be getting longer and longer we might need some more devs, I hope they have TDD experience. Oh, why is everything taking so long, this is supposed to be Agile right?  What do you mean we might need a rewrite as the productivity framework we picked had limitations we couldn’t have anticipated as those stories hadn’t been added to the backlog at the time? Ok, it was a no to a rewrite but I managed to hack around the framework, I hope no one sees this code, I’m so ashamed of myself. Why are people talking about version 2? The product life span can’t be over we haven’t even finished version 1 yet? (ok maybe a little dramatic at the end but I was enjoying myself lol)

Each subsequent layer of process seemed reasonable as we added it but when you put it all together it is laughable. I couldn’t prove it but I really believe only 10-20% of the man hours invested are actually being utilised on the production code. Worse still with so much preoccupation on the process the software seems to have become a secondary consideration and despite a legion of green ticks it seems to be suffering. I think what should be learnt from this experience is that we need to stop growing our techs and approaches in academia and R&D as it never seems to appreciate the realism of working in businesses where the technical team is considered a subsidary support department (regardless of how lost they would be without us).  Development needs to be reactive to circumstances and not nailed to processes that are so brittle they shatter at the slightest pressure.

Read more: http://www.javacodegeeks.com/2012/01/4-warning-signs-that-agile-is-declining.html#ixzz1oNStueN8

What are your thoughts?

PostAgilist

Advertisements

About postagilist

Architect, Manager, Developer, Musician, Post-Agile thought leader
This entry was posted in Uncategorized and tagged , , , . Bookmark the permalink.

7 Responses to Death by a thousand cuts, Agile/Scrum style — an example

  1. Alan Hines says:

    Honestly, my thoughts are that our industry has been dragged into a gravitational sinkwell of Metawork and useless ritual designed to mimic substance, all in a cynical and calculated ploy to create the false impression among the underwriting MBAs that all of this is simply just what software engineering Is, and How It’s Done. The grand design and end game is to permanently inculcate themselves into the landscape; to become perceived as just as much an obvious baseline necessity as monitors and keyboards.

    It is an absolutely brilliant con. Most executives have no idea at all just what’s involved with building software, let alone any accurate perception of it as collaborative Art; like a “band” writing a “song,” to use my favorite metaphor. Explain that to almost any MBA you ever meet, and I promise, you will absolutely blow their minds. The best and rarest of them will immediately grasp and run with that, and thereby profit massively – You Want To Work For Them… The vast majority will think you’re talking BS; but even if they entertain your blah-blah-blah-ing (“oh, those silly nerds; they think they’re artists!”), the bottom line blunt fact is they can’t quantify and measure that; they can’t reduce down to a PowerPoint slide. So when their exorbitantly expensive $crum consultant tells them “well, of COURSE your engineers are only spending 10 percent of their time writing code; ‘Coding’ is not ‘Building Software’. You can have them sitting there and coding all day long if you like; nothing that they do will ever become Software without coming through Us. So the ten percent of their time that they spend coding under $crum is way, WAY more productive than however much they might have spent outside of it – because outside of it, whatever they might do doesn’t exist! See?” – and then follows that up with a bulging fistful of (completely meaningless) “burndown charts” – that’s easily enough to satisfy the mark. Most executives are only ever looking for a concrete number, or if not that, a simple red light / green light. $crum aptly recognizes this, and exploits it with breathtaking ruthlessness and mercenary aplomb.

    The only thing I’ve ever personally seen $crum produce has been superficially persuasive-looking evidence of its effectiveness. That much, they produce in droves; it is the singular priority. Without that, just one probative question of “hey, just exactly what the hell has software engineering actually been DOING for the past month,” and their house of cards collapses. Without that, they would starve.

  2. Pingback: New PM Articles for the Week of March 5 – 11 « The Practicing IT Project Manager

  3. skeptic says:

    In my opinion, a lot of problems start from a lack of good architecture. One of your previous posts says good architecture should be layered like lasagna right? With a strict layered approach, each layer is pretty isolated. With modern frameworks, I’m already decoupling using DI/IOC. I shouldn’t have to do “deep refactoring” on too many occasions since normally i might only need to refactor one layer (and since they are independent, it shouldn’t affect other layers too drastically).

    “The productivity framework had limitations we couldn’t anticipate because those stories weren’t in the backlog”? To start with, hopefully, these external frameworks are decoupled from your code so switching this out does not mean a major re-write on your part. Second, I seem to see this problem a lot when nothing exists outside the scope of agile. In my past experience (perhaps the writers case was different) good leads/architects have brought up limitations to approaches, however, since there is currently no story in the backlog, it is quickly passed over by the scrum master, who is not very technical and is only motivated to get more green check marks next to tasks.

    A lot of problems cease to exist with well thought out modular/layered architectures. But with agile, architecture has to sit in the back.

    • Warren Wise says:

      A lot of problems start from a lack of good architecture. I suppose that’s true. A lot of problems also start from a lack of good communication. Or from a lack of clarity about what it means for the project to be successful. Or from a lack of sound software development practices. The problem is a hydra, and while it’s easy to point at a particular methodology, technology or practice, I think a problem that underlies that is already stated in your post as “Most executives are only ever looking for a concrete number, or if not that, a simple red light / green light”.

      I worked in an IT environment that followed the waterfall approach to managing software development projects, and today I work in one that follows an agile approach. The primary problems I see contributing to a failure to achieve amazing or expected results as reported by others is a lack of knowledge, skill and disciplined application of said knowledge and skill. Why is that? Management wants results without investing in training and organizational change, as if somehow, people will magically do things correctly.

      It’s great to say that good architecture resolves a lot of issues, but how do you create a good architecture? Does it require any unintuitive knowledge? Does it require development of skills that aren’t a natural by-product of working on IT projects? I’m astounded at the degree to which organizations attempt to apply technologies, methodologies and approaches without a sound plan for ensuring everyone is competent to play their role or do their work. Why is that? In my opinion, it’s because of the managerial cluelessness you indicate in your post.

      Replace Agile with OOP, SOA, the Web, Java, .NET and the pattern plays out the same way. Once you understand the principles of software design, project management, strategic planning and organizational change, you’re then qualified to choose (and adapt) any number of tools and methods available, or to invent your own. Placing your faith in a particular approach is for the ignorant. Understanding how things work, then leveraging the resources available to you is the mark of a professional.

      • skeptic says:

        To start off, i do agree with you on picking the right solution for your particular problem. I don’t think there one solution for every problem, although many agile/scrum-bots seem to think they have the definitive answer.

        Just to clarify, though, I’m definitely not proposing “replacing Agile with OOP, SOA, the Web, Java, .NET”, because that doesn’t even make sense. It would be comparing apples to oranges, and well, that’s just silly. Also, I’m definitely not proposing you have to use any particular technology stack or architectural solution.

        What i am saying is that whichever project management methodology you are using should allow for some time to build a sound architecture. Whatever that is for your project is up to you. If you have a small web application, you might not need too much time. If you have a large scalable, multi-tenant, SAAS platform (add any appropriate technology acronym you like), you might need more. Up to you to decide. As far as scrum goes, my experience is that there is not much time allocated for these kinds of activities. Instead, there is just an expectation to do small refactoring tasks as part of sprints. The problem is that these small refactoring tasks never seem to dive deep enough to fix the real underlying issues. (hint: im not saying waterfall is the answer either, but like this blog most likely states already, the answer is probably somewhere in the middle).

      • warren2k says:

        I think I was unclear. I did not mean to suggest that OOP, or any of the other things I mentioned, should be used as a replacement for Agile. I was trying to say that the issue isn’t Scrum because the same arguments that are being made against Scrum can be made against them. I guess I didn’t do a very good job of that. : )

        Another thing I’d like to say is that it’s a misconception that agile methods like Scrum and XP discourage or prohibit creation of a sound architecture. Interestingly, and ironically, the poster child for how Scrum (project management) and XP (software development) transformed a company that was struggling to deliver more than one major release of their software a year into Forbes’ most innovative company of the year three years in a row, is a company that provides a large, scalable, multi-tenant, SAAS platform: Salesforce!

        Agile project management and software development methods transformed Salesforce as an organization, revolutionized their R&D processes, and has gotten recognized as the most innovative company by Forbes (ahead of Apple, Google and Amazon) for the past three years. There are numerous articles on the net about their big-bang approach to adopting agile, but here’s a Slideshare that you can look at, if you’re interested:

        http://www.slideshare.net/sgreene/salesforcecom-agile-transformation-agile-2007-conference

        It’s safer to blame processes, tools and technology for IT project failures. I think the biggest contributing factors to those failures are mostly people related, with poor communication and lack of competence being two that I see repeatedly.

      • postagilist says:

        Warren

        Thanks for your comments. Please try to make them a little more brief tho…

        In terms of “poster” child, I think a good posterchild for Scrum would be Easel, where it was invented, and likewise the C3 project for XP, where it was invented. Both were colossal failures, with C3 being the more colossal of the two.

        Trotting out one of the few Scrum successes (Salesforce? From 7 years ago? That’s all you have?) Compared to their colossal failures, like Myspace and Yahoo!, is disingenuous.

        It’s like saying if you practice Scrumentology you’ll be the next Tom Cruise.

        Just because Scrum didn’t kill Salesforce like it killed Myspace or Yahoo doesn’t mean that they wouldn’t have been better off with another approach, or more importantly, that most organizations wouldn’t be better with another approach.

        Good engineering is what matters, not whether they drink koolaid or practice ritual leaching.

        Ignoring all the failures while counting any success, is not only bad science, it’s pure propaganda and marketing.

        PA

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s