Kanban in a Waterfall Environment

Yoinked from VersionOne’s website – all credit where it’s due.
[vimeo http://www.vimeo.com/35163638 w=400&h=300]

Team Kanban in a Waterfall Environment from VersionOne on Vimeo.

Uploaded here really so I have it always available for reference.  Remember, it’s the organizational culture that defines what kind of process will work.  Even us hard-core agilistas don’t have the oomph to magic a command and control organization into agility.  But Kanban can be a great way for people to get to done more effectively AND gives them a chance to have more of those “aha” moments.

Is Agility like Newton’s Laws?

I’m coming to be of the opinion that if someone starts to seriously get to grips with process, within software development, project management, or knowledge work in general, eventually they end up sliding into the same set of grooves as everyone else who’s trying to do the same thing.

This, after all, is rather how agility was formed – from a set of best practices that no-one thought up, but that many had discovered.  When you do away with things that don’t work, and focus only on things that do work, lo and behold, you’ve rediscovered agility.

I’m aware that this might be bias speaking, and that I’m starting to see the world through agile-tinted lenses.  But it’s feeling more and more like another “aha” moment.  I’m starting to see agility as being a little like discovering that the Earth was round, that the Earth revolves around the Sun, a little like all great discoveries: that these discoveries are only controversial when viewed from the perspective of a previous widely-held misunderstanding.  But that once they are dis-covered, or revealed, or made distinct from the background in which they were previously immersed, then they are recognized as simply being better descriptions of the way things actually work.  In reality, not in imagination.

Any comments?  I want to give this a lot of consideration.

Organizational Culture: The Ingredients for your Agile recipe

I came across this blog http://agilitrix.com/blog/ by Michael Sahota.

Cue sound of trumpets, heavenly sunbeams, moment of revelation!

The short version of his theme of Organizational Culture and Agility is that it’s the culture that provides the ingredients.

Huh?

  • Doesn’t matter how good the recipe it, if the ingredients aren’t available, then you can’t bake that cake.
  • Culture = “How we do things round here”.
  • Culture is very hard to change.  Not impossible, but very hard.
  • If your agile adoption is looking like being a poor orphaned child, then it’s probably because the organizational culture just isn’t aligned to agility.

Read the blog for more details.

For me, it was a huge relief.  I had been getting nowhere with our agile adoption internally.  With a team of external contractors, we were using almost full Scrum and getting results.  If that team had a proper dedicated Product Owner, it would have been full Scrum.  As it was, we knew we had several people who’s combined effort was the equivalent of a Product Owner, but you can imagine, without the single voice, there was some pushing and shoving and waste.

However, with the internal teams – and I use the term “teams” here very loosely – I was getting nowhere.  After reading Sahota’s blog, and the book from which he got the spark for his ideas (The Reengineering Alternative: A plan for making your current culture work by William Schneider) what became clear was that we had a whole organization culture of Command and Control (we get things done by telling you/being told what to do) with a local team culture of Competence (we get things done because we’re good at it) with a smattering of Cultivation (we get things done by helping people grow).

To add to the confusion, we’re working in a bank originally created and developed by a bunch of good old Southern boys (I suspect army buddies) so of course the internal culture is pretty Command and Control, as well as being highly conservative and resistant to change.  That’s what it’s like in the South.  However, the bank had been bought by a huge Spanish banking machine, and they had their own culture.  To be honest, I’m still not sure what that culture is, but it is very different to the Southern culture.

So we have friction as the gears of the two culture grind together.  Ever tried to fit a metric screw into an imperial nut?  Kinda feels like that.

Now, there are loads of people here who pretend that we have a Collaboration culture (we get things done together) but it’s simply not so.  It’s magical thinking.  In fact there’s a lot of magical thinking around here.

What, I hear you ask, is “magical thinking”?  It’s seeing a causal relationship with correlation on one hand (“Every time I whistle Dixie, the wind changes direction!”) and on the other hand assuming that thoughts, wishes, and “rituals” directly affect the external world (“Time for a nor’easter, let’s get a’whistling!”) .

But that is a topic for another post.

Back to my point, it was a huge relief to realize that we weren’t getting anywhere fast with agility because it was like trying to bake a loaf of bread with sand, sump oil, and shaving foam.

So the question has to become “What can I make with sand, sump oil, and shaving foam?”  If I didn’t take a long hard look at what was actually available, then how the heck could I hope to achieve anything?  My mistake had been in the realm of “I didn’t know what I didn’t know”.  I didn’t know that the ingredients weren’t available, nor did I know I too was indulging in magical thinking.  “If I just tell them about agile enough, they MUST see the light, I mean, doesn’t everyone…?”

In the meantime, do please jump across to Sahota’s blog and absorb the wisdom there.

Resumption

Ok, ok, I’m not the worlds most assiduous blogger!

However, in the last few months, I’ve learned so much about agility, process, achievement – you know, all the good stuff.  I’ll start to blog about these topics until we’re back up to speed.

Upcoming:

  • Organizational Culture and the Agile fit.
  • Agile and Lean play nice together.
  • Agile Data Warehousing – sounds nice, but is it even possible?
  • Goal setting: tree-hugging or hard science?  (Hint – it’s hard science.)
  • Book recommendations – what happens to your head when you get a Nook!

PMI goes agile!

To my enormous delight, the Project Management Institute has finally come out with it’s own Agile certification.

See http://www.pmi.org/agile.aspx for more details.

Although I now know some great project management professionals who embrace taking an agile approach, some of the worst projects I’ve been on have been made so by project managers wielding their PMPs to great effect: namely enforcing that most noble of business laws “If something doesn’t work do more of it”.

Saying it like it is

In reading the current batch of blogs about agility, every now and then I come across someone noting that “testing after development is not optimal”.  Or some such.

Anyone mind if I’m a little more blunt?

Testing after development is stupid.  Stupid.  STUPID!

Of course, I know that now.  The thing is that for a long time, before I was introduced to the idea of Test Driven Development, I kinda knew it.  There was always that ghastly feeling “What will they find when they test this?  Or worse, when we don’t have time to test this and the users start using it?  Will I accidentally ruin someone’s livelihood because we didn’t check…?”

It was a miserable feeling.  But until I learned about TDD it tended to end up in the “too hard” basket.  I’d step over it.  We all stepped over it.

So where else are we stepping over stuff?  The Kruger and Dunning Effect says that stupid doesn’t know it’s stupid because it’s too stupid to know it’s stupid!  More kindly, the good folks at Landmark Education, champions of having a great life in this life, point out that we’re disempowered by the stuff that’s living in “Don’t know that we don’t know”.  Either way, it seems to me that once we realize that something is stupid, how on earth can it be ok to just let it carry on?  It’s not.

I see people’s lives being trashed because they’re stuck in a culture where it’s ok to work 70 or more hours in a week.  The crime is that they don’t have to, there are best practices proven to work in multiple contexts that would get them home on time.  It’s what keeps me going in the face of a whole ton of resistance – I know that things can be better.  I couldn’t sleep at night if I didn’t do something about that.

Follow-up to stakeholder pitch

The presentation two days ago went well.  The contractors seem excited, the line of business folks emailed me to say that they are keen to take this new approach, and on the whole, we took a big step forward.

That said, a few feathers got ruffled.  Figures in the project management office got wind of what’s happening, and seem very threatened by it.  According to previous experience, and that of all the great and good who write on the subject, this is a healthy sign, and would indicate that we’re right on track!

At some point, they’ll realize that we all want the same thing: to maximize value, minimize waste, and raise the company’s game.  What, you want our employer to do worse?

The awful mistake of waterfall.

I’m not sure, but every time I review this story or tell it to others, it has me wonder if this isn’t one of the biggest scandals in the history of productivity and process.

Based on what I’ve read in Agile and Iterative Development – A Manager’s Guide by Craig Larman, repeated in Agile Data Warehousing by Ralph Hughes and apparently confirmed in Wikipedia, it would seem that my unease with waterfall has some basis.

I’m going to quote Hughes directly (Hughes, p.48):

The vast majority of corporate IT departments today still utilize methods based upon the waterfall method. How can we foster the right frame of mind among IT managers that they will give Scrum/XP a honest try? We will draw upon two stratagems. First, point out that the predominance of the waterfall method is due to a tremendous error on the part of the U.S. military when it was codifying software development methods, and second, provide a long list as to why Scrum is many times more effective than a traditional approach. We deal with the military’s SNAFU here, and leave the advantages of Scrum for the next section.
The standard waterfall approach became the standard paradigm only through a series of mistakes. In 1970, W.W. Royce first identified the classic waterfall categories as only “essential steps common to all computer program developments, regardless of size or complexity.” In the second half of that same paper he clearly advocated an iterative path for progressing through these steps. To pursue the waterfall steps sequentially from analysis through coding in a single pass, he cautioned, “is risky and invites failure.” The single-pass waterfall method is only the simplest possibility for organizing a development project, and he suggested that “it only works for the most uncomplicated projects.” [Royce 1970]
Undaunted by these warnings from Royce, the Department of Defense proceeded to make the single-pass waterfall method its standard approach for systems development with its official specs MIL-STD-1521B and MIL-STD-2167. One can only speculate that they read just the first half of Royce’s paper and missed the later sections that contained his recommendations. The MIL-STD-2167 spec was particularly influential and soon propagated into many further standards both within the U.S. and internationally. Yet, when interviewed in the mid 1990s, the author of MIL-STD-2167 admitted that he had been unaware of the notion of time boxes when he drafted his recommendations, and with hindsight he would have strongly advocated iterative and incremental development methods instead. [Larman 2004 and Oestereich 2006]
In 1994, DOD replaced MIL-STD-2167 with an IID-promoting standard. Unfortunately, the other governments and standards bodies that based their methods on the earlier spec did not update their policies to match, so the single-pass waterfall approach sadly remains the basis of many methodological standards today. However, one can argue that to cling to a non-iterative, incremental approach is to persist in making a known, fundamental error in software engineering.

(Links to attributions mine.  “Oestereich 2006” attributed by Hughes as “www.isqi.org/isqi/deu/conf/conquest/2006/slides/B3.pdf (accessed Feb 2007)” but not available at the time of writing this blog post.)

I’ll be blogging more about my understanding of the history of agile, but the short version is that the failure of waterfall did not go unnoticed.  Overtime, folks realized that doing more of some things and doing less of others worked better, and that has lead to agile as we know it.

Welcome to Succeedable!

This blog is going to be used to promote and develop agility in the workplace.

I’ll be recording my actual experiences and what I’ve learned from them, and I’ll also be looking into the history of work and how come we do what we do the way we do it, and could we do it differently to give ourselves a better chance of success.

Agility has been mainly adopted in the world of software development using a variety of frameworks.  It’s cultural shift and what that makes possible is applicable to all areas of knowledge work.  Is it also applicable elsewhere?  I would argue that it is, that any project that requires a large portion of work that cannot be pre-defined is a suitable candidate.  I want to see this become a major topic of discussion.

I’ll also be using the blog to capture snippets of thought from my mobile WordPress application as I’m out and about.  These thoughts may well end up being fleshed out into full blown posts.  Who can tell?  Not me!  But this blog can exemplify the great principle of “Make it visible, periodically inspect, then adapt”.