Showing posts with label management. Show all posts
Showing posts with label management. Show all posts

Feb 4, 2012

Why do Developers Underestimate

There’s a very funny response to a question on Quora doing the rounds at the moment.  The question is “Why are software development task estimations regularly off by a factor of 2-3?”

The response (go ahead and read it) makes an analogy to hiking from San Francisco to Los Angeles to visit friends, underestimating the distance, terrain and rate of progress as well as encountering a number of impediments like the weather, colds, blisters and so forth.  It’s a really good read.

It was mentioned on the Australian Alt.Net mailing list, this week and followed up with the question of “so what do we learn from it?”

Two simple lessons come to mind.

Lesson 1: It takes time to produce a reasonable estimate

On any reasonable project don’t expect to be accurate if your estimate takes you less than 5 minutes to arrive at.  In the example, it’s pretty obvious that little time was spent coming up with the estimate for how long the hike might take.  Once they’d decided they were going to hike, taking time to estimate how long it would take was just stealing time from actually hiking!

As developers, we allow ourselves to do this all the time.  “I like to write code, I don’t like to estimate. If I get the estimate done quick then I can get back to doing what I like.”

Stop it.  You’re only storing up pain for yourself down the track.

Lesson 2: Think about what might slow you down

When you estimate think about what might slow you down.  Very few teams do this but it’s critical if you want to be more realistic about how long something will take.

Most developers estimate with the same approach that they test their own code.  They only consider the happy path.

You can hear them in their heads saying “It takes this long to do that. And that long to do that. And … then put it all together and the estimate is X months!!”.

Very few developers then say, “so the baseline is X months.  Now what is likely to slow us down?”.  Yet there are so many things that can do just that.

Want a basic list? How about the following:

  • Have you done anything like this before?  How vague are the requirements you have at the time of estimating?
  • How well do you know the technology you’re using?  Or the design and techniques you’ll be implementing with?
  • How well do you know the people on your team? Are you all co-located? Can you communicate easily?
  • Does your team have all the skills you need to get the job done or will you have to rely on other’s being available?
  • How well do we know the problem you’re being asked to solve?
  • How well can you communicate with your customer(s)? How available will they be?
  • How bad are the worst people on your team and how much will they slow everyone down?
  • How’s team morale? Are people likely to leave mid project and take a whole lot of knowledge with them? How long would it take to replace them and get them up to speed?
  • How sucky is the code you’re working on?
  • How likely is it that people on the team will get pulled off on to other work? And how often?
  • If you’re estimating will you be doing the work? Or are you estimating on behalf of someone else without involving them?

That’s a starting list of things all teams should consider when estimating and all of those items should be used to take your “happy path” estimate and adjust it from there.

 

So next time you need to estimate something, keep these two simple lessons in mind and good luck!

Aug 12, 2011

Outsourcing, Unit Testing and Costs

Let’s say you work in an internal IT department, largely doing maintenance work on software solutions.  Solutions that are by and large built by external vendors through an outsourcing arrangement and then brought in house once they’ve gone live and the initial warranty period has elapsed.

Let’s say every time you look at one of these solutions that you now have to maintain you find the grand sum of zero unit tests.  Without exception. Every single application has nary a unit test to be seen. Not even a hint that one might have existed and been removed at a later date.

“Why?!” you cry to yourself.  In frustration you cry “Why!!?” to management as well.  Their answer: The vendors tell us “it will cost more and take longer to do unit testing” and of course, they reason with you, if that’s the case then why would we pay for it? You know this is a bad argument and the logic is built on sand, so what’s your comeback? How do you get it across to management that a vendor that ignores unit testing is a bad vendor and is actually more expensive overall?

 

So my initial response to their claims would be along the lines of “So do I assume they’re not actually testing the software they write for us then.” and then explain how manual testing takes (much) longer than developing automated tests, assuming of course that the code being written is in any way testable.  If they develop poorly and have a lot of tight coupling between concrete classes or make a lot of static calls to HttpContext or develop in SharePoint for instance then the application might not be easily testable, but hey! That’s their problem not ours. The good news is that anyone with a business brain can understand the benefits of automation for tedious repetitive time consuming processes – heck, we worked that out during the industrial revolution.

Let’s try a different angle in case the first, most obvious one doesn’t work.  If you consider a lack of automated testing to be merely another form of technical debt then it’s just like poor design and architecture, horribly complex methods, N+1 ORM queries, poor UI’s, high bug counts, etc. It’s something that will cost you. So, if a vendor delivers you software with a high level of technical debt and you’ll be maintaining it, then you’re going to have a lower long term ROI and higher TCO from those vendors versus what you might have from a vendor who keeps their debt levels low.  So while the cheap guys may argue that it will cost more and takes longer to write code with unit tests then what they’re really saying is this

“It costs US more and takes US longer to do unit testing because the developers we hire are the cheapest and suckiest we could find. They don’t understand good development practices and we’re not about to teach them because they’ll leave in the next 3 to 6 months. Also manual testers here are cheap-as-chips and lose staff too quickly, so it’s easier for US if we just get the software to ‘it kind of works’ and then give it to you so we can get our money and hope you won’t notice”.

Also, don’t forget that post delivery, all those bugs and problems that snuck through from the vendor will end up becoming your bugs and problems to fix as part of your maintenance work.  So the company is either paying for it with internal staff costs (often seen as a hidden/sunk cost) or, if the vendor is also charging for maintenance, paying for it once a development project is concluded.  Development takes longer, it’s harder to find and fix bugs, it takes longer and longer to test the application manually, etc.  These all add up to time, and in software development time is money!

With that, good luck convincing management to change their vendors, and if you’ve got a better reason to present yours case I’d love to hear it.

Note: I’m not implying manual testing is bad.  In fact I think you should be doing manual testing; but only manual exploratory testing where testers are trying to break your code.  It’s rote, repetitive manual testing that is neither cost efficient or effective.

Sep 22, 2009

Australian Virtual Alt.Net – Influence Strategies

I attended the QCon conference in London earlier this year and during it I was lucky enough to attend one of Linda Rising’s workshops on Influence Strategies.  Recently I presented my own take on her content at the Oz Virtual Alt.Net group, and here it is for your enjoyment.

P.S. The credit for the good content goes to Linda, anything dodgy in there is all me.

Aug 25, 2009

Some “Developers” Just Can’t Develop

I’ve been interviewing candidates for developer positions for over a decade now.  In my early years I was very much hit and miss in terms of finding the right people and it wasn’t until a horror hire that I realised what I was doing wrong.

Previously, and like so many other people I’ve seen, I would interview candidates by looking at their resumes, asking questions about their history and experience and basically trying to get a feel for the person and how they’d fit in the team assuming that if they could talk about programming reasonably well then they must be able to do it as well.  Then, assuming that conversation left me with a positive feeling, I would do the reference checks and make a go/no-go decision.

This resulted in a success rate of maybe 60%, though I may be flattering myself here. But 60% isn’t great especially when you consider the costs of a bad hire.  Even so, the bad hires I made I wouldn’t call shockingly bad, it’s just that they weren’t up to the standard I was aiming for.  At least that’s what I thought until I made a really, really bad hire.

This guy was special.  He was listed as being a .NET team lead, he talked about his history and skills really well, he was engaging and confident, and his references came back glowing with praise so I decided to offer him the job.  This he duly accepted and started shortly thereafter.  Now one of the things I was doing with new hires at the time was getting them to work on bugs – it helped me knock down the backlog and gave them exposure to a variety of areas across the system.  I also gave them plenty of time early on to learn the ropes and get up to speed and it would typically take a few days to get through the first few bugs and start having some ah-ha! moments with the codebase.

So anyway, this particular person starts and is given his first bug.  Quite a simple bug too and yet it took well over a week for him to get it done and even then he needed plenty of help and prodding along the way.  I was also getting complaints from the other developers that this guy was hopeless and a really bad developer which I couldn’t quite believe so I decided to take some time, sit down next to him and do some pairing.

What a shock I got.  It soon became all to apparent that the guy couldn’t program to save himself.  He… literally… couldn’t… code.  At one point I was breaking down the task to fix the bug into very, very small steps and asked him to put in a simple for loop and he just stared at the screen completely blank.  So I asked him what the syntax was for the for loop thinking he was stuck thinking about the larger problem or that he was freaking out with me being next to him growing ever more frustrated.  He couldn’t do it.  He simply couldn’t tell me the syntax.  Not even when intellisense was doing it for him.  How about a while loop?  Nope.  An if statement?  Only just. It was absolutely frightening. How could I get it so wrong?!  I couldn’t believe I’d made such a monumental screw up as to hire this guy.  I also couldn’t believe that the references about him were so great.

Then the slow realisation dawned on me that I’d been lied to. I know – I’m quick off the mark aren’t I? :-)  Imagine someone lying on their resume! The shock of it! Imagine someone getting their friends to pretend to be their former employer and giving them great references. Silly me.

So what was I to do? I remember an article I read around that time from Joel Spolsky that made me slap my forehead.  It basically says if you want to hire a developer, get them to do some code in the interview!  I couldn’t believe I’d been so short sighted as to not do this before.  Gah! Sure, people can lie on their resumes and get their friends to lie for them as well but when you see someone write code, there’s nowhere to hide.  You can either develop or you can’t.

So from that point on I got people to do a debugging exercise.  I figured if the code was there, then all I’m really doing is getting them to read code, understand it and make a few changes.  Pretty much like normal development :-) So I got one of my team to devise a small program that had a number of bugs in it.  You would fix one, uncover another, that sort of thing until you got it working; and all in about 30 lines of code.  When I was given with the code I debugged it first time in a matter of minutes and then used the time it took as a guide to estimate the skill level of the people I was interviewing (with judgement applied based on how nervous they were).  I’ve been using this same test, with various amendments for language changes and so forth for quite a few years now and it’s been a fantastic aid at helping me weed out those who can code and those who can’t.

So just recently I was asked to help another company vet some candidates for a role that didn’t need as high a calibre of person as I normally look for, just a competent one instead, so I figured my debug exercise might prove a little too challenging for the level I was looking for.  I figured that as a change I’d use something simpler try out a variation of Jeff Atwoods FizzBuzz test.  I figured that if the candidate got through that quickly enough then I could always whip out the debug exercise and see how they went with something a little tougher.  Unfortunately I never got past the FizzBuzz exercise since by the time they finished it they’d killed all the time I’d set aside for it (and maybe a little more).  Was I surprised at this? After all it is a pretty simple coding exercise. Well, as it turns out, No. Not really.

If you have a look at Atwood’s post and even the earlier Reg Braithwaite post then in them you’ll see that they both quote this statement “I’ve come to discover that people who struggle to code don’t just struggle on big problems, or even smallish problems (i.e. write a implementation of a linked list). They struggle with tiny problems.”.

How very true.

By doing the fizzbuzz exercise for real, I merely confirmed what I already knew from my experiences with the debugging exercise. So, what’s the lesson here? I think it’s simple. Get your candidates to write some code during the interview.  You don’t need any special tools to do it either – just use a whiteboard and pseudo code if you want to.  But whatever you do, make sure they prove to you that they can actually do the job, not just talk the job.

Remember: There are plenty of developers that just simply can’t develop.  Avoid my mistakes and do what you can to avoid hiring the wrong people.

Aug 20, 2008

An Introduction to Microsoft's ALM (Application Life-cycle Management)

Last year Microsoft launched the "Microsoft Application Platform".  The application platform is Microsoft's vision for how businesses can use Microsoft technology to improve information & knowledge management within their organisation and focuses on 5 key areas as follows, each of which wraps a number of Microsoft product & services offerings:

  • Business Intelligence
  • SOA & Business Process
  • Development
  • User Experience, and
  • Data Management

I'm not going to iterate through the various products that sit under each umbrella, instead I'm going to drill into the Development area as that's the one I find most interesting.

Development Platform

The Development area of the application platform is actually called Development & Application Life-Cycle Management, and normally gets tagged with the ALM acronym.  The good news is that this isn't just an attempt by Microsoft's to sell you on their tools (though obviously that's a major element of it) but it's also an acknowledgement that good processes and practices need to be followed to ensure that applications don't cost an arm and a leg to not only develop, but also to maintain in the future.

From a Microsoft ALM whitepaper comes this little gem: "Acquiring tools to support ALM, such as Microsoft® Visual Studio® Team System (VSTS), is straightforward. Introducing and driving an ALM strategy within your organization, and understanding the necessary changes required within your business, is more difficult."

The introduction section of the same whitepaper talks to the fact that project failures can be traced to a limited number of root causes described as:

  • Lack of visibility into project status
  • Ineffective Team Communication
  • Balancing Business Demands with Project Risk
  • Unpredictable Delivery Times and Quality

These are all valid and legitimate issues.  Issues which both traditional and agile development methodologies attempt to resolve.

What is ALM?

Microsoft's ALM offering is a combination of both development processes and supporting software to help automate whichever ALM provided process you follow.  In the ALM offering Microsoft have included processes for both agile and traditional development models.

Because technology is always on the move a normal development methodology won't specify what technology a team should use, just the processes and practices to follow.  A team can be using .NET, Ruby, Java, PHP, Cobol, whatever.  The difference with ALM is that even though it includes methodologies to follow, it also includes tools and technologies to support those methodologies - namely Visual Studio Team System (VSTS).

Knowing that VSTS is part of the Microsoft ALM offering it might be easy to fall into a trap of thinking that ALM is just VSTS by another name. It's not.  VSTS in and of itself does not require any specific development process.  It is designed to help you automate your development activities now matter what they might be.  How a team then uses VSTS is up to them.  In fact, you can easily find teams using VSTS that have a development process as loosely defined as "write code and see if it works".  That's not something that can be considered as ALM, simply because it fails to address the project management aspects of application development.

How Does VSTS support ALM processes?

So, knowing that VSTS is part of the ALM offering, how does it help? What should the ALM support tool offer us?

According to Wikipedia, ALM offerings are generally expected to cover the following features:

  • Requirements Visualization
  • Requirements Management
  • Feature Management
  • Modelling
  • Design
  • Project Management
  • Change Management
  • Configuration Management
  • Build Management
  • Testing
  • Release Management
  • Deployment
  • Monitoring and Reporting
  • Workflow

And VSTS offers support for all of these items.

Project Management, Requirements Management, Change Management, etc are all process/methodology related items and Team Foundation Server includes process templates for both agile and traditional development methodologies (the MSF Agile and MSF CMMI templates) as well as letting you plug in your own process templates if you wish to use an alternative methodology (e.g the Scrum template).

For more info on VSTS and how it supports ALM have a look at the Team System site.

 

It should be noted that Microsoft has an online assessment tool to help you gauge how well you are performing Application Life-Cycle Management.

Aug 11, 2008

Crunch Time

I was at client a while ago that were after any assistance they could get on a project exhibiting all the symptoms of a classic Death March project.  A waterfall based process gone horribly wrong, developers working ridiculous hours, poor code quality, emergency fixes, misunderstood and late-changing requirements, staff leaving due to stress and so much more.  It was all there.

I was originally going to write about how this all came about, but the more I thought about it the more I realised that how it came about wasn't really that interesting to me.  What was interesting was what crunch time revealed:

1. Communications.  The prevalent form of communication during crunch time was verbal, and typically face to face with a notepad or whiteboard handy.  Hardly any emails, hardly any paperwork, just people talking to people to ensure communications were clear and understanding was mutual.  Fast and effective.

2. Team work.  Pressure can do one of two things to a team - either break it apart or bring it together and form something stronger.  In this case it was the latter.  Watching how the team came together to support each other and achieve a result was fantastic.

3. Blurring of boundaries.  There is usually a very clear departmental delineation between the developers and the infrastructure people (i.e. the team responsible for the production environment).  This is the case in many large firms, however during this particular crunch period the line was much more "malleable" with people from both departments working extremely closely together to find problems and get things resolved as quickly and efficiently as possible.  Why developers couldn't have read-only access to production on a regular basis is still a mystery, but at least they overcame it during the crunch.

4. Impediment Removal.  The manager responsible for the development team did everything in their power to remove obstacles.  Whatever it took, they did it.  This ensured that the team could focus on delivering the project and that nothing else interrupted them.  At one point the CEO even picked people up from their homes to save them time on the commute.

5. One prioritised task at a time.  During the crunch all the remaining tasks were prioritised in a simple excel sheet.  Each developer took a task and worked on it, and nothing else.  No multitasking, no new jobs until the current task was complete.  This avoided the usual situation of having 5 tasks all active at once as trying to task swap between them as the mood takes you.  The excel sheet was communicated to everyone everyday so that all stakeholders were kept up to date on progress.

 

It's really interesting how crunch time forces you to genuinely prioritise what you do. How much regularly wasted time teams usually have and how much of that just isn't necessary when a deadline looms and just how effective people can be if you clear the decks for them.

What's even more interesting is that this is the sort of behaviour we want from an agile team each and every day of the year.  Just without the soul crushing pressure of a death march project :-)

Something to think about, isn't it?

Jul 3, 2008

RDN Dev Days featuring Agile Development

Some of you may be aware that Readify have been running a series of free developer sessions for some time now known as the Readify Developer Network. Past RDN sessions have covered a wide range of topics and have been presented by various people from within Readify, including many of the 10 MVP's we have on board.

Well, it's pleasing to announce that while we are continuing to run the regular RDN sessions we're now also launching an extension of RDN this month called the RDN Dev Day.

Think of it like a one-day tech conference featuring content across 3 tracks presented by 3 of Readify's team, including Hands on Labs to help you get to grips with the latest .NET technology.

The 3 presenters for the inaugural sessions will be Mitch Denny, Paul Stovell and myself with the tracks being:

  • Team System
  • .NET 3.0 & 3.5 (WPF, Silverlight, Linq & WCF)
  • Agile Development

The dev days will be happening in Brisbane, Perth, Sydney & Melbourne over the coming weeks.  Brisbane on July 25th, Perth on July 29th and Sydney & Melbourne in mid-August.  Note that I won't be at the Perth Dev Day, so some of my sessions will be presented by Mitch.

Here's the agenda so you know what's available.

Agenda

AGENDA
Registration 8:30am
Welcome and Key Note 9am-10:30am
Key Note: The .NET Roadmap for 2009 & Crystal Balling - What will .NET look like in the future?
Morning Tea 10:30am
BREAKOUT SESSIONS PRESENTERS
Times Mitch Denny (MVP) Paul Stovell (MVP) Richard Banks
10:55-12pm Team Development with Team Foundation Server (I) Introduction to WPF Introduction to Agile Development
12:10-1:15pm Team Development with Team Foundation Server (II) Introduction to Silverlight Test Driven Development
Lunch 1:15pm-1:45pm
1:45pm-2:45pm Adding Manageability and Scriptability to your Applications Hands on Lab: WPF Introduction to Agile Development [Repeat]
Afternoon Tea 2:45pm-3pm
3pm-4pm Hands on Lab: Setting a Team Project in Team Foundation Server LINQ in Depth Planning & Estimating in Agile Projects
4:05pm-5:15pm Managing the Database Upgrade Process with DBPro and Team Foundation Server Hands on Lab: WCF Basics Agile Q&A

 

Please note that demand is expected to be high and registrations are only open for a short period of time, with Brisbane and Perth open now so get in early and book yourself a spot today.  I'm really looking forward to presenting and I'd love to see you there!

 

UPDATE: In case it wasn't clear, only Brisbane & Perth registrations are open at the moment. Sydney & Melbourne will be available in the next few weeks. Sorry for any confusion.

Jun 19, 2008

What's Wrong With Traditional Software Development

I was just reading some posts on the scrumdevelopment mailing list and saw this from Roy Morien about traditional software development:

 

Traditionally software development processes have been predicated on the following assumptions:

1. It is possible, efficient and effective to create a thorough and complete analysis of the requirements for a system, at the start.

2. Is it possible to plan in detail the development project, even 2-3 years ahead.

3. It is possible to provide accurate estimates of the cost of the project, and of the development time and completion date.

4. Requirements are stable and will remain unchanged during the project period.

5. Following the plan is the best way to ensure project success.

6. A serial or linear phased development approach is the best , most efficient most effective approach to software development.

7. Software development is fundamentally a technical activity that is repeatable, and can be planned and managed according to a preconceived plan.

8. Software projects can be undertaken in the same way as civil engineering projects, which is orderly, linear and sequential.

9. Certainty of outcomes can be achieved if only we properly, correctly and comprehensively plan, analyse, estimate, document and properly manage the project.

10. Project success is counted in terms of 'Within Budget, Within Time, Within Scope'

I then made the statement  "On every count, these assumptions have been demonstrated to be incorrect", and I proceeded to discuss why, and the evidence of that.
Basically I discussed the simple fact of 'Future Uncertainty". This really is the core problem that all of these 10 assumptions fail on.
I was a bit surprised at the audience reaction ... It was almost like many of them had never really thought about this in this way. To have their basic underlying project management principles shown as almost totally flawed seem to come as a bit of a shock to them.

Personally, I'm not at all surprised.  I see the same reaction over and over again in almost every place I go.  Almost every single project manager is taught a one size fits all project management methodology (typically PMBOK or PRINCE2) by an esteemed educational institution and no alternate options are ever discussed, so why would they think that the methodology they've been taught can't work for software development?  Especially if they've never actually done any development themselves.

Hopefully the increasing awareness of the problems with traditional software development and the alternative of agile methodologies will finally result in educational institutions teaching prospective PM's something that actually works in the real world.

Jun 6, 2008

Technical Debt (aka Code Debt)

What Is Technical Debt?

Technical Debt (sometimes called code debt) is a concept that was first talked about by Ward Cunningham way back in 1992 and has since been covered by people like Steve McConnell and Martin Fowler to varying levels.  Rather than repeating everything they've said I'm going to try and I explain it in my own words.  To me technical debt is best thought of as the technical issues and problems we have that slow us down and prevent us from adding as much new functionality to our software as we would like, in the time we have available.  This is typically seen in teams that have a decreasing velocity or noticed when changes that once took half an hour to make now take a couple of hours.

Now, when most people are initially presented with this concept they will immediately think of bugs and it's easy to understand why.  Bugs take time to fix and are an obvious cause for a drop in the time available for adding new functionality.  But bugs aren't the only form of technical debt.

Technical Debt can come through a number of channels, sometimes deliberate but more often unintentional:

1. Bugs.  Obviously.  Bugs have to be fixed.  Time is required to fix them.  However a large number of bugs decreases a developers confidence in the code.  When developers change the code they will either take more time to make sure they haven't broken anything else as a result, or will ignore the other problems and in the process add more bugs.

2. Lack of Automated Testing.  Without automated testing developers are unlikely to be sure that they're changes are correct.  They (or testers on the team) will spend extra time to make sure that the changes haven't broken anything obvious, thus slowing down the team.  Unfortunately, they are unlikely to do a full regression test nor are they likely to try multiple methods to break their code, which means it's likely that the number of bugs in the system will increase.

3. Poor Architecture.  Architectural costs are hard to measure, but impact the team dramatically.  Poor architecture is typically evidenced in hard to test code, where code changes need to be repeated in multiple locations, where different approaches to the same problem are seen in the same application, where things just don't make sense, and where developers openly joke about the crap code that they have to deal with.

4. No Coding Standards.  Coding standards exists to help increase the readability of code.  People should be able to look at code and see a common coding style that anyone on the team is familiar with. Code that is hard to read takes longer to understand and is harder to make a non-buggy change to.  It's also common to end up with multiple coding styles apparent in the same method or file, making it even harder for people in the future to understand it.

5. Highly Complex Code.  Code with a high cyclomatic complexity takes time to understand.  Making a non-breaking change is often even harder.  If effort is taken to refactor the code then that also will take time.

The Problem

"So why is this a problem?" I hear you ask.  Lets think about it.  Let's assume we start a brand new project. No code. No frameworks to deal with. Nothing.  A blank slate.  Now when the project starts out we have the ability to add as much new functionality to the project as we can within an iteration.  And we do, however in that first iteration our team ends up creating bugs.  It's just the nature of the beast.  We're all human, and we all make mistakes.  But since we've only just started we're not that worried about those bugs at the moment.  We'll deal with them later.

Unfortunately our developers are all individuals and each has their own coding styles and standards. And because they haven't really done it before they don't think about unit testing or loosely coupled architectures.  It's a very common scenario.

So the first iteration concludes, and in that first iteration our team delivers a large amount of new functionality and we're really, really pleased about it.  Congratulations all round.  Unfortunately and unseen we've also accrued some technical debt but it's not really a concern at this point, right?

So our next iteration begins.  We now start making changes to existing code to extend on functionality we delivered in iteration 1.  Unfortunately some of those bugs we weren't worried about are stopping us from completing our new work and our team has to spend time fixing those bugs as well as making their changes.  Things they thought would be fairly quick to do end up taking more time than expected and we also find that developers changing each others code are having issues understanding what it's doing and working with other people's coding styles.

At the end of the iteration the team still delivers a good chunk of functionality, but it's not as much as they were hoping for.  The reason - it just took longer than we expected.  While we're not as happy as we were at the end of the first iteration, it's still a good delivery and it's not much below what we were hoping for.

The next iteration we see the same patterns of behaviour as in iteration 2. And this time our team delivers less functionality than ever before and we're not happy.  The developers aren't sure why things are taking longer, they just are.  Maybe it's the tools, maybe it's the BA's fault for not writing better specs, whatever the case, it's not developers.  They're trying just as hard as they were at the start of the project, so it can't be that.

What is the root cause of the problem?  As you probably guessed, it's the technical debt.  As our technical debt increases we have less capacity for adding new functionality resulting in frustration for both our team and our customer.

Left unchecked we will eventually get to a point where we can't add any new code at all because we don't have any spare capacity left.  We end up with a team spending all their time understanding code and fixing bugs just trying to keep the damn thing running.

Here's the problem in graphical form:

image

Solutions - Option 1

OK, so we have a problem.  Our team is stuck in the mire of endless bug fixing.  How do we get out of it?

Here's what most people will do: hire more staff!

This is the worst thing you can do.  It's like getting an overdraft on a loan you can't pay back.  What's worse is that any new staff you get will take time to get up to speed and will undoubtedly make more mistakes initially than anyone else because they haven't got the experience the others have in dealing with how the application works.  Plus, because the root cause of the problem has not been dealt with our code debt will just continue to increase until we're out looking for more staff again.

It looks like this:

 image

Solutions - Option 2

Option 2 is a more tantalising one.  Start again! We can scrap our current code base and start fresh, learning from the mistakes of the past.

This is a very tempting option for a number of reasons.  First it means we can write off all that technical debt we accumulated. We can upgrade our development tools to whatever the latest and greatest release is. We can get excited about doing things differently and we can tell ourselves that because we learnt so much from our previous mistakes that this time will be sooo much better.

This sounds great in theory, but we're forgetting something.  The application itself and the bade code isn't the problem.  A crappy code base and a poor application is a symptom.  The problem lies with our people and the way we work.

If we start fresh and don't deal with the fundamental issues then we're just going to repeat the problem of increasing technical debt, only we'll be doing it with different tools.  Given enough time we'll be right back where we are now, with an unmaintainable application and an unhappy team, only it'll be worse this time because we'll remember that this was a rewrite and weren't things supposed to be different this time 'round?

Solutions - Option 3

OK. So hiring more staff is out.  And starting again is probably not the best idea in the world.  What do we do?

Fix the fundamental problem.  It's a hard thing to do.  We have to press pause on our development work and spend the time to clear some of our debt.  And this is not an easy thing to do - especially if we've got an ingrained and established pattern of debt accrual and a demanding customer or overriding business issues.  Plus you have to somehow convince the business that there's value in doing it and you have to convince the developers to change their ways.  My only suggestion is to try and confront the issue head-on and deal with them openly and honestly.  Call it a learning experience and use the charts above to back up your arguments for change.  You might just get lucky and buy yourself the time you need to save the situation.

 

If you do there here are some suggested starting points for clearing the debt.

Start by agreeing to some coding standards as a team - and follow those standards! No lip service, no saying one thing and doing another.  Make sure that you do code reviews or pair programming and that your team polices each others work.  Use tools like StyleCop and FxCop to assist in your efforts.

Next try to fix some bugs.  Don't try and fix all of them, it'll take too long.  But fix some of the big and ugly ones, the major pain points.  And do so using the newly agreed to coding standards.

Then try to add some automated testing - add unit tests, add functional tests, do anything that will give you some confidence that changes you make don't break something else.  Do what you can to break dependencies between classes, to improve your architecture and to simplify your code.

Anything you can do to make your code more maintainable is going to help in reducing your technical debt.

Finally, after an agreed period of time start taking on new development work again, but this time don't take on as much as you can fit.  Take on some work but keep some capacity for further clearing of technical debt.  It will take time, but done right, you'll break the debt cycle and get back to doing what you do best - delivering great software for your customers.

Oh, and don't be too concerned about clearing all of your debt.  Every team carries some form of technical debt, the difference is that the best team's carry as little as possible.

May 20, 2008

Agile KPI's

I was recently asked by Aaron to do a blog post about KPI's when using Scrum (thanks for the suggestion!).  The basic idea being that many companies want to have role level KPI's, but how do you do that with Scrum, especially when Scrum seems to be about the performance of the team.  I say "seems" because it's often how people who are new to Scrum will look at things, which is unfortunate as they miss the point that scrum is concerned with not just the team's results but with the individuals in that team as well.

So, what KPI's should be considered for members of a scrum team?  At a team level it's easy enough - delivery of working software that provides business value.  But for individuals, what do we do?

The first instinct might be do look for something that's easy to measure such as taking on a certain number of hours worth of tasks per sprint, or being present on time at the daily stand up meetings, but these would be poor KPI's as they measure attendance and allocation of tasks, and not delivery of business value by individuals.  Instead I'd suggest that what we really want to gauge is how much overall value people are adding to the business and the team - and that value isn't always tied directly to their work artifacts/outputs.

Here's some suggestions for KPI's you could apply to individuals in a scrum team:

  • Positive Team Involvement
  • Mentoring
  • Knowledge Sharing
  • Proactive Alerting of Impediments
  • Working According to Team Standards (e.g. TDD, style & naming, code comments & documentation, regular check-ins, etc)

In case it's not implicitly obvious, I'd suggest that individuals in the team would have different expectations on their KPI measures.  For example, the more experienced team members would be expected to spend more time mentoring than the juniors.  The team members with greater domain knowledge would be expected to spend more time sharing that knowledge than those who are new to the problem domain, etc.

So then how do we then measure these factors?  One method might be through peer feedback, others might use independent observation, others might try something completely different.  My suggestion would be a mix of observation and feedback - the scrum master needs to observe the team dynamics and behaviours on an ongoing basis, however they often miss things that the quiet achievers do during the day, so having a peer review process is a good thing to add to the mix.  You might want to get feedback from the team at the end of each sprint or you might choose to have a 6 monthly anonymous feedback mechanism and use that with a mix of judgement to work things out.  Whatever you choose it will end up being a judgement call as to wether people met their KPI's or not.

The most important thing to remember is that in the same way that the team is measured on results, individual KPI's should be trying to measure the individuals value, not their effort or their time spent in the office.

Apr 1, 2008

Deming's Rules of Management and Agile Methods

I saw a post this morning by David Anderson this morning on how a fault tolerant culture helps create great teams.  Admittedly he was talking about the British cycling team's recent gold medal haul but the point is valid regardless.  In his post he referred to W. Edwards Deming whom I'd never heard of as a source for the principle of removing fear from a culture.  Curious, I flicked over to the wikipedia entry and had a look at what the esteemed but unheard of Deming had to say.

Here's an excerpt from the wikipedia entry:

Deming's 14 points

Deming offered fourteen key principles for management for transforming business effectiveness. The points were first presented in his book Out of the Crisis (p. 23-24)[19].

  1. Create constancy of purpose toward improvement of product and service, with the aim to become competitive and stay in business, and to provide jobs.
  2. Adopt the new philosophy. We are in a new economic age. Western management must awaken to the challenge, must learn their responsibilities, and take on leadership for change.
  3. Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.
  4. End the practice of awarding business on the basis of price tag. Instead, minimize total cost. Move towards a single supplier for any one item, on a long-term relationship of loyalty and trust.
  5. Improve constantly and forever the system of production and service, to improve quality and productivity, an thus constantly decrease cost.
  6. Institute training on the job.
  7. Institute leadership (see Point 12 and Ch. 8 of "Out of the Crisis"). The aim of supervision should be to help people and machines and gadgets to do a better job. Supervision of management is in need of overhaul, as well as supervision of production workers.
  8. Drive out fear, so that everyone may work effectively for the company. (See Ch. 3 of "Out of the Crisis")
  9. Break down barriers between departments. People in research, design, sales, and production must work as a team, to foresee problems of production and in use that may be encountered with the product or service.
  10. Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.
  11. a. Eliminate work standards (quotas) on the factory floor. Substitute leadership.
    b. Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership.
  12. a. Remove barriers that rob the hourly worker of his right to pride of workmanship. The responsibility of supervisors must be changed from sheer numbers to quality.
    b. Remove barriers that rob people in management and in engineering of their right to pride of workmanship. This means, inter alia, abolishment of the annual or merit rating and of management by objective (See CH. 3 of "Out of the Crisis").
  13. Institute a vigorous program of education and self-improvement.
  14. Put everybody in the company to work to accomplish the transformation. The transformation is everybody's work.

What struck me as I read this was how much lean and agile processes try to utilise the principles.

Here's my quick take on alignment between Deming's principles and agile methods:

Point 1 - Constancy of purpose.  Best described by the agile manifesto.  It's a clear and unchanging vision of what agile methods are all about and what they are trying to achieve.

Point 2 - Take on leadership for change.  Agile understands and accepts change.  Change happens - let's make the most of it and lead the way towards change.  Change for the better.

Point 3 - Build quality into the product from the start.  Test Driven Development (see video) is a perfect example of this as is the desire to minimise code debt.

Point 4 - Minimize cost.  Time is the number one cost in software development.  Minimizing wasted effort is the single best way to reduce cost.  Minimal code debt also goes a very long way to achieving this

Point 5 - Improve constantly.  Agile calls this principle "inspect and adapt". It's embodied in scrum retrospectives and other review processes.

Point 6 - Training on the job.  For developers this is encapsulated by pair programming.

Point 7 - Institute leadership.  Leadership in an agile project arises from within a team - it is not imposed through management hierarchies (and imposed "leadership" is typically psuedo-leadership - leaders still naturally arise in teams).

Point 8 - Drive out fear.  I don't know that this is explicitly addressed by any agile methodology per se but it is something that all good leaders will endeavour to build into their teams.

Point 9 - Break down barriers.  Agile methods cry out for openness and collaboration.  It's at the heart of what agile is all about.  Barriers are impediments and slow down teams.

Point 10  Eliminate slogans, targets, etc.  The essence of this is that management can't drive people to work harder to improve because of organisational impediments (the system). In software development "the system" is the organisation and the red-tape and processes that organisation imposes on their staff.  Scrum in particular addresses this (indirectly) through the role of Scrum Master - a person responsible for removing impediments to the team.  If the organisation is slow the Scrum Master should be addressing that.

Point 11 - Substitute Leadership.  See point 7.

Point 12 - Remove barriers.  Covered in points 9 & 10.

Point 13 - Self improvement.  Once again the inspect and adapt principle at play - good teams should be applying this personally as well as at a team level.

Point 14 - Everyone works to achieve transformation.  The is the very essence of an organisation with an agile heart.  Everyone working together to achieve an objective and improve what they do.

 

I think I might have to go and hunt for Deming's book and have a read.  I'd be interested in your thoughts as well - especially if you've read Deming's work.

Mar 26, 2008

A Tale of Two Companies

There were once 2 companies who both wanted to develop some custom software to help their businesses enter an evolving market and gain a competitive edge.  Both companies made an initial approximation of the costs involved in getting the software developed and both companies set aside a similar sum of money for the project.  Neither company had any real appetite or skill for developing the software internally and decided to get external companies to help.

Company A

Company A decided to take the traditional, recommended, "best practice" approach for getting the software developed.  It's what they'd always done in the past, it's what their project managers recommended and even though they'd never had a project come in on time before they hoped that if they  did things right this time then it would all be OK.  They also decided to set aside a portion of their budget just in case there was some overrun in the project.

Because Company A wanted to get the best value for their money they decided to put out a tender.  They knew that a tender would give them a result they wanted for the lowest fixed price and they'd get everything they asked for.

They also wanted to get the tender right. They knew how many software projects failed but they figures that that was mainly because requirements in tenders had never been quite right before. If they could get their tender requirements right this time then it would give them their best shot at a successful project and they'd be able to enter this new market with confidence.

Company A decided that the best way to get requirements right was to get in an external expert to help draft and clarify the requirements and to word the tender in the best way possible so that anyone could to clearly understand what it is that they were after.

It takes a week to find the right expert and during the initial meeting the expert listens to the high level overview of the project and estimates about 6 weeks to get the tender ready.  The expert then starts talking to the stakeholders in the company and begins to draw up an initial draft of the tender.

The expert uncovered more requirements than were initially explained and, as this was a perfect opportunity, a number of stakeholders added wish list items to the tender to be included as low priority items. After two months the tender draft is ready and placed before the stakeholders for review.  Everyone in the company understood why the extra 2 weeks were needed to get the draft completed and thought that it was time well spent, especially as some critical requirements were discovered that would otherwise have been missed.  Unfortunately the review also determined that some of the requirements weren't well described and others some seemed to have been misunderstood.

2 weeks later the revised draft was again presented to the stakeholders for review.  There were still a few things that weren't quite right and a couple of the stakeholders were unable to attend due to scheduling conflicts but as the tender had now taken a month longer than estimated the remaining stakeholders signed off on it and got the project moving.

The published tender had over 400 requirements and suppliers were given a 4 week window in which to respond.  Responses when they arrived were larger in both number and size than expected and the review of the responses took 4 weeks to score and complete before a group of 3 potential suppliers were short listed and allocated a day each to present their proposals, 2 weeks after being notified.

Now after 20 weeks the company was at the point where they were ready for vendor presentations.   And they still had no software to show for it. Plus they'd spent 15% of their project budget already.

 

Company B

Company B took a rather different approach.  They'd been burned by tenders and fixed price, fixed scope contracts before and their CIO had read about agile development and some of the success stories that were talked about.

While it was considered a risky approach,  the CIO knew that repeating what had failed before in the hope that the result would be different was the very definition of insanity, and recommended that an agile approach be taken and immediately set about locating a number of potential vendors who could deliver a project using an agile methodology.  After 2 weeks the potential vendors had been contacted and a preferred vendor chosen.  The CIO and the vendor also took an unusual approach to the contract. They agreed that cost would be limited, but that scope would not.  A minimum set of functionality would be delivered but beyond that anything could be changed by the company.  Company B could terminate the contract at any time of their choosing, as long as they paid out a portion of the remaining contract.  The vendor would deliver the highest value software they could using a prioritised list of requirements as determined by Company B. The vendor also stipulated that they wanted to have a single representative of Company B to work with, not a committee. Changes to functionality, priorities and requirements were allowed at any time with no extra cost so long as those things changed were not currently being worked on.  If Company B wanted to add new features or change existing ones then that was their right.  The vendor also agreed to deliver regular releases of the product so that the company could see what was being built and could optionally put that code into production should they deem it appropriate.

It took a few more weeks to get the legals and the resourcing of people sorted out but even so, the project kicked off 4 weeks after the initial "go" decision.

During the initial week of the project stakeholders from Company B and members of the vendor's project team sat together and worked out a prioritised list of high level requirements.  Many of the requirements were still somewhat vague but the vendor wasn't concerned - they would work with Company B's people to clarify those requirements better once they got to them.  Each requirement was given a relative effort estimate based on how much effort there might be compared to other requirements in the list, and was also given a simple set of criteria to help determine when a requirement could be considered complete.  After some consultation with the vendor the prioritised list of requirements was altered slightly to take into consideration any cross-requirement dependencies and pre-requisites but the end result was an order list of things that Company B wanted in their software.

The very next week development work began.

The vendors team started by taking the 4 first items from the requirement list making a commitment to complete them over the next 3 weeks, saying they would have something finished that they could show the people of Company B when the time was up.  However the CIO was a little concerned - 4 items didn't seem like that much to work on.  However after the team talked through how much code and database set up needed to be done in order to complete a single item the CIO was willing to go with it.  At worst they would waste 3 weeks and if they didn't like what they saw they could always terminate the contract.

Over the next 3 weeks the team set to work.  However the teams definition of work was a little unusual, they didn't go and lock themselves away and then turn up 3 weeks later with something to show, they were very visible, spent a lot of time asking people questions, clarifying what they were doing and getting stakeholders together to make sure they understood what needed to be done.Plus there were all these wall charts and sticky notes being stuck up on windows and walls so that anyone could see what was being done by the team and how they were tracking.  For the first time people in Company B could see what those mysterious developers were actually doing.

At the end of the third week of development the team got the stakeholders together and showed them what they had done.  All 4 items they took on were there and working.  No smoke, no mirrors. Plus the people from company B were allowed to play with that version of the software whenever they wanted.  Company B was quietly pleased.

The next week the vendors team took on 12 items to deliver in 3 weeks.  Now the CIO was thinking that they'd taken on too much.  But the same thing happened - lots of communication, lots of conversations and at the end of the next 3 weeks more product was delivered and things were coming together.  Company B was starting to take notice.

Shortly after this some of the Company B people were playing with the software and realised that with what they had already it would be a great idea and would really improve things if they could make a few changes to what they had planned.  They hadn't thought of them before, but now that they had seen the software and how it was shaping up it just seemed so logical.  The vendor was quite happy for the company to add requirements to their list, just as long as the list was re-prioritised and anything they didn't want any more was removed.

This pattern of work for another 4 iterations and at the end of the fourth iteration the CIO and the stakeholders looked at the product and realised that all of their main functionality was already in place.  There were still a lot of items on the requirements list, but they were low priority items and probably not worth the development cost, at least not for an initial release.

After 23 weeks the software was put into production.

As everything that was needed was already delivered Company B decided to terminate the contract and saved around 40% of their budget.  The money saved then went into further efforts to open up a lead in the emerging market giving the company even more of a competitive edge. As for the vendor they became the sole supplier of development services to Company B from that point on.

 

Company B was already up and running with their system and starting to make inroads into the new market before Company A even got the development work started on their project.  Over the next few years Company B continued to push hard providing their customers with regular updates to their offerings, adapting to the changing state of the market and managed to do so at a pace that no-one else could get close to matching.  When Company A finally made their venture into that market it was with a poorly conceived, me-too product that cost way more than budgeted and didn't work anywhere near as well as that of the established players.  Company A withdrew from that market not long after.

 

Before you ask - no, this isn't a story based on any specific companies.  But it is a story I'm sure you can relate to as this sort of thing happens all over the world every single day.  There are still far too many Company A's in this world and we should do all we can to help turn them into Company B's.

I'd love to hear your comments!

Dec 13, 2007

Agile Scope Creep and When is "Done" Done?

So, you decide to take this shiny new agile development methodology that everyone is talking about and use it on a new project you've just gotten sign off for.  You get commitments from your customer (I refer to either internal or external customers here) to have a high degree of involvement and you go and assemble a good team, many of whom have worked on agile projects before.  In terms of skill they can write unit tests, do just in time design, document and test with FitNesse, understand Continuous Integration, Subversion, Trac and everything else you want to use in your development process.

You and the team then sit with the customer for a few days and break out the project into various functional and non-functional requirements.  You come up with a list of items and some approximate sizing of the work involved.  You  group the functional requirements into themes to try to align with iterations and sprints.  Everything is discussed to enough detail to be useful for estimating the project and getting a rough plan of attack put together.

The customer is very aware that the estimates are just that - estimates, and that the project velocity won't really be known until 1 or 2 iterations have passed.  With this understanding you get the project moving and commence work.

Time passes, the first few iterations get completed, and a project velocity is determined.  From there an end date is projected and the customer realises that there's more work than budget so they drop some of the lower priority requirements.  The end result is a backlog with an estimated further 6 months to completion.  You're happy, the team is happy and the customer is happy.

In fact the customer is extremely happy.  They've never had this level of service or involvement before.  They're loving it so much that they regularly help the team in finding bugs and point out areas where they may not have explained things very well in terms of getting the functionality just right.  They happily accept that what was delivered is what they asked for, but now that they look at it they realise a few small changes can improve it no end.  You listen to what they say and decide that in the spirit of providing great customer service and wanting to give the customer the best software you can that you can take on their little corrections and alterations as bonus tasks for the next iteration.

Time passes and you reassess the velocity.  You notice that it has dropped by 20%.  That's weird.  The team hasn't changed and if anything seem to be getting more done now than they were before, and yet they're getting through fewer stories and delivering fewer requirements.  The customer isn't quite as happy as they were now that you've informed them of the drop in velocity and a resulting delay in the projects end date.  What went wrong?  Why the decline?

 

Well, as you may have guessed from the title of the post the problem comes from a misunderstanding of how to manage scope in an agile project.  In a waterfall project each of the little changes would have been written up as a change request, designed, documented, estimated, costed, signed off in both triplicate and in blood and then worked on an delivered as a second phase only after all the other agreed to work was completed.  That's because under waterfall models, an item is treated as done once the customer signs off on a spec.  In waterfall done doesn't mean working delivered software, but rather done means the design and scope is locked down and is as unchangeable as the mountains.  Because agile is all about being responsive and doing away with this kind of silliness it's a common mistake (especially for those just starting out with agile) to think that because we want to be responsive and collaborative that if the customer asks for a change, especially for a small one, we can just slip it in with the other tasks in the next iteration.

This is the agile version of "Death by a Thousand Cuts".

 

In agile, the definition of done is Done! meaning delivered working software requiring no other changes.  Not done meaning mostly done but for a few small things.  Not done meaning it kind of works but we'll have to come back later and clean it up.  Done means Done! Finished! Over! Wrapped Up! Complete! Tie a Bow Around It and Put It Under The Tree! Done!  The story/requirement/backlog item as we know it is done.  There will be no more work on it, there are no known bugs to fix and we do not have any other clean up tasks or refactorings to do for it.  We are DONE! :-)

Now, that doesn't mean the customer can't have their alterations, amendments and ideas as well.  It's just means that the way we handle those requests needs to change.  Instead of trying to jam them into the next iteration as extra tasks (not tied to a backlog item), they need to be treated as new requirements.  When the customer makes a suggestion, grab a pen and jot it down as a story or work item and add it to the product backlog so it can be estimated and prioritised.  Near the end of the iteration the team can provide a quick estimate for the new items and you can then go back to the customer and show them just how much effort is involved in delivering all of their extra suggestions.

The customer then has a choice.  Take on the extra work and expand the projects cost and time budgets or reprioritise the backlog and drop some existing requirements.  This way the fixed time and budget for the project can be respected, and scope creep doesn't get out of control.

It can be a great way to assist customers in managing themselves and helps massively in terms of managing project scope - especially if you have a customer that vacillates over what they really want and flitter back and forward between design ideas.  It may also help them see, possibly for the first time, just what sort of impact a lot of minor revisions can have on a project.

Nov 14, 2007

Personality Types and Agile Development

You've no doubt heard of the various types of personality profiles that exist (you can try a free one at http://www.humanmetrics.com/cgi-win/JTypes1.htm). The most common profile is the Briggs-Myers one though there is also one for a DPSA profile.

The following is from the Marker Consulting description of DPSA profiles

DPSA profiling is an acronym for DRIVER, PROMOTER, SUPPORTER and ADMINISTRATOR.
D - DRIVER: Decisive and Direct. Drivers want to take charge in order to succeed and win.
P - PROMOTER: Outgoing and Optimistic. Promoters want to influence others and inspire them to act.
S - SUPPORTER: Sympathetic and Accommodating. Supporters want to help others and solve conflicts.
A - ADMINISTRATOR: Precise and Reserved. Administrators want to do things right and pay attention to detail.

And if you don't know Briggs-Myers profile it tries to type all people using four classifications:

  • Extraversion - Introversion
  • Sensing - Intuition
  • Thinking - Feeling
  • Judging - Perceiving

The first criterion, Extraversion - Introversion defines the source and direction of energy expression for a person. An extravert has a source and direction of energy expression mainly in the external world while the introvert has a source of energy mainly in the internal world.

Sensing - iNntuition defines the method of information perception by a person. Sensing means that a person believes mainly information he or she receives directly from the external world. Intuition means that a person believes mainly information he or she receives from the internal or imaginative world.

Thinking - Feeling defines how the person processes information. Thinking means that a person makes a decision mainly through logic. Feeling means that, as a rule, he or she makes a decision based on emotion.

The fourth criterion, Judging - Perceiving defines how a person implements the information he or she has processed. Judging means that a person organizes all his life events and acts strictly according to his plans. Perceiving means that he or she is inclined to improvise and seek alternatives.

Personally I came out as strong Driver on the DPSA profile, and an INTJ (Introvert, Intuitive, Thinking, Judging) with a very strong Thinking component.  No real surprise for those who know me.

 

One of the interesting things is that if you look at a basic description of an INTJ you'll see the following:

"When they are in leadership roles, they are quite effective, because they are able to objectively see the reality of a situation, and are adaptable enough to change things which aren't working well."

Why is this interesting?  Simply put, Agile methodologies are all about adapting how things are done to the situation at hand. And this is exactly what an INTJ leans towards.

This then leads on to another question.  Does agile only work if people have the right personality type?

For me, as  a Driver with an INTJ personality, agile is right up my alley.  But what if you're an Administrator or Supporter, or you're the opposite to an INTJ?  Can you still succeed on an agile project?  Will you find it as painful as a visit to the Dentist or as tedious as waiting for a Qantas flight?

It's a tough question, but one worth asking.

If you flip the agile coin and think about Waterfall projects and then have look at the personality overviews you could think that the ISTJ personalities would seem ideally suited to Waterfall.  Fixed requirements delivered using a methodical, traditional approach.  So if you put these types of people in an agile project wouldn't it make them really uncomfortable having to deal with uncertainty day in and day out?  Could that result in team friction, poor morale or a host of other problems?

Does it mean that forming teams means you need to have only people of certain personality types?

It's a really interesting set of questions and something I've been thinking about for quite some time.

 

So, to apply some very non-scientific research to my theory I'd like to do a straw poll of any who's worked in an agile project:

What's your personality type (both DPSA and Briggs-Myers if possible)?  And how much did you/do you enjoy agile development?

Post your responses as a comment.  I'm really curious to see the results.

Oct 29, 2007

Are You A Programmer or a Developer?

Most people would say that a programmer is a developer and a developer is a programmer; and for those of us who write code for a job when asked to describe ourselves, we'd be comfortable using either term depending on the mood we're in.  Most people would think that there's no difference and even if there was I'd have to be nitpicking to find it, right?

Both terms describe someone that writes and maintains software. But if we take a moment and think about this, there's more to creating a software product than just applying technical training.  Why is it that some software is fantastic, some is OK and some is downright horrid?  Why is that some people can product fantastic code that's bug free, easy to maintain, a pleasure to use and exceeds expectations while other people, filling exactly the same role, can produce abominations that tear at the very fabric of space and time and should never, ever, under any circumstances see the light of day?

Is it intelligence?  Is it training? Is it the tools?  Is it management? Is it something else?

I'll scratch intelligence from the list right now.  I've seen people of average intelligence produce great software and I've seen really smart people produce unmitigated rubbish.

I'll also remove training and tools from the list.  Go into any workplace and you'll find people with the same training having wildly different results.  You'll also find those same people using the same toolset.  So let's scratch that as well.

Maybe it's management.  Maybe it's something else.

Here's some 5 characteristics I find strongly evident in people I would call programmers:

  1. Enjoy writing code
  2. Have good technical skills
  3. Turn a design into working software
  4. Very task focused
  5. Stay current with new developer technologies and enjoy using them

Now compare this with the characteristics strongly evident in people I would term developers:

  1. Want to deliver software they can be proud of
  2. Want to keep their customers happy
  3. Understand the non-technical factors of what they do
  4. Very goal focused
  5. Stay current with technologies

There's quite a difference between the two, and to me the difference can be summed up with two words: Perspective & Attitude.

A developer looks at things from the customers point of view and tries to come up with the best solution for the customer.  A programmer looks at things from a technical point of view and tries to come up with a solution that the programmer finds pleasing.  The developer is outward looking, while the programmer is inward looking.

I believe that having a customer based point of view makes a whole world of difference in producing software that meets the needs of the customer and ultimately it's pleasing the customer that ensures they use your software, and call it successful, and invite you back to write more.

What about Attitude?  I think that Developers on the whole take a much more pragmatic approach to problem solving.  They realise that there is no correct solution, that there are usually a multitude of ways to peel the potato, that the latest and greatest developer tools may not be suitable for the task at hand, that all development is inherently risky and that their one and only measure of true success is their ability to produce working software of high quality, delivered on time and enjoyed by their customers.

Programmers on the other hand seem less inclined to pragmatism.  The desire is more towards writing the perfect code, the ultimate algorithm, the most ideal technical solution using with leading edge tools and integrating with all the latest buzz-toys they know of.  If there's a shiny bauble to be added, add it.  If you talk loud enough and passionately enough, you can convince the customer they need it and hang the risk and hang the cost.  And if this thing doesn't ship on time then it's the project managers fault for letting the customer get away with so much scope creep (I hope you see the irony there).

I believe that the days of the programmer are numbered.  Companies want to hire people who create great working software not people who think each project is another excuse for learning new stuff and that never deliver on time.  Customers want to buy software that works as expected and meets or exceed their needs.  They don't want technical wizardry, stuff they don't need and they definitely don't want to be a guinea pig for some in-disguise R&D program.

When I hire people I look for the Developers not the Programmers. When I work with people, I choose to work with the Developers not the Programmers.  And when I need advice I ask the Developers not the Programmers.

 

If you write software you're either a Developer or a Programmer.  Which one are you?

 

Comments welcome.

Oct 9, 2007

Scrum Vs Kanban

Agile project management has certainly come a long way in the last few years and there is strong evidence of a rapidly growing maturity in the space and a commensurate awareness of agile in the maintsream consciousness.  As an example a few years ago trying to hire someone with agile experience would have been a very difficult process, like finding the proverbial needle in a haystack.  Most candidates would have stared at you blankly and wondered what the hell you were talking about.

Fast forward to today and you'll find that the haystack is rapidly getting smaller and many people will now at least acknowledge that they've heard of agile, even if they haven't actually experienced it.

With the growing body of knowledge regarding Agile methodologies there comes an understanding that there are many ways to solve the same problem and a rise in different ways of "being agile".  A question popped up on one of the internal Readify discussion lists recently around the suitability of Kanban for internal development.

For those who don't know, Kanban is an agile methodology created by David Anderson to run his development projects.  David knows a lot about agile development and is widely regarded as a thought leader in this area.  Here's an extract from http://www.agilemanagement.net/Articles/Weblog/KanbaninAction.html (and other writings of his)

“The Kanban system might be visualised as a "Three bin system" - one bin on the factory floor, one bin in the factory store and one bin at the Suppliers' store."

The idea is that there is pool of resources and you set a "kanban limit" which defines how much work-in-progress you're prepared to commit to.

Then as one "kanban" (new feature, bug, change request) is shipped, another "kanban" becomes available. So you go back to the business and ask them which feature/bug/change they want done next.

"The kanban system allows us to deliver on 3 elements of [David's] recipe for success:

  • reduce work-in-progress (in fact it limits it completely);
  • balance capacity against demand (as new CRs can only be introduced when a kanban card frees up after a release);
  • and prioritize.

We hold a business prioritization meeting once per week with vice presidents from around the company. They get to pick new CRs from the backlog to allocate against free kanban cards. This forces them to think about the one, two, or three most important things for them to get done now. It forces prioritization."

 

David's thinking is largely based around the principles of Lean manufacturing and the Theory of Constraints.  In seeking to improve efficiencies in the software "manufacturing process" you have to think about where the constraints are and where the bottlenecks occur, and this is what David has done (and quite well).

The kanban system realises the constraints are squarely within the realm of the development team.  This is almost always the case in any business.  The demand for new software by customers (internal or external) almost always outstrips the ability of the development team to supply that software.  If that isn't the case, then it soon will be as the business will start downsizing development staff pretty quickly (developers are an expensive resource after all).

But what kanban also does is take a subtly different view to the development process itself.  Kanban treats software requests as a stream or work. You've dealt with one request?  Great! What's next in the stream?  It never ends, just keep the work flowing.

Scrum, and all the other agile iteration based methodologies, differ in that they segment the "requirement stream" into chunks of work and process those chunks in a batched manner.  The customer selects what items they want done at the start of the iteration and 2 weeks later the team (hopefully) comes back with the completed work.

 

So the question now Is which approach is better?  Well, as in all things, you need to think about what is best for your team, your customers and your business.  I personally think Kanban can work really well in software maintenance environments where work is coming in as a stream of small amendments and changes, while Scrum may be better suited to new work where delivering lots of thin vertical slices of functionality is more appropriate for customers.  But, hey - we're talking about agile methods here.  Think about what's happening, inspect what is working and what isn't and adapt what you do so you can do it better :-)

Aug 31, 2007

Would You Take on This Project?

clock

A good (and very large) customer calls you up:

"I've got a really exciting project for you!  This is going to be huge for us and something we will talk up a lot in the media.  The exposure for you guys will be world class! What we need developed is SuperWidgetYYY. I'll explain it in detail to you later but it's really cool, and not that complex.  We haven't got it completely figured out yet but we're mostly there, so we'll be relying on your genius for a little advice in rounding off some areas."

"I have a fixed budget of XXX and I need it completed in the next 5 months because of the strategic and competitive advantage it represents for us.  By the way, if you can't do it, let me know now so I can take it to one of your competitors.  Do you think you can you do it for me?"

After a little more discussion, the basic requirement becomes apparent and it doesn't appear to be that hard.  Yes, there's a lot of work involved and a few unknowns and it will be a bit of a push to get it done on time, but you think that you should be able to deliver successfully if there are no major issues, you can get started right away and assuming that the customer is responsive and available.  You figure you'll probably have to use some of the latest technologies and while budget might be a little tight you think that, yeah, it's do-able.

Now put on your business hat.

Would you take on this project or not?

I'd love to hear some thoughts one way or the other.

Jun 20, 2007

Outsourced Agile

Andrew has posted about a guy named Alex in Romania having a big rant about using agile in an outsourcing arrangement.

His argument is founded on everything that can go wrong with an agile project if you miss the fundamentals of agile and forget to "inspect & adapt" your process to fit the situation.  And while he doesn't outright say it, from what I can tell he's tried to use XP (extreme programming) for his engagements.  That in itself is a tough ask - XP is a very demanding methodology to follow (why do you think it's called extreme) and is really intended for internal short term projects, not outsourced arrangements where slightly more rigorous processes are more appropriate.

And in case you're wondering I've run agile projects for large external customers before, so I know a thing or two.

Let's take a look at some of his arguments:

1. The strong points of agile development don't turn out, in practice, to be that strong.  Less paperwork isn't a good thing.

Who said an agile project shouldn't have documentation?  Who said you can't document anything? And, um, are you nuts!! You can document requirements as you go - use cases, user stories, acceptance criteria, signed off UI mock ups, all sorts of things.  Documentation in an agile project should be "just enough documentation" not "no documentation".

2. Stress and Unproductivity

Apparently it's stressful and unproductive to communicate with your customers and tell them what you're doing.  I think someone needs to think seriously about what business they're in.  Outsourced development is a service, and implies that customer communication is important.  Agile methodologies stress the importance of communication.  I don't see how this is a negative.  If anything it's a positive as proper communication will result in improved outcomes for the client.

3. The First Meet-Up.  (You can't estimating the size of the job)

Apparently in an agile project you're not allowed to estimate.  What a crock!  I'd like to point you to Mike Cohn's book on Agile Estimating and Planning.  Agile says "don't lock down your estimates in stone", not "don't estimate".  Every project needs estimates, plans and priorities.  So make an estimate, and revisit that estimate as more information becomes available.

And how does the ability to do an initial up front estimate get impacted by the methodology you use to deliver it?  If a customer wants to know how long the piece of string is, it won't make any difference if you use waterfall or agile, it's still an estimate.  Waterfall just deludes the customer into thinking that the estimate is accurate, all the way up to the missed delivery date!

4. What you deliver has changed from what was asked for and now they're upset

The project draws close to conclusion and the client begins to realise that the delivered solution will be different to what was asked for at the beginning and that this is a surprise to them, and they'll make you redo large parts of it.  HELLO!!  If there's been any communication going on then the variations that have been made will be based on what the customer asked for and put in a prioritised list.  Oh, hang on, he didn't do any documentation and he doesn't like talking to the customer.  Hmm, no wonder things went pear shaped.

The statement that the "In the classic approach, it is impossible for both you and the client to be anything but happy" is just downright ludicrous!  A delivered to spec project does not equal customer happiness.  And while a spec may be "frozen" it's a guarantee that requirements will change.

5. Daily progress and showing the customer each day what has happened is bad.

I agree, so why do it?  Why not show the customer at the end of an iteration.  Every 2 - 4 weeks, and show how you've delivered on commitments (and possibly gotten ahead a little).  The customer doesn't need (or want) to sit shoulder to shoulder with your team 8x5, but they do want to know what's going on.

6 Daily change requests. (when the client changes their mind every day).

Hmmm.  And here I was thinking that in an agile project an iteration had a fixed scope, where commitments are made and delivered.  I think someone missed the point about planing in an agile project.  Change priorities, change scope, change as much as you like, but none of it takes effect until the next iteration.

And the statement that "With the classic approach, change requests like this are impossible, since the specifications are frozen." goes against point 4 above - does anyone really think that a customer with a plethora of unactioned change requests will still be happy when you deliver to the original spec?

7. Clients know programming and will tell you how to do your job.

Is he serious?! You're responsible for your work, the customer is responsible for the specs.  If you've got a customer who thinks they can manage your staff for you, then it won't matter how you run your project, it will fail.

8. Clients who want to run the show.

Why is a client getting involved in the programming?  I just can't fathom it.  The involvement of the customer in an agile project is for the clarification of requirements, business rules, etc.  NOT for assistance with the code.  Honestly, what is going on here?

 

All Alex's rant does is highlight the mistakes he's made.  It's another great article on what NOT to do when adopting agile practices, especially in a client/vendor arrangement.

May 1, 2007

Using LinkedIn for Hiring

Guy Kawasaki has recently posted about using LinkedIn for vetting candidates and tied it to "The No A$$hole Rule" book. The result is a fairly interesting read, especially as I'm a boss.

How would I answer the list of items I wondered? Here's my self assessment, out in public, for the whole word to see. If you know me feel free to comment, though of course I'll probably abuse you and call you names :-)


Q1: Kisses-up and kicks-down: “How does the prospective boss respond to feedback from people higher in rank and lower in rank?”
A1: Fairly well. I usually seek feedback from both my staff and boss. From the staff during reviews, from my boss at semi-regular intervals.

Q2: Can’t take it: “Does the prospective boss accept criticism or blame when the going gets tough?”
A2: Yep, I'll put my hand up when I make a mistake. Here's a public example.

Q3: Short fuse: “In what situations have you seen the prospective boss lose his temper?”
A3: Guilty on this one, though improving (slowly). I've lost my temper 3 times in the last 2 years, that I recall, and it's always been caused by a slow buildup of frustration that eventually erupts. I'm neither happy or proud about it.

Q4: Bad credit: “Which style best describes the prospective boss: gives out gratuitous credit, assigns credit where credit is due, or believes everyone should be their own champion?”
A4: Credit where it's due.

Q5: Canker sore: “What do past collaborators say about working with the prospective boss?”
A5: Nice things I hope - leave a comment. The fact that some people have followed me from previous organisations is probably a good sign.

Q6: Flamer: What kind of email sender is the prospective boss?
A6: Flaming is rare - email is too easy to misinterpret so I try to keep things emotionless.

Q7: Downer: “What types of people find it difficult to work with the prospective boss? What type of people seem to work very well with the prospective boss?”
A7: It's hard to work with me if you don't think for yourself or show little initiative. I like people to be results driven, good communicators, ego-less and open with others.

Q8: Card shark: “Does the prospective boss share information for everyone’s benefit?”
A8: As much as I can. If things are in flux or sensitive in nature I'll withhold details, but once things settle and are clear I'll communicate what I can.

Q9: Army of one: “Would people pick the prospective boss for their team?”
A9: I hope so - leave a comment.

Q10: Open architecture: “How would the prospective boss respond if a copy of the book appeared on her desk?”
A10: I'll always accept freebies :-) Anyone got a copy they want to give me?

Mar 10, 2007

Changing a development culture

Many people can tell when a development culture isn't healthy. Much like being able to tell that the wind is blowing they can see that things aren't right by looking at the evidence around them. High development costs, substandard quality, missed deadlines, code not matching design, business priorities not being met, no responsibility taken by those doing the work, and so on.

Sometimes people think that a better process will, on it's own, cause all these things to improve. As if the process is the problem, and not the people, as if people always follow process to the letter, and as if a process always works in every situation.

The reality is that the right process is important, but that the people using the process are more important. The number one principle on the Agile Manifesto is "Individuals and Interactions over Processes and Tools". Jim Collins in Good to Great talks about this as "getting the right people on the bus".

People are your critical factor to success. So, whether you use CMMI, Scrum, XP, RUP, or any other development methodology, if the people have are poor the results will inevitably be poor. And worse, if the culture is poor, the good people you may have will be unlikely to stay.

So, how do you change and improve on a bad culture? How do you break the habits and attitudes that permeate everything that is done? Is it through charismatic leadership? Through slavish adherence to processes? Is it through ranting and raving at people until your throat is raw?

The answer is none of the above. Cultural change only happens slowly and gradually, and is really a reflection of how the majority of people in your team think and act. The hard part is, that in poor cultures, there is a reinforcement of mediocrity, of striving for the lowest common denominator, for seeking the path of least resistance, for not sticking your neck out and suggesting improvements. It's easier to complain to colleagues about how the culture sucks, than it is to actually try to do anything about it. And a bad culture is very unattractive to good people and those who look for the best, which just makes it harder to turn around.

The good news is that it's not impossible - you just need to have direction, endurance and determination. And you need to be able to motivate your people to want to change.

In other words - you need to use the carrot and the stick.

The carrot is the better method of course. Inspire people to improve through communicating what you expect. Constantly. Every day if you need to. Set your expectations and set them clearly. Do not be afraid to them high and always be prepared to measure every decision and action you make against those expectations. And ensure you meet those expectations yourself, all the time.

Constant communication is critical. People who are constantly told something will eventually believe it. History has countless examples of cultural change occurring through various forms of propaganda (both good and bad). And history shows that people who are constantly told how to behave will eventually conform to that behaviour.

You should also try to get your people passionate about what they do. In this you will have to lead by example. Show yourself getting passionate about what you do, about successes your people have, about improvements you can use as examples. Passion and excitement is infective and others will catch your passion.

And most importantly reward those who change. Those who meet your goals for change should always be rewarded - publicly or privately. It doesn't matter. And rewards don't have to be material - praise and recognition is just as good a reward for many people as a gift.

The stick on the other hand should be reserved for those who refuse to change. Do not hesitate to show displeasure at failure or unmet expectations. Do not hesitate to remove people who do not try to improve. And most importantly do not hesitate to remove the ring leaders seeking to maintain the status quo.

Some people just don't like change. They like their negative cultures, they enjoy highlighting failings in others without admitting their own, and they will actively seek to pull down any initiatives that might improve things. These people are like a cancer and must be cut out. Do not feel guilty and do not feel remorse. Even if they are your most skilled people.

From experience, removing a negative person from a group releases the silent majority from the fear of abuse by that person and gives them the permission they need to improve. It also shows you are serious about change and willing to do what it takes to see that change happen.

Obviously, you need to tailor whatever you do to your own situation, and nothing ever happens overnight, but keep at these things and improvements will occur.

It can be a long and bumpy ride at times. Good luck on your journey.