The Rise and Fall of Waterfall Development

The Waterfall Development model is something we all know and loathe.  Thankfully in recent years we’re seeing the industry slowly turning away from this model and moving back to the iterative development models used in the 1950’s, before the rise of waterfall.   What’s ironic about waterfall development is that it’s creator (Winston Royce) never actually intended it to be a model for software development!

Royce says this in the paper where the model is shown: “I believe in this concept, but the implementation described above is risky and invites failure” and he explicitly mentions that changes can cause anything up to a 100% overrun since it may involve going right back to the drawing board.  He then goes on to evolve that model and introduce safeguards to minimise the risks of change.

So how did this initial model turn into the “One True Way” of software development?

First consider that Royce’s paper was published in an IEEE publication – IEEE is a well respected industry body and anything they publish is going to immediately have substantial respect.  But now consider the background of the author.  To quote again: “I have had various assignments during the past nine years, mostly concerned with the development of software packages for spacecraft mission planning, commanding and post-flight analysis.”  He was working at NASA in the hey-day of the space industry!  The whole world was looking at everything these guys did and reading it as gospel!  After all, these were the people who managed to put man on the moon and bring him back again using nothing more than a tin can, some electronics, a couple of goldfish bowls sewn into to some silver jumpsuits and one freakin’ great big rocket!  Of course people were going to listen to what he said!

Tarmo Toikannen said this about what happened next:

“

If you look at the scientific articles on software engineering that discuss the waterfall, they all cite Royce’s article. In other words, they’re saying something like “The waterfall is a proven method (Royce, 1970).” So they base their claims on an article that actually says the opposite: that the model does not work.

This is how science (unfortunately) often works - researchers just cite something, because everyone else does so as well, and don’t really read the publications that they refer to. So eventually an often cited claim becomes “fact”.

“

But why did so many people reference the broken model?  You might laugh at this, but it’s probably because the model shown was the first diagram in the article.  Instead of looking at the other diagrams to see how the broken model is then evolved into the model that Royce finally proposes they just copied the first image they saw and went from there.

mmmOkay.  But that still doesn’t explain everything.  Why did a model that those in the real world knew didn’t work gain so much traction?  Well, you can thank the US Military for that one – in fact you can specifically thank the geniuses who wrote DOD Standard 2167 – a standard which was not only waterfall based but also heavily documentation focussed.  As soon as the military decided all software should be written to that standard, it became the de-facto for all other large scale development.  You can imaging the thinking “If it’s what the military uses to get high quality software then shouldn’t we do the same?”.  The problem was further compounded when other countries followed the Don’t Reinvent The Wheel principle and adopted the existing US standard for themselves.  Now it was a global standard.

Thankfully the US Military amended their standard in 1995 and multiple times thereafter but by then the damage had been done.  Waterfall was the only thing being taught by academia and the industry was too earning much money by doing things poorly to quickly change to something that made sense.

Only now are we seeing a strong resurgence of the working, iterative, incremental development model of the 1950’s – and it’s got a name now: Agile.

 

P.S. If you want to read something a little more in-depth on this subject have a read of Iterative and Incremental Development: A Brief History (2003).

 

References:

http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf

http://tarmo.fi/blog/2005/09/09/dont-draw-diagrams-of-wrong-practices-or-why-people-still-believe-in-the-waterfall-model/

http://www2.umassd.edu/SWPI/xp/articles/r6047.pdf

http://en.wikipedia.org/wiki/Waterfall_model

http://www.product-lifecycle-management.com/download/DOD-STD-2167A.pdf

http://www.stsc.hill.af.mil/crosstalk/1995/04/Changes.asp

Software is a Service Industry

Ask most developers what they do for their job and they'll tell you that that they write software.  Well, technically that may be the major function performed in the execution of their job, but a developer's goal should not be to write software, but to write software that meets someone's a needs or wants.  To give you some examples:

• If you write business software, you're trying to help people perform their jobs better.
• If you write games then you're trying to meet someone's desire to be entertained.
• If you write security software you're trying to help people keep their information secure.

I think you get the idea.  It's might seem obvious now that it's said,  but it's not something people normally think about.  When spending the major part of your working within Visual Studio then it's very easy to focus on the low level details and forget the high level goals of what it is you're doing.

 

Now if we extend the idea that we are trying to help people through the software we create then we realise that we are trying to service their needs. Any industry that exists to satisfy an individuals needs and wants is a service industry.  What this implies is that the I.T. industry is not a manufacturing industry or a retail industry but rather is a service industry (in fact if you dig to the bottom of any industry you'll see that everything is a service industry).

We have jobs as developers because we are servicing the needs of our customers, and the better we service their needs the more work we get.

So what does this mean?  It means that the customer should be Number One, not us.  We need to remain focused on delivering top quality software that satisfies our customers.  The process of writing code is just one small part of our service and, believe it or not, very few customers care about the internals of our code.

What they want is software that is fit to purpose, easy to use and well supported... with no bugs.  These are the critical elements we need to focus on with software development.  Everything else we do exists only so we can to deliver a service to our customers in a more timely manner.

 

So, the next time you sit down to write some code, think about who you are writing it for and do your best to meet their needs.

Want to write a tech book?

I've been talking recently with Philip Beadle about the book DotNetNuke for Dummies that he co-authored and some of the effort involved in doing it. I was then wandering through the local bookshop shortly afterwards and saw 4 copies of it on the shelves and thought to myself "That's pretty cool".

But is it really worth it to write a book - and how much payback is there in doing it? Here's a somewhat sobering blog post by Scott Mitchell about the realities of authoring a tech book.

It's really worth reading; not as a deterrent but as a reality check. If you want to write a tech book - go for it, just be prepared for it to be a labour of love, not a path to early retirement.

Nintendo's Recovery

Thanks to 800-CEO-READ I just read Business 2.0 article about Nintendo's recovery and their return to completely dominate sales for the current generation of consoles.

It's a very interesting read in terms of strategy, marketing, learning from your mistakes and being willing to take big risks. After all if the Wii failed then Nintendo might well have pulled out of consoles and just concentrated on hand-held gaming. And when you consider that the DS was a huge risk for them as well then they've made 2 bet-the-business gambles in the last 5 years.

Anyone who's read Built to Last will recognize that Nintendo has exhibited many of the traits of visionary companies.

Nintendo has stuck with their core ideology (interactive entertainment), they focus on customers over profits (gameplay comes first), they've made huge gambles (DS & Wii), they've kept leadership promotions internal (Hiroshi Yamauchi - 1950, Satoru Iwata - 2002), were willing to change with the times, and they've tried a bunch of things and kept what works ("game & watch" was a success; gamecube was a failure).

In fact Nintendo corporate history shows that big gambles, trying stuff, etc are a part of their corporate culture, and for a company that was founded in 1889, I suppose it was only a matter of time before they bounced back.

Why Contractors Get A Bad Name

Ok, I know you should never post angry, but I'll run the risk in this case.

Because of the shortage of skilled .NET developers I often have to turn to contractors to get quality people in the door in the timeframe I need.  In the past I've picked up some top notch quality people.  People I am proud to have worked with (or am currently working with), who are integral parts of their teams, who work as much for the team as for themselves and who are genuinely missed when they go.

Then there's the bad eggs.  You hear stories of those who misrepresent themselves in interviews, those who stop putting in after a few weeks elapse, or - like the one I had today - those who commit to a 6 month contract and resign 2 weeks into it because a job they've been holding out for was offered to them.  Being used by someone as a short term cash source while they wait to see if the job they really want is offered to them is an unethical and dishonest way to behave, and hearing excuses about how they "don't normally do this sort of thing" doesn't make their behaviour any more palatable or acceptable.

I really value and appreciate the contractors I've met who are genuine and open about their situations.  This other type of contractor just gives everyone else in the industry a bad name.

Create a Web 2.0 Site

The current dot com boom is focused around Web 2.0 sites. The definition of what a Web 2.0 site is changes depending on who you're talking to, what phase the moon is in and how much interest Google and Yahoo! are paying to it.

The good news is that even if you don't know what one is, you can still create one, sell your hair-brained idea to a venture capitalist who just looks at the visuals and have a big bag of cash come your way! How? By simply following the instructions of course!

And if you just want a visual refresh to your web site so you can look like the rest of the cool kids then these might actually be useful instructions. Enjoy!

Yahoo buys MyBlogLog

OK, so it's probably not the size of the YouTube purchase, but Yahoo has shelled out for MyBlogLog. I posted previously about MyBlogLog as a stats and link tracking system, however one aspect I didn't really dwell on is the social side of it - something which most stats systems don't have.

It appears that this social aspect is the main reason Yahoo purchased them - they want to get a bigger slice of the social pie and this is a different angle on it. I expect that MyBlogLog will change a bit to incorporate other Yahoo social networking tools and that we'll see Yahoo ad's appearing somewhere soon as well. It'll be interesting to see how it pans out.

Congratulations to the MyBlogLog team.

Update: There's an official blog entry about the purchase and what is going to happen. Looks like a lot of improvements will be coming soon.

iWoz

I've just finished reading iWoz - the Steve Wozniak biography. It's a pretty quick and easy read (took me 2 days) detailing how Steve designed the Apple I & Apple II, pulled pranks, organised music festivals and got into teaching.

The book was fairly interesting for me and I did get a bit out of it, but since I've read Accidental Empires previously and seen Triumph Of The Nerds I knew most of the background and apart from adding a bit more detail to what Steve did in the early days and post-apple there wasn't a lot of new history for me. There was however a lot of stuff I learnt about this personality and how he thinks and what makes him tick.

I've read a lot of resumes by I.T. candidates and one thing that comes through time and again is that they talk a lot about their skills and the projects that they've been a part of and have a passion for it, but they never seem to really grasp the details of what else is going on around them, how you can influence a business with your ideas and how the interpersonal dealings are a large and crucial part of what has to happen.

Woz shows a similar type of behaviour in hos bio. He talks about the design of the hardware and the software to run it (and does an awesome job at it too) but it's apparent that in life he's missed a lot of what is happening in terms of business and relationships. For example, his departure from Apple shows this. He was working as an engineer doing what he loves best but wanted to start another company so he tells his manager (2 levels up) that he's leaving but doesn't even think to tell Steve Jobs that he's going. So when a reporter rings the next day and asks why, even then he doesn't think to tell the Apple execs that he's left or that he's just been interviewed. The reporter misquotes (probably deliberately) and there's a big hoo-ha about Woz leaving because he doesn't like Apple. Woz blames the reporter, but doesn't see that the communication about his departure could have been made a lot smoother and not damaged his relationship with Jobs if he'd done a better job on the interpersonal side of things.

In fact, he shows a definite naivety in almost all of his personal dealings and talks about a few of them in the book. It's obvious that money is not a motivator for him - but you shouldn't let yourself get taken advantage of and throughout the book this is one of the things you see repeated.

Is he a great engineer? Undoubtedly! One of the best the we've seen. Is he a well rounded person who is successful outside of engineering? I don't think so. Is it a book worth reading? Absolutely!

Sony's PS3 Plan (or "How to Ruin a Brand")

The SMH has an article about Sony and their late entry to the Australian market.  It's quite an interesting article in that Sony get all hot about their Blu-Ray compatibility.

After reading it though I had to wonder what Sony was really up to with the PS3.

Here's a few comments that made me wonder:

"The Wii-type functionality is more just a sub-element of what the PlayStation 3 offers"

"Wii is a core gaming device. It's a more fun, intuitive sort of product to pick up"

So I guess that the PS3 is not a "core gaming device" anymore.  Then what on earth is it meant to be - it's not a home theatre system is it?  or a HD Set top box?  or a TiVO? At the price it might as well be, but try this quote:

"You have a whole suite of other applications ... such as Blu-ray media playback, the ability to access your music, access your photos and the interoperability with the PlayStation Portable"

Oh, gee, thanks - that makes it so obvious - it's a camera/mp3/blu-ray/PSP Docking station thinga-me-bob.  I assume we can still play games with it, right?

Or how about this little quote:

"Gamers are extremely aware of what they're after and what they want"

Yep - and they want something to play games on, and NOW!  Oh, and they want it for less than the price of a new PC too!  Why would I shell out $830+ for a PS3 when I can buy a new PC from Dell for the same price.  And guess what - a PC plays music and lets me access photo's too, plus it has "a whole suite of other applications" as well.

I'm sorry Sony, but what are you really trying to do?  I suspect you're trying to use the power of the PlayStation brand to make Blu-Ray the next gen DVD platform so that you can make a squillion dollars from all those new Blu-Ray players people will buy and from the licensing costs of the technology.  It's not like DVD is a problem - I'm sure there are absolute millions of people picketing Sony offices crying out for an answer to their viewing woes because of all the pixellation and colour washout they have when watching movies.  And I'm sure that Blu-Ray is the answer for the man on the street, right?  ... No?!  I thought not.

So the PS3 really isn't about competition with core gaming devices like the XBox 360 or the Wii - it's about using a brand that has dominated the gaming industry as a lever into trying to dominate the home entertainment market, and to try and rort their customers along the way.

At least Microsoft show some thought for their customers by giving them choice.  Xbox with HD DVD or not? Then there's skinning options, accessories, add-ons.  The customer has choice and control.  Not so with Sony.

But then again Sony is also a major music label and a strong member of the RIAA right?  The same guys that sued 15 year old girls for downloading a handful of songs from the Internet a few years back.  I'm sorry for being a little cynical about their motives, but their track record doesn't help.

And what does the consumer think of all this?  Japan pre-sold 400,000 Wii's and sold 600,000 in the first 8 days in the US, Sony has sold 154,000 PS3's in Japan and about 200,000 in the US.  The XBox 360, which has had several months head start has sold over 6,000,000.

I think the customer knows what they want, and I think Sony knows how to kill a brand.

GooTube lawsuit reserves

OK, so maybe Google did think the whole lawsuit thing through when they bought YouTube. According to the SMH web site they've set aside $200M for settling lawsuits.

That's some serious cash! And as a honeypot I'm sure it'll attract the lawyers in swarms. I suppose the thinking is that you get all the lawsuits out of the way now, tighten up the copyright controls on YouTube and then get on with life.

I still can't figure out where the return for them is, though the article talks about YouTube having a commercial component to it. Maybe that'll work...

YouTube Sued

I've posted previously about GooTube and the litigation potential. We'll it's apparently started - but this is kinda left field and not at all what I expected.

YouTube is not being sued for copyright infringement, but because their name is so similar to a manufacturing company's name and the company has had to keep moving their web site because it gets flooded by people who can't tell the difference between YouTube and UTube.

Still, I bet this wouldn't have happened if Google didn't buy out YouTube. Given that UTube has already moved their web site a number of times previously, ask yourself what's changed this time to make them get all litigious?

New SSL Certificate Types on their way

As most of you would know SSL is a method for encrypting data in transmission between sites and SSL certificates are used to prove to people that your web site is who you say it is. Unfortunately people assume that data transferred to/from an SSL site (you know - the ones using https) is not only secure, but that the site can also be trusted.

This second assumption is incorrect, however new versions of SSL certificates are being readied that will change this. Adding this verification will help bring an end to phishing, and that's a good thing.

If you're interested have a read of this blog entry for more information.

GooTube Update

I wrote about GooTube (the term coined for the Google & YouTube merger) a little while back and how I didn't really "get it" from a corporate perspective.

The Sydney Morning Herald had an article just recently about the same subject, but from a different perspective that I hadn't considered. Basically they say that Google's attitude is that it's easier to ask for forgiveness than permission. In other words, do it and let the lawyers sort it out.

Apparently the market seems to agree with this attitude as Google shares rose over the last month (when rumours started surfacing) from about US$400 to around the US$480 mark (a 20% increase!).

Kind of scary but I don't suppose there's any other way to push the boundaries is there? Then again I don't see Apple doing the same thing (at least not as blatantly), or Oracle, or IBM, or Netscape, or Sun, or Nintendo, or Sony, or a bunch of others. The only one I can think of who acts like that is Microsoft, and they seem to be improving their attitude and behaviour over time.

Microsoft behaved/s the way they do because they're a monopoly. A monopoly of the desktop and the office suite. I sure hope Google isn't starting to behave the way they seem to be because they're a monopoly of search and internet advertising.

Like all big companies there's plenty to like, but also plenty to improve on. On the GooTube thing - for now I'll hold my judgement.

IE 7 is out

Yes, it's true! You can get it from Microsoft's IE page. Have fun :-)

P.S. It's the English only release for now.

Looks like Firefox 2 isn't far away either. It's at RC3 and not far off it's RTM release.

Google and YouTube silliness

I don't normally make comments on general happenings in the industry, especially negative ones, but I can't help myself on this.

OK, so everyone knows that Google went and paid more than the GDP of some small nations for YouTube. The question I'm still asking myself some days later is why?

I mean, seriously, where's the payback? How on Earth are these guys going to recover that much money in 2 years or less. Why 2 years - because that'll be about how long Gen-Y's attention span lasts before it switches onto some other instant fame fad. And it'll be about that long before the lawsuits start getting decided.

I can imagine that right now the copyright holders for music videos, ads, and movies are all going through YouTube looking for their works, and getting their lawyers amped up and ready to sue. Previosly YouTube had little money and weren't a real lawsuit target but now that they're Googled they'll be rolling in it. They're a rich target, and rich targets end up getting sued.

So why would you buy YouTube? I just can't seem to get my head around it. Surely it's not so that they're "cool", or to stop someone else buying them.

More and more Google are moving away from their core business of search (and advertising) and getting into other areas where they don't have core strengths. They're self proclaimed air of superiority, uber-coolness and the "we're smarter than the rest of you muggles" attitude (as so eloquently expressed by the walking rant machine and flame-bait Steve Yegge) is starting to get tired and wear a bit thin. It's a slippery slope Google are walking on it's likely to end with a fall.

If I had Google stock I'd be thinking about cashing in now while the going's good.


Update: A post on J.LeRoys blog covers similar territory, but looking at Yahoo losing their identity instead of Google.