7 comments | 
Software development projects typically run overtime.
One of the most puzzling things in high technology for business executives is the software development process. It’s the “Black Hole” of the industry.
Endless resources are often poured into a software development project, yet there never seems to be an end in sight. Monitoring a software project can be like peering into the darkness of a bottomless pit.
Why is this so? It seems that we would have long ago figured it such a mainstream activity. Today’s PCs have the same power of supercomputers a few years back; are slapped together like bicycles; and don’t cost much more. One would think that the software development process would be like turning a crank. But it seems to not have advanced much since the dawn of the PC age.
I have been active in technology and software product industries since 1983, and I have never been involved with or even know of a software project that came in exactly on time and under budget. Not once.
That’s really incredible. Of course, there are examples of on-schedule projects out there, but they fall into the overwhelming minority of all software developed.
Software always slips
It’s pretty much accepted in the software business that projects will slip, especially in the case of an actual commercial software product. The businesses I’ve been involved in have tried everything.
We’ve taken every approach imaginable when I’ve had direct responsibility. There’ve been approaches such as “No upfront planning” — by starting coding as soon as possible. On the other end of the spectrum has been “extensive upfront planning” — with a detailed spec and a prototype completed prior to initiating production coding.
We’ve used intermediate steps falling between the two extreme approaches above. We’ve started projects by purchasing as many “pre-written” modules as possible, tried many different languages and platforms, hired dedicated debugging personnel, used code-generators, and assembled both small teams & large teams. You name it-we’ve tried it.
Project schedules have been written with the utmost conservatism at senior management insistence. No matter. Across a number of companies, every project has slipped despite the best efforts of all involved.
Two-week delay for one line of code
Once, I asked our lead programmer to change ONE LINE OF CODE in a well-established product. He estimated it would take just a few seconds to make the change, and a few hours to test it. The change would be final by the end of the day at the latest. Two weeks later, I was still waiting for a solid product.
I’m not writing this to bash software developers. While not every developer I’ve worked with has been outstanding, I’ve had the fortune to work with quite a number whom I consider to be exceptional.
But no matter how much thought, time and effort went into it, our projects always slipped — often by a lot. We usually ended up with a commercially successful product (after all, the competition was slipping too!) But how much successful would we have been bringing the product to market on time?
Art not science
I believe this is so because writing software is still much more of an art than a science. This is surprising until you look a little deeper. There is certainly a plethora of methodology available to guide a team to use sound, time-tested practices in developing software.
However, a software program is really akin to a document written in a foreign language. That’s why developers use Programming “Languages.” So just like in writing a novel you are only guided by syntax, grammar and writing rules, writing a software program is very similar.
In writing a novel, you are creating a unique work that has never been done quite the same way before. This is also true for a software program.
If you knew exactly how to write a novel or software program before you began, there would be no need to write it — it would have already been done. While there are plenty of rules (the science) to writing good software, at the end of the day it’s a unique creation (the art).
Complexity trumps all
Another reason why conquering software development appears daunting is the vastly increasing complexity associated with software projects today.
The average piece of software today does a lot more and is quite a bit larger than at the dawn of the PC era. The era of graphical user interfaces really started the explosion in software code size.
Much more code needs to be written to produce the user-friendly products of today to life. What enabled this was the dawn of the modern operating systems, with breakthroughs like the overcoming of the 640K limit in the original PC DOS operating system.
Modern operating systems, large hard disks and increased memory almost eliminated the need to write software efficiently from a code size perspective. The embedded systems world is the last bastion where writing size-efficient code lives on-it’s a lost art to most of the software world.
If we were still writing in the 640K box today, maybe software development would have evolved to a more predictable science. But the world would be a less productive as well.
What’s a business manager to do?
As you’ve probably gathered, I don’t have a great set of answers on how to bring software to market on time. It’s been one of the great frustrations of my career. I strongly believe that getting the best people you can get will improve your odds of success. I also believe in keeping development teams small, with the minimum of structure necessary to run the project.
Phil Morettini specializes in high tech industry issues.
It’s also wise to plan your product releases more frequent, adding fewer new features per release. This should minimize the pain of each release slipping, since the slip time of each release should be less. Knowing what you’re going to be coding, developing a spec document and sticking to it (no feature creep!) is also sound practice, although its no panacea.
Beyond that, you’re on your own.
Who has a strong opinion on how to bring projects out on time? If you do, post a comment — this is a always a discussion worth having.
Phil Morettini is president of PJM Consulting, a management consulting firm providing general management, marketing and business development services and advice to software and technology companies worldwide.
Comment by: Jim T. Posted: June 24, 2009, 2:10 pm
I call it the ,McDonalds hamburger syndrome. You can make a Hamburger 90 % correct….lose a pickle, put only 3/4 a tomatoe on it, etc…90 % right and people will pay for and Eat it.
But a Software Program 99 % right….isn’t ANY good…and will not Run or work until its 100 %.
Comment by: Tim Colling Posted: July 1, 2009, 10:59 pm
Unit testing! Test harnesses! With those two tools, the process becomes much more predictable!
Comment by: James T. Posted: July 2, 2009, 5:51 am
Coming from over 25 years in the software industry starting out as a programmer and now advising on software engineering processes, I use the same analogy of writing code being the same writing prose all the time. It is an art rather than a science.
Having said that, I think there are two things that stand out in my experience to throw projects off schedule/budget. 1) squishy user requirements 2) poor time estimation/management.
Most projects don’t do a good job of nailing down user requirements (e.g. I don’t like the way the screen looks) which causes the developers to have to write and then rewrite code.
Most good coders are poor estimaters/time managers. They are the creative (it’s an art, not a science) type rather than the organized type. So, while they can write great code, they are poor judges of how long it will take to write that code causing the schedule to be unrealistic.
Now, as most familiar with the Project Management Institute (PMI) will tell you, there are many (20 according to a PMI study a few years ago) causes of project failure, my sense is that the two I mention are particularly reflective of software project failures.
Comment by: Burckhardt Rueffer Posted: July 2, 2009, 5:36 pm
One additional factor that is often overlooked is the localization part of the development cycle. Even if the developers have taken all the necessary steps to “internationalize” their project, getting it localized is often considered an afterthought.
Good planning involves getting a system in place where lines are drawn. These lines identify milestones where all players and parts of a project can be synchronized. Small steps help. Fewer decision makers and a clear definition of what is and what is not included in a release help too.
I have been involved in the localization process for many companies, from Big Players all the way to small companies with a niche product. Unless localization is included in the project plan it will cause delays. By the time the project is localized the coders have fixed some bugs, added some features and the localized product is out of date.
Software localization companies spend much of their time working with developers and project managers. Using the localization cycle as buffer helps identify and fix bugs, including those that would remain undetected if development were only aimed at the local market.
Don’t underestimate the value these localization professionals can add to any software development project - they likely have been through the process often enough, for different clients and projects. Getting them involved early can save time and money and will improve the product.
And - ultimately - it may happen that a project is released on time, with all the features and within budget.
Comment by: Steve Wilcenski Posted: July 3, 2009, 5:44 am
If software project specifications were easily quantified, say a project is to deliver 6 Performance Nicks, and development staff then had only to deliver 6PNs, the development and deployment teams might have a shot at making deadline and budget. Requirements and specifications, however, in every project I’ve known, are allowed to change. This, naturally, makes the expected software change, to say 6.75 PNs. Or more. I’ve never seen less except in cases of cancellation. The mere process of settling-in minor modifications disrupts what’s already in-process.
Requirements and subsequently specifications need to be frozen before development and deployment start. Not realistic? That’s correct. Since reality dictates elasticity to account for real changes and inevitable design, even requirement, omission and error, appropriate changes to time and expense of development are necessary.
IT consumers cannot ask for five pounds of potatoes, and when getting to the checkout counters, after the total has been rung-up, declare the potatoes need to be au gratin, and roast beef, red wine, and Baked Alaska are expected.
Comment by: Mark Posted: July 8, 2009, 6:44 am
There are many ways to combat your issue of slipping time lines and budgets. One use an Agile methodology (it’s hard to be late in a two week sprint), and the second is proper tooling. Too many developers hate being controlled by process and tools–that’s the issue. You must have Executive Level support and implement the right tools and process as well as a Business Intelligence (BI) ability. Then you will have very high predictability for your projects.
I have seen failure turn into success many many times by doing these two things (and in some of America’s largest companies). The BI component is new and there is really only one company that can do it (although a few companies like Motorola have built their own).
Adding BI to your development data just as you would to your customer data gives you HIGHLY accurate info on your current project status as well as predictability for the future.
Development is now a science…not art.
Comment by: Mark Posted: July 8, 2009, 7:16 pm
All of my comments are related to Fortune 500 companies.