minutia press.

I was talking with one of my CSE531 students today who is involved in a large software project at his job. He agreed with me that most projects of any reasonable side are underestimated in terms of how much time it will take to complete them.

One would think that errors in estimation would balance out in a long-term project, so that project lateness might fit a Gaussian curve with a peak at 0 for on-time projects. Not so. A site of megaprojects cites underestimation of completion time as pandemic. Our own beloved Metrolink project went over budget, by a paltry $126M. Besides being late and over budget, the consequences of underestimation included law suits (which Metro lost) costing even more $$.

We don't underestimate everything. When bridges are built, they are usually built to overestimate the load they will experience. We can err on the side of safety if we try.

So I proposed to this student that the reason big projects are underestimated is that if the true estimate were known, people would balk at the expense and give up on the project at the start. If a project is 90% completed and needs 100% more funds to finish, people are more likely to capitulate and spend what is needed to finish up the project.

So this student told me that when he estimates the time to do a task, his management cuts it in half to keep cost down. Then the project ends up going twice as long as budgeted, and management starts yelling. I suggested that he try an estimate of 4x, but he tried that, and management cut it down by a factor of 4 to help the project come in under budget.

Prof. Rouse said that we are still in the early stages of software, so we really don't know what we're doing. We've been building bridges for hundreds of years, so there is lots of build up knowledge about everything involved with building bridges. I wonder if maybe in 100 years everyone on all sides will better understand everything that goes into large scale software projects, leading to a better ability to estimate.

The market should be able to solve this. If a company is good at completing projects on time and on budget, then they could advertise that as one of their assets.

But I guess that assumes there are some companies that can complete projects on time and on budget.

There is another factor at work, called GTM (go-to-market). Product management teams hate engineering teams if the estimates don't match their perceived windows of opportunity. Hence teams are under pressure to keep their estimates low. Teams are under pressure to deliver a lower quality product & continue to fix it with time. Here the problem goes beyond technical realm. Its about $$$