Wednesday, March 26, 2008

Testing

(post dated to reflect when material was discussed in class) Testing is hard... Reading the description of testing the space shuttle engines in Chapter 11 reminded me a great deal of how large software projects are debugged when one doesn't have much of and understanding of the system. You attach a debugger to the running program, try to reproduce the bug, maybe you have a break point set on an entry point that's likely a source of the bug and you follow potentially suspect data down to the point where it gets mangled incorrectly.

If you're lucky you have enough of an understanding of why the program decided to mangle the data that way and you can fix the bug. Otherwise you're left inserting probes and breakpoints in other suspect locations to give you a quick view of data as it moves around the system.

From my own experience top down debugging is tedious and time consuming. Once I've fixed a bug via a top down approach it's really nice if I can pinpoint the source of the problem well enough to test the fix "bottom up". If I can, putting a unit test in place to ensure the bug stays fix solidifies a bottom up test that identifies a failure point and ensures that future changes don't reintroduce the defect -- or, at the least, it identifies a place in the code that my need to be revisited if future changes make a bug fix obsolete.

Monday, March 24, 2008

Re: No Silver Bullet

When I initially started reading No Silver Bullet, by Frederick P Brooks, Jr I was unaware of its first publish date, and first few pages have little temporal evidence. But the farther I got in to the article a couple things show up ... such as referencing time-sharing systems, "high level" languages and unix. All of which have been around for the outset of my programming life and had already reached a more "mature" state long before I came in to the software world.

I suppose the relative agelessness of the first few pages wouldn't have been apparent if I had read the bits of text that dated the article beforehand. I think the agelessness of the text is important. Because it speaks to the fact that the problems Brooks initially defined haven't really changed all the that much in software engineering. I work in a totally different computing world than what was around when the article was written yet the problems he describes don't sound foreign at all to me.

Brooks introduces several things that promised to "solve" the software problem. Yet 20 years later, none of these solutions have broken through to the main stream. People are still getting things done with imperfect software and still making money. Granted, certain methodologies that may've fallen out of the utopia of "perfect software" are very applicable to certain domains.

For most people, software that's "good enough" is all that's needed. For me the most important point Brooks makes is that software is hard, and nothing is going to make software easy, hence the title No Silver Bullet ... pursuing infallible software is fine for academia but it is probably an expensive loosing battle for 80% of software projects. Anecdotally, I've experienced this in my own professional experience through writing my own fair share bugs. The anecdote follows the following premise: write a bug, ship it to customers for period of time, record the period of time that the bug was in wild until it was fixed and released again. If you look at that time period and you look at how much revenue the software generated in that time period you can come up with a rough estimate of the "cost index" of that bug. If you make 150k over a 2 month period selling the software with the imperfection it seems to show that the software was still delivering value, even though it may have an (innumerable) amount of imperfections.