Improving Software Process - A Letter to Upper Management

The following is an email I wrote to the upper management in our company when asked about things we can do to improve the process and what are the next steps of things we need to fix in our application.  It remains mostly unedited and contains my thoughts on what will make our company better when it comes to the software we're writing.  It wasn't written as a blog post.  It wasn't until after I finished with it that I realized that it was decent content for a blog post.  You'll find a very "XP-ish" theme throughout, and that isn't necessarily an accident.  Hope you enjoy! Please leave feedback if something moves you.

Overview

The following contains my thoughts on software development at Super Awesome Company. While I can address the specifics of the project that I see need to be fixed and modified (the “what”), I am instead going to focus on how these items get fixed. I am choosing to do this because over time the new website will have bugs that need to be fixed, enhancements that need to be made, or new features requested. For the long term health of the project it does not matter so much “what” is implemented but rather “how”. For that reason I am avoiding specific project level tasks in this document and speaking at a higher level.

Software has been being developed for over 30 years. While the internet is relatively young, software development by comparison is not. Super Awesome Company will benefit from a commitment to good software development practices. Further, it takes a great deal of commitment and discipline to reap the benefits these practices can bring. Many of my suggestions are based on these principles, which are principles I have committed to learning and adopting over the last several years.

The overarching suggestion I would make is make quality the focus. In order to produce quality, we must slow down. In order to go fast you must go slowly. Sacrificing quality should never be an option. Consider a car manufacturer who needs to get 100 cars built in a single day. If they can only build 80 cars a day, is it better for them in the long run to get 100 cars out the door but have problems or 80 problem free cars? Building deficient cars carries more expense that just fixing the broken cars, such diminished brand reputation. If the goal is 100 cars per day, measures should be sought to increase production while keeping quality high.

We don't need to look too far to see where in a rush we've done something that we've had to revisit several more times because we were "going fast". We've been rushed for nearly a year and I can't say that we are any further along than we would have been had we approached the project more methodically. In some areas we may be further behind where we would have been since we "made sacrifices" to get things done, which have ultimately only put us further behind. Again, to go fast we must go "slow" (methodical). Put another way, quality breeds speed.

Any process, principle, practice we choose should have quality at its heart. Regardless of what process, principles, practices we adopt they will only be successful if we are disciplined enough to follow them.

Planning

Requirements

Requirements need to be clear and understood by all parties. Before developers set out to coding we need to have a clear understanding of what the problem is. The developer is not able to provide a solution until the problem is fully known and understood. Often the perceived solution to a problem is not the best solution. Failure to fully understand the problem/scenario results in rework.

Releases

We need to have planned releases. The timeframe can be discussed and agreed upon by the team. In order to have successful releases we need to plan each release. Each release should be broken down into smaller time frames, iterations. During the iteration the work should be stable for each developer. In other words, they know what they’re going to work on and accomplish (see requirements above) for any given time period.

Metrics

We need to be measuring metrics for the purposes of planning. If we get 10 units of work done each week on average and we have 20 units of work to do, we can easily figure out what is left to do. Without any metrics gathering we can’t do any planning and we’re really flying blind. This will also help with project post mortems. For example, the “web order manager” project was originally estimated to take a week worth of development effort. Three months later, it is still not finished. This is a combination of putting someone with limited knowledge of the business estimating the project, an overly aggressive developer estimate, and having no metric to pull from on how long something should take.

Flexibility

We need to be disciplined in the process of building software, but also realize when the process needs tweaking that we should respond and not be afraid to tweak the process to meet the needs of our team

Design

Simplicity

We should strive for simplicity in our designs. “Clever hacks” are often not healthy for the life time of the project. Simple designs are easier to understand and improve upon. Simple designs also allow other developers ease in working on code. This is much harder to do in practice but it always pays off in the long run.

Design aspects dovetail with testing (later in this document). Typically complex designs prove to be hard to test. When we write code based on simpler designs we’ll find that our code is easier to test. We want both, simple designs and easy-to-test code.

Coding

Standards

We should implement a standard for code development. Right now you can find three different styles/methods of coding in the code base. This makes it hard for developers to work on code they did not author. Projects I work on outside of work require all new code meet the standard, otherwise it is rejected until it meets this standard. As a consumer of these projects I’m always grateful when I can move through any of the project pieces and the code always looks the same.

Testing

Given the environment and complexity of the code, we need to require the developers to be able to test their code in an automated fashion. Visual verification is orders of magnitude slower than automated testing. Once an automated test is written, it runs every time. By having a large collection of tests we can makes changes to certain parts of the application and use existing tests as a safety net to verify nothing else has broken. We currently have over 1,150 tests (about 30% of code has tests) that run whenever someone commits code into source control. Those tests run in about 20 seconds. Imagine how long it would take a user to test each of those cases visually every time a piece of code changes.

Two weeks ago I sat in a meeting adding functionality to shipping and enhancing rules around shipping hazardous items to Canada while people were finding issues because of a solid base of tests. Before the meeting was over both issues discovered were fixed and tests were added for these specific issues so we would be guarded against this happening again.

Unit Tests

All code must have unit tests that can be run in an automated fashion. If the code cannot be unit tested (tested in isolation from other components) it should be restructured in such a way that it can be tested. Very few pieces of code are truly not testable. As such we should have very few areas in our code base which are not tested.

Proving Bug Fixes Through Tests

When bug is found a test should be written before any attempt to fix the bug. This test proves to the developer the bug exists but secondly gives the developer a marker to know when the bug is fixed. As a benefit, this new test acts as a guard against this particular bug ever happening again. This test, when first written, should fail because the developer has not fixed the bug yet. The developer should then work to make the test pass, and by extension fixing the bug. The cost of adding the test is very small when compared against having the test guard against this condition ever happening again.

Code Coverage

We should implement a standard for new code coverage, code that is tested by an automated test. I would recommend somewhere in the 80%-90% range for code coverage. In order to write tests the developers will be forced to think about their code a bit more thoroughly.

Pair Programming

We should adopt pair programming as a method to both increase shared knowledge but also as a means to quality. Pair programming is two developers sitting at one computer for a period of time while code is written. From the outside it may seem impractical to devote two developers to a single computer, thinking it is inefficient. Industry studies however show that code written in a pair has a four-fold benefit:

1. 15% less code– Studies showed that less code is written in order to implement the same functionality. Less code means less to maintain and understand. Less code is also typically simpler.
2. Less bugs – Developers, when working in pairs, can catch each other’s bugs before they ever make it to production. We therefore save time that it would take to fix the bugs that would otherwise go undiscovered.
3. Shared domain knowledge – When “pairing”, each developer is training the other and sharing knowledge and rules around the business. This avoids silos and situations where only one person knows the code.
4. Increased Skill – When “pairing”, design, coding, and testing techniques are transferred between developers. This is often why many development shops pair their senior developers with junior developers; to raise the skill level of the junior developer.

Collective Code Ownership

No one person should be a silo/bottleneck for any area of the code. Any person should be free to make suggestions, fix bugs, or refactor any part of the code. Pair programming (see above) is one tool which seeks to address this.

Conclusion

While the above recommendations may seem like a lot, the above items all really go hand-in-hand. Consider:

• Collective Code Ownership is had through Pair Programming
• Standards are enforced by developers when Pair Programming
• Writing Unit Tests typically leads to Simple Designs
• A commitment to Code Coverage leads to more Unit Tests
• Gathering Metrics leads to better estimates and Release planning

Adopting the above will move us in a positive direction and ensure that all projects coming from our department will be of ever heightening quality. Once the quality bar is set, then we will begin to see the pace pick up; Remember, quality is a prerequisite to speed.

ShareThisPublished Mar 05 2009, 12:49 PM by Tim Barcz

http://devlicio.us/blogs/tim_barcz/archive/2009/03/05/improving-software-process-a-letter-to-upper-management.aspx