Daily clean code

…bugs turn into features at midnight… I once worked on a very large software project. Over time, our bug list grew larger and larger, until everyone realized that something had to be done. A team was dispatched to clean up the bug list; to either fix the bugs or remove them from the list. A couple of months later, management announced with great fanfare that the bug list had been reduced from around a thousand bugs to some six hundred, thus reducing the number of bugs from astronomical to merely unmanageable.

…As a team develops the product, bugs are invariably detected. The usual tendency is to write bug reports and put them on a bug list, where they will be fixed at the appropriate time. But this leads to several problems which ultimately cause the velocity of the team to bog down.

Here we discuss bugs that arise within the sprint. Preexisting bugs should be prioritized by the Product Owner and managed in the Product Backlog. Bugs appearing from outside the sprint such as customer emergencies should be handled by the Illegitimus non Interruptus pattern.

✣  ✣  ✣

{{{wish full}}}

It is natural to want to keep a list of bugs. There are several forces that encourage this.

• One of the most compelling reasons to put bugs on a bug list is that developers are busy with other tasks, and shouldn’t be interrupted.
• Managers can see that adding new features increases revenue, but fixing bugs does not apparently increase revenue. So there is pressure to work on new features rather than fixing bugs.
• New development is more fun than fixing bugs. Therefore, there is a natural tendency to park bugs in a list of things to do when all the fun stuff is done.
• If the team has a fuzzy definition of done, work might be considered Done, with some pieces deferred until later. Likewise, if the team's commitment to Done is weak, the same thing will happen. The work not done yet is basically a bug that is deferred.
• Bugs that arrive might be considered low priority, and it’s nice to have a place to put them.
• If others (e.g. testers, users) find bugs, bug reports and lists give them a convenient way of logging the bugs. But it's easy for developers to leave them on the bug report list. (Maybe they hope that if they leave them there long enough, they will go away.)
• If bugs are allowed to sit on a bug list, further development might even depend on the bugs, making them that much more difficult to fix. Bob Martin relates a story of once fixing a spelling error in an application. However, many customers had built screen capture scripts that depended on the misspelled word. He was ordered to "unfix" misspelled word.

However, keeping a bug list causes numerous problems, such as the following:

• Each bug has to be handled at least twice, once when it arrives, and later when it is fixed.
• The longer a bug sits before it is fixed, the greater chance there is of losing information about the bug, how to reproduce it, and how to fix it. Written bug reports are designed to capture this information, but they often miss important information, especially about how to reproduce the bug. The original finder of the bug may forget, or worse, no longer be available. And good bug reports take time and effort to write.
• Bug lists are generally prioritized, but how do their priorities match against the priorities of development items? Keeping separate lists causes this question to be revisited constantly. This creates work and causes context switching in the team members.
• Bug lists tend to grow. If they are allowed to get large, it virtually ensures that the lowest priority bugs will never get fixed.
• Bug lists usually contain user-visible bugs. this creates the possibility that another user finds the bug. If the user writes a bug report, you have duplicates, and managing duplicate bugs is simply messy. And a user who finds a known bug is needlessly inconvenienced.

In short, deferring the fixing of bugs until later is borrowing against your future velocity.

Therefore:

{{{therefore full}}}

✣  ✣  ✣

This is simply practicing good housekeeping. If you find a cockroach in your house, you kill it right away.

In order to make this work, avoid the temptation to keep a bug list. You already have a list of work to do, the Sprint Backlog. As bugs are identified, they are either fixed immediately or go on the sprint backlog to be fixed later in the day or the very next day. This allows developers to complete their current task without interruption, if necessary. Naturally, if it's a bug you find, fix it now, while it is fresh in your mind.

✣  ✣  ✣

Note that putting the bug on the sprint backlog does cause the bug to be handled twice, so the ideal is to handle the bug immediately. But there is a tradeoff between handling it immediately and continuing current work without interruption.

If a bug cannot be fixed within one day, then that may indicate a more serious problem with the design of the product. After all, no bug should take very long to fix. Causes of such bugs should be discussed in sprint retrospectives.

It is still usually necessary to have a bug reporting mechanism for customers to write up bugs. If you have one, use it only as a holder for bug reports that have not been examined yet. Once you examine a bug report, handle it immediately as described above. Resist the urge to read it and leave it in the bug reporting system to be handled later.

Note that while the focus of this pattern is on fixing bugs, one should also treat refactoring the same way. After all, refactoring is a way of cleaning up the code, and the at the end of the day, the code should be as clean as possible. Like bugs, deferred refactorings are technical debt.

How does this help velocity?

• It reduces the overhead of handling bugs. You don’t need to devote effort to maintaining a list of bugs and periodically reviewing them and prioritizing them. Each bug is handled just once. It also basically eliminates duplicate bug reports, which are traditionally cumbersome to handle.
• It eliminates two separate lists of work to do, making it easier for the team to focus.
• It keeps the code clean. Developers won’t be basing code on top of bugs. The consistent quality of the code is probably the major contributor to the improvement in velocity.
• It eliminates the need for the periodic “bugathon” or “bug bash”, where the team stops what they are doing to spend time cleaning up the code. Not only do these activities take time, but they can breed other bugs. (High intensity bug fixing induces fatigue and encourages one to cut quality and verification corners. These lead to additional bugs.)

If a team already maintains a bug list, then moving to this pattern will be painful and require a lot of work. However, fixing bugs immediately will radically reduce time needed for testing. Since testing is usually the bottleneck, deployment will go faster. For these reasons, the change over to immediate fixing of bugs discovered in a sprint is recommended. A plan for remediation of technical debt introduced through old bugs should be developed and implemented.

This is a radical break from the traditional way of thinking. Teams that maintain a bug list must not only eliminate their existing bug list, but must change their way of thinking. Strong commitment from the team is needed, along with constant reminders, especially from the Scrum Master (see DoneMaster and Sheepdog).

The source of data on this is a paper on Scrum and CMMI where a CMMI Level 5 company drove the doubling of productivity on every time using control charts that manages bug fixes to less than a day (ideally averaging two hours). C. Jakobsen and J. Sutherland, "Scrum and CMMI – Going from Good to Great: are you ready-ready to be done-done?," in Agile 2009, Chicago, 2009.

Related Patterns:

• definition of done is necessary to establish the criteria for bug-free code, and done master is used to enforce it.
• completion headroom can be used to accommodate time for fixing bugs.
• illegitimus non interruptus can be used for handling serious user-found bugs.
• The scrum master must often be a sheepdog to help remind developers to follow this pattern until it becomes a habit.

Testing is about giving people information and that’s the role of the tester—to give people information about the quality attributes of the solution, not necessarily “is it right”—two very different things. Testing is a structured, considered, and focused approach that looks for a particular quality attribute of your solution.

Extra Notes

Forces:

• Business looks for market growth and delivery and engaging with customers, wants high velocity, predictability, and sustainable pace.
• Resources (people, budgets, equipment) is limited, scarce in some cases.
• Deferring the fixing of bugs until later is borrowing against your future velocity with compound interest (hence, exponential).
• Velocity chokes because a team spends time dealing with too many bugs.
• Fixing bugs immediately will radically reduce time needed for testing.
• Bug lists en technical debt tends to grow ever larger.
• Focus on defining done and defining good and then focus on delivering that, not just delivering “your bit”.
• Whenever you see anyone thinking, “We’ll test that later,” what they’re actually saying is, “Never”.
• Having testing as second nature speeds up development and pulls completion dates closer.

Therefore:

• Test early, test often.
• Don't change code without a test:
  • don't fix a bug before you have a broken test;
  • a feature is only done once you have tests for it;
  • don't refactor before you have sufficient test coverage.
• Assert against improper behavior:
  • If it can't happen, use assertions to assure it won't.
• Maximum method weight:
  • a method should be replaceable by one of your colleagues in less than 30 minutes.
• Beware of Heisenbugs:
  • “The measurement of position necessarily disturbs a particle's momentum, and vice versa” (Werner Heisenberg);
  • Similarly, inspecting a running program might alter the behavior of the program, known as a Heisenbug.
• Clean builds
  • “Keeping the build clean is not just about keeping it free of compilation errors or test failures. Warnings are also an important and critical part of code hygiene.” (Johannes Brodwall)
• Code and test are one
  • When writing the code, think of the test.
  • When writing the test, think of the code.
  • When you think of code and test as one, testing is easy and code is beautiful.
• Build only what you can test
  • When you can't test everything, you're building too much
  • When a story isn't tested, it simply is not ready for production. This means that focus should be on testing throughout the sprint, by the whole team. No new work should be picked up if there is test work still open.
  • Maximizing cohesion and minimizing coupling reduces complexity and increases testability.
• Fixing a bug is less expensive when found earlier in the development process

---

Web: ScrumPLoP » daily clean code