Technical Debt

With borrowed money you can do something sooner than you might otherwise. If you are willing to incur technical debt you can release a product earlier than you might otherwise.

We all recognize these stereotypes: The sales team is willing to (and sometimes do) sell a product and cash in the money long before development is finished. While the engineers are reluctant to let go of their baby because there are always things that can be improved. A successful business needs engineers and salespeople that are willing to compromise and cooperate on this conflict of interest. Technical debt is a powerful metaphor that can be used to work on a compromise, especially when we are talking about software development.

Technical debt in a software project includes internal things that you choose not to do now, but will impede future development if left undone [1]. Examples of technical debt might be: We need to upgrade our compiler to a more recent version, but let us ship the product now and upgrade the compiler later. We do not properly understand how to implement this feature properly anyway, but this hack seems to make the customer happy for now. We have identified some dirty code that is slowing us down, but we choose to fix it in the next release instead. These are all examples of prudent and deliberate reasons [2] for taking on technical debt which can be compared to borrowing money for sensible housing. There are also less responsible ways of incurring technical debt though, perhaps caused by; lust, gluttony, greed, sloth, wrath, envy or pride. Examples might include: writing bad code, skipping analysis and design, over-engineering, résumé-driven development and so on. This kind of technical debt is more like unauthorized overdrafts and check bouncing, and is best avoided if you have a long-term vision for your product.

Like financial debt, a technical loan will incur interests, and if you are not able or willing to pay back the loan then you risk go into bankruptcy. The nice feature of software however is that paying back is usually both possible and comparatively cheap. While making effective and strategic decisions about what internal qualities to postpone you should keep track of them and write down an estimated effort needed to do it properly. This will give you an idea of how much you owe at any point in time. Then, after rushing a release out of the door, you can immediately start to pay back by doing the postponed things properly and get a flying start into the next release. Retrofitting stuff like this might appear to be more expensive than “doing it right the first time”, but since we are dealing with software it is often the right approach.

Perhaps the most powerful feature of the Technical Debt metaphor is that it communicates well between technical and non-technical people [3]. By quantifying the current technical debt in your product it should be possible to get management both interested and involved in the importance of controlling the debt burden.

References:
[1] www.c2.com
[2] martinfowler.com
[3] blogs.construx.com/blogs/stevemcc

Your codebase is like a kitchen

The state of your codebase determines what you can achieve. In some ways your codebase is like a kitchen. I just presented a lightning talk about this analogy at the ACCU Conference. Here are the slides.

Advanced Feedback-driven Development

For any non-trivial project: Software development should be considered a continuous learning process and a cooperative game of communication between professionals. Effective software development can be achieved through frequently repeating cycles of preparing, changing, observing, reflecting, and learning.

While the statement above is obvious to many, it is easy to miss the key points. For instance, you must make sure that you facilitate the learning process by implementing effective feedback mechanisms, do frequent iterations and be willing to re-plan the project continuously. You must also implement information radiators, enable osmotic communication, and get rid of things that hinders communication (yes, I am a fan of Alistair Cockburn). But first of all, you must assume that your developers are professionals that know what to do given a vision, trust and enough information. You should certainly not treat your developers as mere resources that need to be directed and told what to do and how to do it.

I am fortunate in that I work for a company that really gets it. I invite you to take a look at these slides. The key thing that this particular project do better than most is to do fast and automatic testing on all levels which gives developers confidence when making changes. Automatic feedback from all levels was a key success factor to get the first version out of the door, and now we have a workflow that will support further development for years. I think it make sense to describe what we do as advanced feedback-driven development – AFDD.

Software Architecture is a key enabler for your Business Strategy

Last month I organized an internal 4-day workshop in Software Architecture at my company. Twelve lead developers representing several product groups and development sites attended. The instructor was Dana Bredemeyer himself.

The workshop was excellent, although quite different from what I expected. Bredemeyers workshop was all about techniques for translating your strategy into architecture. Little was said about how to translate architecture into implementation, but that was ok. I have to admit that so far in my career I have thought about software architecture as something that first of all is useful for the implementation of software. But now I realize that for a business relying on complex software, focus on architecture is indeed a key enabler for your business strategy.

All software implementations have an architecture. By studying the code base and a running system you might be able to both illustrate the architecture and explain the expected behavior of the system. For many, perhaps most, development efforts this is a sensible way of working – apply most of your energy on writing code and just take a brief look at the resulting architecture at regular intervals to see if it makes sense. Let the user requirements drive the implementation, and use your test scenarios to verify the product. Because you will never get exactly what you plan for, the resulting product will determine which business strategy you can use to make money and the architecture will restrict how to evolve your business. The products drive the business (see figure 1).

For a business with only a few products on the market this can be a very efficient way of making money. Your product might be successful and the business attracts money that can be invested in making the same product even more fancy to attract even more money. Adjust your business strategy according to the products that are available.

However, as the business grows and the product portfolio diversifies, you need to focus more on architecture. Why? Because you might want a mechanism to let a strategy run the business, not the products. The key is to focus more on architecture so that you can drive it with your business strategy, and let the architecture strongly influence the implementation so that you get products that support the strategy. It is then possible to let a strategy drive your business (see figure 2).

Beware, I am not trying to tell you that heavy focus on architecture is the only way to succeed. It is certainly possible for huge businesses to experience massive success with large and complex software systems without paying too much attention on the architecture. However, the key point is that if you want to control your business by defining strategies you need to focus on the architecture – otherwise not much will happen if you position yourself inside the strategy bubble and try to twist the knobs.

C++ Idioms by Example

I read a lot of code. Often I like what I see, but sometimes I see code that is perfectly OK according to the definition of the language but which is flawed because it breaks too many established idioms and conventions of the language. There are plenty of really good books about this topic, but most of them start at a slightly too advanced level – they assume that the reader already know about the basic stuff.

I have tried to make a little contribution by illustrating some of the really basic C++ idioms and conventions using a contrived but still complete example. Please take a look at this presentation.

When presenting this stuff, I usually hand out the first code snippet to a group of people, then I do online coding while the group suggests improvements and we discuss why it is so. Interesting enough, it seems like we always end up roughly at the same place. This presentation is just one of several paths that might happen when a group decides what to improve or not.

Code Entropy

While preparing my talk at Smidig 2008 I kept thinking about the second law of thermodynamics.

Given two snippets of code, A and B, that have exactly the same external behaviour. If an expert programmer is more likely to change A to B than B to A, then snippet A has higher code entropy than snippet B.

Let us consider a very simple example:

{ int a=3; while (a<9) { ... ; a++; } } // snippet A

and

{ for (int a=3; a<9; a++) { ... } } // snippet B

The external behaviour for these two code snippets are exactly the same. However, most programmers would agree that snippet A is better rewritten into snippet B. So, in this example, during a refactoring session, it is likely that someone will change A into B, but unlikely that someone will change B into A. Hence snippet A has higher code entropy than snippet B.

Now, extend this idea into larger functions, classes, modules, applications, software design and architecture. Can entropy be used to describe the state of a codebase?

Code Cleaning

Take care of your codebase. Keep it clean. A good codebase will enable you to create more great solutions for your demanding customers. If you do not take care of your codebase, then it will rot and your projects (and company) will probably fail.

There is only one way to keep it in a healthy state, follow the boys scout rule: always leave the codebase in a better state than you found it. Whenever you make a change, try to write clean code yourself and clean up surrounding code.

I just gave a lightening talk at Smidig 2008. My talk was about a selection of techniques for improving existing code, how to keep your codebase healthy and why you should do it. Here are the slides and a recording of the talk.

JavaZone – Where to go next?

JavaZone 2008 was yet another great success. Two days, ~2300 geeks, 6 tracks and 100 sessions. This was my sixth time in a row attending the conference.

The conference has grown rapidly from ~400 people in 2003 at Chateau Neuf. The conference then moved to Radisson SAS at Holbergs plass, with ~900 people in 2004, ~1000 in 2005 and ~1400 in 2006. When Radisson SAS was too small to host the conference, JavaZone was moved to Spektrum in 2007 and the number of delegates could be lifted to ~2300. From a technical point of view JavaZone 2008 was more or less exactly as in 2007, the capacity has already been reached. The amazing thing is that the conference has been completely sold out every year, usually several weeks before the event. So where to go next?

I believe that as long as the program commitee does a good job to renew itself, it is in theory possible to keep the current format for another couple of years. Even if the arrangement is exactly the same, people will come as long as the program changes every year. However, how fun is it to arrange the same thing over and over again? Given a decline in the funfactor for the organisers, greed will sneak in and erode enthusiasm and positive attitude. For people motivated by money greed is usually not a problem, but JavaZone is a community driven conference based on a lot of voluntary work – greed will destroy. The dilemma is that JavaZone might die if it continues to follow its winning strategy.

I propose to move the conference to another city next year. Let a local javaBin group be responsible for arranging JavaZone in their home town. I would surely concider to travel a bit to attend JavaZone myself. Actually, I think I would very much enjoy travelling and staying at a hotel with friends from Oslo and meeting local geeks in a new city. Arranging JavaZone in another city will probably also reduce the size of the conference. But so what? A 600 delegate conference in Stavanger, say, might be just as valuable as a 6000 delegate conference near Oslo. Or, perhaps JavaZone 2009 should be in Trondheim, as a kind of a preparation for XP2010?

The next generation of superprogrammers

Most students coming out of university these days are completely clueless when it comes to computer programming. This seems to be getting worse for every year. However, in between all the garbage (where garbage in this context means students not really interested in or capable of programming a computer) there are some really exceptional and brilliant talents that might just need a few more years of experiences to become really efficient superprogrammers. This number is growing. The challenge is to find and recognize these talents, and are they willing to work for you?

I believe the next generation of superprogrammers is to be found in the free and open source software communities. They have little motivation to work with traditional closed source and proprietary software. Why should they? Yes, they want to work on challenging and interesting software projects, but also they want to share their best ideas with others – where the latter is perhaps the most important motivation factor. They want their work to be viewed by others, they want respect, recognition and to be admired by their friends, they want to be part of a world-wide community. In order to hire these people you need to realize that money, internal recognition and a “career” is not enough. You might need to change your business strategy.

Consider hiring a brilliant poet, painter or musician where the message is that their work can only be presented, displayed or heard inside the company – a ridiculous proposal and of course they will refuse. The same thing is true for the next generation of superprogrammers. Given a choice they will work for a company where their work is visible for the outside world, or even better, where their work is a contribution to the software community as a whole.

We see already that many companies have a hard time realizing the impact of free and open source software. Those who do not get it will die. Those who manage to attract the next generation of superprogrammers will win.

Daily Stand-up Meetings – Perhaps the third question should go first?

It is about 10 years since the first time I participated in a project using daily stand-up meeting. I was immediately fascinated. Since then I have always encouraged development teams to do daily stand-up meetings and to get rid of the traditional weekly status meeting.

Modern software development is about taking advantage of the fact that most often you have a group of very intelligent and capable individuals working together as a team. Given accurate information developers will know what is the right thing to do for the project to succeed. Given enough freedom (this is often forgotten) the developers will also be able to do the right thing. The daily stand-up meeting is an effective mechanism for making sure that everybody on the team have enough information to do the right thing.

There are a lot of formats for running the daily stand-up meeting (ref). But common for most of them is that everybody on the team should answer three questions in this particular order:

1. Did?
2. Will Do?
3. Impediments?

Sometimes I observe stand-up meetings where the third question is skipped. Nobody in the team mentions anything about problems, hindrance, obstacles or anything that impedes their work. It is very unlikely that this is true. In a software project there is always something that can be improved to speed up the team.

Actually, of the three questions, the third question is by far the most important in a daily stand-up meeting. The “Did?” and “Will do?” question is also important, but the information is often also available in the software repository and the task bord. Apart from the daily stand-up meeting, there is often no formal mechanism for making sure that everybody knows about what slows down the team.

Therefore, I propose that we should order the questions according to their importance. Allow me to suggest that in the daily stand-up meeting the team members should answer the following questions:

1. Any impediments in your way?
2. What are you working on today?
3. What have you finished since yesterday?