Starting a project with the end in mind

In our company, we learned a number of valuable lessons about project development. We learned how to get to a more predictable process which delivers the most value with little overhead. In this article I want to share some of these lessons.

Years ago, our company was just a small machine manufacturer. There were a few software developers. They knew all there was to know: they knew the source code inside out, they knew the machines, they knew the customers. They went to customers to fix issues on-site regularly. So they exactly knew what customers were doing, what they wanted and why they wanted it. They knew what was important for the customers and what wasn’t.

This also reflected the way of working during machine development. When a new machine was to be developed, they knew what to make from just a few talks with the sales people, then they were able to build everything.

Overcoming growing pains

The consequence of this is that this was process heavily relied on knowledge in people’s heads. Very little needed to be written down, so why bother to write this down at all? When one software developer can oversee it all, there is no need for extensive requirements and design documents and no need for review procedures.

With the years passing by, the company kept growing. The company’s product portfolio grew. More developers were hired and the software department grew larger and larger. But this introduced an unexpected problem: the old way of working does not scale. In the past, every developer could know the design ideas of most of the code, but this became more and more difficult with every new hire: more developers meant that new concepts were introduced faster and faster and it simply became impossible to keep up with everything. This meant that software development was becoming more and more difficult. Projects finished too late more and more. Until we hit some projects where it was actually unclear whether the machine could be shipped or not: it was simply not clear what the machine should actually be doing beyond the basic functions.

Time to fix that. Something needed to change. But how to get grip on this situation? We needed to think about a project with the end in mind: when the project is finished, what will be delivered? The main problem was that it was unclear whether we could ship or not, so that was the first issue to solve. When are we ready to ship? What would the stakeholders expect?

So our first step was to talk with the relevant stakeholders. We had the idea to get them to tell what they would like the result to be. But this was not an easy exercise, as it is very difficult to drill down to the actual wishes; people envision the product and then tell you how they want it to be solved, not why. We used a ‘black box’ approach to phrase a list of requirements. The product was framed as a kind of black box where you put things in and where things come out, but you can’t peek inside. No requirement could name actual parts of the machine. This resulted in a list of requirements which mostly listed the actual needs.

However, this list of requirements was not sufficient. You cannot make a reliable project planning (does such a thing exist?) of a project this size with just a list of requirements. You will inevitably forget the unknown unknowns, the integration issues, forget to cope with known risks. It also gives no handle what to implement first. So we had to make yet another transition.

Feature based development

When viewing such a list of requirements, it is inevitable that the set of requirements describes a set of topics; in other words, the requirements can be grouped into topics. When the topics are formulated as deliverable features, then suddenly a list of deliverable work packages results.

These features can then be analyzed: when is the feature done, which other things do we need first? When this analysis is performed for a feature, we can define user stories for it. The nice thing is that this can all be done according to the INVEST principle which leads to very well defined work packages.

The INVEST principle causes features to be deliverable one by one. This allows to use the list of features to monitor project progress. As features are independent, they can be swapped when we learn more so we deliver the most useful things first.

When all feature are completed, then we’re done! But… When running a project this size, all kinds of unexpected work will pop up. There are several ways to handle that:

• You can keep a time buffer for unexpected work (like 20% of the total time that was estimated);
• Since features are relatively independent, you can trade new features for lower priority ones;
• You can accept that the project takes longer when everything absolutely has to be included;

When the first few features and stories are ready, actual development can start already. There is no need to have a complete list of features. This is a very important trait to this way of working, because developing the product is the only way to find out about the unknown problems. Discovering those problems can be sped up by first picking up the items that carry the most risk.

Summarized, this approach results in a high level overview of the entire project, defined by the list of features. This tells what needs to be done, and what the current status is. When combined with information about the time spent, the total duration of the project can be estimated. Simultaneously, development can start as early as possible. Using the same list of features, we can monitor project progress early on.