Over the last few years I have grown increasingly passionate about the agile/lean movements and truly believe that for the majority of the projects I have encountered these methodologies would actually provide a better way to manage their complexity. What I aim to write in this post is some of the key learnings, they are simply re-iterations of what has been previously said about agile and lean but I think that there are some core concepts that can be taken across to any project.
Everyone involved in a software project from the business to the developers/testers need to understand the development process but they also need to buy into that process. No single process is applicable to all situations and there must be a carefully considered decision about which methodology to use and to get people to understand why this process has been chosen.
I have been on a number of "agile" projects where significant stakeholders just wanted to write some vague requirements at the start and then don't want to be bothered, where testers don't really want to know about the project until it is delivered to them with weeks before the project is needed live; on other projects, managers have demanded that everything be written down, again on an "agile" project, and e-mailed to all concerned parties who then take ages to respond. These project are incredibly difficult to run and manage because the development team are looking for quick feedback on both requirements and from testing but when this does not come the team ends up reverting back to a waterfall approach and spend a lot of effort "protecting themselves" from any potential comeback.
One thought is that on these projects were actually waterfall but the developers were trying to go more agile but either way, where there is this conflict there is going to be uncertainty and risk.
Regardless of whether you are working in an agile manner, a waterfall manner or even in a post-agile manner, the key to a successful project is communication. I believe that the business, architects, developers and testers should be talking constantly about the project to highlight and tackle issues sooner rather than later.
Good communication will help to build trust between the business and the development team, a regular show of progress can help to allay fears and to also help communicate the amount of effort that the developers are putting into the project.
When things are faltering communication becomes even more important, letting people know that there are issues allows everyone involved to prioritise them, tackle the issues immediately or even just plan to deal with them. The sooner you know there is an issue, the more options you have available and therefore the more likely you are to find a workable solution.
This really is a sub-set of communication but it is something that I consider vitally important. Businesses should constantly be focussed on the risks to their projects and one of the greatest risks that I can see is concerned with the timescales of feedback.
In a most waterfall projects of any significant size (6 months or bigger) that I have been a part of the feedback loop between the developers and the testers is huge. Several months of work before the testers even start to look at the system means that bugs entered into the system at the very start do not surface until close to the end. The sooner we can start testing, the sooner these bugs will surface and, once again, the more options the development team will have to fix the bug.
Also, we need to be looking at the feedback loop between the business and the developers/testers. The business define their requirements at the very start, usually with the analysts, and will sign-off the design but if they are not involved in the day-to-day development issues will arise that the customers will have no control over and decisions will be made that may end up not delivering what is actually needed but can be interpreted as meeting the requirements. The business will end up seeing the final system when they get to perform UAT but the developers have not received the feedback that what they have developed is correct until it is too late to make any significant changes.
This is quite an agile concept, about reducing the feedback loops to improve quality and importantly to keep the project on course. It does not mean that you have to give up the up-front-design or even give up on having the analysis, design, build and test stages of a project but at least regularly show your work and get that feedback.
Quite often we expect teams to commit all their time to something, either to the current project, or to support, or to the design/analysis of the next project and there is never any slack built into the system. When you have the slack you can burst when there are issues that need to be addressed immediately but most of the time you can spend your time learning, improving your skills or even tackling technical debt issues that are normally left because there is no budget.
I believe that development teams should be constantly working, constantly developing new software, but at a predictable pace, if you have them running at 100% you will lose that predictability as problems have a proportionally larger effect than if you are working at 75% where the problems can be absorbed by the slack in the system.
One major advantage of building deliberate slack into the system (and having the discipline to maintain it under most circumstances) is that developers will have time to research into new techniques or APIs and become better developers.
I have realised that I much prefer working as part of a consistent team, or at least working as a team. I want the team members to be passionate developers who won't let me get away with making stupid decisions or not thinking about the impact of a decision, people who will question me and force me to justify my thought processes but will also accept when a decision has been made.
I find that if I work alone then I will be less productive and should I fall ill, or even take a holiday, the knowledge of the software has left the company and there is no one who understands what has been done or how it has been done.
Also, perhaps one that most people don't consider important, is that I find I will work harder because of the people in my team. It is often much harder to feel a level of loyalty to a lot of companies but if you are working with a good team, if you feel part of that team you will feel loyalty to that team and you will work harder for them.
I think finally I would like to say that quite often projects seem to flounder despite the lack of strong leadership and I think that is partially due to the fact that the team do not fully feel that the project belongs to them. The project belongs to all of the stakeholders, including the users, analysts, developers and testers. If each of these people has a personal stake in the project, if they feel that they are empowered to raise issues, to guide and make decisions based on the ultimate goals of the project then they will strive harder to make that work. People who feel that they are simply doing something for someone tend to approach that task with less vigour.
In summary, I think that regardless of whether you are doing projects in an agile manner or a waterfall manner there are a number things that can be done to drive that project forward more successfully, the teams will feel more involved and will build a culture of success.