Rituals and Processes - How to make them work, for real

As the summer end and we are slowly crawling back to work from a long time working from home, people will start looking at new ways to work. Again. New buzzwords like mixed workplaces and work anywhere will mix with things like SAFe and Tribes, all driven from top management that need something new to call things. Here is the thing, though... You don't need it. Because the work has not changed and your need for control is due to a lack of trust. So let me explain why the processes are identical all over the world and how rituals may vary to improve them.

Process of making software and development.

I have to admit that I am constantly amazed over how little people understand the process of making software and develop things. Almost everywhere I go, people are so focused on the rituals that they seem completely unaware of the overlying process. Instead, they talk about Agile, Lean, SAFe and other rituals as if they were the process to make code. They are not. They are rituals on different levels. Their purpose is to add naming and rituals to the overlying process. The problem is that without understanding the process, they are just small pieces of a larger puzzle that more often than not, does not fit with other puzzle pieces.

So what is this overlying process, or master process if you will? It is the steps every development must go through, regardless of size. That is what a process is, after all. A series of activities that you repeat over and over. What are these steps then that you always take, and how can they work regardless of scale? Let us define them and see if you agree with me.

Ideation

Ideation usually involve visual design or some technical design, but it can also be feedback on the current solution. The purpose is to form some idea of something that can create value. I do not mean value for the end users, but for the company. This is an important distinction because some rituals only talk about end user value, which is actually irrelevant unless that lead to value creating for the company. This is also why some rituals are completely separated, or ignore the finance part of the process. The ideation part can be a very small tweak, a feature, a project or even a new product.

Definition

Once an idea has been formed, we define what the idea is. We do this with the purpose to make a cost and risk analysis to see if the idea can create enough value to be developed. This involves not just design and management, but input is also required from development and test to get complexity and high level estimates. Again, scale does not matter here, as we do the same for any level with more detailed work required the larger the scale becomes.

Finance

In this part, we match the value creation with cost of creation. Many rituals ignore this step and place it implicit on the role that own the financial responsibility. At large scale this has its own rituals like project planning and portfolio management, but even for small features or even tweaks we always make a cost benefit analysis. Or we should at least.

Specification

In the specification phase, we ensure that everyone involved understand what to build. We also set the rules for how to verify and accept. Again we make this at any scale where at the smallest level this might just be a verbal confirmation and an update to documentation, while in others it can be hundreds of hours of workshop and documentation. The aim is always the same: to ensure we understand each other and know how to verify that we have done things according to the need.

Realization

This is where we take the specification and realize it. Depending on our definition of done, we document what we build with the purpose to keep records of the system, so others can build upon the work.

Verification

In this phase, we verify that the specification has been fulfilled. That is all. If something fail to meet the specification, then it is returned to Realization in the form of a defect, or in large scale circumstanses, as contractual issues.

Acceptance

In the acceptance phase, we approve the development based on the specification, but we also try to determine if the specification was the right one. In the case that the specification did not yield the correct result, we can add new things into the ideation phase to correct. Making the wrong specification is not a fault of the realization, so it can not be defects.In large scale situations this is a contractual issue that is managed as a contactual conflict.

Presentation

The presentation phase is when the idea is finally presented to the end users. Usually this is done by releasing code to a production environment, but it can be done in various ways depending on the product.

Maintenance

This is the last step in the process where problems are managed within the build and presentation phases. This step can be either a time limited one, such as a post go-live support period, or it can be as a part of a maintenance project. Regardless, this phase is a bit different as it has a predefined finance attached to it and it only deals with problems such as incidents and repeating problems.

You always go through these phases

It does not matter at what level you are, you always go through these phases. This is why you will feel pain when your way of working skip one or more of these steps, and in my experience it almost always happen to be Finance, Specification and Acceptance at the lower levels. This is almost always based on the rituals, however, where misinterpretation of Agile along with poor definition of management responsibilities seem to be the biggest reason as I have seen it.

You can do some very small things to adjust this, however, ranging from a small checklist to a more detailed process mapping on need. A checklist can easily be done with just a few questions, where we just verify each phase to ensure we have not skipped any of them.

1. What is your idea?
2. Is it technically feasible?
3. How difficult would it be to realize it in terms of time and risk?
4. Will the idea generate more value than it cost to realize it?
5. Is this idea more important than other ideas in terms of priority?
6. Do you understand what I want, and can you realize it within the time we have set?
7. Is what we have realized in accordance with the specification?
8. Are the end users happy with what we have presented?
9. --- Maintenance below is usually a bit on the side ---
10. Are there any problems that need to be taken care of?

Define the inputs and outputs

If you want to make this a more reliable process, then the next step is to have a set of workshops with each group to define requirements of input and output. Anyone who have spent time defining work processes have probably touched on this, but usually this is a top-down process where you focus on rituals, not the actual process.

I can not tell you how many times I have sat down with people who are very skilled with processes only to see completely blank faces when asking what the requirements for the input is from the developers. They never tend to get that far down, as they start at the top with finance and portfolio management down to maybe project management.

My suggestion is to start with the part where you will make the biggest impact, and that is at the bottom. If you shave off 10 hours a month for every project manager, that might seem to be a huge win. Until you compare it with shaving off 30 minutes of time for every developer and tester. In one company I worked for, I made an extremely low estimate that they lost more than one hundred million Swedish crowns every year due to lacking in the verification part alone. In the whole work process, they probably lost more than 10 times that...

Start with the Testers

NOTE: Depending on your organization and your rituals, you may have people with multiple roles. That is ok as long as they are clear on what role they are representing in each workshop. It may be required that you direct them from time to time if they drift to another role.

So I always start with the testers. This is because despite them being very important in the process, they rarely get to be involved where they should be. This is the same as for design, but their impact is less on the overall process since they exist in the ideation and specification parts. The format for this workshop is quite simple. Put a group of testers of different levels in a room, put up a square on a blackboard or big piece of paper with one side marked as "Input", the other side as "Output" and on the bottom you add "Dependencies".

Divide the group into 3-5 people each and ask them to write post-it's for each of the three sides. Input is what do you need to do your job, Output is what do you deliver to acceptance and dependencies are things that can affect your job. Give the group 20 minutes to write these things down, put it on the square. After 20 minutes, go over the notes together and define what the testers need to do their job, what their definition of done is and what dependencies they have.

The next step is to move over to the work process and discuss in what phase they see themselves participating and what their role is there. Rinse and repeat, but limit the time to 10 minutes for each phase, since most should already be up on the big square. Discuss again and create multiple squares, one for each phase.

NOTE: If you want, you can take this opportunity to define roles and responsibilities at the same time, or keep this for a separate session.

Then move on to the Developers

The next step is to talk to the developers with a similar exercise, but at the end bring up the testers requirements. The aim here is to match the developer's output with the input of the testers. This will show the gap between expectations of these groups, which you can then discuss. From this exercise, you might need to go back to the testers and make a separate gap analysis with them, and in some cases you might need to bring both groups together to get the gap closed properly.

Move on to the business side next

While it would make sense to take the Specification phase next we purposely skip that and lump the business aspekts into one big session. The aim for this session is to identify and separate the large scale aspects of the business side since we will handle them separately. So we try to focus the sesion on two parts: project management and line work. As both work with fixed budgets this allow us to focus on what input is needed to make good finanical decisions and how to prioritize.

In this session we focus on the Definition and Finance phase to get the input needed from Ideation and how that support the financial decisions. In this case we start with Finance, even if it comes last in the chain. That is because it is usually easier to define what input is needed in order to make financial decision. That will make the definition phase easier to define as well.

The trick for this workshop is to avoid getting into discussion of rituals. How decisions are made, in what cadence or what the decisions are called is irrelevant. The only thing that is important is what information is required to make decisions. If your organization have rituals that require you to submit a form-501 for change request that include a certain financial code, then that is a ritual, not a part of a process. Just note that down as that you need financial information about billing recipients and a dependecy to the portfolio process, or whatever is relevant to your organization.

Once you know what information you need to make informed decisions, then it is time to talk about the definition phase. How do we get the information we need from the Ideation phase may sound easy, but this is where things get a bit tricky. On one hand you want the perfect information for financial decisions, but on the other hand you don't want a 3 day documentation process to submit new ideas. So this is where the participants will get a bit frustrated, because we will kill their darlings.

That is right, we will take the requirements they made for the Financial phase and we will do a YAGNI excercise on it. Yagni stands for "You Ain't Going to Need It" and we will use that to mark all data we previously marked as input to the Financial phase. The aim is to remove as much as possible, or rather to simplify to the point where we can take good decisions without to much risk.

This can be a very tricky exercise if your organization have low trust, but it is crucial for you to get a proper process.

Bring Specification to the table

We now move on to the most crucial part of the process: The Specification Phase. This is the zone between the business parts and the realization parts. As such it is the most fragile part and it is prone to a lot of confusion within most organizations. This is why we take this after we do the Finance and Development phases.

Since we now have the requirements from the Development side and we have the expected outputs from the Finance and Definition side we have a starting point. In this workshop I tend to add multiple groups because we need Developent, Test as well as Design and Finance since they are the one that need to understand each other.

I tend to start with defining the purpose of the Specification phase first so everyone is on the same page. 

1. The main purpse of Specification (Requirements) is to ensure everyone knows what the definition of done is.
2. Specifications are legally binding. Payment is made based on them and defects are verified by them.
3. Not all Specifications will be Realized (financial backloop)
4. Specifications are historical records.

With that in mind we can now start discussing the input and output of this phase. You will most likely see that there is a big gap here and that roles such as project managers, requirement analysts and product owners will express concern because the requirements on their job is much higher than they thought. You will most likely aslo see a big change in attitude from the development and test  when the people on the business side start to realise just how much time these two groups spend chasing information that should have come from the Specification phase.

In this workshop it is important that we do not fall into a WE vs THEM situation. This can sometimes happen if the gap between the business side and the realization side is very big. If this happen, remind everyone that we are not there to complain on what is missing, but to collaborate to find ways to add things that we need. Together.

You probably also need to explain to the visual designers that they are part of two phases, the Ideation and the Specification. Many do not know the difference and they often think everything is exploration and ideation, which of course is not true. In the Specification phase we talk about design guides and approved design specifications, nothing else.

Mind the Gap(s) and sign the process.

With this we have pretty much defined all aspects of the process with input and outputs. You may have noticed that there are some phases that we have not covered yet, but don't worry. We actually have covered all the phases, but we just have not had workshops for all of them yet.

Ideation have an expected output from the Definition phase. The Acceptance actually get their expected input from Specification since that is what they will accept towards. This is also true for Maintenance that will focus on Incidents and repeated problems that both have to be defined in the Specification.

That leaves us with Presentation that is the phase where we deploy to production and release it to the end users. Since this usually does not entail a lot of requirements more than knowing what to release and when, there are situations where a more defined process is required. if this is the case for your organization, then follow the same workshop setup as the other ones and map input, output and dependencies.

Once you have all phases done you need to check for gaps that might still exist. So ask everyone that have participated in the workshops to verify their phases, but don't just ask for a thumbs up. Request that they fomally sign a decision document where they say they approve the definitions as they are stated. This is very hard in organizations that handle approvals through a consensus decision ritual rather than a formal decision ritual, but I suggest you insist. Not only will it make everyone take responsibility and thereby look a bit extra, it also visualize that someone actually have thought these things through!

That is something that is far more important than you think.

Brace yourself, change is coming!

Once a process is set everyone will examine it and they will apply their opinions to it. This is natural and it is a good thing, if you can manage it. Change for the sake of keeping things the way they are should be shut down as soon as it comes up. Change that improve the process should always be encouraged. Sometimes the way things are will fall into the category where it improves the process and if that happens improvements always win.

One thing you will notice is that everyone who bring forward thoughts on the process do so from a position of wanting to improve things. No one ever want to make things worse. Because they do not understand the process, or the difference between process and ritual however, they will make suggestions that are making things more bloated and complex. This is because they are blurring the lines between process and ritual and if you are not careful you will end up with a custom made ritual and that can be very expensive.

So what I do is I make a proper channel for input of great ideas. Each channel is tied to one of the process phases and each phase have a group that receive and review these changes. If a change is beneficial for everyone in that phase, then it is implemented into the process. If it is not, then it will not be included in the process. The key here is to make sure that the groups that are responsible for the input and output are the only groups that can approve or deny changes to that process phase.

Remember that the process itself rarely need any changes, but the rituals often require many and frequent updates.

How we use the Process with Rituals

Before I answer that we need to define what Rituals are. In my definition Rituals are a set of nomenclature (set of words), activities and roles. They can exist to cover one phase in the process, or multiple ones. They can also exist on different heirarchies, or scale, such as strategic and operative. They all have the same benefit and problem and that is that while they provide a unified way to discuss around things, they rarely play well with other rituals and they quickly grow to be bloated and encumbersome.

So if we take an operative ritual such as Agile for example, then we know that it is not designed to work on a strategic level. We would then use that preferrably on a line work setup where we have a defined budget, but not a defined scope. For this setup we would then take our process and use it to ensure we get all the input and output done correctly as part of the retrospective.

We can also easily see where the ritual have their weakpoints. Agile for example is high in Ideation and Definition is done collaborative as a parallell ritual. The weakest point of Agile is Specification because Agile is usually more verbal than documenting. So we define together how specification fit within the Agile ritual and add that to our definition of done.

By doing this we can easily discover shortcomings of our way of working with Agile, by examining output and input to ensure we don't miss anything. We can also make small adjustments to the ritual where there are gaps.

Is this overkill for small companies?

Not at all. Understanding what you do and how your work relates to the overall picture is very useful. I almost always hear people asking the same questions in every organization I work: what is expected of me? Even having this as a two hour exercise in a single team will make you work much better together. There is no size limit for when understanding expectations on you and the work you do stop being useful.

It seems very hard to do.

I assure you it is not.

Anyone can do this because even in worst case scenario you will still have had a healthy discussion on expectations in a pretty fun and interesting workshop. This workshop also doubles as a team excercise and an excellent opportunity to bring up problems and issues that may otherwise be difficult to raise.

The tricky part is to find the right level for the input and output. If you take the input for Realization for example one requirement is of course to understand what I am supposed to build. That is to generic so we define that in more detail so we say that for visual elements I need a design or wireframe as the minimum, or referral to components if you work with that. I also need acceptance criterias so I can verify my work. Perhaps I have dependencies to instructions on how the system works or code standards so I know how to write my code.

If you do not get it right the first time around, don't worry to much. You can always iterate and improve when needed.

Defining a process is a great starting point

Once you have a process that you can use as a base of what you do on a daily basis, then you have a very good start. It is very much the foundation upon which you can build your Taj Mahal of work processes that will make your employees create new wonders for the world to see with joy and serenity in their hearts.

So don't focus on interior decoration first, it will just be a bunch of fancy named furnitures thrown into the mud.

It is easier than you think!