I wrote a two-tiered Kanban tool

Why?

Why you might ask? At Zope Corporation, we pursued an agile process mainly as a way to make our team more self organizing. We tried Scrum and it was an improvement, but we eventually found a better process based on two-tiered Kanban.

Scrum

The most popular agile process is Scrum and we started with it [1]. When starting something new, it's usually a good idea to stay "on the rails" and Scrum rails are well laid and documented.

Adopting Scrum was a major improvement for us. The shared Scrum task board:

• Allowed developers to pull tasks. Rather than team leads taking time to assign and monitor tasks, we wanted developers to pull tasks according to their availability and interest and to respond to team needs.

• Helped to focus the team on common goals.

• Provided better management visibility.

• Reinforced process.

  We were building a culture of code review and making that an explicit state reinforced the requirement for code review.

  • Code review tasks were pulled. Reviewers didn't need to be recruited.
  • Review tasks tended to back up, because most developers would rather create than review, but tasks visibly backing up in the review state made review tasks harder to shirk.

We used Scrum for several months.

Scrum had some major downsides for us:

• Fixed sprint lengths didn't provide any value for us. We didn't have issues trusting our developers to get things done, so we didn't need a mechanism around promised accomplishments.

• Most of the various Scrum meetings were time consuming, and not perceived to provide a lot of value.

• Estimation tended to take a lot of time just to decide what to do in an arbitrary fixed time-frame.

• Retrospectives are hard. What makes them harder is that they happen at the end of a sprint, after memory of pains and gains have faded. We were more inclined to "inspect and adapt" in real time. Why wait?

• Scrum focuses on completing tasks. For us, that meant getting software written, tested, reviewed and merged.

  At the time, we were using mostly manual deployment processes. Software could sit in the repository a long time (and sometimes rot) before it was providing value. I felt we were missing something.

  Note that the relationship between the backlog and value could be stronger. Better run Scrum teams probably align value and backlog items much better than we did. I suspect though that most teams get this wrong, as we did.

Two-tiered Kanban

I investigated Kanban [2] and liked it's emphasis on continuous value flow. At first, it wasn't clear how to apply it to development projects, as Kanban boards are typically just flat collections of tasks. But then I came across two-tiered Kanban, which tracks both features, which provide value, and the tasks needed to implement them.

With two-tiered Kanban, units of value are represented by top-level tasks, typically called "features". A feature may encompass multiple stories [3], which may require multiple tasks. When a feature enters development, the stories and tasks making up the feature are presented and tracked using sub-states or nested task boards.

An important goal of Kanban is to provide continuous value flow, so features should be as small as practical and still provide value. Features may require a single story or task [4], which is great, but they typically require more.

Initially, we implemented two-tiered Kanban using a physical board (as we had when initially implementing Scrum).

Large white cards were used for features and sticky notes were used for tasks. Initially, tasks were stuck to feature cards. When a feature entered development, it's tasks were moved across development states. The physical board extended over an area of 4 feet (1.3 meters) high by 16 feet (5 meters) wide.

We abandoned sprints and most Scrum meetings, but adapted and kept:

Story time

Story time is a good way to socialize and gain a shared vision of a feature. Someone (typically a product manager) representing the stakeholders (typically users) would describe a new feature to the development team. This typically occurred in two meetings.

In the first meeting, the team would assess feasibility and challenges and point out gaps, if any, in the feature description.

A second meeting was used to confirm the shared understanding as reflected in the updated feature description. There could be more intermediate meetings, if iteration was necessary.

WAG estimation

• We guessed feature size to help prioritize features.
• Soon before starting development, someone would volunteer to do initial feature work break down, with wild guesses on task sizes. This served as a check on the feature-level guess and provided a way to roughly track progress.

Kanban-style stand-ups.

In Scrum stand-ups, team members go around saying what they did, what they were going to do, and what challenges/blockers they had. This is mostly information that should be on the board. In my experience, both at Zope and at another company, this tends to be mostly an annoying waste of time.

In Kanban stand-ups, the focus is on exceptional situations and dealing with blockers. Kanban stand-ups are typically a lot shorter.

For successful stand-ups, it's extremely important that the relevant information of visible on the board. This puts a huge demand on board simplicity and readability, which is a challenge for most tools.

Retrospectives

The idea of regularly monitoring the process and adjusting is very appealing. We kept holding retrospective meetings periodically, but in practice, they were only occasionally useful. I think amongst the reasons for this:

• We tended to inspect and adapt continuously. Kanban made this a bit easier, because we could do so without upsetting sprint schedules.
• By the time of a periodic retrospective, problems and opportunities are easily forgotten.

While the physical task board was a good way to start, it was cumbersome. I was unable to find a tool that implemented two-tiered Kanban as I understood it.

Asana-based implementation

I created a quick hack on top of Asana:

• Features were managed in Asana as tasks. Feature tasks were managed as Asana subtasks.

• A shared on-line two-tiered Kanban board provided a visualization of the underlying tasks.

  • Features and tasks could be dragged on the board, which updated underlying task and subtask states (represented as Asana tags).
  • When a task was dragged to a working state, then the task was automatically assigned to the user doing the dragging.

  Changes made via the board were updated in real-time.

• The Asana API wasn't well suited to this sort of an application, so an intermediate server process was used that provided extensive caching. Updates made in Asana weren't reflected in the on-line board until the cache was manually invalidated.

This was a big improvement over the physical board and worked pretty well initially. Having to use both the Asana API and the board to manage tasks was cumbersome. Over time, the Asana API became much slower to the point that we were unable to provide enough caching to make the board responsive. At about this time, we started using an off-shore development team that we didn't want to share Asana access with and we stopped using the two-tiered Kanban.

Later, I began prototyping an implementation that used ZODB rather than an external service as a back end.

I joined a company that was using and struggling with Scrum. I presented my prototype and they were interested in trying it, but it wasn't far enough along to be usable.

Feature-flow and Valuenator

For the last year or so, I've been an independent developer focusing on various open source projects, especially relating to ZODB. A recent project has been to complete [5] the two-tiered Kanban prototype:

https://github.com/feature-flow/twotieredkanban

This is largely functionally complete, and documented:

http://feature-flow.readthedocs.io

I chose to brand the practice as "feature-flow" to put more emphasis on practice rather than on mechanism.

I think this practice and tool can benefit a lot of teams. Now, I need to get the word out.

Trying it should be pretty easy:

• There's an in-browser demo version that requires no installation or sign up.
• An online version is available.
• There's a docker image that you can run yourself.

And, of course, the source is available to do with what you will.

I'm told that many of the commercial tools can provide two-tiered Kanban. That's great if true [6]. I care more about the practice than the tool. I simply wrote the tool to enable the practice.