04 Projection, 04 The Winding Road

The Iterative Roadmap

Reading Time: 6 minutes

In the previous chapter we saw that in a time of need, to commit to shorter Roadmaps of roughly 4-5 Deliverables would avoid potential disappointments. It also has the potential to keep us further away from wrong rabbit holes.

So, if it is beneficial, maybe it should be just always done. In this chapter, we are going to create, define and practice a Roadmap based on shorter commitments. To iteratively commit to roughly 4-5 Deliverables with higher certainties, while keeping the Roadmap in mind to avoid Blockages and disappointments.

Static Design

At RapidAPI (2020) we had a business problem originating from a technical problem. One of the data pipelines, owned by the Data team, had a very low freshness time of about 7 minutes end-to-end. Plainly put it was really slow.

It incorrectly led to over charging tens of customers by hundreds and thousands of dollars, and we had to constantly refund them. But mostly, it hit our company’s credibility. We wanted to get this fixed once and for all.

We were also aware of a future product requirement, for freshness to be near real time. Obviously resolving the future product demand today would also resolve today’s problem. So naturally we designed a technically oriented “two birds one stone” solution.

Jumping to solutions (AI Art)

It would have led to resolving today’s problem, just not today. All Tasks would have to be eventually deployed in order to fix the charging problem, maybe even all deployed at once. And that would take a long time, meaning a long fruition process until we end up satisfying our customers. Customer disappointment was guaranteed to happen, as they already were.

There was also no element of progress to getting it done this way. We’d only be moving the system from one state to another, at once. And that is exactly the difference between the two: Designs and solutions show static states, Roadmaps are a progress to getting those done.

Dynamic Response

There was nothing wrong with the design/solution we thought of. It was only the way to get it done and the goal it was focused on. We would end up not resolving the immediate charging issue any time soon.

So instead of setting the technicality of freshness as the goal, we’ve set the charging problem as the immediate goal, and near real time as the long term goal. And both can and will be measured by freshness, which we already did and know it correlates to the number of incorrect charges. As long as we lower it, we are still hitting two birds with one stone. 

In order to resolve the immediate charging problem, we needed to make progress towards the In order to resolve the immediate charging problem, we needed to make progress towards its resolution. To lower the freshness, which would not not only express progress, but also will express value. To customers, and to the company.

An engineer, finding a way (AI Art)

We started throwing around beneficial ideas. We anticipated their impact on freshness while turning them into Deliverables. Doing so, they became independent of one another, giving us the option to gradually execute them until there won’t be any charging issues left.

Nobody knew how many of those Deliverables it would take, it was uncertain. However, we did know that once the immediate goal is met, we can just stop. Stop and waste no more time on the matter at hand. Because that’s what beneficial Roadmaps do.

We had the long-term goal in mind and its solution, so we kept those as our guiding light. With a Roadmap in our hands, we did our best to ensure no Blockages will be added. None that would prevent the accomplishment of the long-term goal, when it will be eventually needed.

We’ve had everything we needed to start executing the Roadmap. In order to avoid committing to a disappointment, we only committed to one Deliverable at time. Starting with the one who had the most expected impact on freshness. Only when we were done with it, we estimated the next one and committed to it only. 

We were making progress one Sprint after the other. And we properly conveyed it, the uncertainty of what we do and the certainty we had in our Roadmap. Deviations had occurred, but they only Changed the design/solution and the Roadmap, with no Change to past work. Exactly what a Roadmap is for.

After the 4th Deliverable, the freshness had dropped from 7 minutes to 3 seconds. It was enough to resolve the charging issues completely. And we decided to stop, and move on the more pressing issues. 

Recipe for Iteration

Here’s the algorithm to create and execute an Iterative Roadmap. It can be applied to a technically-oriented Roadmap, a product-oriented one, and any mixture of the two. 

Phase 0: Init

  1. Take the immediate business/product problem (charging issues).
  2. Find the technical or product cause (low freshness).
  3. Find a metric that measures the cause (freshness time).
    • Make sure that any increase/decrease in the metric expresses a value that would eventually solve the immediate business goal (the lower the freshness time, the less charging issues).
    • If no metric exists, find another way to express value by progress.
  4. Set a long term goal based on the metric. If there isn’t already one, fantasize one beyond your reach. This would make sure our Roadmap is open to future demands that we can not be aware of yet (freshness time will be 10ms).
  5. Design a solution for the long term goal. This will be used as a technical guideline, and will make sure Blockages would not be added to the Roadmap. Actual progress towards the long term goal is optional, not a must.
  6. Map/rediscover the Current State. Know where you are.

Phase 1: Brainstorm a Roadmap

  1. Throw a bunch of technical or product solutions, ideas.
  2. Anticipate each one’s impact on the metric. That is the expected  and expressed value. 
  3. Technical solution + expected value = A potential Deliverable
  4. Take all the potential ones, and turn them into actual Deliverables. They must be minimal, independent (to be executed on the Current State), have certain enough value, complete, and open for Feedback. Do not yet come up with a technical solution to it, hold our horses.
  5. Sort the list of Deliverables by their expected value.
  6. That is our Iterative Roadmap.

Phase 2: Iterate

  1. As the Roadmap is sorted by expected value, take out the first/next Deliverable.
  2. Finish its process of Uncovering Work to divide to sub-tasks, and estimate with Uncovering Effort. While doing so, make sure no contradiction has risen to the design/solution/technical guideline of the long term goal. Make sure it does not create Blockages.
  3. If estimation is longer than a Sprint or a Deadline:
  • Further divide to Deliverables, and put back to the sorted list according to their expected value.
  • If not possible, either let go of the time restriction or take the next Deliverable.
  1. Commit only to this Deliverable, to avoid disappointment.
  2. Execute it – get it done.
  3. Measure the change in the metric. See if it was as expected.
  4. Revalidate the immediate business goal. If it has been resolved enough, stop.
  5. Deviate from the Roadmap – get feedback, reinspect all Deliverables against the new Current State. Change the design/solution if needed. Resort the list according to the expected value.
  6. Go back to step 1.

It may be hard to notice what is intentionally missing in this Roadmap. Notice that the Details, the exact how, were avoided. Or more correctly to say postponed as much as possible. Because Details can have non-beneficial effects on the Roadmap. On this, in the next chapter.

Leave a Reply