In the previous series, we’ve modeled and introduced The Wheel of Change. The cycle that continuously generates our work. We’ve deeply examined it through the perspective of Change. We’ve learned of planning for the spin of the wheel; of throttling the spin and uncontrollable spins; of the process of fruition, the relationship between timing and value; which all led us to a lesson on how to avoid unnecessary tasks.
In this series, we’d be further exploring the Wheel, this time through the aspect of Feedback. As most Changes are born out of it, Feedback fuels, throttles and sets the direction of The Wheel of Change. We would only be talking about Feedback caused by people, processes, communication and relationships. We won’t be talking about monitoring and other technical Feedback. These are discussed here and here.
In this chapter, we’d first be talking about the importance of Feedback itself. With stories and use cases from Non-Feedback processes.
Down the Drain
A few months after the first Covid lockdown of March 2020, I started looking for a new job. It was also a few months after I left Silo which closed down after 3.5 years without launching the product at all. It was the hardest time to find a job as everyone was so damn picky.
I remember one recruiting process I had for an Israeli R&D center of a big American corporation, not FAGMA. My second interview was with the VP R&D and the third one was with their chief architect while the VP R&D was listening in. I was presenting my work on Silo’s “bullet proof” architecture. For about 15 minutes, they’ve tried to find faults in it with no luck, because I design to fail. Alas I had to tell the truth and conclude the session with “it’s a shame that the company shutdown before it went live”.
The interviewing architect pondered for a few seconds and decided that “you’ve really over engineered this”. It wasn’t the first time I got that kind of reaction. I’ve always led myself into a defensive position. Now it was up to me to overcome the interviewer’s mindset. No matter how much I’ve explained that the right decisions were made. About hardware being slower than software and why reliability was extremely important. About how because of a six months delay in hardware, it was better to switch from Agile to Waterfall.
His verdict of over engineering might have been wrong, but he was not wrong at all that the work was somewhat worthless. The architecture might have been perfect, but there was no Feedback to it. It didn’t go live. Just by telling it so, makes it look like I’ve taken the company down the wrong rabbit hole, of wasting 3.5 years.
It took me a while to understand what I was doing wrong, because no one gave Feedback on me interviewing. I had to figure out on my own that I need to tell the story differently. Stop telling it as my own personal tragedy, of the work that went down the drain.
Instead, I’ve talked about Silo’s PoC/MVP (3D printed versions of the final product). How I’ve learned about reliability and choosing the architecture needed. How I minimized it to fit and to be tested during the PoC/MVP. Telling on how we shipped and monitored 30 units from thousands of miles away. How we gathered data to learn the product’s usage and what we’ve learned from it. On how fast we were to react, that it was only a matter of days to fix all the bugs and product issues we could not anticipate.
And lastly, on what we learned, how we learned and how it affected the final architecture. What was done and what was left for later and why. “Alas, three weeks before launch the money ran out and I had to fire almost everyone starting with myself”. That is a story full of Feedback and learning from Feedback that ended misfortunately. Two weeks later I found myself with two job offers to choose from.
The No Feedback Cycle
One of these offers were for RapidAPI, where I joined in as a group leader and headed the Monetization team as well. RapidAPI had two platforms. One was the publicly available SaaS platform. The other was for enterprises in private tenancies, an instance of the platform was set up for each enterprise customer independently. Features that were coded for the SaaS had taken 3-6 months to reach the enterprise instances. It’s an old practice but a common and sometimes a needed one.
One of the major projects the team was working on is bringing monetization as-is to enterprises. To allow them to charge others for their API usage as part of the Public API Economy. To do so, we had to start decoupling ourselves from several 3rd party vendors. Stripe was the major one. It required some heavy refactoring to the monetization service, adding several APIs and introducing an entirely new event scheduling mechanism.
It was not an easy task at all. It was a 2 months effort of about 2.5 people to get it done. In a perfect world it would have taken less, but we were working with some very old Monoliths. In total, from time of design to delivery it was about 4 months. And then another 3 months for it to be delivered to the first excited customer. And then nothing happened. We joked around saying “we know nobody uses it because no one complained it isn’t working”. It turns out there was some truth to it.
At the sales pitch, the sales team is demoing the platform’s features on a dedicated enterprise instance. As it never has all the features, they also demo glimpses of features available only on the SaaS platform. And once a prospective customer says “I want this”, we start moving the feature as-is. It was hopeful that by the time we finished moving it, the prospective customer would already have signed the contract and get what was promised during the sales pitch. That is the outcome of Sales Driven Development.
So it’s about 6 months from the sales pitch, until a customer first tries the monetization feature, tests it and integrates with it. Meaning only 6 months after we got the work done, we got the first Feedback. And it was bad. Turns out it is 95% worthless to the customer. Because monetization for enterprises is a whole other matter than for the Public API Economy. That was the result of a process that had no Feedback at all during the entirety of it.
There was no internal Feedback between engineering and product, and no external ones between the company and its customers. In this series we’ll be exploring them both. I guess we only told ourselves that we were Agile. The reality was of private tenancies for enterprises causing a 6 months fruition/delivery process. That is not Agile, that is a waterfall. Agile without Feedback is a waterfall.
I had to face the team. There is no good way to say “remember all the good work you did 3 months ago? All those long hours and that one weekend spent where you had to resolve this critical bug before delivery? It was for nothing. I’m sorry we went down the wrong rabbit hole”. There are some damages I just wasn’t able to fix, but it could have been worse. We could have gone further down. There were more tasks in the backlog that we actively withheld, because we were waiting for proper Feedback before getting them done. No more gut feeling driven development, which not coincidentally is the title of the next chapter.