In the previous chapter, we talked about lack of Feedback. In this chapter, we’d be exploring the effect of Feedback on Change. On how the right Feedback at the right time leads us away from a non-beneficial Change, which then leads us away from the wrong rabbit holes. We’ll see what happens when insufficient Feedback is gathered, the wrong one, or at the wrong time.
An Incomplete Validation
Feedback is more than a toolbox full of customer interviews, customer surveys, mocks and others. It is an art of its own. As such it sometimes requires another person’s perspective, outside of engineering.
Silo’s founder (2019) was a well experienced mechanical/system engineer and quite a genius. He came up with the Silo’s patent, and we ended up with manufacturing deep vacuum containers with Alexa and a mobile app (we were at least very close to do so). One day he had an idea for an improvement. The plastic containers should be numbered so you can use your mobile app to find them in your fridge or cabinets.
He grabbed the designer and another mechanical engineer. They all spent a week designing and printing a 3D model numbered container to show us all. It was quite beautiful and valid for manufacturing. In his mind, a decision was already made to add this to our MVP.
Back then, I was the only software engineer. I was the only one who had a course in introduction to computer science. I knew that a 3 digit number can not be unique enough in mass manufacturing. Back then, we had no product manager. I was the only one who had some knowledge of cognition processing as I had an entire course about Introduction to Human Factors Engineering. I knew that a 3 digit number is one of the worst possible ways to do a cognitive search for real life objects.
I knew it, and yet did the research required. Research does not so easily overcome a mind that has already made a decision. The research indeed showed that numbering containers is not a beneficial idea. It also showed that doing a cognitive search based on size and color, which the containers already had, is the right way to go. It is only a small change to the mobile app that can be eventually done. That was proper validation helping us avoid an execution of a long fruition process that would have ended with a negative actual value. But we should have done the validation earlier.
I hope we noticed the above story ended with a waste of time for 3 people (founder, designer, engineer). The research took me three hours. A research the founder wasn’t capable of doing, as he never knew what cognitive search is. It’s no one’s fault that we can’t Google what we don’t know of. Alas, he hasn’t shared his idea with the right people at the right time. He could have gotten Feedback sooner and avoided the waste. Proper and beneficial validation should be done at the right time. Otherwise, it’s just a gut feeling that could lead us down the wrong rabbit hole.
Justify the Means
Design or spec does not come out of nowhere. Product/project managers sit down with customers, sales, marketing and other teams and constantly iterate on them. A waterfall project can end up with a 400 pages long spec. An Agile project, due to smaller iterations, ends up only with a few pages of design or spec. For both, we already know there is nothing like the concrete and viable Feedback of the end result. The actual value.
The practice of Continuous Deployment of the Agile Methodology allows us to iterate faster and get Feedback sooner, and reach the end result sooner. But it is our binary thinking mind saying “if there is nothing like the end result and it is so easy to get there, let’s skip all the way to it!”.
This would be incorrectly throttling The Wheel of Change. It would be throttled by Change first and not by Feedback first. And Change takes time. Coding takes time.
Change done by coding with no proper validation is neglecting and disregarding important Feedback. Having no Feedback is exactly like trusting your gut feelings. And we’ll soon see why our guts should not be trusted, as suggested by Nobel Prize Winner, Professor Daniel Kahneman. in his book Thinking, Fast and Slow.
Coding the entire product as one big validation, dirty-coding until meeting a product market fit, also has non-beneficial results. Neglecting and disregarding coding practices speeds up the Wasteful Applicative Evolution. Specifically, the code would soon be such a Monolith, that iterating for validation itself would be inefficient and a waste of time.
And it would be hitting us exactly at the wrong time, when the company needs to grow. Agile is not a free ticket to do your job with disregard to consequences. Ending up with a product coded like one big Proof of Concept (PoC) is not a beneficial result. It would take a lot of effort to straighten it out.
Validate Coding
The Middle Way would be to treat PoC just like any other application. An application engineered for fast iteration. Yes, it takes time today to prevent being inefficient next week. And it is beneficial according to Martin Fowler.
As time passes the PoC application would grow. It would take longer and longer to iterate on the PoC. We can look at effort as an equilibrium point between a PoC and a product. If iterating/changing a PoC takes as much time as for a product – it is a product. It may be a point to stop and look at the value made.
If insufficient actual value has come to fruition, it might be the time to let it go, to drop the PoC. If we won’t, we’d be going down a rabbit hole and take many people with us. If we already coded it properly like a product, there would be no reason to Change the existing code, only to extend it. A properly coded application would be agnostic to this decision made.
By properly validating first, we’d making our development more efficient today; by avoiding unnecessary tasks. A fine equilibrium between the two would be to invest equally between validation and coding. If it takes an engineer 3 days to code it, we can spend 3 days on proper validation. Preferably a PM should be there for it.
There is so much more to proper validation that we can say it is part of engineering itself. Refining and focusing our MVP/PoC; Postponing what’s not needed yet, Designing with Eventualism; Differentiating between internal technical feasibility or value and external customer ones; Maybe postponing some tests to later in the fruition process, once the value is more certain. Read The Lean Startup, It’s all about how to refine work with proper minimal effort validations.
Lastly, we should be aware of Analysis Paralysis, to catch ourselves before we over validate.
Coding Validations
At DealPly (2015) I was called in for a meeting with the product manager, to advise on a technical concern. He had an idea, don’t remember the details of it but it sounded fantastic and with great potential to increase the company’s income. And we all agreed it would.
DealPly’s architecture was based on the assumption of a 1:1 relationship between web page views and server API requests. At that time we had millions of daily active users generating billions of daily API requests. The suggested feature had a potential of inducing a 1:3 to 1:10 relationship.
I was afraid it would bring down the entirety of our servers if it were to go live with current API and architecture. I estimated that creating a new API would be a matter of weeks, but neither I nor my superiors could anticipate whether the downstream servers would be able to handle that backpressure.
“Can we code this feature, a very simple version of it, but run it on 0.1% of the traffic using the existing server API? Just so we can make sure the potential we expect does exist?”. It was not an engineer who suggested it, but the product manager. What made it great is not the solution suggested but suggesting validating first. And so we did.
We tested it for 0.1% * 10 = 1% increase in API requests, behind a feature flag. We shortened the fruition time. And although it sounded like an amazing idea to everyone, the customers did not like it at all. It dropped our KPIs for reasons we could not anticipate. It was not luck that we avoided wasting weeks of coding. We did not go down that wrong rabbit hole on our gut feelings. (Read this by the legendary Martin Fowler. It’s everything you need to know about flags).
These are the kinds of validation to code. Quick, short and simple. Their expected change should only be small tunings measured in minutes of effort, and then removed in its entirety within 2-3 Feedback-Change cycles. Engineer it to be contained within the code, and easily deleted from it.
Getting the right Feedback at the right time is a must. But it isn’t sufficient on its own. In the next chapter we’d see why understanding Feedback is a must as well.