Ever since the rise of Microservices oriented architecture, the size of an application has remained an open question. A lot of considerations had been reviewed, none had focused on the development workflow like this series. From a financial perspective, engineers’s time is a critical factor that can not be disregarded. And we can not disregard Delays in product deliveries due to issues caused by Bottlenecks in our development or company workflow. Both are some of the reasons this book focuses on these matters and how to ease and resolve them.
We’ve seen that any kind of split would have an effect on our development workflow. On our engineers and on our product delivery. Not all of them would have an eventually beneficial outcome, and we’ve seen what to consider in order to validate it would be one.
Will these considerations also answer what is the correct size of an application? Maybe before answering it, we better first pose a more important question – why and how it matters.
Size Doesn’t Matter
Should an application be big or small, or arguing whether it’s too big or too small, is a fallacy of our designing minds. More than that, it is actually a fallacy of our binary thinking minds. If we try to group our applications into two buckets of “big” and “small”, where would we place an application that is “not so small”? It’s nothing but a topic to argue about on and on without anyone convincing another. To overcome this fallacy, we’ve learned it is better to decide based on what would be an eventually beneficial decision, and how to avoid non-beneficial decisions. Doing what matters, matters. Size doesn’t matter.
Which brings us to the second point. Size is not constant, size is an outcome. Applications grow and evolve, and are also refactored. Not only is it not a constant, an application’s life cycle is actually an unpredictable Stream of Changes, Changes constantly applied to our application by our engineers. One day we’d start to work on a new small Feature, and four months later it turned into an entire Product without us even realizing it. One day our Product Manager will have a completely new idea and we won’t be touching our application at all for a few good months. And one day maybe the company will pivot due to reasons beyond our control, or go through a reorganization. An application will probably be affected by all of the above, which is why it should be agnostic to it. Change matters. Size doesn’t matter.
Size is not even an outcome of a Stream of Changes alone. Size is also an outcome of Throughput of Changes meeting the application’s internal design. Splitting an application does not guarantee to have an effect on an application’s size. It may only roll back the clock on its size but eventually it will grow back again. Because time, Throughput and design matters. Size doesn’t matter.
Lastly, let’s be honest. Not getting a Task done because “our application is too big”, would not be perceived as a valid reason neither to the Product Manager or the CEO. Getting it done by creating a new application is indeed a valid response and solution. But that would delay our delivery for a few days and sometimes it might be an Inefficiency. And if done frequently enough, it would be a Bottleneck in our development workflow. Getting the job done matters. Size doesn’t matter.
If an application’s size doesn’t matter, factoring it into our design considerations would be wrong. So how come we do, and should we?
Size Does Matter
A Serverless Function such as AWS Lambda is limited only to 50mb zipped. If our application is bigger than that, we’d have to use Serverless Containers instead. It would also require us to invest effort into handling high availability, load balancing, scale and maintain a web application. Effort Matters. Size does matter.
When another Container instance needs to be launched by the orchestrator and its image does not yet exist on the physical underlying server, it would need to be copied to it. The bigger our application is, the bigger the container image is, the longer it would take for it to be copied from the Container Registry. The longer it would take for our application to spin up, the harder it is to match a sudden spike in scale. From a certain size, it would be noticeable and image sizes can and should be optimized. Exactly why a Function is limited to only 50MBs in advance, so it can scale up extremely fast. Scaling Matters. Size does matter.
Size also has an effect on our deployment’s duration. The longer it takes our application to be copied and booted up, the longer it would take to replace all of the running instances when we deploy a Change. If the duration is long enough, it becomes a Bottleneck and we lose Throughput. Splitting applications to smaller sizes is a way to remove or ease this Bottleneck, up to a certain point. Throughput matters. Size does matter.
And lastly, build/compile and tests. The smaller our application is, the less time it takes to build or compile our application. The smaller it is, the less tests it has and the less it takes to complete them. And just like a deployment’s duration, when it takes long enough it is a Bottleneck. Bottlenecks matter. Size does matter. [Note: there are many ways to avoid and optimize long running tests and builds. But that would be in the fourth book currently titled Assurance. There is no start date for me to write it.]
The Penalty Tradeoff
I speculate the reasons why size matters is what started the talk about size. If we go all the way back to the 90’s, one outcome of Service Oriented Architecture was indeed shorter build durations. It allowed splitting an extremely large single application, which we may mistakenly call today a Monolith, to smaller applications. Not objectively small, but relatively smaller. They still weighed in hundreds of MBs. Instead of a single build running for a few days, these companies had a few shorter builds. Each one was measured in hours and could fit into nightly builds, while our engineers are asleep.
As for their development workflows, their Throughput of Changes had been split between multiple applications. As a result, a big Bottleneck was removed and many smaller ones added to each application, to be further eased later. If they wouldn’t have done this split, they would have paid a penalty in the form of Inefficiencies. That would be our engineers sitting idly waiting for some very long running builds to complete. It is just not feasible in today’s Agile, where we’d wish to deploy several times a day.
Only later in history, when we moved to the cloud and auto scaling infrastructure became a need and a challenge, size had become a technical limitation. And when we incorrectly force it and perform a split, it adds a potential Bottleneck to the development workflow. Given enough Throughput and it becomes a Bottleneck. Although it can be eased through optimizations per application, it can only be removed by merging applications, which is a very strange form of refactor. It would have been better to avoid the split in the first place.
To get a sense why applying a technical limitation in advance would be problematic, let’s have a look at an interesting contradiction within Serverless Architecture. Its most common definition is “a way to build and run applications and services without having to manage infrastructure”. But its most common implementation is only with Serverless Functions (see here, here and here). On the contrary, Serverless Containers and managed orchestrators both fit the same definition of Serverless and are mostly disregarded in Serverless Architecture.
If we follow the implementation to the letter, we’d tightly couple ourselves only to Functions. We’d be forcing a technical Restriction, forcing all of our applications to be limited to 50MB. We’d be creating an architecture that places not one potential Bottleneck, but many and everywhere. Given enough time and Throughput, the eventual result is guaranteed to be fragmentation. To make a single product Change, would be to make multiple code Changes followed by multiple deployments.
As multiple applications can’t Change together and can’t Change at once, it would require careful planning, maintaining backwards compatibilities, and plan and execute a Rollout. That is a penalty to pay. If that happens frequently enough, that penalty would also be an Inefficiency and a Bottleneck in our development workflow. And it happened because our architecture forces us to split applications even when it is guaranteed to be non-beneficial. An architecture that limits sizes is unintentionally eventually designed to cause these.
That was generally speaking of an entire architecture, but the penalties described are the same for every single split. We pay a penalty when we can’t split, or postponing a much needed split.
We also pay a penalty when we incorrectly split. All of these penalties are comparable as they both result in Inefficiencies measured in hours and minutes. Both depend on frequencies and durations, and lucky us they both can be extracted from our CI/CD system that tracks these. We can do some math of frequency*duration for any split we think of, and get some input to make a better decision.
Or we can go beyond calculating frequency*duration, and avoid it all in advance. Throughout this book, we’ve learned principles to identify Inefficiencies and design accordingly. If we design by considering frequency of Change and Cohesion of Causes, we’d end up splitting beneficially and avoiding non-beneficial splits in advance. We’d know when we’ll be gaining Reliability and at what cost, that Isolation prevents Instabilities and Inefficiencies, and we’d be preventing Bottlenecks that entails Inefficiencies.
Size doesn’t matter. Cohesion matters. Because sizes and Inefficiencies are an outcome of cohesion. Cohesion comes first. But wait! If it’s not about size, what is a microservice? On this, in the next chapter.