November 2014 was a big day for cloud computing, but it happened not on the 13th the day that ECS was launched but a day after on the 14th. While AWS were continually pushing forward Containers technology they were working on an entirely new revolutionary concept named Function as a Service, which once again completely changed the landscape of cloud computing.
Unlike virtual servers and containers, which are basically the same both for “on promise” and for all cloud providers, the following article will refer only to AWS Lambda, a specific implementation of a specific cloud provider of the Function as a Service concept as I’ve gained no experience or investigated other equivalent solutions. Other major cloud providers have implemented something similar and open source solutions are available as well (e.g. OpenStack) which still require underlying server maintenance, thus are not Serverless in Ops perspective.
There are at least two aspects to the benefits Function as a Service.
Minimize set up and maintenance
FaaS is first about simplifying and minimizing setup and maintenance. Even with Containers, a lot is needed to launch a highly available “simple” web application:
- Create a VPC with both a public and private subnet (firewall)
- Place three containers with load balancers in the public subnet, each one in a different availability zone (each AZ is a completely independent and isolated data center at AWS).
- Place three containers with your web application in the private subnet, each one in a different availability zone.
- Create a “secure tunnel” between the load balancers and the web application instances.
- Scale up and down both the load balancers (register and deregister their IPs with the DNS) and the web applications instances (register and deregister with all the LB instances)
- (So many more things that we’ve never covered in these articles)
- Don’t forget that all the above applications need to be constantly maintained (update the LB and Web App versions, e.g.)
- And most importantly – you actually have to deploy your code to the web application!
Visually speaking, it looks like this:
Or a more technical view would be:
That may be the first time that you see how much effort you need to invest just to get “your code” running. That is a lot of effort otherwise spent towards your money maker, which from the business perspective is the most important thing. Not Ops, not set up and definitely not maintenance.
Both for the developer and the Ops that’s a lot of work to do just to get a simple code to run. Although many Infrastructure as a Code tools exist, that is still a lot for the Ops to set up and a huge burden to make sure all the above is always up. Function as a Service as a concept is to take care of everything related to getting your code to run, letting you get back to what’s really important – the money maker.
Is DevOps Dead?
One must ponder. Between Serverless Containers and Serverless Functions and if no underlying servers are in need any more, wouldn’t that make the Ops guy obsolete? I really hate these yes and no questions because the truth is always somewhere in between. Quite frankly I’m not an oracle and no else is really as well. No one has any idea what the world of software is going to look like exactly ten years from now.
Let’s have a look the present facts (Silo 2019):
- Serverless requires almost zero set up and maintenance
- Server infrastructure, network and application distribution design and maintenance – less needed
- DevOps these days is a field of expertise in high demand, thus an experienced DevOps engineer is hard to get and very expensive (In Israel, the annual cost of one was around 180K $)
Due to these high costs, At Silo we’ve tried to avoid hiring one as long as possible. We’ve managed without one all the way up to product launch time (which unfortunately never came). That was our backend developer and I managed this entirely on our own. For us, DevOps was adding DevOps as an acquiredable skill thanks to Terraform (Infrastructure as a Code tool). I would say that any good developer can learn the basics of it. That would mean a good developer can be “his own” DevOps. Once we almost ran into trouble, and saw that it could be outsourced as well. There are a lot of independent DevOps companies, consultants and experts available worldwide.
The above is true for the set up and development phase of Silo. We’ve managed to pull through a PoC without any downtime and no need for a dedicated DevOps. My expectations were that once we launch we’d require for starters no more than one (maybe not a senior one as we’ve gained a lot of experience ourselves). As almost everything was expected to be Serverless, he wouldn’t be doing only maintenance and just like his colleagues – he’d also be going home to have a good night sleep knowing that everything is running. Happy customers and happy engineers.
An Easy Start
Just like many things in my life, it started with a real life conversation over a cup of coffee. Around the end of 2019 due to continuous delays in product launch I’ve scaled down my work in Silo to only 3 days a week. In my free time I used to hang around my favorite coffee shop in Tel Aviv. It had the best coffee in Israel, maybe even in the entire middle east. It was worth the 15 minutes walk.
I overheard a conversation between two guys. As one of them went to the bathroom I told the other “if that is a job interview, you are doing very well!”. He laughed hard and said that he is only helping a friend who has an idea for a startup company, something to do with construction projects management and optimization. I’ve asked if I can tap in into the conversation, maybe I could be of assistance.
Let’s treat the guy with the idea as the future CEO and the tech guy as the future CTO. The CTO has just finished his army service at one of Israeli Defence Force’s most notable computers unit. He told his CEO that you’d need to launch a server, host a web application on it, an additional MySQL server. A design very similar to the simple 3-Tier architecture we started this series of articles with. He really did not need anything more at that. You could say that the CTO knows what he’s doing.
I have not said a word yet and I’ve let the two finish the conversation just pulling in all they were saying. They signaled me when they were done and I did my Zen thing, asked for a few minutes to process it, took a deep breath and started petting my beard.
I’ve wondered how many customers they were expecting to have. I was told that they have 2-3 design partners and within a year they’ll have about 20. Maybe if they’re successful they’ll have hundreds within a few years. I’ve wondered how many times do you expect a customer to use your product. “A few times a week, several minutes each time”, I was told. “Would you say that your product is about 99% in an idle state?”.
The CTO flinched an eye and said I’m correct. “I’ve got to say,”, I addressed the CTO with a wink to the CEO, “that your design is good, solid, correct and matches the use case”, suggesting that he was not wrong because he was not. “But you’ve just left the army. Am I presuming correctly that you’ve never worked with a cloud before?”. He nodded. “That’s okay, don’t worry about it there is always a first. Write the following keywords down and google it: AWS API Gateway, Cognito, Lambda and Aurora Serverless.”. These are the minimal components required to create a 100% Serverless 3-Tier solution, where you only pay per use and maintain absolutely zero servers and applications (besides your own code).
I’ve done some quick math and showed that his solution would cost somewhere around 600$ a month, not including high availability which he had no knowledge of. Have you ever stopped and wondered why he’d be paying for servers running 24/7 while his product is 99% idle? Why should you pay for this extreme under utilisation?
But you can’t have less than one server, you’d need to run your code somewhere and I’ve already stated that you’d need at least three. No, I am not contradicting myself at all. With Function as a Service you are paying only for the actual compute resources used. This is why the monthly bill would be reduced from 600$ to around 3$ and he’d no longer need to outsource his DevOps. This is insane.
We’ll dive deeply into real use cases from in the series of articles about Functions v.s. Containers.
The Smallest Container
[this will be deeply reviewed in the series of articles about Serverless Development]
Technically speaking and in the simplest form, Lambda is a Micro Container that consists of:
- Minimal OS Kernel
- A very thin runtime with required application dependencies (e.g. Node.JS runtime with AWS SDK) which is optimized for minimalicity and fast deployment
- Your very thin code (up to tens of MBs)
Plus, it securely shares with the host:
- Networking – to receive and reply requests (called Event and Context to be exact)
- A non-persistent small storage
This minimalism is maintained for security reasons and in order to launch more micro controller instances as fast as possible. This is important because unlike a regular Container that can process multiple requests / events at a time – a Micro Container can only process one at a time. In order to handle scale and concurrent processing, an unknown orchestrator launches more and more of these Micro Containers and makes sure to pass exactly one request at a time for each one of them.
What happens when there are too many of them available? Functions will be frozen and thawed upon demand and after a while being unused and frozen, the instance will be completely removed. This is why and how you’re paying only for actual time used. This mechanism is completely not under your control, no way to control neither to which instance a request is going to nor which to remove. You’d be losing control, but also losing the need of self maintenance. Maintenance done by someone who knows better than you leads to a more resilient system.
You should be aware that due to technical caveats not every code or application can run as-is as a Lambda function. The code may require adjustments and it may not be possible at all to run as a function.
Further Optimization
A common misconception is that Serverless means your code does not run on servers. Serverless only means that you, a cloud provider customer, are not maintaining any underlying servers. But as your cloud provider still does and as these series of articles talk about cloud computing evolution, it is interesting to know how AWS further optimized Lambda, as AWS’s Lambda/Fargate server fleet was very much under utilized with entire servers being idle.
When AWS launched Lambda back in 2014, they used to manage a fleet of dedicated EC2 virtual servers per customer. Meaning that if Silo had just one Lambda function and Tupperware had just one Lambda function, when invoked Silo would get a virtual server and Tupperware will get another one. It was due to security reasons and standards that AWS needs to meet with, which makes sense because a customer wouldn’t want his code to run at his rival’s servers.
Once they validated that Function as a Service is a valid concept with a demand for, they moved towards optimizing the solution. That would be that the Lambda Service itself had an entire pool of virtual servers, to allocate these micro controllers without the per customer restriction.
The solution was Firecracker, a micro virtual machine, that will run the Container or Micro Container within it. It ensures that even if Silo’s and Tupperware’s code concurrently run on the same virtual server they will be entirely isolated and would not even share the host’s OS kernel directly. This led to a dramatic decrease in the number of virtual servers AWS used to run Fargate and Lambda, to huge savings. It made sure that these services would be profitable and also led to a price reduction to the customers themselves. This is how Ops optimisation could lead to tech superiority and market dominance.