Let’s say that you are a software engineer working on an e-commerce product, it’s near the end of the year and the Black Friday event is coming, currently, you are working in a monolithic application, and the server is already overwhelmed with the number of active users. Now you need to find a way to be able to receive more incoming traffic in the next few weeks. how would you solve this problem?

Scalability

To solve this problem the server should be scaled, and there is basically two way to scale a server to receive more incoming traffic, vertical and horizontal scaling.

Vertical Scaling

This basically means adding more compute resources for the server as CPU and RAM, in this way the server can process more requests and handle the overload.

The advantage of this approach is simplicity, is very simple to change the resources of a machine through a cloud provider console.

This strategy works very well for small applications, but it also brings some disadvantages:

• Machines, in general, have a physical limit then will be impossible to scale a machine infinitely.
• Having just one server starts to be a single point of failure, that is if the server dies your application will stop working until the problem be fixed.

Horizontal Scaling

The horizontal scaling comes to solve these two problems.

With this approach the server can be scaled infinitely, you can add as many machines as you want, also by adding more machines the problem with the single point of failure is eliminated.

But how distribute the load between those machines, how to orchestrate the incoming requests to distribute the load? This is when the load balancer comes in.

Load Balancer

The load balancer even can be hardware or software it will receive all the income requests and will distribute this load between the replicated machines.

It makes usage of an algorithm that basically will determine a strategy to handle the load.

Many algorithms can be used, and each one will fit better in cert types of problems, below I will let some examples:

Round Robin

For round robin the requests are distributed for the server in rotation, for example, the first request will be redirected to the first server, the second request will be redirected to the second server, and so on and so forth.

Least Connections

This strategy will take into consideration the number of active connections in each server, and it will send the requests for the machines with the least connections.

Source IP hash

It will take the customer and the server IP address to generate a hash key that will be used to decide which server will receive the request.

Layer 4 and Layer 7

The load balancer may operate on two distinct layers from the OSI Layers, layer 4 (Transport) and layer 7 (Application)

Layer 4 load balancer operates on the transport layer that will route the income requests regardless of the content. Layer 4 by itself is very straightforward, it just redirects the network packages without doing any decryption or inspecting the content.

Pros of Layer 4 load balancing:

• It is faster and more efficient because the content is not considered.
• More secure given that the content is not inspected.
• It doesn’t need any decryption for the content.
• Keep just one connection between the client and the server.

Cons of Layer 4 load balancing:

• It’s a dumb load balancing can’t apply any smart balancing based on the content.
• It’s not possible to use a cache, since the data can’t be accessed

Layer 7 load balancer operates on the application layer, this layer has access to all the content transported and can make decisions based on that. For all requests made two connections are created, one between the client and the load balancer and another one between the load balancer and the server, Layer 7 load balancer also supports cache.

Pros of Layer 7 load balancing:

• It is a smart load balance since the load balancer has access to the content.
• Supports cache.

Cons of Layer 7 load balancing:

• Needs more than one connection.
• To have access to the content needs to implement data decryption.
• Sharing of TSL certificates.

Example

All the concepts are extremely important, but I believe that a real example always helps to get the idea more deeply, thinking about it, I create this Github repository that mimics a real-world example, feel free to take a look. 🙂

https://github.com/wevnasc/load-balancer

Thanks for reading!