Apache Thrift - Load Balancing

In distributed systems, load balancing and service discovery ensure high availability, fault tolerance, and efficient utilization of resources.

They help distribute traffic evenly and allow systems to adapt to changes in the environment, such as new instances being added or existing ones going down.

Load Balancing

Load balancing involves distributing client requests across multiple server instances to prevent any single server from becoming overwhelmed.

This ensures better resource utilization, improves response times, and provides high availability.

Types of Load Balancing

Following are the primary types of load balancing −

Client-Side Load Balancing

In client-side load balancing, the client is responsible for deciding which server to send each request to. The client maintains a list of available servers and selects one based on predefined strategies or algorithms.

Description: The client application directly interacts with multiple server instances and decides where to route each request. This approach can help distribute the load evenly and adapt to changes in server availability dynamically.
Example: Libraries such as Ribbon in Java provide client-side load balancing capabilities. Ribbon allows clients to load balance requests across multiple server instances by choosing among them based on configurable rules and algorithms.

Server-Side Load Balancing

Server-side load balancing involves using an intermediary load balancer that receives incoming requests and forwards them to one of the available server instances. The load balancer is responsible for distributing traffic according to its configured rules.

Description: The load balancer sits between the client and the server pool, managing and distributing incoming requests. This approach centralizes load balancing logic and simplifies client configuration.
Example: Popular server-side load balancers include HAProxy and NGINX. These tools can distribute traffic based on various algorithms like round-robin, least connections, or IP hash, and provide features like health checks and session persistence.

DNS-Based Load Balancing

DNS-based load balancing uses DNS to distribute incoming requests among multiple server instances. By resolving a single domain name to multiple IP addresses, DNS can direct clients to different servers, balancing the load across them.

Description: DNS entries are configured to return multiple IP addresses for a single domain name. DNS servers handle the distribution of requests by rotating through the list of IP addresses or using other strategies.
Example: Services like Amazon Route 53 offer DNS-based load balancing. Route 53 can provide features such as weighted routing, latency-based routing, and geo-routing to manage traffic distribution effectively.

Implementing Client-Side Load Balancing

Client-side load balancing is managed by the client application, which maintains a list of servers and decides which server to route each request to.

Libraries or frameworks typically handle this process by applying load balancing algorithms to distribute requests efficiently.

Example in Java using Ribbon

The following example demonstrates how to configure and use Ribbon for client-side load balancing in a Java application.

It shows how to include Ribbon as a dependency, set up server lists, create a load balancer, and send requests using Ribbon's load balancing capabilities −

Include Ribbon Dependency: Add Ribbon as a dependency in your "pom.xml" file to use it in your project −

<dependency>
  <groupId>com.netflix.ribbon</groupId>
  <artifactId>ribbon</artifactId>
  <version>2.3.0</version>
</dependency>

Configure Ribbon: Set up the list of available servers for Ribbon to use. This configuration specifies which servers Ribbon will consider for load balancing −

ConfigurationManager.getConfigInstance().setProperty(
   "myClient.ribbon.listOfServers", "localhost:8081,localhost:8082");

Create Load Balancer: Initialize the load balancer with Ribbon's configuration. The load balancer will use the list of servers to distribute incoming requests −

ILoadBalancer loadBalancer = LoadBalancerBuilder.newBuilder()
   .withClientConfig(DefaultClientConfigImpl.create("myClient"))
   .buildDynamicServerListLoadBalancer();

Send Requests: Use the load balancer to choose a server and send a request. The load balancer will select one of the servers based on its algorithm −

Server server = loadBalancer.chooseServer(null);
URI uri = new URI("http://" + server.getHost() + ":" + server.getPort() + "/path");
HttpResponse response = HttpClientBuilder.create().build().execute(new HttpGet(uri));

Implementing Server-Side Load Balancing

Server-side load balancing uses a dedicated load balancer to distribute incoming requests among multiple server instances. This approach centralizes load balancing and can handle various distribution strategies.

Example using HAProxy

The following example demonstrates how to set up HAProxy for server-side load balancing, including installing HAProxy, configuring it to distribute requests among multiple servers, and starting the service to manage load distribution effectively −

Install HAProxy: Install HAProxy on your server. This tool will act as the load balancer for distributing requests −

sudo apt-get install haproxy

Configure HAProxy: Set up the HAProxy configuration file (haproxy.cfg) to define how requests should be distributed among servers −

frontend myfrontend
   bind *:80
   default_backend mybackend

backend mybackend
   balance roundrobin
   server server1 localhost:8081 check
   server server2 localhost:8082 check

Here,

frontend myfrontend: Configures HAProxy to listen on port 80 and forward requests to the back-end.
backend mybackend: Defines the servers to which requests will be routed, using a round-robin load balancing strategy.

Start HAProxy: Start the HAProxy service to begin load balancing requests based on your configuration.

sudo service haproxy start

Service Discovery

Service discovery is the method by which a system automatically detects and maintains a list of available service instances.

This dynamic process allows clients to locate and connect to services without needing hard coded addresses, making it easier to manage and scale services in a distributed environment.

Types of Service Discovery

Following are the primary types of service discovery −

Client-Side Service Discovery

In this approach, the client queries a service registry to obtain a list of available service instances and then selects one to connect to. This method gives the client control over how it connects to services.

Example: Using libraries like Eureka in Java for managing service instance information.

Server-Side Service Discovery

Here, the client sends requests to a load balancer, which then queries the service registry and forwards the request to an appropriate service instance. This method centralizes the discovery process and simplifies client configuration.

Example: Using tools like Consul in combination with NGINX for managing service instance routing.

Implementing Client-Side Service Discovery

Client-side service discovery involves using a service registry to dynamically locate and connect to available service instances.

Example in Java using Eureka

The following example demonstrates how to integrate Eureka for client-side service discovery in Java, enabling the application to dynamically locate and connect to available service instances −

Include Eureka Client Dependency: Add the Eureka client dependency to your "pom.xml" to enable service discovery features in your Java application −

<dependency>
  <groupId>com.netflix.eureka</groupId>
  <artifactId>eureka-client</artifactId>
  <version>1.10.11</version>
</dependency>

Configure Eureka Client: Set up the Eureka client configuration to specify the URL of the Eureka server −

eureka.client.serviceUrl.defaultZone=http://localhost:8761/eureka/

Discover Services: Use the Eureka client to query the service registry, retrieve available instances, and connect to a specific instance −

Application application = eurekaClient.getApplication("myservice");
InstanceInfo instanceInfo = application.getInstances().get(0);
URI uri = new URI("http://" + instanceInfo.getIPAddr() + ":" + instanceInfo.getPort() + "/path");
HttpResponse response = HttpClientBuilder.create().build().execute(new HttpGet(uri));

Implementing Server-Side Service Discovery

Server-side service discovery integrates a service registry with a load balancer to manage request routing.

Example using Consul with NGINX

This example shows how to use Consul for server-side service discovery with NGINX, allowing NGINX to route requests to services registered with Consul for dynamic load balancing and failover −

Install Consul: Install Consul on your system to enable service registration and discovery −

sudo apt-get install consul

Register Services with Consul: Create a JSON configuration file to register your service with Consul, including health checks −

{
  "service": {
    "name": "myservice",
    "port": 8081,
    "check": {
      "http": "http://localhost:8081/health",
      "interval": "10s"
    }
  }
}

Configure NGINX to Use Consul: Configure NGINX to route requests to the service instances registered with Consul −

http {
   upstream myservice {
      server localhost:8081;
      server localhost:8082;
   }

   server {
      listen 80;
      location / {
         proxy_pass http://myservice;
      }
   }
}

Start NGINX: Start or restart NGINX to apply the new configuration and begin load balancing requests −

sudo service nginx start

Save or share this article