Consistent Hashing - System Design

Consistent hashing is a technique used in distributed systems to distribute data or requests across multiple servers while minimizing key reassignment when servers are added or removed. Unlike traditional modulo-based hashing, it uses a logical hash ring to make scaling and changes in server membership more manageable.

• Minimal Key Remapping: Only keys in affected hash ranges are reassigned when servers join or leave.
• Scalable Distribution: Supports adding or removing servers without remapping the entire keyspace.
• Improved Load Distribution: Uses virtual nodes to distribute keys more evenly across physical servers.

Example: In a distributed cache, adding a new server using traditional hashing may change the server assignments of many cached keys. Consistent hashing reduces this disruption by moving only the keys that fall into the new server's assigned range.

Issues with Traditional Hashing

Traditional hash-based distribution often assigns keys to servers using a modulo operation:

serverIndex = hash(key) % numberOfServers

This works when the number of servers remains fixed. However, adding or removing servers changes the modulo value, potentially causing many keys to be reassigned.

• Large-Scale Remapping: Changes in server count can cause many keys to map to different servers.
• Data Movement: Remapped keys may require redistribution, increasing network and storage overhead.
• Scaling Disruption: Adding or removing servers can increase operational complexity.
• Cache Misses: In distributed caches, key reassignment may temporarily reduce cache hit rates.

Consistent hashing addresses these issues by limiting reassignment to specific portions of the hash ring.

How Consistent Hashing Works

Consistent hashing uses a circular hash space, known as a hash ring, where both servers and data keys are mapped using a deterministic hash function.

• Create the Hash Ring: Define a circular range of hash values representing the available hash space.
• Place Servers: Hash each server identifier to determine its position on the ring.
• Map Keys: Hash each data key into the same hash space.
• Assign Keys: Move clockwise from a key's position and select the first server encountered.
• Handle Node Changes: When a server is added or removed, only the affected key ranges are reassigned.

Adding and Removing Servers

One of the main benefits of consistent hashing is its ability to handle changes in cluster membership without redistributing the entire keyspace.

1. Adding a Server

When a new server is added, its hash position is placed on the ring.

• The new server takes responsibility for keys in the range between its predecessor and itself.
• Only the affected keys are reassigned from the next server clockwise.
• Keys in other ranges retain their existing assignments.

2. Removing a Server

When a server fails or is removed, the ring is updated to exclude its position.

• The removed server's keys are assigned to the next available server clockwise.
• Other server assignments remain unchanged.
• Data availability depends on replication, backups, or recovery mechanisms.

Virtual Nodes

A basic consistent-hashing ring can distribute keys unevenly when each physical server occupies only one position. To improve distribution, a physical server can be represented by multiple positions called virtual nodes (vnodes).

• Improves key distribution across physical servers.
• Reduces the effect of uneven node placement.
• Makes node addition and removal smoother.
• Can assign different numbers of virtual nodes to servers with different capacities.

Implementation of Consistent Hashing algorithm

#include <bits/stdc++.h>

using namespace std;

class ConsistentHashRing {
private:
    map<int, string> ring;
    set<int> sorted_keys;
    int replicas;

    int get_hash(const string& value) {
        hash<string> hash_function;
        return hash_function(value);
    }

public:
    ConsistentHashRing(int replicas = 3) : replicas(replicas) {}

      // Function to add Node in the ring
    void add_node(const string& node) {
        for (int i = 0; i < replicas; ++i) {
            int replica_key = get_hash(node + "_" + to_string(i));
            ring[replica_key] = node;
            sorted_keys.insert(replica_key);
        }
    }

      // Function to remove Node from the ring
    void remove_node(const string& node) {
        for (int i = 0; i < replicas; ++i) {
            int replica_key = get_hash(node + "_" + to_string(i));
            ring.erase(replica_key);
            sorted_keys.erase(replica_key);
        }
    }

    string get_node(const string& key) {
        if (ring.empty()) {
            return "";
        }

        int hash_value = get_hash(key);
        auto it = sorted_keys.lower_bound(hash_value);

        if (it == sorted_keys.end()) {
            // Wrap around to the beginning of the ring
            it = sorted_keys.begin();
        }

        return ring[*it];
    }
};

int main() {
    ConsistentHashRing hash_ring;

    // Add nodes to the ring
    hash_ring.add_node("Node_A");
    hash_ring.add_node("Node_B");
    hash_ring.add_node("Node_C");

    // Get the node for a key
    string key = "first_key";
    string node = hash_ring.get_node(key);

    cout << "The key '" << key << "' is mapped to node: " << node << endl;

    return 0;
}

import java.util.*;
import java.util.stream.*;

class ConsistentHashRing {
    private Map<Integer, String> ring = new HashMap<>();
    private TreeSet<Integer> sortedKeys = new TreeSet<>();
    private int replicas;

    private int getHash(String value) {
        return value.hashCode();
    }

    public ConsistentHashRing(int replicas) {
        this.replicas = replicas;
    }

    public void addNode(String node) {
        for (int i = 0; i < replicas; ++i) {
            int replicaKey = getHash(node + "_" + i);
            ring.put(replicaKey, node);
            sortedKeys.add(replicaKey);
        }
    }

    public void removeNode(String node) {
        for (int i = 0; i < replicas; ++i) {
            int replicaKey = getHash(node + "_" + i);
            ring.remove(replicaKey);
            sortedKeys.remove(replicaKey);
        }
    }

    public String getNode(String key) {
        if (ring.isEmpty()) {
            return "";
        }

        int hashValue = getHash(key);
        Integer it = sortedKeys.ceiling(hashValue);

        if (it == null) {
            it = sortedKeys.first();
        }

        return ring.get(it);
    }
}

public class Main {
    public static void main(String[] args) {
        ConsistentHashRing hashRing = new ConsistentHashRing(3);

        hashRing.addNode("Node_A");
        hashRing.addNode("Node_B");
        hashRing.addNode("Node_C");

        String key = "first_key";
        String node = hashRing.getNode(key);

        System.out.println("The key '" + key + "' is mapped to node: " + node);
    }
}

from hashlib import md5
from bisect import bisect, insort

class ConsistentHashRing:
    def __init__(self, replicas):
        self.ring = {}
        self.sorted_keys = []
        self.replicas = replicas

    def get_hash(self, value):
        return int(md5(value.encode()).hexdigest(), 16)

    def add_node(self, node):
        for i in range(self.replicas):
            replica_key = self.get_hash(node + "_" + str(i))
            self.ring[replica_key] = node

            # Insert replica key while keeping the list sorted
            insort(self.sorted_keys, replica_key)

    def remove_node(self, node):
        for i in range(self.replicas):
            replica_key = self.get_hash(node + "_" + str(i))

            if replica_key in self.ring:
                del self.ring[replica_key]
                self.sorted_keys.remove(replica_key)

    def get_node(self, key):
        if not self.ring:
            return ""

        hash_value = self.get_hash(key)

        # Find the first server clockwise from the key
        idx = bisect(self.sorted_keys, hash_value)

        # Wrap around to the beginning of the ring
        idx = idx % len(self.sorted_keys)

        return self.ring[self.sorted_keys[idx]]


if __name__ == "__main__":
    hash_ring = ConsistentHashRing(3)

    hash_ring.add_node("Node_A")
    hash_ring.add_node("Node_B")
    hash_ring.add_node("Node_C")

    key = "first_key"
    node = hash_ring.get_node(key)

    print(f"The key '{key}' is mapped to node: {node}")

The key 'first_key' is mapped to node: Node_C

Note: This example uses a simple hash function and a binary search to find the position on the ring.

Uses of Consistent Hashing

Consistent hashing is useful in distributed systems where data or requests must be distributed across changing sets of servers.

• Distributed Caching: Maps cache keys to cache nodes while reducing remapping during cluster changes.
• Database Sharding: Can distribute records or partitions across multiple database nodes.
• Request Routing: Assigns requests or identifiers to backend servers using stable hash-based routing.
• Distributed Storage: Helps determine ownership of data partitions across storage nodes.
• Cluster Scaling: Supports server additions and removals with limited key reassignment.

Advantages of Consistent Hashing

Consistent hashing offers several benefits for managing distributed workloads.

• Minimal Key Remapping: Only the affected key ranges are reassigned when nodes are added or removed.
• Scalability: Supports incremental changes to cluster capacity without redistributing the entire keyspace.
• Improved Load Distribution: Virtual nodes can reduce uneven key distribution across physical servers.
• Reduced Data Movement: Limits the amount of data that may need relocation during cluster changes.
• Operational Flexibility: Makes cluster membership changes easier to manage compared with simple modulo-based partitioning.

Limitations of Consistent Hashing

Despite its advantages, consistent hashing has several limitations.

• Uneven Distribution: Poor hash functions or too few virtual nodes can cause unbalanced key assignments and hotspots.
• Implementation Complexity: Maintaining the ring, virtual nodes, and cluster membership requires additional logic.
• Rebalancing Overhead: Adding or removing nodes still requires data movement or cache repopulation.
• No Built-In Fault Tolerance: Consistent hashing does not automatically provide replication, failover, or data recovery.
• Key Distribution Limitations: Evenly distributing keys does not guarantee balanced server workloads because some keys may receive significantly more requests than others.