Scale with Redis Cluster Horizontal scaling with Redis Cluster

Redis scales horizontally with a deployment topology called Redis Cluster. This topic will teach you how to set up, test, and operate Redis Cluster in production. You will learn about the availability and consistency characteristics of Redis Cluster from the end user's point of view.

If you plan to run a production Redis Cluster deployment or want to understand better how Redis Cluster works internally, consult the Redis Cluster specification. To learn how Redis Enterprise handles scaling, see Linear Scaling with Redis Enterprise.

Redis Cluster 101 Redis Cluster provides a way to run a Redis installation where data is automatically sharded across multiple Redis nodes. Redis Cluster also provides some degree of availability during partitions—in practical terms, the ability to continue operations when some nodes fail or are unable to communicate. However, the cluster will become unavailable in the event of larger failures (for example, when the majority of masters are unavailable).

So, with Redis Cluster, you get the ability to:

Automatically split your dataset among multiple nodes. Continue operations when a subset of the nodes are experiencing failures or are unable to communicate with the rest of the cluster. Redis Cluster TCP ports Every Redis Cluster node requires two open TCP connections: a Redis TCP port used to serve clients, e.g., 6379, and second port known as the cluster bus port. By default, the cluster bus port is set by adding 10000 to the data port (e.g., 16379); however, you can override this in the cluster-port configuration.

Cluster bus is a node-to-node communication channel that uses a binary protocol, which is more suited to exchanging information between nodes due to little bandwidth and processing time. Nodes use the cluster bus for failure detection, configuration updates, failover authorization, and so forth. Clients should never try to communicate with the cluster bus port, but rather use the Redis command port. However, make sure you open both ports in your firewall, otherwise Redis cluster nodes won't be able to communicate.

For a Redis Cluster to work properly you need, for each node:

The client communication port (usually 6379) used to communicate with clients and be open to all the clients that need to reach the cluster, plus all the other cluster nodes that use the client port for key migrations. The cluster bus port must be reachable from all the other cluster nodes. If you don't open both TCP ports, your cluster will not work as expected.

Redis Cluster and Docker Currently, Redis Cluster does not support NATted environments and in general environments where IP addresses or TCP ports are remapped.

Docker uses a technique called port mapping: programs running inside Docker containers may be exposed with a different port compared to the one the program believes to be using. This is useful for running multiple containers using the same ports, at the same time, in the same server.

To make Docker compatible with Redis Cluster, you need to use Docker's host networking mode. Please see the --net=host option in the Docker documentation for more information.

Redis Cluster data sharding Redis Cluster does not use consistent hashing, but a different form of sharding where every key is conceptually part of what we call a hash slot.

There are 16384 hash slots in Redis Cluster, and to compute the hash slot for a given key, we simply take the CRC16 of the key modulo 16384.

Every node in a Redis Cluster is responsible for a subset of the hash slots, so, for example, you may have a cluster with 3 nodes, where: