How To Configure a Redis Cluster on Ubuntu 14.04

Introduction #

Redis is an open source key-value data store, using an in-memory storage model with optional disk writes for persistence. It features transactions, pub/sub, and automatic failover, among other functionality. It is recommended to use Redis with Linux for production environments, but the developers also mention OS X as a platform on which they develop and test. Redis has clients written in most languages, with recommended ones featured on their website.
For production environments, replicating your data across at least two nodes is considered the best practice. Redundancy allows for recovery in case of environment failure, which is especially important when the user base of your application grows.
By the end of this guide, we will have set up two Redis Droplets on DigitalOcean, as follows:

one Droplet for the Redis master server
one Droplet for the Redis slave server

We will also demonstrate how to switch to the slave server and set it up as a temporary master.
Feel free to set up more than one slave server.
This article focuses on setting up a master-slave Redis cluster; to learn more about Redis in general and its basic usage as a database, see this usage tutorial.

Prerequisites #

While this may work on earlier releases and other Linux distributions, we recommend Ubuntu 14.04.
For testing purposes, we will use small instances as there is no real workload to be handled, but production environments may require larger servers.

Ubuntu 14.04 LTS
Two Droplets, of any size you need; one master and one or more slave(s)
Access to your machines via SSH with a sudo non-root user as explained in Initial Server Setup with Ubuntu 14.04

Step 1 — Install Redis #

Starting with the Droplet that will host our master server, our first step is to install Redis. First we need to add Chris Lea’s Redis repository (as always, take extreme caution when adding third party repositories; we are using this one because its maintainer is a reputable figure):

sudo add-apt-repository ppa:chris-lea/redis-server

Press ENTER to accept the repository.
Run the following command to update our packages:

sudo apt-get update

Install the Redis server:

sudo apt-get install redis-server

Check that Redis is up and running:

redis-benchmark -q -n 1000 -c 10 -P 5

The above command is saying that we want redis-benchmark to run in quiet mode, with 1000 total requests, 10 parallel connections and pipeline 5 requests. For more information on running benchmarks for Redis, typing redis-benchmark --help in your terminal will print useful information with examples.
Let the benchmark run. After it’s finished, you should see output similar to the following:

PING_INLINE: 166666.67 requests per second
PING_BULK: 249999.98 requests per second
SET: 249999.98 requests per second
GET: 499999.97 requests per second
INCR: 333333.34 requests per second
LPUSH: 499999.97 requests per second
LPOP: 499999.97 requests per second
SADD: 499999.97 requests per second
SPOP: 499999.97 requests per second
LPUSH (needed to benchmark LRANGE): 499999.97 requests per second
LRANGE_100 (first 100 elements): 111111.12 requests per second
LRANGE_300 (first 300 elements): 27777.78 requests per second
LRANGE_500 (first 450 elements): 8333.33 requests per second
LRANGE_600 (first 600 elements): 6369.43 requests per second
MSET (10 keys): 142857.14 requests per second

Now repeat this section for the Redis slave server. If you are configuring more Droplets, you may set up as many slave servers as necessary.
At this point, Redis is installed and running on our two nodes. If the output of any node is not similar to what is shown above, repeat the setup process carefully and check that all prerequisites are met

Step 2 — Configure Redis Master #

Now that Redis is up and running on our two-Droplet cluster, we have to edit their configuration files. As we will see, there are minor differences between configuring the master server and the slave.
Let’s first start with our master.
Open /etc/redis/redis.conf with your favorite text editor:

sudo nano /etc/redis/redis.conf

Edit the following lines.
Set a sensible value to the keepalive timer for TCP:
/etc/redis/redis.conf

tcp-keepalive 60

Make the server accessible to anyone on the web by commenting out this line:
/etc/redis/redis.conf

#bind 127.0.0.1

Given the nature of Redis, and its very high speeds, an attacker may brute force the password without many issues. That is why we recommend uncommenting the requirepass line and adding a complex password (or a complex passphrase, preferably):
/etc/redis/redis.conf

requirepass your_redis_master_password

Depending on your usage scenario, you may change the following line or not. For the purpose of this tutorial, we assume that no key deletion must occur. Uncomment this line and set it as follows:
/etc/redis/redis.conf

maxmemory-policy noeviction

Finally, we want to make the following changes, required for backing up data. Uncomment and/or set these lines as shown:
/etc/redis/redis.conf

appendonly yes
appendfilename redis-staging-ao.aof

Save your changes.
Restart the Redis service to reload our configuration changes:

sudo service redis-server restart

If you want to go the extra mile, you can add some unique content to the master database by following the Redis Operations sections in this tutorial, so we can later see how it gets replicated to the slave server.
Now that we have the master server ready, let’s move on to our slave machine.

Step 3 — Configure Redis Slave #

We need to make some changes that allow our slave server to connect to our master instance:
Open /etc/redis/redis.conf with your favorite text editor:

sudo nano /etc/redis/redis.conf

Edit the following lines; some settings will be similar to the master’s.
Make the server accessible to anyone on the web by commenting out this line:
/etc/redis/redis.conf

#bind 127.0.0.1

The slave server needs a password as well so we can give it commands (such as INFO). Uncomment this line and set a server password:
/etc/redis/redis.conf

requirepass your_redis_slave_password

Uncomment this line and indicate the IP address where the master server can be reached, followed by the port set on that machine. By default, the port is 6379:
/etc/redis/redis.conf

slaveof your_redis_master_ip 6379

Uncomment the masterauth line and provide the password/passphrase you set up earlier on the master server:
/etc/redis/redis.conf

masterauth your_redis_master_password

Now save these changes, and exit the file. Next, restart the service like we did on our master server:

sudo service redis-server restart

This will reinitialize Redis and load our modified files.
Connect to Redis:

redis-cli -h 127.0.0.1 -p 6379 

Authorize with the slave server’s password:

AUTH your_redis_slave_password

At this point we are running a functional master-slave Redis cluster, with both machines properly configured.

Step 4 — Verify the Master-Slave Replication #

Testing our setup will allow us to better understand the behavior of our Redis Droplets, once we want to start scripting failover behavior. What we want to do now is make sure that our configuration is working correctly, and our master is talking with the slave Redis instances.
First, we connect to Redis via our terminal, on the master server:
First connect to the local instance, running by default on port 6379. In case you’ve changed the port, modify the command accordingly.

redis-cli -h 127.0.0.1 -p 6379

Now authenticate with Redis with the password you set when configuring the master:

AUTH your_redis_master_password

And you should get an OK as a response. Now, you only have to run:

INFO

You will see everything you need to know about the master Redis server. We are especially interested in the #Replication section, which should look like the following output:

. . .

# Replication
role:master
connected_slaves:1
slave0:ip=111.111.111.222,port=6379,state=online,offset=407,lag=1
master_repl_offset:407
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:406

. . .

Notice the connected_slaves:1 line, which indicates our other instance is talking with the master Droplet. You can also see that we get the slave IP address, along with port, state, and other info.
Let’s now take a look at the #Replication section on our slave machine. The process is the same as for our master server. Log in to the Redis instance, issue the INFO command, and view the output:

. . .

# Replication
role:slave
master_host:111.111.111.111
master_port:6379
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:1401
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

. . .

We can see that this machine has the role of slave, is communicating with the master Redis server, and has no slaves of its own.

Step 5 — Switch to the Slave #

Building this architecture means that we also want failures to be handled in such a way that we ensure data integrity and as little downtime as possible for our application. Any slave can be promoted to be a master. First, let’s test switching manually.
On a slave machine, we should connect to the Redis instance:

redis-cli -h 127.0.0.1 -p 6379

Now authenticate with Redis with the password you set when configuring the slave

AUTH your_redis_slave_password

Turn off slave behavior:

SLAVEOF NO ONE

The response should be OK. Now type:

INFO

Look for the # Replication section to find the following output:

. . .

# Replication
role:master
connected_slaves:0
master_repl_offset:1737
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

. . .

As we expected, the slave has turned into a master, and is now ready to accept connections from other machines (if any). We can use it as a temporary backup while we debug our main master server.

If you have multiple slaves that depended on the initial master, they all have to be pointed towards the newly promoted master.

This can be scripted easily, with the following steps needing to be implemented once a failure is detected:

From the application, send all requests for Redis to a slave machine
On that slave, execute the SLAVEOF NO ONE command. Starting with Redis version 1.0.0, this command tells the slave to stop replicating data, and start acting as a master server
On all remaining slaves (if any), running SLAVEOF hostname port will instruct them to stop replicating from the old master, discard the now deprecated data completely, and start replicating from the new master. Make sure to replace hostname and port with the correct values, from your newly promoted master
After analyzing the issue, you may return to having your initial server as master, if your particular setup requires it

There are many ways of accomplishing the steps explained above. However, it is up to you to implement an adequate solution for your environment, and make sure to test it thoroughly before any actual failures occur.

Step 6 — Reconnect to the Master #

Let’s reconnect to the original master server. On the slave server, log in to Redis and execute the following:

SLAVEOF your_redis_master_ip 6379

If you run the INFO command again, you should see we have returned to the original setup.

Conclusion #

We have properly set up an enviroment consisting of two servers, one acting as Redis master, and the other replicating data as a slave. This way, if the master server ever goes offline or loses our data, we know how to switch to one of our slaves for recovery until the issue is taken care of.
Next steps might include scripting the automated failover procedure, or ensuring secure communications between all your Droplets by the use of VPN solutions such as OpenVPN or Tinc. Also, testing procedures and scripts are vital for validating your configurations.
Additionally, you should take precautions when deploying this kind of setup in production environments. The Redis Documentation page should be studied and you must have a clear understanding of what security model is adequate for your application. We often use Redis as a session store, and the information it contains can be valuable to an attacker. Common practice is to have these machines accessible only via private network, and place them behind multiple layers of security.
This is a simple starting point on which your data store may be built; by no means an exhaustive guide on setting up Redis to use master-slave architecture. If there is anything that you consider this guide should cover, please leave comments below. For more information and help on this topic, the DigitalOcean Q&A is a good place to start.