"Missing" Keys on Redis Replicas

If you use expiring (also known as “volatile”) keys in Redis, you may be in for a surprise when you attach a new Redis replica to your Redis master: the key count on your replica may be up to 25% lower than the key count on your master. This is especially common if you have a large number of volatile keys.

Is your Redis replica missing keys? Did you just lose data? The short answer is, “no”. However, it is helpful to understand why, exactly, your Redis replica is reporting a lower key count even though you haven’t lost any data. This involves two implementation details: how Redis expires keys and how a Redis master sends your dataset to new replicas.

How does Redis expire volatile keys?

In Redis, volatile keys are not expunged from memory at the exact moment that they are set to expire. Instead, they are deleted via one of two methods:

  1. When a Redis client performs a read or write operation on a key, the Redis server will first check to see if that key exists and has an expiration time. If it exists and has an expiration time in the past, Redis will immediately expunge the key from memory before handling the command.

  2. To prevent volatile keys sticking around in memory forever if they are not accessed, Redis uses a simple passive algorithm to expire keys: Every 10 milliseconds, it grabs 100 random volatile keys and immediately expunges any keys whose expiration times have past. If 25 or more keys are expunged, Redis immediately grabs another 100 keys and begins again.

The second method above is important because it means that up to 25% of your keys may be volatile keys that are already “expired” but that have not yet been expunged from memory yet1. Redis will continue to include these keys in the “keys” and “expires” counts in your INFO output until they are expunged.

How does Redis send the dataset to new replicas?

When a Redis replica attaches itself to a Redis master, the master server creates an RDB snapshot of its dataset and sends it to the replica. And when Redis creates RDB snapshots, it does not include any keys with expiration dates in the past, even if they haven’t been expunged from memory yet.

So why is my replica’s key count lower than the master’s?

When you attach a replica to a master Redis instance, that replica is receiving a dataset that does not include the volatile keys are expired but that have not yet been expunged from memory. And because up to 25% of your key count can be keys that are expired but not yet expunged from memory, your replica may show a key count up to 25% lower than that of the master. Additionally, this is the same reason that your key count may drop when restoring a Redis server from an RDB backup.

Special thanks goes to @mattsta for help tracking down the RDB behavior.

  1. In practice, this number is usually much lower due to the fact that Redis is constantly looking for keys to delete. 

Last updated 16 Oct 2018. Originally written by Tyson Mote

← Back to docs