Redis
Common Scenarios
| Scenario | Applications |
|---|---|
| Caching | Data caching, session management, page caching |
| Counters | Visit counts, online users, PV, UV |
| Message queues | Async tasks, real-time message push, event notification |
| Distributed locks | Distributed transactions, resource coordination, preventing duplicate submission |
| Geolocation | LBS, social applications, ride-hailing services |
| Full-text search | Search engines, text analysis, keyword extraction |
Basic Data Structures
| Structure | Characteristics | Scenarios |
|---|---|---|
| String | The most basic type; binary-safe, it can store any kind of data | Caching, counters, rate limiting, distributed locks |
| List | Stores an ordered list of strings, supporting insertion and popping from both ends | Message queues, task queues, leaderboards, timelines |
| Set | Stores multiple unique strings and supports set operations such as union, intersection, and difference | Tag systems, friend relationships, voting systems, recommendation systems |
| Hash | Stores multiple key-value pairs, supporting add, delete, and get on individual fields | Object storage, product attributes, user information, configuration |
| Zset (sorted set) | Stores multiple members, each with a score, supporting ordering by score and range lookup | Leaderboards, scoreboards, game rankings |
Deployment Approaches
| Approach | Description | Scenarios |
|---|---|---|
| Standalone | Redis runs on a single server, without data sharding, using one configuration file. | Small-scale environments such as development and testing. |
| Primary-replica replication | Redis instances are divided into primary and replica nodes. The primary handles writes, replicas handle reads. If the primary fails, a replica can be promoted automatically. | Applications with high data consistency requirements, such as e-commerce and finance. |
| Sentinel | Sentinel is Redis's high availability solution: multiple Sentinel processes monitor the Redis primary and replicas, and promote a replica automatically when the primary fails. | Applications needing high availability and data consistency, such as finance and healthcare. |
| Cluster | Redis Cluster distributes data across multiple nodes for storage and management, supporting automatic sharding and load balancing. | Applications needing high scalability, such as internet and IoT applications. |
Configuration Requirements
| Approach | Memory | CPU | Disk | Concurrency | Response target |
|---|---|---|---|---|---|
| Standalone | 1GB-4GB | Single or dual core | 10GB-100GB | 10K-100K QPS | Response time generally in the millisecond range |
| Primary-replica replication | 4GB-32GB | Four or eight cores | 100GB-1TB | 10K-100K QPS | Writes on the primary generally in the millisecond range; reads on replicas faster |
| Sentinel | 8GB-64GB | Four or eight cores | 100GB-1TB | 10K-100K QPS | Generally milliseconds; failover when the primary fails is generally in the seconds range |
| Cluster | 32GB-512GB | Multiple cores | Multiple disks | 100K-1M QPS | Response time generally in the millisecond range |
Implementing a Distributed Lock
Redis can implement a distributed lock with the SETNX (set if not exists) command. The steps are:
- Define a key that names the lock, for example lock:my_lock.
- Before acquiring the lock, set an expiry time so that a crashed client or a network fault does not leave the lock held forever. Use SETNX to set the value: if it returns 1, the set succeeded and the lock is acquired; if it returns 0, the lock is already held and acquisition failed.
- If the lock was acquired, release it promptly after finishing the task by deleting the key with the DEL command.
A simple example:
def get_lock(conn, lock_name, expire_time=10):
# try to acquire the lock
is_locked = conn.setnx(lock_name, 'locked')
if is_locked:
# set the lock expiry
conn.expire(lock_name, expire_time)
return True
else:
return False
def release_lock(conn, lock_name):
# release the lock
conn.delete(lock_name)
When using a distributed lock, take network latency and lock granularity into account. And to avoid deadlock, set a sensible expiry and check on release that the lock is the one you hold.
Redis Key Expiry Mechanisms
| Mechanism | Description | Scenarios | Pros | Cons |
|---|---|---|---|---|
| Timed expiry | When an expiry time is set on a key, Redis deletes it automatically when that time arrives | Scenarios needing automatic cleanup of expired data, such as caching | Simple to implement and understand; ensures expired keys are deleted promptly | Blocks the main thread when a key expires, which may hurt performance; cannot handle a key updated or deleted before it expires |
| Lazy expiry | Once a key expires, it is deleted only when accessed | Scenarios with very large cached data, avoiding a mass deletion that hurts performance | Reduces the performance impact of expiry and optimizes memory use; handles keys updated or deleted before expiry | Cannot guarantee prompt deletion; some expired keys may linger |
| Periodic expiry | Redis scans for expired keys periodically and deletes them, ensuring not too many linger | Scenarios needing expired keys removed but unable to absorb a mass deletion at once | Ensures reasonably prompt deletion with relatively small performance impact; handles keys updated or deleted before expiry | Cannot guarantee prompt deletion; the scan interval needs tuning for the actual situation |
Redis Key Eviction Policies
| Policy | Description | Scenarios | Pros | Cons |
|---|---|---|---|---|
| LRU (Least Recently Used) | Evicts the least recently used keys | Read-heavy, write-light scenarios such as caching | Ensures frequently used keys survive; simple to implement and performant | Hurts write performance; unsuited to irregular access patterns |
| LFU (Least Frequently Used) | Evicts the least frequently accessed keys | Scenarios with fairly stable access frequency, such as hot data | Ensures frequently accessed keys survive; adapts to changes in access patterns | Requires tracking access counts, adding memory overhead; unsuited to rapidly shifting access patterns |
| Random | Picks a key at random to evict | Any scenario | Simple, with no access counting needed | Cannot ensure frequently used keys survive; eviction is inefficient |
| TTL | Evicts the key with the shortest remaining time | Time-sensitive data such as session data | Ensures keys with short expiry do not occupy too much memory | Only applies to time-sensitive data; cannot handle frequently accessed keys |
The stop the world Problem
Redis is a single-threaded in-memory database, meaning it uses one main thread to handle all client requests. Redis is designed around non-blocking IO, so some operations do not block the main thread. That avoids most "stop the world" problems.
However, some operations can still block the main thread. For example, commands such as BGSAVE, BGREWRITEAOF, FLUSHALL, and FLUSHDB perform a persistence operation that writes in-memory data to disk. During these operations the Redis main thread is blocked until the operation finishes. If the database is very large, this can take a while, causing a long pause.
Blocking can also occur when the main thread performs a lot of computation. For example, running a complex Lua script or a large set operation can occupy the main thread, so client requests cannot be answered promptly.
So while Redis avoids "stop the world" in most cases, it can still occur in some. To avoid it, Redis usage needs careful planning and tuning so that operations complete within acceptable time.
Competitor Analysis
| Competitor | Differences | Pros | Cons | Scenarios |
|---|---|---|---|---|
| Memcached | - No persistence - No data structures - Supports distributed locks |
- Simple and easy to use - High performance - High concurrency - Efficient memory use |
- Only caching scenarios - No persistence - No complex data structures |
High-concurrency caching with a high read/write ratio and large data volumes |
| RocksDB | - Storage medium not limited to memory - Can persist data on disk - Supports transactions |
- Supports many data structures - High performance - Highly scalable |
- No distributed support | Scenarios needing persistent storage of large data volumes |
| LevelDB | - Storage medium not limited to memory - Can persist data on disk |
- High performance - Supports transactions - Lightweight |
- No complex data structures - No distributed support |
Scenarios needing persistent storage of large data volumes |
| Cassandra | - Supports distribution - Horizontally scalable - Supports multi-datacenter deployment |
- High availability - High reliability - High scalability |
- High complexity - Costly to deploy and maintain |
Scenarios needing to store massive data with high availability and reliability |
| MongoDB | - Supports complex data structures - Supports transactions |
- High availability - High flexibility - Supports distributed deployment |
- Weaker performance - No ACID support |
Scenarios needing complex data structures with high availability and flexibility |
powered by ChatGPT