Redis

Common Scenarios

Scenario Applications
Caching Data caching, session management, page caching
Counters Visit counts, online users, PV, UV
Message queues Async tasks, real-time message push, event notification
Distributed locks Distributed transactions, resource coordination, preventing duplicate submission
Geolocation LBS, social applications, ride-hailing services
Full-text search Search engines, text analysis, keyword extraction

Basic Data Structures

Structure Characteristics Scenarios
String The most basic type; binary-safe, it can store any kind of data Caching, counters, rate limiting, distributed locks
List Stores an ordered list of strings, supporting insertion and popping from both ends Message queues, task queues, leaderboards, timelines
Set Stores multiple unique strings and supports set operations such as union, intersection, and difference Tag systems, friend relationships, voting systems, recommendation systems
Hash Stores multiple key-value pairs, supporting add, delete, and get on individual fields Object storage, product attributes, user information, configuration
Zset (sorted set) Stores multiple members, each with a score, supporting ordering by score and range lookup Leaderboards, scoreboards, game rankings

Deployment Approaches

Approach Description Scenarios
Standalone Redis runs on a single server, without data sharding, using one configuration file. Small-scale environments such as development and testing.
Primary-replica replication Redis instances are divided into primary and replica nodes. The primary handles writes, replicas handle reads. If the primary fails, a replica can be promoted automatically. Applications with high data consistency requirements, such as e-commerce and finance.
Sentinel Sentinel is Redis's high availability solution: multiple Sentinel processes monitor the Redis primary and replicas, and promote a replica automatically when the primary fails. Applications needing high availability and data consistency, such as finance and healthcare.
Cluster Redis Cluster distributes data across multiple nodes for storage and management, supporting automatic sharding and load balancing. Applications needing high scalability, such as internet and IoT applications.

Configuration Requirements

Approach Memory CPU Disk Concurrency Response target
Standalone 1GB-4GB Single or dual core 10GB-100GB 10K-100K QPS Response time generally in the millisecond range
Primary-replica replication 4GB-32GB Four or eight cores 100GB-1TB 10K-100K QPS Writes on the primary generally in the millisecond range; reads on replicas faster
Sentinel 8GB-64GB Four or eight cores 100GB-1TB 10K-100K QPS Generally milliseconds; failover when the primary fails is generally in the seconds range
Cluster 32GB-512GB Multiple cores Multiple disks 100K-1M QPS Response time generally in the millisecond range

Implementing a Distributed Lock

Redis can implement a distributed lock with the SETNX (set if not exists) command. The steps are:

  1. Define a key that names the lock, for example lock:my_lock.
  2. Before acquiring the lock, set an expiry time so that a crashed client or a network fault does not leave the lock held forever. Use SETNX to set the value: if it returns 1, the set succeeded and the lock is acquired; if it returns 0, the lock is already held and acquisition failed.
  3. If the lock was acquired, release it promptly after finishing the task by deleting the key with the DEL command.

A simple example:

def get_lock(conn, lock_name, expire_time=10):
    # try to acquire the lock
    is_locked = conn.setnx(lock_name, 'locked')
    if is_locked:
        # set the lock expiry
        conn.expire(lock_name, expire_time)
        return True
    else:
        return False

def release_lock(conn, lock_name):
    # release the lock
    conn.delete(lock_name)

When using a distributed lock, take network latency and lock granularity into account. And to avoid deadlock, set a sensible expiry and check on release that the lock is the one you hold.

Redis Key Expiry Mechanisms

Mechanism Description Scenarios Pros Cons
Timed expiry When an expiry time is set on a key, Redis deletes it automatically when that time arrives Scenarios needing automatic cleanup of expired data, such as caching Simple to implement and understand; ensures expired keys are deleted promptly Blocks the main thread when a key expires, which may hurt performance; cannot handle a key updated or deleted before it expires
Lazy expiry Once a key expires, it is deleted only when accessed Scenarios with very large cached data, avoiding a mass deletion that hurts performance Reduces the performance impact of expiry and optimizes memory use; handles keys updated or deleted before expiry Cannot guarantee prompt deletion; some expired keys may linger
Periodic expiry Redis scans for expired keys periodically and deletes them, ensuring not too many linger Scenarios needing expired keys removed but unable to absorb a mass deletion at once Ensures reasonably prompt deletion with relatively small performance impact; handles keys updated or deleted before expiry Cannot guarantee prompt deletion; the scan interval needs tuning for the actual situation

Redis Key Eviction Policies

Policy Description Scenarios Pros Cons
LRU (Least Recently Used) Evicts the least recently used keys Read-heavy, write-light scenarios such as caching Ensures frequently used keys survive; simple to implement and performant Hurts write performance; unsuited to irregular access patterns
LFU (Least Frequently Used) Evicts the least frequently accessed keys Scenarios with fairly stable access frequency, such as hot data Ensures frequently accessed keys survive; adapts to changes in access patterns Requires tracking access counts, adding memory overhead; unsuited to rapidly shifting access patterns
Random Picks a key at random to evict Any scenario Simple, with no access counting needed Cannot ensure frequently used keys survive; eviction is inefficient
TTL Evicts the key with the shortest remaining time Time-sensitive data such as session data Ensures keys with short expiry do not occupy too much memory Only applies to time-sensitive data; cannot handle frequently accessed keys

The stop the world Problem

Redis is a single-threaded in-memory database, meaning it uses one main thread to handle all client requests. Redis is designed around non-blocking IO, so some operations do not block the main thread. That avoids most "stop the world" problems.

However, some operations can still block the main thread. For example, commands such as BGSAVE, BGREWRITEAOF, FLUSHALL, and FLUSHDB perform a persistence operation that writes in-memory data to disk. During these operations the Redis main thread is blocked until the operation finishes. If the database is very large, this can take a while, causing a long pause.

Blocking can also occur when the main thread performs a lot of computation. For example, running a complex Lua script or a large set operation can occupy the main thread, so client requests cannot be answered promptly.

So while Redis avoids "stop the world" in most cases, it can still occur in some. To avoid it, Redis usage needs careful planning and tuning so that operations complete within acceptable time.

Competitor Analysis

Competitor Differences Pros Cons Scenarios
Memcached - No persistence
- No data structures
- Supports distributed locks
- Simple and easy to use
- High performance
- High concurrency
- Efficient memory use
- Only caching scenarios
- No persistence
- No complex data structures
High-concurrency caching with a high read/write ratio and large data volumes
RocksDB - Storage medium not limited to memory
- Can persist data on disk
- Supports transactions
- Supports many data structures
- High performance
- Highly scalable
- No distributed support Scenarios needing persistent storage of large data volumes
LevelDB - Storage medium not limited to memory
- Can persist data on disk
- High performance
- Supports transactions
- Lightweight
- No complex data structures
- No distributed support
Scenarios needing persistent storage of large data volumes
Cassandra - Supports distribution
- Horizontally scalable
- Supports multi-datacenter deployment
- High availability
- High reliability
- High scalability
- High complexity
- Costly to deploy and maintain
Scenarios needing to store massive data with high availability and reliability
MongoDB - Supports complex data structures
- Supports transactions
- High availability
- High flexibility
- Supports distributed deployment
- Weaker performance
- No ACID support
Scenarios needing complex data structures with high availability and flexibility

powered by ChatGPT

results matching ""

    No results matching ""