The first part mainly covers optimistic and pessimistic locking. Locking guarantees data consistency under concurrency at the database level. Understanding locks helps you understand concurrency solutions. The second part mainly covers how to guarantee data consistency in concurrent scenarios within Django.

1. About Locks

1.1 Optimistic Locking

The starting point of optimistic locking is that the same row of data rarely conflicts from concurrent modification; it suits read-heavy, write-light scenarios and is used to improve throughput.

The implementation: read a field, run the processing logic, and when the data needs to be updated, check once more whether that field is still identical to what was read the first time. If it is, update the data; otherwise reject the update, re-read, and submit again.

1.2 Pessimistic Locking

The starting point of pessimistic locking is that while a row of data is being modified, no other operation on that row is allowed.

The implementation: after reading a field, take a lock and disallow any other read or write operations. Run the processing logic, and release the lock once the data has been updated.

1.3 Comparison

The overhead of optimistic locking is far lower than that of pessimistic locking.

Pessimistic locking can cause deadlock. When A has locked resource a and needs resource b, while b is locked by B and B is waiting for resource a, a deadlock results. This problem can, however, be handled by setting a timeout.

Pessimistic locking can effectively reduce the number of retries after a conflict.

Optimistic locking can improve response speed.

2. Transactions in Django

By default, Django commits every database operation to the database immediately.

This leads to a problem: if a series of database operations must either all execute or none of them execute, what do you do?

This is where transactions come in. Set a series of database operations as one transaction and submit it to the database for execution.

Django provides the atomic decorator to start a transaction.
atomic takes a parameter that specifies the database name. If you do not set a value, Django uses the system’s default database.

2.1 Opening a Transaction for the Whole View Function

1
2
3
4
5
6

from django.db import transaction

@transaction.atomic
def viewfunc(request):
    # This code executes inside a transaction.
    do_stuff()

2.2 Opening a Transaction for Part of a Function, do_more_stuff().

1
2
3
4
5
6
7
8
9

from django.db import transaction

def viewfunc(request):
    # This code executes in autocommit mode (Django's default).
    do_stuff()

    with transaction.atomic():
        # This code executes inside a transaction.
        do_more_stuff()

2.3 Do Not Handle Exceptions Inside a Transaction

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

from django.db import transaction

def viewfunc(request):
    do_stuff()
    try:
        with transaction.commit_on_success():
            do_more_stuff_1() # in transaction
            try:
                do_more_stuff_2() # not in transaction
            except:
                pass
            do_more_stuff_3() # in transaction
    except:
        pass

When exiting an atomic block, Django checks whether it exited normally or whether there was an exception to decide whether to commit or roll back.

If you catch and handle an exception inside an atomic block, you may hide the fact that a problem occurred in Django. This can cause unexpected behavior.

3. Using the F Function to Update Computations

Normally, updating data in the database requires reading the object into memory. You modify it in memory and then write it back to the database.

If operations in memory happen concurrently, the computation logic can end up wrong.

What the F() function does is generate the SQL statement directly, so the object that needs updating does not have to be read into memory. This avoids the data inconsistency caused by concurrency.

1
2
3
4
5

from django.db.models import F

reporter = Reporters.objects.get(name='OICQ')
reporter.stories_filed = F('stories_filed') + 1
reporter.save()

4. Using the select_for_update Function

select_for_update uses pessimistic locking.

The select for update function uses the database query statement select ... for update to operate on the database.

This is a database-level way to solve the problem of fetching data concurrently and then modifying it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

def mark_as_readed(self, notification_id):
    # 让s elect for update 和 update 语句发生在一个完整的事务里面
    with transaction.commit_on_success():
        # 使用select_for_update 来保证并发请求同时只有一个请求在处理，其他的请求
        # 等待锁释放
        notification = Notification.objects.select_for_update().get(pk=notification_id)
        # 没有必要重复标记一个已经读过的通知
        if notication.has_readed:
            return
        notification.has_readed = True
        notification.save()
        # 在这里更新我们的计数器，嗯，我感觉好极了
        self.update_unread_count(-1)