Chapter 36 Isolation and Locking j 315

activity records associated with a given messageid:

1select count(*) filter(where action = 'rt')

2- count(*) filter(where action = 'de-rt')

3as rts,

4count(*) filter(where action = 'fav')

5- count(*) filter(where action = 'de-fav')

6as favs

7from tweet.activity

9where messageid = :id;

Reading the current counter value has become quite complex when compared to just adding a column to your query output list. On the other hand, when adding a rt or a fav action to a message, we transform the SQL:

1update tweet.message set rts = rts +1 where messageid = :id;

This is what we use instead:

1insert into tweet.activity(messageid, action) values(:id, 'rt');

The reason why replacing an update with an insert is interesting is concurrency behavior and locking. In the rst version, retweeting has to wait until all concurrent retweets are done, and the business model wants to sustain as many concurrent activities on the same small set of messages as possible (read about influencer accounts).

The insert has no concurrency because it targets a row that doesn’t exist yet. We register each action into its own tuple and require no locking to do that, allowing our production setup of PostgreSQL to sustain a much larger load.

Now, computing the counters each time we want to display them is costly. And the counters are displayed on every tweet message. We need a way to cache that information, and we’ll see about that in the Computing and Caching in SQL section.

Putting Concurrency to the Test

When we benchmark the concurrency properties of the two statements above, we quickly realize that the activity table is badly designed. The unique constraint includes a timestamptz eld, which in PostgreSQL is only precise down to the microsecond.

Chapter 36 Isolation and Locking j 316

This kind of made-up unique constraint means we now have these errors to deal with:

The best course of action here is to do this:

1alter table tweet.activity

2drop constraint activity_messageid_datetime_action_key;

Now we can properly compare the concurrency scaling of the insert and the update based version. In case you might be curious about it, here’s the testing code that’s been used:

1 (defpackage #:concurrency

2(:use #:cl #:appdev)

3(:import-from #:lparallel

8(:import-from #:cl-postgres-error

17(defparameter *insert-rt*

18"insert into tweet.activity(messageid, action) values($1, 'rt')")

19

20(defparameter *update-rt*

21"update tweet.message set rts = coalesce(rts, 0) + 1 where messageid = $1")

25(format t "Starting benchmark for updates~%")

26(with-timing (rts seconds)

27(run-workers workers retweets messageid *update-rt* connspec)

28(format t "Updating took ~f seconds, did ~d rts~%" seconds rts))

32(format t "Starting benchmark for inserts~%")

33(with-timing (rts seconds)

34(run-workers workers retweets messageid *insert-rt* connspec)

35(format t "Inserting took ~f seconds, did ~d rts~%" seconds rts)))

Chapter 36 Isolation and Locking j 317

39(let* ((*kernel* (lparallel:make-kernel workers))

40(channel (lparallel:make-channel)))

41(loop repeat workers

42do (lparallel:submit-task channel #'retweet-many-times

Chapter 36 Isolation and Locking j 318

for a free slot.

In another test with even more concurrency pressure at 50 retweets per worker, we can show that the results are repeatable:

1 CL-USER> (concurrency::concurrency-test 100 50 6)

2Starting benchmark for updates

3Updating took 5.070135 seconds, did 5000 rts

4

5Starting benchmark for inserts

6Inserting took 3.739505 seconds, did 5000 rts

If you know that your application has to scale, think about how to avoid concurrent activity that competes against a single shared resource. Here, this shared resource is the rts eld of the tweet.message row that you target, and the concurrency behavior is going to be ne if the retweet activity is well distributed. As soon as many users want to retweet the same message, then the update solution has a non-trivial scalability impact.

Now, we’re going to implement the tweet.activity based model. In this model, the number of retweets needs to be computed each time we display it, and it’s part of the visible data. Also, in the general case, it’s impossible for our users to know for sure how many retweets have been made so that we can implement a cache with eventual consistency properties.