improve dequeue performances with r/sqlite

[elv-gilles]

improve dequeue performances with r/sqlite

• new priority type: iwrr (interleaved weighted round robin)
  • iwrr is available for de-queuing with r/sqlite and
  • uses buffered dequeue, limited by queue concurrency & processor concurrency
• add processor 'global' concurrency
• remove backward compatible map[string]int in QueueConfig (replaced with a constructor)

De-queueing vs Queues Priority Types

The asynq processor is initialised with queues, each of them has a priority integer.

The processor works by polling these queues, calling the Dequeue function of the broker

Dequeue(serverID string, qnames ...string) (msg *TaskMessage, deadline time.Time, err error)

The broker loops through the list, returning a task from the first queue that has one ready.

So far, there were 2 priority types (or mode) used in asynq:

• strict: in strict mode the qnames parameter is the list of names of the queues ordered following the priority of the queues.
  With this mode, tasks in lower priority queues are dequeued only when higher priority queues are empty.
  We never use this mode.
• lenient: in this mode the list of queue names are ordered 'statistically' with priorities.
  The frequency at which queues are at the head of the list is proportional to their priority.

Performances

These performance tests are run by enqueuing 1000 to 10000 tasks at a given rate and looking at the time it takes to have the task dequeued.
Two broker are tested redis (running locally) and sqlite, which is our target.

Three tests are shown (with the name of the corresponding test in code):

• alone (TestSingleQueueAloneAtRate): configuration has one single queue
• single (TestSingleQueueAtRate): configuration has multiple queues. The test enqueues in the higher priority queue only.
• all (TestAllQueuesAtRate): configuration has multiple queues and the test enqueues in all of them.

To make tables more readable, the results below only show some of the tested rates with the more relevant numbers.

Note on enqueuing rate

• redis: the broker always responds quickly to an enqueuing requests, but - as shown by 'max' outliers in the first table below - the backend sometimes needs to digest (this my interpretation and I did not dig more)
• sqlite: looking at the total time needed to enqueue all tasks, it seems difficult with sqlite to enqueue faster than ~500 tasks/sec. Nevertheless, for our tests, this just means the test takes longer to execute and will show accumulated lateness when de-queuing.

alone

Queues configuration has a single queue

Dequeue-ing time (millis) - lenient mode

type	enqueue rate (tasks/sec)	50	100	200	400	1000
redis	max (ms)	6	0	0	4	37002
	avg (ms)	0	0	0	0	3
sqlite	max (ms)	8	3	3	70	17
	avg (ms)	1	1	1	7	1

single

Queues configuration

queues {
  "q1": 1,
  "q2": 2,
  "q3": 3,
  "q4": 4,
  "q5": 5
}

Only q5 is used

Dequeue-ing time (millis) - lenient mode

type	enqueue rate (tasks/sec)	50	100	200	400	1000
redis	max (ms)	25	14	1	5	44000
	avg (ms)	0	0	0	0	4
sqlite	max (ms)	3	4	8	4694	15932
	avg (ms)	1	1	1	2908	11345

all

Queues configuration

queues {
  "q1": 1,
  "q2": 2,
  "q3": 3,
  "q4": 4,
  "q5": 5
}

All queues are used

Dequeue-ing average time (millis) - lenient mode

redis

queue / enqueue rate	50	100	200	400	1000
q1	1	1	1	0	130
q2	0	0	0	0	0
q3	0	0	0	39	0
q4	0	0	0	16	0
q5	0	167	0	25	0

sqlite

queue / enqueue rate	50	100	200	400	1000
q1	10	1038	994	5276	5048
q2	6	410	535	2287	2335
q3	5	43	30	231	99
q4	3	16	19	15	17
q5	3	10	11	9	10

IWRR

The previous results show that even when not used the presence of other queues is sufficient to decrease performances

• with a single queue, this shows at around 300 tasks/second
• with 5 queues in use, the lowest priority queue sees degraded performances between 50 and 100 tasks/second

The reason for degraded performances is the time a single dequeue request takes with sqlite and the number of requests needed with the lenient mode.

To solve the issue we use the iwrr mode coupled with buffered dequeue-ing (available only with sqlite):

• iwrr is Interleaved Weighted Round Robin described in this Wikipedia article
• buffered dequeue allows to dequeue N tasks from the broker

DequeueN(serverID string, qname string, count int, ret []Task) ([]Task, error)

When using buffered dequeue, the processor takes care of limiting the size of the buffer according to:

• the concurrency of the queue
• the global concurrency of the processor. When the number of available active workers decreases, the
  size allowed to the dequeue buffer is proportional to the priority of the queue.

performances

alone

type	enqueue rate (tasks/sec)	50	100	200	400	1000
sqlite	max (ms)	5	3	4	4	16
	avg (ms)	1	1	1	1	1

single

type	enqueue rate (tasks/sec)	50	100	200	400	1000
sqlite	max (ms)	9	10	15	46	41
	avg (ms)	3	3	3	9	15

all (avg ms)

queue / enqueue rate	50	100	200	400	1000
q1	7	12	11	12	12
q2	4	6	6	6	6
q3	4	1	1	1	1
q4	5	8	9	9	9
q5	4	4	3	4	3

Note that the max dequeue-ing time never exceeded 100ms in this test, for all queues.