@@ -18,13 +18,21 @@ TaskTiger
1818Features
1919--------
2020
21- - Per-task fork
22-
23- TaskTiger forks a subprocess for each task, This comes with several benefits:
24- Memory leaks caused by tasks are avoided since the subprocess is terminated
25- when the task is finished. A hard time limit can be set for each task, after
26- which the task is killed if it hasn't completed. To ensure performance, any
27- necessary Python modules can be preloaded in the parent process.
21+ - Per-task fork or synchronous worker
22+
23+ By default, TaskTiger forks a subprocess for each task, This comes with
24+ several benefits: Memory leaks caused by tasks are avoided since the
25+ subprocess is terminated when the task is finished. A hard time limit can be
26+ set for each task, after which the task is killed if it hasn't completed. To
27+ ensure performance, any necessary Python modules can be preloaded in the
28+ parent process.
29+
30+ TaskTiger also supports synchronous workers, which allows for better
31+ performance due to no forking overhead, and tasks have the ability to reuse
32+ network connections. To prevent memory leaks from accumulating, workers can
33+ be set to shutdown after a certain amount of time, at which point a
34+ supervisor can restart them. Workers also automatically exit on on hard
35+ timeouts to prevent an inconsistent process state.
2836
2937- Unique queues
3038
@@ -552,6 +560,18 @@ Workers support the following options:
552560
553561 Store tracebacks with execution history (config defaults to ``True ``).
554562
563+ - ``--executor ``
564+
565+ Can be ``fork `` (default) or ``sync ``. Whether to execute tasks in a separate
566+ process via fork, or execute them synchronously in the same proces. See
567+ "Features" section for the benefits of either approach.
568+
569+ - ``--exit-after ``
570+
571+ Exit the worker after the time in minutes has elapsed. This is mainly useful
572+ with the synchronous executor to prevent memory leaks from accumulating.
573+
574+
555575In some cases it is convenient to have a custom TaskTiger launch script. For
556576example, your application may have a ``manage.py `` command that sets up the
557577environment and you may want to launch TaskTiger workers using that script. To
0 commit comments