Add the ability to run parallel tasks

[rroblf01]

This pull request adds support for concurrent task processing in the database-backed worker by introducing multi-threading. The worker can now claim and process multiple tasks in parallel, controlled by a new --max-workers option. The changes also update the core query and locking logic to support batch task retrieval, and adjust related tests to reflect the new behavior.

Concurrency and worker configuration:

• Added a --max-workers command-line option to the worker, allowing configuration of the maximum number of concurrent worker threads (default is 1, set by MAX_WORKERS) (django_tasks/backends/database/management/commands/db_worker.py, django_tasks/base.py). [1] [2] [3]
• Updated the worker initialization and argument parsing to accept and use the max_workers parameter (django_tasks/backends/database/management/commands/db_worker.py). [1] [2] [3] [4]

Task claiming and processing logic:

• Changed the worker loop to claim and process up to max_workers tasks concurrently using threads, instead of a single task at a time (django_tasks/backends/database/management/commands/db_worker.py). [1] [2]
• Modified the get_locked method in the queryset to return a batch of locked tasks (as a queryset slice) instead of a single result, supporting batch locking (django_tasks/backends/database/models.py).

Testing updates:

• Adjusted tests to accommodate the new batch locking and threading behavior, including changes to query counts and task locking logic (tests/tests/test_database_backend.py). [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]

These changes collectively enable the worker to process multiple tasks in parallel, improving throughput and efficiency for database-backed task queues.

[rroblf01]

There are some things in this PR that need to be taken into account, such as that with my proposed changes, signal.SIGINT cannot terminate a running task, signal.SIGINT cannot terminate threads other than the main thread; as I have left it, the task would finish and then the worker would close.