Added introduction and PostgreSQL warning.

This commit is contained in:
Hans van Zijst 2024-12-22 16:49:20 +01:00
parent 04c14636a2
commit c41ea155f2
No known key found for this signature in database
GPG key ID: ECF8564FB15A8216

View file

@ -3,7 +3,7 @@ gitea: none
include_toc: true
---
# Worker-based setup
# Introduction to a worker-based setup
Very busy servers are brought down because a single thread can't keep up with
the load. So you want to create several threads for different types of work.
@ -22,6 +22,27 @@ freeze the rest.
Workers communicate with each other via socket files and Redis.
**Important note**
While the use of workers can drastically improve speed, the law of diminished
returns applies. Splitting off more and more workers will not further improve
speed after a certain point. Plus: you need to understand what the most
resource-consuming tasks are before you can start to plan how many workers for
what tasks you need.
In this document we'll basically create a worker for every task, and several
workers for a few heavy tasks, as an example. You mileage may not only vary, it
will.
Tuning the rest of the machine and network also counts, especially PostgreSQL.
A well-tuned PostgreSQL can make a really big difference and should probably
be considered even before configuring workers.
With workers, PostgreSQL's configuration should be changed accordingly: see
[Tuning PostgreSQL for a Matrix Synapse
server](https://tcpipuk.github.io/postgres/tuning/index.html) for hints and
examples.
# Redis