Fediversity/matrix/synapse/workers.md

Table of Contents

• Worker-based setup
• Redis
• Synapse
• Workers

Worker-based setup

Very busy servers are brought down because a single thread can't keep up with the load. So you want to create several threads for different types of work.

See this Matrix blog for some background information.

Redis

First step is to install Redis.

apt install redis-server

For less overhead we use a UNIX socket instead of a network connection to localhost. Disable the TCP listener and enable the socket in /etc/redis/redis.conf:

port 0

unixsocket /run/redis/redis-server.sock
unixsocketperm 770

Our matrix user (matrix-synapse) has to be able to read from and write to that socket, which is created by Redis and owned by redis:redis, so we add user matrix-synapse to the group redis.

adduser matrix-synapse redis

Restart Redis for these changes to take effect. Check if port 6379 is no longer active, and if the socketfile /run/redis/redis-server.sock exists.

Synapse

First, create the directory where all the socket files for workers will come, and give it the correct user, group and permission:

mkdir /run/matrix-synapse
dpkg-statoverride --add --update matrix-synapse matrix-synapse 0770 /run/matrix-synapse

Add a replication listener:

listeners:

  ...

  - path: /run/matrix-synapse/replication.sock
    mode: 0660
    type: http
    resources:
      - names:
        - replication 

Check if the socket is created and has the correct permissions. Now point Synapse at Redis in conf.d/redis.yaml:

redis:
  enabled: true
  path: /run/redis/redis-server.sock

Check if Synapse can connect to Redis via the socket, you should find log entries like this:

synapse.replication.tcp.redis - 292 - INFO - sentinel - Connecting to redis server UNIXAddress('/run/redis/redis-server.sock')
synapse.util.httpresourcetree - 56 - INFO - sentinel - Attaching <synapse.replication.http.ReplicationRestResource object at 0x7f95f850d150> to path b'/_synapse/replication'
synapse.replication.tcp.redis - 126 - INFO - sentinel - Connected to redis
synapse.replication.tcp.redis - 138 - INFO - subscribe-replication-0 - Sending redis SUBSCRIBE for ['matrix.example.com/USER_IP', 'matrix.example.com']
synapse.replication.tcp.redis - 141 - INFO - subscribe-replication-0 - Successfully subscribed to redis stream, sending REPLICATE command
synapse.replication.tcp.redis - 146 - INFO - subscribe-replication-0 - REPLICATE successfully sent

Workers

Workers are Synapse instances that perform a single job (or a set of jobs). Their configuration goes into /etc/matrix-synapse/workers, which we have to create first.