From 63d0b8d6f592551ca2a766786ef6a3a919ac6f34 Mon Sep 17 00:00:00 2001
From: Hans van Zijst <hans@procolix.com>
Date: Mon, 23 Dec 2024 19:35:03 +0100
Subject: [PATCH] Updated maps documentation a bit and added first bit of
documentation for upstreams.
---
matrix/nginx/workers/README.md | 61 ++++++++++++++++++++++++----------
1 file changed, 44 insertions(+), 17 deletions(-)
diff --git a/matrix/nginx/workers/README.md b/matrix/nginx/workers/README.md
index 2f8a9e5f..214130b5 100644
--- a/matrix/nginx/workers/README.md
+++ b/matrix/nginx/workers/README.md
@@ -5,10 +5,10 @@ include_toc: true
# Reverse proxy for Synapse with workers
-Changing nginx's configuration from a reverse proxy for a normal, monolithi
+Changing nginx's configuration from a reverse proxy for a normal, monolithic
Synapse to one for a Synapse that uses workers, quite a lot has to be changed.
-As mentioned in [Synapse with workers](../synapse/workers.md#synapse), we're
+As mentioned in [Synapse with workers](../../synapse/workers.md#synapse), we're
changing from network sockets to UNIX sockets.
Because we're going to have to forward a lot of specific requests to all kinds
@@ -28,27 +28,24 @@ need to include them several times in different places.
# Maps
-A map sets a variable based on, usually, another variable. One case we use is
-determining the type of sync a client is doing. A normal sync, simply
-updating an existing session, is a rather lightweight operation.
-
-An initial sync, meaning a full sync because the session is brand new, is not
-so lightweight.
+A map sets a variable based on, usually, another variable. One case we use this
+is in determining the type of sync a client is doing. A normal sync, simply
+updating an existing session, is a rather lightweight operation. An initial sync,
+meaning a full sync because the session is brand new, is not so lightweight.
A normal sync can be recognised by the `since` bit in the request: it tells
the server when its last sync was. If there is no `since`, we're dealing with
an initial sync.
-Initial syncs are forwarded to the `initial_sync` workers, the normal syncs to
-`normal_sync`.
+We want to forward requests for normal syncs to the `normal_sync` workers, and
+the initial syncs to the `initial_sync` workers.
We decide to which type of worker to forward the sync request to by looking at
the presence or absence of `since`: if it's there, it's a normal sync and we
set the variable `$sync` to `normal_sync`. If it's not there, we set `$sync` to
-`initial_sync`.
+`initial_sync`. The content of `since` is irrelevant for nginx.
-The last bit is to foward the request to the upstream `$sync`. This is what
-that map looks like:
+This is what the map looks like:
```
map $arg_since $sync {
@@ -57,17 +54,47 @@ map $arg_since $sync {
}
```
-We filter the argument `since` by using `$arg_since`, see [the index of
+We evaluate `$arg_since` to set `$sync`: `$arg_since` is nginx's variable `$arg_`
+followed by `since`, the argument we want. See [the index of
variables in nginx](https://nginx.org/en/docs/varindex.html) for more
-variables we can use.
+variables we can use in nginx.
By default we set `$sync` to `normal_sync`, unless the argument `since` is
-empty (absent); then we set `$sync` to `initial_sync`.
+empty (absent); then we set it to `initial_sync`.
-After that, we forward the request:
+After this mapping, we forward the request to the correct worker like this:
```
proxy_pass http://$sync;
```
+# Upstreams
+
+In our configuration, nginx is not only a reverse proxy, it's a load balancer.
+Just like what `haproxy` does, it can forward requests to "servers" behind it.
+Such a server is the inbound UNIX socket of a worker, and there can be several
+of them in one group.
+
+Two of these upstreams are the sync workers: `normal_sync` and `initial_sync`,
+both consisting of several "servers":
+
+```
+upstream initial_sync {
+ hash $mxid_localpart consistent;
+ server unix:/run/matrix-synapse/inbound_initial_sync1.sock max_fails=0;
+ server unix:/run/matrix-synapse/inbound_initial_sync2.sock max_fails=0;
+ keepalive 10;
+}
+
+upstream normal_sync {
+ hash $mxid_localpart consistent;
+ server unix:/run/matrix-synapse/inbound_normal_sync1.sock max_fails=0;
+ server unix:/run/matrix-synapse/inbound_normal_sync2.sock max_fails=0;
+ server unix:/run/matrix-synapse/inbound_normal_sync3.sock max_fails=0;
+ keepalive 10;
+}
+```
+
+The `hash` bit is to make sure requests are always forwarded to the same
+worker.