This is an old revision of the document!
- synapse
- Jonathan Haack
- Haack's Networking
- webmaster@haacksnetworking.org
synapse
This tutorial is for users of Debian GNU/Linux who want to create their own Synapse instance. The official documentation was pretty solid, but/and I also used some online tutorials, especially the one at Hack Liberty. Although I give credit to these sites, I must say that they both had tons of small to medium mistakes which, combined with the complexity of the project, made this a fairly challenging instance to create. I am quite glad the VM is built, backed up and tarballed. Okay, so first, install synapse and add the gpg keys for their repo, etc.:
sudo apt install -y lsb-release wget apt-transport-https sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/matrix-org.list sudo apt update sudo apt install matrix-synapse-py3
You now need to install postgresql and create a database with a dedicated non-root user:
sudo apt install postgresql sudo -u postgres bash createuser --pwprompt synapse_user createdb --encoding=UTF8 --locale=C --template=template0 --owner=synapse_user synapse exit
After creating the database, inform synapse of how to reach it in the pg_hba.conf file as follows:
nano /etc/postgresql/13/main/pg_hba.conf <host synapse synapse_user ::1/128 md5> sudo systemctl reload postgresql
It's now time to edit the file /etc/matrix-synapse/homeserver.yaml. Remove the default database configuration, and replace it with the credentials you just made:
<database:> <name: psycopg2> <txn_limit: 10000> <args:> <user: synapse_user> <password: secretpassword> <database: synapse> <host: localhost> <port: 5432> <cp_min: 5> <cp_max: 10>
There are now some options that you can configure based on personal preference. Hack Liberty has its own recommendations, and I agreed with some and not with others. Moreover, I also found that Matrix/Synapse is currently requiring a stricter recipe for the yaml config than their template or even the official docs recommend. I was unable to get to the homeserver.yaml to work without adding a base_url line and a Google V2 challenge. Make sure to refer to Synapse's official docs for your use case and so that you understand what they each do. Here are the “optional” configurations that I have active, some of which I had to configure to make everything work:
<public_baseurl: "https://gnulinux.club"> <require_auth_for_profile_requests: true> <limit_profile_requests_to_users_who_share_rooms: true> <include_profile_data_on_invite: false> <allow_public_rooms_over_federation: true> <allow_profile_lookup_over_federation: true> <allow_device_name_lookup_over_federation: true> <enable_registration: True> <enable_registration_captcha: True> <recaptcha_public_key: "enter pub key here"> <recaptcha_private_key: "enter priv key here"> <registration_shared_secret: "yourmomismykey">
In my case, matrix was not currently allowing un-challenged, or un-tokened user registration, so adding the Google Challenge was required in order to make it functional (keep verify origin off). I also had to the base_url explicitly and the enable_registration explicitly. From researching forums and reddits online, I was able to ascertain that this is because Matrix changes their criteria/allowances depending on exploits, current threats, etc., actively, meaning that what might be allowed on day x is no longer supported on day y, etc. Now that synapse is more or less configured, it's time to install your web server of choice. I prefer apache2 since I have a long history of making reverse proxy configs for it, and even got some changes committed(see June 8, 2022) to the recipes project on gitlab (yay!). Note: Since ACME's cert-only command always fails for me, whenever I do apache reverse proxies, I first set up a slim website with stock virtualhost, A records on my DNS host, pull in certs on it, then disable it, as follows:
sudo apt install apache2 sudo certbot --authenticator standalone --installer apache -d gnulinux.club --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2" a2dissite 000-default.conf
Refer to apache survival if you need help on setting up virtual hosts, etc. and/or apache/LAMP stacks. Once you built the certs and disabled the dummy website, then create two reverse proxy virtual hosts like these:
[[https://repo.haacksnetworking.org/oemb1905/haackingclub/-/blob/master/apache/virtualhosts/synapse.conf|virtualhost-80.conf]] [[https://repo.haacksnetworking.org/oemb1905/haackingclub/-/blob/master/apache/virtualhosts/synapse-ssl.conf|virtualhost-443.conf]]
Once the reverse proxies are setup, restart the service with systemctl restart apache2.service. It's now time to set up a turn server so clients behind NAT can function properly. To do that, do as follows:
sudo apt install coturn mkdir -p /etc/coturn/certs sudo chown -R turnserver:turnserver /etc/coturn/ chmod -R 700 /etc/coturn/ pwgen -s 64 1 <it dumps a key> sudo nano /etc/turnserver.conf <use-auth-secret> <static-auth-secret=YOUR-STATIC-AUTH-SECRET-HERE> <realm=yoursite.com:5349> <no-tcp-relay> <denied-peer-ip=10.0.0.0-10.255.255.255> <denied-peer-ip=192.168.0.0-192.168.255.255> <denied-peer-ip=172.16.0.0-172.31.255.255> <allowed-peer-ip=10.0.0.1> <user-quota=12> <total-quota=1200> <cert=/etc/coturn/certs/fullchain.pem> <pkey=/etc/coturn/certs/privkey.pem> sudo systemctl restart coturn
Now that you have configured a turn server, make sure that synapse knows how to leverage it by adding a configuration block to /etc/matrix-synapse/homeserver.yaml as follows:
turn_uris: [ "turns:yoursite.com:5349?transport=udp", "turns:yoursite.com:5349?transport=tcp" ]` turn_shared_secret: "YOUR-STATIC-AUTH-SECRET-HERE" turn_user_lifetime: 86400000 turn_allow_guests: false
Now, let's make sure that the turn server can use TLS encryption by copying the certs from Let's Encrypt to it's working directory. (Note: I think these should be updated to symlinks.)
cp /etc/letsencrypt/live/gnulinux.club/fullchain.pem /etc/coturn/certs/ cp /etc/letsencrypt/live/gnulinux.club/privkey.pem /etc/coturn/certs/ sudo chown turnserver:turnserver -R /etc/coturn/ sudo service coturn force-reload sudo service coturn restart
So since I could not get token user creation to work, I ended up using Google Challenge instead. Nevertheless, I include this here because I did create this infrastructure, and will leverage it if/when token authentication begins to work again. Make sure to record all keys/passwords, etc., in a secure manager or FDE drive that's offline. Begin by creating two users for the instance, one that's the administrator for it, and another that's used as a proxy for registration requests with a token.
register_new_matrix_user -c homeserver.yaml http://localhost:8008 register_new_matrix_user -c homeserver.yaml http://localhost:8008
After the database users are thus created and passwords specified and recorded, you now need to install matrix/synapse's registration tool with python-pip.
sudo apt install python3-pip pip3 install matrix-registration==1.0.0.dev7 pip3 install psycopg2-binary
Once that's done, you will need to create a database user and database for the accounts. You will need to alternate between the user and root to do this and might have to tweak permissions.
sudo -u postgres bash createuser --pwprompt matrix_reg_user createdb --owner=matrix_reg_user matrix_reg
Now that the database registration user and password have been created, configure the synapse/matrix instance to leverage those credentials by appending this additional entry to the pg_hba.conf file:
nano /etc/postgresql/13/main/pg_hba.conf host matrix_reg matrix_reg_user ::1/128 md5
You now need to specify how the reverse proxy for registration can find the base url and the synapse/matrix services, etc. To do that, you create a config.yaml file as follows:
sudo nano /home/registration/matrix-registration/config.yaml
The contents of config.yaml should look like this:
[[https://repo.haacksnetworking.org/oemb1905/haackingclub/-/blob/master/configs/config.yaml|config.yaml]]
It's now best practice to create a systemd unit for the registration service itself since this is accordingly a public instance and might need the registration toolchain restarted etc. To do that, the official docs call for creating a systemd unit as follows:
sudo useradd -m registration sudo nano /etc/systemd/system/matrix-registration.service
The contents of the matrix-registration.service file should have locations that match the configurations you made for registration above. Something like the following is recommended:
[[https://repo.haacksnetworking.org/oemb1905/haackingclub/-/blob/master/configs/matrix-registration.service|matrix-registration.service]]
Once the systemd unit is created and the locations in the block linked to your production locations, then enable the unit, reload the units, and start the service. Note: As the first block states, you won't need to individually restart this service as it will restart when you restart the default service systemctl restart matrix-synapse.service.
systemctl daemon-reload systemctl --user enable matrix-registration systemctl --user start matrix-registration
Once the registration and account creation services are installed and functional, it is now time to create the web client, which will be used for clients who want to register or test out the instance, and also for users to directly create new matrix accounts on your instance. To dot that, let's add a dedicated user for element, which is the standard/default web client, and install the element-web content management system:
sudo useradd -m elementweb cd /home/elementweb/ sudo --user elementweb wget https://github.com/vector-im/element-web/releases/download/v1.10.12/element-v1.10.12.tar.gz sudo --user elementweb tar -xvf element-v1.10.12.tar.gz mv element-v1.10.12 /var/www/element cd /var/www/element nano config.json
The config.json file should like something like what I have below. Note: this file is notorious for causing the “host cannot be found error” that is all over forums. Remember to change all the entries to your domain and make sure the base_url matches what you have in /etc/matrix-synapse/homserver.yaml. Here's what mine looks like:
[[https://repo.haacksnetworking.org/oemb1905/haackingclub/-/blob/master/configs/config.json|config.json]]
If you have not already done so, make sure to add an A record for element.domain.tld to your DNS host. It's now time to create an apache virtual host for the subdomain as follows:
sudo nano /etc/apache2/sites-enabled/element80.conf [[https://repo.haacksnetworking.org/oemb1905/haackingclub/-/blob/master/apache/virtualhosts/element.matrix.conf|element80.conf]] sudo nano /etc/apache2/sites-enabled/element443.conf [[https://repo.haacksnetworking.org/oemb1905/haackingclub/-/blob/master/apache/virtualhosts/element.matrix-ssl.conf|element443.conf]] sudo certbot --authenticator standalone --installer apache -d element.gnulinux.club --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2" a2ensite element80.conf a2ensite element443.conf systemctl restart apache2.service
Once the apache web server is restarted, it's time to debug your instance. To do that, remember these two essential skills:
tail -f /var/log/example.log sudo apache2ctl configtest
Between these two debugging tools, you should be in good shape to get everything up and running! Okay, that took a lot to document but it's worth it. Also, if you are serving others in a public facing community, make sure you have appropriate backup scripts for this VM. Utilize rsync or similar to ensure you have those backups offsite as well as on site, and also provision a hot-spare in case your self-hosted or co-located hardware fails.
— oemb1905 2022/11/20 00:24
