Taking Back Control of my Messaging with Matrix

Back in the olden times when you wanted to send a message to someone you mailed them a letter. Remember those? Real pieces of paper which contained actual writing and were physically delivered to someone. Time progressed and advancements in technology gave us the ability to send messages electronically, there was FAX then email and SMS. All of these communication methods had two things in common:

1. They were decentralized and interoperable; that is to say anyone could send a message to anyone else with the right equipment, and
2. You were ultimately in control of the message history. In the case of letters and FAX, you retained physical possession of the media, and in the case of SMS or email messages were stored on a device you physically controlled.

Fast-forward another decade and the old tried and true methods are increasingly finding themselves taking a backseat to newer communications tools such as Slack, Messenger or Discord. While these systems offer new features and improved ease of use they all have one thing in common – they are proprietary, centralized and gated, which is ultimately contrary to the original intentions of the Internet. In order to use any of these services, a person needs to register an account, or otherwise subscribe to the service, and if the organization or company hosting the data decides to shut it down; all the associated data and history is lost to the world forever. It also leaves a person with a pile of disparate accounts and no easy way to manage them all, other then to login to a dozen different apps or websites to retrieve communications.

None of these problems really crossed my mind, and like most people the number of communications apps on my phone kept growing, and growing until the day my personal Slack server that I share with a few friends passed the free 10,000 message limit, forever condemning our older message history to obscurity behind a corporate paywall. To be clear, I’m not blaming Slack – they are very clear about the limitations of a free account – but it did make me realize that modern communications were becoming increasingly ephemeral and I couldn’t help but think there had to be a better way.

Matrix Chat has Entered the Room

Enter the concept of Matrix Chat. First imagined in 2015, it is designed to be a decentralized, federated and open instant messaging communication platform – essentially promising to become the email of the IM world, or as my friend would put it; the email of Web 3.0. In a nutshell, anyone can build their own server or client from scratch and interoperate seamlessly with other Matrix servers across the Internet. Just like with email, if one of the servers was to drop offline, messages are qued and delivered when the server comes back online and in the event that a server drops offline permanently, only the users of that server are effected, the rest of the network continues chugging along unimpeded; sending and receiving messages like nothing even happened.

This openness is rare in today’s modern world of closed-source, proprietary communications protocols and it’s seemingly easy-to-use applications coupled with a moderately large uptake (28 million users in 2021) made it an obvious candidate for further investigation. After a bit more research I discovered the concept of bridging, which is the ability to integrate other chat applications into Matrix by way of third party binaries, in theory allowing me to consolidate all of my dozens of different chat systems into one centralized platform. At this point I was sold – open standard? federation? and the promise to integrate other chat platforms? Sign me up!

Using Matrix

In order to use Matrix you need, at a minimum, a client and a server (referred to as a homeserver). Because Matrix is a protocol, rather then a piece of software, I was a bit thrown off when trying to search for things like “how to install Matrix server.” I realize now it was a bit like asking “how do I install SMTP?” You, of course, don’t install SMTP, you install a server (like Postfix or Exim) which speaks SMTP. Eventually, I landed upon Synapse, which is the reference implementation of a Matrix-protocol server developed by the authors of the Matrix-protocol themselves. Synapse is the de-facto standard when it comes to running a Matrix-protocol server, and it’s configuration and installation is pretty simple. This isn’t a guide on how to use the software, so I’ll forgo the step by step directions, but in a nutshell it boils down to pip3 install synapse and you’re off to the races. Within minutes I was up and running.

Once the homeserver is up and running, it’s time to choose a client. Element is by far the most popular client and is developed by the same core team who worked on the protocol specification and the Synapse server implementation. Element comes in web, desktop and mobile flavors and while I opted to use it on my desktop machines, I ended up preferring a different client, SchildiChat on mobile. This is the beauty of an open protocol specification; you can use any client you want, you could even write your own if so inclined!

No matter which client you choose, setup is going to be fairly similar. When you launch it, you choose a server (matrix.org is the default in most cases) and then sign in or register an account. As I was using this on my brand new homeserver, I needed to enable registrations in the server config (they are disabled by default) and then register an account to my server via the app. Once I was registered I was sure to disable new registrations since I don’t want anyone being able to register an account on my homeserver, doing so would be a bit like hosting email accounts for the general public.

When registering an account you are also prompted to setup encryption keys, it’s important to back these up since you will need them if you ever need to re-install your client and want to access encrypted messages you’ve sent. It’s difficult to understate the importance of keeping these keys backed up, if you loose them and loose access to your client any encrypted messages you sent will be lost FOREVER. You can reset the keys (as long as you have your account password) and send new messages, but your historical ones are GONE.

Signing in on additional devices can be a bit cumbersome. You either need to supply the backup of the encryption keys (which you received above) or you will need to use one of your existing devices to ‘verify’ the new device by scanning a QR code or validating with a series of screens. While it’s not the most intuitive process, I appreciate that the process is in place to protect the security of messages, and if one was to save the encryption backup file on a cloud-storage service it would be reasonably trivial to use it in order to validate a newly signed in device.

Now that I was registered on the homeserver, it was time to start chatting! When you register or sign in to a new account you are given a completely blank workspace. The next steps can be a bit unclear. The first thing I did was create a new ‘channel.’ Like with other chat services, channels are a common communication bus that multiple people can join; any messages posted in the channel can be viewed by any other members. When creating a new channel you are given a few options, the first is whether to configure ‘end-to-end’ encryption on the channel. End to end encryption means that your messages are encrypted at rest, so even if the homeserver was compromised, an attacker would not be able to read the content of the messages without your personal decryption keys. In my case, I decided to forgo end-to-end encryption for the time being to reduce complexity, particularly since I was running the homeserver myself and trust it implicitly.

Another option you have is whether to make the room public or not. In the case of a public room, anyone with the room’s address can join it, including people on other Matrix homeservers (remember, the network is decentralized) while private rooms require you to send out invitations for people to join. You then name the room and set an optional topic and that’s it! You can now start posting messages to… yourself, or anyone else in the channel. Once the channel is created, there are other options you can configure by clicking the … icon and selecting settings which I won’t go into in this post. Suffice to say there are a lot of options here and you can customize to your heart’s content.

Exploring Federation

OK, now you’ve got your first room and you’ve asked all your friends to join your homeserver so you can chat, but this is only part of what matrix is all about. Up until this point I had basically replicated a self-hosted version of existing models of chat services, something that has already been done by projects like RocketChat and Mattermost. The real power of the open Matrix protocol is that you should be able to talk to ANYONE using Matrix registered to ANY homeserver. Just like with email where you don’t need an account on every single email provider to reach someone, with Matrix you should only have to register in one place.

It turns out sending a message to another Matrix user on another homeserver is dead simple. In your client you simply click the + box to start a new chat and enter the user’s username in the format of @USERNAME:MATRIXDOMAIN. So for instance if I wanted to chat with user ‘blabla123’ on homeserver ‘matrix.org’ I would enter @blabla123:matrix.org and the user ‘blabla123’ would be sent an invitation to chat. In order to test this, I created an account on the matrix.org homeserver and started a chat with myself on my own homeserver, and wouldn’t you know it, within seconds I was receiving the invitation to chat! Messages sent back and forth were delayed by a couple of seconds, but overall the experience is quite smooth and if I wasn’t sitting here staring at both screens waiting for messages to propagate the delay would be imperceptible.

The same principal applies to rooms. If you know the address of a public room on a homesever, you can simply start a chat with that room from any other homeserver and be up and running in seconds. In the case of private rooms, you need to be invited, but invites can also be sent across the matrix network!

Next, I wanted to validate what would happen if a homeserver was unreachable for a period of time, so I took the lazy approach of patching my homeserver’s hosts file to point matrix.org to a dummy IP. I then sent a couple messages to my federated chat and verified that the messages were not received at the matrix.org homeserver. After some time, I fixed the hosts file on my server and, as expected, the messages were successfully delivered after a period of time! They even had accurate timestamps! Just like with email, temporary connectivity problems are handled by a retry mechanism within the protocol. Pretty damn slick!

Integrating with other Chat Protocols

At this point I was pretty sold on Matrix. Even just as a federated and open standard it was miles better then anything else out there, and the fact that it was so simple to run my own homeserver and thus retain ultimate control over my messaging history meant that I was already going to use the system, however, the icing on this very delicious cake was the fact that the Synapse server supports a system called ‘bridges’ which allows you write software that integrates with other chat protocols. Matrix.org provides official bridges for IRC, Slack, and a few others and unofficial, community developed bridges for Messenger, Signal, SMS and dozens of other protocols. To date, I have managed to integrate my homeserver with Signal, Messenger, IRC, SMS and Slack, though the Messenger implementation is a little buggy and I wouldn’t recommend it for users who aren’t highly technical.

Setting up a bridge requires more technical inclination then setting up the homeserver or client, but for someone with experience modifying config files and compiling software it’s pretty easy to get a handle on. At a high level, you compile the application (it could be in a Docker container, written in Python, Node or another language), generate a registration file from the application, edit your homeserver config to reference the registration, then start both the bridge and your homeserver. Interaction with the bridge is usually via a ‘bot’ – you start a direct chat with the ‘bot’ and send it commands to link rooms, login to accounts and send messages. In most cases, once the account is linked on both sides, if someone sends you a message on the external protocol your Matrix server will invite you to a new chat (similar to the invitation you would receive if someone from the Matrix network invited you to chat), and any messages sent to that chat will be replicated out to the external protocol.

In my brief experience, the official bridge implementations (such as Slack) work quite well. Messages sent to me in Slack are replicated to Matrix, and replies I type in Matrix are sent to Slack. Threads are supported natively, and editing messages (when supported by the external protocol) works without a fuss Some of the community implementations (such as with Messenger) are significantly more buggy, requiring some babysitting and fiddling to get “just right.” This is not unexpected, since in many cases these bridges are doing something that, the other protocol was not deigned to handle, and I’m hoping as development goes on the experience will improve.

One irritation that is common across all bridges, is that messages are not reliably marked as ‘read’ in the external protocol. For instance, when I sign into the Slack app directly (something that I only need to do rarely now), I am bombarded by dozens of messages that are ‘unread’ even though I have already responded to them via Matrix. This does seem to be an issue that is being worked on, and it’s a small price to pay in order to have a complete, searchable archive of my chat history.

Closing Thoughts

The Matrix protocol is still relatively new and is constantly being changed and iterated upon as requirements develop. Running your own homeserver is definitely a reasonably technical endeavor and staying up to date with the latest changes is a must. While the overall experience is fairly smooth, there are definitely areas for improvement, particularly around the Element Desktop client where I experience random refresh issues and occasional problems organizing chat rooms into spaces. The app is being constantly updated though, so in time I’m sure the experience will improve.

One area where I feel Matrix desperately needs to improve is the process of migrating to a new homeserver. Right now, if your current homeserver was to go offline permanently, you would be stuck with an orphaned chat history and the need to register on a new homeserver. While it is possible to export your chat history, there is currently no functionality to import it to a new homeserver, so while you would be able to retain a ‘read-only’ copy of your message history new messages would appear in a completely new chat which breaks the continuity of a continuous message stream. There are projects such as Matrix Migration which aim to make it easier to move to a new homeserver, but it doesn’t have quite the same functionality that you would have while migrating to a new email server for instance, where you can import your old messages via IMAP. This is definitely a well-requested feature, and I’m confident it will be addressed at some point, so I’ll stay tuned for updates.

Despite the few shortcomings, the Matrix-protocol is hands down the best self-hosted chat solution I have come across. As more communications systems move to a proprietary, closed off model Matrix stands out as being one of the few good guys who believe in maintaining interoperability and decentralization in an increasingly centralized Internet. I look forward to continuing to use the system and following it as it continues to develop and mature.