<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Technobabble</title>
    <link>https://blog.passtheballsocrates.com/</link>
    <description></description>
    <pubDate>Fri, 17 Apr 2026 00:47:22 -0500</pubDate>
    <item>
      <title>How to reclaim database space from synapse</title>
      <link>https://blog.passtheballsocrates.com/how-to-reclaim-database-space-from-synapse</link>
      <description>&lt;![CDATA[So, synapse is a neat tool, but it sucks at managing storage space. The built-in automatic room history purges works, but it doesn&#39;t delete the state data. If a matrix room gets spammed or just too large, you can&#39;t remove the state data without kicking all the users out of the room, deleting the data, then letting them rejoin. This can be mitigated with the rust compressor tool for synapse, which apparently has amazing squishing power, but I went a different route. Here are my notes from the process. &#xA;&#xA;!--more--&#xA;&#xA;This is mostly based on forest&#39;s notes and this script.&#xA;&#xA;Purge Remote Media&#xA;get a timestamp in unix seconds: date +%s000 --date &#34;10 days ago&#34;&#xA;    everything before this date will be purged&#xA;curl -k -X POST &#34;http://localhost:8008/synapse/admin/v1/purgemediacache?beforets=$your-timestamp-here&amp;accesstoken=$your-access-token&#34;&#xA;&#xA;Purge Big Rooms&#xA;use some forest scripts&#xA;&#xA;Purge Rooms from Database&#xA;get a list of the biggest rooms&#xA;    docker compose exec postgresql psql -U synapse -d synapse -c &#34;select roomid, count() from stategroupsstate group by roomid order by count() DESC LIMIT 100&#34;&#xA;    run this inside the synapse directory&#xA;    copy biggest rooms to a text file somewhere&#xA;use this kick-user-purge-room script  for each room id (must be wrapped in single quotes)&#xA;&#xA;!/bin/bash&#xA;&#xA;roomid=&#34;$1&#34;&#xA;accesstoken=&#34;insert-matrix-access-token-here&#34;&#xA;&#xA;[ -z &#34;$roomid&#34; ] &amp;&amp; exit 1&#xA;&#xA;echo &#34;getting list of users in $roomid...&#34;                                                                             &#xA;&#xA;curl -sS &#34;localhost:8008/synapse/admin/v1/rooms/$roomid/members?accesstoken=$accesstoken&#34; | jq -r &#39;.members[]&#39; | grep &#39;your-matrix-server-domain.com&#39;&#xA;&#xA;echo &#34;deleting $roomid...&#34;                                                                                             &#xA;&#xA;curl -H &#34;Content-Type: application/json&#34; -X DELETE &#34;localhost:8008/synapse/admin/v2/rooms/$roomid?accesstoken=$accesstoken&#34; \&#xA;        --data &#39;{ &#34;block&#34;: false, &#34;forcepurge&#34;: true, &#34;purge&#34;: true, &#34;message&#34;: &#34;This room is being cleaned, stand by...&#34; }&#39;&#xA;&#xA;echo &#39;&#39; &#xA;echo &#39;GET /synapse/admin/v2/rooms/$roomid/deletestatus : &#39;                                                           &#xA;&#xA;curl -sS &#34;localhost:8008/synapse/admin/v2/rooms/$roomid/deletestatus?accesstoken=$accesstoken&#34;&#xA;&#xA;    ./kick-user-purge-room &#39;!some-room-id&#39;&#xA;&#xA;Purge State Group IDs&#xA;Get all the state group ids from the deleted rooms&#xA;    docker compose exec postgresql psql -U synapse -d synapse -c &#34; SELECT id from stategroups where roomid = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39; OR roomid = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;; &#34;   stategroups1.txt&#xA;    docker compose exec postgresql psql -U synapse -d synapse -c &#34; SELECT id from stategroups where roomid = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39; OR roomid = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;; &#34;   stategroups2.txt&#xA;remove header and footer from the stategroups files with a text editor&#xA;sort the state group ids and convert them to postgres delete statements&#xA;    cat ./stategroups1.txt | sort | sed -E &#39;s/([0-9]+)/DELETE FROM stategroupsstate where stategroup = \1;/&#39;   stategroups1.sql&#xA;    cat ./stategroups2.txt | sort | sed -E &#39;s/([0-9]+)/DELETE FROM stategroupsstate where stategroup = \1;/&#39;   stategroups2.sql&#xA;move the files into the docker container sudo cp ./stategroups{1,2}.sql ./postgresdata/&#xA;delete the state group state!&#xA;    docker compose exec postgresql psql -U synapse -d synapse -f /var/lib/postgresql/data/stategroups1.sql&#xA;    docker compose exec postgresql psql -U synapse -d synapse -f /var/lib/postgresql/data/stategroups2.sql&#xA;&#xA;Cleanup&#xA;delete from other state group related tables&#xA;    docker compose exec postgresql psql -U synapse -d synapse -c &#34;DELETE FROM stategroupedges where stategroup in (SELECT id from stategroups where roomid = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;); &#34;&#xA;    docker compose exec postgresql psql -U synapse -d synapse -c &#34;DELETE FROM eventtostategroups where stategroup in (SELECT id from stategroups where roomid = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;); &#34;&#xA;    docker compose exec postgresql psql -U synapse -d synapse -c &#34;DELETE FROM stategroups where roomid = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;; &#34;&#xA;&#xA;Remove table and index bloat&#xA;commands from this excellent article on cleaning up synapse bloat&#xA;    I didn&#39;t use the rust compressor tool because I used forest&#39;s notes to delete the state of the massive rooms instead&#xA;reindex the synapse database&#xA;    docker compose exec postgresql psql -U synapse -d synapse -c &#34;REINDEX (VERBOSE) DATABASE synapse&#34;;&#xA;vaccum up the table bloat&#xA;    docker compose exec postgresql psql -U synapse -d synapse -c &#34;VACUUM FULL VERBOSE&#34;;&#xA;&#xA;DONE!&#xA;]]&gt;</description>
      <content:encoded><![CDATA[<p>So, synapse is a neat tool, but it sucks at managing storage space. The built-in automatic room history purges works, but it doesn&#39;t delete the state data. If a matrix room gets spammed or just too large, you can&#39;t remove the state data without kicking all the users out of the room, deleting the data, then letting them rejoin. This can be mitigated with the rust compressor tool for synapse, which apparently has amazing squishing power, but I went a different route. Here are my notes from the process.</p>



<p>This is mostly based on <a href="https://picopublish.sequentialread.com/files/matrix-manual-room-bonk.txt">forest&#39;s notes</a> and <a href="https://codeberg.org/DecaTec/Matrix-Synapse-Helpers/src/branch/main/CleanupMedia.sh">this script</a>.</p>

<h2 id="purge-remote-media">Purge Remote Media</h2>
<ul><li>get a timestamp in unix seconds: <code>date +%s000 --date &#34;10 days ago&#34;</code>
<ul><li>everything before this date will be purged</li></ul></li>
<li><code>curl -k -X POST &#34;http://localhost:8008/_synapse/admin/v1/purge_media_cache?before_ts=$your-timestamp-here&amp;access_token=$your-access-token&#34;</code></li></ul>

<h2 id="purge-big-rooms">Purge Big Rooms</h2>
<ul><li>use some forest scripts</li></ul>

<h3 id="purge-rooms-from-database">Purge Rooms from Database</h3>
<ul><li>get a list of the biggest rooms
<ul><li><code>docker compose exec postgresql psql -U synapse -d synapse -c &#34;select room_id, count(*) from state_groups_state group by room_id order by count(*) DESC LIMIT 100&#34;</code></li>
<li>run this inside the synapse directory</li>
<li>copy biggest rooms to a text file somewhere</li></ul></li>
<li>use this kick-user-purge-room script  for each room id (must be wrapped in single quotes)</li></ul>

<pre><code>#!/bin/bash

roomid=&#34;$1&#34;
accesstoken=&#34;insert-matrix-access-token-here&#34;

[ -z &#34;$roomid&#34; ] &amp;&amp; exit 1

echo &#34;getting list of users in $roomid...&#34;                                                                             

curl -sS &#34;localhost:8008/_synapse/admin/v1/rooms/$roomid/members?access_token=$accesstoken&#34; | jq -r &#39;.members[]&#39; | grep &#39;your-matrix-server-domain.com&#39;


echo &#34;deleting $roomid...&#34;                                                                                             

curl -H &#34;Content-Type: application/json&#34; -X DELETE &#34;localhost:8008/_synapse/admin/v2/rooms/$roomid?access_token=$accesstoken&#34; \
        --data &#39;{ &#34;block&#34;: false, &#34;force_purge&#34;: true, &#34;purge&#34;: true, &#34;message&#34;: &#34;This room is being cleaned, stand by...&#34; }&#39;

echo &#39;&#39; 
echo &#39;GET /_synapse/admin/v2/rooms/$roomid/delete_status : &#39;                                                           

curl -sS &#34;localhost:8008/_synapse/admin/v2/rooms/$roomid/delete_status?access_token=$accesstoken&#34;
</code></pre>

<p>    – <code>./kick-user-purge-room &#39;!some-room-id&#39;</code></p>

<h3 id="purge-state-group-ids">Purge State Group IDs</h3>
<ul><li>Get all the state group ids from the deleted rooms
<ul><li><code>docker compose exec postgresql psql -U synapse -d synapse -c &#34; SELECT id from state_groups where room_id = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39; OR room_id = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;; &#34; &gt; stategroups1.txt</code></li>
<li><code>docker compose exec postgresql psql -U synapse -d synapse -c &#34; SELECT id from state_groups where room_id = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39; OR room_id = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;; &#34; &gt; stategroups2.txt</code></li></ul></li>
<li>remove header and footer from the stategroups files with a text editor</li>
<li>sort the state group ids and convert them to postgres delete statements
<ul><li><code>cat ./stategroups1.txt | sort | sed -E &#39;s/([0-9]+)/DELETE FROM state_groups_state where state_group = \1;/&#39; &gt; stategroups1.sql</code></li>
<li><code>cat ./stategroups2.txt | sort | sed -E &#39;s/([0-9]+)/DELETE FROM state_groups_state where state_group = \1;/&#39; &gt; stategroups2.sql</code></li></ul></li>
<li>move the files into the docker container <code>sudo cp ./stategroups{1,2}.sql ./postgresdata/</code></li>
<li>delete the state group state!
<ul><li><code>docker compose exec postgresql psql -U synapse -d synapse -f /var/lib/postgresql/data/stategroups1.sql</code></li>
<li><code>docker compose exec postgresql psql -U synapse -d synapse -f /var/lib/postgresql/data/stategroups2.sql</code></li></ul></li></ul>

<h2 id="cleanup">Cleanup</h2>
<ul><li>delete from other state group related tables
<ul><li><code>docker compose exec postgresql psql -U synapse -d synapse -c &#34;DELETE FROM state_group_edges where state_group in (SELECT id from state_groups where room_id = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;); &#34;</code></li>
<li><code>docker compose exec postgresql psql -U synapse -d synapse -c &#34;DELETE FROM event_to_state_groups where state_group in (SELECT id from state_groups where room_id = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;); &#34;</code></li>
<li><code>docker compose exec postgresql psql -U synapse -d synapse -c &#34;DELETE FROM state_groups where room_id = &#39;&#34;&#39;!&#39;&#34;some-room-id&#39;; &#34;</code></li></ul></li></ul>

<h2 id="remove-table-and-index-bloat">Remove table and index bloat</h2>
<ul><li>commands from <a href="https://levans.fr/shrink-synapse-database.html">this excellent article</a> on cleaning up synapse bloat
<ul><li>I didn&#39;t use the rust compressor tool because I used forest&#39;s notes to delete the state of the massive rooms instead</li></ul></li>
<li>reindex the synapse database
<ul><li><code>docker compose exec postgresql psql -U synapse -d synapse -c &#34;REINDEX (VERBOSE) DATABASE synapse&#34;;</code></li></ul></li>
<li>vaccum up the table bloat
<ul><li><code>docker compose exec postgresql psql -U synapse -d synapse -c &#34;VACUUM FULL VERBOSE&#34;;</code></li></ul></li></ul>

<p>DONE!</p>
]]></content:encoded>
      <guid>https://blog.passtheballsocrates.com/how-to-reclaim-database-space-from-synapse</guid>
      <pubDate>Thu, 06 Apr 2023 10:21:16 +0000</pubDate>
    </item>
  </channel>
</rss>