Erik Johnston
483ba85c7a
Analyze user_ips before running deduplication
...
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
2019-02-12 11:55:27 +00:00
Erik Johnston
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating
...
add updating of backup versions
2019-02-11 13:45:49 +00:00
Richard van der Hoff
56710c7df5
Fix 'no unique or exclusion constraint' error ( #4591 )
...
Add more tables to the list of tables which need a background update to
complete before we can upsert into them, which fixes a race against the
background updates.
2019-02-08 18:30:46 +00:00
Hubert Chathi
8248637173
add new endpoint to update backup versions
2019-02-06 17:57:10 -05:00
Andrew Morgan
627ecd358e
Filter user directory state query to a subset of state events ( #4462 )
...
* Filter user directory state query to a subset of state events
* Add changelog
2019-02-05 12:16:28 +00:00
Andrew Morgan
c45fd0dda0
Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable
...
Copy over non-federatable trait on room upgrade
2019-01-31 20:28:37 +00:00
Andrew Morgan
d239f67c25
Raise an exception instead of returning None
2019-01-31 18:34:15 +00:00
Andrew Morgan
3ed3cb4339
New function for getting room's create event
2019-01-31 18:21:39 +00:00
Richard van der Hoff
e707e7b38d
Fix infinite loop when an event is redacted in a v3 room ( #4535 )
2019-01-31 15:34:17 +00:00
Amber Brown
3f189c902e
Fix flake8 ( #4519 )
2019-01-30 10:53:17 +00:00
Erik Johnston
a696c48133
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah
2019-01-29 22:00:33 +00:00
Erik Johnston
c21b7cbc09
Update synapse/storage/events_worker.py
2019-01-29 21:53:48 +00:00
Erik Johnston
4db252c073
Check redaction state when event is pulled out of the database
2019-01-29 21:48:36 +00:00
Erik Johnston
6d23ec2111
Fix typo
2019-01-29 21:45:53 +00:00
Erik Johnston
b5d510ad64
Remove unused arg
2019-01-29 21:45:28 +00:00
Richard van der Hoff
82165eeb05
Update synapse/storage/events_worker.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-01-29 21:14:39 +00:00
Erik Johnston
7709d2bd16
Implement rechecking of redactions
2019-01-29 11:56:20 +00:00
Erik Johnston
be47cfa9c9
Refactor event building into EventBuilder
...
This is so that everything is done in one place, making it easier to
change the event format based on room version
2019-01-29 11:13:00 +00:00
Amber Brown
94fb63e44f
Fix typo in upserts code ( #4505 )
...
* fix obvious problem :|
* changelog
2019-01-29 10:04:23 +00:00
Amber Brown
7072fe3084
Fix UPSERTs on SQLite 3.24+ ( #4477 )
2019-01-28 15:43:32 +00:00
Andrew Morgan
4026d555fa
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-28 14:08:24 +00:00
Erik Johnston
b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
...
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
Erik Johnston
431e485914
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sqlite_native_upsert
2019-01-25 14:11:17 +00:00
Andrew Morgan
0b3fd1401f
Don't require sqlite3 when using postgres ( #4466 )
2019-01-25 22:25:02 +11:00
Andrew Morgan
821b65aeb5
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-25 11:09:53 +00:00
Andrew Morgan
b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
...
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
Erik Johnston
1953067136
Disable native upserts for sqlite, as they don't work
2019-01-25 10:46:49 +00:00
Erik Johnston
678a92cb56
Replace missed usages of FrozenEvent
2019-01-25 10:32:30 +00:00
Erik Johnston
be6a7e47fa
Revert "Require event format version to parse or create events"
2019-01-25 10:23:51 +00:00
Erik Johnston
efb8ed1d45
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/require_format_version
2019-01-24 18:52:34 +00:00
Erik Johnston
80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
...
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
Erik Johnston
664b7a2920
Merge pull request #4437 from matrix-org/erikj/event_format_version_v2
...
Add support for persisting event format versions
2019-01-24 18:32:30 +00:00
Erik Johnston
b8082a5445
Use term 'out of band membership' instead
2019-01-24 17:33:19 +00:00
Andrew Morgan
a383289b0d
Merge branch 'anoa/full_search_upgraded_rooms' of github.com:matrix-org/synapse into anoa/full_search_upgraded_rooms
2019-01-24 17:23:51 +00:00
Andrew Morgan
e1781b043b
Remove redundant create event None check
2019-01-24 17:23:39 +00:00
Richard van der Hoff
03c85335d1
Apply suggestions from code review
...
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:22:09 +00:00
Amber Brown
0e27501ee5
Fix UPSERT check ( #4459 )
2019-01-24 22:57:41 +11:00
Erik Johnston
e8c9f15397
Replace missed usages of FrozenEvent
2019-01-24 11:14:07 +00:00
Amber Brown
58f6c48183
Use native UPSERTs where possible ( #4306 )
2019-01-24 21:31:54 +11:00
Erik Johnston
17898a5ba6
Merge branch 'erikj/fixup_rejecting_invites' of github.com:matrix-org/synapse into erikj/require_format_version
2019-01-23 20:13:07 +00:00
Erik Johnston
7c288c2250
Clarify the invite flows
2019-01-23 20:07:47 +00:00
Erik Johnston
886e5acc76
Store rejected remote invite events as outliers
...
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.
In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
2019-01-23 20:04:57 +00:00
Erik Johnston
2a8edbaf74
Merge branch 'erikj/get_pdu_versions' into erikj/require_format_version
2019-01-23 17:27:49 +00:00
Erik Johnston
be1065af59
isort
2019-01-23 11:48:16 +00:00
Erik Johnston
c5a296b10c
Add support for persisting event format versions
...
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.
We specify the format version rather than the room version as:
1. We don't necessarily know the room version, existing events may be
either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
events in the same room reported to be of different versions, which
sounds annoying.
2019-01-23 11:30:01 +00:00
Erik Johnston
90743c9d89
Fixup removal of duplicate user_ips
rows ( #4432 )
...
* Remove unnecessary ORDER BY clause
* Add logging
* Newsfile
2019-01-23 19:45:18 +11:00
Erik Johnston
7f503f83b9
Refactor to rewrite the SQL instead
2019-01-22 16:31:05 +00:00
Erik Johnston
1c9704f8ab
Don't shadow params
2019-01-22 16:20:33 +00:00
Erik Johnston
2557531f0f
Fix bug when removing duplicate rows from user_ips
...
This was caused by accidentally overwritting a `last_seen` variable
in a for loop, causing the wrong value to be written to the progress
table. The result of which was that we didn't scan sections of the table
when searching for duplicates, and so some duplicates did not get
deleted.
2019-01-22 13:33:46 +00:00
Andrew Morgan
277e50462d
Do not return in a deferred function
2019-01-22 12:40:26 +00:00