Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-13 01:14:20 -05:00
Code Issues Packages Projects Releases Wiki Activity
16,169 Commits 4 Branches 609 Tags 60 MiB
2cad8baa70
Commit Graph

11355 Commits

Author SHA1 Message Date
Brendan Abolivier
8363588237
Fix typo 2020-01-15 19:13:22 +00:00
Brendan Abolivier
855af069a4
Fix instantiation of message retention purge jobs 
When figuring out which topological token to start a purge job at, we
need to do the following:

1. Figure out a timestamp before which events will be purged
2. Select the first stream ordering after that timestamp
3. Select info about the first event after that stream ordering
4. Build a topological token from that info

In some situations (e.g. quiet rooms with a short max_lifetime), there
might not be an event after the stream ordering at step 3, therefore we
abort the purge with the error `No event found`. To mitigate that, this
patch fetches the first event _before_ the stream ordering, instead of
after.
 2020-01-15 18:56:18 +00:00
Erik Johnston
19a1aac48c
Fix purge_room admin API (#6711) 2020-01-15 18:13:47 +00:00
Andrew Morgan
edc244eec4
Remove duplicate session check in web fallback servlet (#6702) 2020-01-15 18:05:18 +00:00
Richard van der Hoff
107f256cd8 Merge branch 'develop' into rav/module_api_extensions 2020-01-15 16:00:24 +00:00
Richard van der Hoff
8f5d7302ac
Implement RedirectException (#6687) 
Allow REST endpoint implemnentations to raise a RedirectException, which will
redirect the user's browser to a given location.
 2020-01-15 15:58:55 +00:00
Erik Johnston
28c98e51ff
Add local_current_membership table (#6655) 
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
 2020-01-15 14:59:33 +00:00
Erik Johnston
b5ce7f5874
Process EDUs in parallel with PDUs. (#6697) 
This means that things like to device messages don't get blocked behind
processing PDUs, which can potentially take *ages*.
 2020-01-14 14:08:35 +00:00
Erik Johnston
e8b68a4e4b
Fixup synapse.replication to pass mypy checks (#6667) 2020-01-14 14:08:06 +00:00
Andrew Morgan
1177d3f3a3
Quarantine media by ID or user ID (#6681) 2020-01-13 18:10:43 +00:00
Richard van der Hoff
326c893d24
Kill off RegistrationError (#6691) 
This is pretty pointless. Let's just use SynapseError.
 2020-01-13 12:48:22 +00:00
Richard van der Hoff
2d07c73777
Don't assign numeric IDs for empty usernames (#6690) 
Fix a bug where we would assign a numeric userid if somebody tried registering
with an empty username
 2020-01-13 12:47:30 +00:00
Richard van der Hoff
3cfac9593c
Merge pull request #6689 from matrix-org/rav/saml_mapping_provider_updates 
Updates to the SAML mapping provider API
 2020-01-13 12:44:55 +00:00
Richard van der Hoff
8039685051
Allow additional_resources to implement Resource directly (#6686) 
AdditionalResource really doesn't add any value, and it gets in the way for
resources which want to support child resources or the like. So, if the
resource object already implements the IResource interface, don't bother
wrapping it.
 2020-01-13 12:42:44 +00:00
Richard van der Hoff
feee819973
Fix exceptions on requests for non-ascii urls (#6682) 
Fixes #6402
 2020-01-13 12:41:51 +00:00
Richard van der Hoff
da4e52544e comment for run_in_background 2020-01-12 21:53:47 +00:00
Richard van der Hoff
dc69a1cf43 Pass client redirect URL into SAML mapping providers 2020-01-12 21:40:49 +00:00
Richard van der Hoff
47e63cc67a Pass the module_api into the SamlMappingProvider 
... for consistency with other modules, and because we'll need it sooner or
later and it will be a pain to introduce later.
 2020-01-12 21:40:49 +00:00
Richard van der Hoff
01243b98e1 Handle config not being set for synapse plugin modules 
Some modules don't need any config, so having to define a `config` property
just to keep the loader happy is a bit annoying.
 2020-01-12 21:34:36 +00:00
Richard van der Hoff
473d3801b6 Cleanups and additions to the module API 
Add some useful things, such as error types and logcontext handling, to the
API.

Make `hs` a private member to dissuade people from using it (hopefully
they aren't already).

Add a couple of new methods (`record_user_external_id` and
`generate_short_term_login_token`).
 2020-01-12 21:31:44 +00:00
Richard van der Hoff
bf46821180 Refuse to start if sqlite is older than 3.11.0 2020-01-09 18:11:04 +00:00
Richard van der Hoff
e48ba84e0b Check postgres version in check_database 
this saves doing it on each connection, and will allow us to pass extra options
in.
 2020-01-09 18:05:59 +00:00
Richard van der Hoff
e97d1cf001 Modify check_database to take a connection rather than a cursor 
We might not need the cursor at all.
 2020-01-09 18:05:50 +00:00
Manuel Stahl
d2906fe666 Allow admin users to create or modify users without a shared secret (#6495) 
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
 2020-01-09 13:31:00 +00:00
Erik Johnston
d773290cb1 Merge branch 'master' into develop 2020-01-09 13:25:48 +00:00
Erik Johnston
24b2c940fb 1.8.0 2020-01-09 11:39:29 +00:00
Erik Johnston
7c232bd98b
Merge pull request #6664 from matrix-org/erikj/media_admin_apis 
Fix media repo admin APIs when using a media worker.
 2020-01-08 15:50:06 +00:00
Erik Johnston
d74054afda Shuffle the code 2020-01-08 14:57:45 +00:00
Erik Johnston
bca3455b38 Comments 2020-01-08 14:27:35 +00:00
Erik Johnston
187dc6ad02 Do not rely on streaming events, as media repo doesn't 2020-01-08 14:24:28 +00:00
Richard van der Hoff
b064a41291 Merge remote-tracking branch 'origin/release-v1.8.0' into develop 2020-01-08 13:27:17 +00:00
Manuel Stahl
7caaa29daa Fix GET request on /_synapse/admin/v2/users endpoint (#6563) 
Fixes #6552
 2020-01-08 13:26:40 +00:00
Erik Johnston
1adf27c82a Import RoomStore in media worker to fix admin APIs 2020-01-08 13:26:20 +00:00
Erik Johnston
3cf7d6d5b6 Move media admin store functions to worker store 2020-01-08 13:26:20 +00:00
Richard van der Hoff
573fee759c
Back out ill-advised notary server hackery (#6657) 
This was ill-advised. We can't modify verify_keys here, because the response
object has already been signed by the requested key.

Furthermore, it's somewhat unnecessary because existing versions of Synapse
(which get upset that the notary key isn't present in verify_keys) will fall
back to a direct fetch via `/key/v2/server`.

Also: more tests for fetching keys via perspectives: it would be nice if we actually tested when our fetcher can't talk to our notary impl.
 2020-01-08 13:24:10 +00:00
Erik Johnston
be29ed7ad8
Correctly proxy remote group HTTP errors. (#6654) 
e.g. if remote returns a 404 then that shouldn't be treated as an error
but should be proxied through.
 2020-01-07 15:36:41 +00:00
Brendan Abolivier
2b6b7f482a
Merge pull request #6621 from matrix-org/babolivier/purge_job_config_typo 
Fix a typo in the purge jobs configuration example
 2020-01-07 16:17:40 +01:00
Brendan Abolivier
391fb47791
Reword 2020-01-07 14:54:32 +00:00
Brendan Abolivier
3a86477162
Change the example from 5min to 12h 
Have a purge job running every 5min is probably not something we want to advise admins to do as a sort-of default.
 2020-01-07 14:53:07 +00:00
Erik Johnston
7f0e706ebf 1.8.0rc1 2020-01-07 14:31:13 +00:00
Brendan Abolivier
0ab5853ec9
Merge pull request #6652 from matrix-org/babolivier/depth_missing_events 
Fix conditions failing if min_depth = 0
 2020-01-07 15:22:07 +01:00
Richard van der Hoff
85db7f73be
Add a background update to clear tombstoned rooms from the directory (#6648) 
* Add a background update to clear tombstoned rooms from the directory

* use the ABC metaclass
 2020-01-07 14:18:43 +00:00
Richard van der Hoff
9824a39d80
Async/await for background updates (#6647) 
so that bg update routines can be async
 2020-01-07 14:12:42 +00:00
Richard van der Hoff
1ff5491117
Merge pull request #6645 from matrix-org/rav/fix_synchrotron_error 
Fix exceptions in the synchrotron worker log when events are rejected.
 2020-01-07 14:02:14 +00:00
Brendan Abolivier
cd428a93e2
Fix conditions failing if min_depth = 0 
This could result in Synapse not fetching prev_events for new events in the room if it has missed some events.
 2020-01-07 12:08:58 +00:00
Richard van der Hoff
1807db5e73
Merge pull request #6629 from matrix-org/rav/kill_event_reference_hashes 
Remove a bunch of unused code from event creation
 2020-01-06 17:46:31 +00:00
Richard van der Hoff
26c5d3d398 Fix exceptions in log when rejected event is replicated 2020-01-06 17:16:28 +00:00
Richard van der Hoff
c74de81bfc async/await for SyncReplicationHandler.process_and_notify 2020-01-06 17:14:28 +00:00
Richard van der Hoff
bc42da4ab8 Clarify documentation on get_event* methods 
Make it clearer how they behave in the face of rejected and/or missing events.
 2020-01-06 17:12:06 +00:00
Richard van der Hoff
ba897a7590
Fix some test failures when frozen_dicts are enabled (#6642) 
Fixes #4026
 2020-01-06 15:22:46 +00:00
Erik Johnston
9f6c1befbb
Add experimental 'databases' config (#6580) 2020-01-06 14:44:01 +00:00
Richard van der Hoff
ab4b4ee6a7
Fix an error which was thrown by the PresenceHandler _on_shutdown handler. (#6640) 2020-01-06 14:34:02 +00:00
Richard van der Hoff
a7d2e5b37f Remove unused get_latest_event_ids_and_hashes_in_room 2020-01-06 13:45:33 +00:00
Richard van der Hoff
dc41fbf0dd Remove unused get_prev_events_and_hashes_for_room 2020-01-06 13:45:33 +00:00
Richard van der Hoff
38e0829a4c Remove unused hashes and depths from _update_membership params 2020-01-06 13:45:33 +00:00
Richard van der Hoff
3bef62488e Remove unused hashes and depths from create_event params 2020-01-06 13:45:33 +00:00
Richard van der Hoff
66ca914dc0 Remove unused hashes and depths from create_new_client_event params 2020-01-06 13:45:33 +00:00
Richard van der Hoff
15720092ac replace get_prev_events_and_hashes_for_room with get_prev_events_for_room in create_new_client_event 2020-01-06 13:45:33 +00:00
Richard van der Hoff
5a04781643 rename get_prev_events_for_room to get_prev_events_and_hashes_for_room 
... to make way for a new method which just returns the event ids
 2020-01-06 13:45:33 +00:00
Richard van der Hoff
4b36b482e0
Fix exception when fetching notary server's old keys (#6625) 
Lift the restriction that *all* the keys used for signing v2 key responses be
present in verify_keys.

Fixes #6596.
 2020-01-06 12:33:56 +00:00
Richard van der Hoff
18674eebb1
Workaround for error when fetching notary's own key (#6620) 
* Kill off redundant SynapseRequestFactory

We already get the Site via the Channel, so there's no need for a dedicated
RequestFactory: we can just use the right constructor.

* Workaround for error when fetching notary's own key

As a notary server, when we return our own keys, include all of our signing
keys in verify_keys.

This is a workaround for #6596.
 2020-01-06 12:28:58 +00:00
Andrew Morgan
01c3c6c929 Fix power levels being incorrectly set in old and new rooms after a room upgrade (#6633) 
Modify a copy of an upgraded room's PL before sending to the new room
 2020-01-06 09:53:07 +00:00
Richard van der Hoff
08815566bc
Automate generation of the sample and debian log configs (#6627) 2020-01-03 17:14:00 +00:00
Richard van der Hoff
e484101306
Raise an error if someone tries to use the log_file config option (#6626) 
This has caused some confusion for people who didn't notice it going away.
 2020-01-03 17:11:29 +00:00
Richard van der Hoff
98247c4a0e
Remove unused, undocumented "content repo" resource (#6628) 
This looks like it got half-killed back in #888.

Fixes #6567.
 2020-01-03 17:10:52 +00:00
Richard van der Hoff
b6b57ecb4e
Kill off redundant SynapseRequestFactory (#6619) 
We already get the Site via the Channel, so there's no need for a dedicated
RequestFactory: we can just use the right constructor.
 2020-01-03 14:19:48 +00:00
Richard van der Hoff
6964ea095b
Reduce the reconnect time when replication fails. (#6617) 2020-01-03 14:19:09 +00:00
Brendan Abolivier
4efe1d4d3f
Fix a typo in the purge jobs configuration example 2020-01-03 12:57:24 +01:00
Richard van der Hoff
dc96943d51 Merge branch 'master' into develop 2019-12-31 11:01:06 +00:00
Richard van der Hoff
77661ce81a 1.7.3 2019-12-31 10:45:12 +00:00
Richard van der Hoff
92eac974b9
Hacks to work around #6605 (#6608) 
When we have an event which refers to non-existent auth_events, ignore said events rather than exploding in a ball of fire.

Fixes #6605.
 2019-12-31 10:41:44 +00:00
Richard van der Hoff
56ad230efa Merge branch 'master' into develop 2019-12-20 11:11:34 +00:00
Richard van der Hoff
29794c6bc8 1.7.2 2019-12-20 10:58:07 +00:00
Erik Johnston
75d8f26ac8
Split state groups into a separate data store (#6296) 2019-12-20 10:48:24 +00:00
Richard van der Hoff
4caab0e95e
Backport fixes to sqlite upgrade from develop (#6578) 
Only run prepare_database on connection for in-memory databases.

Fixes #6569.
 2019-12-20 10:46:46 +00:00
Erik Johnston
fa780e9721
Change EventContext to use the Storage class (#6564) 2019-12-20 10:32:02 +00:00
Richard van der Hoff
03d3792f3c
Fix exceptions when attempting to backfill (#6576) 
Fixes #6575
 2019-12-20 09:55:45 +00:00
Erik Johnston
0b5dbadd96
Explode on duplicate delta file names. (#6565) 2019-12-19 15:07:37 +00:00
Erik Johnston
3d46124ad0
Port some admin handlers to async/await (#6559) 2019-12-19 15:07:28 +00:00
Richard van der Hoff
bca30cefee
Improve diagnostics on database upgrade failure (#6570) 
`Failed to upgrade database` is not helpful, and it's unlikely that UPGRADE.rst
has anything useful.
 2019-12-19 14:53:15 +00:00
Richard van der Hoff
0b794cbd7b
Fix sdnotify with acme enabled (#6571) 
If acme was enabled, the sdnotify startup hook would never be run because we
would try to add it to a hook which had already fired.

There's no need to delay it: we can sdnotify as soon as we've started the
listeners.
 2019-12-19 14:52:52 +00:00
Richard van der Hoff
b95b762560
Add an export_signing_key script (#6546) 
I want to do some key rotation, and it is silly that we don't have a way to do
this.
 2019-12-19 11:11:14 +00:00
Richard van der Hoff
d6752ce5da
Clean up startup for the pusher (#6558) 
* Remove redundant python2 support code

`str.decode()` doesn't exist on python3, so presumably this code was doing
nothing

* Filter out pushers with corrupt data

When we get a row with unparsable json, drop the row, rather than returning a
row with null `data`, which will then cause an explosion later on.

* Improve logging when we can't start a pusher

Log the ID to help us understand the problem

* Make email pusher setup more robust

We know we'll have a `data` member, since that comes from the database. What we
*don't* know is if that is a dict, and if that has a `brand` member, and if
that member is a string.
 2019-12-18 14:26:58 +00:00
Andrew Morgan
7963ca83cb
Add delta file to fix missing default table data (#6555) 2019-12-18 11:13:33 +00:00
Erik Johnston
2284eb3a53
Add database config class (#6513) 
This encapsulates config for a given database and is the way to get new
connections.
 2019-12-18 10:45:12 +00:00
Richard van der Hoff
91ccfe9f37 Merge branch 'master' into develop 2019-12-18 10:16:39 +00:00
Richard van der Hoff
6e8f8e14f2 Merge release-v1.7.1 into develop 2019-12-18 09:51:51 +00:00
Richard van der Hoff
d656e91fc2 1.7.1 2019-12-18 09:38:08 +00:00
Erik Johnston
5029422530
Fix bug where we added duplicate event IDs as auth_events (#6560) 2019-12-17 15:06:08 +00:00
Erik Johnston
02553901ce
Remove unused get_pagination_rows methods. (#6557) 
Remove unused get_pagination_rows methods
 2019-12-17 11:44:32 +00:00
Erik Johnston
5ca2cfadc3 Add auth events as per spec. (#6556) 
Previously we tried to be clever and filter out some unnecessary event
IDs to keep the auth chain small, but that had some annoying
interactions with state res v2 so we stop doing that for now.
 2019-12-16 17:05:01 +00:00
Erik Johnston
3fbe5b7ec3
Add auth events as per spec. (#6556) 
Previously we tried to be clever and filter out some unnecessary event
IDs to keep the auth chain small, but that had some annoying
interactions with state res v2 so we stop doing that for now.
 2019-12-16 16:59:32 +00:00
Brendan Abolivier
6316530366
Merge pull request #6553 from matrix-org/babolivier/fix-context-filter 
Use the filtered version of an event when responding to /context requests for that event
 2019-12-16 16:24:10 +00:00
Will Hunt
bfb95654c9 Add option to allow profile queries without sharing a room (#6523) 2019-12-16 16:11:55 +00:00
Brendan Abolivier
a820069549
Incorporate review 2019-12-16 16:00:18 +00:00
Richard van der Hoff
bbb75ff6ee Exclude rejected state events when calculating state at backwards extrems (#6527) 
This fixes a weird bug where, if you were determined enough, you could end up with a rejected event forming part of the state at a backwards-extremity. Authing that backwards extrem would then lead to us trying to pull the rejected event from the db (with allow_rejected=False), which would fail with a 404.
 2019-12-16 14:07:29 +00:00
Richard van der Hoff
ff773ff724 Persist auth/state events at backwards extremities when we fetch them (#6526) 
The main point here is to make sure that the state returned by _get_state_in_room has been authed before we try to use it as state in the room.
 2019-12-16 14:07:11 +00:00
Richard van der Hoff
83895316d4 sanity-checking for events used in state res (#6531) 
When we perform state resolution, check that all of the events involved are in
the right room.
 2019-12-16 13:53:53 +00:00
Richard van der Hoff
6577f2d887 Sanity-check room ids in event auth (#6530) 
When we do an event auth operation, check that all of the events involved are
in the right room.
 2019-12-16 13:47:34 +00:00
Richard van der Hoff
35bbe4ca79 Check the room_id of events when fetching room state/auth (#6524) 
When we request the state/auth_events to populate a backwards extremity (on
backfill or in the case of missing events in a transaction push), we should
check that the returned events are in the right room rather than blindly using
them in the room state or auth chain.

Given that _get_events_from_store_or_dest takes a room_id, it seems clear that
it should be sanity-checking the room_id of the requested events, so let's do
it there.
 2019-12-16 13:47:07 +00:00