Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-22 09:34:58 -05:00
Code Issues Packages Projects Releases Wiki Activity
19,231 Commits 4 Branches 609 Tags 60 MiB
947dbbdfd1
Commit Graph

620 Commits

Author SHA1 Message Date
Erik Johnston
a4bf72c30c Censor redactions in DB after a month 2019-08-30 11:29:17 +01:00
Hubert Chathi
e3d3fbf63f Merge branch 'uhoreg/e2e_cross-signing_merged' into cross-signing_keys 2019-08-28 17:36:46 -07:00
Hubert Chathi
72d296a7f3 Merge branch 'develop' into uhoreg/e2e_cross-signing_merged 2019-08-28 17:33:44 -07:00
Hubert Chathi
3b0b22cb05 use stream ID generator instead of timestamp 2019-08-28 17:17:21 -07:00
Jorik Schellekens
8767b63a82
Propagate opentracing contexts through EDUs (#5852) 
Propagate opentracing contexts through EDUs
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2019-08-22 18:21:10 +01:00
Richard van der Hoff
c9f11d09fc
Add missing index on users_in_public_rooms. (#5894) 2019-08-22 10:43:13 +01:00
Richard van der Hoff
4dab867288
Drop some unused tables. (#5893) 
These tables are never used, so we may as well drop them.
 2019-08-21 13:16:28 +01:00
Hubert Chathi
b7f7cc7ace add the version field to the index for e2e_room_keys 2019-08-14 17:14:40 -07:00
Hubert Chathi
f63ba7a795
Cross-signing [1/4] -- hidden devices (#5759) 
* allow devices to be marked as "hidden"

This is a prerequisite for cross-signing, as it allows us to create other things
that live within the device namespace, so they can be used for signatures.
 2019-08-12 15:14:37 -07:00
Hubert Chathi
fac1cdc562 make changes from PR review 2019-08-01 21:51:19 -04:00
Hubert Chathi
336c546d6a Merge branch 'cross-signing_hidden' into cross-signing_keys 2019-08-01 16:31:40 -04:00
Hubert Chathi
c1f0a5636c Merge branch 'develop' into cross-signing_hidden 2019-07-30 23:33:06 -04:00
Hubert Chathi
bc95890822 Merge branch 'cross-signing_hidden' into cross-signing_keys 2019-07-30 23:21:26 -04:00
Hubert Chathi
781ade836b apply changes from PR review 2019-07-30 23:09:50 -04:00
Erik Johnston
123c04daa7 Don't recreate column 2019-07-30 16:42:48 +01:00
Erik Johnston
15056ca208 Fix current_state_events membership background update. 
Turns out not all rooms are in `rooms`, so lets fetch the room list from
`current_state_events`. We move the delta file to force it to be run
again.
 2019-07-30 14:51:41 +01:00
Erik Johnston
105e7f6ed3 Remove lost comment 2019-07-29 16:09:48 +01:00
Hubert Chathi
c659b9f94f allow uploading keys for cross-signing 2019-07-25 11:08:24 -04:00
Hubert Chathi
d1c7c2a98a allow devices to be marked as "hidden" 
This is a prerequisite for cross-signing, as it allows us to create other things
that live within the device namespace, so they can be used for signatures.
 2019-07-24 23:21:52 -04:00
Erik Johnston
32768e96d4 Add function to get all forgotten rooms for user 
This will allow us to efficiently filter out rooms that have been
forgotten in other queries without having to join against the
`room_memberships` table.
 2019-07-24 11:44:23 +01:00
Erik Johnston
ebc5ed1296 Update comment for new column 2019-07-19 13:29:02 +01:00
Erik Johnston
c618a5d348 Add background update for current_state_events.membership column 2019-07-18 14:16:01 +01:00
Erik Johnston
6de09e07a6 Add membership column to current_state_events table. 
It turns out that doing a join is surprisingly expensive for the DB to
do when room_membership table is larger than the disk cache.
 2019-07-18 14:15:57 +01:00
Richard van der Hoff
5f158ec039
Implement access token expiry (#5660) 
Record how long an access token is valid for, and raise a soft-logout once it
expires.
 2019-07-12 17:26:02 +01:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Brendan Abolivier
d0530382ee
Track deactivated accounts in the database (#5378) 2019-06-14 13:18:24 +01:00
Erik Johnston
1b8cb64393
Merge pull request #5320 from matrix-org/hawkowl/full-schema-v1 
Make a full SQL schema
 2019-06-06 18:35:38 +01:00
Amber H. Brown
ed872db8df fix maybe 2019-06-07 02:53:47 +10:00
Andrew Morgan
3719680ee4
Add ability to perform password reset via email without trusting the identity server (#5377) 
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.

This PR is a culmination of 3 smaller PRs which have each been separately reviewed:

* #5308
* #5345
* #5368
 2019-06-06 17:34:07 +01:00
Amber H. Brown
837e32ef55 just user dir? 2019-06-07 01:49:25 +10:00
Amber H. Brown
3b6645d3bf remove background updates that arent needed 2019-06-07 01:20:58 +10:00
Amber Brown
6362e3af14 add more comments 2019-06-06 04:20:35 +10:00
Amber Brown
2198b7ce2a add stuff in bg updates 2019-06-04 01:06:00 +10:00
Amber Brown
4e75c5e02a WHY IS THIS CALLED A SLIGHTLY DIFFERENT THING 2019-06-03 22:42:12 +10:00
Amber Brown
be452fc9ac more fix 2019-06-03 22:24:23 +10:00
Amber Brown
7f81b967ca fix schemas 2019-06-03 22:23:40 +10:00
Amber Brown
dc72b90cd6 full schema 2019-06-03 22:03:28 +10:00
Erik Johnston
31d44ec4bd
Merge pull request #5294 from matrix-org/erikj/speed_up_room_stats 
Speed up room stats background update
 2019-05-31 10:48:51 +01:00
Erik Johnston
5037326d66 Add indices. Remove room_ids accidentally added 
We have to do this by re-inserting a background update and recreating
tables, as the tables only get created during a background update and
will later be deleted.

We also make sure that we remove any entries that should have been
removed but weren't due to a race that has been fixed in a previous
commit.
 2019-05-31 10:26:56 +01:00
Erik Johnston
6cdfb0207e Add index to temp table 2019-05-30 15:02:31 +01:00
Erik Johnston
640fcbb07f Fixup comments and logging 2019-05-30 10:55:55 +01:00
Erik Johnston
d79c9994f4 Add DB bg update to cleanup extremities. 
Due to #5269 we may have extremities in our DB that we shouldn't have,
so lets add a cleanup task such to remove those.
 2019-05-29 15:11:26 +01:00
Brendan Abolivier
a97d4e218a
Merge pull request #5268 from matrix-org/babolivier/account_validity_fix_schema 
Fix schema update for account validity
 2019-05-28 10:30:07 +01:00
Brendan Abolivier
ba17de7fbc
Fix schema update for account validity 2019-05-28 10:11:38 +01:00
Richard van der Hoff
b75537beaf Store key validity time in the storage layer 
This is a first step to checking that the key is valid at the required moment.

The idea here is that, rather than passing VerifyKey objects in and out of the
storage layer, we instead pass FetchKeyResult objects, which simply wrap the
VerifyKey and add a valid_until_ts field.
 2019-05-23 11:52:22 +01:00
Amber Brown
4a30e4acb4
Room Statistics (#4338) 2019-05-21 11:36:50 -05:00
Erik Johnston
efe3c7977a Add simple send_relation API and track in DB 2019-05-15 13:36:51 +01:00
Brendan Abolivier
91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration 
Send out emails with links to extend an account's validity period
 2019-04-17 14:57:39 +01:00
Brendan Abolivier
20f0617e87 Send out emails with links to extend an account's validity period 2019-04-17 14:42:20 +01:00
Erik Johnston
e446921def
Merge pull request #5033 from matrix-org/erikj/fix_schema_delta 
Fix schema upgrade when dropping tables
 2019-04-10 10:22:35 +01:00
Brendan Abolivier
bfc8fdf1fc
Merge pull request #5027 from matrix-org/babolivier/account_expiration 
Add time-based account expiration
 2019-04-09 17:02:41 +01:00
Brendan Abolivier
747aa9f8ca Add account expiration feature 2019-04-09 16:46:04 +01:00
Erik Johnston
50d2a3059d Fix schema upgrade when dropping tables 
We need to drop tables in the correct order due to foreign table
constraints (on `application_services`), otherwise the DROP TABLE
command will fail.

Introduced in #4992.
 2019-04-09 14:39:18 +01:00
Richard van der Hoff
3352baac4b
Remove unused server_tls_certificates functions (#5028) 
These have been unused since #4120, and with the demise of perspectives, it is
unlikely that they will ever be used again.
 2019-04-08 21:50:18 +01:00
Neil Johnson
2d951686a7
drop tables listed in #1830 (#4992) 
Tables dropped: 
 * application_services, 
 * application_services_regex, 
 * transaction_id_to_pdu, 
 * stats_reporting
 * current_state_resets
 * event_content_hashes
 * event_destinations
 * event_edge_hashes
 * event_signatures
 * feedback
 * room_hosts
 * state_forward_extremities
 2019-04-08 15:37:26 +01:00
Neil Johnson
e8419554ff
Remove presence lists (#4989) 
Remove presence list support as per MSC 1819
 2019-04-03 11:11:15 +01:00
Erik Johnston
c75e2017f1 Fixup docstrings 2019-04-02 11:20:06 +01:00
Erik Johnston
3715c124b3 Grandfather in existing user threepids 
We assume, as we did before, that users bound their threepid to one of
the trusted identity servers. So we simply fill the new table with all
threepids in `user_threepids` joined with the trusted identity servers.
 2019-04-01 15:25:19 +01:00
Erik Johnston
1666c0696a Track IS used to bind 3PIDs 
This will then be used to know which IS to default to when unbinding the
threepid.
 2019-04-01 15:23:01 +01:00
Erik Johnston
e9eeca1314 Fix user directory background update (#4887) 2019-03-19 22:13:53 +11:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Amber Brown
797b6a63fc fixup 2019-03-13 01:17:51 +11:00
Amber Brown
10480c4348 fixup 2019-03-12 21:47:14 +11:00
Amber Brown
30a8deeb68 Merge remote-tracking branch 'origin/develop' into hawkowl/userdir-search 2019-03-11 21:12:30 +11:00
Amber Brown
4abc988c6a
initial 2019-03-11 21:11:36 +11:00
Erik Johnston
fe6c12e6cd Add comment to schema 2019-03-08 16:38:23 +00:00
Amber Brown
f6135d06cf
Rewrite userdir to be faster (#4537) 2019-03-07 01:22:53 -08:00
Erik Johnston
483ba85c7a Analyze user_ips before running deduplication 
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
 2019-02-12 11:55:27 +00:00
Erik Johnston
c5a296b10c Add support for persisting event format versions 
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.

We specify the format version rather than the room version as:

1. We don't necessarily know the room version, existing events may be
   either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
   events in the same room reported to be of different versions, which
   sounds annoying.
 2019-01-23 11:30:01 +00:00
Amber Brown
a35c66a00b
Remove duplicates in the user_ips table and add an index (#4370) 2019-01-12 06:21:50 +11:00
Neil Johnson
d2f7c4e6b1
create support user (#4141) 
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
 2018-12-14 18:20:59 +00:00
Aaron Raimist
704c5298f0
Drop sent_transactions 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2018-12-01 23:07:35 -06:00
David Baker
bca3b91c2d Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers 2018-11-09 18:35:02 +00:00
Richard van der Hoff
350f654e7b Add unique indexes to a couple of tables 
The indexes on device_lists_remote_extremeties can be unique, and they
therefore should, to ensure that the db remains consistent.
 2018-11-02 10:36:13 +00:00
David Baker
64fa557f80 Try & make it work on postgres 2018-10-30 09:51:04 +00:00
David Baker
563f9b61b1 Make e2e backup versions numeric in the DB 
We were doing max(version) which does not do what we wanted
on a column of type TEXT.
 2018-10-29 21:01:22 +00:00
Erik Johnston
056f099126 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-19 15:48:59 +01:00
David Baker
497444f1fd Don't reuse backup versions 
Since we don't actually delete the keys, just mark the versions
as deleted in the db rather than actually deleting them, then we
won't reuse versions.

Fixes https://github.com/vector-im/riot-web/issues/7448
 2018-10-05 15:08:36 +01:00
Erik Johnston
4917ff5523 Add state_group index to event_to_state_groups 
This is needed to efficiently check for unreferenced state groups during
purge.
 2018-10-04 16:03:08 +01:00
David Baker
bc74925c5b WIP e2e key backups 
Continues from uhoreg's branch

This just fixed the errcode on /room_keys/version if no backup and
updates the schema delta to be on the latest so it gets run
 2018-09-13 17:02:59 +01:00
Hubert Chathi
83caead95a
Merge branch 'develop' into e2e_backups 2018-08-24 11:44:26 -04:00
Matthew Hodgson
15d513f16f fix idiocies and so make tests pass 2018-08-12 19:14:31 -04:00
Matthew Hodgson
0abb205b47 blindly incorporate PR review - needs testing & fixing 2018-08-12 19:14:31 -04:00
Matthew Hodgson
69e51c7ba4 make /room_keys/version work 2018-08-12 19:14:31 -04:00
Matthew Hodgson
8ae64b270f implement /room_keys/version too (untested) 2018-08-12 19:14:31 -04:00
Matthew Hodgson
6b8c07abc2 make it work and fix pep8 2018-08-12 19:13:09 -04:00
Matthew Hodgson
53ace904b2 total WIP skeleton for /room_keys API 2018-08-12 18:23:10 -04:00
Neil Johnson
312ae74746 typos 2018-08-08 13:33:16 +01:00
Neil Johnson
16d78be315 make use of _simple_select_one_onecol, improved comments 2018-08-06 17:51:15 +01:00
Neil Johnson
165e067033 Revert "change monthly_active_users table to be a single column" 
This reverts commit ec716a35b2.
 2018-08-02 10:59:58 +01:00
Neil Johnson
ec716a35b2 change monthly_active_users table to be a single column 2018-08-01 17:54:37 +01:00
Neil Johnson
f9f5559971 fix comment 2018-08-01 12:03:42 +01:00
Neil Johnson
6ef983ce5c api into monthly_active_users table 2018-07-31 16:36:24 +01:00
Neil Johnson
df2235e7fa coding style 2018-07-31 13:16:20 +01:00
Richard van der Hoff
51d7df1915 Create the column nullable 
There's no real point in ever making the column non-nullable, and doing so
breaks the sytests.
 2018-07-26 14:54:04 +01:00
Richard van der Hoff
5c1d301fd9 Stop populating events.content 
This field is no longer read from, so we should stop populating it. Once we're
happy that this doesn't break everything, and a rollback is unlikely, we can
think about dropping the column.
 2018-07-26 14:43:02 +01:00
Richard van der Hoff
1b4d73fa52 comment on event_edges 2018-07-26 12:53:51 +01:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00
Erik Johnston
244484bf3c Revert "Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"" 
This reverts commit 1d009013b3.
 2018-06-25 13:42:55 +01:00
Richard van der Hoff
1d009013b3 Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility" 
This reverts commit ce0d911156, reversing
changes made to b4a5d767a9.
 2018-06-22 16:35:10 +01:00