Commit Graph

503 Commits

Author SHA1 Message Date
Erik Johnston
483ba85c7a Analyze user_ips before running deduplication
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
2019-02-12 11:55:27 +00:00
Erik Johnston
c5a296b10c Add support for persisting event format versions
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.

We specify the format version rather than the room version as:

1. We don't necessarily know the room version, existing events may be
   either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
   events in the same room reported to be of different versions, which
   sounds annoying.
2019-01-23 11:30:01 +00:00
Amber Brown
a35c66a00b
Remove duplicates in the user_ips table and add an index (#4370) 2019-01-12 06:21:50 +11:00
Neil Johnson
d2f7c4e6b1
create support user (#4141)
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
Aaron Raimist
704c5298f0
Drop sent_transactions
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2018-12-01 23:07:35 -06:00
David Baker
bca3b91c2d Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers 2018-11-09 18:35:02 +00:00
Richard van der Hoff
350f654e7b Add unique indexes to a couple of tables
The indexes on device_lists_remote_extremeties can be unique, and they
therefore should, to ensure that the db remains consistent.
2018-11-02 10:36:13 +00:00
David Baker
64fa557f80 Try & make it work on postgres 2018-10-30 09:51:04 +00:00
David Baker
563f9b61b1 Make e2e backup versions numeric in the DB
We were doing max(version) which does not do what we wanted
on a column of type TEXT.
2018-10-29 21:01:22 +00:00
Erik Johnston
056f099126 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups 2018-10-19 15:48:59 +01:00
David Baker
497444f1fd Don't reuse backup versions
Since we don't actually delete the keys, just mark the versions
as deleted in the db rather than actually deleting them, then we
won't reuse versions.

Fixes https://github.com/vector-im/riot-web/issues/7448
2018-10-05 15:08:36 +01:00
Erik Johnston
4917ff5523 Add state_group index to event_to_state_groups
This is needed to efficiently check for unreferenced state groups during
purge.
2018-10-04 16:03:08 +01:00
David Baker
bc74925c5b WIP e2e key backups
Continues from uhoreg's branch

This just fixed the errcode on /room_keys/version if no backup and
updates the schema delta to be on the latest so it gets run
2018-09-13 17:02:59 +01:00
Hubert Chathi
83caead95a
Merge branch 'develop' into e2e_backups 2018-08-24 11:44:26 -04:00
Matthew Hodgson
15d513f16f fix idiocies and so make tests pass 2018-08-12 19:14:31 -04:00
Matthew Hodgson
0abb205b47 blindly incorporate PR review - needs testing & fixing 2018-08-12 19:14:31 -04:00
Matthew Hodgson
69e51c7ba4 make /room_keys/version work 2018-08-12 19:14:31 -04:00
Matthew Hodgson
8ae64b270f implement /room_keys/version too (untested) 2018-08-12 19:14:31 -04:00
Matthew Hodgson
6b8c07abc2 make it work and fix pep8 2018-08-12 19:13:09 -04:00
Matthew Hodgson
53ace904b2 total WIP skeleton for /room_keys API 2018-08-12 18:23:10 -04:00
Neil Johnson
312ae74746 typos 2018-08-08 13:33:16 +01:00
Neil Johnson
16d78be315 make use of _simple_select_one_onecol, improved comments 2018-08-06 17:51:15 +01:00
Neil Johnson
165e067033 Revert "change monthly_active_users table to be a single column"
This reverts commit ec716a35b2.
2018-08-02 10:59:58 +01:00
Neil Johnson
ec716a35b2 change monthly_active_users table to be a single column 2018-08-01 17:54:37 +01:00
Neil Johnson
f9f5559971 fix comment 2018-08-01 12:03:42 +01:00
Neil Johnson
6ef983ce5c api into monthly_active_users table 2018-07-31 16:36:24 +01:00
Neil Johnson
df2235e7fa coding style 2018-07-31 13:16:20 +01:00
Richard van der Hoff
51d7df1915 Create the column nullable
There's no real point in ever making the column non-nullable, and doing so
breaks the sytests.
2018-07-26 14:54:04 +01:00
Richard van der Hoff
5c1d301fd9 Stop populating events.content
This field is no longer read from, so we should stop populating it. Once we're
happy that this doesn't break everything, and a rollback is unlikely, we can
think about dropping the column.
2018-07-26 14:43:02 +01:00
Richard van der Hoff
1b4d73fa52 comment on event_edges 2018-07-26 12:53:51 +01:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00
Erik Johnston
244484bf3c Revert "Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility""
This reverts commit 1d009013b3.
2018-06-25 13:42:55 +01:00
Richard van der Hoff
1d009013b3 Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"
This reverts commit ce0d911156, reversing
changes made to b4a5d767a9.
2018-06-22 16:35:10 +01:00
Richard van der Hoff
3ff8a619f5 UserErasureStore
to store which users have been erased
2018-06-12 09:52:22 +01:00
Neil Johnson
4986b084f8 remove unnecessary INSERT 2018-06-01 10:50:40 +01:00
Neil Johnson
ab0ef31dc7 create users index on creation_ts 2018-05-29 17:51:08 +01:00
Richard van der Hoff
9ea219c514 Send users a server notice about consent
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
2018-05-22 11:54:51 +01:00
Neil Johnson
dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics
user visit data
2018-05-16 10:09:24 +00:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler
ConsentResource to gather policy consent from users
2018-05-16 07:19:18 +01:00
Richard van der Hoff
47815edcfa ConsentResource to gather policy consent from users
Hopefully there are enough comments and docs in this that it makes sense on its
own.
2018-05-15 15:11:59 +01:00
Neil Johnson
977765bde2 Merge branch 'develop' of https://github.com/matrix-org/synapse into cohort_analytics 2018-05-14 09:31:42 +01:00
David Baker
46df23f581 Add the schema file 2018-05-09 15:07:54 +01:00
Neil Johnson
d0857702e8 add inidexes based on usage 2018-05-01 12:12:57 +01:00
Matthew Hodgson
99a54bf2af
Merge pull request #3129 from matrix-org/matthew/fix_group_dups
remove duplicates from groups tables
2018-04-30 11:47:25 +01:00
Matthew Hodgson
adaf3ec87f fix missing import 2018-04-28 22:39:15 +01:00
Matthew Hodgson
006e18b6bb pep8 2018-04-28 22:32:24 +01:00
Matthew Hodgson
42c89c8215 make it work with sqlite 2018-04-28 22:27:30 +01:00
Adrian Tschira
d82b6ea9e6 Move more xrange to six
plus a bonus next()

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-28 13:57:00 +02:00
Neil Johnson
617bf40924 Generate user daily stats 2018-04-25 17:37:29 +01:00
Matthew Hodgson
e3a373f002 remove duplicates from groups tables
and rename inconsistently named indexes.
Based on https://github.com/matrix-org/synapse/pull/3128 - thanks @vurpo\!
2018-04-25 14:58:43 +01:00