Erik Johnston
816605a137
Merge pull request #2307 from matrix-org/erikj/user_ip_batch
...
Batch upsert user ips
2017-06-27 15:08:32 +01:00
Erik Johnston
78cefd78d6
Make workers report to master for user ip updates
2017-06-27 14:58:10 +01:00
Erik Johnston
a0a561ae85
Fix up client ips to read from pending data
2017-06-27 14:46:12 +01:00
Erik Johnston
ed3d0170d9
Batch upsert user ips
2017-06-27 13:37:04 +01:00
Erik Johnston
976128f368
Update version and changelog
2017-06-26 16:14:56 +01:00
Erik Johnston
d04d672a80
Merge pull request #2290 from matrix-org/erikj/ensure_round_trip
...
Reject local events that don't round trip the DB
2017-06-26 15:12:02 +01:00
Erik Johnston
1bce3e6b35
Remove unused variables
2017-06-26 14:03:27 +01:00
Erik Johnston
e3cbec10c1
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip
2017-06-26 14:02:44 +01:00
Erik Johnston
8abdd7b553
Fix up indices for users_who_share_rooms
2017-06-26 14:01:30 +01:00
Erik Johnston
ff13c5e7af
Merge pull request #2301 from xwiki-labs/push-redact-content
...
Add configuration parameter to allow redaction of content from push m…
2017-06-24 13:13:51 +01:00
Caleb James DeLisle
27bd0b9a91
Change the config file generator to more descriptive explanation of push.redact_content
2017-06-24 10:32:12 +02:00
Caleb James DeLisle
bce144595c
Fix TravisCI tests for PR #2301 - Fat finger mistake
2017-06-23 15:26:09 +02:00
Caleb James DeLisle
75eba3b07d
Fix TravisCI tests for PR #2301
2017-06-23 15:15:18 +02:00
Caleb James DeLisle
1591eddaea
Add configuration parameter to allow redaction of content from push messages for google/apple devices
2017-06-23 13:01:04 +02:00
Erik Johnston
7fe8ed1787
Store URL cache preview downloads seperately
...
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
Erik Johnston
44c722931b
Make some more params configurable
2017-06-22 14:59:52 +01:00
Erik Johnston
2d520a9826
Typo. ARGH.
2017-06-22 14:42:39 +01:00
Erik Johnston
24d894e2e2
Fix thinko in unhandled user spam
2017-06-22 14:39:05 +01:00
Matthew Hodgson
ccfcef6b59
Merge branch 'master' into develop
2017-06-22 13:03:44 +01:00
Erik Johnston
e0004aa28a
Add desc
2017-06-22 10:03:48 +01:00
Erik Johnston
b668112320
Merge pull request #2296 from matrix-org/erikj/dont_appserver_shar
...
Don't work out users who share room with appservice users
2017-06-21 14:50:24 +01:00
Erik Johnston
dae9a00a28
Initialise exclusive_user_regex
2017-06-21 14:19:33 +01:00
Erik Johnston
8177563ebe
Fix for workers
2017-06-21 13:57:49 +01:00
Krombel
4202fba82a
Merge branch 'develop' into avoid_duplicate_filters
2017-06-21 14:48:21 +02:00
Krombel
812c030e87
replaced json.dumps with encode_canonical_json
2017-06-21 14:48:12 +02:00
Erik Johnston
1217c7da91
Don't work out users who share room with appservice users
2017-06-21 12:00:41 +01:00
Erik Johnston
385dcb7c60
Handle thumbnail urls
2017-06-19 17:48:28 +01:00
Erik Johnston
b8b936a6ea
Add API to quarantine media
2017-06-19 17:39:21 +01:00
Erik Johnston
e5ae386ea4
Handle all cases of sending membership events
2017-06-19 16:07:54 +01:00
Erik Johnston
36e51aad3c
Remove unused import
2017-06-19 14:42:21 +01:00
Erik Johnston
b490299a3b
Change to create new room and join other users
2017-06-19 14:10:13 +01:00
Erik Johnston
5db7070dd1
Forget room
2017-06-19 12:40:29 +01:00
Erik Johnston
d7fe6b356c
Add shutdown room API
2017-06-19 12:37:27 +01:00
Erik Johnston
fcf01dd88e
Reject local events that don't round trip the DB
2017-06-19 11:33:40 +01:00
Johannes Löthberg
4f66312df8
python_dependencies: Use bcrypt module instead of py-bcrypt
...
py-bcrypt has been unmaintained for a long while, while bcrypt is
actively maintained. And since ff8b87118d
2017-06-17 17:39:35 +02:00
Erik Johnston
dfeca6cf40
Merge pull request #2286 from matrix-org/erikj/split_out_user_dir
...
Split out user directory to a separate process
2017-06-16 13:01:19 +01:00
Erik Johnston
6aa5bc8635
Initial worker impl
2017-06-16 11:47:11 +01:00
Erik Johnston
d8f47d2efa
Merge pull request #2280 from matrix-org/erikj/share_room_user_dir
...
Include users who you share a room with in user directory
2017-06-16 11:06:00 +01:00
Krombel
1ff419d343
allow Authorization header which handling got implemented in #1098
...
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-06-16 11:21:14 +02:00
Erik Johnston
fdf1ca30f0
Bump version and changelog
2017-06-15 12:59:06 +01:00
Erik Johnston
052c5d19d5
Merge pull request #2281 from matrix-org/erikj/phone_home_stats
...
Fix phone home stats
2017-06-15 12:46:23 +01:00
Erik Johnston
5ddd199870
Typo
2017-06-15 10:49:10 +01:00
Erik Johnston
a9d6fa8b2b
Include users who share room with requester in user directory
2017-06-15 10:17:21 +01:00
Erik Johnston
4564b05483
Implement updating users who share rooms on the fly
2017-06-15 10:17:17 +01:00
Erik Johnston
72613bc379
Implement initial population of users who share rooms table
2017-06-15 09:59:04 +01:00
Erik Johnston
ebcd55d641
Add DB schema for tracking users who share rooms
2017-06-15 09:45:48 +01:00
Erik Johnston
4b461a6931
Add some more stats
2017-06-15 09:39:39 +01:00
Erik Johnston
617304b2cf
Fix phone home stats
2017-06-14 19:47:15 +01:00
Erik Johnston
d9fd937e39
Fix user directory insertion due to missing room_id
2017-06-13 11:50:24 +01:00
Erik Johnston
505e7e8b9d
Fix up sql
2017-06-13 11:19:18 +01:00
Erik Johnston
6fd7e6db3d
Fix user dir to not assume existence of user
2017-06-13 11:11:26 +01:00
Erik Johnston
fdca6e36ee
Merge pull request #2274 from matrix-org/erikj/cache_is_host_joined
...
Add cache for is_host_joined
2017-06-13 10:58:53 +01:00
Erik Johnston
90ae0cffec
Merge pull request #2275 from matrix-org/erikj/tweark_user_directory_search
...
Tweak the ranking of PG user dir search
2017-06-13 10:58:43 +01:00
Erik Johnston
de4cb50ca6
Merge pull request #2276 from matrix-org/erikj/fix_user_di
...
Don't assume existence of events when updating user directory
2017-06-13 10:55:41 +01:00
Erik Johnston
48d2949416
Throw exception when not retrying when downloading media
2017-06-13 10:23:14 +01:00
Erik Johnston
6ae8373d40
Don't assume existance of events when updating user directory
2017-06-13 10:19:26 +01:00
Erik Johnston
b58e24cc3c
Tweak the ranking of PG user dir search
2017-06-13 10:16:31 +01:00
Erik Johnston
d53fe399eb
Add cache for is_host_joined
2017-06-13 09:56:18 +01:00
Erik Johnston
a837765e8c
Merge pull request #2266 from matrix-org/erikj/host_in_room
...
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
Erik Johnston
8060974344
Fix replication
2017-06-09 16:40:52 +01:00
Erik Johnston
b0d975e216
Comments
2017-06-09 16:25:42 +01:00
Erik Johnston
e54d7d536e
Cache state deltas
2017-06-09 16:24:00 +01:00
Erik Johnston
1e9b4d5a95
Merge pull request #2268 from matrix-org/erikj/entity_has_changed
...
Fix has_any_entity_changed
2017-06-09 15:30:55 +01:00
Erik Johnston
efc2b7db95
Rewrite conditional
2017-06-09 13:35:15 +01:00
Erik Johnston
bfd68019c2
Merge pull request #2267 from matrix-org/erikj/missing_notifier
...
Fix removing of pushers when using workers
2017-06-09 13:07:29 +01:00
Erik Johnston
1664948e41
Comment
2017-06-09 13:05:05 +01:00
Erik Johnston
935e588799
Tweak SQL
2017-06-09 13:01:23 +01:00
Erik Johnston
eed59dcc1e
Fix has_any_entity_changed
...
Occaisonally has_any_entity_changed would throw the error: "Set changed
size during iteration" when taking the max of the `sorteddict`. While
its uncertain how that happens, its quite inefficient to iterate over
the entire dict anyway so we change to using the more traditional
`bisect_*` functions.
2017-06-09 11:44:01 +01:00
Erik Johnston
2cac7623a5
Add missing notifier
2017-06-09 11:24:41 +01:00
Erik Johnston
298d83b340
Fix replication
2017-06-09 11:01:28 +01:00
Erik Johnston
0185b75381
Change is_host_joined to use current_state table
...
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston
7132e5cdff
Mark remote invite rejections as outliers
2017-06-09 10:08:18 +01:00
Erik Johnston
ea11ee09f3
Ensure we don't use unpersisted state group as prev group
2017-06-08 11:59:57 +01:00
Erik Johnston
c62c480dc6
Merge pull request #2259 from matrix-org/erikj/fix_state_woes
...
Fix bug where state_group tables got corrupted
2017-06-07 17:51:25 +01:00
Erik Johnston
197bd126f0
Fix bug where state_group tables got corrupted
...
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.
We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
2017-06-07 17:39:36 +01:00
Erik Johnston
f45f07ab86
Merge pull request #2258 from matrix-org/erikj/user_dir
...
Don't start user_directory handling on workers
2017-06-07 14:04:50 +01:00
Erik Johnston
a053ff3979
Merge pull request #2248 from matrix-org/erikj/state_fixup
...
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
Erik Johnston
ecdd2a3658
Don't start user_directory handling on workers
2017-06-07 12:02:53 +01:00
Erik Johnston
2f34ad31ac
Add some logging to user directory
2017-06-07 11:50:44 +01:00
Erik Johnston
64ed74c01e
When pruning, delete from device_lists_outbound_last_success
2017-06-07 11:20:47 +01:00
Erik Johnston
1a81a1898e
Keep pruning background task
2017-06-07 11:16:56 +01:00
Erik Johnston
6ba21bf2b8
Comments
2017-06-07 11:08:36 +01:00
Erik Johnston
09e4bc0501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup
2017-06-07 11:05:23 +01:00
Erik Johnston
6e2a7ee1bc
Remove spurious log lines
2017-06-07 11:05:17 +01:00
Erik Johnston
65f0513a33
Split up device_lists_outbound_pokes table for faster updates.
2017-06-07 11:02:38 +01:00
Erik Johnston
6f83c4537c
Increase size of IP cache
2017-06-07 10:18:44 +01:00
Erik Johnston
cca94272fa
Fix typo when getting app name
2017-06-06 11:50:07 +01:00
Erik Johnston
66b121b2fc
Fix wrong number of arguments
2017-06-06 11:46:38 +01:00
Erik Johnston
1a01af079e
Handle profile updates in user directory
2017-06-01 15:39:51 +01:00
Erik Johnston
4d039aa2ca
Fix sqlite
2017-06-01 14:58:48 +01:00
Erik Johnston
21e255a8f1
Split the table in two
2017-06-01 14:50:46 +01:00
Erik Johnston
d5477c7afd
Tweak search query
2017-06-01 13:28:01 +01:00
Erik Johnston
02a6108235
Tweak search query
2017-06-01 13:16:40 +01:00
Erik Johnston
7233341eac
Comments
2017-06-01 13:11:38 +01:00
Erik Johnston
8be6fd95a3
Check if host is still in room
2017-06-01 13:05:39 +01:00
Erik Johnston
59dbb47065
Remove spurious inlineCallbacks
2017-06-01 11:41:29 +01:00
Erik Johnston
9c7db2491b
Fix removing users
2017-06-01 11:36:50 +01:00
Erik Johnston
0fe6f3c521
Bug fixes and logging
...
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
2017-06-01 11:09:49 +01:00
Erik Johnston
036362ede6
Order by if they have profile info
2017-06-01 09:41:08 +01:00
Erik Johnston
a757dd4863
Use prefix matching
2017-06-01 09:40:37 +01:00
Erik Johnston
f5cc22bdc6
Comment on why arbitrary comments
2017-05-31 17:30:26 +01:00
Erik Johnston
5dd1b2c525
Use unique indices
2017-05-31 17:29:12 +01:00
Erik Johnston
cc7609aa9f
Comment briefly on how we keep user_directory up to date
2017-05-31 17:11:18 +01:00
Erik Johnston
f1378aef91
Convert to int
2017-05-31 17:03:08 +01:00
Erik Johnston
b2d8d07109
Lifts things into separate function
2017-05-31 17:00:24 +01:00
Erik Johnston
f9791498ae
Typos
2017-05-31 16:50:57 +01:00
Erik Johnston
f091061711
Fix tests
2017-05-31 16:34:40 +01:00
Erik Johnston
4abcff0177
Fix typo
2017-05-31 16:22:36 +01:00
Erik Johnston
63c58c2a3f
Limit number of things we fetch out of the db
2017-05-31 16:17:58 +01:00
Erik Johnston
304880d185
Add stream change cache
2017-05-31 15:46:36 +01:00
Erik Johnston
5d79d728f5
Split out directory and search tables
2017-05-31 15:23:49 +01:00
Erik Johnston
dc51af3d03
Pull max id from correct table
2017-05-31 15:13:49 +01:00
Erik Johnston
350622a107
Handle the server leaving a public room
2017-05-31 15:11:36 +01:00
Erik Johnston
63fda37e20
Add comments
2017-05-31 15:00:29 +01:00
Erik Johnston
293ef29655
Weight differently
2017-05-31 14:29:32 +01:00
Erik Johnston
535c99f157
Use POST
2017-05-31 14:15:45 +01:00
Erik Johnston
45a5df5914
Add REST API
2017-05-31 14:11:55 +01:00
Erik Johnston
3b5f22ca40
Add search
2017-05-31 14:00:01 +01:00
Erik Johnston
b5db4ed5f6
Update room column when room becomes unpublic
2017-05-31 13:40:28 +01:00
Erik Johnston
168524543f
Add call later
2017-05-31 11:59:36 +01:00
Erik Johnston
3e123b8497
Start later
2017-05-31 11:56:27 +01:00
Erik Johnston
42137efde7
Don't go round in circles
2017-05-31 11:55:13 +01:00
Erik Johnston
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
Erik Johnston
dd48f7204c
Add comment
2017-05-30 15:01:22 +01:00
Erik Johnston
04095f7581
Add clobbered event_id
2017-05-30 14:53:01 +01:00
Erik Johnston
a584a81b3e
Add current_state_delta_stream table
2017-05-30 14:44:09 +01:00
Erik Johnston
619e8ecd0c
Handle None state group correctly
2017-05-26 10:46:03 +01:00
Erik Johnston
23da638360
Fix typing tests
2017-05-26 10:02:04 +01:00
Erik Johnston
dfbda5e025
Faster cache for get_joined_hosts
2017-05-25 17:24:44 +01:00
Erik Johnston
2b03751c3c
Don't return weird prev_group
2017-05-25 14:47:39 +01:00
Erik Johnston
dbc0dfd2d5
Remove unused options
2017-05-25 14:28:34 +01:00
Erik Johnston
11f139a647
Merge pull request #2247 from matrix-org/erikj/auth_event
...
Only store event_auth for state events
2017-05-24 16:46:34 +01:00
Erik Johnston
6e614e9e10
Add background task to clear out old event_auth
2017-05-24 15:23:34 +01:00
Erik Johnston
c049472b8a
Only store event_auth for state events
2017-05-24 15:23:31 +01:00
Erik Johnston
9a804b2812
Merge pull request #2243 from matrix-org/matthew/fix-url-preview-length-again
...
actually trim oversize og:description meta
2017-05-23 13:26:28 +01:00
Erik Johnston
fbbc40f385
Merge pull request #2237 from matrix-org/erikj/sync_key_count
...
Add count of one time keys to sync stream
2017-05-23 11:18:13 +01:00
Erik Johnston
8cf9f0a3e7
Remove redundant invalidation
2017-05-23 09:46:59 +01:00
Erik Johnston
e6618ece2d
Missed an invalidation
2017-05-23 09:36:52 +01:00
Matthew Hodgson
836d5c44b6
actually trim oversize og:description meta
2017-05-22 21:14:20 +01:00
Erik Johnston
11c2a3655f
Only load jinja2 templates once
...
Instead of every time a new email pusher is created, as loading jinja2
templates is slow.
2017-05-22 17:48:58 +01:00
Erik Johnston
539aa4d333
Merge pull request #2241 from matrix-org/erikj/fix_notifs
...
Correctly calculate push rules for member events
2017-05-22 16:46:58 +01:00
Erik Johnston
f85a415279
Add missing storage function to slave store
2017-05-22 16:31:24 +01:00
Erik Johnston
6489455bed
Comment
2017-05-22 16:22:04 +01:00
Erik Johnston
d668caa79c
Remove spurious log level guards
2017-05-22 16:21:06 +01:00
Erik Johnston
74bf4ee7bf
Stream count_e2e_one_time_keys cache invalidation
2017-05-22 16:19:22 +01:00
Erik Johnston
33ba90c6e9
Merge pull request #2240 from matrix-org/erikj/cache_list_fix
...
Update list cache to handle one arg case
2017-05-22 16:10:46 +01:00
Erik Johnston
bd7bb5df71
Pull out if statement from for loop
2017-05-22 15:12:19 +01:00
Erik Johnston
e3417a06e2
Update list cache to handle one arg case
...
We update the normal cache descriptors to handle caches with a single
argument specially so that the key wasn't a 1-tuple. We need to update
the cache list to be aware of this.
2017-05-22 15:04:42 +01:00
Erik Johnston
7fb80b5eae
Check if current event is a membership event
2017-05-22 15:02:12 +01:00
Erik Johnston
2d17b09a6d
Add debug logging
2017-05-22 15:01:36 +01:00
