Erik Johnston
4b3403ca9b
Stream cache invalidations for room membership storage functions
2017-01-30 17:28:22 +00:00
Erik Johnston
1c13c9f6b6
Don't have such a large cache
2017-01-30 17:12:14 +00:00
Erik Johnston
c7a26b7c32
Fix unit tests
2017-01-30 17:11:24 +00:00
Erik Johnston
fd1c18c088
Use DB cache of joined users for presence
2017-01-30 17:00:24 +00:00
Erik Johnston
c2c9a78db9
Noop device key changes if they're the same
2017-01-30 16:55:04 +00:00
Erik Johnston
e75a779d9e
Fix query
2017-01-30 16:38:20 +00:00
Erik Johnston
828db669ec
Use get_users_in_room and declare it iterable
2017-01-30 16:37:22 +00:00
Erik Johnston
9636b2407d
Merge pull request #1857 from matrix-org/erikj/device_list_stream
...
Implement device lists updates over federation
2017-01-30 14:35:21 +00:00
Erik Johnston
3670025e64
Rename func
2017-01-30 14:11:31 +00:00
Erik Johnston
4ac363a168
Remove debug logging
2017-01-30 14:10:12 +00:00
Erik Johnston
d360c97ae1
Clear out old destination pokes.
2017-01-30 10:14:37 +00:00
Erik Johnston
76100203ab
Always use the latest stream_id, sent or unsent
2017-01-30 10:14:25 +00:00
Erik Johnston
d1e1fd6210
Add ts column to device_lists_outbound_pokes
2017-01-27 15:23:48 +00:00
Erik Johnston
252b503fc8
Hook device list updates to replication
2017-01-27 14:31:35 +00:00
Erik Johnston
84a35f32c7
Comment
2017-01-27 10:35:12 +00:00
Erik Johnston
c517a19c2d
Comment
2017-01-27 10:33:26 +00:00
Erik Johnston
738a2867c8
SQL param ordering
2017-01-27 10:31:29 +00:00
Erik Johnston
755adff0e4
User if rather than for
2017-01-27 10:31:06 +00:00
Erik Johnston
888c59c955
Better name
2017-01-27 10:29:47 +00:00
Erik Johnston
f25a4a4692
Remove unused param
2017-01-27 10:27:39 +00:00
Erik Johnston
31aca5589c
Fix on sqlite: use left rather than outer join
2017-01-26 16:55:50 +00:00
Erik Johnston
76d40f4904
Handle users leaving rooms
2017-01-26 16:39:33 +00:00
Erik Johnston
fbfad76c03
Add comments
2017-01-26 16:33:21 +00:00
Erik Johnston
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
Erik Johnston
51e9fe36e4
Fix up sending of m.device_list_update edus
2017-01-25 16:55:21 +00:00
Erik Johnston
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
Paul "LeoNerd" Evans
10e48d8310
Don't clobber a displayname or avatar_url if provided by an m.room.member event
2017-01-24 18:06:07 +00:00
Erik Johnston
ba8e144554
Merge branch 'erikj/current_state_fix' into develop
2017-01-23 16:15:10 +00:00
Erik Johnston
f5b46482f4
Merge pull request #1840 from matrix-org/erikj/current_state_fix
...
Insert delta of current_state_events to be more efficient
2017-01-23 16:14:34 +00:00
Erik Johnston
fdf2a31a51
Typo
2017-01-23 16:14:14 +00:00
Erik Johnston
c77b24c092
Refactor to calculate state delta outside transaction
2017-01-23 14:51:33 +00:00
Erik Johnston
5d2134d485
Comments
2017-01-20 17:13:24 +00:00
Erik Johnston
a55fa2047f
Insert delta of current_state_events to be more efficient
2017-01-20 17:10:18 +00:00
Erik Johnston
3d9d48fffb
Merge pull request #1836 from matrix-org/erikj/current_state_fix
...
Derive current_state_events from state groups
2017-01-20 15:14:05 +00:00
Erik Johnston
d0897dead5
Spelling
2017-01-20 15:05:11 +00:00
Erik Johnston
567aa35b67
Update all call sites after rename
2017-01-20 14:40:31 +00:00
Erik Johnston
f2f40e64a9
Comments
2017-01-20 14:38:13 +00:00
Erik Johnston
4c6a31cd6e
Calculate the forward extremeties once
2017-01-20 14:28:53 +00:00
Erik Johnston
86063d4321
Merge pull request #1835 from matrix-org/erikj/fix_workers
...
Make worker listener config backwards compat
2017-01-20 11:55:56 +00:00
Erik Johnston
09eb08f910
Derive current_state_events from state groups
2017-01-20 11:52:51 +00:00
Erik Johnston
97efe99ae9
Make worker listener config backwards compat
2017-01-20 11:45:29 +00:00
Marvin Steadfast
86e6165687
Added default config for turn username and password
2017-01-19 14:35:55 +01:00
Marvin Steadfast
1e38be3a7a
Added username and password for turn server
...
It makes it possible to use a turn server that needs a username and
password instead of a token.
2017-01-19 14:08:20 +01:00
Erik Johnston
841c228533
Merge pull request #1828 from matrix-org/erikj/iterable_cache_size
...
Update LruCache size estimate on clear
2017-01-18 14:57:54 +00:00
Erik Johnston
c430111d0e
Update LruCache size estimate on clear
2017-01-18 14:55:23 +00:00
David Baker
97d3918377
Merge pull request #1811 from aperezdc/unhardcode-riot-urls
...
Allow configuring the Riot URL used in notification emails
2017-01-18 14:38:49 +00:00
David Baker
8c5009b628
Lowercase all email addresses before querying db
...
Since we store all emails in the DB in lowercase
(https://github.com/matrix-org/synapse/pull/1170 )
2017-01-18 13:25:56 +00:00
Erik Johnston
ae7b4da4cc
Merge pull request #1823 from matrix-org/erikj/load_events_logs
...
Remove loading events logs
2017-01-18 11:07:58 +00:00
Erik Johnston
fc7cae8aa3
Merge pull request #1824 from matrix-org/erikj/retry_host_log
...
Lower the not retrying host log line to debug
2017-01-18 11:07:51 +00:00
Erik Johnston
f9058ca785
Merge pull request #1822 from matrix-org/erikj/statE_logging
...
Change resolve_state_groups call site logging to DEBUG
2017-01-18 11:02:03 +00:00
Erik Johnston
f648313f98
Merge pull request #1821 from matrix-org/erikj/cache_metrics_string_intern
...
Measure metrics of string_cache
2017-01-18 10:57:39 +00:00
Erik Johnston
15f012032c
Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split
...
Optimise state resolution
2017-01-18 10:53:00 +00:00
Erik Johnston
4ec1cf49e2
Lower loading events log to DEBUG
2017-01-17 17:28:32 +00:00
Erik Johnston
f878f64f43
Lower the not retrying host log line to debug
2017-01-17 17:20:39 +00:00
Erik Johnston
5f027d1fc5
Change resolve_state_groups call site logging to DEBUG
2017-01-17 17:07:15 +00:00
Erik Johnston
380dba1020
Measure metrics of string_cache
2017-01-17 17:04:46 +00:00
Erik Johnston
ed4d176152
PEP8
2017-01-17 15:27:28 +00:00
Mark Haines
c6064a7ba6
Only construct sets when necessary
2017-01-17 15:23:07 +00:00
Erik Johnston
a8594fd19f
Use better names
2017-01-17 14:59:03 +00:00
Erik Johnston
7fae460402
Merge pull request #1820 from matrix-org/erikj/push_tools
...
Get state at event rather than for room in push
2017-01-17 14:58:26 +00:00
Erik Johnston
37b4c7d8a9
Fix typo in return type
2017-01-17 14:43:32 +00:00
Erik Johnston
e5d2df9c34
Use better variable name
2017-01-17 14:32:53 +00:00
Erik Johnston
04006bb7f0
Get state at event rather than for room in push
2017-01-17 14:31:21 +00:00
Erik Johnston
ce59a2faad
Correctly handle case of rejected events in state res
2017-01-17 14:18:53 +00:00
Erik Johnston
633f97151c
Check event is in state_map
2017-01-17 13:33:54 +00:00
Erik Johnston
e6153e1bd1
Fix couple of federation state bugs
2017-01-17 13:22:34 +00:00
Erik Johnston
5d6bad1b3c
Optimise state resolution
2017-01-17 13:22:19 +00:00
Erik Johnston
e8ecbb6f20
Merge pull request #1812 from matrix-org/erikj/state_auth_splitout_split
...
Split out static state methods from StateHandler
2017-01-17 11:55:18 +00:00
Erik Johnston
d6c75cb7c2
Rename and comment tree_to_leaves_iterator
2017-01-17 11:47:03 +00:00
Erik Johnston
1ccd5676e3
Remove needless call to evict()
2017-01-17 11:42:26 +00:00
Erik Johnston
d906206049
Increase state_group_cache_size
2017-01-17 11:31:08 +00:00
Erik Johnston
f85b6ca494
Speed up cache size calculation
...
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.
This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
2017-01-17 11:18:13 +00:00
Erik Johnston
6d00213e80
Use OrderedDict in ExpiringCache
2017-01-16 15:33:22 +00:00
Erik Johnston
897f8752da
Up cache max entries for state
2017-01-16 15:08:17 +00:00
Erik Johnston
beda469bc6
Put staticmethods at module level
2017-01-16 15:05:24 +00:00
Erik Johnston
46aebbbcbf
Add support for 'iterable' to ExpiringCache
2017-01-16 14:57:23 +00:00
Erik Johnston
01521299c7
Increase cache size limit
2017-01-16 11:56:51 +00:00
Erik Johnston
2fae34bd2c
Optionally measure size of cache by sum of length of values
2017-01-13 17:46:17 +00:00
Erik Johnston
95a22ae194
Merge pull request #1810 from matrix-org/erikj/state_auth_splitout_split
...
Split out static auth methods from Auth object
2017-01-13 16:32:27 +00:00
Erik Johnston
ec0a523ac3
Split out static state methods from StateHandler
2017-01-13 15:25:06 +00:00
Erik Johnston
e178feca3f
Remove unused function
2017-01-13 15:16:45 +00:00
Erik Johnston
f0325a9ccc
Merge pull request #1793 from matrix-org/erikj/change_device_inbox_index
...
Change device_inbox stream index to include user
2017-01-13 15:14:51 +00:00
Erik Johnston
c050f493dd
Add comment
2017-01-13 15:14:41 +00:00
Adrian Perez de Castro
a3e4a198e3
Allow configuring the Riot URL used in notification emails
...
The URLs used for notification emails were hardcoded to use either matrix.to
or vector.im; but for self-hosted setups where Riot is also self-hosted it
may be desirable to allow configuring an alternative Riot URL.
Fixes #1809 .
Signed-off-by: Adrian Perez de Castro <aperez@igalia.com>
2017-01-13 17:12:04 +02:00
Erik Johnston
8b2fa38256
Split event auth code into seperate module
2017-01-13 15:07:32 +00:00
Erik Johnston
641ccdbb14
Merge pull request #1795 from matrix-org/erikj/port_defaults
...
Restore default bind address
2017-01-13 13:02:59 +00:00
Erik Johnston
ebf94aff8d
Fix spurious Unhandled Error log lines
2017-01-12 17:19:47 +00:00
Erik Johnston
bf5c9706d9
Remove full_twisted_stacktraces option
...
The debug 'full_twisted_stacktraces' flag caused synapse to rewrite
twisted deferreds to always fire the callback on the next reactor tick.
This was to force the deferred to always store the stacktraces on
exceptions, and thus be more likely to have a full stacktrace when it
reaches the final error handlers and gets printed to the logs.
Dynamically rewriting things is generally bad, and in particular this
change violates assumptions of various bits of Twisted. This wouldn't
necessarily be so bad, but it turns out this option has been turned on
on some production servers.
Turning the option can cause e.g. #1778 .
For now, lets just entirely nuke this option.
2017-01-12 10:32:52 +00:00
Erik Johnston
7b62d0bc70
Add missing None check
2017-01-11 10:57:03 +00:00
Erik Johnston
7e6c2937c3
Split out static auth methods from Auth object
2017-01-10 18:16:54 +00:00
Erik Johnston
b1dfd20292
Pop bind_address
2017-01-10 17:23:18 +00:00
Erik Johnston
edd6cdfc9a
Restore default bind address
2017-01-10 17:21:41 +00:00
Erik Johnston
8a0fddfd73
Remove spurious for..else..
2017-01-10 16:30:53 +00:00
Erik Johnston
d524bc9110
Merge pull request #1792 from matrix-org/erikj/limit_cache_prefill_device
...
Limit number of entries to prefill from cache
2017-01-10 15:42:00 +00:00
Erik Johnston
d2b00d0866
Merge pull request #1790 from matrix-org/erikj/linearizer
...
Add paranoia exception catch in Linearizer
2017-01-10 15:38:30 +00:00
Erik Johnston
ab655dca33
Explicitly close the cursor
2017-01-10 15:15:25 +00:00
Erik Johnston
5a32e9273e
Don't disable autocommit
2017-01-10 15:11:27 +00:00
Erik Johnston
caddadfc5a
Change device_inbox stream index to include user
...
This makes fetching the nost recently changed users much tricker, and
brings it in line with e.g. presence_stream indices.
2017-01-10 15:04:57 +00:00
Erik Johnston
dd52d4de4c
Limit number of entries to prefill from cache
...
Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.
In future, we should add a better index to make it even faster.
2017-01-10 14:34:50 +00:00
Mark Haines
32019c9897
Log which files we saved attachments to in the media_repository
2017-01-10 14:19:50 +00:00
Erik Johnston
657488113e
Merge pull request #1789 from matrix-org/erikj/decouple_presence
...
Don't block messages sending on bumping presence
2017-01-10 14:06:05 +00:00
Erik Johnston
3b4de17d2b
Comment
2017-01-10 14:05:53 +00:00
Erik Johnston
7d0981b312
Merge pull request #1787 from matrix-org/erikj/linearize_member
...
Linearize updates to membership via PUT /state/
2017-01-10 14:04:54 +00:00
Erik Johnston
f477370c0c
Add paranoia exception catch in Linearizer
2017-01-10 14:04:13 +00:00
Erik Johnston
586f474a44
Don't block messages sending on bumping presence
2017-01-10 12:46:00 +00:00
Erik Johnston
6823fe5241
Linearize updates to membership via PUT /state/
2017-01-09 18:25:13 +00:00
Erik Johnston
f7085ac84f
Name linearizer's for better logs
2017-01-09 17:17:10 +00:00
Erik Johnston
9898bbd9dc
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-01-09 14:51:17 +00:00
Erik Johnston
9a8ae6f1bf
Bump version and changelog
2017-01-09 14:47:56 +00:00
Matthew Hodgson
2f4b2f4783
gah, fix mangled merge of 0.18.7 into develop
2017-01-07 04:00:42 +00:00
Matthew
6d363cea9d
Merge branch 'release-v0.18.7' into develop
2017-01-07 03:46:16 +00:00
Matthew
f0e4bac64e
bump changelog & version
2017-01-07 03:45:38 +00:00
Matthew
4304e7e593
do the discard check in the right place to avoid grabbing dependent events
2017-01-07 03:44:18 +00:00
Matthew Hodgson
8c48971b51
Merge branch 'release-v0.18.7' into develop
2017-01-07 02:23:37 +00:00
Matthew
e10c527930
Discard PDUs from invalid origins due to #1753 in 0.18.[56]
2017-01-07 02:13:14 +00:00
Matthew Hodgson
2f5be2d8dc
oops, this should have been rc1
2017-01-07 01:11:56 +00:00
Matthew Hodgson
4086026524
move logging to right place
2017-01-07 00:41:46 +00:00
Matthew
19e2fb4386
bump version
2017-01-06 23:38:22 +00:00
Matthew
b2850e62db
fix typo breaking the fix to #1753
2017-01-06 23:23:37 +00:00
Mark Haines
06c00bd19b
Merge branch 'release-v0.18.6' into develop
2017-01-06 14:46:27 +00:00
Mark Haines
b42a972b71
Bump version and changelog
2017-01-06 14:44:28 +00:00
Patrik Oldsberg
1ef6084b75
handlers/room_member: fix guest access check when joining rooms
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2017-01-06 14:36:56 +00:00
Matthew Hodgson
bd85434cb3
Merge branch 'release-v0.18.6' into develop
2017-01-05 13:58:19 +00:00
Mark Haines
c18f7fc410
Fix flake8 and update changelog
2017-01-05 13:50:22 +00:00
Matthew Hodgson
dafd50d178
Merge pull request #1767 from matrix-org/matthew/resolve_state_group_logging
...
log call paths for resolve_state_group
2017-01-05 13:47:42 +00:00
Matthew Hodgson
883ff92a7f
Fix case
2017-01-05 13:45:02 +00:00
Matthew Hodgson
d79d165761
add logging for all the places we call resolve_state_groups. my kingdom for a backtrace that actually works.
2017-01-05 13:40:39 +00:00
Matthew Hodgson
8cfc0165e9
fix annoying typos
2017-01-05 13:39:43 +00:00
Mark Haines
62451800e7
Bump version and changelog to v0.18.6-rc3
2017-01-05 13:36:10 +00:00
Matthew Hodgson
b31ed22738
Merge branch 'release-v0.18.6' into develop
2017-01-05 13:03:02 +00:00
Mark Haines
dd3df11c55
More logging for the linearizer and for get_events
2017-01-05 12:32:47 +00:00
Matthew Hodgson
468749c9fc
fix comment
2017-01-05 12:00:11 +00:00
Matthew Hodgson
eedf400d05
limit total timeout for get_missing_events to 10s
2017-01-05 11:58:15 +00:00
Matthew Hodgson
8e82611f37
fix comment
2017-01-05 11:44:44 +00:00
Mark Haines
f784980d2b
Only send events that originate on this server.
...
Or events that are sent via the federation "send_join" API.
This should match the behaviour from before v0.18.5 and #1635 landed.
2017-01-05 11:26:30 +00:00
Mark Haines
e02bdaf08b
Get the destinations from the state from before the event
...
Rather than the state after then event.
2017-01-04 15:17:15 +00:00
Mark Haines
b6b67715ed
Send ALL membership events to the server that was affected.
...
Send all membership changes to the server that was affected.
This ensures that if the last member of a room on a server
was kicked or banned they get told about it.
2017-01-04 13:56:20 +00:00
Matthew Hodgson
555d702e34
limit total timeout for get_missing_events to 10s
2016-12-31 15:21:37 +00:00
Matthew Hodgson
899a3a1268
Merge branch 'release-v0.18.6' into develop
2016-12-31 02:38:26 +00:00
Mark Haines
f3de4f8cb7
Bump version and changelog
2016-12-30 20:21:04 +00:00
Mark Haines
62ce3034f3
s/aquire/acquire/g
2016-12-30 20:04:44 +00:00
Mark Haines
0aff09f6c9
Add more useful logging when we block fetching events
2016-12-30 20:00:44 +00:00
Mark Haines
cc50b1ae53
Remove fallback from get_missing_events.
...
get_missing_events used to fallback to fetching the missing events
individually requesting from every server in the room, one by one.e
This could be unacceptably slow, possibly causing #1732
2016-12-30 18:13:15 +00:00
Mark Haines
f576c34594
Merge remote-tracking branch 'origin/release-v0.18.6' into develop
2016-12-30 15:13:49 +00:00
Mark Haines
822cb39dfa
Use the new twisted logging framework.
...
Hopefully adding an observer to the new framework will avoid a memory
leak https://twistedmatrix.com/trac/ticket/8164
2016-12-30 11:09:24 +00:00
Mark Haines
342fb8dae9
Merge branch 'release-v0.18.6' into develop
2016-12-29 17:33:46 +00:00
Mark Haines
f023be9293
Bump changelog and version
2016-12-29 16:18:04 +00:00
Mark Haines
97ffc5690b
Manually abort the underlying TLS connection.
...
The abort() method calls loseConnection() which tries to shutdown the
TLS connection cleanly. We now call abortConnection() directly which
should promptly close both the TLS connection and the underlying TCP
connection.
I also added some TODO markers to consider cancelling the old previous
timeout rather than checking time.time(). But given how urgently we want
to get this code released I'd rather leave the existing code with the
duplicate timeouts and the time.time() check.
2016-12-29 15:51:04 +00:00
Erik Johnston
b4bc6fef5b
Respect long_retries param and default to off
2016-12-29 00:58:34 +00:00
Erik Johnston
68030fd37b
Spelling and comments
2016-12-29 00:10:49 +00:00