Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-22 08:14:57 -05:00
Code Issues Packages Projects Releases Wiki Activity
10,622 Commits 4 Branches 609 Tags 60 MiB
a8d8bf92e0
Commit Graph

395 Commits

Author SHA1 Message Date
Richard van der Hoff
4bd597d9fc push federation retry limiter down to matrixfederationclient 
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
 2017-03-23 09:28:46 +00:00
Richard van der Hoff
19b9366d73 Fix a couple of logcontext leaks 
Use preserve_fn to correctly manage the logcontexts around things we don't want
to yield on.
 2017-03-23 00:17:46 +00:00
Richard van der Hoff
95f21c7a66 Fix caching of remote servers' signature keys 
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing
its `num_args` parameter, which meant that it was returning the wrong key for
any server which had more than one recorded key.

By way of a fix, change the default for `num_args` to be *all* arguments. To
implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
 2017-03-22 15:11:30 +00:00
Richard van der Hoff
bd08ee7a46 Merge pull request #2026 from matrix-org/rav/logcontext_docs 
Logcontext docs
 2017-03-20 12:05:21 +00:00
Richard van der Hoff
f40c2db05a Stop preserve_fn leaking context into the reactor 
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.

Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
 2017-03-18 00:07:43 +00:00
Richard van der Hoff
d2d146a314 Logcontext docs 2017-03-17 23:59:28 +00:00
Richard van der Hoff
2abe85d50e Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join 
Queue up federation PDUs while a room join is in progress
 2017-03-17 11:32:44 +00:00
Richard van der Hoff
29ed09e80a Fix assertion to stop transaction queue getting wedged 
... and update some docstrings to correctly reflect the types being used.

get_new_device_msgs_for_remote can return a long under some circumstances,
which was being stored in last_device_list_stream_id_by_dest, and was then
upsetting things on the next loop.
 2017-03-15 12:16:55 +00:00
Richard van der Hoff
b5d1c68beb Implement reset_context_after_deferred 
to correctly reset the context when we fire off a deferred we aren't going to
wait for.
 2017-03-15 02:21:07 +00:00
David Baker
73a5f06652 Support registration / login with phone number 
Changes from https://github.com/matrix-org/synapse/pull/1971
 2017-03-13 17:27:51 +00:00
Erik Johnston
7eae6eaa2f Revert "Support registration & login with phone number" 2017-03-13 09:59:33 +00:00
Erik Johnston
3545e17f43 Add setdefault key to ExpiringCache 2017-03-10 10:30:49 +00:00
David Baker
9d0d40fc15 Docs 2017-03-08 19:05:29 +00:00
David Baker
3edc57296d Incorrectly copied copyright 
This file post-dates OM
 2017-03-08 19:00:51 +00:00
David Baker
1c99934b28 pep8 2017-03-08 11:58:20 +00:00
David Baker
a9e2b9ec16 Add msisdn util file 2017-03-08 11:53:36 +00:00
Erik Johnston
6b61060b51 Comment 2017-02-02 14:47:15 +00:00
Erik Johnston
9efcc3f3be Comment 2017-02-02 13:50:22 +00:00
Erik Johnston
df4ecff5a9 Correctly raise exceptions for ratelimitng. Ratelimit on 401 2017-02-01 15:42:19 +00:00
Erik Johnston
fe08db2713 Remove explicit < 400 check as apparently this is confusing 2017-01-31 15:21:32 +00:00
Erik Johnston
4c0ec15bdc Comment 2017-01-31 13:53:46 +00:00
Erik Johnston
85c590105f Comment 2017-01-31 13:46:38 +00:00
Erik Johnston
ae7a132f38 Better handle 404 response for federation /send/ 2017-01-31 13:40:09 +00:00
Erik Johnston
c430111d0e Update LruCache size estimate on clear 2017-01-18 14:55:23 +00:00
Erik Johnston
380dba1020 Measure metrics of string_cache 2017-01-17 17:04:46 +00:00
Erik Johnston
37b4c7d8a9 Fix typo in return type 2017-01-17 14:43:32 +00:00
Erik Johnston
d6c75cb7c2 Rename and comment tree_to_leaves_iterator 2017-01-17 11:47:03 +00:00
Erik Johnston
1ccd5676e3 Remove needless call to evict() 2017-01-17 11:42:26 +00:00
Erik Johnston
f85b6ca494 Speed up cache size calculation 
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.

This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
 2017-01-17 11:18:13 +00:00
Erik Johnston
6d00213e80 Use OrderedDict in ExpiringCache 2017-01-16 15:33:22 +00:00
Erik Johnston
46aebbbcbf Add support for 'iterable' to ExpiringCache 2017-01-16 14:57:23 +00:00
Erik Johnston
2fae34bd2c Optionally measure size of cache by sum of length of values 2017-01-13 17:46:17 +00:00
Erik Johnston
bf5c9706d9 Remove full_twisted_stacktraces option 
The debug 'full_twisted_stacktraces' flag caused synapse to rewrite
twisted deferreds to always fire the callback on the next reactor tick.
This was to force the deferred to always store the stacktraces on
exceptions, and thus be more likely to have a full stacktrace when it
reaches the final error handlers and gets printed to the logs.

Dynamically rewriting things is generally bad, and in particular this
change violates assumptions of various bits of Twisted. This wouldn't
necessarily be so bad, but it turns out this option has been turned on
on some production servers.

Turning the option can cause e.g. #1778.

For now, lets just entirely nuke this option.
 2017-01-12 10:32:52 +00:00
Erik Johnston
f477370c0c Add paranoia exception catch in Linearizer 2017-01-10 14:04:13 +00:00
Mark Haines
dd3df11c55 More logging for the linearizer and for get_events 2017-01-05 12:32:47 +00:00
Mark Haines
62ce3034f3 s/aquire/acquire/g 2016-12-30 20:04:44 +00:00
Mark Haines
0aff09f6c9 Add more useful logging when we block fetching events 2016-12-30 20:00:44 +00:00
Erik Johnston
d53a80af25 Merge pull request #1620 from matrix-org/erikj/concurrent_room_access 
Limit the number of events that can be created on a given room concurrently
 2016-12-12 10:30:23 +00:00
Erik Johnston
fbaf868f62 Correctly handle timeout errors 2016-12-09 16:30:29 +00:00
Erik Johnston
11bfe438a2 Use correct var 2016-11-24 15:26:53 +00:00
Erik Johnston
aaecffba3a Correctly handle 500's and 429 on federation 2016-11-24 15:04:49 +00:00
Erik Johnston
90565d015e Invalidate retry cache in both directions 2016-11-22 17:45:44 +00:00
Erik Johnston
f8ee66250a Handle sending events and device messages over federation 2016-11-17 15:48:04 +00:00
Erik Johnston
d56c39cf24 Use external ldap auth pacakge 2016-11-15 13:03:19 +00:00
Kegan Dougal
3991b4cbdb Clean transactions based on time. Add HttpTransactionCache tests. 2016-11-14 11:19:24 +00:00
Erik Johnston
64038b806c Comments 2016-11-11 10:42:08 +00:00
Erik Johnston
d073cb7ead Add Limiter: limit concurrent access to resource 2016-11-10 16:29:51 +00:00
Erik Johnston
27d3f2e7ab Explicitly set authentication mode in ldap3 
This only makes a difference for versions of ldap3 before 1.0, but a)
its best to be explicit and b) there are distributions that package
ancient versions for ldap3 (e.g. debian).
 2016-11-08 14:35:25 +00:00
Erik Johnston
850b103b36 Implement pluggable password auth 
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
 2016-10-03 10:36:40 +01:00
Erik Johnston
955f34d23e Change get_pos_of_last_change to return upper bound 2016-09-15 15:12:07 +01:00
Erik Johnston
cb3edec6af Use stream_change cache to make get_forward_extremeties_for_room cache more effective 2016-09-15 14:28:13 +01:00
Erik Johnston
7356d52e73 Fix up push to use get_current_state_ids 2016-08-25 18:35:49 +01:00
Erik Johnston
9219139351 Preserve some logcontexts 2016-08-24 11:58:40 +01:00
Erik Johnston
e65bc7d315 Merge pull request #1031 from matrix-org/erikj/measure_notifier 
Add more Measure blocks
 2016-08-22 12:13:07 +01:00
Erik Johnston
8731197e54 Only abort Measure on Exceptions 2016-08-19 18:23:45 +01:00
Erik Johnston
45fd2c8942 Ensure invalidation list does not grow unboundedly 2016-08-19 16:09:16 +01:00
Erik Johnston
c0d7d9d642 Rename to on_invalidate 2016-08-19 15:13:58 +01:00
Erik Johnston
dc76a3e909 Make cache_context an explicit option 2016-08-19 15:02:38 +01:00
Erik Johnston
ba214a5e32 Remove lru option 2016-08-19 14:17:11 +01:00
Erik Johnston
4161ff2fc4 Add concept of cache contexts 2016-08-19 14:17:07 +01:00
Erik Johnston
ca8abfbf30 Clean up TransactionQueue 2016-08-10 16:24:16 +01:00
Erik Johnston
3bc9629be5 Measure federation send transaction resources 2016-08-10 10:56:38 +01:00
Erik Johnston
24f36469bc Add federation /version API 2016-08-05 16:36:07 +01:00
Erik Johnston
7b0f6293f2 Merge pull request #940 from matrix-org/erikj/fed_state_cache 
Cache federation state responses
 2016-08-02 15:21:37 +01:00
Mark Haines
bf81e38d36 Fix retry utils to check if the exception is a subclass of CME 2016-07-28 10:47:02 +01:00
David Baker
389c890f14 Don't include name of room for invites in push 
Avoids insane pushes like, "Bob invited you to invite from Bob"
 2016-07-28 10:20:47 +01:00
Matthew Hodgson
242c52d607 typo 2016-07-26 10:09:25 +02:00
Erik Johnston
248e6770ca Cache federation state responses 2016-07-21 10:30:12 +01:00
Erik Johnston
7335f0adda Add ReadWriteLock 2016-07-05 15:23:17 +01:00
David Baker
870c45913e Use similar naming we use in email notifs for push 
Fixes https://github.com/vector-im/vector-web/issues/1654
 2016-06-24 11:41:11 +01:00
Mark Haines
9f1800fba8 Remove registered_users from the distributor. 
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.

This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
 2016-06-17 19:14:16 +01:00
Erik Johnston
3b096c5f5c Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop 2016-06-03 12:00:33 +01:00
Erik Johnston
58a224a651 Pull out update_results_dict 2016-06-03 11:47:07 +01:00
Erik Johnston
73c7112433 Change CacheMetrics to be quicker 
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
 2016-06-03 11:26:52 +01:00
Erik Johnston
e043ede4a2 Small optimisation to CacheListDescriptor 2016-06-03 11:19:22 +01:00
Erik Johnston
597013caa5 Make cachedList go a bit faster 2016-06-03 11:13:29 +01:00
Matthew Hodgson
680f1d9387 catch thinko in presentable names 2016-05-23 22:55:11 +01:00
David Baker
c7c75e87dc Docstring 2016-04-29 19:47:35 +01:00
David Baker
ec9cbe847d pep8 newline 2016-04-29 10:07:30 +01:00
David Baker
acded821c4 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 10:05:20 +01:00
David Baker
ebbabc4986 Handle room invites in email notifs 2016-04-28 11:49:36 +01:00
David Baker
7b4715bad7 More variable calculation for email notifs 
Include name of the person we're sending to and add summary text at the top giving an overview of what's happened.
 2016-04-25 18:27:04 +01:00
Mark Haines
72e2fafa20 Add a metrics listener and a ssh listener to the pusher 2016-04-25 17:34:25 +01:00
Mark Haines
f22f46f4f9 Move the listenTCP call outside the manhole function 2016-04-25 14:59:21 +01:00
David Baker
290f125a13 Typo 2016-04-25 14:42:59 +01:00
David Baker
05e49ffbdf No we don't: it's just the display name 2016-04-22 18:44:17 +01:00
David Baker
c5b3c6e101 Sort member events 
So names of people in a room are given in order
 2016-04-22 18:33:36 +01:00
David Baker
e8701e64b9 Implement group-of-people names 2016-04-22 17:28:42 +01:00
Mark Haines
5905f36f05 Split out setting up the manhole to a separate file 2016-04-22 17:09:15 +01:00
Mark Haines
9e7aa98c22 Split out create_resource_tree to a separate file 2016-04-22 15:40:51 +01:00
David Baker
c10ed26c30 Flesh out email templating 
Mostly WIP porting the room name calculation logic from the web client so our room names in the email mirror the clients.
 2016-04-21 19:19:07 +01:00
Erik Johnston
eb8619e256 Create log context in Measure if one doesn't exist 2016-04-18 16:08:32 +01:00
Erik Johnston
cb9c465707 Use SynapseError 504 for Timeout errors 2016-04-15 10:21:32 +01:00
Mark Haines
82d7eea7e3 Move the versionstring code out of app.homeserver into util 2016-04-11 14:57:09 +01:00
Mark Haines
7e2f971c08 Remove some unused functions (#711) 
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
 2016-04-08 14:01:56 +01:00
Mark Haines
b9ee5650b0 Move all the wrapper functions for distributor.fire 
Move the functions inside the distributor and import them
where needed. This reduces duplication and makes it possible
for flake8 to detect when the functions aren't used in a
given file.
 2016-04-08 11:01:38 +01:00
Erik Johnston
95ac3078da Rename things 2016-04-07 16:07:16 +01:00
Erik Johnston
ee5aef6c72 Log contexts and squash things together 2016-04-07 15:34:21 +01:00
Erik Johnston
639cd07d6d Add comment 2016-04-07 14:24:12 +01:00
Erik Johnston
af03ecf352 Deduplicate joins 2016-04-07 14:19:02 +01:00
Mark Haines
87f2dec8d4 Make the cache objects be per instance rather than being global 2016-04-06 13:08:05 +01:00
Erik Johnston
3f4eb4c924 Comment 2016-04-01 14:15:27 +01:00
Erik Johnston
8d73cd502b Add concurrently_execute function 2016-04-01 14:06:00 +01:00
Mark Haines
77cba688ed Fix typo 2016-03-24 18:02:37 +00:00
Mark Haines
191c7bef6b Deduplicate identical /sync requests 2016-03-24 17:47:31 +00:00
Erik Johnston
8122ad7bab Simplify intern_dict 2016-03-23 16:41:54 +00:00
Erik Johnston
acdfef7b14 Intern all the things 2016-03-23 16:25:54 +00:00
Erik Johnston
75daede92f String intern 2016-03-23 14:53:53 +00:00
Erik Johnston
c4a8cbd15a Make LruCache use a dedicated _Node class 2016-03-22 16:06:21 +00:00
Erik Johnston
a547e2df85 Return list, not generator. 2016-03-14 15:30:19 +00:00
Mark Haines
239badea9b Use syntax that works on both py2.7 and py3 2016-03-07 20:13:10 +00:00
Erik Johnston
374f9b2f07 Limit stream change cache size too 2016-03-01 13:30:15 +00:00
Erik Johnston
ce2cdced61 Move cache size fiddling to descriptors only. Fix tests 2016-03-01 13:21:46 +00:00
Erik Johnston
910fc0f28f Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1 2016-03-01 12:56:39 +00:00
Erik Johnston
72165e5b77 Reraise exception 2016-03-01 11:00:10 +00:00
Erik Johnston
ff2d7551c7 Correct cache miss detection 2016-03-01 10:59:17 +00:00
Erik Johnston
278d6c0527 Report size of ExpiringCache 2016-02-23 16:46:21 +00:00
Erik Johnston
c77dae7a1a Change the way we figure out presence updates for small deltas 2016-02-23 14:54:40 +00:00
Erik Johnston
5614b4dafb Add presence metrics 2016-02-19 09:50:54 +00:00
Erik Johnston
b8cdec92c7 WheelTimer: Don't scan list, use index. 2016-02-18 16:33:07 +00:00
Erik Johnston
e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Erik Johnston
9e7900da1e Add wheeltimer impl 2016-02-17 14:29:28 +00:00
Erik Johnston
e557dc80b8 Merge pull request #566 from matrix-org/erikj/logcontext 
Don't bother copying records on parent context
 2016-02-10 11:41:45 +00:00
Erik Johnston
4eb8f9ca8a Remove comment 2016-02-10 11:29:21 +00:00
Erik Johnston
00c9ad49df s/parent_context/previous_context/ 2016-02-10 11:25:19 +00:00
Erik Johnston
9777c5f49a Set parent context on instansiation 2016-02-10 11:23:32 +00:00
Erik Johnston
82631c5f94 Fix unit tests 2016-02-09 13:50:37 +00:00
Erik Johnston
9daa4e2a85 Don't create new logging context 2016-02-09 11:06:19 +00:00
Erik Johnston
3e2fcd67b2 Don't bother copying records on parent context 2016-02-09 10:50:31 +00:00
Erik Johnston
241b71852e Fix bug in util.metrics.Measure 2016-02-09 10:28:13 +00:00
Erik Johnston
97294ef2fd Create new context when measuring 2016-02-09 10:12:00 +00:00
Erik Johnston
c486b7b41c Change logcontext warns to debug 2016-02-09 09:20:38 +00:00
Erik Johnston
2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Erik Johnston
13e6262659 Add metrics to pushers 2016-02-08 14:26:45 +00:00
Erik Johnston
d4f72a5bfb Allowing tagging log contexts 2016-02-03 13:52:27 +00:00
Daniel Wagner-Hall
d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston
e70165039c If stream pos is greater then earliest known key and entity hasn't changed, then entity hasn't changed 2016-01-29 16:41:32 +00:00
Erik Johnston
18579534ea Prefill stream change caches 2016-01-29 14:37:59 +00:00
Erik Johnston
b18114e19e Merge pull request #536 from matrix-org/erikj/sync 
Make /sync "better".
 2016-01-29 13:04:51 +00:00
Erik Johnston
fb7299800f Directly set self.value 2016-01-29 11:29:14 +00:00
Erik Johnston
c046630c33 Remove spurious self.size 2016-01-29 11:17:54 +00:00
Erik Johnston
a30364c1f9 Correctly bookkeep the size of TreeCache 2016-01-29 10:44:46 +00:00
Erik Johnston
766526e114 Make TreeCache keep track of its own size. 2016-01-29 10:11:21 +00:00
Erik Johnston
50e18938a9 Reset size on clear 2016-01-29 10:00:45 +00:00
Erik Johnston
3f5dd18bd4 If the same as the earliest key, assume nothing has changed. 2016-01-28 18:11:41 +00:00
Erik Johnston
40431251cb Correctly update _entity_to_key 2016-01-28 18:05:43 +00:00
Erik Johnston
82cf3a8043 Fix inequalities 2016-01-28 17:44:04 +00:00
Erik Johnston
0663c5bd52 Include cache hits with has_entity_changed 2016-01-28 17:27:28 +00:00
Erik Johnston
45cf827c8f Change name and doc has_entity_changed 2016-01-28 16:39:18 +00:00
Erik Johnston
00cb3eb24b Cache tags and account data 2016-01-28 16:37:41 +00:00