721 Commits

Author	SHA1	Message	Date
Erik Johnston	042eedfa2b	Add hacky cache factor override system	2018-06-04 15:39:28 +01:00
Amber Brown	c936a52a9e	Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy (#3307)	2018-05-31 19:03:47 +10:00
Amber Brown	debff7ae09	Merge pull request #3281 from NotAFile/py3-six-isinstance ... remaining isintance fixes	2018-05-30 12:44:46 +10:00
Adrian Tschira	7873cde526	pep8	2018-05-29 17:35:55 +02:00
Amber Brown	57ad76fa4a	fix up tests	2018-05-28 19:51:53 +10:00
Amber Brown	3ef5cd74a6	update to more consistently use seconds in any metrics or logging	2018-05-28 19:39:27 +10:00
Amber Brown	357c74a50f	add comment about why unreg	2018-05-28 19:14:41 +10:00
Amber Brown	754826a830	Merge remote-tracking branch 'origin/develop' into 3218-official-prom	2018-05-28 18:57:23 +10:00
Adrian Tschira	4ee4450d66	fix recursion error	2018-05-24 21:44:10 +02:00
Adrian Tschira	dd068ca979	remaining isintance fixes ... Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-05-24 20:55:08 +02:00
Amber Brown	36501068d8	Merge pull request #3247 from NotAFile/py3-misc ... Misc Python3 fixes	2018-05-24 12:58:37 -05:00
Amber Brown	2aff6eab6d	Merge pull request #3245 from NotAFile/batch-iter ... Add batch_iter to utils	2018-05-24 12:54:12 -05:00
Amber Brown	53cc2cde1f	cleanup	2018-05-22 17:32:57 -05:00
Amber Brown	071206304d	cleanup pep8 errors	2018-05-22 16:54:22 -05:00
Amber Brown	85ba83eb51	fixes	2018-05-22 16:28:23 -05:00
Amber Brown	a8990fa2ec	Merge remote-tracking branch 'origin/develop' into 3218-official-prom	2018-05-22 10:50:26 -05:00
Erik Johnston	7948ecf234	Comment	2018-05-22 11:39:43 +01:00
Erik Johnston	020377a550	Fix logcontext resource usage tracking	2018-05-22 11:16:07 +01:00
Amber Brown	df9f72d9e5	replacing portions	2018-05-21 19:47:37 -05:00
Adrian Tschira	45b55e23d3	Add batch_iter to utils ... There's a frequent idiom I noticed where an iterable is split up into a number of chunks/batches. Unfortunately that method does not work with iterators like dict.keys() in python3. This implementation works with iterators. Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-05-19 17:48:30 +02:00
Adrian Tschira	73cbdef5f7	fix py3 intern and remove unnecessary py3 encode ... Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-05-19 17:35:31 +02:00
Richard van der Hoff	093d8c415a	Merge remote-tracking branch 'origin/develop' into rav/warn_on_logcontext_fail	2018-05-03 14:59:29 +01:00
Richard van der Hoff	a7fe62f0cb	Fix logcontext leaks in rate limiter	2018-05-03 12:31:59 +01:00
Richard van der Hoff	415c6b672e	Merge branch 'develop' into rav/more_logcontext_leaks	2018-05-02 16:16:01 +01:00
Richard van der Hoff	f22e7cda2c	Fix a class of logcontext leaks ... So, it turns out that if you have a first `Deferred` `D1`, you can add a callback which returns another `Deferred` `D2`, and `D2` must then complete before any further callbacks on `D1` will execute (and later callbacks on `D1` get the *result* of `D2` rather than `D2` itself). So, `D1` might have `called=True` (as in, it has started running its callbacks), but any new callbacks added to `D1` won't get run until `D2` completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield` will 'block'. In conclusion: some of our assumptions in `logcontext` were invalid. We need to make sure that we don't optimise out the logcontext juggling when this situation happens. Fortunately, it is easy to detect by checking `D1.paused`.	2018-05-02 11:58:00 +01:00
Richard van der Hoff	e482f8cd85	Fix incorrect reference to StringIO ... This was introduced in 4f2f5171	2018-05-02 09:12:26 +01:00
Richard van der Hoff	fdb6849b81	Merge pull request #3144 from matrix-org/rav/run_in_background_exception_handling ... Trap exceptions thrown within run_in_background	2018-04-30 10:23:02 +01:00
Richard van der Hoff	db75c86e84	Merge branch 'develop' into py3-xrange-1	2018-04-30 01:02:25 +01:00
Richard van der Hoff	049b0b5af2	Merge pull request #3154 from NotAFile/py3-stringio ... Replace stringIO imports with six	2018-04-30 00:59:04 +01:00
Richard van der Hoff	dbf6f28d64	Merge pull request #3155 from NotAFile/py3-bytes-1 ... more bytes strings	2018-04-30 00:38:21 +01:00
Richard van der Hoff	aab2e4da60	Merge pull request #3140 from matrix-org/rav/use_run_in_background ... Use run_in_background in preference to preserve_fn	2018-04-30 00:34:28 +01:00
Adrian Tschira	e9143b6593	more bytes strings ... Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-29 00:13:57 +02:00
Adrian Tschira	d82b6ea9e6	Move more xrange to six ... plus a bonus next() Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-28 13:57:00 +02:00
Adrian Tschira	4f2f5171b7	replace stringIO imports	2018-04-28 13:46:23 +02:00
Richard van der Hoff	fc149b4eeb	Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background	2018-04-27 14:31:23 +01:00
Richard van der Hoff	6146332387	Merge remote-tracking branch 'origin/develop' into rav/deferred_timeout	2018-04-27 14:18:00 +01:00
Richard van der Hoff	2a13af23bc	Use run_in_background in preference to preserve_fn ... While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.	2018-04-27 12:55:51 +01:00
Richard van der Hoff	9d2c1b8429	Backport deferred.addTimeout ... Twisted 16.0 doesn't have addTimeout, so let's backport it.	2018-04-27 12:52:30 +01:00
Richard van der Hoff	13843f771e	Trap exceptions thrown within run_in_background ... Turn any exceptions that get thrown synchronously within run_in_background into Failures instead.	2018-04-27 12:17:13 +01:00
Richard van der Hoff	9255a6cb17	Improve exception handling for background processes ... There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background.	2018-04-27 11:07:40 +01:00
Richard van der Hoff	1ea904b9f0	Use deferred.addTimeout instead of time_bound_deferred ... This doesn't feel like a wheel we need to reinvent.	2018-04-23 00:53:18 +01:00
Richard van der Hoff	8dc4a6144b	Merge pull request #3107 from NotAFile/py3-bool-nonzero ... add __bool__ alias to __nonzero__ methods	2018-04-20 15:43:39 +01:00
Richard van der Hoff	c09a6daf09	Merge pull request #3110 from NotAFile/py3-six-queue ... Replace Queue with six.moves.queue	2018-04-20 15:35:00 +01:00
Richard van der Hoff	11a67b7c9d	Merge pull request #3093 from matrix-org/rav/response_cache_wrap ... Refactor ResponseCache usage	2018-04-20 11:31:17 +01:00
Adrian Tschira	878995e660	Replace Queue with six.moves.queue ... and a six.range change which I missed the last time Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-16 00:46:21 +02:00
Adrian Tschira	f63ff73c7f	add __bool__ alias to __nonzero__ methods ... Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-15 20:40:47 +02:00
Richard van der Hoff	d3347ad485	Revert "Use sortedcontainers instead of blist" ... This reverts commit 9fbe70a7dc. It turns out that sortedcontainers.SortedDict is not an exact match for blist.sorteddict; in particular, `popitem()` removes things from the opposite end of the dict. This is trivial to fix, but I want to add some unit tests, and potentially some more thought about it, before we do so.	2018-04-13 11:16:43 +01:00
Richard van der Hoff	60f6014bb7	ResponseCache: fix handling of completed results ... Turns out that ObservableDeferred.observe doesn't return a deferred if the result is already completed. Fix handling and improve documentation.	2018-04-13 07:32:29 +01:00
Richard van der Hoff	b78395b7fe	Refactor ResponseCache usage ... Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a (get, set) pair, and then use it throughout the codebase. This will be largely non-functional, but does include the following functional changes: * federation_server.on_context_state_request: drops use of _server_linearizer which looked redundant and could cause incorrect cache misses by yielding between the get and the set. * RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks * the wrap function includes some logging. I'm hoping this won't be too noisy on production.	2018-04-12 13:02:15 +01:00
Richard van der Hoff	d5c74b9f6c	Merge pull request #3092 from matrix-org/rav/response_cache_metrics ... Add metrics for ResponseCache	2018-04-12 12:59:36 +01:00
Richard van der Hoff	261124396e	Merge pull request #3059 from matrix-org/rav/doc_response_cache ... Document the behaviour of ResponseCache	2018-04-12 11:22:30 +01:00
Richard van der Hoff	b3384232a0	Add metrics for ResponseCache	2018-04-10 23:14:47 +01:00
Vincent Breitmoser	9fbe70a7dc	Use sortedcontainers instead of blist ... This commit drop-in replaces blist with SortedContainers. They are written in pure python so work with pypy, but perform as good as native implementations, at least in a couple benchmarks: http://www.grantjenks.com/docs/sortedcontainers/performance.html	2018-04-10 11:29:51 +02:00
Richard van der Hoff	13decdbf96	Revert "Merge pull request #3066 from matrix-org/rav/remove_redundant_metrics" ... We aren't ready to release this yet, so I'm reverting it for now. This reverts commit d1679a4ed7, reversing changes made to e089100c62.	2018-04-09 12:59:12 +01:00
Richard van der Hoff	3449da3bc7	Merge pull request #3068 from matrix-org/rav/fix_cache_invalidation ... Improve database cache performance	2018-04-05 17:21:44 +01:00
Richard van der Hoff	01afc563c3	Fix overzealous cache invalidation ... Fixes an issue where a cache invalidation would invalidate *all* pending entries, rather than just the entry that we intended to invalidate.	2018-04-05 16:24:04 +01:00
Richard van der Hoff	518f6de088	Remove redundant metrics which were deprecated in 0.27.0.	2018-04-04 19:46:28 +01:00
Richard van der Hoff	a9a74101a4	Document the behaviour of ResponseCache ... it looks like everything that uses ResponseCache expects to have to `make_deferred_yieldable` its results. It's debatable whether that is the best approach, but let's document it for now to avoid further confusion.	2018-04-04 09:06:22 +01:00
Richard van der Hoff	05630758f2	Use static JSONEncoders ... using json.dumps with custom options requires us to create a new JSONEncoder on each call. It's more efficient to create one upfront and reuse it.	2018-03-29 23:13:33 +01:00
Matthew Hodgson	8cbbfaefc1	404 correctly on missing paths via NoResource ... fixes https://github.com/matrix-org/synapse/issues/2043 and https://github.com/matrix-org/synapse/issues/2029	2018-03-23 10:32:50 +00:00
Erik Johnston	9a0d783c11	Add comments	2018-03-19 11:35:53 +00:00
Richard van der Hoff	5a6e54264d	Make 'unexpected logging context' into warnings ... I think we've now fixed enough of these that the rest can be logged at warning.	2018-03-15 18:40:38 +00:00
Erik Johnston	7c7706f42b	Fix bug where state cache used lots of memory ... The state cache bases its size on the sum of the size of entries. The size of the entry is calculated once on insertion, so it is important that the size of entries does not change. The DictionaryCache modified the entries size, which caused the state cache to incorrectly think it was smaller than it actually was.	2018-03-15 15:46:54 +00:00
Richard van der Hoff	20f40348d4	Factor run_in_background out from preserve_fn ... It annoys me that we create temporary function objects when there's really no need for it. Let's factor the gubbins out of preserve_fn and start using it.	2018-03-08 11:50:11 +00:00
Richard van der Hoff	3a75de923b	Rewrite make_deferred_yieldable avoiding inlineCallbacks ... ... because (a) it's actually simpler (b) it might be marginally more performant?	2018-03-01 12:40:05 +00:00
Richard van der Hoff	bc496df192	report metrics on number of cache evictions	2018-02-05 15:34:01 +00:00
Matthew Hodgson	ab9f844aaf	Add federation_domain_whitelist option (#2820) ... Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network	2018-01-22 19:11:18 +01:00
Matthew Hodgson	d84f65255e	Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid ... add registrations_require_3pid and allow_local_3pids	2018-01-22 13:57:22 +00:00
Matthew Hodgson	8fe253f19b	fix PR nitpicking	2018-01-19 18:23:45 +00:00
Matthew Hodgson	447f4f0d5f	rewrite based on PR feedback: ... * [ ] split config options into allowed_local_3pids and registrations_require_3pid * [ ] simplify and comment logic for picking registration flows * [ ] fix docstring and move check_3pid_allowed into a new util module * [ ] use check_3pid_allowed everywhere @erikjohnston PTAL	2018-01-19 15:33:55 +00:00
Erik Johnston	b6dc7044a9	Merge pull request #2804 from matrix-org/erikj/file_consumer ... Add decent impl of a FileConsumer	2018-01-18 16:31:33 +00:00
Richard van der Hoff	d57765fc8a	Fix bugs in block metrics ... ... which I introduced in #2785	2018-01-18 12:24:42 +00:00
Erik Johnston	be0dfcd4a2	Do logcontexts correctly	2018-01-18 11:57:57 +00:00
Erik Johnston	1432f7ccd5	Move test stuff to tests	2018-01-18 11:57:57 +00:00
Erik Johnston	2f18a2647b	Make all fields private	2018-01-18 11:57:54 +00:00
Erik Johnston	dc519602ac	Ensure we registerProducer isn't called twice	2018-01-18 11:07:17 +00:00
Erik Johnston	17b54389fe	Fix _notify_empty typo	2018-01-18 11:05:34 +00:00
Erik Johnston	28b338ed9b	Move definition of paused_producer to __init__	2018-01-18 11:04:41 +00:00
Erik Johnston	a177325b49	Fix comments	2018-01-18 11:02:43 +00:00
Erik Johnston	bc67e7d260	Add decent impl of a FileConsumer ... Twisted core doesn't have a general purpose one, so we need to write one ourselves. Features: - All writing happens in background thread - Supports both push and pull producers - Push producers get paused if the consumer falls behind	2018-01-17 16:43:03 +00:00
Richard van der Hoff	3d12d97415	Track DB scheduling delay per-request ... For each request, track the amount of time spent waiting for a db connection. This entails adding it to the LoggingContext and we may as well add metrics for it while we are passing.	2018-01-16 17:23:32 +00:00
Richard van der Hoff	6324b65f08	Track db txn time in millisecs ... ... to reduce the amount of floating-point foo we do.	2018-01-16 15:53:18 +00:00
Richard van der Hoff	44a498418c	Optimise LoggingContext creation and copying ... It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit.	2018-01-16 15:49:42 +00:00
Richard van der Hoff	39f4e29d01	Reorganise request and block metrics ... In order to circumvent the number of duplicate foo:count metrics increasing without bounds, it's time for a rearrangement. The following are all deprecated, and replaced with synapse_util_metrics_block_count: synapse_util_metrics_block_timer:count synapse_util_metrics_block_ru_utime:count synapse_util_metrics_block_ru_stime:count synapse_util_metrics_block_db_txn_count:count synapse_util_metrics_block_db_txn_duration:count The following are all deprecated, and replaced with synapse_http_server_response_count: synapse_http_server_requests synapse_http_server_response_time:count synapse_http_server_response_ru_utime:count synapse_http_server_response_ru_stime:count synapse_http_server_response_db_txn_count:count synapse_http_server_response_db_txn_duration:count The following are renamed (the old metrics are kept for now, but deprecated): synapse_util_metrics_block_timer:total -> synapse_util_metrics_block_time_seconds synapse_util_metrics_block_ru_utime:total -> synapse_util_metrics_block_ru_utime_seconds synapse_util_metrics_block_ru_stime:total -> synapse_util_metrics_block_ru_stime_seconds synapse_util_metrics_block_db_txn_count:total -> synapse_util_metrics_block_db_txn_count synapse_util_metrics_block_db_txn_duration:total -> synapse_util_metrics_block_db_txn_duration_seconds synapse_http_server_response_time:total -> synapse_http_server_response_time_seconds synapse_http_server_response_ru_utime:total -> synapse_http_server_response_ru_utime_seconds synapse_http_server_response_ru_stime:total -> synapse_http_server_response_ru_stime_seconds synapse_http_server_response_db_txn_count:total -> synapse_http_server_response_db_txn_count synapse_http_server_response_db_txn_duration:total synapse_http_server_response_db_txn_duration_seconds	2018-01-15 17:09:44 +00:00
Richard van der Hoff	b2cd6accf5	Remove __PreservingContextDeferred too	2017-11-14 23:00:10 +00:00
Richard van der Hoff	7e6fa29cb5	Remove preserve_context_over_{fn, deferred} ... Both of these functions ae known to leak logcontexts. Replace the remaining calls to them and kill them off.	2017-11-14 11:22:42 +00:00
Richard van der Hoff	bf993db11c	Logging and logcontext fixes for Limiter ... Add some logging to the Limiter in a similar spirit to the Linearizer, to help debug issues. Also fix a logcontext leak. Also refactor slightly to avoid throwing exceptions.	2017-11-07 00:48:57 +00:00
Richard van der Hoff	0be99858f3	fix vars named l ... E741 says "do not use variables named ‘l’, ‘O’, or ‘I’".	2017-10-23 15:56:38 +01:00
Richard van der Hoff	eaaabc6c4f	replace 'except:' with 'except Exception:' ... what could possibly go wrong	2017-10-23 15:52:32 +01:00
Richard van der Hoff	2e9f5ea31a	Fix logcontext handling for persist_events ... * don't use preserve_context_over_deferred, which is known broken. * remove a redundant preserve_fn. * add/improve some comments	2017-10-17 10:59:30 +01:00
Richard van der Hoff	cc794d60e7	Merge pull request #2532 from matrix-org/rav/fix_linearizer ... Fix stackoverflow and logcontexts from linearizer	2017-10-11 17:29:32 +01:00
Richard van der Hoff	f30c4ed2bc	logformatter: fix AttributeError ... make sure we have the relevant fields before we try to log them.	2017-10-11 17:26:17 +01:00
Richard van der Hoff	4fad8efbfb	Fix stackoverflow and logcontexts from linearizer ... 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts.	2017-10-11 15:05:05 +01:00
Richard van der Hoff	3cc852d339	Fancy logformatter to format exceptions better ... This is a bit of an experimental change at this point; the idea is to see if it helps us track down where our stack overflows are coming from by logging the stack when the exception was caught and turned into a Failure. (We'll also need edf2704420). If we deploy this, we'll be able to enable it via the log config yaml.	2017-10-09 17:44:42 +01:00
Richard van der Hoff	148428ce76	Fix logcontext handling for concurrently_execute ... Avoid preserve_context_over_deferred, which is broken.	2017-10-06 22:24:28 +01:00
David Baker	8ad5f34908	pep8	2017-09-26 19:21:41 +01:00
David Baker	9fd086e506	unnecessary parens	2017-09-26 17:59:46 +01:00
David Baker	0b03a97708	Add module_loader.py	2017-09-26 17:56:41 +01:00
Erik Johnston	495f075b41	Increase default cache factor size.	2017-07-04 09:58:32 +01:00
Erik Johnston	b5e8d529e6	Define CACHE_SIZE_FACTOR once	2017-07-04 09:56:44 +01:00
Erik Johnston	c72058bcc6	Use an ExpiringCache for storing registration sessions ... This is because pruning them was a significant performance drain on matrix.org	2017-06-29 14:08:37 +01:00
Erik Johnston	efc2b7db95	Rewrite conditional	2017-06-09 13:35:15 +01:00
Erik Johnston	eed59dcc1e	Fix has_any_entity_changed ... Occaisonally has_any_entity_changed would throw the error: "Set changed size during iteration" when taking the max of the `sorteddict`. While its uncertain how that happens, its quite inefficient to iterate over the entire dict anyway so we change to using the more traditional `bisect_*` functions.	2017-06-09 11:44:01 +01:00
Erik Johnston	304880d185	Add stream change cache	2017-05-31 15:46:36 +01:00
Erik Johnston	bd7bb5df71	Pull out if statement from for loop	2017-05-22 15:12:19 +01:00
Erik Johnston	e3417a06e2	Update list cache to handle one arg case ... We update the normal cache descriptors to handle caches with a single argument specially so that the key wasn't a 1-tuple. We need to update the cache list to be aware of this.	2017-05-22 15:04:42 +01:00
Erik Johnston	bbfe4e996c	Make get_state_groups_from_groups faster. ... Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy.	2017-05-17 15:12:15 +01:00
Erik Johnston	ffad4fe35b	Don't update event cache hit ratio from get_joined_users ... Otherwise the hit ration of plain get_events gets completely skewed by calls to get_joined_users* functions.	2017-05-08 16:06:17 +01:00
Erik Johnston	d2d8ed4884	Optimise caches with single key	2017-05-04 14:18:46 +01:00
Richard van der Hoff	2e996271fe	Instantiate DeferredTimedOutError correctly ... Call `super` correctly, so that we correctly initialise the `errcode` field. Fixes https://github.com/matrix-org/synapse/issues/2179.	2017-05-02 13:26:17 +01:00
Erik Johnston	d9aa645f86	Reduce size of joined_user cache ... The _get_joined_users_from_context cache stores a mapping from user_id to avatar_url and display_name. Instead of storing those in a dict, store them in a namedtuple as that uses much less memory. We also try converting the string to ascii to further reduce the size.	2017-04-25 14:38:51 +01:00
Erik Johnston	efab1dadde	Remove DEBUG_CACHES	2017-04-25 10:54:09 +01:00
Erik Johnston	119cb9bbcf	Reduce cache size by not storing deferreds ... Currently the cache descriptors store deferreds rather than raw values, this is a simple way of triggering only one database hit and sharing the result if two callers attempt to get the same value. However, there are a few caches that simply store a mapping from string to string (or int). These caches can have a large number of entries, under the assumption that each entry is small. However, the size of a deferred (specifically the size of ObservableDeferred) is signigicantly larger than that of the raw value, 2kb vs 32b. This PR therefore changes the cache descriptors to store the raw values rather than the deferreds. As a side effect cached storage function now either return a deferred or the actual value, as the cached list decriptor already does. This is fine as we always end up just yield'ing on the returned value eventually, which handles that case correctly.	2017-04-25 10:23:11 +01:00
Erik Johnston	d134d0935e	Only intern ascii strings	2017-04-24 14:07:48 +01:00
Richard van der Hoff	e2eebf1696	Fix fixme in preserve_fn ... `preserve_fn` is no longer used as a decorator anywhere, so we can safely fix a fixme therein.	2017-04-03 15:38:02 +01:00
Erik Johnston	4d17add8de	Remove unused instance variable	2017-03-31 09:38:27 +01:00
Erik Johnston	5b5b171f3e	Docs	2017-03-30 17:05:53 +01:00
Erik Johnston	b282fe7170	Revert log context change	2017-03-30 17:03:59 +01:00
Erik Johnston	6194a64ae9	Doc new instance variables	2017-03-30 14:19:10 +01:00
Erik Johnston	014fee93b3	Manually calculate cache key as getcallargs is expensive ... This is because getcallargs recomputes the getargspec, amongst other things, which we don't need to do as its already been done	2017-03-30 14:14:46 +01:00
Erik Johnston	86780a8bc3	Don't convert to deferreds when not necessary	2017-03-30 14:14:36 +01:00
Richard van der Hoff	f9b4bb05e0	Fix the logcontext handling in the cache wrappers (#2077) ... The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.	2017-03-30 13:22:24 +01:00
Richard van der Hoff	9397edb28b	Merge pull request #2050 from matrix-org/rav/federation_backoff ... push federation retry limiter down to matrixfederationclient	2017-03-23 22:27:01 +00:00
Richard van der Hoff	06ce7335e9	Merge pull request #2052 from matrix-org/rav/time_bound_deferred ... Fix time_bound_deferred to throw the right exception	2017-03-23 22:22:54 +00:00
Richard van der Hoff	5a16cb4bf0	Ignore backoff history for invites, aliases, and roomdirs ... Add a param to the federation client which lets us ignore historical backoff data for federation queries, and set it for a handful of operations.	2017-03-23 12:23:22 +00:00
Richard van der Hoff	b88a323ffb	Fix time_bound_deferred to throw the right exception ... Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing.	2017-03-23 12:07:11 +00:00
Richard van der Hoff	4bd597d9fc	push federation retry limiter down to matrixfederationclient ... rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.	2017-03-23 09:28:46 +00:00
Richard van der Hoff	19b9366d73	Fix a couple of logcontext leaks ... Use preserve_fn to correctly manage the logcontexts around things we don't want to yield on.	2017-03-23 00:17:46 +00:00
Richard van der Hoff	95f21c7a66	Fix caching of remote servers' signature keys ... The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.	2017-03-22 15:11:30 +00:00
Richard van der Hoff	bd08ee7a46	Merge pull request #2026 from matrix-org/rav/logcontext_docs ... Logcontext docs	2017-03-20 12:05:21 +00:00
Richard van der Hoff	f40c2db05a	Stop preserve_fn leaking context into the reactor ... Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.	2017-03-18 00:07:43 +00:00
Richard van der Hoff	d2d146a314	Logcontext docs	2017-03-17 23:59:28 +00:00
Richard van der Hoff	2abe85d50e	Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join ... Queue up federation PDUs while a room join is in progress	2017-03-17 11:32:44 +00:00
Richard van der Hoff	29ed09e80a	Fix assertion to stop transaction queue getting wedged ... ... and update some docstrings to correctly reflect the types being used. get_new_device_msgs_for_remote can return a long under some circumstances, which was being stored in last_device_list_stream_id_by_dest, and was then upsetting things on the next loop.	2017-03-15 12:16:55 +00:00
Richard van der Hoff	b5d1c68beb	Implement reset_context_after_deferred ... to correctly reset the context when we fire off a deferred we aren't going to wait for.	2017-03-15 02:21:07 +00:00
David Baker	73a5f06652	Support registration / login with phone number ... Changes from https://github.com/matrix-org/synapse/pull/1971	2017-03-13 17:27:51 +00:00
Erik Johnston	7eae6eaa2f	Revert "Support registration & login with phone number"	2017-03-13 09:59:33 +00:00
Erik Johnston	3545e17f43	Add setdefault key to ExpiringCache	2017-03-10 10:30:49 +00:00
David Baker	9d0d40fc15	Docs	2017-03-08 19:05:29 +00:00
David Baker	3edc57296d	Incorrectly copied copyright ... This file post-dates OM	2017-03-08 19:00:51 +00:00
David Baker	1c99934b28	pep8	2017-03-08 11:58:20 +00:00
David Baker	a9e2b9ec16	Add msisdn util file	2017-03-08 11:53:36 +00:00
Erik Johnston	6b61060b51	Comment	2017-02-02 14:47:15 +00:00
Erik Johnston	9efcc3f3be	Comment	2017-02-02 13:50:22 +00:00
Erik Johnston	df4ecff5a9	Correctly raise exceptions for ratelimitng. Ratelimit on 401	2017-02-01 15:42:19 +00:00
Erik Johnston	fe08db2713	Remove explicit < 400 check as apparently this is confusing	2017-01-31 15:21:32 +00:00
Erik Johnston	4c0ec15bdc	Comment	2017-01-31 13:53:46 +00:00
Erik Johnston	85c590105f	Comment	2017-01-31 13:46:38 +00:00
Erik Johnston	ae7a132f38	Better handle 404 response for federation /send/	2017-01-31 13:40:09 +00:00
Erik Johnston	c430111d0e	Update LruCache size estimate on clear	2017-01-18 14:55:23 +00:00
Erik Johnston	380dba1020	Measure metrics of string_cache	2017-01-17 17:04:46 +00:00
Erik Johnston	37b4c7d8a9	Fix typo in return type	2017-01-17 14:43:32 +00:00
Erik Johnston	d6c75cb7c2	Rename and comment tree_to_leaves_iterator	2017-01-17 11:47:03 +00:00
Erik Johnston	1ccd5676e3	Remove needless call to evict()	2017-01-17 11:42:26 +00:00
Erik Johnston	f85b6ca494	Speed up cache size calculation ... Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory.	2017-01-17 11:18:13 +00:00
Erik Johnston	6d00213e80	Use OrderedDict in ExpiringCache	2017-01-16 15:33:22 +00:00
Erik Johnston	46aebbbcbf	Add support for 'iterable' to ExpiringCache	2017-01-16 14:57:23 +00:00
Erik Johnston	2fae34bd2c	Optionally measure size of cache by sum of length of values	2017-01-13 17:46:17 +00:00
Erik Johnston	bf5c9706d9	Remove full_twisted_stacktraces option ... The debug 'full_twisted_stacktraces' flag caused synapse to rewrite twisted deferreds to always fire the callback on the next reactor tick. This was to force the deferred to always store the stacktraces on exceptions, and thus be more likely to have a full stacktrace when it reaches the final error handlers and gets printed to the logs. Dynamically rewriting things is generally bad, and in particular this change violates assumptions of various bits of Twisted. This wouldn't necessarily be so bad, but it turns out this option has been turned on on some production servers. Turning the option can cause e.g. #1778. For now, lets just entirely nuke this option.	2017-01-12 10:32:52 +00:00
Erik Johnston	f477370c0c	Add paranoia exception catch in Linearizer	2017-01-10 14:04:13 +00:00
Mark Haines	dd3df11c55	More logging for the linearizer and for get_events	2017-01-05 12:32:47 +00:00
Mark Haines	62ce3034f3	s/aquire/acquire/g	2016-12-30 20:04:44 +00:00
Mark Haines	0aff09f6c9	Add more useful logging when we block fetching events	2016-12-30 20:00:44 +00:00
Erik Johnston	d53a80af25	Merge pull request #1620 from matrix-org/erikj/concurrent_room_access ... Limit the number of events that can be created on a given room concurrently	2016-12-12 10:30:23 +00:00
Erik Johnston	fbaf868f62	Correctly handle timeout errors	2016-12-09 16:30:29 +00:00
Erik Johnston	11bfe438a2	Use correct var	2016-11-24 15:26:53 +00:00
Erik Johnston	aaecffba3a	Correctly handle 500's and 429 on federation	2016-11-24 15:04:49 +00:00
Erik Johnston	90565d015e	Invalidate retry cache in both directions	2016-11-22 17:45:44 +00:00
Erik Johnston	f8ee66250a	Handle sending events and device messages over federation	2016-11-17 15:48:04 +00:00
Erik Johnston	d56c39cf24	Use external ldap auth pacakge	2016-11-15 13:03:19 +00:00
Kegan Dougal	3991b4cbdb	Clean transactions based on time. Add HttpTransactionCache tests.	2016-11-14 11:19:24 +00:00
Erik Johnston	64038b806c	Comments	2016-11-11 10:42:08 +00:00
Erik Johnston	d073cb7ead	Add Limiter: limit concurrent access to resource	2016-11-10 16:29:51 +00:00
Erik Johnston	27d3f2e7ab	Explicitly set authentication mode in ldap3 ... This only makes a difference for versions of ldap3 before 1.0, but a) its best to be explicit and b) there are distributions that package ancient versions for ldap3 (e.g. debian).	2016-11-08 14:35:25 +00:00
Erik Johnston	850b103b36	Implement pluggable password auth ... Allows delegating the password auth to an external module. This also moves the LDAP auth to using this system, allowing it to be removed from the synapse tree entirely in the future.	2016-10-03 10:36:40 +01:00
Erik Johnston	955f34d23e	Change get_pos_of_last_change to return upper bound	2016-09-15 15:12:07 +01:00
Erik Johnston	cb3edec6af	Use stream_change cache to make get_forward_extremeties_for_room cache more effective	2016-09-15 14:28:13 +01:00
Erik Johnston	7356d52e73	Fix up push to use get_current_state_ids	2016-08-25 18:35:49 +01:00
Erik Johnston	9219139351	Preserve some logcontexts	2016-08-24 11:58:40 +01:00
Erik Johnston	e65bc7d315	Merge pull request #1031 from matrix-org/erikj/measure_notifier ... Add more Measure blocks	2016-08-22 12:13:07 +01:00
Erik Johnston	8731197e54	Only abort Measure on Exceptions	2016-08-19 18:23:45 +01:00
Erik Johnston	45fd2c8942	Ensure invalidation list does not grow unboundedly	2016-08-19 16:09:16 +01:00
Erik Johnston	c0d7d9d642	Rename to on_invalidate	2016-08-19 15:13:58 +01:00
Erik Johnston	dc76a3e909	Make cache_context an explicit option	2016-08-19 15:02:38 +01:00
Erik Johnston	ba214a5e32	Remove lru option	2016-08-19 14:17:11 +01:00
Erik Johnston	4161ff2fc4	Add concept of cache contexts	2016-08-19 14:17:07 +01:00
Erik Johnston	ca8abfbf30	Clean up TransactionQueue	2016-08-10 16:24:16 +01:00
Erik Johnston	3bc9629be5	Measure federation send transaction resources	2016-08-10 10:56:38 +01:00
Erik Johnston	24f36469bc	Add federation /version API	2016-08-05 16:36:07 +01:00
Erik Johnston	7b0f6293f2	Merge pull request #940 from matrix-org/erikj/fed_state_cache ... Cache federation state responses	2016-08-02 15:21:37 +01:00
Mark Haines	bf81e38d36	Fix retry utils to check if the exception is a subclass of CME	2016-07-28 10:47:02 +01:00
David Baker	389c890f14	Don't include name of room for invites in push ... Avoids insane pushes like, "Bob invited you to invite from Bob"	2016-07-28 10:20:47 +01:00
Matthew Hodgson	242c52d607	typo	2016-07-26 10:09:25 +02:00
Erik Johnston	248e6770ca	Cache federation state responses	2016-07-21 10:30:12 +01:00
Erik Johnston	7335f0adda	Add ReadWriteLock	2016-07-05 15:23:17 +01:00
David Baker	870c45913e	Use similar naming we use in email notifs for push ... Fixes https://github.com/vector-im/vector-web/issues/1654	2016-06-24 11:41:11 +01:00
Mark Haines	9f1800fba8	Remove registered_users from the distributor. ... The only place that was observed was to set the profile. I've made it so that the profile is set within store.register in the same transaction that creates the user. This required some slight changes to the registration code for upgrading guest users, since it previously relied on the distributor swallowing errors if the profile already existed.	2016-06-17 19:14:16 +01:00
Erik Johnston	3b096c5f5c	Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop	2016-06-03 12:00:33 +01:00
Erik Johnston	58a224a651	Pull out update_results_dict	2016-06-03 11:47:07 +01:00
Erik Johnston	73c7112433	Change CacheMetrics to be quicker ... We change it so that each cache has an individual CacheMetric, instead of having one global CacheMetric. This means that when a cache tries to increment a counter it does not need to go through so many indirections.	2016-06-03 11:26:52 +01:00
Erik Johnston	e043ede4a2	Small optimisation to CacheListDescriptor	2016-06-03 11:19:22 +01:00
Erik Johnston	597013caa5	Make cachedList go a bit faster	2016-06-03 11:13:29 +01:00
Matthew Hodgson	680f1d9387	catch thinko in presentable names	2016-05-23 22:55:11 +01:00
David Baker	c7c75e87dc	Docstring	2016-04-29 19:47:35 +01:00
David Baker	ec9cbe847d	pep8 newline	2016-04-29 10:07:30 +01:00
David Baker	acded821c4	Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs	2016-04-29 10:05:20 +01:00
David Baker	ebbabc4986	Handle room invites in email notifs	2016-04-28 11:49:36 +01:00
David Baker	7b4715bad7	More variable calculation for email notifs ... Include name of the person we're sending to and add summary text at the top giving an overview of what's happened.	2016-04-25 18:27:04 +01:00
Mark Haines	72e2fafa20	Add a metrics listener and a ssh listener to the pusher	2016-04-25 17:34:25 +01:00
Mark Haines	f22f46f4f9	Move the listenTCP call outside the manhole function	2016-04-25 14:59:21 +01:00
David Baker	290f125a13	Typo	2016-04-25 14:42:59 +01:00
David Baker	05e49ffbdf	No we don't: it's just the display name	2016-04-22 18:44:17 +01:00
David Baker	c5b3c6e101	Sort member events ... So names of people in a room are given in order	2016-04-22 18:33:36 +01:00
David Baker	e8701e64b9	Implement group-of-people names	2016-04-22 17:28:42 +01:00
Mark Haines	5905f36f05	Split out setting up the manhole to a separate file	2016-04-22 17:09:15 +01:00
Mark Haines	9e7aa98c22	Split out create_resource_tree to a separate file	2016-04-22 15:40:51 +01:00
David Baker	c10ed26c30	Flesh out email templating ... Mostly WIP porting the room name calculation logic from the web client so our room names in the email mirror the clients.	2016-04-21 19:19:07 +01:00
Erik Johnston	eb8619e256	Create log context in Measure if one doesn't exist	2016-04-18 16:08:32 +01:00
Erik Johnston	cb9c465707	Use SynapseError 504 for Timeout errors	2016-04-15 10:21:32 +01:00
Mark Haines	82d7eea7e3	Move the versionstring code out of app.homeserver into util	2016-04-11 14:57:09 +01:00
Mark Haines	7e2f971c08	Remove some unused functions (#711) ... * Remove some unused functions * get_room_events_stream is only used in tests * is_exclusive_room might actually be something we want	2016-04-08 14:01:56 +01:00
Mark Haines	b9ee5650b0	Move all the wrapper functions for distributor.fire ... Move the functions inside the distributor and import them where needed. This reduces duplication and makes it possible for flake8 to detect when the functions aren't used in a given file.	2016-04-08 11:01:38 +01:00
Erik Johnston	95ac3078da	Rename things	2016-04-07 16:07:16 +01:00
Erik Johnston	ee5aef6c72	Log contexts and squash things together	2016-04-07 15:34:21 +01:00
Erik Johnston	639cd07d6d	Add comment	2016-04-07 14:24:12 +01:00
Erik Johnston	af03ecf352	Deduplicate joins	2016-04-07 14:19:02 +01:00
Mark Haines	87f2dec8d4	Make the cache objects be per instance rather than being global	2016-04-06 13:08:05 +01:00
Erik Johnston	3f4eb4c924	Comment	2016-04-01 14:15:27 +01:00
Erik Johnston	8d73cd502b	Add concurrently_execute function	2016-04-01 14:06:00 +01:00
Mark Haines	77cba688ed	Fix typo	2016-03-24 18:02:37 +00:00
Mark Haines	191c7bef6b	Deduplicate identical /sync requests	2016-03-24 17:47:31 +00:00
Erik Johnston	8122ad7bab	Simplify intern_dict	2016-03-23 16:41:54 +00:00
Erik Johnston	acdfef7b14	Intern all the things	2016-03-23 16:25:54 +00:00
Erik Johnston	75daede92f	String intern	2016-03-23 14:53:53 +00:00
Erik Johnston	c4a8cbd15a	Make LruCache use a dedicated _Node class	2016-03-22 16:06:21 +00:00
Erik Johnston	a547e2df85	Return list, not generator.	2016-03-14 15:30:19 +00:00
Mark Haines	239badea9b	Use syntax that works on both py2.7 and py3	2016-03-07 20:13:10 +00:00
Erik Johnston	374f9b2f07	Limit stream change cache size too	2016-03-01 13:30:15 +00:00
Erik Johnston	ce2cdced61	Move cache size fiddling to descriptors only. Fix tests	2016-03-01 13:21:46 +00:00
Erik Johnston	910fc0f28f	Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1	2016-03-01 12:56:39 +00:00
Erik Johnston	72165e5b77	Reraise exception	2016-03-01 11:00:10 +00:00
Erik Johnston	ff2d7551c7	Correct cache miss detection	2016-03-01 10:59:17 +00:00
Erik Johnston	278d6c0527	Report size of ExpiringCache	2016-02-23 16:46:21 +00:00
Erik Johnston	c77dae7a1a	Change the way we figure out presence updates for small deltas	2016-02-23 14:54:40 +00:00
Erik Johnston	5614b4dafb	Add presence metrics	2016-02-19 09:50:54 +00:00
Erik Johnston	b8cdec92c7	WheelTimer: Don't scan list, use index.	2016-02-18 16:33:07 +00:00
Erik Johnston	e5999bfb1a	Initial cut	2016-02-17 15:40:50 +00:00
Erik Johnston	9e7900da1e	Add wheeltimer impl	2016-02-17 14:29:28 +00:00
Erik Johnston	e557dc80b8	Merge pull request #566 from matrix-org/erikj/logcontext ... Don't bother copying records on parent context	2016-02-10 11:41:45 +00:00
Erik Johnston	4eb8f9ca8a	Remove comment	2016-02-10 11:29:21 +00:00
Erik Johnston	00c9ad49df	s/parent_context/previous_context/	2016-02-10 11:25:19 +00:00
Erik Johnston	9777c5f49a	Set parent context on instansiation	2016-02-10 11:23:32 +00:00
Erik Johnston	82631c5f94	Fix unit tests	2016-02-09 13:50:37 +00:00
Erik Johnston	9daa4e2a85	Don't create new logging context	2016-02-09 11:06:19 +00:00
Erik Johnston	3e2fcd67b2	Don't bother copying records on parent context	2016-02-09 10:50:31 +00:00
Erik Johnston	241b71852e	Fix bug in util.metrics.Measure	2016-02-09 10:28:13 +00:00
Erik Johnston	97294ef2fd	Create new context when measuring	2016-02-09 10:12:00 +00:00
Erik Johnston	c486b7b41c	Change logcontext warns to debug	2016-02-09 09:20:38 +00:00
Erik Johnston	2c1fbea531	Fix up logcontexts	2016-02-08 14:26:45 +00:00
Erik Johnston	13e6262659	Add metrics to pushers	2016-02-08 14:26:45 +00:00
Erik Johnston	d4f72a5bfb	Allowing tagging log contexts	2016-02-03 13:52:27 +00:00
Daniel Wagner-Hall	d83d004ccd	Fix flake8 warnings for new flake8	2016-02-02 17:18:50 +00:00
Erik Johnston	e70165039c	If stream pos is greater then earliest known key and entity hasn't changed, then entity hasn't changed	2016-01-29 16:41:32 +00:00
Erik Johnston	18579534ea	Prefill stream change caches	2016-01-29 14:37:59 +00:00
Erik Johnston	b18114e19e	Merge pull request #536 from matrix-org/erikj/sync ... Make /sync "better".	2016-01-29 13:04:51 +00:00
Erik Johnston	fb7299800f	Directly set self.value	2016-01-29 11:29:14 +00:00
Erik Johnston	c046630c33	Remove spurious self.size	2016-01-29 11:17:54 +00:00
Erik Johnston	a30364c1f9	Correctly bookkeep the size of TreeCache	2016-01-29 10:44:46 +00:00
Erik Johnston	766526e114	Make TreeCache keep track of its own size.	2016-01-29 10:11:21 +00:00
Erik Johnston	50e18938a9	Reset size on clear	2016-01-29 10:00:45 +00:00
Erik Johnston	3f5dd18bd4	If the same as the earliest key, assume nothing has changed.	2016-01-28 18:11:41 +00:00
Erik Johnston	40431251cb	Correctly update _entity_to_key	2016-01-28 18:05:43 +00:00
Erik Johnston	82cf3a8043	Fix inequalities	2016-01-28 17:44:04 +00:00
Erik Johnston	0663c5bd52	Include cache hits with has_entity_changed	2016-01-28 17:27:28 +00:00
Erik Johnston	45cf827c8f	Change name and doc has_entity_changed	2016-01-28 16:39:18 +00:00
Erik Johnston	00cb3eb24b	Cache tags and account data	2016-01-28 16:37:41 +00:00
Erik Johnston	c23a8c7833	Ensure keys to RoomStreamChangeCache are ints	2016-01-28 15:55:26 +00:00
Erik Johnston	e1941442d4	Invalidate caches properly. Remove unused arg	2016-01-28 15:02:41 +00:00
Erik Johnston	b97f6626b6	Add cache to room stream	2016-01-27 17:33:26 +00:00
David Baker	7cd418d38e	Don't add the member functiopn if we're not using treecache	2016-01-22 13:40:37 +00:00
David Baker	cd80019eec	docs	2016-01-22 12:21:13 +00:00
David Baker	d552861346	Revert all the bits changing keys of eeverything that used LRUCaches to tuples	2016-01-22 12:18:14 +00:00
David Baker	10f76dc5da	Make LRU cache not default to treecache & add options to use it	2016-01-22 12:10:33 +00:00
David Baker	5b142788d2	Add __contains__	2016-01-22 11:49:59 +00:00
David Baker	eaa836e8ca	Docs for treecache	2016-01-22 11:47:22 +00:00
David Baker	8acc5cb60f	Add invalidate_many here too	2016-01-22 11:22:32 +00:00
David Baker	330be18ec5	peppate	2016-01-21 19:17:32 +00:00
David Baker	f1f8122120	Change LRUCache to be tree-based so we can delete subtrees.	2016-01-21 19:16:25 +00:00
Matthew Hodgson	6c28ac260c	copyrights	2016-01-07 04:26:29 +00:00
Mark Haines	d12c00bdc3	Add some docstring explaining the snapshot cache does	2015-12-23 15:18:11 +00:00
Mark Haines	7fa71e3267	Add a unit test for the snapshot cache	2015-12-23 11:48:03 +00:00
Mark Haines	9ac417fa88	Add a cache for initialSync responses that expires after 5 minutes	2015-12-22 18:27:56 +00:00
Mark Haines	6a5ff5f223	Track the time spent in the database per request. ... and track the number of transactions that request started.	2015-12-07 17:56:11 +00:00
Mark Haines	3dd1630848	Add a setter for the current log context. ... Move the resource tracking inside that setter so that it is easier to make sure that the resource tracking isn't double counting the resource usage.	2015-12-07 10:51:18 +00:00
Mark Haines	5231737369	Add comments to explain why we are hardcoding RUSAGE_THREAD	2015-12-04 11:53:38 +00:00
Mark Haines	d6059bdd2a	Fix warnings	2015-12-04 11:34:23 +00:00
Mark Haines	48a2526d62	Track the cpu used in the main thread by each logging context	2015-12-03 21:03:01 +00:00
Mark Haines	a412b9a465	Run the background updates when starting synapse.	2015-11-10 15:50:58 +00:00
Daniel Wagner-Hall	2cebe53545	Exchange 3pid invites for m.room.member invites	2015-11-05 16:43:19 +00:00
Erik Johnston	5bc690408d	Merge pull request #340 from matrix-org/erikj/server_retries ... Retry dead servers a lot less often	2015-11-05 16:15:50 +00:00
Mark Haines	c452dabc3d	Remove the LockManager class because it wasn't being used	2015-11-04 14:08:15 +00:00
Erik Johnston	eacb068ac2	Retry dead servers a lot less often	2015-11-02 16:56:30 +00:00
Daniel Wagner-Hall	9261ef3a15	Merge pull request #312 from matrix-org/daniel/3pidinvites ... Stuff signed data in a standalone object	2015-10-19 15:52:34 +01:00
Mark Haines	9371a35e89	Merge pull request #306 from matrix-org/markjh/unused_methods ... Remove some login classes from synapse.	2015-10-16 18:18:41 +01:00
Daniel Wagner-Hall	0e5239ffc3	Stuff signed data in a standalone object ... Makes both generating it in sydent, and verifying it here, simpler at the cost of some repetition	2015-10-16 17:45:48 +01:00
Daniel Wagner-Hall	c225d63e9e	Add signing host and keyname to signatures	2015-10-16 15:07:56 +01:00
Daniel Wagner-Hall	b8dd5b1a2d	Verify third party ID server certificates	2015-10-16 14:54:54 +01:00
Daniel Wagner-Hall	643b5fcdc8	Look for keys on the right objects	2015-10-15 13:10:30 +01:00
Daniel Wagner-Hall	f38df51e8d	Merge branch 'develop' into daniel/3pidinvites	2015-10-15 11:51:55 +01:00
Mark Haines	5338220d3a	synapse.util.emailutils was unused	2015-10-15 10:39:33 +01:00
Daniel Wagner-Hall	d82c5f7b5c	Use more descriptive error code	2015-10-13 18:02:00 +01:00
Daniel Wagner-Hall	0c38e8637f	Remove unnecessary class-wrapping	2015-10-13 18:00:38 +01:00
Mark Haines	32d66738b0	Fix pep8 warnings.	2015-10-13 17:18:29 +01:00
Mark Haines	7639c3d9e5	Bounce all deferreds through the reactor to make debugging easier. ... If all deferreds wait a reactor tick before resolving then there is always a chance to add an errback to the deferred so that stacktraces get reported, rather than being discarded.	2015-10-13 17:13:04 +01:00
Daniel Wagner-Hall	17dffef5ec	Move event contents into third_party_layout field	2015-10-13 15:48:12 +01:00
Daniel Wagner-Hall	5b3e9713dd	Implement third party identifier invites	2015-10-01 17:49:52 +01:00
Mark Haines	372ac60375	synapse/util/__init__.py:unwrap_deferred was unused	2015-09-22 18:16:07 +01:00
Mark Haines	e85c7873dc	Allow non-ascii filenames for attachments	2015-08-26 16:26:37 +01:00
Erik Johnston	8e254862f4	Don't assume @cachedList function returns keys for everything	2015-08-18 11:11:33 +01:00
Erik Johnston	cfa62007a3	Docstring	2015-08-12 16:42:46 +01:00
Erik Johnston	d7451e0f22	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache	2015-08-12 10:30:30 +01:00
Erik Johnston	4807616e16	Wire up the dictionarycache to the metrics	2015-08-12 10:13:35 +01:00
Erik Johnston	2df8dd9b37	Move all the caches into their own package, synapse.util.caches	2015-08-11 18:00:59 +01:00
Erik Johnston	dcefac3b06	Comments	2015-08-10 14:16:24 +01:00
Erik Johnston	f596ff402e	Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache	2015-08-06 13:37:56 +01:00
Erik Johnston	7eea3e356f	Make @cached cache deferreds rather than the deferreds' values	2015-08-06 13:33:34 +01:00
Erik Johnston	07507643cb	Use dictionary cache to do group -> state fetching	2015-08-05 15:11:42 +01:00
Erik Johnston	c67ba143fa	Move DictionaryCache	2015-08-04 15:58:28 +01:00
Erik Johnston	9beaedd164	Enforce ascii filenames for uploads	2015-06-30 10:31:59 +01:00
Erik Johnston	7fa1363fb0	Merge pull request #192 from matrix-org/erikj/fix_log_context ... Fix log context when sending requests	2015-06-19 16:21:40 +01:00
Erik Johnston	a68abc79fd	Add comment on cancellation of observers	2015-06-19 11:48:55 +01:00
Erik Johnston	653533a3da	Fix log context when sending requests	2015-06-19 11:46:49 +01:00
Erik Johnston	22049ea700	Refactor the notifier.wait_for_events code to be clearer. Add _NotifierUserStream.new_listener that accpets a token to avoid races.	2015-06-18 15:49:24 +01:00
Erik Johnston	dd74436ffd	Unused import	2015-05-28 15:47:20 +01:00
Erik Johnston	31ade3b3e9	Remove a deep copy	2015-05-28 13:45:23 +01:00
Erik Johnston	4429e720ae	Merge branch 'master' of github.com:matrix-org/synapse into develop	2015-05-22 10:33:00 +01:00
Mark Haines	d61ce3f670	Add a cache for get_current_state with state_key	2015-05-21 16:41:39 +01:00
Erik Johnston	7cd6a6f6cf	Awful idea for speeding up fetching of events	2015-05-14 15:34:02 +01:00
Erik Johnston	cdb3757942	Refactor _get_events	2015-05-14 13:31:55 +01:00
Mark Haines	ec07dba29e	Merge pull request #143 from matrix-org/erikj/SYN-375 ... SYN-375 - Lots of unhandled deferred exceptions.	2015-05-12 15:25:54 +01:00
Erik Johnston	8022b27fc2	Make distributer.fire work as it did	2015-05-12 13:14:48 +01:00
Erik Johnston	95dedb866f	Unwrap defer.gatherResults failures	2015-05-12 13:14:29 +01:00
Erik Johnston	2236ef6c92	Fix up leak. Add warnings.	2015-05-08 19:53:34 +01:00
Erik Johnston	da1aa07db5	Add some docs	2015-05-08 16:52:49 +01:00
Erik Johnston	476899295f	Change the way we do logging contexts so that they survive divergences	2015-05-08 16:32:18 +01:00
Erik Johnston	fca28d243e	Change the way we create observers to deferreds so that we don't get spammed by 'unhandled errors'	2015-05-08 16:28:08 +01:00
Erik Johnston	6f8e2d517e	Merge branch 'develop' of github.com:matrix-org/synapse into postgres	2015-04-27 14:41:40 +01:00
Erik Johnston	1c82fbd2eb	Implement create_observer. ... `create_observer` takes a deferred and create a new deferred that *observers* the original deferred. Any callbacks added to the observing deferred will *not* affect the origin deferred.	2015-04-27 13:59:37 +01:00
Erik Johnston	806f380a8b	Make LruCache thread safe, as its used for event cache	2015-04-15 16:07:59 +01:00
Erik Johnston	a43b40449b	Merge branch 'develop' of github.com:matrix-org/synapse into mysql	2015-04-08 16:46:56 +01:00