forked-synapse

mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00

Author	SHA1	Message	Date
Richard van der Hoff	9dcd667ac2	Merge pull request #2836 from matrix-org/rav/resolve_state_events_docstring Docstring fixes	2018-02-02 15:26:49 +01:00
Travis Ralston	6e87b34f7b	Merge branch 'develop' into travis/admin-list-media	2018-02-01 18:05:47 -07:00
Richard van der Hoff	d5352cbba8	Handle url_previews with no content-type avoid failing with an exception if the remote server doesn't give us a Content-Type header. Also, clean up the exception handling a bit.	2018-02-02 00:53:46 +00:00
Richard van der Hoff	14737ba495	doc arg types for _seperate	2018-02-01 12:41:34 +00:00
Richard van der Hoff	e15d4ea248	More docstring fixes Fix a couple of errors in docstrings	2018-02-01 12:41:34 +00:00
Richard van der Hoff	a18828c129	Fix docstring for StateHandler.resolve_state_groups The return type was a complete lie, so fix it	2018-02-01 12:41:34 +00:00
Richard van der Hoff	6da4c4d3bd	Factor out resolve_state_groups to a separate handler We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer.	2018-02-01 12:40:04 +00:00
Richard van der Hoff	0cbda53819	Rename resolve_state_groups -> resolve_state_groups_for_events (to make way for a method that actually just does the state group resolution)	2018-02-01 12:40:00 +00:00
Richard van der Hoff	77c0629ebc	Merge pull request #2837 from matrix-org/rav/fix_quarantine_media Fix sql error in quarantine_media	2018-02-01 11:45:58 +01:00
Travis Ralston	e16e45b1b4	pep8 Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-31 15:30:38 -07:00
Richard van der Hoff	e1e4ec9f9d	factor _get_new_state_after_events out of _calculate_state_delta This reduces the scope of a bunch of variables	2018-01-31 21:32:09 +00:00
Richard van der Hoff	78e7e05188	Merge pull request #2838 from matrix-org/rav/fix_logging_on_dns_fail Remove spurious log argument	2018-01-31 22:18:46 +01:00
Richard van der Hoff	ad48dfe73d	Merge pull request #2835 from matrix-org/rav/remove_event_type_param Remove unused "event_type" param on state.get_current_state_ids	2018-01-31 22:17:57 +01:00
Richard van der Hoff	518a74586c	Merge pull request #2834 from matrix-org/rav/better_persist_event_exception_handling Improve exception handling in persist_event	2018-01-31 22:17:38 +01:00
Richard van der Hoff	d1fe4db882	Merge pull request #2833 from matrix-org/rav/pusher_hacks Logging and metrics for the http pusher	2018-01-31 22:16:59 +01:00
Richard van der Hoff	421d68ca8c	Merge pull request #2817 from matrix-org/rav/http_conn_pool Use a connection pool for the SimpleHttpClient	2018-01-31 22:14:22 +01:00
Travis Ralston	63c4383927	Documentation and naming Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-31 08:07:52 -07:00
Richard van der Hoff	af19f5e9aa	Remove spurious log argument ... which would cause scary-looking and unhelpful errors in the log on dns fail	2018-01-30 17:52:03 +00:00
Richard van der Hoff	773f0eed1e	Fix sql error in quarantine_media	2018-01-30 15:02:51 +00:00
Richard van der Hoff	adfc0c9539	docstring for get_current_state_ids	2018-01-29 17:39:55 +00:00
Richard van der Hoff	d413a2ba98	Remove unused "event_type" param on state.get_current_state_ids this param doesn't seem to be used, and is a bit pointless anyway because it can easily be replicated by the caller. It is also horrible, because it changes the return type of the method.	2018-01-29 17:06:57 +00:00
Richard van der Hoff	b387ee17b6	Improve exception handling in persist_event 1. use `deferred.errback()` instead of `deferred.errback(e)`, which means that a Failure object will be constructed using the current exception state, including its stack trace - so the stack trace is saved in the Failure, leading to better exception reports. 2. Set `consumeErrors=True` on the ObservableDeferred, because we know that there will always be at least one observer - which avoids a spurious "CRITICAL: unhandled exception in Deferred" error in the logs	2018-01-29 17:05:33 +00:00
Richard van der Hoff	03dd745fe2	Better logging when pushes fail	2018-01-29 15:49:06 +00:00
Richard van der Hoff	e051abd20b	add appid/device_display_name to to pusher logging	2018-01-29 15:04:16 +00:00
Richard van der Hoff	02ba118f81	Increase http conn pool size	2018-01-29 14:30:15 +00:00
Richard van der Hoff	46022025ea	Fix SQL for user search fix some syntax errors for user search when search_all_users is enabled fixes #2801, hopefully	2018-01-27 17:21:57 +00:00
Richard van der Hoff	4b0f06e99c	Merge pull request #2830 from matrix-org/rav/factor_out_get_conn Factor out get_db_conn to HomeServer base class	2018-01-27 17:56:25 +01:00
Neil Johnson	e98f0f9112	Merge pull request #2827 from matrix-org/fix_server_500_on_public_rooms_call_when_no_rooms_exist Fix server 500 on public rooms call when no rooms exist	2018-01-26 20:22:40 +00:00
Richard van der Hoff	25adde9a04	Factor out get_db_conn to HomeServer base class This function is identical to all subclasses, so we may as well push it up to the base class to reduce duplication (and make use of it in the tests)	2018-01-26 00:56:49 +00:00
Neil Johnson	73560237d6	add white space line	2018-01-26 00:15:10 +00:00
Neil Johnson	86c4f49a31	rather than try reconstruct the results object, better to guard against the xrange step argument being 0	2018-01-26 00:12:02 +00:00
Neil Johnson	f632083576	fix return type, should be a dict	2018-01-25 23:52:17 +00:00
Neil Johnson	6c6e197b0a	fix PEP8 violation	2018-01-25 23:47:46 +00:00
Neil Johnson	d02e43b15f	remove white space	2018-01-25 23:29:46 +00:00
Neil Johnson	349c739966	synapse 500s on a call to publicRooms in the case where the number of public rooms is zero, the specific cause is due to xrange trying to use a step value of zero, but if the total room number really is zero then it makes sense to just bail and save the extra processing	2018-01-25 23:28:44 +00:00
Matthew Hodgson	9a72b70630	fix thinko on 3pid whitelisting	2018-01-24 11:07:47 +01:00
Richard van der Hoff	25e2456ee7	Merge pull request #2816 from matrix-org/rav/metrics_comments Add some comments about the reactor tick time metric	2018-01-23 19:39:26 +00:00
Matthew Hodgson	d32385336f	add ?ts massaging for ASes (#2754 ) blindly implement ?ts for AS. untested	2018-01-23 09:59:06 +01:00
Richard van der Hoff	4528dd2443	Fix logging and add user_id	2018-01-22 20:15:42 +00:00
Richard van der Hoff	93efd7eb04	logging and debug for http pusher	2018-01-22 18:14:10 +00:00
Matthew Hodgson	ab9f844aaf	Add federation_domain_whitelist option (#2820 ) Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network	2018-01-22 19:11:18 +01:00
Matthew Hodgson	d84f65255e	Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid add registrations_require_3pid and allow_local_3pids	2018-01-22 13:57:22 +00:00
Travis Ralston	a94d9b6b82	Appease the linter These are ids anyways, not mxc uris. Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-20 22:49:46 -07:00
Travis Ralston	5552ed9a7f	Add an admin route to get all the media in a room This is intended to be used by administrators to monitor the media that is passing through their server, if they wish. Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-20 22:37:53 -07:00
Richard van der Hoff	2c8526cac7	Use a connection pool for the SimpleHttpClient In particular I hope this will help the pusher, which makes many requests to sygnal, and is currently negotiating SSL for each one.	2018-01-20 00:55:44 +00:00
Richard van der Hoff	87b7d72760	Add some comments about the reactor tick time metric	2018-01-19 23:51:04 +00:00
Matthew Hodgson	49fce04624	fix typo (thanks sytest)	2018-01-19 19:55:38 +00:00
Richard van der Hoff	b0d9e633ee	Merge pull request #2814 from matrix-org/rav/fix_urlcache_thumbs Use the right path for url_preview thumbnails	2018-01-19 18:57:15 +00:00
Richard van der Hoff	ad7ec63d08	Use the right path for url_preview thumbnails This was introduced by #2627: we were overwriting the original media for url previews with the thumbnails :/ (fixes https://github.com/vector-im/riot-web/issues/6012, hopefully)	2018-01-19 18:29:39 +00:00
Matthew Hodgson	62d7d66ae5	oops, check all login types	2018-01-19 18:23:56 +00:00
Matthew Hodgson	8fe253f19b	fix PR nitpicking	2018-01-19 18:23:45 +00:00
Matthew Hodgson	293380bef7	trailing commas	2018-01-19 15:38:53 +00:00
Matthew Hodgson	447f4f0d5f	rewrite based on PR feedback: * [ ] split config options into allowed_local_3pids and registrations_require_3pid * [ ] simplify and comment logic for picking registration flows * [ ] fix docstring and move check_3pid_allowed into a new util module * [ ] use check_3pid_allowed everywhere @erikjohnston PTAL	2018-01-19 15:33:55 +00:00
Matthew Hodgson	9d332e0f79	fix up v1, and improve errors	2018-01-19 00:53:58 +00:00
Matthew Hodgson	0af58f14ee	fix pep8	2018-01-19 00:33:51 +00:00
Matthew Hodgson	28a6ccb49c	add registrations_require_3pid lets homeservers specify a whitelist for 3PIDs that users are allowed to associate with. Typically useful for stopping people from registering with non-work emails	2018-01-19 00:19:58 +00:00
Erik Johnston	cd871a3057	Fix storage provider bug introduced when renamed to store_local	2018-01-18 18:37:59 +00:00
Erik Johnston	8ff6726c0d	Merge pull request #2812 from matrix-org/erikj/media_storage_provider_config Make storage providers configurable	2018-01-18 18:33:57 +00:00
Erik Johnston	d69768348f	Fix passing wrong config to provider constructor	2018-01-18 17:14:05 +00:00
Erik Johnston	8e85220373	Remove duplicate directory test	2018-01-18 17:12:35 +00:00
Erik Johnston	3fe2bae857	Missing staticmethod	2018-01-18 17:11:45 +00:00
Erik Johnston	aae77da73f	Fixup comments	2018-01-18 17:11:29 +00:00
Erik Johnston	b6dc7044a9	Merge pull request #2804 from matrix-org/erikj/file_consumer Add decent impl of a FileConsumer	2018-01-18 16:31:33 +00:00
Erik Johnston	9a89dae8c5	Fix typo in thumbnail resource causing access times to be incorrect	2018-01-18 15:06:24 +00:00
Erik Johnston	0af5dc63a8	Make storage providers more configurable	2018-01-18 14:07:21 +00:00
Richard van der Hoff	d57765fc8a	Fix bugs in block metrics ... which I introduced in #2785	2018-01-18 12:24:42 +00:00
Erik Johnston	2cf6a7bc20	Use better file consumer	2018-01-18 12:00:46 +00:00
Erik Johnston	4a53f3a3e8	Ensure media is in local cache before thumbnailing	2018-01-18 12:00:46 +00:00
Erik Johnston	be0dfcd4a2	Do logcontexts correctly	2018-01-18 11:57:57 +00:00
Erik Johnston	1432f7ccd5	Move test stuff to tests	2018-01-18 11:57:57 +00:00
Erik Johnston	2f18a2647b	Make all fields private	2018-01-18 11:57:54 +00:00
Richard van der Hoff	d6af5512bb	Merge pull request #2809 from matrix-org/rav/metrics_errors better exception logging in callbackmetrics	2018-01-18 11:46:37 +00:00
Richard van der Hoff	ce236f8ac8	better exception logging in callbackmetrics when we fail to render a metric, give a clue as to which metric it was	2018-01-18 11:30:49 +00:00
Erik Johnston	dc519602ac	Ensure we registerProducer isn't called twice	2018-01-18 11:07:17 +00:00
Erik Johnston	17b54389fe	Fix _notify_empty typo	2018-01-18 11:05:34 +00:00
Erik Johnston	28b338ed9b	Move definition of paused_producer to __init__	2018-01-18 11:04:41 +00:00
Erik Johnston	a177325b49	Fix comments	2018-01-18 11:02:43 +00:00
Richard van der Hoff	36da256cc6	Merge pull request #2805 from matrix-org/rav/log_state_res Log room when doing state resolution	2018-01-17 18:05:04 +00:00
Richard van der Hoff	1224612a79	Log room when doing state resolution Mostly because it helps figure out what is prompting the resolution	2018-01-17 17:11:59 +00:00
Erik Johnston	bc67e7d260	Add decent impl of a FileConsumer Twisted core doesn't have a general purpose one, so we need to write one ourselves. Features: - All writing happens in background thread - Supports both push and pull producers - Push producers get paused if the consumer falls behind	2018-01-17 16:43:03 +00:00
Erik Johnston	a87006f9c7	Merge pull request #2783 from matrix-org/erikj/media_last_accessed Keep track of last access time for local media	2018-01-17 16:39:02 +00:00
Matthew Hodgson	06db5c4b76	Merge pull request #2803 from matrix-org/matthew/fix-userdir-sql fix SQL when searching all users	2018-01-17 16:27:54 +00:00
Matthew Hodgson	2d9ab533f9	fix SQL when searching all users	2018-01-17 15:58:52 +00:00
Richard van der Hoff	390093d45e	Split resolve_events into two functions ... so that the return type doesn't depend on the arg types	2018-01-17 15:44:31 +00:00
Erik Johnston	2fb3a28c98	Remove lost comment	2018-01-17 14:59:44 +00:00
Richard van der Hoff	a7e4ff9cca	Merge pull request #2795 from matrix-org/rav/track_db_scheduling Track DB scheduling delay per-request	2018-01-17 14:29:37 +00:00
Richard van der Hoff	f884cfffb9	Merge pull request #2797 from matrix-org/rav/user_id_checking Sanity checking for user ids	2018-01-17 14:29:12 +00:00
Richard van der Hoff	a5213df1f7	Sanity checking for user ids Check the user_id passed to a couple of APIs for validity, to avoid "IndexError: list index out of range" exception which looks scary and results in a 500 rather than a more useful error. Fixes #1432, among other things	2018-01-17 14:28:54 +00:00
Richard van der Hoff	e8f7541d3f	Merge remote-tracking branch 'origin/develop' into rav/track_db_scheduling	2018-01-17 14:01:57 +00:00
Richard van der Hoff	fb6563b4be	Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millis Track db txn time in millisecs	2018-01-17 13:52:42 +00:00
Richard van der Hoff	1954e867b4	Merge pull request #2796 from matrix-org/rav/fix_closed_connection_errors Fix 'NoneType' object has no attribute 'writeHeaders'	2018-01-17 13:52:24 +00:00
Richard van der Hoff	11ab2f56f5	Merge branch 'develop' into rav/fix_closed_connection_errors	2018-01-17 11:25:11 +00:00
Richard van der Hoff	0486a7814a	Merge branch 'develop' into rav/rework_run_interaction	2018-01-17 11:24:38 +00:00
Richard van der Hoff	90c14da992	Merge branch 'develop' into rav/db_txn_time_in_millis	2018-01-17 11:19:55 +00:00
Richard van der Hoff	38506773eb	Merge remote-tracking branch 'origin/develop' into rav/optimise_logging_context	2018-01-17 10:57:13 +00:00
Erik Johnston	300edc2348	Update last access time when thumbnails are viewed	2018-01-17 10:24:43 +00:00
Erik Johnston	05f98a2224	Keep track of last access time for local media	2018-01-17 10:24:43 +00:00
Erik Johnston	3cb2dabaad	Merge pull request #2789 from matrix-org/erikj/fix_thumbnails Fix thumbnailing remote files	2018-01-17 10:22:20 +00:00
Erik Johnston	d728c47142	Add docstring	2018-01-17 10:06:14 +00:00
Richard van der Hoff	936482d507	Fix 'NoneType' object has no attribute 'writeHeaders' Avoid throwing a (harmless) exception when we try to write an error response to an http request where the client has disconnected. This comes up as a CRITICAL error in the logs which tends to mislead people into thinking there's an actual problem	2018-01-16 17:58:16 +00:00
Richard van der Hoff	3d12d97415	Track DB scheduling delay per-request For each request, track the amount of time spent waiting for a db connection. This entails adding it to the LoggingContext and we may as well add metrics for it while we are passing.	2018-01-16 17:23:32 +00:00
Richard van der Hoff	0f5d2cc37c	Merge branch 'rav/db_txn_time_in_millis' into rav/track_db_scheduling	2018-01-16 17:09:15 +00:00
Richard van der Hoff	8615f19d20	rework runInteraction in terms of runConnection ... so that we can share the code	2018-01-16 17:08:29 +00:00
Matthew Hodgson	5e97ca7ee6	fix typo	2018-01-16 16:52:35 +00:00
Erik Johnston	d863f68cab	Use local vars	2018-01-16 16:24:15 +00:00
Erik Johnston	6368e5c0ab	Change _generate_thumbnails to take media_type	2018-01-16 16:17:38 +00:00
Erik Johnston	0a90d9ede4	Move setting of file_id up to caller	2018-01-16 16:03:05 +00:00
Richard van der Hoff	6324b65f08	Track db txn time in millisecs ... to reduce the amount of floating-point foo we do.	2018-01-16 15:53:18 +00:00
Richard van der Hoff	44a498418c	Optimise LoggingContext creation and copying It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit.	2018-01-16 15:49:42 +00:00
Erik Johnston	5dfc83704b	Fix typo	2018-01-16 14:32:56 +00:00
Richard van der Hoff	febdca4b37	Merge pull request #2785 from matrix-org/rav/reorganise_metrics_again Reorganise request and block metrics	2018-01-16 14:10:21 +00:00
Richard van der Hoff	f5f89fda21	Merge pull request #2790 from matrix-org/rav/preserve_event_logcontext_leak Fix a logcontext leak in persist_events	2018-01-16 14:09:13 +00:00
Erik Johnston	307f88dfb6	Fix up log lines	2018-01-16 13:53:52 +00:00
Richard van der Hoff	807e848f0f	Merge pull request #2787 from matrix-org/rav/worker_event_counts Metrics for events processed in appservice and fed sender	2018-01-16 13:14:09 +00:00
Richard van der Hoff	4a31a61ef9	Merge pull request #2786 from matrix-org/rav/rdata_metrics Metrics for number of RDATA commands received	2018-01-16 13:13:53 +00:00
Erik Johnston	9795b9ebb1	Correctly use server_name/file_id when generating/fetching remote thumbnails	2018-01-16 12:02:06 +00:00
Erik Johnston	c5b589f2e8	Log when we respond with 404	2018-01-16 12:01:40 +00:00
Richard van der Hoff	64ddec1bc0	Fix a logcontext leak in persist_events ObserveableDeferred expects its callbacks to be called without any logcontexts, whereas it turns out we were calling them with the logcontext of the request which initiated the persistence loop. It seems wrong that we are attributing work done in the persistence loop to the request that happened to initiate it, so let's solve this by dropping the logcontext for it. (I'm not sure this actually causes any real problems other than messages in the debug log, but let's clean it up anyway)	2018-01-16 11:47:36 +00:00
Erik Johnston	a4c5e4a645	Fix thumbnailing remote files	2018-01-16 11:37:50 +00:00
Erik Johnston	1159abbdd2	Merge pull request #2767 from matrix-org/erikj/media_storage_refactor Refactor MediaRepository to separate out storage	2018-01-16 10:23:50 +00:00
Richard van der Hoff	a027c2af8d	Metrics for events processed in appservice and fed sender More metrics I wished I'd had	2018-01-15 18:23:24 +00:00
Richard van der Hoff	5c3c32f16f	Metrics for number of RDATA commands received I found myself wishing we had this.	2018-01-15 17:45:55 +00:00
Richard van der Hoff	39f4e29d01	Reorganise request and block metrics In order to circumvent the number of duplicate foo:count metrics increasing without bounds, it's time for a rearrangement. The following are all deprecated, and replaced with synapse_util_metrics_block_count: synapse_util_metrics_block_timer:count synapse_util_metrics_block_ru_utime:count synapse_util_metrics_block_ru_stime:count synapse_util_metrics_block_db_txn_count:count synapse_util_metrics_block_db_txn_duration:count The following are all deprecated, and replaced with synapse_http_server_response_count: synapse_http_server_requests synapse_http_server_response_time:count synapse_http_server_response_ru_utime:count synapse_http_server_response_ru_stime:count synapse_http_server_response_db_txn_count:count synapse_http_server_response_db_txn_duration:count The following are renamed (the old metrics are kept for now, but deprecated): synapse_util_metrics_block_timer:total -> synapse_util_metrics_block_time_seconds synapse_util_metrics_block_ru_utime:total -> synapse_util_metrics_block_ru_utime_seconds synapse_util_metrics_block_ru_stime:total -> synapse_util_metrics_block_ru_stime_seconds synapse_util_metrics_block_db_txn_count:total -> synapse_util_metrics_block_db_txn_count synapse_util_metrics_block_db_txn_duration:total -> synapse_util_metrics_block_db_txn_duration_seconds synapse_http_server_response_time:total -> synapse_http_server_response_time_seconds synapse_http_server_response_ru_utime:total -> synapse_http_server_response_ru_utime_seconds synapse_http_server_response_ru_stime:total -> synapse_http_server_response_ru_stime_seconds synapse_http_server_response_db_txn_count:total -> synapse_http_server_response_db_txn_count synapse_http_server_response_db_txn_duration:total synapse_http_server_response_db_txn_duration_seconds	2018-01-15 17:09:44 +00:00
Richard van der Hoff	992018d1c0	mechanism to render metrics with alternative names	2018-01-15 17:04:39 +00:00
Richard van der Hoff	80fa610f9c	Add some comments to metrics classes	2018-01-15 16:52:52 +00:00
Richard van der Hoff	19d274085f	Make Counter render floats Prometheus handles all metrics as floats, and sometimes we store non-integer values in them (notably, durations in seconds), so let's render them as floats too. (Note that the standard client libraries also treat Counters as floats.)	2018-01-12 23:49:44 +00:00
Richard van der Hoff	21bf87a146	Reinstate media download on thumbnail request We need to actually download the remote media when we get a request for a thumbnail.	2018-01-12 15:38:06 +00:00
Erik Johnston	694f1c1b18	Fix up comments	2018-01-12 15:02:46 +00:00
Erik Johnston	e21370ba54	Correctly reraise exception	2018-01-12 14:44:02 +00:00
Erik Johnston	85a4d78213	Make Responder a context manager	2018-01-12 13:32:03 +00:00
Erik Johnston	dcc8eded41	Add missing class var	2018-01-12 13:16:27 +00:00
Erik Johnston	fefeb0ab0e	Merge pull request #2774 from matrix-org/erikj/synctl When using synctl with workers, don't start the main synapse automatically	2018-01-12 12:00:07 +00:00
Erik Johnston	81391fa162	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/media_storage_refactor	2018-01-12 11:28:49 +00:00
Erik Johnston	1e4edd1717	Remove unnecessary condition	2018-01-12 11:28:32 +00:00
Erik Johnston	c6c009603c	Remove unused variables	2018-01-12 11:24:05 +00:00
Erik Johnston	4d88958cf6	Make class var local	2018-01-12 11:23:54 +00:00
Erik Johnston	227c491510	Comments	2018-01-12 11:22:41 +00:00
Erik Johnston	f4d93ae424	Actually make it work	2018-01-12 10:39:27 +00:00
Erik Johnston	f68e4cf690	Refactor	2018-01-12 10:11:12 +00:00
Richard van der Hoff	5f23b6d5ea	Merge pull request #2766 from matrix-org/rav/room_event Add /room/{id}/event/{id} to synapse	2018-01-11 14:47:18 +00:00
Erik Johnston	7cd34512d8	When using synctl with workers, don't start the main synapse automatically	2018-01-11 11:37:39 +00:00
Erik Johnston	07ab948c38	Merge branch 'master' of github.com:matrix-org/synapse into develop	2018-01-11 11:23:40 +00:00
Erik Johnston	825a07a974	Merge pull request #2773 from matrix-org/erikj/hash_bg Do bcrypt hashing in a background thread	2018-01-10 18:11:41 +00:00
Erik Johnston	f8e1ab5fee	Do bcrypt hashing in a background thread	2018-01-10 18:01:28 +00:00
Michael Telatynski	5f07f5694c	fix order of operations derp and also use `.get` to default to {} Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>	2018-01-10 15:11:35 +00:00
Michael Telatynski	8c9d5b4873	Fix publicised groups API (singular) over federation which was missing its fed client API, since there is no other API it might as well reuse the bulk one and unwrap it Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>	2018-01-10 15:04:51 +00:00
Richard van der Hoff	c175a5f0f2	Merge pull request #2770 from matrix-org/rav/fix_request_metrics Update http request metrics before calling servlet	2018-01-10 14:53:40 +00:00
Richard van der Hoff	d90e8ea444	Update http request metrics before calling servlet Make sure that we set the servlet name in the metrics object before calling the servlet, in case the servlet throws an exception.	2018-01-09 18:27:35 +00:00
hera	174eacc8ba	oops	2018-01-09 18:14:32 +00:00
Matthew Hodgson	a66f489678	fix GIST->GIN switch	2018-01-09 16:55:51 +00:00
Matthew Hodgson	e79db0a673	switch back from GIST to GIN indexes	2018-01-09 16:37:48 +00:00
Matthew Hodgson	e365ad329f	oops, tweak work_mem when actually storing	2018-01-09 16:30:30 +00:00
Matthew Hodgson	19f9227643	avoid 80s GIN inserts by tweaking work_mem see https://github.com/matrix-org/synapse/issues/2753 for details	2018-01-09 16:25:04 +00:00
Erik Johnston	8f03aa9f61	Add StorageProvider concept	2018-01-09 16:16:12 +00:00
Erik Johnston	2442e9876c	Make PreviewUrlResource use MediaStorage	2018-01-09 16:15:07 +00:00
Erik Johnston	9d30a7691c	Make ThumbnailResource use MediaStorage	2018-01-09 16:15:07 +00:00
Erik Johnston	9e20840e02	Use MediaStorage for remote media	2018-01-09 16:15:07 +00:00
Erik Johnston	dd3092c3a3	Use MediaStorage for local files	2018-01-09 16:15:07 +00:00
Erik Johnston	ada470bccb	Add MediaStorage class	2018-01-09 16:15:07 +00:00
Erik Johnston	1ee787912b	Add some helper classes	2018-01-09 16:15:07 +00:00
Erik Johnston	47ca5eb882	Split out add_file_headers	2018-01-09 16:15:07 +00:00
Erik Johnston	b6c9deffda	Remove dead TODO	2018-01-09 15:53:23 +00:00
Richard van der Hoff	51c9d9ed65	Add /room/{id}/event/{id} to synapse Turns out that there is a valid usecase for retrieving event by id (notably having received a push), but event ids should be scoped to room, so /event/{id} is wrong.	2018-01-09 14:39:12 +00:00
Erik Johnston	b30cd5b107	Remove dead code related to default thumbnails	2018-01-09 14:38:33 +00:00
Richard van der Hoff	cb66a2d387	Merge pull request #2763 from matrix-org/rav/fix_config_uts Fix broken config UTs	2018-01-09 12:08:08 +00:00
Richard van der Hoff	0211464ba2	Fix broken config UTs https://github.com/matrix-org/synapse/pull/2755 broke log-config generation, which in turn broke the unit tests.	2018-01-09 11:28:33 +00:00
Richard van der Hoff	3a556f1ea0	Make indentation of generated log config consistent (we had a mix of 2- and 4-space indents)	2018-01-09 11:27:19 +00:00
Travis Ralston	eccfc8e928	Fix templating error with unban permission message Fixes https://github.com/matrix-org/synapse/issues/2759 Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-07 19:52:58 -07:00
Richard van der Hoff	840f72356e	Remove 'verbosity'/'log_file' from generated cfg ... because these only really exist to confuse people nowadays. Also bring log config more into line with the generated log config, by making `level_for_storage` apply to the `synapse.storage.SQL` logger rather than `synapse.storage`.	2018-01-05 12:30:28 +00:00
Erik Johnston	864a6d2977	Bump version and changelog	2018-01-05 10:54:01 +00:00
Richard van der Hoff	6e375f4597	Merge pull request #2744 from matrix-org/rav/login_logging Better logging when login can't find a 3pid	2018-01-05 09:54:39 +00:00
Richard van der Hoff	bd91857028	Check missing fields in event_from_pdu_json Return a 400 rather than a 500 when somebody messes up their send_join	2017-12-30 18:40:19 +00:00
Richard van der Hoff	3079f80d4a	Factor out `event_from_pdu_json` turns out we have two copies of this, and neither needs to be an instance method	2017-12-30 18:40:19 +00:00
Richard van der Hoff	65abc90fb6	federation_server: clean up imports	2017-12-30 18:40:19 +00:00
Richard van der Hoff	a7b726ad18	federation_client: clean up imports	2017-12-30 18:40:19 +00:00
Richard van der Hoff	75c1b8df01	Better logging when login can't find a 3pid	2017-12-20 19:31:00 +00:00
Richard van der Hoff	3f9f1c50f3	Merge pull request #2683 from seckrv/fix_pwd_auth_prov_typo synapse/config/password_auth_providers: Fixed bracket typo	2017-12-18 22:37:21 +00:00
Silke	df0f602796	Implement listen_tcp method in remaining workers Signed-off-by: Silke <silke@slxh.eu>	2017-12-18 20:00:42 +01:00
Silke	26cd3f5690	Remove logger argument and do not catch replication listener Signed-off-by: Silke <silke@slxh.eu>	2017-12-18 20:00:42 +01:00
Silke Hofstra	ed48ecc58c	Add methods for listening on multiple addresses Add listen_tcp and listen_ssl which implement Twisted's reactor.listenTCP and reactor.listenSSL for multiple addresses. Signed-off-by: Silke Hofstra <silke@slxh.eu>	2017-12-17 13:15:48 +01:00
Silke Hofstra	37d1a90025	Allow binds to both :: and 0.0.0.0 Binding on 0.0.0.0 when :: is specified in the bind_addresses is now allowed. This causes a warning explaining the behaviour. Configuration changed to match. See #2232 Signed-off-by: Silke Hofstra <silke@slxh.eu>	2017-12-17 13:10:31 +01:00
Willem Mulder	3e59143ba8	Adapt the default config to bind on IPv6. Most deployments are on Linux (or Mac OS), so this would actually bind on both IPv4 and IPv6. Resolves #1886. Signed-off-by: Willem Mulder <willemmaster@hotmail.com>	2017-12-17 13:07:37 +01:00
Erik Johnston	069ae2a5d6	Bump changelog and version	2017-12-13 15:11:22 +00:00
Erik Johnston	d8a6c734fa	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/createroom_content	2017-12-07 14:24:01 +00:00
Erik Johnston	ef045dcd71	Copy dict in update_membership too	2017-12-07 14:17:15 +00:00
Matthew Hodgson	33cb7ef0b7	Merge pull request #2723 from matrix-org/matthew/search-all-local-users Add all local users to the user_directory and optionally search them	2017-12-05 11:09:47 +00:00
Matthew Hodgson	cdc2cb5d11	fix StoreError syntax	2017-12-05 11:09:31 +00:00
Richard van der Hoff	16ec3805e5	Fix error when deleting devices This was introduced in `d7ea8c4` / PR #2728	2017-12-05 09:49:22 +00:00
Richard van der Hoff	da1010c83a	support custom login types for validating users Wire the custom login type support from password providers into the UI-auth user-validation flows.	2017-12-05 09:43:30 +00:00
Richard van der Hoff	d7ea8c4800	Factor out a validate_user_via_ui_auth method Collect together all the places that validate a logged-in user via UI auth.	2017-12-05 09:42:30 +00:00
Richard van der Hoff	d5f9fb06b0	Refactor UI auth implementation Instead of returning False when auth is incomplete, throw an exception which can be caught with a wrapper.	2017-12-05 09:40:05 +00:00
Matthew Hodgson	c22e73293a	speed up the rate of initial spam for users	2017-12-04 18:05:28 +00:00
Matthew Hodgson	b11dca2025	better doc	2017-12-04 17:51:33 +00:00
Matthew Hodgson	74e0cc74ce	fix pep8 and tests	2017-12-04 15:11:38 +00:00
Matthew Hodgson	1bd40ca73e	switch to a simpler 'search_all_users' button as per review feedback	2017-12-04 14:58:39 +00:00
Matthew Hodgson	f397153dfc	Merge branch 'develop' into matthew/search-all-local-users	2017-11-30 01:51:38 +00:00
Matthew Hodgson	5406392f8b	specify default user_directory_include_pattern	2017-11-30 01:45:34 +00:00
Matthew Hodgson	f61e107f63	remove null constraint on user_dir.room_id	2017-11-30 01:43:50 +00:00
Matthew Hodgson	4b1fceb913	fix alternation operator for FTS4 - how did this ever work!?	2017-11-30 01:34:03 +00:00
Matthew Hodgson	a4bb133b68	fix thinkos galore	2017-11-30 01:17:15 +00:00
Matthew Hodgson	cd3697e8b7	kick the user_directory index when new users register	2017-11-29 18:33:34 +00:00
Matthew Hodgson	3241c7aac3	untested WIP but might actually work	2017-11-29 18:27:05 +00:00
Richard van der Hoff	624c46eb06	Merge pull request #2721 from matrix-org/rav/get_user_by_access_token_comments Improve comments on get_user_by_access_token	2017-11-29 17:57:06 +00:00
Matthew Hodgson	47d99a20d5	Add user_directory_include_pattern config param to expand search results to additional users Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive. It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users... ...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too. Committing it here as a WIP	2017-11-29 16:46:45 +00:00
Richard van der Hoff	ad7e570d07	Delete devices in various logout situations Make sure that we delete devices whenever a user is logged out due to any of the following situations: * /logout * /logout_all * change password * deactivate account (by the user or by an admin) * invalidate access token from a dynamic module Fixes #2672.	2017-11-29 16:44:35 +00:00
Richard van der Hoff	ae31f8ce45	Move set_password into its own handler Non-functional refactoring to move set_password. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.	2017-11-29 16:44:35 +00:00
Richard van der Hoff	7ca5c68233	Move deactivate_account into its own handler Non-functional refactoring to move deactivate_account. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.	2017-11-29 16:44:35 +00:00
Richard van der Hoff	2c6d63922a	Remove pushers when deleting access tokens Whenever an access token is invalidated, we should remove the associated pushers.	2017-11-29 16:44:35 +00:00
Richard van der Hoff	97d1a1dc01	Merge pull request #2718 from matrix-org/rav/notify_logcontexts Clear logcontext before starting fed txn queue runner	2017-11-29 16:01:46 +00:00
Richard van der Hoff	8b45de90a4	Merge pull request #2719 from matrix-org/rav/handle_missing_hashes Fix 500 when joining matrix-dev	2017-11-29 16:01:33 +00:00
Richard van der Hoff	7303ed65e1	Fix 500 when joining matrix-dev matrix-dev has an event (`$/6ANj/9QWQyd71N6DpRQPf+SDUu11+HVMeKSpMzBCwM:zemos.net`) which has no `hashes` member. Check for missing `hashes` element in events.	2017-11-29 16:00:46 +00:00
Richard van der Hoff	da562bd6a1	Improve comments on get_user_by_access_token because I have to reverse-engineer this every time.	2017-11-29 15:52:41 +00:00
Richard van der Hoff	d4fb4f7c52	Clear logcontext before starting fed txn queue runner These processes take a long time compared to the request, so there is lots of "Entering\|Restoring dead context" in the logs. Let's try to shut it up a bit.	2017-11-28 15:26:14 +00:00
Erik Johnston	dfbc45302e	PEP8	2017-11-28 15:23:26 +00:00
Erik Johnston	c4c1d170af	Fix wrong avatars when inviting multiple users when creating room We reused the `content` dictionary between invite requests, which meant they could end up reusing the profile info for a previous user	2017-11-28 15:19:15 +00:00
Luke Barnard	ab1b2d0ff2	Allow guest access to group APIs for reading	2017-11-28 11:23:00 +00:00
Richard van der Hoff	5a4da5bf78	Merge pull request #2697 from matrix-org/rav/fix_urlcache_index_error Fix error on sqlite 3.7	2017-11-27 12:25:48 +00:00
Richard van der Hoff	84b31a3e7a	Merge pull request #2713 from matrix-org/rav/no_upsert_forever Avoid retrying forever on IntegrityError	2017-11-27 12:19:35 +00:00
Richard van der Hoff	df6c72ede3	Merge pull request #2711 from matrix-org/rav/fix_dns_errhandler Fix error handling on dns lookup	2017-11-27 12:19:18 +00:00
Richard van der Hoff	04bb79f139	Merge pull request #2710 from matrix-org/rav/remove_dead_code Tiny code cleanups	2017-11-27 12:15:44 +00:00
Richard van der Hoff	e828a7380a	Merge pull request #2708 from matrix-org/rav/replication_logcontext_leaks Fix some logcontext leaks in replication resource	2017-11-27 12:15:33 +00:00
Richard van der Hoff	7ef22a41a3	Merge pull request #2707 from matrix-org/rav/fix_urlpreview Fix OPTIONS on preview_url	2017-11-27 12:15:14 +00:00
Richard van der Hoff	6be01f599b	Improve tracebacks on exceptions Use failure.Failure to recover our failure, which will give us a useful stacktrace, unlike the rethrown exception.	2017-11-27 12:05:58 +00:00
Richard van der Hoff	63ccaa5873	Avoid retrying forever on IntegrityError	2017-11-27 12:00:07 +00:00
Richard van der Hoff	8b38096a89	Fix error handling on dns lookup pass the right arguments to the errback handler Fixes "TypeError('eb() takes exactly 2 arguments (1 given)',)"	2017-11-24 16:47:48 +00:00
Richard van der Hoff	795b0849f3	Add a comment which might save some confusion	2017-11-24 00:34:56 +00:00
Richard van der Hoff	7f14f0ae38	Remove dead sync_callback This is never used; let's remove it to stop confusing things.	2017-11-24 00:32:04 +00:00
Richard van der Hoff	0edf085b68	Fix some logcontext leaks in replication resource The @measure_func annotations rely on the wrapped function respecting the logcontext rules. Add the necessary yields to make this work.	2017-11-23 23:19:43 +00:00
Richard van der Hoff	8132a6b7ac	Fix OPTIONS on preview_url Fixes #2706	2017-11-23 17:52:31 +00:00
Richard van der Hoff	6b48b3e277	fix sql fails	2017-11-22 18:06:24 +00:00
Richard van der Hoff	2908f955d1	Check database in has_completed_background_updates so that the right thing happens on workers.	2017-11-22 18:02:15 +00:00
Richard van der Hoff	68ca864141	Add config option to disable media_repo on main synapse ... to stop us doing the cache cleanup jobs on the master.	2017-11-22 16:20:27 +00:00
Richard van der Hoff	e1fd4751de	Build MediaRepositoryResource as a homeserver dependency This avoids the scenario where we have four different PreviewUrlResources configured on a single app, each of which have their own caches and cache clearing jobs.	2017-11-22 16:19:49 +00:00
Richard van der Hoff	d5a7c56ef9	Merge pull request #2698 from matrix-org/rav/remove_dead_dependencies Clean up dependency list	2017-11-21 17:38:42 +00:00
Richard van der Hoff	0b4aa2dc21	Merge pull request #2689 from matrix-org/rav/unlock_account_data_upsert Avoid locking account_data tables for upserts	2017-11-21 13:39:14 +00:00
Richard van der Hoff	7298ed7c51	Clean up dependency list remove those that aren't used at all, and replace the ones that don't have builders with simple getters rather than dynamically-generated methods.	2017-11-21 11:15:41 +00:00
Richard van der Hoff	7098b65cb8	Fix error on sqlite 3.7 Create the url_cache index on local_media_repository as a background update, so that we can detect whether we are on sqlite or not and create a partial or complete index accordingly. To avoid running the cleanup job before we have built the index, add a bailout which will defer the cleanup if the bg updates are still running. Fixes https://github.com/matrix-org/synapse/issues/2572.	2017-11-21 11:14:17 +00:00
Oliver Kurz	83d8d4d8cd	Allow use of higher versions of saml2 The package was pinned to <4.0 with `07cf96eb` because "from saml2 import config" did not work. This seems to have been fixed in the mean time in the saml2 package and therefore should not stop to use a more recent version. Signed-off-by: Oliver Kurz <okurz@suse.de>	2017-11-20 11:14:39 +01:00
Richard van der Hoff	59a7275258	Merge pull request #2688 from matrix-org/rav/unlock_more_upsert Avoid locking for upsert on pushers tables	2017-11-17 13:09:31 +00:00
Richard van der Hoff	d8a05418f9	Merge branch 'master' into develop	2017-11-17 12:13:14 +00:00
Luke Barnard	cdf6fc15b0	Merge pull request #2686 from matrix-org/luke/as-flair Add automagical AS Publicised Group(s)	2017-11-17 10:13:46 +00:00
Richard van der Hoff	74bbeb4373	Bump version in __init__.py	2017-11-17 10:10:53 +00:00
Jurek	eded7084d2	Fix auth handler #2678	2017-11-17 10:07:27 +00:00
Matthew Hodgson	34c3d0a386	typo	2017-11-17 01:54:02 +00:00
Luke Barnard	7fb0e98b03	Extract group_id from the dict for multiple use	2017-11-16 18:18:30 +00:00
Richard van der Hoff	c46139a17e	Avoid locking account_data tables for upserts	2017-11-16 18:08:01 +00:00
Luke Barnard	d8391f0541	Remove unused GROUP_ID_REGEX	2017-11-16 18:05:57 +00:00
Luke Barnard	4e8374856d	Document get_groups_for_user	2017-11-16 18:03:46 +00:00
Luke Barnard	270f9cd23a	Flake8	2017-11-16 18:03:31 +00:00
Luke Barnard	9d83d52027	Use a generator instead of a list	2017-11-16 17:57:34 +00:00
Luke Barnard	5b48eec4a1	Make sure we check AS groups for lookup on bulk	2017-11-16 17:55:15 +00:00
Luke Barnard	b1edf26051	Check group_id belongs to this domain	2017-11-16 17:54:27 +00:00
Richard van der Hoff	06e5bcfc83	Avoid locking for upsert on pushers tables * replace the upsert into deleted_pushers with an insert * no need to lock for upsert on pusher_throttle	2017-11-16 17:52:23 +00:00
Jurek	624a8bbd67	Fix auth handler #2678	2017-11-16 17:19:02 +00:00
Luke Barnard	97bd18af4e	Add automagical AS Publicised Group(s) via registration file "users" namespace: ```YAML ... namespaces: users: - exclusive: true regex: '.luke.' group_id: '+all_the_lukes:hsdomain' ... ``` This is part of giving App Services their own groups for matching users. With this, ghost users will be given the appeareance that they are in a group and that they have publicised the fact, but _only_ from the perspective of the `get_publicised_groups_for_user` API.	2017-11-16 16:44:55 +00:00
Richard van der Hoff	77a1227870	Fix broken ref to IntegrityError	2017-11-16 16:03:38 +00:00
Richard van der Hoff	7ab2b69e18	Avoid locking `pushers` table on upsert Now that _simple_upsert will retry on IntegrityError, we don't need to lock the table.	2017-11-16 15:32:01 +00:00
Richard van der Hoff	10aaa1bc15	_simple_upsert: retry on IntegrityError wrap the call to _simple_upsert_txn in a loop so that we retry on an integrityerror: this means we can avoid locking the table provided there is an unique index.	2017-11-16 15:30:15 +00:00
Richard van der Hoff	cdc9e50a5d	Cleanup in _simple_upsert_txn Bail out early to reduce indentation	2017-11-16 15:29:10 +00:00
Richard von Seck	6f05de0e5e	synapse/config/password_auth_providers: Fixed bracket typo Signed-off-by: Richard von Seck <richard.von-seck@gmx.net>	2017-11-16 15:59:38 +01:00
Richard van der Hoff	f959c01600	Merge pull request #2661 from matrix-org/rav/statereadstore Pull out bits of StateStore to a mixin	2017-11-15 17:23:01 +00:00
Richard van der Hoff	30d2730ee2	Declare support for r0.3.0	2017-11-15 16:24:22 +00:00
Richard van der Hoff	1b83c09c03	Merge pull request #2675 from matrix-org/rav/remove_broken_logcontext_funcs Remove preserve_context_over_{fn, deferred}	2017-11-15 11:13:53 +00:00
David Baker	7190a550dc	Merge pull request #2650 from matrix-org/dbkr/push_include_content_option Rename redact_content option to include_content	2017-11-15 10:47:38 +00:00
Richard van der Hoff	b2cd6accf5	Remove __PreservingContextDeferred too	2017-11-14 23:00:10 +00:00
Richard van der Hoff	008aa2fc6d	Merge pull request #2671 from matrix-org/rav/room_list_fixes Reshuffle room list request code	2017-11-14 18:11:02 +00:00
Erik Johnston	03feb7a34d	Bump version and changelog	2017-11-14 14:51:25 +00:00
Richard van der Hoff	35a4b63240	Pull out bits of StateStore to a mixin ... so that we don't need to secretly gut-wrench it for use in the slaved stores. I haven't done the other stores yet, but we should. I'm tired of the workers breaking every time we tweak the stores because I forgot to gut-wrench the right method. fixes https://github.com/matrix-org/synapse/issues/2655.	2017-11-14 11:43:58 +00:00
Richard van der Hoff	4dd1bfa8c1	Revert "Revert "move _state_group_cache to statestore"" We're going to fix this properly on this branch, so that the _state_group_cache can end up in StateGroupReadStore. This reverts commit `ab335edb02`.	2017-11-14 11:43:58 +00:00
Richard van der Hoff	6caa379ba1	Merge pull request #2658 from matrix-org/rav/store_heirarchy_init Make __init__ consistent across Store hierarchy	2017-11-14 11:43:12 +00:00
Richard van der Hoff	7e6fa29cb5	Remove preserve_context_over_{fn, deferred} Both of these functions ae known to leak logcontexts. Replace the remaining calls to them and kill them off.	2017-11-14 11:22:42 +00:00
Richard van der Hoff	44a1bfd6a6	Reshuffle room list request code I'm not entirely sure if this will actually help anything, but it simplifies the code and might give further clues about why room list search requests are blowing out the get_current_state_ids caches.	2017-11-14 10:29:58 +00:00
Richard van der Hoff	1fc66c7460	Add a load of logging to the room_list handler So we can see what it gets up to.	2017-11-14 10:23:47 +00:00
Richard van der Hoff	7bd6c87eca	Merge pull request #2668 from turt2live/travis/whoami Add a route for determining who you are	2017-11-14 09:54:21 +00:00
Travis Ralston	812c191939	Remove redundent call Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-11-13 12:44:21 -07:00
Richard van der Hoff	c741ba59c9	Merge pull request #2669 from matrix-org/rav/cache_urlpreview_failure Cache failures in url_preview handler	2017-11-13 18:36:24 +00:00
David Baker	45ab288e07	Print instead of logging because we had to wait until the logger was set up	2017-11-13 18:32:08 +00:00
Richard van der Hoff	8b33ac8f6c	Merge branch 'develop' into rav/invalid_request_utf8	2017-11-13 11:56:22 +00:00
Richard van der Hoff	6cfee09be9	Make __init__ consitstent across Store heirarchy Add db_conn parameters to the `__init__` methods of the *Store classes, so that they are all consistent, which makes the multiple inheritance work correctly (and so that we can later extract mixins which can be used in the slavedstores)	2017-11-13 10:46:07 +00:00
Erik Johnston	ab335edb02	Revert "move _state_group_cache to statestore" This reverts commit `f5cf3638e9`.	2017-11-13 10:05:33 +00:00
Erik Johnston	bfbf1e1f1a	Up cache size of get_global_account_data_by_type_for_user	2017-11-13 09:52:11 +00:00
Travis Ralston	2d314b771f	Add a route for determining who you are Useful for applications which may have an access token, but no idea as to who owns it. Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-11-12 23:39:38 -07:00
Richard van der Hoff	5d15abb120	Bit more logging	2017-11-10 16:58:04 +00:00
Richard van der Hoff	46790f50cf	Cache failures in url_preview handler Reshuffle the caching logic in the url_preview handler so that failures are cached (and to generally simplify things and fix the logcontext leaks).	2017-11-10 16:50:50 +00:00
Richard van der Hoff	e508145c9b	Add some more comments appservice user registration Explain why we don't validate userids registered via app services	2017-11-10 12:39:45 +00:00
Richard van der Hoff	e0ebd1e4bd	Downcase userids for shared-secret registration	2017-11-10 12:39:05 +00:00
Richard van der Hoff	f90649eb2b	Fix 500 on invalid utf-8 in request If somebody sends us a request where the the body is invalid utf-8, we should return a 400 rather than a 500. (json.loads throws a UnicodeError in this situation) We might as well catch all Exceptions here: it seems very unlikely that we would get a request that *isn't caused by invalid json.	2017-11-10 09:15:39 +00:00
Richard van der Hoff	9b599bc18d	Downcase userid on registration Force username to lowercase before attempting to register https://github.com/matrix-org/synapse/issues/2660	2017-11-09 22:20:01 +00:00
Richard van der Hoff	9b803ccc98	Revert "Allow upper-case characters in mxids" This reverts commit `b70b646903`.	2017-11-09 21:57:24 +00:00
Richard van der Hoff	b70b646903	Allow upper-case characters in mxids Because we're never going to be able to fix this :'(	2017-11-09 19:36:13 +00:00
Erik Johnston	2dce6b15c3	Fix typo	2017-11-09 15:56:16 +00:00
Erik Johnston	4e2b2508af	Register group servlet	2017-11-09 15:49:42 +00:00
Erik Johnston	13735843c7	Namespace visibility options for groups	2017-11-09 15:27:18 +00:00
Richard van der Hoff	618c7b816a	Merge pull request #2656 from matrix-org/rav/fix_deactivate Fix 'NoneType' not iterable in /deactivate	2017-11-09 15:20:35 +00:00
Richard van der Hoff	889102315e	Fix 'NoneType' not iterable in /deactivate make sure we actually return a value from user_delete_access_tokens	2017-11-09 15:15:33 +00:00
David Baker	b2a788e902	Make the commented config have the default	2017-11-09 10:11:42 +00:00
Erik Johnston	82e4bfb53d	Add brackets	2017-11-09 10:06:42 +00:00
Erik Johnston	e8814410ef	Have an explicit API to update room config	2017-11-08 16:13:27 +00:00
Erik Johnston	94ff2cda73	Revert "Modify group room association API to allow modification of is_public"	2017-11-08 15:43:34 +00:00
Erik Johnston	d305987b40	Merge pull request #2631 from xyzz/fix_appservice_event_backlog Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services	2017-11-08 11:54:10 +00:00
Erik Johnston	167eb01d83	Merge pull request #2637 from spantaleev/avoid-noop-media-deletes Avoid no-op media deletes	2017-11-08 11:53:27 +00:00
David Baker	ad408beb66	better comments	2017-11-08 11:50:08 +00:00
David Baker	1b870937ae	Log if any of the old config flags are set	2017-11-08 11:46:24 +00:00
David Baker	2a98ba0ed3	Rename redact_content option to include_content The redact_content option never worked because it read the wrong config section. The PR introducing it (https://github.com/matrix-org/synapse/pull/2301) had feedback suggesting the name be changed to not re-use the term 'redact' but this wasn't incorporated. This reanmes the option to give it a less confusing name, and also means that people who've set the redact_content option won't suddenly see a behaviour change when upgrading synapse, but instead can set include_content if they want to. This PR also updates the wording of the config comment to clarify that this has no effect on event_id_only push. Includes https://github.com/matrix-org/synapse/pull/2422	2017-11-08 10:35:30 +00:00
Richard van der Hoff	02a9a93bde	Merge pull request #2649 from matrix-org/rav/fix_delta_on_state_res Fix bug in state group storage	2017-11-08 09:22:13 +00:00
Richard van der Hoff	e148438e97	s/items/iteritems/	2017-11-08 09:21:41 +00:00
Ilya Zhuravlev	d46386d57e	Remove useless assignment in notify_interested_services	2017-11-07 22:23:22 +03:00
Matthew Hodgson	228ccf1fe3	Merge pull request #2643 from matrix-org/matthew/user_dir_typos Fix various embarrassing typos around user_directory and add some doc.	2017-11-07 17:31:11 +00:00
Richard van der Hoff	780dbb378f	Update deltas when doing auth resolution Fixes a bug where the persisted state groups were different to those actually being used after auth resolution.	2017-11-07 16:43:00 +00:00
Richard van der Hoff	1ca4288135	factor out _update_context_for_auth_events This is duplicated, so let's factor it out before fixing it	2017-11-07 16:43:00 +00:00
Richard van der Hoff	f5cf3638e9	move _state_group_cache to statestore this is internal to statestore, so let's keep it there.	2017-11-07 16:43:00 +00:00
Erik Johnston	5ef5e14ecc	Merge pull request #2636 from farialima/me-master Fix for #2635: correctly update rooms avatar/display name when modified by admin	2017-11-07 13:49:27 +00:00
Erik Johnston	76c9af193c	Revert "Merge branch 'master' of github.com:matrix-org/synapse into develop" This reverts commit `f9b255cd62`, reversing changes made to `1bd654dabd`.	2017-11-07 13:32:35 +00:00
Erik Johnston	f9b255cd62	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-11-07 13:31:03 +00:00
Luke Barnard	38b265cb51	Remember to pick is_admin out of the db	2017-11-07 11:24:04 +00:00
Luke Barnard	5561c09091	Return whether a user is an admin within a group	2017-11-07 11:18:45 +00:00
Matthew Hodgson	3db5ff69b2	Merge pull request #2576 from maximevaillancourt/exclude-noscript-url-preview Ignore <noscript> tags when generating URL preview descriptions	2017-11-07 11:09:22 +00:00
Matthew Hodgson	631fa4a1b7	create new indexes before dropping old ones to keep safetynet in place	2017-11-07 10:41:55 +00:00
Richard van der Hoff	bf993db11c	Logging and logcontext fixes for Limiter Add some logging to the Limiter in a similar spirit to the Linearizer, to help debug issues. Also fix a logcontext leak. Also refactor slightly to avoid throwing exceptions.	2017-11-07 00:48:57 +00:00
Matthew Hodgson	4ad883398f	s/users_in_pubic_room/users_in_public_rooms/g	2017-11-04 19:39:40 +00:00
Matthew Hodgson	d802e8ca6a	s/users_in_pubic_room/users_in_public_rooms/g	2017-11-04 19:38:13 +00:00
Matthew Hodgson	a100700630	fix copyright....	2017-11-04 19:35:49 +00:00
Matthew Hodgson	b6b075fd49	s/popualte/populate/	2017-11-04 19:35:33 +00:00
Matthew Hodgson	d1622e080f	s/intial/initial/	2017-11-04 19:35:14 +00:00
Slavi Pantaleev	805196fbeb	Avoid no-op media deletes If there are no media entries to delete, avoid creating transactions, prepared statements and unnecessary log entries. Signed-off-by: Slavi Pantaleev <slavi@devture.com>	2017-11-04 09:50:15 +02:00
Francois Granade	f103b91ffa	removed unused import flagged by flake8a	2017-11-03 18:45:49 +01:00
Francois Granade	fa4f337b49	Fix for issue 2635: correctly update rooms avatar/display name when modified by admin	2017-11-03 18:25:04 +01:00
Ilya Zhuravlev	45fbe4ff67	Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services	2017-11-02 22:49:43 +03:00
David Baker	f851bc8182	Merge pull request #2630 from matrix-org/luke/fix-rooms-in-group Make the get_rooms_in_group API more sane	2017-11-02 17:23:17 +00:00
David Baker	9e09a1800b	Merge pull request #2629 from matrix-org/rav/register_inhibit_login support inhibit_login in /register	2017-11-02 16:51:35 +00:00
Luke Barnard	a34c586a89	Make the get_rooms_in_group API more sane Return entries with is_public = True when they're public and is_public = False otherwise.	2017-11-02 16:42:30 +00:00
Richard van der Hoff	6c3a02072b	support inhibit_login in /register Allow things to pass inhibit_login when registering to ... inhibit logins.	2017-11-02 16:31:07 +00:00
David Baker	4b36897cd9	Merge remote-tracking branch 'origin/develop' into rav/module_api_hooks	2017-11-02 15:19:17 +00:00
David Baker	d4553818a0	Merge pull request #2627 from matrix-org/rav/custom_rest_endpoints Add a hook for custom rest endpoints	2017-11-02 15:18:37 +00:00
David Baker	6b6f03ae05	Merge pull request #2626 from matrix-org/rav/refactor_module_api Factor _AccountHandler proxy out to ModuleApi	2017-11-02 15:15:30 +00:00
Richard van der Hoff	6b60f7dca0	Add more hooks to ModuleApi add `get_user_by_req` and `invalidate_access_token`	2017-11-02 14:37:39 +00:00
Richard van der Hoff	fcdfc911ee	Add a hook for custom rest endpoints Let the user specify custom modules which can be used for implementing extra endpoints.	2017-11-02 14:36:55 +00:00
Richard van der Hoff	1189be43a2	Factor _AccountHandler proxy out to ModuleApi We're going to need to use this from places that aren't password auth, so let's move it to a proper class.	2017-11-02 14:36:11 +00:00
Richard van der Hoff	6650a07ede	Factor out _configure_named_resource This was a bit of a code vomit, so let's factor it out to preserve some sanity	2017-11-02 14:33:37 +00:00
David Baker	b19d9e2174	Merge pull request #2624 from matrix-org/rav/password_provider_notify_logout Notify auth providers on logout	2017-11-02 10:55:17 +00:00
David Baker	1f080a6c97	Merge pull request #2623 from matrix-org/rav/callbacks_for_auth_providers Allow password_auth_providers to return a callback	2017-11-02 10:49:03 +00:00
David Baker	04897c9dc1	Merge pull request #2622 from matrix-org/rav/db_access_for_auth_providers Let auth providers get to the database	2017-11-02 10:41:25 +00:00
Richard van der Hoff	979eed4362	Fix user-interactive password auth this got broken in the previous commit	2017-11-01 17:03:20 +00:00
Richard van der Hoff	bc8a5c0330	Notify auth providers on logout Provide a hook by which auth providers can be notified of logouts.	2017-11-01 16:51:51 +00:00
Richard van der Hoff	4c8f94ac94	Allow password_auth_providers to return a callback ... so that they have a way to record access tokens.	2017-11-01 16:51:03 +00:00
Richard van der Hoff	846a94fbc9	Merge pull request #2620 from matrix-org/rav/auth_non_password Let password auth providers handle arbitrary login types	2017-11-01 16:45:33 +00:00
Richard van der Hoff	3cd6b22c7b	Let password auth providers handle arbitrary login types Provide a hook where password auth providers can say they know about other login types, and get passed the relevant parameters	2017-11-01 16:43:57 +00:00
David Baker	c9b9ef575b	Merge pull request #2621 from matrix-org/rav/refactor_accesstoken_delete Move access token deletion into auth handler	2017-11-01 16:26:06 +00:00
Matthew Hodgson	275826f234	Merge pull request #2617 from matrix-org/matthew/auto-displayname automatically set default displayname on register	2017-11-01 16:21:16 +00:00
David Baker	4f0488b307	Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete	2017-11-01 16:20:19 +00:00
David Baker	e5e930aec3	Merge pull request #2615 from matrix-org/rav/break_auth_device_dep Break dependency of auth_handler on device_handler	2017-11-01 16:06:31 +00:00
David Baker	fbbacb284e	Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens Remove the last vestiges of refresh_tokens	2017-11-01 15:57:35 +00:00
Matthew Hodgson	9f7a555b4e	switch to setting default displayname in the storage layer to avoid clobbering guest user displaynames on registration	2017-11-01 15:51:30 +00:00
Richard van der Hoff	dd13310fb8	Move access token deletion into auth handler Also move duplicated deactivation code into the auth handler. I want to add some hooks when we deactivate an access token, so let's bring it all in here so that there's somewhere to put it.	2017-11-01 15:46:22 +00:00
David Baker	0bb253f37b	Apparently this is python	2017-11-01 14:02:52 +00:00
David Baker	59e7e62c4b	Log login requests Carefully though, to avoid logging passwords	2017-11-01 13:58:01 +00:00
Matthew Hodgson	f8420d6279	automatically set default displayname on register to avoid leaking ugly MXIDs and cluttering up the timeline with displayname changes as well as membership joins for autojoin rooms (e.g. the status autojoin rooms), automatically set the displayname to match the localpart of the mxid upon registration.	2017-11-01 13:15:41 +00:00
Luke Barnard	99354b430e	Merge pull request #2612 from matrix-org/luke/groups-room-relationship-is-public Modify group room association API to allow modification of is_public	2017-11-01 11:08:36 +00:00
Richard van der Hoff	74c56f794c	Break dependency of auth_handler on device_handler I'm going to need to make the device_handler depend on the auth_handler, so I need to break this dependency to avoid a cycle. It turns out that the auth_handler was only using the device_handler in one place which was an edge case which we can more elegantly handle by throwing an error rather than fixing it up.	2017-11-01 10:27:06 +00:00
Luke Barnard	318a249c8b	Leave `is_public` as required argument of update_room_group_association	2017-11-01 09:36:01 +00:00
Luke Barnard	207fabbc6a	Update docs for updating room group association	2017-11-01 09:35:15 +00:00
Richard van der Hoff	356bcafc44	Remove the last vestiges of refresh_tokens	2017-10-31 20:35:58 +00:00
Richard van der Hoff	3e0aaad190	Let auth providers get to the database Somewhat open to abuse, but also somewhat unavoidable :/	2017-10-31 17:22:29 +00:00
Luke Barnard	13b3d7b4a0	Flake8	2017-10-31 17:20:11 +00:00
Luke Barnard	20fe347906	Modify group room association API to allow modification of is_public also includes renamings to make things more consistent.	2017-10-31 17:04:28 +00:00
Richard van der Hoff	1650eb5847	DB schema interface for password auth providers Provide an interface by which password auth providers can register db schema files to be run at startup	2017-10-31 14:01:53 +00:00
David Baker	c31a7c3ff6	Merge pull request #2609 from matrix-org/rav/refactor_login Refactor some logic from LoginRestServlet into AuthHandler	2017-10-31 13:51:36 +00:00
Richard van der Hoff	1b65ae00ac	Refactor some logic from LoginRestServlet into AuthHandler I'm going to need some more flexibility in handling login types in password auth providers, so as a first step, move some stuff from LoginRestServlet into AuthHandler. In particular, we pass everything other than SAML, JWT and token logins down to the AuthHandler, which now has responsibility for checking the login type and fishing the password out of the login dictionary, as well as qualifying the user_id if need be. Ideally SAML, JWT and token would go that way too, but there's no real need for it right now and I'm trying to minimise impact. This commit should be non-functional.	2017-10-31 10:48:41 +00:00
Richard van der Hoff	ffc574a6f9	Clean up backwards-compat hacks for ldap try to make the backwards-compat flows follow the same code paths as the modern impl. This commit should be non-functional.	2017-10-31 10:47:02 +00:00
Luke Barnard	9bc17fc5fb	Fix wording on group creation error	2017-10-30 15:17:23 +00:00
Matthew Hodgson	208a6647f1	fix typo	2017-10-29 20:54:20 +00:00
Erik Johnston	d0abb4e8e6	Fix typo when checking if user is invited to group	2017-10-27 16:57:19 +01:00
Erik Johnston	977078f06d	Fix bad merge	2017-10-27 15:10:50 +01:00
Erik Johnston	6980c4557e	Merge branch 'erikj/attestation_jitter' of github.com:matrix-org/synapse into develop	2017-10-27 15:09:05 +01:00
Erik Johnston	af92f5b00f	Revert "Add jitter to validity period of attestations"	2017-10-27 15:07:21 +01:00
Erik Johnston	4ab8abbc2b	Merge branch 'erikj/attestation_local_fix' of github.com:matrix-org/synapse into develop	2017-10-27 15:07:08 +01:00
Erik Johnston	6af3656deb	Merge pull request #2595 from matrix-org/erikj/attestation_commnet Add comment about attestations	2017-10-27 14:20:19 +01:00
Richard van der Hoff	4d83632009	Merge pull request #2591 from matrix-org/rav/device_delete_auth Device deletion: check UI auth matches access token	2017-10-27 12:30:10 +01:00
Richard van der Hoff	110b373e9c	Merge pull request #2589 from matrix-org/rav/as_deactivate_account Allow ASes to deactivate their own users	2017-10-27 12:29:32 +01:00
Erik Johnston	ca571b0ec3	Add jitter to validity period of attestations This helps ensure that the renewals of attestations are spread out more evenly.	2017-10-27 11:57:27 +01:00
Luke Barnard	d8c26162a1	Merge pull request #2582 from matrix-org/luke/group-is-public Add is_public to groups table to allow for private groups	2017-10-27 11:41:13 +01:00
Erik Johnston	c067088747	Add comment about attestations	2017-10-27 11:35:41 +01:00
Luke Barnard	5451cc7792	Request is_public from database	2017-10-27 11:27:43 +01:00
Luke Barnard	124314672f	group is dict	2017-10-27 11:08:19 +01:00
Luke Barnard	6362298fa5	Create groups with is_public = True	2017-10-27 11:04:20 +01:00
Richard van der Hoff	8b56977b6f	Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header Front-end proxy: pass through auth header	2017-10-27 11:01:50 +01:00
Richard van der Hoff	173567a7f2	Docstring for post_urlencoded_get_json	2017-10-27 10:59:50 +01:00
Luke Barnard	c7d9f25d22	Fix create_group to pass requester_user_id	2017-10-27 10:57:20 +01:00
Erik Johnston	e27b76d117	Import logger	2017-10-27 10:54:02 +01:00
Richard van der Hoff	8854c039f2	Merge pull request #2585 from matrix-org/rav/unstable_to_r0 Support /keys/upload on /r0 as well as /unstable	2017-10-27 10:53:48 +01:00
Richard van der Hoff	14f581abc2	Merge pull request #2584 from matrix-org/rav/fix_httpclient_logcontexts Fix logcontext leaks in httpclient	2017-10-27 10:53:29 +01:00
Luke Barnard	2ca46c7afc	Correct logic for checking private group membership	2017-10-27 10:48:01 +01:00
Erik Johnston	82d8c1bacb	Fixup	2017-10-27 10:30:21 +01:00
Erik Johnston	2fd9831f7c	Merge pull request #2574 from matrix-org/erikj/room_list_fixes Add logging and fix log contexts for publicRooms	2017-10-27 10:01:23 +01:00
Erik Johnston	195abfe7a5	Remove incorrect attestations	2017-10-27 09:58:13 +01:00
Erik Johnston	d8dde19f04	Log if we try to do attestations for our own user and group	2017-10-27 09:55:01 +01:00
Erik Johnston	585972b51a	Don't generate group attestations for local users	2017-10-27 09:46:56 +01:00
Richard van der Hoff	7a6546228b	Device deletion: check UI auth matches access token (otherwise there's no point in the UI auth)	2017-10-27 00:04:31 +01:00
Richard van der Hoff	785bd7fd75	Allow ASes to deactivate their own users	2017-10-27 00:01:00 +01:00
Richard van der Hoff	c89e6aadff	Merge pull request #2581 from matrix-org/rav/fix_init_with_no_logfile Fix error when running synapse with no logfile	2017-10-26 22:16:57 +01:00
Richard van der Hoff	54a2525133	Front-end proxy: pass through auth header So that access-token-in-an-auth-header works.	2017-10-26 18:19:01 +01:00
Richard van der Hoff	0a5866bec9	Support /keys/upload on /r0 as well as /unstable (So that we can stop riot relying on it in /unstable)	2017-10-26 18:18:23 +01:00
Richard van der Hoff	0d8e3ad48b	Fix logcontext leaks in httpclient `preserve_context_over_fn` is borked	2017-10-26 18:17:10 +01:00
Richard van der Hoff	12ef02dc3d	SimpleHTTPClient: add support for headers Sometimes we need to pass headers into these methods	2017-10-26 17:59:50 +01:00
Luke Barnard	69e8a05f35	Make it work	2017-10-26 17:55:58 +01:00
Luke Barnard	007cd48af6	Recreate groups table instead of adding column Adding a column with non-constant default not possible in sqlite3	2017-10-26 17:55:22 +01:00
Luke Barnard	713e60b9b6	Awful hack to get default true	2017-10-26 17:38:14 +01:00
Luke Barnard	cfa4e658e0	Bump schema version to 46	2017-10-26 17:23:49 +01:00
Luke Barnard	595fe67f01	delint	2017-10-26 17:20:24 +01:00
Luke Barnard	9b2feef9eb	Add is_public to groups table to allow for private groups Prevent group API access to non-members for private groups Also make all the group code paths consistent with `requester_user_id` always being the User ID of the requesting user.	2017-10-26 16:51:32 +01:00
Richard van der Hoff	f7f90e0c8d	Fix error when running synapse with no logfile Fixes 'UnboundLocalError: local variable 'sighup' referenced before assignment'	2017-10-26 16:45:20 +01:00
Krombel	8299b323ee	add release endpoints for /thirdparty	2017-10-26 16:58:20 +02:00
Krombel	9b436c8b4c	register some /unstable endpoints in /r0 as well	2017-10-26 15:22:50 +02:00
Erik Johnston	566e21eac8	Update room_list.py	2017-10-26 11:39:54 +01:00
Maxime Vaillancourt	5287e57c86	Ignore noscript tags when generating URL previews	2017-10-25 20:44:34 -04:00
Erik Johnston	2a7e9faeec	Do logcontexts outside ResponseCache	2017-10-25 15:21:08 +01:00
Erik Johnston	1ad1ba9e6a	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-25 10:27:23 +01:00
Erik Johnston	33a9026cdf	Add logging and fix log contexts for publicRooms	2017-10-25 10:26:06 +01:00
Matthew Hodgson	efd0f5a3c5	tip for generating tls_fingerprints	2017-10-24 18:49:49 +01:00
Erik Johnston	6ba4fabdb9	Bump version and changelog	2017-10-24 14:15:27 +01:00
Erik Johnston	39dc52157d	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/group_fed_update_profile	2017-10-24 09:16:20 +01:00
Richard van der Hoff	0be99858f3	fix vars named `l` E741 says "do not use variables named ‘l’, ‘O’, or ‘I’".	2017-10-23 15:56:38 +01:00
Richard van der Hoff	eaaabc6c4f	replace 'except:' with 'except Exception:' what could possibly go wrong	2017-10-23 15:52:32 +01:00
Erik Johnston	ce6d4914f4	Correctly wire in update group profile over federation	2017-10-23 15:21:24 +01:00
Richard van der Hoff	ecf198aab8	Merge pull request #2566 from matrix-org/rav/media_logcontext_leak Fix a logcontext leak in the media repo	2017-10-23 14:47:49 +01:00
Richard van der Hoff	3267b81b81	Merge pull request #2561 from matrix-org/rav/id_checking Updates to ID checking	2017-10-23 14:39:20 +01:00
Richard van der Hoff	d03cfc4258	Fix a logcontext leak in the media repo	2017-10-23 14:34:27 +01:00
Erik Johnston	1de557975f	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-23 13:18:12 +01:00
Erik Johnston	13e16cf302	Bump version and changelog	2017-10-23 13:13:31 +01:00
Richard van der Hoff	1135193dfd	Validate group ids when parsing May as well do it whenever we parse a Group ID. We check the sigil and basic structure here so it makes sense to check the grammar in the same place.	2017-10-21 00:30:39 +01:00
Richard van der Hoff	29812c628b	Allow = in mxids and groupids ... because the spec says we should.	2017-10-20 23:42:53 +01:00
Richard van der Hoff	58fbbe0f1d	Disallow capital letters in userids Factor out a common function for checking user ids and group ids, which forbids capitals.	2017-10-20 23:37:22 +01:00
Richard van der Hoff	631d7b87b5	Remove pointless create() method It just calls the constructor, so we may as well kill it rather than having random codepaths.	2017-10-20 22:14:55 +01:00
Erik Johnston	6070647774	Correctly bump version	2017-10-19 16:40:20 +01:00
Erik Johnston	0ef0aeceac	Bump version and changelog	2017-10-19 13:48:33 +01:00
Erik Johnston	c7d46510d7	Flake8	2017-10-19 13:36:06 +01:00
Erik Johnston	ffd3f1a783	Add missing file...	2017-10-19 12:17:30 +01:00
Erik Johnston	29bafe2f7e	Add config to enable group creation	2017-10-19 12:13:44 +01:00
Erik Johnston	513c23bfd9	Enforce sensible group IDs	2017-10-19 12:01:01 +01:00
Erik Johnston	011d03a0f6	Fix typo	2017-10-19 11:22:48 +01:00
Erik Johnston	9ab859f27b	Fix typo in group attestation handling	2017-10-19 10:55:52 +01:00
Erik Johnston	bd5718d0ad	Fix typo in thumbnail generation	2017-10-19 10:27:18 +01:00
Erik Johnston	161a862ffb	Fix typo	2017-10-19 10:17:43 +01:00
Richard van der Hoff	69994c385a	Merge pull request #2553 from matrix-org/rav/fix_500_on_event_send Fix 500 error when we get an error handling a PDU	2017-10-18 11:25:44 +01:00
Richard van der Hoff	b5dbbac308	Merge pull request #2552 from matrix-org/rav/fix_500_on_dodgy_powerlevels Fix 500 error when fields missing from power_levels event	2017-10-17 20:53:30 +01:00
Richard van der Hoff	582bd19ee9	Fix 500 error when we get an error handling a PDU FederationServer doesn't have a send_failure (and nor does its subclass, ReplicationLayer), so this was failing. I'm not really sure what the idea behind send_failure is, given (a) we don't do anything at the other end with it except log it, and (b) we also send back the failure via the transaction response. I suspect there's a whole lot of dead code around it, but for now I'm just removing the broken bit.	2017-10-17 20:52:40 +01:00
Richard van der Hoff	74f99f227c	Doc some more dynamic Homeserver methods	2017-10-17 20:51:29 +01:00
Richard van der Hoff	c2bd177ea0	Fix 500 error when fields missing from power_levels event If the users or events keys were missing from a power_levels event, then we would throw 500s when trying to auth them.	2017-10-17 17:05:42 +01:00
Erik Johnston	fe6e9f580b	Merge pull request #2550 from krombel/fix_thumbnail_2548 fix thumbnailing (#2548)	2017-10-17 15:35:18 +01:00
Richard van der Hoff	7216c76654	Improve error handling for missing files (#2551 ) `os.path.exists` doesn't allow us to distinguish between permissions errors and the path actually not existing, which repeatedly confuses people. It also means that we try to overwrite existing key files, which is super-confusing. (cf issues #2455, #2379). Use os.stat instead. Also, don't recomemnd the the use of --generate-config, which screws everything up if you're using debian (cf #2455).	2017-10-17 14:46:17 +01:00
Richard van der Hoff	4cc8bb0767	Merge pull request #2549 from matrix-org/rav/event_persist_logcontexts Fix logcontext handling for persist_events	2017-10-17 13:20:35 +01:00
David Baker	4e242b3e20	Merge pull request #2545 from matrix-org/dbkr/auto_join_rooms Add config option to auto-join new users to rooms	2017-10-17 11:45:49 +01:00
Krombel	a6245478c8	fix thumbnailing (#2548 ) in commit `0e28281a` the code for thumbnailing got refactored and the renaming of this variables was not done correctly. Signed-Off-by: Matthias Kesler <krombel@krombel.de>	2017-10-17 12:45:33 +02:00
Richard van der Hoff	2e9f5ea31a	Fix logcontext handling for persist_events * don't use preserve_context_over_deferred, which is known broken. * remove a redundant preserve_fn. * add/improve some comments	2017-10-17 10:59:30 +01:00
David Baker	a9c2e930ac	pep8	2017-10-17 10:13:13 +01:00
David Baker	c05e6015cc	Add config option to auto-join new users to rooms New users who register on the server will be dumped into all rooms in auto_join_rooms in the config.	2017-10-16 17:57:27 +01:00
Luke Barnard	85f5674e44	Delint	2017-10-16 15:52:17 +01:00
Luke Barnard	c43e8a9736	Make it work. Warn about lack of user profile	2017-10-16 15:50:39 +01:00
Luke Barnard	a3ac4f6b0a	_create_rererouter for get_invited_users_in_group	2017-10-16 15:41:03 +01:00
Luke Barnard	5dfd0350c7	Merge branch 'develop' into luke/groups-invited-users	2017-10-16 15:34:36 +01:00
Luke Barnard	ca96d609e4	Merge pull request #2543 from matrix-org/luke/fix-on-group-invite-no-profile Log a warning when no profile for invited member	2017-10-16 16:33:53 +02:00
Luke Barnard	2c5972f87f	Implement GET /groups/$groupId/invited_users	2017-10-16 15:31:11 +01:00
Luke Barnard	6079d0027a	Log a warning when no profile for invited member And return empty profile	2017-10-16 14:20:45 +01:00
David Baker	9342bcfce0	Omit the *s for @room notifications They're just redundant	2017-10-16 13:38:10 +01:00
Erik Johnston	1b6b0b1e66	Add try/finally block to close t_byte_source	2017-10-13 15:34:08 +01:00
Erik Johnston	6b725cf56a	Remove old comment	2017-10-13 15:23:41 +01:00
Erik Johnston	2b24416e90	Don't reuse source but instead copy from primary media store to backup	2017-10-13 14:11:34 +01:00
Erik Johnston	b92a8e6e4a	PEP8	2017-10-13 13:58:57 +01:00
Erik Johnston	31aa7bd8d1	Move type into key	2017-10-13 13:47:38 +01:00
Erik Johnston	ad1911bbf4	Comment	2017-10-13 13:47:05 +01:00
Erik Johnston	c021c39cbd	Remove spurious addition	2017-10-13 13:46:53 +01:00
Erik Johnston	1f43d22397	Don't needlessly rename variable	2017-10-13 11:42:07 +01:00
Erik Johnston	a675bd08bd	Add paths back in...	2017-10-13 11:41:06 +01:00
Erik Johnston	4d7e1dde70	Remove unnecessary diff	2017-10-13 11:36:32 +01:00
Erik Johnston	ae5d18617a	Make things be absolute paths again	2017-10-13 11:35:44 +01:00
Erik Johnston	9732ec6797	s/write_to_file/write_to_file_and_backup/	2017-10-13 11:34:41 +01:00
Erik Johnston	0e28281a02	Fix up	2017-10-13 11:33:49 +01:00
Erik Johnston	505371414f	Fix up thumbnailing function	2017-10-13 11:23:53 +01:00
Erik Johnston	e3428d26ca	Fix typo	2017-10-13 10:39:59 +01:00
Erik Johnston	35332298ef	Fix up comments	2017-10-13 10:39:32 +01:00
Erik Johnston	64db043a71	Move makedirs to thread	2017-10-13 10:25:01 +01:00
Erik Johnston	b60859d6cc	Use make_deferred_yieldable	2017-10-13 10:24:19 +01:00
Erik Johnston	d76621a47b	Fix comments	2017-10-12 18:16:25 +01:00
Erik Johnston	4ae85ae121	Don't close prematurely..	2017-10-12 17:57:31 +01:00
Erik Johnston	cc505b4b5e	getvalue closes buffer	2017-10-12 17:52:30 +01:00
Erik Johnston	1259a76047	Get len before close	2017-10-12 17:39:23 +01:00
Erik Johnston	802ca12d05	Don't close file prematurely	2017-10-12 17:37:21 +01:00
Erik Johnston	e283b555b1	Copy everything to backup	2017-10-12 17:31:24 +01:00
Erik Johnston	b77a13812c	Typo	2017-10-12 15:32:32 +01:00
Erik Johnston	6dfde6d485	Remove dead code	2017-10-12 15:30:26 +01:00
Erik Johnston	c8eeef6947	Fix typos	2017-10-12 15:28:24 +01:00
Erik Johnston	67cb89fbdf	Fix typo	2017-10-12 15:23:41 +01:00
Erik Johnston	bf4fb1fb40	Basic implementation of backup media store	2017-10-12 15:20:59 +01:00
hera	f807f7f804	log when we get an exception handling replication updates	2017-10-12 11:51:24 +01:00
David Baker	b8d8ed1ba9	Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages Allow error strings from spam checker	2017-10-12 10:31:03 +01:00
Richard van der Hoff	cc794d60e7	Merge pull request #2532 from matrix-org/rav/fix_linearizer Fix stackoverflow and logcontexts from linearizer	2017-10-11 17:29:32 +01:00
Richard van der Hoff	8dd0c85ac5	Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handling log pdu_failures from incoming transactions	2017-10-11 17:29:14 +01:00
Richard van der Hoff	76fa695241	Merge pull request #2515 from matrix-org/rav/fix_receipt_logcontext A logformatter which includes the stack when the exception was caught when logging exceptions.	2017-10-11 17:28:01 +01:00
Richard van der Hoff	f30c4ed2bc	logformatter: fix AttributeError make sure we have the relevant fields before we try to log them.	2017-10-11 17:26:17 +01:00
Erik Johnston	b752507b48	Fix fetching remote summaries	2017-10-11 16:59:18 +01:00
Erik Johnston	818b08d0e4	peeeeeeeeep8888888888888888888888888888	2017-10-11 15:54:00 +01:00
Erik Johnston	ea18996f54	Fix group stream replication The stream update functions expect the storage function to return a list of tuples.	2017-10-11 15:44:39 +01:00
Richard van der Hoff	68fd82e840	Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext fix a logcontext leak in read receipt handling	2017-10-11 15:08:53 +01:00
Richard van der Hoff	4fad8efbfb	Fix stackoverflow and logcontexts from linearizer 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts.	2017-10-11 15:05:05 +01:00
David Baker	b78bae2d51	fix isinstance	2017-10-11 14:49:09 +01:00
Erik Johnston	271f5601f3	Fix typo in invite to group	2017-10-11 14:45:33 +01:00
David Baker	c3b7a45e84	Allow error strings from spam checker	2017-10-11 14:39:22 +01:00
Richard van der Hoff	c3e190ce67	fix a logcontext leak in read receipt handling	2017-10-11 14:37:20 +01:00
Richard van der Hoff	b75d443caf	log pdu_failures from incoming transactions ... even if we have no EDUs. This appears to have been introduced in `476899295f`.	2017-10-11 14:36:13 +01:00
Erik Johnston	27e727a146	Fix typo	2017-10-11 14:32:40 +01:00
Erik Johnston	4ce4379235	Fix attestations to check correct server name	2017-10-11 14:11:43 +01:00
Erik Johnston	c2c47550f9	Fix schema delta versions	2017-10-11 13:23:15 +01:00
Erik Johnston	535cc49f27	Merge pull request #2466 from matrix-org/erikj/groups_merged Initial Group Implementation	2017-10-11 13:20:07 +01:00
Erik Johnston	dfbf73408c	Merge pull request #2501 from matrix-org/dbkr/channel_notifications Support for channel notifications	2017-10-11 13:19:29 +01:00
Erik Johnston	bc7f3eb32f	Merge pull request #2483 from jeremycline/unfreeze-ujson-dump Unfreeze event before serializing with ujson	2017-10-11 13:18:52 +01:00
Erik Johnston	ec954f47fb	Validate room ids	2017-10-11 13:15:44 +01:00
David Baker	81a5e0073c	pep8	2017-10-10 15:53:34 +01:00
David Baker	ab1bc9bf5f	Don't KeyError if no power_levels event	2017-10-10 15:34:05 +01:00
David Baker	0f1eb3e914	Use notification levels in power_levels Rather than making the condition directly require a specific power level. This way the level require to notify a room can be configured per room.	2017-10-10 15:23:00 +01:00
Erik Johnston	84e27a592d	Merge pull request #2490 from matrix-org/erikj/drop_left_room_events Ignore incoming events for rooms that we have left	2017-10-10 11:58:32 +01:00
David Baker	c9f034b4ac	There was already a constant for this also update copyright	2017-10-10 11:47:10 +01:00
David Baker	a9f9d68631	More optimisation	2017-10-10 11:38:31 +01:00
David Baker	707374d5dc	What year is it!? Who's the president!?	2017-10-10 11:21:41 +01:00
David Baker	89fa00ddff	Merge branch 'develop' into dbkr/channel_notifications	2017-10-10 11:20:17 +01:00
Richard van der Hoff	6a6cc27aee	fed server: process PDUs for different rooms in parallel With luck, this will give a real-time improvement when there are many rooms and the server ends up calling out to fetch missing events.	2017-10-09 18:30:31 +01:00
Richard van der Hoff	4c7c4d4061	Fed server: use a linearizer for ongoing transactions We don't want to process the same transaction multiple times concurrently, so use a linearizer.	2017-10-09 18:30:10 +01:00
Richard van der Hoff	ba5b9b80a5	fed server: refactor on_incoming_transaction Move as much as possible to after the have_responded check, and reduce the number of times we iterate over the pdu list.	2017-10-09 18:10:53 +01:00
Richard van der Hoff	a6e3222fe5	Fed server: Move origin-check code to _handle_received_pdu The response-building code expects there to be an entry in the `results` list for each entry in the pdu_list, so the early `continue` was messing this up. That doesn't really matter, because all that the federation client does is log any errors, but it's pretty poor form.	2017-10-09 17:53:32 +01:00
Richard van der Hoff	3cc852d339	Fancy logformatter to format exceptions better This is a bit of an experimental change at this point; the idea is to see if it helps us track down where our stack overflows are coming from by logging the stack when the exception was caught and turned into a Failure. (We'll also need `edf2704420`). If we deploy this, we'll be able to enable it via the log config yaml.	2017-10-09 17:44:42 +01:00
Richard van der Hoff	0eeaa25694	Merge pull request #2508 from matrix-org/rav/federation_queue_logcontexts Fix up logcontext handling in (federation) TransactionQueue	2017-10-09 17:43:48 +01:00
Richard van der Hoff	aa3fac8057	Merge pull request #2507 from matrix-org/rav/execute_concurrently_log_contexts Fix logcontext handling for concurrently_execute	2017-10-09 17:43:32 +01:00
Erik Johnston	e8496efe84	Fix up comment	2017-10-09 15:17:34 +01:00
Richard van der Hoff	01bbacf3c4	Fix up logcontext handling in (federation) TransactionQueue Avoid using preserve_context_over_function, which has problems with respect to logcontexts.	2017-10-06 22:39:25 +01:00
Richard van der Hoff	148428ce76	Fix logcontext handling for concurrently_execute Avoid preserve_context_over_deferred, which is broken.	2017-10-06 22:24:28 +01:00
Richard van der Hoff	c8f568ddf9	Fix up deferred handling in federation.py * Avoid preserve_context_over_deferred, which is broken * set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled failure" erros	2017-10-06 22:14:24 +01:00
Richard van der Hoff	3ddda939d3	some comments in the state res code	2017-10-05 14:58:17 +01:00
David Baker	f878e6f8af	Spam checking: add the invitee to user_may_invite	2017-10-05 14:02:28 +01:00
David Baker	269af961e9	Make be faster	2017-10-05 13:27:12 +01:00
David Baker	ed80c6b6cc	Add fastpath optimisation	2017-10-05 13:20:22 +01:00
David Baker	e433393c4f	pep8	2017-10-05 13:08:02 +01:00
David Baker	985ce80375	They're called rooms	2017-10-05 13:03:44 +01:00
David Baker	b9b9714fd5	Get rule type right	2017-10-05 13:02:19 +01:00
David Baker	fa969cfdde	Support for channel notifications Add condition type to check the sender's power level and add a base rule using it for @channel notifications.	2017-10-05 12:39:18 +01:00
David Baker	0c8da8b519	Use better method for word boundary searching From `ebc95667b8`	2017-10-05 11:57:43 +01:00
Erik Johnston	eaaa837e00	Don't corrupt cache	2017-10-05 11:43:22 +01:00
David Baker	cbe3c3fdd4	pep8	2017-10-05 11:43:10 +01:00
David Baker	6748f0a579	Fix notif kws that start/end with non-word chars Only prepend / append word bounary characters if the search expression starts or ends with a word character, otherwise they don't work because there's no word bounary between whitespace and a non-word char.	2017-10-05 11:33:30 +01:00
David Baker	d8ce68b09b	spam check room publishing	2017-10-04 14:29:33 +01:00
David Baker	78d4ced829	un-double indent	2017-10-04 12:44:27 +01:00
David Baker	197c14dbcf	Add room creation checks to spam checker Lets the spam checker deny attempts to create rooms and add aliases to them.	2017-10-04 10:47:54 +01:00

... 9 10 11 12 13 ...

7768 Commits