10077 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Erik Johnston	3d33eef6fc	Store state groups separately from events (#2784) ... * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints	2018-02-06 14:31:24 +00:00
Richard van der Hoff	b31bf0bb51	Merge pull request #2849 from matrix-org/rav/clean_up_state_delta ... Remove redundant return value from _calculate_state_delta	2018-02-05 17:42:20 +01:00
Richard van der Hoff	9a304ef2b0	Merge pull request #2848 from matrix-org/rav/refactor_search_insert ... Factor out common code for search insert	2018-02-05 17:42:09 +01:00
Richard van der Hoff	ebfe64e3d6	Use StateResolutionHandler to resolve state in persist events ... ... and thus benefit (hopefully) from its cache.	2018-02-05 16:23:26 +00:00
Richard van der Hoff	225dc3b4cb	Flatten _get_new_state_after_events ... rejig the if statements to simplify the logic and reduce indentation	2018-02-05 16:23:25 +00:00
Richard van der Hoff	9fcbbe8e7d	Check that events being persisted have state_group	2018-02-05 16:23:25 +00:00
Richard van der Hoff	447aed42d2	Add event_map param to resolve_state_groups	2018-02-05 16:23:25 +00:00
Richard van der Hoff	ee6fb4cf85	Remove redundant return value from _calculate_state_delta ... we already have the state from _get_new_state_after_events, so returning it from _calculate_state_delta is just confusing.	2018-02-05 16:23:20 +00:00
Richard van der Hoff	3c7b480ba3	Factor out common code for search insert ... we can reuse the same code as is used for event insert, for doing the background index population.	2018-02-05 16:12:14 +00:00
Erik Johnston	25c0a020f4	Updates tests	2018-02-05 16:01:48 +00:00
Erik Johnston	3fa362502c	Update places where we create events	2018-02-05 16:01:48 +00:00
Erik Johnston	5ff3d23564	Split event creation into a separate handler	2018-02-05 16:01:48 +00:00
Richard van der Hoff	c46e75d3d8	Move store_event_search_txn to SearchStore ... ... as a precursor to making event storing and doing the bg update share some code.	2018-02-05 15:43:22 +00:00
Richard van der Hoff	db91e72ade	Merge pull request #2844 from matrix-org/rav/evicted_metrics ... montoring metrics for number of cache evictions	2018-02-05 16:34:36 +01:00
Richard van der Hoff	bc496df192	report metrics on number of cache evictions	2018-02-05 15:34:01 +00:00
Erik Johnston	a1beca0e25	Fix broken unit test for media storage	2018-02-05 12:44:03 +00:00
Erik Johnston	b5049d2e5c	Add .vscode to gitignore	2018-02-05 12:06:46 +00:00
Erik Johnston	1f881e0746	Merge pull request #2791 from matrix-org/erikj/media_storage_refactor ... Ensure media is in local cache before thumbnailing	2018-02-05 11:28:52 +00:00
Richard van der Hoff	80b8a28100	Factor out common code for search insert ... we can reuse the same code as is used for event insert, for doing the background index population.	2018-02-04 00:23:06 +00:00
Richard van der Hoff	bd25f9cf36	Clean up work_mem handling ... Add some comments and improve exception handling when twiddling work_mem for the search update	2018-02-03 23:05:41 +00:00
Richard van der Hoff	4eeae7ad65	Move store_event_search_txn to SearchStore ... ... as a precursor to making event storing and doing the bg update share some code.	2018-02-03 22:59:45 +00:00
Richard van der Hoff	bb9f0f3cdb	Merge branch 'develop' into matthew/gin_work_mem	2018-02-03 22:40:28 +00:00
Richard van der Hoff	6b02fc80d1	Reinstate event_search_postgres_gist handler ... People may have queued updates for this, so we can't just delete it.	2018-02-02 14:32:51 +00:00
Richard van der Hoff	9c9356512e	Merge pull request #2845 from matrix-org/rav/urlcache_error_handling ... Handle url_previews with no content-type	2018-02-02 15:27:52 +01:00
Richard van der Hoff	18eae413af	Merge pull request #2842 from matrix-org/rav/state_resolution_handler ... Factor out resolve_state_groups to a separate handler	2018-02-02 15:27:35 +01:00
Richard van der Hoff	78d6ddba86	Merge pull request #2841 from matrix-org/rav/refactor_calc_state_delta ... factor _get_new_state_after_events out of _calculate_state_delta	2018-02-02 15:27:15 +01:00
Richard van der Hoff	9dcd667ac2	Merge pull request #2836 from matrix-org/rav/resolve_state_events_docstring ... Docstring fixes	2018-02-02 15:26:49 +01:00
Richard van der Hoff	33cac3dc29	Merge pull request #2818 from turt2live/travis/admin-list-media ... Add an admin route to get all the media in a room	2018-02-02 11:41:03 +01:00
Travis Ralston	6e87b34f7b	Merge branch 'develop' into travis/admin-list-media	2018-02-01 18:05:47 -07:00
Richard van der Hoff	d5352cbba8	Handle url_previews with no content-type ... avoid failing with an exception if the remote server doesn't give us a Content-Type header. Also, clean up the exception handling a bit.	2018-02-02 00:53:46 +00:00
Richard van der Hoff	14737ba495	doc arg types for _seperate	2018-02-01 12:41:34 +00:00
Richard van der Hoff	e15d4ea248	More docstring fixes ... Fix a couple of errors in docstrings	2018-02-01 12:41:34 +00:00
Richard van der Hoff	a18828c129	Fix docstring for StateHandler.resolve_state_groups ... The return type was a complete lie, so fix it	2018-02-01 12:41:34 +00:00
Richard van der Hoff	6da4c4d3bd	Factor out resolve_state_groups to a separate handler ... We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer.	2018-02-01 12:40:04 +00:00
Richard van der Hoff	0cbda53819	Rename resolve_state_groups -> resolve_state_groups_for_events ... (to make way for a method that actually just does the state group resolution)	2018-02-01 12:40:00 +00:00
Richard van der Hoff	77c0629ebc	Merge pull request #2837 from matrix-org/rav/fix_quarantine_media ... Fix sql error in quarantine_media	2018-02-01 11:45:58 +01:00
Travis Ralston	e16e45b1b4	pep8 ... Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-31 15:30:38 -07:00
Richard van der Hoff	e1e4ec9f9d	factor _get_new_state_after_events out of _calculate_state_delta ... This reduces the scope of a bunch of variables	2018-01-31 21:32:09 +00:00
Richard van der Hoff	78e7e05188	Merge pull request #2838 from matrix-org/rav/fix_logging_on_dns_fail ... Remove spurious log argument	2018-01-31 22:18:46 +01:00
Richard van der Hoff	ad48dfe73d	Merge pull request #2835 from matrix-org/rav/remove_event_type_param ... Remove unused "event_type" param on state.get_current_state_ids	2018-01-31 22:17:57 +01:00
Richard van der Hoff	518a74586c	Merge pull request #2834 from matrix-org/rav/better_persist_event_exception_handling ... Improve exception handling in persist_event	2018-01-31 22:17:38 +01:00
Richard van der Hoff	d1fe4db882	Merge pull request #2833 from matrix-org/rav/pusher_hacks ... Logging and metrics for the http pusher	2018-01-31 22:16:59 +01:00
Richard van der Hoff	421d68ca8c	Merge pull request #2817 from matrix-org/rav/http_conn_pool ... Use a connection pool for the SimpleHttpClient	2018-01-31 22:14:22 +01:00
Richard van der Hoff	326189c25a	Script to move remote media to another media store	2018-01-31 18:45:10 +00:00
Travis Ralston	3af53c183a	Add admin api documentation for list media endpoint ... Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-31 08:15:59 -07:00
Travis Ralston	63c4383927	Documentation and naming ... Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-31 08:07:52 -07:00
Richard van der Hoff	af19f5e9aa	Remove spurious log argument ... ... which would cause scary-looking and unhelpful errors in the log on dns fail	2018-01-30 17:52:03 +00:00
Richard van der Hoff	773f0eed1e	Fix sql error in quarantine_media	2018-01-30 15:02:51 +00:00
Richard van der Hoff	adfc0c9539	docstring for get_current_state_ids	2018-01-29 17:39:55 +00:00
Richard van der Hoff	d413a2ba98	Remove unused "event_type" param on state.get_current_state_ids ... this param doesn't seem to be used, and is a bit pointless anyway because it can easily be replicated by the caller. It is also horrible, because it changes the return type of the method.	2018-01-29 17:06:57 +00:00