875 Commits

Author	SHA1	Message	Date
Erik Johnston	f407cbd2f1	PEP8	2015-05-20 13:02:01 +01:00
Erik Johnston	227f8ef031	Split out _get_event_from_row back into defer and _txn version	2015-05-20 13:00:57 +01:00
Erik Johnston	2bc60c55af	Fix _get_backfill_events to return events in the correct order	2015-05-20 12:57:00 +01:00
Erik Johnston	9084cdd70f	Ensure event_results is a set	2015-05-19 16:34:31 +01:00
Erik Johnston	aa729349dd	Fix event_backwards_extrem insertion to ignore outliers	2015-05-19 15:27:00 +01:00
Mark Haines	99914ec9f8	Merge pull request #152 from matrix-org/notifier_performance ... Notifier performance	2015-05-18 17:49:59 +01:00
Erik Johnston	165eb2dbe6	Comments and shuffle of functions	2015-05-18 15:18:41 +01:00
Erik Johnston	65878a2319	Remove unused metric	2015-05-18 14:06:30 +01:00
Erik Johnston	4d1b6f4ad1	Remove rejected events if we don't want rejected events	2015-05-18 14:03:46 +01:00
Mark Haines	0b0033c40b	Merge branch 'develop' into notifier_performance	2015-05-18 13:50:01 +01:00
Erik Johnston	10f1bdb9a2	Move get_events functions to storage.events	2015-05-18 10:21:40 +01:00
Erik Johnston	c71176858b	Newline, remove debug logging	2015-05-18 10:11:14 +01:00
Erik Johnston	c3b37abdfd	PEP8	2015-05-15 16:59:58 +01:00
Erik Johnston	6c74fd62a0	Revert limiting of fetching, it didn't help perf.	2015-05-15 16:45:35 +01:00
Erik Johnston	9ff7f66a2b	init j	2015-05-15 16:36:03 +01:00
Erik Johnston	70f272f71c	Don't completely drain the list	2015-05-15 16:34:17 +01:00
Erik Johnston	8763dd80ef	Don't fetch prev_content for current_state	2015-05-15 15:33:01 +01:00
Erik Johnston	acb12cc811	Make store.get_current_state fetch events asyncly	2015-05-15 15:20:05 +01:00
Erik Johnston	d62dee7eae	Remove more debug logging	2015-05-15 15:06:37 +01:00
Erik Johnston	0f29cfabc3	Remove debug logging	2015-05-15 14:06:42 +01:00
Erik Johnston	e275a9c0d9	preserve log context	2015-05-15 11:54:51 +01:00
Erik Johnston	aa32bd38e4	Add a wait	2015-05-15 11:35:04 +01:00
Erik Johnston	372d4c6d7b	Srsly. Don't use closures. Baaaaaad	2015-05-15 11:26:00 +01:00
Erik Johnston	575ec91d82	Correctly pass through params	2015-05-15 11:15:10 +01:00
Erik Johnston	de01438a57	Sort out error handling	2015-05-15 11:00:50 +01:00
Erik Johnston	a2c4f3f150	Fix daedlock	2015-05-15 10:54:04 +01:00
Erik Johnston	1d566edb81	Remove race condition	2015-05-14 16:54:35 +01:00
Erik Johnston	ef3d8754f5	Call from right thread	2015-05-14 15:41:55 +01:00
Erik Johnston	142934084a	Count and loop	2015-05-14 15:40:21 +01:00
Erik Johnston	96c5b9f87c	Don't start up more fetch_events	2015-05-14 15:36:04 +01:00
Erik Johnston	7cd6a6f6cf	Awful idea for speeding up fetching of events	2015-05-14 15:34:02 +01:00
Erik Johnston	7f4105a5c9	Turn off preemptive transactions	2015-05-14 14:51:06 +01:00
Erik Johnston	f4d58deba1	PEP8	2015-05-14 14:45:42 +01:00
Erik Johnston	386b7330d2	Move from _base to events	2015-05-14 14:45:22 +01:00
Erik Johnston	7d6a1dae31	Jump out early	2015-05-14 14:27:58 +01:00
Erik Johnston	656223fbd3	Actually, we probably want to run this in a transaction	2015-05-14 14:26:35 +01:00
Erik Johnston	2f7f8e1c2b	Preemptively jump into a transaction if we ask for get_prev_content	2015-05-14 14:17:36 +01:00
Erik Johnston	e1e9f0c5b2	loop -> gatherResults	2015-05-14 13:58:49 +01:00
Erik Johnston	ab78a8926e	Err, we probably want a bigger limit	2015-05-14 13:47:16 +01:00
Erik Johnston	f6f902d459	Move fetching of events into their own transactions	2015-05-14 13:45:48 +01:00
Erik Johnston	cdb3757942	Refactor _get_events	2015-05-14 13:31:55 +01:00
Erik Johnston	36ea26c5c0	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/perf	2015-05-14 12:01:38 +01:00
David Baker	7c549dd557	Add ID generator for push_rules_enable to #resolve SYN-378	2015-05-14 11:44:03 +01:00
Mark Haines	899d4675dd	Merge branch 'notifier_unify' into notifier_performance	2015-05-14 11:36:44 +01:00
Erik Johnston	4f1d984e56	Add index on events	2015-05-13 17:22:26 +01:00
Erik Johnston	968b01a91a	Actually use async method	2015-05-13 17:02:46 +01:00
Erik Johnston	4071f29653	Fetch events from events_id in their own transactions	2015-05-13 16:59:41 +01:00
Erik Johnston	a988361aea	Typo	2015-05-13 15:44:15 +01:00
Erik Johnston	8888982db3	Don't insert None	2015-05-13 15:43:32 +01:00
Erik Johnston	cf706cc6ef	Don't return None	2015-05-13 15:31:25 +01:00
Erik Johnston	5971d240d4	Limit batch size	2015-05-13 15:26:49 +01:00
Erik Johnston	ca4f458787	Fetch events in bulk	2015-05-13 15:13:42 +01:00
Erik Johnston	6edff11a88	Don't fetch redaction and rejection stuff for each event, so we can use index only scan	2015-05-13 14:39:05 +01:00
Mark Haines	63878c0379	Don't bother checking for updates if the stream token hasn't advanced for a user	2015-05-13 13:42:21 +01:00
Erik Johnston	02590c3e1d	Temp turn off checking for rejections and redactions	2015-05-13 11:31:28 +01:00
Erik Johnston	619a21812b	defer.gatherResults loop	2015-05-13 11:29:03 +01:00
Erik Johnston	fec4485e28	Batch fetching of events for state groups	2015-05-13 11:22:42 +01:00
Erik Johnston	409bcc76bd	Load events for state group seperately	2015-05-13 11:13:31 +01:00
Erik Johnston	e122685978	You need to call contextmanager	2015-05-12 16:12:37 +01:00
Mark Haines	a6fb2aa2a5	Merge pull request #144 from matrix-org/erikj/logging_context ... Preserving logging contexts	2015-05-12 15:23:50 +01:00
Mark Haines	1fce36b111	Merge pull request #149 from matrix-org/erikj/backfill ... Backfill support	2015-05-12 15:20:32 +01:00
Erik Johnston	8b28209c60	Err, delete the right stuff	2015-05-12 15:02:53 +01:00
Erik Johnston	30c72d377e	Newlines	2015-05-12 14:47:40 +01:00
Erik Johnston	e4eddf9b36	We do actually want to delete rows out of event_backward_extremities	2015-05-12 14:47:23 +01:00
Erik Johnston	c1779a79bc	Fix up _handle_prev_events to not try to insert duplicate rows	2015-05-12 14:41:50 +01:00
Erik Johnston	74850d7f75	Do state groups persistence /after/ checking if we have already persisted the event	2015-05-12 14:14:58 +01:00
Erik Johnston	da6a7bbdde	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/logging_context	2015-05-12 13:10:42 +01:00
Mark Haines	5002056b16	SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called from the main thread after the transaction completes, not from database thread before the transaction completes.	2015-05-12 11:20:40 +01:00
Erik Johnston	4df11b5039	Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards'	2015-05-12 10:28:10 +01:00
Erik Johnston	84e6b4001f	Initial hack at wiring together pagination and backfill	2015-05-11 18:01:31 +01:00
Erik Johnston	17653a5dfe	Move storage.stream._StreamToken to types.RoomStreamToken	2015-05-11 18:01:01 +01:00
Erik Johnston	b036596b75	Prefer to use _simple_*.	2015-05-11 11:24:01 +01:00
Erik Johnston	cd525c0f5a	push_rules table expects an 'id' field	2015-05-11 11:24:01 +01:00
Erik Johnston	22d2f498fa	Fix push rule bug: can't insert bool into small int column	2015-05-10 10:50:51 +01:00
Erik Johnston	476899295f	Change the way we do logging contexts so that they survive divergences	2015-05-08 16:32:18 +01:00
Mark Haines	1c7912751e	Drop the old table not the new table	2015-05-08 16:04:32 +01:00
Mark Haines	9d36eb4eab	Rename unique constraint	2015-05-08 16:01:55 +01:00
Mark Haines	b0f71db3ff	Remove unsigned	2015-05-08 15:59:51 +01:00
Mark Haines	84e1cacea4	Bump schema version	2015-05-08 15:58:14 +01:00
Mark Haines	6538d445e8	Make the timestamps in server_keys_json bigints	2015-05-08 15:55:17 +01:00
Erik Johnston	0c4ac271ca	Merge branch 'erikj/executemany' of github.com:matrix-org/synapse into erikj/SYN-371	2015-05-05 18:21:19 +01:00
Erik Johnston	0cf7e480b4	And use buffer(...) there as well	2015-05-05 18:20:01 +01:00
Erik Johnston	ed2584050f	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemany	2015-05-05 18:15:20 +01:00
Erik Johnston	977338a7af	Use buffer(...) when inserting into bytea column	2015-05-05 18:12:53 +01:00
Mark Haines	31049c4d72	Merge pull request #139 from matrix-org/bugs/SYN-369 ... Fix race with cache invalidation. SYN-369	2015-05-05 17:46:13 +01:00
Mark Haines	deb0237166	Add some doc-string	2015-05-05 17:45:11 +01:00
Mark Haines	d18f37e026	Collect the invalidate callbacks on the transaction object rather than passing around a separate list	2015-05-05 17:32:21 +01:00
Erik Johnston	9951542393	Add a comment about the zip(*[zip(sorted(...),...)])	2015-05-05 17:06:55 +01:00
Mark Haines	041b6cba61	SYN-369: Add comments to the sequence number logic in the cache	2015-05-05 16:32:44 +01:00
Mark Haines	63075118a5	Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access	2015-05-05 16:24:04 +01:00
Erik Johnston	531d7955fd	Don't insert without deduplication. In this case we never actually use this table, so simply remove the insert entirely	2015-05-05 16:12:28 +01:00
Mark Haines	bfa4a7f8b0	Invalidate the room_member cache if the current state events updates	2015-05-05 15:43:49 +01:00
Mark Haines	d0fece8d3c	Missing return for when the event was already persisted	2015-05-05 15:39:09 +01:00
Erik Johnston	bdcd7693c8	Fix indentation	2015-05-05 15:14:48 +01:00
Erik Johnston	43c2e8deae	Add support for using executemany	2015-05-05 15:13:25 +01:00
Erik Johnston	1692dc019d	Don't call 'encode_parameter' no-op	2015-05-05 15:00:30 +01:00
Mark Haines	a9aea68fd5	Invalidate the caches from the correct thread	2015-05-05 14:57:08 +01:00
Mark Haines	261d809a47	Sequence the modifications to the cache so that selects don't race with inserts	2015-05-05 14:13:50 +01:00
Erik Johnston	d9cc5de9e5	Correctly name transaction	2015-05-05 10:24:10 +01:00
Erik Johnston	b8940cd902	Remove some unused indexes	2015-05-01 16:14:25 +01:00