2449 Commits

Author	SHA1	Message	Date
Richard van der Hoff	9dbd903f41	background updates: Fix assertion to do something	2016-07-25 14:05:23 +01:00
Richard van der Hoff	465117d7ca	Fix background_update tests ... A bit of a cleanup for background_updates, and make sure that the real background updates have run before we start the unit tests, so that they don't interfere with the tests.	2016-07-25 12:10:42 +01:00
Richard van der Hoff	363786845b	PEP8	2016-07-22 13:21:07 +01:00
Richard van der Hoff	ec5717caf5	Create index on user_ips in the background ... user_ips is kinda big, so really we want to add the index in the background once we're running. Replace the schema delta with one which will do that. I've done this in a way that's reasonably easy to reuse as there a few other indexes I need, and I don't suppose they will be the last.	2016-07-22 13:16:39 +01:00
Erik Johnston	aede7248ab	Split out a FederationReader process	2016-07-21 17:37:44 +01:00
Richard van der Hoff	c445f5fec7	storage/client_ips: remove some dead code	2016-07-21 11:58:47 +01:00
Richard van der Hoff	7314bf4682	Merge branch 'develop' into rav/get_devices_api ... (pick up PR #938 in the hope of fixing the UTs)	2016-07-20 17:40:00 +01:00
Richard van der Hoff	bc8f265f0a	GET /devices endpoint ... implement a GET /devices endpoint which lists all of the user's devices. It also returns the last IP where we saw that device, so there is some dancing to fish that out of the user_ips table.	2016-07-20 16:42:32 +01:00
Richard van der Hoff	ec041b335e	Record device_id in client_ips ... Record the device_id when we add a client ip; it's somewhat redundant as we could get it via the access_token, but it will make querying rather easier.	2016-07-20 16:41:03 +01:00
Richard van der Hoff	053e83dafb	More doc-comments ... Fix some more comments on some things	2016-07-20 16:40:28 +01:00
David Baker	e967bc86e7	Merge pull request #932 from matrix-org/rav/register_refactor ... Further registration refactoring	2016-07-20 11:03:33 +01:00
Erik Johnston	1e2a7f18a1	Merge pull request #922 from matrix-org/erikj/file_api2 ... Feature: Add filter to /messages. Add 'contains_url' to filter.	2016-07-20 10:40:48 +01:00
Erik Johnston	f91faf09b3	Comment	2016-07-20 10:18:09 +01:00
Richard van der Hoff	40cbffb2d2	Further registration refactoring ... * `RegistrationHandler.appservice_register` no longer issues an access token: instead it is left for the caller to do it. (There are two of these, one in `synapse/rest/client/v1/register.py`, which now simply calls `AuthHandler.issue_access_token`, and the other in `synapse/rest/client/v2_alpha/register.py`, which is covered below). * In `synapse/rest/client/v2_alpha/register.py`, move the generation of access_tokens into `_create_registration_details`. This means that the normal flow no longer needs to call `AuthHandler.issue_access_token`; the shared-secret flow can tell `RegistrationHandler.register` not to generate a token; and the appservice flow continues to work despite the above change.	2016-07-19 18:46:19 +01:00
Richard van der Hoff	022b9176fe	schema fix ... device_id should be text, not bigint.	2016-07-19 11:44:05 +01:00
Richard van der Hoff	f863a52cea	Add device_id support to /login ... Add a 'devices' table to the storage, as well as a 'device_id' column to refresh_tokens. Allow the client to pass a device_id, and initial_device_display_name, to /login. If login is successful, then register the device in the devices table if it wasn't known already. If no device_id was supplied, make one up. Associate the device_id with the access token and refresh token, so that we can get at it again later. Ensure that the device_id is copied from the refresh token to the access_token when the token is refreshed.	2016-07-18 16:39:44 +01:00
Erik Johnston	f52565de50	Fix /purge_history bug ... This was caused by trying to insert duplicate backward extremeties	2016-07-15 14:23:15 +01:00
Erik Johnston	978fa53cc2	Pull out min stream_ordering from ex_outlier_stream	2016-07-15 10:22:30 +01:00
Erik Johnston	eec9609e96	event_backwards_extremeties may not be empty	2016-07-15 10:22:09 +01:00
Erik Johnston	d554ca5e1d	Add support for filters in paginate_room_events	2016-07-14 15:59:04 +01:00
Erik Johnston	b64aa6d687	Add sender and contains_url field to events table	2016-07-14 15:35:43 +01:00
Mark Haines	dfde67a6fe	Add a comment explaining allow_none	2016-07-08 15:57:06 +01:00
Mark Haines	10c843fcfb	Ensure that the guest user is in the database when upgrading accounts	2016-07-08 15:15:55 +01:00
Erik Johnston	58930da52b	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-07-08 14:11:37 +01:00
Erik Johnston	b92e7955be	Comment	2016-07-07 11:42:15 +01:00
Erik Johnston	c98e1479bd	Return 400 rather than 500	2016-07-07 11:41:07 +01:00
Erik Johnston	67f2c901ea	Add rest servlet. Fix SQL.	2016-07-06 15:56:59 +01:00
Erik Johnston	a17e7caeb7	Merge branch 'erikj/shared_secret' into erikj/test2	2016-07-06 14:46:31 +01:00
Erik Johnston	be3548f7e1	Remove spurious txn	2016-07-05 17:46:51 +01:00
Erik Johnston	4adf93e0f7	Fix for postgres	2016-07-05 17:34:25 +01:00
Erik Johnston	651faee698	Add an admin option to shared secret registration	2016-07-05 17:30:22 +01:00
Erik Johnston	8f8798bc0d	Add ReadWriteLock for pagination and history prune	2016-07-05 15:30:25 +01:00
Mark Haines	dd2ccee27d	Fix typo	2016-07-05 14:06:07 +01:00
Mark Haines	b6b0132ac7	Make get_events_around more efficient on sqlite3	2016-07-05 13:55:18 +01:00
Mark Haines	d44d11d864	Use true/false for boolean parameter inclusive to avoid potential for sqli, and possibly make the code clearer	2016-07-05 10:39:13 +01:00
Erik Johnston	2d21d43c34	Add purge_history API	2016-07-05 10:28:51 +01:00
Mark Haines	0fb76c71ac	Use different SQL for postgres and sqlite3 for when using multicolumn indexes	2016-07-04 19:44:55 +01:00
Erik Johnston	a67bf0b074	Add storage function to purge history for a room	2016-07-04 16:02:50 +01:00
Mark Haines	f18d7546c6	Use a query that postgresql optimises better for get_events_around	2016-07-04 15:48:25 +01:00
Erik Johnston	f328d95cef	Feature: Add deactivate account admin API ... Allows server admins to "deactivate" accounts, which: - Revokes all access tokens - Removes all threepids - Removes password The API is a POST to `/admin/deactivate/<user_id>`	2016-06-30 15:40:58 +01:00
Mark Haines	6783534a0f	Merge pull request #886 from matrix-org/markjh/async_commit ... Optionally make committing to postgres asynchronous.	2016-06-29 15:21:58 +01:00
Erik Johnston	a70688445d	Implement purge_media_cache admin API	2016-06-29 14:57:59 +01:00
Erik Johnston	314b146b2e	Track approximate last access time for remote media	2016-06-29 11:41:20 +01:00
David Baker	aa3a4944d5	more pep8	2016-06-24 11:45:23 +01:00
David Baker	870c45913e	Use similar naming we use in email notifs for push ... Fixes https://github.com/vector-im/vector-web/issues/1654	2016-06-24 11:41:11 +01:00
David Baker	b5fb7458d5	Actually we need to order these properly ... otherwise we'll end up returning the wrong 20	2016-06-22 18:07:14 +01:00
David Baker	f73fdb04a6	Style	2016-06-22 17:51:40 +01:00
David Baker	3a4120e49a	Put most recent 20 messages in notif ... Fixes https://github.com/vector-im/vector-web/issues/1648	2016-06-22 17:47:18 +01:00
Mark Haines	d5fb561709	Optionally make committing to postgres asynchronous. ... Useful when running tests when you don't care whether the server will lose data that it claims that it has committed.	2016-06-20 17:53:38 +01:00
Mark Haines	41e4b2efea	Add the create_profile method back since the tests use it	2016-06-17 19:20:47 +01:00
Mark Haines	9f1800fba8	Remove registered_users from the distributor. ... The only place that was observed was to set the profile. I've made it so that the profile is set within store.register in the same transaction that creates the user. This required some slight changes to the registration code for upgrading guest users, since it previously relied on the distributor swallowing errors if the profile already existed.	2016-06-17 19:14:16 +01:00
Erik Johnston	1a815fb04f	Don't hit DB for noop replications queries	2016-06-08 11:33:30 +01:00
Mark Haines	0b2158719c	Remove dead code. ... Loading push rules now happens in the datastore, so we can remove the methods that loaded them outside the datastore. The ``waiting_for_join_list`` in federation handler is populated by anything, so can be removed. The ``_get_members_events_txn`` method isn't called from anywhere so can be removed.	2016-06-07 15:07:11 +01:00
Erik Johnston	84379062f9	Fix AS retries, but with correct ordering	2016-06-07 10:24:50 +01:00
Erik Johnston	310197bab5	Fix AS retries	2016-06-07 09:34:50 +01:00
Erik Johnston	e4134c5e13	Merge pull request #841 from matrix-org/erikj/event_counter ... Add metric counter for number of persisted events	2016-06-06 14:17:40 +01:00
Erik Johnston	7aa778fba9	Add metric counter for number of persisted events	2016-06-06 11:58:09 +01:00
Erik Johnston	70aee0717c	Add events to cache when we persist them	2016-06-06 11:34:53 +01:00
Erik Johnston	cffe46408f	Don't rely on options when inserting event into cache	2016-06-03 18:25:21 +01:00
Erik Johnston	10ea3f46ba	Change the way we cache events	2016-06-03 17:57:50 +01:00
Erik Johnston	05e01f21d7	Remove event fetching from DB threads	2016-06-03 17:22:13 +01:00
Mark Haines	eef541a291	Move insert_client_ip to a separate class	2016-06-03 14:42:35 +01:00
Erik Johnston	d06f2a229e	Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room ... Make get_joined_hosts_for_room use get_users_in_room	2016-06-03 11:50:30 +01:00
Erik Johnston	722472b48c	Merge pull request #825 from matrix-org/erikj/cache_push_rules ... Load push rules in storage layer so that they get cached	2016-06-03 11:44:32 +01:00
Mark Haines	b09f348530	Merge pull request #824 from matrix-org/markjh/slaved_presence_store ... Add a slaved store for presence	2016-06-03 11:26:33 +01:00
Erik Johnston	ccb56fc24b	Make get_joined_hosts_for_room use get_users_in_room	2016-06-03 11:20:23 +01:00
Erik Johnston	6a0afa582a	Load push rules in storage layer, so that they get cached	2016-06-03 11:10:00 +01:00
Mark Haines	3ae915b27e	Add a slaved store for presence	2016-06-03 11:05:53 +01:00
Erik Johnston	9c26b390a2	Only get local users	2016-06-03 11:04:31 +01:00
Erik Johnston	065e739d6e	Merge pull request #811 from matrix-org/erikj/state_users_in_room ... Use state to calculate get_users_in_room	2016-06-03 10:58:27 +01:00
Erik Johnston	696d7c5937	Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room ... Add get_users_with_read_receipts_in_room cache	2016-06-03 10:58:24 +01:00
Mark Haines	80f34d7b57	Fix setting the _clock in SQLBaseStore	2016-06-02 15:23:56 +01:00
Erik Johnston	a04a2d043c	Merge pull request #807 from matrix-org/erikj/push_rules_cache ... Ensure we always return boolean in push rules	2016-06-01 18:07:48 +01:00
Erik Johnston	0f06b496d1	Merge pull request #806 from matrix-org/erikj/hash_cache ... Cache get_event_reference_hashes	2016-06-01 18:07:42 +01:00
Erik Johnston	c8285564a3	Use state to calculate get_users_in_room	2016-06-01 15:25:25 +01:00
David Baker	d60eed0710	Limit number of notifications in an email notification	2016-06-01 11:45:43 +01:00
Erik Johnston	43db0d9f6a	Add get_users_with_read_receipts_in_room cache	2016-06-01 10:54:32 +01:00
Erik Johnston	dea9f20f8c	Force boolean	2016-05-31 17:24:30 +01:00
Mark Haines	c8c5bf950a	Fix synapse/storage/schema/delta/30/as_users.py	2016-05-31 17:10:40 +01:00
Erik Johnston	c9ca285d33	Merge pull request #805 from matrix-org/erikj/push_rules_cache ... Fix GET /push_rules	2016-05-31 16:42:21 +01:00
Erik Johnston	1d4ee854e2	Fix typo	2016-05-31 15:45:53 +01:00
Erik Johnston	cca0093fa9	Change fix	2016-05-31 15:44:08 +01:00
Erik Johnston	aefd2d1cbc	Cache get_event_reference_hashes	2016-05-31 15:32:32 +01:00
Erik Johnston	10de8c2631	Merge pull request #804 from matrix-org/erikj/push_rules_cache ... Add caches to bulk_get_push_rules*	2016-05-31 15:04:40 +01:00
Erik Johnston	e5b0bbcd33	Add caches to bulk_get_push_rules*	2016-05-31 13:46:58 +01:00
Erik Johnston	85b992f621	Fix to allow start with postgres	2016-05-27 10:44:44 +01:00
David Baker	37b7e84620	Include the ts the notif was received at	2016-05-24 11:33:32 +01:00
David Baker	a24bc5b2dc	Add GET /notifications API	2016-05-23 18:33:51 +01:00
David Baker	31b5395ab6	Remove debug logging	2016-05-23 16:32:01 +01:00
David Baker	c2da3406fc	Oops, missing comma	2016-05-20 18:03:31 +01:00
David Baker	ccffb0965d	Remove stale line	2016-05-20 17:59:10 +01:00
David Baker	18d68bfee4	Handle empty events table	2016-05-20 17:58:09 +01:00
David Baker	d4503e25ed	Make deleting push actions more efficient ... There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.	2016-05-20 17:56:10 +01:00
David Baker	149fa411e2	Only delete push actions after 30 days	2016-05-20 15:25:12 +01:00
Erik Johnston	42368ea8db	Add desc to get_presence_for_users	2016-05-18 11:38:10 +01:00
Mark Haines	6a30a0bfd3	Move the functions for parsing app service config	2016-05-17 11:28:58 +01:00
Mark Haines	816df9f267	get_room_members is unused now	2016-05-16 19:51:43 +01:00
Mark Haines	1a3a2002ff	Spell "domain" correctly ... s/domian/domain/g	2016-05-16 19:17:23 +01:00
Mark Haines	b7381d5338	Allow receipts for events we haven't seen in the db	2016-05-13 15:46:41 +01:00
Erik Johnston	ba5c616ff4	Merge pull request #778 from matrix-org/erikj/add_pusher ... Fixup add_pusher	2016-05-13 14:43:23 +01:00
Erik Johnston	0c11c1be88	Spelling	2016-05-13 14:42:25 +01:00
Erik Johnston	e00e8f2166	Merge pull request #769 from matrix-org/erikj/push_actions_delete ... Delete old pushers	2016-05-13 14:41:36 +01:00
Erik Johnston	fd8e921b6e	Merge pull request #779 from matrix-org/erikj/receipts ... Use tree cache for get_linearized_receipts_for_room	2016-05-13 14:41:21 +01:00
Erik Johnston	c9aff0736c	Remove topics table	2016-05-13 14:40:38 +01:00
Mark Haines	9295fa30a8	Annotate the removed indicies with why they were removed.	2016-05-13 14:16:57 +01:00
Erik Johnston	5e50058473	Remove unused indices ... This includes removing both unused indices and indices that are subsets of other indices.	2016-05-13 13:28:07 +01:00
Erik Johnston	6da7f39d95	Use tree cache for get_linearized_receipts_for_room	2016-05-13 11:41:23 +01:00
Erik Johnston	13d37c3c56	Fixup add_pusher	2016-05-13 11:25:02 +01:00
Matrix	3b1930e8ec	unbreak schema	2016-05-10 16:42:37 +01:00
Matthew Hodgson	fe97b81c09	Merge pull request #759 from matrix-org/dbkr/email_notifs ... Send email notifications for missed messages	2016-05-10 16:30:05 +02:00
David Baker	997db04648	Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs	2016-05-10 14:40:19 +02:00
Erik Johnston	c9eb6dfc1b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user	2016-05-09 13:21:06 +01:00
Erik Johnston	3f84da139c	Merge pull request #773 from matrix-org/erikj/get_domian_from_id ... Add and use get_domain_from_id	2016-05-09 13:21:00 +01:00
Erik Johnston	def64d6ef3	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user	2016-05-09 13:05:09 +01:00
Erik Johnston	8715731559	Merge pull request #772 from matrix-org/erikj/get_user_cache ... Add cache to get_user_by_id	2016-05-09 11:12:11 +01:00
Erik Johnston	08dfa8eee2	Add and use get_domian_from_id	2016-05-09 10:36:03 +01:00
Erik Johnston	4ea762c1a2	Add cache to get_user_by_id	2016-05-09 10:08:21 +01:00
Erik Johnston	012cb5416c	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/push_actions_delete	2016-05-06 15:59:20 +01:00
Erik Johnston	fcb2c3f0db	Remove unused import	2016-05-06 15:47:40 +01:00
Erik Johnston	fd85b167ec	Pull loop one level up	2016-05-06 15:38:42 +01:00
Erik Johnston	b6e0be701e	Queue events for persistence	2016-05-06 14:31:44 +01:00
Erik Johnston	d13459636f	Pull prev txn from in memory	2016-05-06 11:30:55 +01:00
Erik Johnston	1d275dba69	Don't needlessly enter transaction	2016-05-06 11:25:58 +01:00
Erik Johnston	56b5e83e36	Reduce database inserts when sending transactions	2016-05-06 11:20:18 +01:00
Mark Haines	9c272da05f	Add an openidish mechanism for proving to third parties that you own a given user_id	2016-05-05 13:42:44 +01:00
Erik Johnston	1f0f5ffa1e	Add bulk fetch storage API	2016-05-05 10:03:15 +01:00
Erik Johnston	fee1118a20	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user	2016-05-04 19:08:27 +01:00
Erik Johnston	fcd1eb642d	Add primary key	2016-05-04 16:51:51 +01:00
Erik Johnston	8e6a163f27	Add timestamp and auto incrementing ID	2016-05-04 15:19:12 +01:00
David Baker	80be396464	Correct SQL statement for postgres ... In standard sql, join binds tighter than comma, so we were joining on the wrong table. Postgres follows the standard (apparently).	2016-05-04 13:19:59 +01:00
Erik Johnston	984d4a2c0f	Add /report endpoint	2016-05-04 11:28:10 +01:00
Erik Johnston	a438a6d2bc	Implement basic ignore user	2016-05-04 10:16:46 +01:00
Erik Johnston	183f23f10d	Delete old pushers	2016-05-03 14:22:33 +01:00
David Baker	35b7b8e4bc	Remove unused function	2016-04-29 20:10:34 +01:00
David Baker	b0a1036d93	Use explicit join	2016-04-29 19:28:56 +01:00
David Baker	8f99cd5996	Oops, actually specify the user id	2016-04-29 19:27:03 +01:00
David Baker	60f44c098d	Remove unnecessary if	2016-04-29 19:17:10 +01:00
David Baker	50ad8005e4	Put spaces at start of line	2016-04-29 19:16:15 +01:00
David Baker	acded821c4	Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs	2016-04-29 10:05:20 +01:00
Mark Haines	871357d539	Check that somethign has happend before running the selects	2016-04-27 11:54:13 +01:00
Erik Johnston	183cacac90	Simplify query and handle finishing correctly	2016-04-22 10:01:57 +01:00
Erik Johnston	ae571810f2	Order NULLs first	2016-04-21 18:14:18 +01:00
Erik Johnston	3ddbb1687c	Fix query	2016-04-21 18:02:36 +01:00
Erik Johnston	8fae3d7b1e	Use special UPDATE syntax	2016-04-21 18:01:49 +01:00
Erik Johnston	b57dcb4b51	Typo	2016-04-21 17:49:00 +01:00
Erik Johnston	26db18bc90	Need to do _background_update_progress_txn in actual transaction	2016-04-21 17:45:56 +01:00
Erik Johnston	e395eb1108	Update progress when creating index	2016-04-21 17:39:24 +01:00
Erik Johnston	3b0fa77f50	Fix SQL statement	2016-04-21 17:37:42 +01:00
Erik Johnston	129e403487	Create index must be on a conn	2016-04-21 17:35:51 +01:00
Erik Johnston	51bb339ab2	Create index concurrently	2016-04-21 17:16:11 +01:00
Erik Johnston	b743c1237e	Add missing run_upgrade	2016-04-21 17:12:04 +01:00
Erik Johnston	c877f0f034	Optimise event_search in postgres	2016-04-21 16:56:14 +01:00
David Baker	f63bd4ff47	Send a rather basic email notif ... Also pep8 fixes	2016-04-20 13:02:01 +01:00
Erik Johnston	61c7edfd34	Add cache to _get_state_groups_from_groups	2016-04-19 17:22:03 +01:00
David Baker	07d765209d	First bits of emailpusher ... Mostly logic of when to send an email	2016-04-19 14:24:36 +01:00
Erik Johnston	d213d69fe3	Add desc arg	2016-04-14 11:36:23 +01:00
Erik Johnston	96bcfb29c7	Add index	2016-04-14 11:26:33 +01:00
Erik Johnston	a2546b9082	Fix query for get_unread_push_actions_for_user_in_range	2016-04-14 11:08:31 +01:00
David Baker	767fc0b739	pep8	2016-04-13 14:23:27 +01:00
David Baker	54d08c8868	Only return unread notifications ... Make get_unread_push_actions_for_user_in_range only return unread event actions, being more true to its name. Done in two separate sql queries to get actions after a read receipt and those in a room wiht no receipt at all. SQL queries by Erik.	2016-04-13 14:16:45 +01:00
David Baker	d33d623f0d	Merge pull request #716 from matrix-org/dbkr/get_pushers ... Add get endpoint for pushers	2016-04-12 14:40:37 +01:00
David Baker	7984ffdc6a	Unneccessarywhitespaceisunnecessary	2016-04-12 13:55:57 +01:00
David Baker	c1267d04c5	Oops, forgot the desc.	2016-04-12 13:55:32 +01:00
David Baker	44891b4a0a	Tidy up get_pusher functions ... Decodes pushers rows on the main thread rather than the db thread and uses _simple_select_list. Also do the same to the function I copied and factor out the duplication into a helper function.	2016-04-12 13:47:17 +01:00
David Baker	7b39bcdaae	Mis-named function	2016-04-12 13:35:08 +01:00
Erik Johnston	762ada1e07	Add back backfilled parameter that was removed	2016-04-12 11:58:04 +01:00
Erik Johnston	cccf86dd05	Check if we've already backfilled events	2016-04-12 11:19:32 +01:00
David Baker	8a76094965	Add get endpoint for pushers ... As per https://github.com/matrix-org/matrix-doc/pull/308	2016-04-11 18:00:03 +01:00
David Baker	2547dffccc	Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions ... Change pushers to use the event_actions table	2016-04-11 12:58:55 +01:00
Matthew Hodgson	4bd3d25218	Merge pull request #688 from matrix-org/matthew/preview_urls ... URL previewing support	2016-04-11 10:40:29 +01:00
Matthew Hodgson	6ff7a79308	move local_media_repository_url_cache.sql to schema v31	2016-04-08 19:09:02 +01:00
Matthew Hodgson	2460d904bd	fix error checking for new SQL	2016-04-08 19:04:29 +01:00
Matthew Hodgson	fb83f6a1fc	fix SQL based on PR feedback	2016-04-08 18:55:38 +01:00
David Baker	ed3979df5f	Fix invite pushes ... * If the event is an invite event, add the invitee to list of user we run push rules for (if they have a pusher etc) * Move invite_for_me to be higher prio than member events otherwise member events matches them * Spell override right	2016-04-08 15:29:59 +01:00
Mark Haines	7e2f971c08	Remove some unused functions (#711) ... * Remove some unused functions * get_room_events_stream is only used in tests * is_exclusive_room might actually be something we want	2016-04-08 14:01:56 +01:00
David Baker	d9f38561c8	Literally a dictionary	2016-04-07 17:45:01 +01:00
David Baker	4836864f56	generate id in the main thread	2016-04-07 17:38:48 +01:00
David Baker	3fb35cbd6f	Oops, inequality fail	2016-04-07 17:33:37 +01:00
Matthew Hodgson	d6e7333ae4	Merge branch 'develop' into matthew/preview_urls	2016-04-07 17:26:44 +01:00
Mark Haines	ceb599e789	Add tests for redactions	2016-04-07 16:52:07 +01:00
David Baker	05d044aac3	pep8	2016-04-07 16:45:38 +01:00
David Baker	2d5c693fd3	Fix port script for changes merged from develop	2016-04-07 16:43:54 +01:00
David Baker	9c99ab4572	Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions	2016-04-07 16:35:22 +01:00
David Baker	92e3071623	Send badge count pushes. ... Also fix bugs with retrying.	2016-04-07 15:39:53 +01:00
David Baker	0fd1cd2400	pep8	2016-04-06 16:50:47 +01:00
Mark Haines	e815763b7f	Merge pull request #697 from matrix-org/markjh/slaveI ... Add a slaved events store class	2016-04-06 16:19:25 +01:00
David Baker	7e2c89a37f	Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do: ... * Make badges work again * Remove old, unused code	2016-04-06 15:42:15 +01:00
Erik Johnston	b713934b2e	Merge pull request #698 from matrix-org/erikj/port_script_fix ... Don't require config to create database	2016-04-06 14:32:45 +01:00
Mark Haines	75fb9ac1be	Add a slaved events store class ... Add a test to check that get_room_names_and_aliases does the same thing on both the master and on the slave data store.	2016-04-06 14:18:35 +01:00
Erik Johnston	8aab9d87fa	Don't require config to create database	2016-04-06 14:15:45 +01:00
Mark Haines	7d11f825aa	Merge pull request #694 from matrix-org/markjh/caches ... Move _get_cache_dict into the SQLBaseStore	2016-04-06 13:21:25 +01:00
Mark Haines	87f2dec8d4	Make the cache objects be per instance rather than being global	2016-04-06 13:08:05 +01:00
Mark Haines	a1e0d316ea	Move _get_cache_dict into the SQLBaseStore	2016-04-06 13:05:19 +01:00
Erik Johnston	b29f98377d	Merge pull request #691 from matrix-org/erikj/member ... Fix stuck invites	2016-04-05 12:44:39 +01:00
Erik Johnston	df727f2126	Fix stuck invites ... If rejecting a remote invite fails with an error response don't fail the entire request; instead mark the invite as locally rejected. This fixes the bug where users can get stuck invites which they can neither accept nor reject.	2016-04-05 11:13:24 +01:00
Erik Johnston	7a77f8b6d5	Merge pull request #690 from matrix-org/erikj/member ... Store invites in a separate table.	2016-04-05 09:12:27 +01:00
Erik Johnston	0c53d750e7	Docs and indents	2016-04-04 18:02:48 +01:00
Erik Johnston	92ab45a330	Add upgrade path, rename table	2016-04-04 17:07:43 +01:00
Erik Johnston	3d76b7cb2b	Store invites in a separate table.	2016-04-04 16:30:15 +01:00
Erik Johnston	bf14883a04	Merge pull request #689 from matrix-org/erikj/member ... Do checks for memberships before creating events	2016-04-04 11:56:40 +01:00
Matthew Hodgson	9f7dc2bef7	Merge branch 'develop' into matthew/preview_urls	2016-04-04 00:38:21 +01:00
Matthew Hodgson	8b98a7e8c3	pep8	2016-04-03 12:56:29 +01:00
Matthew Hodgson	7426c86eb8	add a persistent cache of URL lookups, and fix up the in-memory one to work	2016-04-03 00:31:57 +01:00
Erik Johnston	d76d89323c	Use computed prev event ids	2016-04-01 17:39:32 +01:00
Mark Haines	89e6839a48	Merge pull request #686 from matrix-org/markjh/doc_strings ... Use google style doc strings.	2016-04-01 16:20:09 +01:00
Mark Haines	2a37467fa1	Use google style doc strings. ... pycharm supports them so there is no need to use the other format. Might as well convert the existing strings to reduce the risk of people accidentally cargo culting the wrong doc string format.	2016-04-01 16:12:07 +01:00
Mark Haines	f2b916534b	Merge pull request #684 from matrix-org/markjh/backfill_id_gen ... Use a stream id generator for backfilled ids	2016-04-01 15:13:14 +01:00
Mark Haines	9bc5b4c663	Assert that the step != 0	2016-04-01 15:08:20 +01:00
Mark Haines	35b5c4ba1b	use google style doc strings	2016-04-01 15:07:01 +01:00
Mark Haines	a2866e2e6a	Rename direction to step, apply checks consistently	2016-04-01 13:50:54 +01:00
Mark Haines	e36bfbab38	Use a stream id generator for backfilled ids	2016-04-01 13:29:05 +01:00
Mark Haines	7753fc6570	Fix the invalidation of the names and aliases cache	2016-04-01 10:34:51 +01:00
Mark Haines	5d06929169	Move the check for backfilled outside the for loop	2016-03-31 15:09:09 +01:00
Mark Haines	76503f95ed	Remove the is_new_state argument to persist event. ... Move the checks for whether an event is new state inside persist event itself. This was harder than expected because there wasn't enough information passed to persist event to correctly handle invites from remote servers for new rooms.	2016-03-31 15:00:42 +01:00
Mark Haines	62e395f0e3	Merge pull request #676 from matrix-org/markjh/replicate_stateIII ... Add replication streams for ex outliers and current state resets	2016-03-31 11:20:57 +01:00
Mark Haines	2ec5426035	Use a namedtuple rather than tuple unpacking	2016-03-31 10:33:02 +01:00
Mark Haines	1fbb094c6f	Add replication streams for ex outliers and current state resets	2016-03-30 17:19:56 +01:00
Mark Haines	8b8052909f	return the state_group for backfill	2016-03-30 16:20:07 +01:00
Mark Haines	61407986b4	Add a entry to current_state_resets table when the current state is reset	2016-03-30 16:18:46 +01:00
Mark Haines	31a9eceda5	Add a replication stream for state groups	2016-03-30 16:01:58 +01:00
Mark Haines	1e25f62ee6	Use a stream id generator to assign state group ids	2016-03-30 12:55:02 +01:00
Erik Johnston	2f0180b09e	Don't bother interning keys that are already interned	2016-03-23 16:29:46 +00:00
Erik Johnston	acdfef7b14	Intern all the things	2016-03-23 16:25:54 +00:00
Erik Johnston	75daede92f	String intern	2016-03-23 14:53:53 +00:00
Erik Johnston	fbdeb1778d	Merge pull request #664 from matrix-org/erikj/public_room_list ... Don't require alias in public room list.	2016-03-23 14:42:01 +00:00
Erik Johnston	b275765545	Comment about weird SQL	2016-03-23 14:15:32 +00:00
Erik Johnston	0c1a27b787	SQLite and postgres doesn't share a true literal	2016-03-23 14:10:49 +00:00
Erik Johnston	84afeb41f3	Ensure all old public rooms have aliases	2016-03-23 13:59:34 +00:00
Erik Johnston	2749da542c	Merge pull request #663 from matrix-org/erikj/invite_for_user ... Make get_invites return RoomsForUser	2016-03-23 13:19:26 +00:00
Erik Johnston	e14baa7a3b	Merge pull request #665 from matrix-org/erikj/dont_cache_events ... Only cache events in the event cache	2016-03-23 13:19:12 +00:00
Erik Johnston	d87a846ebc	Don't cache events in get_recent_events_for_room	2016-03-23 11:42:50 +00:00
Erik Johnston	8b0dfc9fc4	Don't cache events in get_current_state_for_key	2016-03-23 11:42:17 +00:00
Erik Johnston	b6507869cd	Make get_invites return RoomsForUser	2016-03-23 10:32:10 +00:00
Erik Johnston	99f929f36b	Make StateHandler._state_cache only store event_ids.	2016-03-22 16:06:04 +00:00
Erik Johnston	6cf0ba1466	Bump get_unread_event_push_actions_by_room_for_user cache	2016-03-22 14:18:21 +00:00
Erik Johnston	76d18a5776	Bump get_aliases_for_room cache	2016-03-22 14:08:13 +00:00
Erik Johnston	fa2f96c2e3	Merge pull request #660 from matrix-org/erikj/state_cache ... Don't cache events in _state_group_cache	2016-03-22 13:12:06 +00:00
Erik Johnston	f93304e77f	Merge pull request #659 from matrix-org/erikj/state_cache_factor ... Make stateGroupCache honour CACHE_SIZE_FACTOR	2016-03-22 13:12:01 +00:00
Erik Johnston	2c86187a1b	Don't cache events in _state_group_cache ... Instead, simply cache the event ids, relying on the event cache to cache the actual events. The problem was that while the state groups cache was limited in the number of groups it could hold, each individual group could consist of thousands of events.	2016-03-22 12:00:09 +00:00
Erik Johnston	d6ac752538	Merge pull request #657 from matrix-org/erikj/roomlist ... Add published room list edit API	2016-03-22 11:57:39 +00:00
Erik Johnston	b591277620	Make stateGroupCache honour CACHE_SIZE_FACTOR	2016-03-22 10:32:50 +00:00
Mark Haines	5244c0b48e	Remove unused backfilled parameter from persist_event	2016-03-21 18:06:08 +00:00
Erik Johnston	3e7fac0d56	Add published room list edit API	2016-03-21 15:06:07 +00:00
Erik Johnston	67ed8065db	Dedupe requested event list in _get_events	2016-03-18 14:31:31 +00:00
Mark Haines	673c96ce97	Remove dead code left over from presence changes	2016-03-17 11:03:47 +00:00
Mark Haines	ba660ecde2	Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id)	2016-03-16 10:35:00 +00:00
Mark Haines	ee32d622ce	Fix a couple of errors when deleting pushers	2016-03-15 17:47:36 +00:00
Mark Haines	b6e8420aee	Add replication stream for pushers	2016-03-15 17:33:10 +00:00
Erik Johnston	15122da0e2	Thats not how transactions work.	2016-03-11 16:45:27 +00:00
Erik Johnston	b13035cc91	Implement logout	2016-03-11 16:27:50 +00:00
Erik Johnston	c081228439	Fix SQL statement	2016-03-11 15:09:17 +00:00
Erik Johnston	b5afe6bc38	Merge pull request #641 from matrix-org/dbkr/fix_change_password ... Fix logout on password change	2016-03-11 14:48:20 +00:00
David Baker	2dee03aee5	more pep8	2016-03-11 14:38:23 +00:00
David Baker	af59826a2f	Make select more sensible when dseleting access tokens, rename pusher deletion to match access token deletion and make exception arg optional.	2016-03-11 14:34:09 +00:00
David Baker	f523177850	Delete old, unused methods and rename new one to just be user_delete_access_tokens with an except_token_ids argument doing what it says on the tin.	2016-03-11 14:29:01 +00:00
David Baker	57c444b3ad	Dear PyCharm, please indent sensibly for me. Thx.	2016-03-11 14:25:05 +00:00
Erik Johnston	58443a022d	Merge pull request #635 from matrix-org/erikj/sync_order ... Use topological orders for initial sync timeline	2016-03-11 13:17:49 +00:00
David Baker	aa11db5f11	Fix cache invalidation so deleting access tokens (which we did when changing password) actually takes effect without HS restart. Reinstate the code to avoid logging out the session that changed the password, removed in 415c2f0549	2016-03-11 13:14:18 +00:00
Daniel Wagner-Hall	465605d616	Store appservice ID on register	2016-03-10 15:58:22 +00:00
Erik Johnston	9669a99d1a	Update users table in a batched manner	2016-03-10 15:12:19 +00:00
Erik Johnston	8a88684736	Add comment	2016-03-09 16:51:22 +00:00
Erik Johnston	af2fe6110c	Return the correct token form	2016-03-09 16:11:53 +00:00
Erik Johnston	3ecaabc7fd	Use topological orders for initial sync timeline	2016-03-09 15:45:34 +00:00
Erik Johnston	158a322e82	Ensure integer is an integer	2016-03-09 10:20:48 +00:00
Mark Haines	5f5817ab05	Merge pull request #632 from matrix-org/markjh/py3v2 ... Fix relative imports so they work in both py3 and py27	2016-03-08 18:11:10 +00:00
Daniel Wagner-Hall	3b97797c8d	Merge branch 'develop' into daniel/ick	2016-03-08 17:35:09 +00:00
Daniel Wagner-Hall	edca2d9891	Idempotent-ise schema update script ... If any ASes don't have an ID, the schema will fail, and then it will error when trying to add the column again.	2016-03-08 17:32:29 +00:00
Mark Haines	7076082ae6	Fix relative imports so they work in both py3 and py27	2016-03-08 11:45:50 +00:00
Mark Haines	9848b54cac	Prefill from the correct stream	2016-03-04 16:20:22 +00:00
Mark Haines	deda48068c	prefill the push rules stream change cache	2016-03-04 16:19:42 +00:00
Mark Haines	ebcbb23226	s/stream_ordering/event_stream_ordering/ in push	2016-03-04 16:15:23 +00:00
Mark Haines	7e9fc9b6af	/FNARG/d	2016-03-04 15:54:09 +00:00
Mark Haines	ec7460b4f2	Merge branch 'develop' into markjh/pushrule_stream	2016-03-04 14:44:34 +00:00
Mark Haines	1b4f4a936f	Hook up the push rules stream to account_data in /sync	2016-03-04 14:44:01 +00:00
Erik Johnston	dd463e246d	Merge pull request #614 from matrix-org/erikj/alias_delete ... Allow alias creators to delete aliases	2016-03-04 09:02:33 +00:00
Daniel Wagner-Hall	95481e7ba7	Merge pull request #571 from matrix-org/daniel/asids ... Mark AS users with their AS's ID	2016-03-03 13:56:28 +00:00
Mark Haines	2223204eba	Hook push rules up to the replication API	2016-03-02 17:26:20 +00:00
Mark Haines	a1cf9e3bf3	Add a stream for push rule updates	2016-03-01 18:16:37 +00:00
Mark Haines	a612ce6659	Merge pull request #489 from matrix-org/markjh/replication ... Add a /replication API for extracting the updates that happened on synapse.	2016-03-01 15:08:24 +00:00
Mark Haines	60a0f81c7a	Add a /replication API for extracting the updates that happened on ... synapse This is necessary for replicating the data in synapse to be visible to a separate service because presence and typing notifications aren't stored in a database so won't be visible to another process. This API can be used to either get the raw data by requesting the tables themselves or to just receive notifications for updates by following the streams meta-stream. Returns updates for each table requested a JSON array of arrays with a row for each row in the table. Each table is prefixed by a header row with the: name of the table, current stream_id position for the table, number of rows, number of columns and the names of the columns. This is followed by the rows that have been added to the server since the requester last asked. The API has a timeout and is hooked up to the notifier so that a slave can long poll for updates.	2016-03-01 14:49:41 +00:00
Erik Johnston	f9af8962f8	Allow alias creators to delete aliases	2016-03-01 14:46:31 +00:00
Mark Haines	54172924c8	Load the current id in the IdGenerator constructor ... Rather than loading them lazily. This allows us to remove all the yield statements and spurious arguments for the get_next methods. It also allows us to replace all instances of get_next_txn with get_next since get_next no longer needs to access the db.	2016-03-01 14:32:56 +00:00
Mark Haines	de27f7fc79	Add support for changing the actions for default rules ... See matrix-org/matrix-doc#283 Works by adding dummy rules to the push rules table with a negative priority class and then using those rules to clobber the default rule actions when adding the default rules in ``list_with_base_rules``	2016-02-26 14:28:19 +00:00
Daniel Wagner-Hall	33300673b7	Generate guest access token on 3pid invites ... This means that following the same link across multiple sessions or devices can re-use the same guest account. Note that this is somewhat of an abuse vector; we can't throw up captchas on this flow, so this is a way of registering ephemeral accounts for spam, whose sign-up we don't rate limit.	2016-02-24 14:41:25 +00:00
Erik Johnston	6451fcd085	Create a new stream_id per presence update	2016-02-23 15:51:39 +00:00
Erik Johnston	e5ad2e5267	Merge pull request #582 from matrix-org/erikj/presence ... Rewrite presence for performance.	2016-02-19 09:37:50 +00:00
Erik Johnston	e6c5e3f28a	Close cursor	2016-02-18 16:39:28 +00:00
Erik Johnston	42109a62a4	Remove unused param from get_max_token	2016-02-18 16:37:28 +00:00
Mark Haines	b9977ea667	Remove dead code for setting device specific rules. ... It wasn't possible to hit the code from the API because of a typo in parsing the request path. Since no-one was using the feature we might as well remove the dead code.	2016-02-18 16:05:13 +00:00
Erik Johnston	112283e230	Prefix TS fields with _ts	2016-02-18 10:11:43 +00:00
Daniel Wagner-Hall	7e90fb6a57	Merge branch 'develop' into daniel/roomcleanupincremental ... Conflicts: synapse/rest/client/v1/room.py	2016-02-17 15:53:59 +00:00
Erik Johnston	e5999bfb1a	Initial cut	2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall	a4e278bfe7	Respond to federated invite with non-empty context ... Currently, we magically perform an extra database hit to find the inviter, and use this to guess where we should send the event. Instead, fill in a valid context, so that other callers relying on the context actually have one.	2016-02-17 15:25:12 +00:00
Mark Haines	a9c9868957	Make adding push rules idempotent ... Also remove the **kwargs from the add_push_rule method. Fixes https://matrix.org/jira/browse/SYN-391	2016-02-16 15:53:38 +00:00
Daniel Wagner-Hall	763360594d	Mark AS users with their AS's ID	2016-02-11 17:26:42 +00:00
Erik Johnston	ce14c7a995	Fix SYN-627, events are in incorrect room in /sync	2016-02-11 15:02:56 +00:00
Erik Johnston	0eff740523	Return events in correct order for /events	2016-02-11 10:07:27 +00:00
Erik Johnston	5189bfdef4	Batch fetch _get_state_groups_from_groups	2016-02-10 13:24:42 +00:00
Erik Johnston	24f00a6c33	Use _simple_select_many for _get_state_group_for_events	2016-02-10 12:57:50 +00:00
Erik Johnston	f7ef5c1d57	Merge pull request #568 from matrix-org/erikj/unread_notif ... Atomically persit push actions when we persist the event	2016-02-10 11:25:32 +00:00
Erik Johnston	0214745239	Rename functions	2016-02-10 11:09:56 +00:00
Erik Johnston	7b0d846407	Atomically persit push actions when we persist the event	2016-02-09 16:19:15 +00:00
Erik Johnston	78d6c1b5be	Change a log from debug to info	2016-02-09 14:44:12 +00:00
Erik Johnston	70a8608749	Invalidate get_last_receipt_event_id_for_user cache	2016-02-09 14:27:29 +00:00
Erik Johnston	eff12e838c	Don't load all ephemeral state for a room on every sync	2016-02-09 13:55:59 +00:00
Erik Johnston	2c1fbea531	Fix up logcontexts	2016-02-08 14:26:45 +00:00
Daniel Wagner-Hall	79a1c0574b	Allocate guest user IDs numericcally ... The current random IDs are ugly and confusing when presented in UIs. This makes them prettier and easier to read. Also, disable non-automated registration of numeric IDs so that we don't need to worry so much about people carving out our automated address space and us needing to keep retrying ID registration.	2016-02-05 11:22:30 +00:00
Erik Johnston	4d36e73230	Actually return something sensible	2016-02-03 16:35:00 +00:00
Erik Johnston	aa4af94c69	We return dicts now.	2016-02-03 16:29:32 +00:00
Erik Johnston	b84d59c5f0	Add descriptions	2016-02-03 16:22:35 +00:00
Erik Johnston	8450114098	Merge pull request #554 from matrix-org/erikj/event_push ... Change event_push_actions_rm_tokens schema	2016-02-03 15:02:47 +00:00
Erik Johnston	f8aae79a72	Simplify get_rooms	2016-02-03 13:24:35 +00:00
Erik Johnston	6f52e90065	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_room_fix	2016-02-03 11:06:29 +00:00
Erik Johnston	771528ab13	Change event_push_actions_rm_tokens schema	2016-02-03 10:50:49 +00:00
Erik Johnston	b32121a5d1	Unused import	2016-02-03 10:30:56 +00:00
Daniel Wagner-Hall	d83d004ccd	Fix flake8 warnings for new flake8	2016-02-02 17:18:50 +00:00
Erik Johnston	8a391e33ae	s/get_room_changes_for_user/get_membership_changes_for_user/	2016-02-02 16:12:10 +00:00
Erik Johnston	477b1ed6cf	Fetch events in a separate transaction. ... This has a couple of benefits: - It reduces the time of transactions, allowing other database requests to run. - Fetching events is given a dedicated database thread, and so can't starve other database requests.	2016-02-02 15:58:14 +00:00
Erik Johnston	65e92eca49	Change the way we do public room list fetching	2016-02-02 15:21:10 +00:00
Erik Johnston	89b40b225c	Order things correctly	2016-02-01 16:32:46 +00:00
Erik Johnston	4bf448be25	Switch over /events to use per room caches	2016-02-01 16:26:51 +00:00
Erik Johnston	2d3837bec7	Merge pull request #543 from matrix-org/erikj/sync ... Cache get_room_changes_for_user	2016-02-01 15:05:06 +00:00
Erik Johnston	ceb6b8680a	Only use room_ids if in get_room_events_stream if is_guest	2016-02-01 10:33:52 +00:00
Erik Johnston	25c311eaf6	Cache get_room_changes_for_user	2016-01-29 16:52:48 +00:00
Erik Johnston	cc9c97e0dc	Invalidate _account_data_stream_cache correctly	2016-01-29 16:41:51 +00:00
Erik Johnston	8da95b6f1b	Comment. Remove superfluous order by	2016-01-29 15:39:17 +00:00
Erik Johnston	b5dbced938	Don't prefill account data	2016-01-29 14:53:59 +00:00
Erik Johnston	3d60686c0c	Actually use cache	2016-01-29 14:49:11 +00:00
Erik Johnston	45488e0ffa	Max is not a function	2016-01-29 14:42:01 +00:00
Erik Johnston	f67d60496a	Convert param style	2016-01-29 14:41:16 +00:00
Erik Johnston	18579534ea	Prefill stream change caches	2016-01-29 14:37:59 +00:00
Erik Johnston	ebc5f00efe	Bump AccountDataAndTagsChangeCache size	2016-01-29 13:37:40 +00:00
Erik Johnston	b18114e19e	Merge pull request #536 from matrix-org/erikj/sync ... Make /sync "better".	2016-01-29 13:04:51 +00:00
Erik Johnston	467c27a1f9	Amalgamate tags and account data stream caches	2016-01-28 18:20:00 +00:00
Erik Johnston	03b2c2577c	Don't use defer.returnValue	2016-01-28 17:29:24 +00:00
Erik Johnston	8fe8951a8d	Cache filters	2016-01-28 17:09:09 +00:00
Erik Johnston	fdca8ec418	Add events index	2016-01-28 16:41:59 +00:00
Erik Johnston	45cf827c8f	Change name and doc has_entity_changed	2016-01-28 16:39:18 +00:00
Erik Johnston	00cb3eb24b	Cache tags and account data	2016-01-28 16:37:41 +00:00
Erik Johnston	c23a8c7833	Ensure keys to RoomStreamChangeCache are ints	2016-01-28 15:55:26 +00:00
Erik Johnston	e1941442d4	Invalidate caches properly. Remove unused arg	2016-01-28 15:02:41 +00:00
Erik Johnston	19fd425928	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync	2016-01-28 14:46:08 +00:00
Erik Johnston	167d1df699	Merge pull request #534 from matrix-org/erikj/setup ... Add a Homeserver.setup method	2016-01-28 14:45:36 +00:00
Erik Johnston	7ed2bbeb11	Clean up a bit. Add comment	2016-01-28 14:32:05 +00:00
Erik Johnston	4e7948b47a	Allow paginating backwards from stream token	2016-01-28 11:52:34 +00:00
Erik Johnston	ba8931829b	Return correct type of token	2016-01-28 11:34:17 +00:00
Erik Johnston	61eaa6ec64	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync	2016-01-28 10:50:57 +00:00
Erik Johnston	c5e7c0e436	Up get_rooms_for_user cache size	2016-01-28 09:58:45 +00:00
Erik Johnston	e7febf4fbb	PEP8	2016-01-27 17:33:27 +00:00
Erik Johnston	aca3193efb	Use the same path for incremental with gap or without gap	2016-01-27 17:33:27 +00:00
Erik Johnston	b97f6626b6	Add cache to room stream	2016-01-27 17:33:26 +00:00
Erik Johnston	a955cbfa49	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/setup	2016-01-27 17:22:35 +00:00
Mark Haines	8c94833b72	Fix adding push rules relative to other rules	2016-01-27 10:24:20 +00:00
Erik Johnston	87f9477b10	Add a Homeserver.setup method. ... This is for setting up dependencies that require work on startup. This is useful for the DataStore that wants to read a bunch from the database before initiliazing.	2016-01-26 15:51:06 +00:00
Erik Johnston	8431f62ebb	Merge pull request #525 from matrix-org/erikj/select_many ... Implement a `_simple_select_many_batch`	2016-01-25 16:30:36 +00:00
Erik Johnston	f091b73e69	Merge pull request #527 from matrix-org/erikj/push_cache ... Push: Use storage apis that are cached	2016-01-25 16:16:34 +00:00
Erik Johnston	aea5da0ef6	Guard against empty iterables	2016-01-25 15:59:29 +00:00
Erik Johnston	1ebf5e3d03	Correct docstring	2016-01-25 15:53:36 +00:00
Erik Johnston	86896408b0	Add index to event_push_actions	2016-01-25 15:30:32 +00:00
Erik Johnston	53cb173663	Push: Use storage apis that are cached	2016-01-25 13:55:18 +00:00
Erik Johnston	ddd25def01	Implement a _simple_select_many_batch	2016-01-25 13:36:02 +00:00
David Baker	52bdd1b834	Overlay the push_rules_enabled map for users, otherwise they won't be able to disable server default rules.	2016-01-22 14:58:19 +00:00
David Baker	10f76dc5da	Make LRU cache not default to treecache & add options to use it	2016-01-22 12:10:33 +00:00
David Baker	42eae4634f	Use new invalidate_many cache invalidation to invalidate the event_push_actions cache appropriately.	2016-01-22 11:22:48 +00:00
Erik Johnston	8f66fe6392	Cache get_unread_event_push_actions_by_room_for_user	2016-01-21 15:02:07 +00:00
David Baker	367cfab4e6	peppate	2016-01-20 16:05:09 +00:00
David Baker	d4315bbf6b	Add index by user id on receipts_linearized	2016-01-20 15:33:27 +00:00
David Baker	3fa344c037	Add storage function to get all receipts for a user. Also add some cache invalidation to the receipts storage because there wasn't any, and remove a method that was unused.	2016-01-20 15:30:31 +00:00
Erik Johnston	5a7d1ecffc	Add regex cache. Only caculate push actions for users that have sent read receipts, and are on that server	2016-01-19 16:01:05 +00:00
Erik Johnston	3adcc4c86a	Return highlight_count in /sync	2016-01-19 11:35:50 +00:00
Erik Johnston	5cd2126a6a	Remove dead code	2016-01-18 16:49:46 +00:00
Erik Johnston	2c176e02ae	Make unit tests work	2016-01-18 14:48:50 +00:00
Erik Johnston	f59b564507	Make notifications go quicker	2016-01-18 14:48:29 +00:00
Erik Johnston	cc66a9a5e3	Allow filtering events for multiple users at once	2016-01-18 14:43:50 +00:00
Daniel Wagner-Hall	2680043bc6	Require ID and as_token be unique for ASs ... Defaults ID to as_token if not specified. This will change when IDs are fully supported.	2016-01-14 14:34:01 +00:00
Daniel Wagner-Hall	27927463a1	Merge pull request #494 from matrix-org/daniel/2 ... Don't start server if ASes are invalidly configured	2016-01-14 11:06:19 +00:00
Daniel Wagner-Hall	fcb6df45e5	Merge pull request #493 from matrix-org/daniel/1 ... Delete unused code	2016-01-14 11:06:14 +00:00
Daniel Wagner-Hall	f6fcff3602	Don't start server if ASes are invalidly configured	2016-01-13 17:09:24 +00:00
Daniel Wagner-Hall	244b356a37	Delete unused code	2016-01-13 17:03:58 +00:00
Mark Haines	9c1f853d58	Rename 'user_name' to 'user_id' in push to make it consistent with the rest of the code	2016-01-13 13:32:59 +00:00
Mark Haines	f4dad9f639	Merge remote-tracking branch 'origin/erikj/bulk_get_push_rules' into markjh/table_name ... Conflicts: synapse/storage/push_rule.py	2016-01-13 11:46:07 +00:00
Erik Johnston	8740e4e94a	bulk_get_push_rules should handle empty lists	2016-01-13 11:37:17 +00:00
Mark Haines	c0a279e808	Delete the table objects from TransactionStore	2016-01-13 11:15:20 +00:00
Mark Haines	96e400fee5	Remove the RoomsTable object	2016-01-13 11:07:32 +00:00
Mark Haines	a8e9e0b916	Remove the PushersTable and EventPushActionsTable objects	2016-01-12 14:41:26 +00:00
Mark Haines	31de2953a3	Remove the PushRuleTable and PushRuleEnableTable objects	2016-01-12 14:36:16 +00:00
David Baker	b5d33a656f	Postgres doesn't like booleans	2016-01-11 17:13:52 +00:00
David Baker	fe56138142	Remove rogue 'admin'	2016-01-11 17:09:03 +00:00
David Baker	c232780081	Merge pull request #456 from matrix-org/store_event_actions ... Send unread notification counts	2016-01-08 14:47:15 +00:00
David Baker	daadcf36c0	This comma is actually important	2016-01-07 10:15:35 +00:00
Matthew Hodgson	6c28ac260c	copyrights	2016-01-07 04:26:29 +00:00
David Baker	442fcc02f7	Merge remote-tracking branch 'origin/develop' into store_event_actions	2016-01-06 17:28:55 +00:00
David Baker	b6a585348a	Adding is_guest here won't work because it just constructs a dict of uid -> password hash	2016-01-06 17:16:02 +00:00
Daniel Wagner-Hall	5880de186b	Log when we skip daily messages	2016-01-06 13:52:16 +00:00
David Baker	992928304f	Delete notifications for redacted events	2016-01-06 11:58:46 +00:00
David Baker	ae1262a241	Add schema change file for is_guest flag	2016-01-06 11:58:20 +00:00
David Baker	c79f221192	Add is_guest flag to users db to track whether a user is a guest user or not. Use this so we can run _filter_events_for_client when calculating event_push_actions.	2016-01-06 11:38:09 +00:00
David Baker	eb03625626	Merge remote-tracking branch 'origin/develop' into store_event_actions	2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall	cfd07aafff	Allow guests to upgrade their accounts	2016-01-05 18:01:18 +00:00
David Baker	4eb7b950c8	= not == in sql	2016-01-04 18:11:17 +00:00
David Baker	c914d67cda	Rename event-actions to event_push_actions as per PR request	2016-01-04 14:05:37 +00:00
David Baker	3051c9d002	Address minor PR issues	2016-01-04 13:39:29 +00:00
David Baker	4c8f6a7e42	Insert push actions in a single db query rather than one per user/profile_tag	2015-12-22 17:04:31 +00:00
David Baker	65c451cb38	Add bulk push rule evaluator which actually still evaluates rules one by one, but does far fewer db queries to fetch the rules	2015-12-22 15:19:34 +00:00
David Baker	c061b47c57	Merge remote-tracking branch 'origin/develop' into store_event_actions	2015-12-21 15:30:26 +00:00
David Baker	091c545c4f	pep8	2015-12-21 10:14:57 +00:00
David Baker	413d0d6a24	Make unread notification count sending work: put the correct count in incremental syncs too, where necessary, and fix silly bugs like only select the event actions for that user...	2015-12-18 17:47:00 +00:00
Richard van der Hoff	a64f9bbfe0	Fix 500 error when back-paginating search results ... We were mistakenly adding pagination clauses to the count query, which then failed because the count query doesn't join to the events table.	2015-12-17 12:50:46 +00:00
David Baker	42ad49f5b7	still very WIP, but now sends unread_notifications_count in the room object on sync (only actually corrrect in a full sync: hardcoded to 0 in incremental syncs).	2015-12-16 18:42:09 +00:00
Mark Haines	4fab578b43	Merge branch 'release-v0.12.0' into develop	2015-12-16 11:18:45 +00:00
Mark Haines	dcfc70e8ed	Allow users to change which account a 3pid is bound to	2015-12-15 17:02:21 +00:00
Mark Haines	15e9885197	Merge branch 'release-v0.12.0' into develop	2015-12-14 14:46:55 +00:00
Mark Haines	98dfa7d24f	Skip events that where the body, name or topic isn't a string when back populating the FTS index	2015-12-14 13:55:46 +00:00
Daniel Wagner-Hall	76e69cc8de	Fix typo	2015-12-14 12:38:55 +00:00
Mark Haines	bfc52a2342	Fix typo in sql for full text search on sqlite3	2015-12-14 11:38:11 +00:00
Mark Haines	deeebbfcb7	Merge branch 'release-v0.12.0' into develop	2015-12-12 14:21:49 +00:00
Erik Johnston	d9a5c56930	Include approximate count of search results	2015-12-11 11:40:23 +00:00
Erik Johnston	51fb590c0e	Use more efficient query form	2015-12-11 11:12:57 +00:00
David Baker	aa667ee396	Save event actions to the db	2015-12-10 17:51:15 +00:00
Mark Haines	7d6b313312	Add caches for whether a room has been forgotten by a user	2015-12-10 17:49:34 +00:00
David Baker	21f135ba76	Very first cut of calculating actions for events as they come in. Doesn't store them yet. Not very efficient.	2015-12-10 16:26:08 +00:00
Mark Haines	6a5ff5f223	Track the time spent in the database per request. ... and track the number of transactions that request started.	2015-12-07 17:56:11 +00:00
Mark Haines	d57c5cda71	Bump schema version. ... As we released version 26 in v0.11.1	2015-12-04 15:28:39 +00:00
Erik Johnston	b8680b82c3	Merge pull request #414 from matrix-org/erikj/if_not_exists ... Older versions of SQLite don't like IF NOT EXISTS in virtual tables	2015-12-03 13:21:09 +00:00
David Baker	e515b48929	Just replace the table definition with the one from full_schema 16	2015-12-02 17:23:52 +00:00
David Baker	8810eb8c39	Fix schema delta 15 on postgres in the very unlikley event that anyone upgrades to 15...	2015-12-02 17:19:11 +00:00
Erik Johnston	f721fdbf87	Merge pull request #412 from matrix-org/erikj/search ... Search: Add prefix matching support	2015-12-02 13:56:41 +00:00
Erik Johnston	976cb5aaa8	Throw if unrecognized DB type	2015-12-02 13:50:43 +00:00
Erik Johnston	b2def42bfd	Older versions of SQLite don't like IF NOT EXISTS in virtual tables	2015-12-02 13:29:14 +00:00
Erik Johnston	b9acef5301	Fix so highlight matching works again	2015-12-02 13:28:13 +00:00
Erik Johnston	7dd6e5efca	Remove deuplication. Add comment about regex.	2015-12-02 13:09:37 +00:00
Erik Johnston	477da77b46	Search: Add prefix matching support	2015-12-02 11:40:52 +00:00
Mark Haines	95f30ecd1f	Add API for setting account_data globaly or on a per room basis	2015-12-01 18:41:32 +00:00
Erik Johnston	4dcaa42b6d	Allow paginating search ordered by recents	2015-11-30 17:45:31 +00:00
Erik Johnston	76936f43ae	Return words to highlight in search results	2015-11-27 16:40:42 +00:00
Daniel Wagner-Hall	df7cf6c0eb	Fix SQL for postgres again	2015-11-23 18:54:41 +00:00
Daniel Wagner-Hall	3e573a5c6b	Fix SQL for postgres	2015-11-23 18:48:53 +00:00
Daniel Wagner-Hall	7dfa455508	Remove size specifier for database column ... Postgres doesn't support them like this. We don't have a bool type in common between postgres and sqlite.	2015-11-23 18:35:25 +00:00
Daniel Wagner-Hall	95c3306798	Merge branch 'daniel/forgetrooms' of github.com:matrix-org/synapse into daniel/forgetrooms	2015-11-19 15:00:14 -05:00
Daniel Wagner-Hall	df6824a008	Ignore forgotten rooms in v2 sync	2015-11-19 14:54:47 -05:00
Paul "LeoNerd" Evans	1cfda3d2d8	Merge branch 'develop' into daniel/forgetrooms	2015-11-19 16:53:13 +00:00
Daniel Wagner-Hall	9da4c5340d	Simplify code	2015-11-19 10:07:21 -05:00
Daniel Wagner-Hall	bed7889703	Apply forgetting properly to historical events	2015-11-18 18:11:12 -05:00
Mark Haines	d153f482dd	Rename the database table	2015-11-18 15:33:02 +00:00
Mark Haines	1c960fbb80	s/private_user_data/account_data/	2015-11-18 15:31:04 +00:00
Daniel Wagner-Hall	ba26eb3d5d	Allow users to forget rooms	2015-11-17 17:17:30 -05:00
Richard van der Hoff	e4d622aaaf	Implementation of state rollback in /sync ... Implementation of SPEC-254: roll back the state dictionary to how it looked at the start of the timeline. Merged PR https://github.com/matrix-org/synapse/pull/373	2015-11-13 10:58:56 +00:00
Richard van der Hoff	fddedd51d9	Fix a few race conditions in the state calculation ... Be a bit more careful about how we calculate the state to be returned by /sync. In a few places, it was possible for /sync to return slightly later state than that represented by the next_batch token and the timeline. In particular, the following cases were susceptible: * On a full state sync, for an active room * During a per-room incremental sync with a timeline gap * When the user has just joined a room. (Refactor check_joined_room to make it less magical) Also, use store.get_state_for_events() (and thus the existing stategroups) to calculate the state corresponding to a particular sync position, rather than state_handler.compute_event_context(), which recalculates from first principles (and tends to miss some state). Merged from PR https://github.com/matrix-org/synapse/pull/372	2015-11-13 10:39:09 +00:00
Erik Johnston	3de46c7755	Trailing whitespace	2015-11-12 15:36:43 +00:00
Erik Johnston	8fd8e72cec	Expand comment	2015-11-12 15:33:47 +00:00
Erik Johnston	764e79d051	Comment	2015-11-12 15:19:56 +00:00
Erik Johnston	320408ef47	Fix SQL syntax	2015-11-12 15:09:45 +00:00
Erik Johnston	14a9d805b9	Use a (hopefully) more efficient SQL query for doing recency based room search	2015-11-12 14:48:39 +00:00
Erik Johnston	39de87869c	Fix bug where assumed dict was namedtuple	2015-11-12 14:47:48 +00:00
Erik Johnston	e21cef9bb5	Merge pull request #359 from matrix-org/markjh/incremental_indexing ... Incremental background updates for db indexes	2015-11-11 17:19:51 +00:00
Mark Haines	e1627388d1	Fix param style to work on both sqlite and postgres	2015-11-11 17:14:56 +00:00
Mark Haines	940a161192	Fix the background update	2015-11-11 13:59:40 +00:00
Daniel Wagner-Hall	cf437900e0	Return world_readable and guest_can_join in /publicRooms	2015-11-10 17:10:27 +00:00
Mark Haines	90b503216c	Use a background task to update databases to use the full text search	2015-11-10 16:20:13 +00:00
Mark Haines	a412b9a465	Run the background updates when starting synapse.	2015-11-10 15:50:58 +00:00
Mark Haines	2ede7aa8a1	Add background update task for reindexing event search	2015-11-09 19:29:32 +00:00
Mark Haines	c6a01f2ed0	Add storage module for tracking background updates. ... The progress for each background update is stored as a JSON blob in the database. Each background update is broken up into separate batches. The batch size is automatically tuned to try avoid blocking single threaded databases for too long.	2015-11-09 17:26:27 +00:00
Mark Haines	bbe10e8be7	Merge branch 'develop' into markjh/SYN-513 ... Conflicts: synapse/storage/tags.py	2015-11-09 15:01:59 +00:00
Mark Haines	c4135d85e1	SYN-513: Include updates for rooms that have had all their tags deleted	2015-11-09 14:53:08 +00:00
Matthew Hodgson	dd40fb68e4	fix comedy important missing comma breaking recent-ordered FTS on sqlite	2015-11-08 16:04:37 +00:00
Matthew Hodgson	767c20a869	add a key existence check to tags_by_room to avoid /events 500'ing when testing against vector	2015-11-06 20:49:57 +01:00
Daniel Wagner-Hall	f2c4ee41b9	Remove accidentally added ID column	2015-11-06 14:27:49 +00:00
Erik Johnston	3640ddfbf6	Error handling	2015-11-05 16:10:54 +00:00
Erik Johnston	729ea933ea	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search	2015-11-05 15:43:52 +00:00
Erik Johnston	7301e05122	Implement basic pagination for search results	2015-11-05 15:04:08 +00:00
Daniel Wagner-Hall	ca2f90742d	Open up /events to anonymous users for room events only ... Squash-merge of PR #345 from daniel/anonymousevents	2015-11-05 14:32:26 +00:00
Erik Johnston	05c326d445	Implement order and group by	2015-11-04 17:57:44 +00:00
Daniel Wagner-Hall	4e62ffdb21	Merge branch 'develop' of github.com:matrix-org/synapse into develop	2015-11-04 17:31:01 +00:00
Daniel Wagner-Hall	f522f50a08	Allow guests to register and call /events?room_id= ... This follows the same flows-based flow as regular registration, but as the only implemented flow has no requirements, it auto-succeeds. In the future, other flows (e.g. captcha) may be required, so clients should treat this like the regular registration flow choices.	2015-11-04 17:29:07 +00:00
Mark Haines	33b3e04049	Merge branch 'develop' into daniel/removesomelies ... Conflicts: synapse/notifier.py	2015-11-04 16:01:00 +00:00
Erik Johnston	6a3a840b19	Merge pull request #343 from matrix-org/erikj/fix_retries ... Fix broken cache for getting retry times.	2015-11-03 17:51:49 +00:00
Mark Haines	6797fcd9ab	Merge pull request #335 from matrix-org/markjh/room_tags ... Add APIs for adding and removing tags from rooms	2015-11-03 16:45:53 +00:00
Erik Johnston	7ce264ce5f	Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should	2015-11-03 16:24:03 +00:00
Mark Haines	5897e773fd	Spell "deferred" more correctly	2015-11-03 14:27:35 +00:00
Daniel Wagner-Hall	771ca56c88	Remove more unused parameters	2015-11-02 15:31:57 +00:00
Mark Haines	ddd8566f41	Store room tag content and return the content in the m.tag event	2015-11-02 15:11:31 +00:00
Mark Haines	0e36756383	Merge branch 'develop' into markjh/room_tags	2015-11-02 10:57:00 +00:00
Daniel Wagner-Hall	621e84d9a0	Add missing column	2015-10-30 16:25:53 +00:00
Mark Haines	f40b0ed5e1	Inform the client of new room tags using v1 /events	2015-10-29 15:21:09 +00:00
Mark Haines	a89b86dc47	Fix pyflakes errors	2015-10-28 16:45:57 +00:00
Mark Haines	892e70ec84	Add APIs for adding and removing tags from rooms	2015-10-28 16:06:57 +00:00
Erik Johnston	56dbcd1524	Docs	2015-10-28 14:05:50 +00:00
Erik Johnston	5cb298c934	Add room context api	2015-10-28 13:45:56 +00:00
Erik Johnston	4cf633d5e9	Pull out sender when computing search results	2015-10-23 15:41:36 +01:00
Erik Johnston	0c36098c1f	Implement rank function for SQLite FTS	2015-10-23 13:23:48 +01:00
Erik Johnston	671ac699f1	Actually filter results	2015-10-22 16:54:56 +01:00
Erik Johnston	fb0fecd0b9	LESS THAN	2015-10-22 16:18:35 +01:00
Erik Johnston	232beb3a3c	Use namedtuple as return value	2015-10-22 15:02:35 +01:00
Erik Johnston	ba02bba88c	Limit max number of SQL vars	2015-10-22 13:25:27 +01:00
Erik Johnston	8a98f0dc5b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search	2015-10-22 13:16:35 +01:00
Erik Johnston	f142898f52	PEP8	2015-10-22 11:18:01 +01:00
Erik Johnston	4d25bc6c92	Move FTS to delta 25	2015-10-22 11:12:28 +01:00
Mark Haines	e94ffd89d6	Merge pull request #316 from matrix-org/markjh/v2_sync_archived ... Add rooms that the user has left under archived in v2 sync.	2015-10-21 15:46:41 +01:00
Mark Haines	e3d75f564a	Include banned rooms in the archived section of v2 sync	2015-10-21 11:15:48 +01:00
Mark Haines	68b7fc3e2b	Add rooms that the user has left under archived in v2 sync.	2015-10-19 17:26:18 +01:00
Erik Johnston	46d39343d9	Explicitly check for Sqlite3Engine	2015-10-16 16:58:00 +01:00
Erik Johnston	f2d698cb52	Typing	2015-10-16 16:46:48 +01:00
Erik Johnston	33646eb000	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search	2015-10-16 15:35:35 +01:00
Mark Haines	fc012aa8dc	Fix FilteringStore.get_user_filter to work with postgres	2015-10-16 15:28:43 +01:00
Erik Johnston	12122bfc36	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search	2015-10-16 14:46:32 +01:00
Erik Johnston	edb998ba23	Explicitly check for Sqlite3Engine	2015-10-16 14:37:14 +01:00
Erik Johnston	3cf9948b8d	Add docstring	2015-10-16 11:28:12 +01:00
Erik Johnston	73260ad01f	Comment on the LIMIT 500	2015-10-16 11:24:02 +01:00
Erik Johnston	22a8c91448	Split up run_upgrade	2015-10-16 11:19:44 +01:00
Mark Haines	6296590bf7	Encode the filter JSON as UTF-8 before storing in the database. ... Because we are using a binary column type to store the filter JSON.	2015-10-16 10:50:32 +01:00
Erik Johnston	bcfb653816	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search	2015-10-15 16:37:32 +01:00
Mark Haines	a059760954	Merge pull request #305 from matrix-org/markjh/v2_sync_api ... Update the v2 sync API to work as specified in the current spec.	2015-10-14 13:56:23 +01:00
Mark Haines	f50c43464c	Merge branch 'develop' into markjh/v2_sync_api	2015-10-14 11:40:45 +01:00
Erik Johnston	99c7fbfef7	Fix to work with SQLite	2015-10-14 09:52:40 +01:00
Erik Johnston	858634e1d0	Remove unused room_id arg	2015-10-14 09:31:20 +01:00
Erik Johnston	7ecd11accb	Add paranoia limit	2015-10-13 15:50:56 +01:00
Erik Johnston	3e2a1297b5	Remove constraints in preperation of using filters	2015-10-13 15:22:14 +01:00
Mark Haines	cacf0688c6	Add a get_invites_for_user method to the storage to find out the rooms a user is invited to	2015-10-13 14:08:38 +01:00
Erik Johnston	88971fd034	Merge branch 'erikj/store_engine' into erikj/search	2015-10-13 14:03:30 +01:00
Erik Johnston	17c80c8a3d	rename schema_prepare to prepare_database	2015-10-13 13:56:22 +01:00
Erik Johnston	cfd39d6b55	Add SQLite support	2015-10-13 13:47:50 +01:00
Erik Johnston	f9340ea0d5	Merge branch 'erikj/store_engine' into erikj/search	2015-10-13 13:29:02 +01:00
Erik Johnston	ec398af41c	Expose error more nicely	2015-10-13 11:43:43 +01:00
Erik Johnston	40b6a5aad1	Split out the schema preparation and update logic into its own module	2015-10-13 11:38:48 +01:00
Erik Johnston	1a40afa756	Add sqlite schema	2015-10-13 10:36:25 +01:00
Erik Johnston	ca53ad7425	Filter events to only thsoe that the user is allowed to see	2015-10-12 15:52:55 +01:00
Erik Johnston	927004e349	Remove unused room_id parameter	2015-10-12 15:06:14 +01:00
Erik Johnston	61561b9df7	Keep FTS indexes up to date. Only search through rooms currently joined	2015-10-12 10:49:53 +01:00
Erik Johnston	c85c912562	Add basic full text search impl.	2015-10-09 15:48:31 +01:00
Mark Haines	301141515a	Merge pull request #288 from matrix-org/markjh/unused_definitions ... Remove some of the unused definitions from synapse	2015-09-28 14:22:44 +01:00
Mark Haines	cf1100887b	Fix order of ON constraints in _get_rooms_for_user_where_membership_is_txn	2015-09-24 17:35:10 +01:00
Mark Haines	1cd65a8d1e	synapse/storage/state.py: _make_group_id was unused	2015-09-23 10:37:58 +01:00
Mark Haines	973ebb66ba	Remove unused functions from synapse/storage/signatures.py	2015-09-23 10:36:33 +01:00
Mark Haines	e51aa4be96	synapse/storage/roommember.py:_get_members_query was unused	2015-09-23 10:35:10 +01:00
Mark Haines	92d8d724c5	Remove unused functions from synapse/storage/events.py	2015-09-23 10:33:06 +01:00
Mark Haines	c292dba70c	Remove unused functions from synapse/storage/event_federation.py	2015-09-23 10:31:25 +01:00
Mark Haines	396834f1c0	synapse/storage/_base.py:_simple_max_id was unused	2015-09-23 10:30:38 +01:00
Mark Haines	1d9036aff2	synapse/storage/_base.py:_simple_delete was unused	2015-09-23 10:30:25 +01:00
Mark Haines	1ee3d26432	synapse/storage/_base.py:_simple_selectupdate_one was unused	2015-09-23 10:30:03 +01:00
Mark Haines	3559a835a2	synapse/storage/event_federation.py:_get_auth_events is unused	2015-09-22 18:39:46 +01:00
Mark Haines	7dd4f79c49	synapse/storage/_base.py:_execute_and_decode was unused	2015-09-22 18:37:07 +01:00
Mark Haines	527d95dea0	synapse/storage/_base.py:Table was unused	2015-09-22 18:14:15 +01:00
Daniel Wagner-Hall	6d59ffe1ce	Add some docstrings	2015-09-22 13:47:40 +01:00
Daniel Wagner-Hall	eb011cd99b	Add docstring	2015-09-22 13:29:36 +01:00
Daniel Wagner-Hall	7213588083	Implement configurable stats reporting ... SYN-287 This requires that HS owners either opt in or out of stats reporting. When --generate-config is passed, --report-stats must be specified If an already-generated config is used, and doesn't have the report_stats key, it is requested to be set.	2015-09-22 12:57:40 +01:00
Mark Haines	ee2d722f0f	Merge pull request #276 from matrix-org/markjh/history_for_rooms_that_have_been_left ... SPEC-216: Allow users to view the history of rooms that they have left.	2015-09-21 14:38:13 +01:00
Erik Johnston	2048388cfd	Merge pull request #281 from matrix-org/erikj/update_extremeties ... When updating a stored event from outlier to non-outlier, remember to update the extremeties	2015-09-15 16:57:25 +01:00
Erik Johnston	8148c48f11	"Comments"	2015-09-15 16:54:48 +01:00
Erik Johnston	3b05b67c89	When updating a stored event from outlier to non-outlier, remember to update the extremeties	2015-09-15 16:34:42 +01:00
Erik Johnston	dffc9c4ae0	Drop unused index	2015-09-14 14:41:37 +01:00
Mark Haines	09cb5c7d33	Allow users that have left a room to get the messages that happend in the room before they left	2015-09-09 17:31:09 +01:00
Mark Haines	3c166a24c5	Remove undocumented and unimplemented 'feedback' parameter from the Client-Server API	2015-09-09 16:05:09 +01:00
Mark Haines	89ae0166de	Allow room initialSync for users that have left the room, returning a snapshot of how the room was when they left it	2015-09-09 13:25:22 +01:00
Mark Haines	417485eefa	Include the event_id and stream_ordering of membership events when looking up which rooms a user is in	2015-09-08 18:14:54 +01:00
Erik Johnston	b62c1395d6	Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop	2015-09-01 13:11:55 +01:00
Erik Johnston	cd800ad99a	Lower size of 'stateGroupCache' now that we have data from matrix.org to support doing so	2015-09-01 10:09:03 +01:00
Mark Haines	a7122692d9	Merge branch 'release-v0.10.0' into develop ... Conflicts: synapse/handlers/auth.py synapse/python_dependencies.py synapse/rest/client/v1/login.py	2015-08-28 11:15:27 +01:00
Daniel Wagner-Hall	d3c0e48859	Merge erikj/user_dedup to develop	2015-08-26 13:42:45 +01:00
Daniel Wagner-Hall	a2355fae7e	Merge pull request #251 from matrix-org/removeadmin ... Stop looking up "admin", which we never read	2015-08-25 17:23:05 +01:00
Daniel Wagner-Hall	ee3fa1a99c	Merge pull request #248 from matrix-org/deviceid ... Remove completely unused concepts from codebase	2015-08-25 17:19:06 +01:00
Daniel Wagner-Hall	a9d8bd95e7	Stop looking up "admin", which we never read	2015-08-25 16:29:39 +01:00
Daniel Wagner-Hall	a0b181bd17	Remove completely unused concepts from codebase ... Removes device_id and ClientInfo device_id is never actually written, and the matrix.org DB has no non-null entries for it. Right now, it's just cluttering up code. This doesn't remove the columns from the database, because that's fiddly.	2015-08-25 16:23:06 +01:00
Mark Haines	1c847af28a	Merge pull request #243 from matrix-org/markjh/remove_syutil ... Replace syutil dependency with smaller, single-purpose libraries	2015-08-25 10:52:16 +01:00
Mark Haines	037481a033	Remove autoincrement since we incrementing the ID in the storage layer	2015-08-24 17:48:57 +01:00
Mark Haines	01fc3943f1	Fix indent	2015-08-24 17:18:58 +01:00
Mark Haines	78323ccdb3	Remove syutil dependency in favour of smaller single-purpose libraries	2015-08-24 16:17:38 +01:00
Erik Johnston	457970c724	Don't insert events into 'event_*_extremeties' tables if they're outliers	2015-08-23 13:44:23 +01:00
Erik Johnston	1bd1a43073	Actually check if event_id isn't returned by _get_state_groups	2015-08-21 14:30:34 +01:00
Erik Johnston	745b72660a	Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop	2015-08-21 11:39:38 +01:00
Erik Johnston	42f12ad92f	When logging in fetch user by user_id case insensitively, *unless* there are multiple case insensitive matches, in which case require the exact user_id	2015-08-21 11:38:44 +01:00
Erik Johnston	aa3c9c7bd0	Don't allow people to register user ids which only differ by case to an existing one	2015-08-21 10:57:47 +01:00
Erik Johnston	1f7642efa9	Fix bug where we didn't correctly serialize the redacted_because key over federation	2015-08-21 09:36:07 +01:00
Daniel Wagner-Hall	e8cf77fa49	Merge branch 'develop' into refresh ... Conflicts: synapse/rest/client/v1/login.py	2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall	cecbd636e9	/tokenrefresh POST endpoint ... This allows refresh tokens to be exchanged for (access_token, refresh_token). It also starts issuing them on login, though no clients currently interpret them.	2015-08-20 16:21:35 +01:00
Daniel Wagner-Hall	13a6517d89	s/by_token/by_access_token/g ... We're about to have two kinds of token, access and refresh	2015-08-20 16:01:29 +01:00
Mark Haines	a0b8e5f2fe	Merge pull request #211 from matrix-org/email_in_use ... Changes for unique emails	2015-08-20 10:04:04 +01:00
Erik Johnston	e624cdec64	Merge pull request #228 from matrix-org/erikj/_get_state_for_groups ... Ensure we never return a None event from _get_state_for_groups	2015-08-18 16:30:17 +01:00
Erik Johnston	38a965b816	Merge pull request #227 from matrix-org/erikj/receipts_take2 ... Re-enable receipts API.	2015-08-18 16:30:04 +01:00
Erik Johnston	a82938416d	Remove newline because vertical whitespace makes mjark sad	2015-08-18 16:28:13 +01:00
Erik Johnston	0bfdaf1f4f	Rejig the code to make it nicer	2015-08-18 16:26:07 +01:00
Erik Johnston	128ed32e6b	Bump size of get_presence_state cache	2015-08-18 15:51:23 +01:00
Erik Johnston	f704c10f29	Rename unhelpful variable name	2015-08-18 11:54:03 +01:00
Erik Johnston	8199475ce0	Ensure we never return a None event from _get_state_for_groups	2015-08-18 11:44:10 +01:00
Erik Johnston	85d0bc3bdc	Reduce cache size from obscenely large to quite large	2015-08-18 11:00:38 +01:00
Erik Johnston	cfc503681f	Comments	2015-08-18 10:49:23 +01:00
Erik Johnston	8f4165628b	Add index receipts_linearized_room_stream	2015-08-17 14:43:54 +01:00
Erik Johnston	d3d582bc1c	Remove unused import	2015-08-17 13:38:09 +01:00
Erik Johnston	afef6f5d16	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts_take2	2015-08-17 13:23:44 +01:00
Erik Johnston	2d97e65558	Remember to invalidate caches	2015-08-17 10:46:55 +01:00
Erik Johnston	1a9510bb84	Implement a batched presence_handler.get_state and use it	2015-08-17 10:40:23 +01:00
Erik Johnston	47abebfd6d	Add batched version of store.get_presence_state	2015-08-17 09:50:50 +01:00
Erik Johnston	f9d4da7f45	Fix bug where we were leaking None into state event lists	2015-08-17 09:39:45 +01:00
Daniel Wagner-Hall	30883d8409	Merge pull request #221 from matrix-org/auth ... Simplify LoginHander and AuthHandler	2015-08-14 17:02:22 +01:00
Erik Johnston	68b255c5a1	Batch _get_linearized_receipts_for_rooms	2015-08-14 15:06:22 +01:00
Erik Johnston	129ee4e149	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts_take2	2015-08-13 17:28:43 +01:00
Erik Johnston	2bb2c02571	Remove some vertical space	2015-08-13 17:11:30 +01:00
Erik Johnston	57877b01d7	Replace list comprehension	2015-08-13 17:00:17 +01:00
Erik Johnston	5db5677969	Add metrics to the receipts cache	2015-08-13 16:58:23 +01:00
Erik Johnston	0fbed2a8fa	Comment	2015-08-12 17:22:54 +01:00
Erik Johnston	c10ac7806e	Explain why we're prefilling dict with Nones	2015-08-12 17:16:30 +01:00
Erik Johnston	101ee3fd00	Better variable name	2015-08-12 17:08:05 +01:00
Erik Johnston	df361d08f7	Split _get_state_for_group_from_cache into two	2015-08-12 17:06:21 +01:00
Daniel Wagner-Hall	415c2f0549	Simplify LoginHander and AuthHandler ... * Merge LoginHandler -> AuthHandler * Add a bunch of documentation * Improve some naming * Remove unused branches I will start merging the actual logic of the two handlers shortly	2015-08-12 15:49:37 +01:00
Erik Johnston	2df8dd9b37	Move all the caches into their own package, synapse.util.caches	2015-08-11 18:00:59 +01:00
Erik Johnston	53a817518b	Comments	2015-08-11 11:40:40 +01:00
Erik Johnston	6eaa116867	Comment	2015-08-11 11:35:24 +01:00
Erik Johnston	4762c276cb	Docs	2015-08-11 11:33:41 +01:00
Erik Johnston	1b994a97dd	Fix application of ACLs	2015-08-11 10:41:40 +01:00
Erik Johnston	10b874067b	Fix state cache	2015-08-11 09:12:41 +01:00
Erik Johnston	017b798e4f	Clean up StateStore	2015-08-10 15:01:06 +01:00
Erik Johnston	2c019eea11	Remove unused function	2015-08-10 14:44:41 +01:00
Erik Johnston	bb0a475c30	Comments	2015-08-10 14:27:38 +01:00
Mark Haines	559c51debc	Use TypeError instead of ValueError and give a nicer error mesasge ... when someone calls Cache.invalidate with the wrong type.	2015-08-10 14:07:17 +01:00
Erik Johnston	7ce71f2ffc	Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache	2015-08-10 13:47:51 +01:00
Erik Johnston	8c3a62b5c7	Merge pull request #215 from matrix-org/erikj/cache_varargs_interface ... Change Cache to not use *args in its interface	2015-08-10 13:47:45 +01:00
Erik Johnston	86eaaa885b	Rename keyargs to args in CacheDescriptor	2015-08-10 13:44:44 +01:00
Erik Johnston	e0b6e49466	Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache	2015-08-10 10:39:22 +01:00
Erik Johnston	2cd6cb9f65	Rename keyargs to args in Cache	2015-08-10 10:38:47 +01:00
Erik Johnston	aa88582e00	Do bounds check	2015-08-10 10:08:15 +01:00
Erik Johnston	5119e416e8	Line length	2015-08-10 10:05:30 +01:00
Erik Johnston	8f04b6fa7a	Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache	2015-08-07 19:30:25 +01:00
Erik Johnston	7dec0b2bee	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache	2015-08-07 19:28:39 +01:00
Erik Johnston	06218ab125	Merge pull request #212 from matrix-org/erikj/cache_deferreds ... Make CacheDescriptor cache deferreds rather than the deferreds' values	2015-08-07 19:28:05 +01:00
Erik Johnston	2352974aab	Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface	2015-08-07 19:26:54 +01:00
Erik Johnston	9c5385b53a	s/observed/observer/	2015-08-07 19:26:38 +01:00
Erik Johnston	ffab798a38	Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface	2015-08-07 19:18:47 +01:00
Erik Johnston	62126c996c	Propogate stale cache errors to calling functions	2015-08-07 19:17:58 +01:00
Erik Johnston	3213ff630c	Remove unnecessary cache	2015-08-07 19:14:05 +01:00
Erik Johnston	20addfa358	Change Cache to not use *args in its interface	2015-08-07 18:32:47 +01:00
Erik Johnston	9eb5b23d3a	Batch up various DB requests for event -> state	2015-08-07 18:16:02 +01:00
Erik Johnston	0211890134	Implement a CacheListDescriptor	2015-08-07 18:14:49 +01:00
Paul "LeoNerd" Evans	0db40d3e93	Don't complain about extra .pyc files we find while hunting for database schemas	2015-08-07 17:22:11 +01:00
Erik Johnston	b2c7bd4b09	Cache get_recent_events_for_room	2015-08-07 14:42:34 +01:00
Erik Johnston	b3768ec10a	Remove unncessary cache	2015-08-07 13:41:05 +01:00
Erik Johnston	b8e386db59	Change Cache to not use *args in its interface	2015-08-07 11:52:21 +01:00
Erik Johnston	fe994e728f	Store absence of state in cache	2015-08-07 10:17:38 +01:00
Erik Johnston	1d08bf7c17	Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache	2015-08-06 14:03:15 +01:00
Erik Johnston	63b1eaf32c	Docs	2015-08-06 14:02:50 +01:00
Erik Johnston	b811c98574	Remove failed deferreds from cache	2015-08-06 14:01:27 +01:00
Erik Johnston	433314cc34	Re-implement DEBUG_CACHES flag	2015-08-06 14:01:05 +01:00
Erik Johnston	f596ff402e	Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache	2015-08-06 13:37:56 +01:00
Erik Johnston	2efb93af52	Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds	2015-08-06 13:35:28 +01:00
Erik Johnston	953dbd28a7	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cached_keyword_args	2015-08-06 13:35:03 +01:00
Erik Johnston	7eea3e356f	Make @cached cache deferreds rather than the deferreds' values	2015-08-06 13:33:34 +01:00
Erik Johnston	3e1b77efc2	Merge branch 'erikj/cached_keyword_args' of github.com:matrix-org/synapse into erikj/dictionary_cache	2015-08-05 16:45:56 +01:00
Erik Johnston	b52b4a84ec	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache	2015-08-05 15:41:20 +01:00
Erik Johnston	1e62a3d3a9	Up the cache size for 'get_joined_hosts_for_room' and 'get_users_in_room'	2015-08-05 15:40:40 +01:00
Erik Johnston	a89559d797	Use LRU cache by default	2015-08-05 15:39:47 +01:00
Erik Johnston	07507643cb	Use dictionary cache to do group -> state fetching	2015-08-05 15:11:42 +01:00
Erik Johnston	c67ba143fa	Move DictionaryCache	2015-08-04 15:58:28 +01:00
Erik Johnston	e7768e77f5	Add basic dictionary cache	2015-08-04 15:56:56 +01:00
David Baker	e124128542	Bump schema version	2015-08-04 14:50:31 +01:00
David Baker	c77048e12f	Add endpoint that proxies ID server request token and errors if the given email is in use on this Home Server.	2015-08-04 14:37:09 +01:00
Erik Johnston	413a4c289b	Add comment	2015-08-04 11:08:07 +01:00
Erik Johnston	4d6cb8814e	Speed up event filtering (for ACL) logic	2015-08-04 09:32:23 +01:00
Erik Johnston	39e21ea51c	Add support for using keyword arguments with cached functions	2015-07-27 13:57:29 +01:00
David Baker	20c0324e9c	Dodesn't seem to make any difference: guess it does work with the object reference	2015-07-21 16:21:37 -07:00
David Baker	cf7a40b08a	I think this was what was intended...	2015-07-21 16:08:00 -07:00
Mark Haines	3b5823c74d	s/take/claim/ for end to end key APIs	2015-07-20 18:23:54 +01:00
Matrix	4f973eb657	Up default cache size for _RoomStreamChangeCache	2015-07-18 19:07:33 +01:00
Erik Johnston	b6d4a4c6d8	Merge pull request #199 from matrix-org/erikj/receipts ... Implement read receipts.	2015-07-16 18:18:36 +01:00
Erik Johnston	baa55fb69e	Merge pull request #193 from matrix-org/erikj/bulk_persist_event ... Add bulk insert events API	2015-07-14 10:49:24 +01:00
Erik Johnston	4624d6035e	Docs	2015-07-14 10:19:07 +01:00
Erik Johnston	5989637f37	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts	2015-07-13 13:50:57 +01:00
Erik Johnston	e5991af629	Comments	2015-07-13 13:30:43 +01:00
Erik Johnston	17bb9a7eb9	Remove commented out code	2015-07-10 14:07:57 +01:00
Erik Johnston	532fcc997a	Merge pull request #196 from matrix-org/erikj/room_history ... Add ability to restrict room history.	2015-07-10 13:47:04 +01:00
Erik Johnston	0b3389bcd2	Merge pull request #194 from matrix-org/erikj/bulk_verify_sigs ... Implement bulk verify_signed_json API	2015-07-10 13:46:53 +01:00
Erik Johnston	0d7f0febf4	Uniquely name unique constraint	2015-07-10 13:43:03 +01:00
Erik Johnston	ed88720952	Handle error slightly better	2015-07-09 16:14:46 +01:00
Mark Haines	bf0d59ed30	Don't bother with a timeout for one time keys on the server.	2015-07-09 14:04:03 +01:00
Erik Johnston	1af188209a	Change format of receipts to allow inclusion of data	2015-07-09 11:39:30 +01:00
Mark Haines	8fb79eeea4	Only remove one time keys when new one time keys are added	2015-07-08 17:04:29 +01:00
Erik Johnston	304343f4d7	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts	2015-07-08 15:37:33 +01:00
Erik Johnston	af812b68dd	Add a cache to fetching of receipt streams	2015-07-08 15:35:00 +01:00
Erik Johnston	87311d1b8c	Hook up receipts to v1 initialSync	2015-07-08 11:02:04 +01:00
Erik Johnston	ca041d5526	Wire together receipts and the notifer/federation	2015-07-07 15:25:30 +01:00
Erik Johnston	716e426933	Fix various typos	2015-07-07 10:55:31 +01:00
Mark Haines	b5770f8947	Add store for client end to end keys	2015-07-06 18:46:47 +01:00
Erik Johnston	65e69dec8b	Don't explode if we don't recognize one of the event_ids in the backfill request	2015-07-06 09:33:03 +01:00
Erik Johnston	1a60545626	Add basic impl for room history ACL on GET /messages client API	2015-07-02 16:20:10 +01:00
Erik Johnston	ac78e60de6	Add stream_id index	2015-07-02 13:18:41 +01:00
Erik Johnston	80a61330ee	Add basic storage functions for handling of receipts	2015-07-01 17:19:12 +01:00
Erik Johnston	901f56fa63	Add tables for receipts	2015-06-30 15:29:47 +01:00
Erik Johnston	b5f55a1d85	Implement bulk verify_signed_json API	2015-06-26 10:39:34 +01:00
Erik Johnston	5130d80d79	Add bulk insert events API	2015-06-25 17:29:34 +01:00
Erik Johnston	6924852592	Batch SELECTs in _get_auth_chain_ids_txn	2015-06-23 11:01:04 +01:00
Erik Johnston	2eda996a63	Add a dummy.sql into delta/20 as pip isn't packinging the pushers.py	2015-06-13 12:21:58 +01:00
Erik Johnston	1e365e88bd	Bump schema version	2015-06-11 15:50:39 +01:00
David Baker	da84946de4	pep8	2015-06-04 16:43:45 +01:00
David Baker	63a7b3ad1e	Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them.	2015-06-04 16:16:01 +01:00
Erik Johnston	0d7d9c37b6	Add cache to get_state_groups	2015-06-03 14:45:55 +01:00
Erik Johnston	d8866d7277	Caches should be bound to instances. ... Before, caches were global and so different instances of the stores would share caches. This caused problems in the unit tests.	2015-06-03 14:45:17 +01:00
Erik Johnston	6e17463228	Don't explode if we don't have the event	2015-06-01 16:39:43 +01:00
Erik Johnston	a7b65bdedf	Add config option to turn off freezing events. Use new encode_json api and ujson.loads	2015-05-29 12:17:33 +01:00
Mark Haines	d9f60e8dc8	Merge pull request #163 from matrix-org/markjh/presence_list_cache ... Add a cache for the presence list	2015-05-22 17:02:23 +01:00
Mark Haines	0e42dfbe22	Merge pull request #164 from matrix-org/markjh/pusher_performance_2 ... Add a cache for get_push rules for user, fix cache invalidation	2015-05-22 17:01:56 +01:00
Mark Haines	1a9a9abcc7	Add a cache for getting the presence list for a user	2015-05-22 16:11:17 +01:00
Erik Johnston	74b7de83ec	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes	2015-05-22 16:10:42 +01:00
Mark Haines	36317f3dad	Merge pull request #156 from matrix-org/erikj/join_perf ... Make joining #matrix:matrix.org over federation quicker	2015-05-22 16:09:54 +01:00
Mark Haines	b6adfc59f5	Invalidate the get_latest_event_ids_in_room cache when deleting from event_forward_extremities	2015-05-22 13:01:03 +01:00
Mark Haines	a04cde613e	Add a cache for get_push rules for user, fix cache invalidation	2015-05-22 10:39:45 +01:00
Erik Johnston	4429e720ae	Merge branch 'master' of github.com:matrix-org/synapse into develop	2015-05-22 10:33:00 +01:00
Mark Haines	2043527b9b	Don't try to use a txn when not in one, remove spurious debug logging	2015-05-21 16:53:03 +01:00
Mark Haines	53447e9cd3	Add caches for things requested by the pushers	2015-05-21 16:41:39 +01:00
Mark Haines	d61ce3f670	Add a cache for get_current_state with state_key	2015-05-21 16:41:39 +01:00
Erik Johnston	a910984b58	Actually return something from lambda	2015-05-21 15:58:41 +01:00
Erik Johnston	e309b1045d	Sort backfill events	2015-05-21 15:57:35 +01:00
Erik Johnston	0180bfe4aa	Remove dead code	2015-05-21 15:53:41 +01:00
Erik Johnston	1f3d1d85a9	Only get non-state	2015-05-21 15:52:29 +01:00
Erik Johnston	39a3340f73	Skip events we've already seen	2015-05-21 15:48:56 +01:00
Erik Johnston	ae3bff3491	Correctly prepopulate queue	2015-05-21 15:46:07 +01:00
Erik Johnston	dc085ddf8c	Don't prepopulate event_results	2015-05-21 15:44:05 +01:00
Erik Johnston	73d23c6ae8	Don't readd things that are already in event_results	2015-05-21 15:40:22 +01:00
Erik Johnston	6189d8e54d	PriorityQueue gives lowest first	2015-05-21 15:38:08 +01:00
Erik Johnston	115ef3ddac	Correctly capture Queue.Empty exception	2015-05-21 15:37:43 +01:00
Erik Johnston	c2633907c5	Merge branch 'erikj/join_perf' of github.com:matrix-org/synapse into erikj/backfill_fixes	2015-05-21 14:58:47 +01:00
Erik Johnston	ebfdd2eb5b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf	2015-05-21 14:54:52 +01:00
Erik Johnston	a551c5dad7	Merge pull request #155 from matrix-org/erikj/perf ... Bulk and batch retrieval of events.	2015-05-21 14:54:40 +01:00
Erik Johnston	27e4b45c06	s/for events/for requests for events/	2015-05-21 14:52:23 +01:00
Erik Johnston	ac5f2bf9db	s/for events/for requests for events/	2015-05-21 14:50:57 +01:00
Erik Johnston	80a167b1f0	Add comments	2015-05-21 11:19:04 +01:00
Erik Johnston	9118a92862	Split up _get_events into defer and txn versions	2015-05-20 13:27:16 +01:00
Erik Johnston	ab45e12d31	Make not return a deferred _get_event_from_row_txn	2015-05-20 13:07:19 +01:00
Erik Johnston	f407cbd2f1	PEP8	2015-05-20 13:02:01 +01:00
Erik Johnston	227f8ef031	Split out _get_event_from_row back into defer and _txn version	2015-05-20 13:00:57 +01:00
Erik Johnston	2bc60c55af	Fix _get_backfill_events to return events in the correct order	2015-05-20 12:57:00 +01:00
Erik Johnston	9084cdd70f	Ensure event_results is a set	2015-05-19 16:34:31 +01:00
Erik Johnston	aa729349dd	Fix event_backwards_extrem insertion to ignore outliers	2015-05-19 15:27:00 +01:00
Mark Haines	99914ec9f8	Merge pull request #152 from matrix-org/notifier_performance ... Notifier performance	2015-05-18 17:49:59 +01:00
Erik Johnston	165eb2dbe6	Comments and shuffle of functions	2015-05-18 15:18:41 +01:00
Erik Johnston	65878a2319	Remove unused metric	2015-05-18 14:06:30 +01:00
Erik Johnston	4d1b6f4ad1	Remove rejected events if we don't want rejected events	2015-05-18 14:03:46 +01:00
Mark Haines	0b0033c40b	Merge branch 'develop' into notifier_performance	2015-05-18 13:50:01 +01:00
Erik Johnston	10f1bdb9a2	Move get_events functions to storage.events	2015-05-18 10:21:40 +01:00
Erik Johnston	c71176858b	Newline, remove debug logging	2015-05-18 10:11:14 +01:00
Erik Johnston	c3b37abdfd	PEP8	2015-05-15 16:59:58 +01:00
Erik Johnston	6c74fd62a0	Revert limiting of fetching, it didn't help perf.	2015-05-15 16:45:35 +01:00
Erik Johnston	9ff7f66a2b	init j	2015-05-15 16:36:03 +01:00
Erik Johnston	70f272f71c	Don't completely drain the list	2015-05-15 16:34:17 +01:00
Erik Johnston	8763dd80ef	Don't fetch prev_content for current_state	2015-05-15 15:33:01 +01:00
Erik Johnston	acb12cc811	Make store.get_current_state fetch events asyncly	2015-05-15 15:20:05 +01:00
Erik Johnston	d62dee7eae	Remove more debug logging	2015-05-15 15:06:37 +01:00
Erik Johnston	0f29cfabc3	Remove debug logging	2015-05-15 14:06:42 +01:00
Erik Johnston	e275a9c0d9	preserve log context	2015-05-15 11:54:51 +01:00
Erik Johnston	aa32bd38e4	Add a wait	2015-05-15 11:35:04 +01:00
Erik Johnston	372d4c6d7b	Srsly. Don't use closures. Baaaaaad	2015-05-15 11:26:00 +01:00
Erik Johnston	575ec91d82	Correctly pass through params	2015-05-15 11:15:10 +01:00
Erik Johnston	de01438a57	Sort out error handling	2015-05-15 11:00:50 +01:00
Erik Johnston	a2c4f3f150	Fix daedlock	2015-05-15 10:54:04 +01:00
Erik Johnston	1d566edb81	Remove race condition	2015-05-14 16:54:35 +01:00
Erik Johnston	ef3d8754f5	Call from right thread	2015-05-14 15:41:55 +01:00
Erik Johnston	142934084a	Count and loop	2015-05-14 15:40:21 +01:00
Erik Johnston	96c5b9f87c	Don't start up more fetch_events	2015-05-14 15:36:04 +01:00
Erik Johnston	7cd6a6f6cf	Awful idea for speeding up fetching of events	2015-05-14 15:34:02 +01:00
Erik Johnston	7f4105a5c9	Turn off preemptive transactions	2015-05-14 14:51:06 +01:00
Erik Johnston	f4d58deba1	PEP8	2015-05-14 14:45:42 +01:00
Erik Johnston	386b7330d2	Move from _base to events	2015-05-14 14:45:22 +01:00
Erik Johnston	7d6a1dae31	Jump out early	2015-05-14 14:27:58 +01:00
Erik Johnston	656223fbd3	Actually, we probably want to run this in a transaction	2015-05-14 14:26:35 +01:00
Erik Johnston	2f7f8e1c2b	Preemptively jump into a transaction if we ask for get_prev_content	2015-05-14 14:17:36 +01:00
Erik Johnston	e1e9f0c5b2	loop -> gatherResults	2015-05-14 13:58:49 +01:00
Erik Johnston	ab78a8926e	Err, we probably want a bigger limit	2015-05-14 13:47:16 +01:00
Erik Johnston	f6f902d459	Move fetching of events into their own transactions	2015-05-14 13:45:48 +01:00
Erik Johnston	cdb3757942	Refactor _get_events	2015-05-14 13:31:55 +01:00
Erik Johnston	36ea26c5c0	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/perf	2015-05-14 12:01:38 +01:00
David Baker	7c549dd557	Add ID generator for push_rules_enable to #resolve SYN-378	2015-05-14 11:44:03 +01:00
Mark Haines	899d4675dd	Merge branch 'notifier_unify' into notifier_performance	2015-05-14 11:36:44 +01:00
Erik Johnston	4f1d984e56	Add index on events	2015-05-13 17:22:26 +01:00
Erik Johnston	968b01a91a	Actually use async method	2015-05-13 17:02:46 +01:00
Erik Johnston	4071f29653	Fetch events from events_id in their own transactions	2015-05-13 16:59:41 +01:00
Erik Johnston	a988361aea	Typo	2015-05-13 15:44:15 +01:00
Erik Johnston	8888982db3	Don't insert None	2015-05-13 15:43:32 +01:00
Erik Johnston	cf706cc6ef	Don't return None	2015-05-13 15:31:25 +01:00
Erik Johnston	5971d240d4	Limit batch size	2015-05-13 15:26:49 +01:00
Erik Johnston	ca4f458787	Fetch events in bulk	2015-05-13 15:13:42 +01:00
Erik Johnston	6edff11a88	Don't fetch redaction and rejection stuff for each event, so we can use index only scan	2015-05-13 14:39:05 +01:00
Mark Haines	63878c0379	Don't bother checking for updates if the stream token hasn't advanced for a user	2015-05-13 13:42:21 +01:00
Erik Johnston	02590c3e1d	Temp turn off checking for rejections and redactions	2015-05-13 11:31:28 +01:00
Erik Johnston	619a21812b	defer.gatherResults loop	2015-05-13 11:29:03 +01:00
Erik Johnston	fec4485e28	Batch fetching of events for state groups	2015-05-13 11:22:42 +01:00
Erik Johnston	409bcc76bd	Load events for state group seperately	2015-05-13 11:13:31 +01:00
Erik Johnston	e122685978	You need to call contextmanager	2015-05-12 16:12:37 +01:00
Mark Haines	a6fb2aa2a5	Merge pull request #144 from matrix-org/erikj/logging_context ... Preserving logging contexts	2015-05-12 15:23:50 +01:00
Mark Haines	1fce36b111	Merge pull request #149 from matrix-org/erikj/backfill ... Backfill support	2015-05-12 15:20:32 +01:00
Erik Johnston	8b28209c60	Err, delete the right stuff	2015-05-12 15:02:53 +01:00
Erik Johnston	30c72d377e	Newlines	2015-05-12 14:47:40 +01:00
Erik Johnston	e4eddf9b36	We do actually want to delete rows out of event_backward_extremities	2015-05-12 14:47:23 +01:00
Erik Johnston	c1779a79bc	Fix up _handle_prev_events to not try to insert duplicate rows	2015-05-12 14:41:50 +01:00
Erik Johnston	74850d7f75	Do state groups persistence /after/ checking if we have already persisted the event	2015-05-12 14:14:58 +01:00
Erik Johnston	da6a7bbdde	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/logging_context	2015-05-12 13:10:42 +01:00
Mark Haines	5002056b16	SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called from the main thread after the transaction completes, not from database thread before the transaction completes.	2015-05-12 11:20:40 +01:00
Erik Johnston	4df11b5039	Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards'	2015-05-12 10:28:10 +01:00
Erik Johnston	84e6b4001f	Initial hack at wiring together pagination and backfill	2015-05-11 18:01:31 +01:00
Erik Johnston	17653a5dfe	Move storage.stream._StreamToken to types.RoomStreamToken	2015-05-11 18:01:01 +01:00
Erik Johnston	b036596b75	Prefer to use _simple_*.	2015-05-11 11:24:01 +01:00
Erik Johnston	cd525c0f5a	push_rules table expects an 'id' field	2015-05-11 11:24:01 +01:00
Erik Johnston	22d2f498fa	Fix push rule bug: can't insert bool into small int column	2015-05-10 10:50:51 +01:00
Erik Johnston	476899295f	Change the way we do logging contexts so that they survive divergences	2015-05-08 16:32:18 +01:00
Mark Haines	1c7912751e	Drop the old table not the new table	2015-05-08 16:04:32 +01:00
Mark Haines	9d36eb4eab	Rename unique constraint	2015-05-08 16:01:55 +01:00
Mark Haines	b0f71db3ff	Remove unsigned	2015-05-08 15:59:51 +01:00
Mark Haines	84e1cacea4	Bump schema version	2015-05-08 15:58:14 +01:00
Mark Haines	6538d445e8	Make the timestamps in server_keys_json bigints	2015-05-08 15:55:17 +01:00
Erik Johnston	0c4ac271ca	Merge branch 'erikj/executemany' of github.com:matrix-org/synapse into erikj/SYN-371	2015-05-05 18:21:19 +01:00
Erik Johnston	0cf7e480b4	And use buffer(...) there as well	2015-05-05 18:20:01 +01:00
Erik Johnston	ed2584050f	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemany	2015-05-05 18:15:20 +01:00
Erik Johnston	977338a7af	Use buffer(...) when inserting into bytea column	2015-05-05 18:12:53 +01:00
Mark Haines	31049c4d72	Merge pull request #139 from matrix-org/bugs/SYN-369 ... Fix race with cache invalidation. SYN-369	2015-05-05 17:46:13 +01:00
Mark Haines	deb0237166	Add some doc-string	2015-05-05 17:45:11 +01:00
Mark Haines	d18f37e026	Collect the invalidate callbacks on the transaction object rather than passing around a separate list	2015-05-05 17:32:21 +01:00
Erik Johnston	9951542393	Add a comment about the zip(*[zip(sorted(...),...)])	2015-05-05 17:06:55 +01:00
Mark Haines	041b6cba61	SYN-369: Add comments to the sequence number logic in the cache	2015-05-05 16:32:44 +01:00
Mark Haines	63075118a5	Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access	2015-05-05 16:24:04 +01:00
Erik Johnston	531d7955fd	Don't insert without deduplication. In this case we never actually use this table, so simply remove the insert entirely	2015-05-05 16:12:28 +01:00
Mark Haines	bfa4a7f8b0	Invalidate the room_member cache if the current state events updates	2015-05-05 15:43:49 +01:00
Mark Haines	d0fece8d3c	Missing return for when the event was already persisted	2015-05-05 15:39:09 +01:00
Erik Johnston	bdcd7693c8	Fix indentation	2015-05-05 15:14:48 +01:00
Erik Johnston	43c2e8deae	Add support for using executemany	2015-05-05 15:13:25 +01:00
Erik Johnston	1692dc019d	Don't call 'encode_parameter' no-op	2015-05-05 15:00:30 +01:00
Mark Haines	a9aea68fd5	Invalidate the caches from the correct thread	2015-05-05 14:57:08 +01:00
Mark Haines	261d809a47	Sequence the modifications to the cache so that selects don't race with inserts	2015-05-05 14:13:50 +01:00
Erik Johnston	d9cc5de9e5	Correctly name transaction	2015-05-05 10:24:10 +01:00
Erik Johnston	b8940cd902	Remove some unused indexes	2015-05-01 16:14:25 +01:00
David Baker	eb9bd2d949	user_id now in user_threepids	2015-05-01 15:04:37 +01:00
David Baker	9eeb03c0dd	Don't use self.execute: it's designed for fetching stuff	2015-05-01 14:21:25 +01:00
David Baker	7693f24792	No id field on user	2015-05-01 13:55:42 +01:00
David Baker	92b20713d7	More missed get_user_by_id API changes	2015-05-01 13:45:54 +01:00
Erik Johnston	4011cf1c42	Cache latest_event_ids_in_room	2015-05-01 13:06:26 +01:00
Erik Johnston	657298cebd	Don't lock user_ips table for upsert.	2015-05-01 10:46:48 +01:00
Erik Johnston	fabb7acd45	Fix bug where we reconnected to the database on every query.	2015-05-01 10:24:24 +01:00
Erik Johnston	23c639ff32	Split a storage function in two so that we don't have to do extra work.	2015-05-01 10:17:19 +01:00
Erik Johnston	8be5284e91	Remove pointless join	2015-04-30 18:49:26 +01:00
Erik Johnston	503e4d3d52	Fix broken SQL	2015-04-30 18:44:47 +01:00
Erik Johnston	00718ae7a9	Need more yield	2015-04-30 18:43:39 +01:00
Erik Johnston	0465560c1a	Add missing param	2015-04-30 18:42:44 +01:00
Erik Johnston	61d05daab1	More join conditions	2015-04-30 18:41:05 +01:00
Erik Johnston	6ead27ddda	Add more conditions on JOINs to make postgres go a little faster.	2015-04-30 18:32:03 +01:00
Erik Johnston	f383d5a801	Fix up get_current_state and get_room_name_and_aliases queries to parse events in transaction	2015-04-30 17:12:52 +01:00
Erik Johnston	69d4063651	Add get_rooms_for_user cache	2015-04-30 16:47:51 +01:00
Erik Johnston	3c4c229788	Don't use sub queries, it makes postgres sad	2015-04-30 16:16:53 +01:00
Mark Haines	29400b45b9	SYN-367: Use upsert rather than insert_or_replace	2015-04-30 15:21:31 +01:00
Erik Johnston	2366d28780	Don't needlessly join on state_events	2015-04-30 14:02:06 +01:00
Erik Johnston	1d7702833d	Make simple query rather than long one and then throw away half the results	2015-04-30 10:16:12 +01:00
Erik Johnston	b1ca784aca	Correctly decode, for sqlite and postgres, rows from pushers table	2015-04-29 19:41:14 +01:00
Erik Johnston	4a9dc5b2f5	pushkey' are also bytes.	2015-04-29 19:27:02 +01:00
Erik Johnston	0ade2712d1	Typo	2015-04-29 19:17:25 +01:00
Erik Johnston	50f96f256f	Also remove yield from within lock in the other generator	2015-04-29 19:17:00 +01:00
Erik Johnston	d2d61a8288	Fix deadlock in id_generators. No idea why this was an actual deadlock.	2015-04-29 19:15:23 +01:00
Erik Johnston	3864b3a8e6	Actually return rows	2015-04-29 18:07:36 +01:00
Erik Johnston	0618978238	Typo, args wrong way round	2015-04-29 18:04:35 +01:00
Erik Johnston	09177f4f2e	Decode buffers in same thread	2015-04-29 18:03:42 +01:00
Erik Johnston	472be88674	We store pusher data as bytes	2015-04-29 17:43:46 +01:00
David Baker	12d381bd5d	Decode the data json in the storage layer (was moved but this part was missed)	2015-04-29 17:13:51 +01:00
David Baker	f8c30faf25	Oops, update the contraint too	2015-04-29 16:58:42 +01:00
David Baker	61cd5d9045	Be more postgressive	2015-04-29 16:57:14 +01:00
David Baker	fb95035a65	Be postgressive	2015-04-29 16:53:41 +01:00
David Baker	4669def000	Oops, forgot the schema delta file	2015-04-29 16:50:16 +01:00
Erik Johnston	0337eaf321	txn.execute doesn't return cursors	2015-04-29 16:43:39 +01:00
Erik Johnston	884fb88e28	txn.execute doesn't return cursors	2015-04-29 16:35:20 +01:00
Erik Johnston	d76c058eea	Fix invalid SQL to work in postgres land	2015-04-29 16:30:25 +01:00
Erik Johnston	c8ed9bd278	pushers table requires an access_token	2015-04-29 15:33:27 +01:00
Erik Johnston	970a9b9d2b	We can't use REPLACE when upgrading databases now we have postgres	2015-04-29 13:55:44 +01:00
Erik Johnston	64991b0c8b	Merge pull request #129 from matrix-org/key_distribution ... Key distribution v2	2015-04-29 13:34:38 +01:00
Mark Haines	e26a3d8d9e	bump database schema version	2015-04-29 13:32:32 +01:00
Mark Haines	a9549fdce3	Use bytea rather than BLOB	2015-04-29 13:16:09 +01:00
Mark Haines	4ad8b45155	Merge branch 'develop' into key_distribution ... Conflicts: synapse/config/homeserver.py	2015-04-29 13:15:14 +01:00
Erik Johnston	cd0864121b	Make postgres database error slightly more helpful	2015-04-29 12:12:25 +01:00
Erik Johnston	4932a7e2d9	Use __all__ instead of assert to stop pyflakes from warning about unused import in __init__	2015-04-29 12:12:25 +01:00
Erik Johnston	a08bf11138	Appease PEP8	2015-04-29 11:44:48 +01:00
Erik Johnston	204132a998	Check that postgres database has correct charset set	2015-04-29 11:42:28 +01:00
Erik Johnston	327ca883ec	Merge branch 'develop' of github.com:matrix-org/synapse into postgres	2015-04-28 13:39:42 +01:00
Erik Johnston	18f8247701	Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit	2015-04-28 12:41:33 +01:00
Mark Haines	9182f87664	Merge pull request #126 from matrix-org/csauth ... Client / Server Auth Refactor	2015-04-28 11:00:27 +01:00
Mark Haines	55e1bc8920	And don't bump the schema version unnecessarily	2015-04-28 10:54:15 +01:00
Mark Haines	b96c133034	Add server_keys.sql to the current delta rather than creating a new delta	2015-04-28 10:50:00 +01:00
Erik Johnston	8558e1ec73	Make get_max_token into inlineCallbacks so that the lock works.	2015-04-27 15:19:44 +01:00
Erik Johnston	6f8e2d517e	Merge branch 'develop' of github.com:matrix-org/synapse into postgres	2015-04-27 14:41:40 +01:00
Erik Johnston	2732be83d9	Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible.	2015-04-27 13:22:30 +01:00
Erik Johnston	e4c4664d73	Handle the fact that postgres databases can be restarted from under us	2015-04-27 12:40:49 +01:00
David Baker	8a5be236e0	pep8	2015-04-27 11:49:18 +01:00
David Baker	b02e1006b9	Run database check before daemonizing, at the cost of database hygiene.	2015-04-27 11:46:00 +01:00
David Baker	f8152f2708	rename db method to be more informative	2015-04-27 10:16:26 +01:00
David Baker	2f475bd5d5	pep8	2015-04-24 18:15:07 +01:00
David Baker	a7b51f4539	Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266	2015-04-24 18:11:21 +01:00
Mark Haines	eede182df7	Merge branch 'develop' into key_distribution	2015-04-24 10:35:49 +01:00
Mark Haines	4e2f8b8722	Copyright notices	2015-04-24 10:35:29 +01:00
Mark Haines	f30d47c876	Implement remote key lookup api	2015-04-22 14:21:08 +01:00
David Baker	91c8f828e1	pep8	2015-04-17 19:56:04 +01:00
David Baker	8db6832db8	Password reset, finally.	2015-04-17 19:53:47 +01:00
David Baker	117f35ac4a	Add endpoint to get threepids from server	2015-04-17 17:20:18 +01:00
David Baker	4eea5cf6c2	pep8	2015-04-17 16:46:45 +01:00
David Baker	f96ab9d18d	make add3pid servlet work	2015-04-17 16:44:49 +01:00
Erik Johnston	e3417bbbe0	Revert needless change to storage.profile	2015-04-17 16:13:14 +01:00
David Baker	cb03fafdf1	Merge branch 'develop' into csauth	2015-04-17 13:51:10 +01:00
Erik Johnston	0ec346d942	Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea	2015-04-17 11:10:20 +01:00
Erik Johnston	e1c0970c11	PEP8	2015-04-16 11:18:45 +01:00
Erik Johnston	b8092fbc82	Go back to storing JSON in TEXT	2015-04-16 11:17:52 +01:00
Erik Johnston	bc9e69e160	Move encoding and decoding of JSON into storage layer	2015-04-16 11:01:09 +01:00
Erik Johnston	ffad75bd62	Remove mysql/maria support	2015-04-15 17:00:50 +01:00
Mark Haines	a429515bdd	Add methods for storing and retrieving the raw key json	2015-04-15 16:58:35 +01:00
Erik Johnston	cf04cedf21	Change full_schemas/11 to work with postgres	2015-04-15 16:53:47 +01:00
Erik Johnston	5b31afcbd1	Remove debug logging	2015-04-15 16:27:04 +01:00
Erik Johnston	6e91f14d09	Add missing yield	2015-04-15 16:25:07 +01:00
Erik Johnston	ed26e4012b	pushers table requires a unique id.	2015-04-15 16:24:14 +01:00
Erik Johnston	a5c72780e6	Don't pass in removed flag	2015-04-15 15:13:22 +01:00
Erik Johnston	4af32a2817	Postgres does not allow you to continue using a cursor after a DB exception has been raised, so move _simple_insert or_ignore flag out of transaction	2015-04-15 14:51:21 +01:00
Erik Johnston	25d2914fba	Turn off persistance of sent_transactions.response_json	2015-04-15 14:09:35 +01:00
Erik Johnston	cce5d057d3	Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses	2015-04-15 11:43:25 +01:00
Erik Johnston	a971fa9d58	Use try..finally in contextlib.contextmanager	2015-04-15 10:25:43 +01:00
Erik Johnston	ded4128965	Use True for True rather than 1	2015-04-15 10:24:24 +01:00
Erik Johnston	f9e12f79ca	Add missing yield in storage func	2015-04-15 10:24:07 +01:00
Erik Johnston	c756dfeb14	Correctly identify deadlocks	2015-04-15 10:23:42 +01:00
Mark Haines	32e14d8181	Return a sha256 fingerprint rather than the entire tls certificate	2015-04-14 19:10:09 +01:00
Erik Johnston	127fad17dd	Add postgres database engine	2015-04-14 14:50:29 +01:00
Erik Johnston	5a95cd4442	Rename user_ips.user -> user_id	2015-04-14 13:54:09 +01:00
Erik Johnston	58d8339966	Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS.	2015-04-14 13:53:20 +01:00
Erik Johnston	3c741682e5	Correctly increment the _next_id initially	2015-04-14 09:54:44 +01:00
Erik Johnston	86fc9b617c	For backwards compat, make state_groups.id have a type of int, not varchar	2015-04-13 17:03:49 +01:00
Erik Johnston	93937b2b31	Remove duplicate rows	2015-04-10 18:47:03 +01:00
Erik Johnston	c5365dee56	Use case sensitive collations	2015-04-10 18:46:33 +01:00
Erik Johnston	4103b1c470	DROP indexes before recreating them	2015-04-10 18:46:09 +01:00
Erik Johnston	4d5b098626	Use LONGBLOB and TEXT for arbitary length rows	2015-04-10 18:45:09 +01:00
Erik Johnston	7ed2ec3061	Handle the fact that in sqlite binary data might be stored as unicode or bytes	2015-04-10 13:41:54 +01:00
Erik Johnston	ce797ad373	Bump schema version	2015-04-10 11:22:30 +01:00
Erik Johnston	7e863c51e6	Use unsigned bigint	2015-04-10 11:22:04 +01:00
Erik Johnston	0f12772e32	SQLite wants INTEGER and not BIGINT for primary keys	2015-04-10 11:16:09 +01:00
Erik Johnston	d5d4281647	Update full_schemas/16 to match delta files. Add delta/16 scripts	2015-04-10 10:59:46 +01:00
Erik Johnston	cda4a6f93f	Revert non-trivial changes to upgrade scripts	2015-04-10 10:19:50 +01:00
Erik Johnston	e2722f58ee	Fix schema again	2015-04-10 10:16:29 +01:00
Erik Johnston	a1665c5094	Revert non-trivial schema changes and move them to a new schema version.	2015-04-10 10:05:44 +01:00
Erik Johnston	2ded344620	Remove unused import	2015-04-09 13:46:06 +01:00
Erik Johnston	9707acfc40	Remove spurious spaces	2015-04-09 13:45:20 +01:00
Erik Johnston	8bf285e082	Merge branch 'develop' of github.com:matrix-org/synapse into mysql	2015-04-09 13:13:26 +01:00
Erik Johnston	8ad0f4912e	Stream ordering and out of order insertions. ... Handle the fact that events can be persisted out of order, and so to get the "current max" stream token becomes non trivial - as we need to make sure that *all* stream tokens less than the current max have also successfully been persisted.	2015-04-09 11:41:36 +01:00
Erik Johnston	279a547a8b	Use generic db exceptions rather than sqlite3 specific ones	2015-04-08 16:53:48 +01:00
Mark Haines	83f5125d52	Merge pull request #114 from matrix-org/improve_get_event_cache ... Improve get event cache	2015-04-08 16:50:27 +01:00
Erik Johnston	3887350e47	Merge branch 'develop' of github.com:matrix-org/synapse into mysql	2015-04-08 14:11:07 +01:00
Erik Johnston	c1b34af441	Move database timer logging to seperate logger	2015-04-08 13:31:19 +01:00
Erik Johnston	9a05795619	Retry transaction, not SQL query	2015-04-08 13:11:28 +01:00
Erik Johnston	24d8134ac1	Fix maria engine to correctly recognize deadlocks	2015-04-08 13:10:54 +01:00
Erik Johnston	4fe95094d1	Merge branch 'develop' of github.com:matrix-org/synapse into mysql	2015-04-07 18:05:39 +01:00
Erik Johnston	49d6aa1394	Retry on deadlock	2015-04-07 15:28:37 +01:00
Erik Johnston	0bfa78b39b	PEP8	2015-04-07 12:16:05 +01:00
Erik Johnston	6bc9edd8b2	Fix prepare_sqlite3_database's convert_param_style	2015-04-07 12:13:58 +01:00
Erik Johnston	05a35d62b6	Bump database version	2015-04-07 12:10:15 +01:00
Erik Johnston	8574bf62dc	Add index to presence table	2015-04-07 12:09:36 +01:00
Erik Johnston	0af5f5efaf	Don't use multiple UNIQUE constraints; it will cause deadlocks	2015-04-07 12:08:35 +01:00
Erik Johnston	c8d3f6486d	Implement or_ignore flag on inserts	2015-04-07 12:06:22 +01:00
Erik Johnston	304111afd0	Don't use AUTOINCREMENT, use an in memory version	2015-04-07 12:05:36 +01:00