Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-20 23:54:19 -05:00
Code Issues Packages Projects Releases Wiki Activity
4,753 Commits 4 Branches 609 Tags 60 MiB
234d6f9f3e
Commit Graph

3110 Commits

Author SHA1 Message Date
Erik Johnston
b2c7bd4b09 Cache get_recent_events_for_room 2015-08-07 14:42:34 +01:00
Erik Johnston
b3768ec10a Remove unncessary cache 2015-08-07 13:41:05 +01:00
Erik Johnston
b8e386db59 Change Cache to not use *args in its interface 2015-08-07 11:52:21 +01:00
Erik Johnston
fe994e728f Store absence of state in cache 2015-08-07 10:17:38 +01:00
Erik Johnston
1d08bf7c17 Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 14:03:15 +01:00
Erik Johnston
63b1eaf32c Docs 2015-08-06 14:02:50 +01:00
Erik Johnston
b811c98574 Remove failed deferreds from cache 2015-08-06 14:01:27 +01:00
Erik Johnston
433314cc34 Re-implement DEBUG_CACHES flag 2015-08-06 14:01:05 +01:00
Erik Johnston
f596ff402e Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 13:37:56 +01:00
Erik Johnston
2efb93af52 Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds 2015-08-06 13:35:28 +01:00
Erik Johnston
953dbd28a7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cached_keyword_args 2015-08-06 13:35:03 +01:00
Erik Johnston
7eea3e356f Make @cached cache deferreds rather than the deferreds' values 2015-08-06 13:33:34 +01:00
Erik Johnston
3e1b77efc2 Merge branch 'erikj/cached_keyword_args' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-05 16:45:56 +01:00
Erik Johnston
b52b4a84ec Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-05 15:41:20 +01:00
Erik Johnston
1e62a3d3a9 Up the cache size for 'get_joined_hosts_for_room' and 'get_users_in_room' 2015-08-05 15:40:40 +01:00
Erik Johnston
a89559d797 Use LRU cache by default 2015-08-05 15:39:47 +01:00
Erik Johnston
07507643cb Use dictionary cache to do group -> state fetching 2015-08-05 15:11:42 +01:00
David Baker
185ac7ee6c Allow sign in using email address 2015-08-04 16:29:54 +01:00
David Baker
a0dea6eaed Remember to yield: not much point testing is a deferred is not None 2015-08-04 16:18:17 +01:00
Erik Johnston
c67ba143fa Move DictionaryCache 2015-08-04 15:58:28 +01:00
Erik Johnston
e7768e77f5 Add basic dictionary cache 2015-08-04 15:56:56 +01:00
David Baker
883aabe423 splt long line 2015-08-04 15:20:35 +01:00
David Baker
e124128542 Bump schema version 2015-08-04 14:50:31 +01:00
David Baker
c77048e12f Add endpoint that proxies ID server request token and errors if the given email is in use on this Home Server. 2015-08-04 14:37:09 +01:00
Erik Johnston
2e35a733cc Merge branch 'develop' of github.com:matrix-org/synapse into erikj/acl_perf 2015-08-04 13:00:52 +01:00
Erik Johnston
413a4c289b Add comment 2015-08-04 11:08:07 +01:00
Erik Johnston
4d6cb8814e Speed up event filtering (for ACL) logic 2015-08-04 09:32:23 +01:00
David Baker
7148aaf5d0 Don't try & check the username if we don't have one (which we won't if it's been saved in the auth layer) 2015-08-03 17:03:27 +01:00
David Baker
28d07a02e4 Add vector.im as trusted ID server 2015-08-03 15:31:21 +01:00
Kegan Dougal
11b0a34074 Use the same reg paths as register v1 for ASes. 
Namely this means using registration_handler.appservice_register.
 2015-07-29 10:00:54 +01:00
Kegan Dougal
a4d62ba36a Fix v2_alpha registration. Add unit tests. 
V2 Registration forced everyone (including ASes) to create a password for a
user, when ASes should be able to omit passwords. Also unbreak AS registration
in general which checked too early if the given username was claimed by an AS;
it was checked before knowing if the AS was the one doing the registration! Add
unit tests for AS reg, user reg and disabled_registration flag.
 2015-07-28 17:34:12 +01:00
Erik Johnston
39e21ea51c Add support for using keyword arguments with cached functions 2015-07-27 13:57:29 +01:00
Mark Haines
2da3b1e60b Get the end-to-end key federation working 2015-07-24 18:26:46 +01:00
Mark Haines
62c010283d Add federation support for end-to-end key requests 2015-07-23 16:03:38 +01:00
Erik Johnston
459085184c Factor out thumbnail() 2015-07-23 15:59:53 +01:00
Erik Johnston
2b4f47db9c Generate local thumbnails on a thread 2015-07-23 14:52:29 +01:00
Erik Johnston
33d83f3615 Fix remote thumbnailing 2015-07-23 14:24:21 +01:00
Erik Johnston
ff7c2e41de Always return a thumbnail of the requested size. 
Before, we returned a thumbnail that was at least as big (if possible)
as the requested size. Now, if we don't have a thumbnail of the given
size we generate (and persist) one of that size.
 2015-07-23 14:12:49 +01:00
Erik Johnston
103e1c2431 Pick larger than desired thumbnail for 'crop' 2015-07-23 11:12:49 +01:00
Matrix
4e2e67fd50 Disable receipts for now 2015-07-22 16:13:46 +01:00
David Baker
a56eccbbfc Query for all the ones we were asked about, not just the last... 2015-07-21 16:38:16 -07:00
David Baker
20c0324e9c Dodesn't seem to make any difference: guess it does work with the object reference 2015-07-21 16:21:37 -07:00
David Baker
cf7a40b08a I think this was what was intended... 2015-07-21 16:08:00 -07:00
Erik Johnston
90dbd71c13 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-07-21 09:25:30 +01:00
Mark Haines
3b5823c74d s/take/claim/ for end to end key APIs 2015-07-20 18:23:54 +01:00
Daniel Wagner-Hall
53d1174aa9 Improve naming 2015-07-20 06:32:12 -07:00
Kegan Dougal
ddef5ea126 Remove semicolon. 2015-07-20 14:02:36 +01:00
Kegan Dougal
b6ee0585bd Parse the ID given to /invite|ban|kick to make sure it looks like a user ID. 2015-07-20 13:55:19 +01:00
Matrix
4f973eb657 Up default cache size for _RoomStreamChangeCache 2015-07-18 19:07:33 +01:00
Matrix
4cab2cfa34 Don't do any database hits in receipt handling if from_key == to_key 2015-07-18 19:07:12 +01:00
Erik Johnston
b6d4a4c6d8 Merge pull request #199 from matrix-org/erikj/receipts 
Implement read receipts.
 2015-07-16 18:18:36 +01:00
Erik Johnston
d155b318d2 Merge pull request #203 from matrix-org/erikj/room_creation_presets 
Implement presets at room creation
 2015-07-16 18:18:11 +01:00
Erik Johnston
a2ed7f437c Merge pull request #202 from matrix-org/erikj/power_level_sanity 
Change power level semantics.
 2015-07-16 18:18:04 +01:00
Erik Johnston
c456d17daf Implement specifying custom initial state for /createRoom 2015-07-16 15:25:29 +01:00
David Baker
09489499e7 pep8 + debug line 2015-07-15 19:39:18 +01:00
David Baker
4da05fa0ae Add back in support for remembering parameters submitted to a user-interactive auth call. 2015-07-15 19:28:57 +01:00
Matthew Hodgson
8cedf3ce95 bump up image quality a bit more as it looks crap 2015-07-14 23:53:13 +01:00
Erik Johnston
baa55fb69e Merge pull request #193 from matrix-org/erikj/bulk_persist_event 
Add bulk insert events API
 2015-07-14 10:49:24 +01:00
Erik Johnston
002a44ac1a s/everyone_ops/original_invitees_have_ops/ 2015-07-14 10:37:42 +01:00
David Baker
62b4b72fe4 Close, but no cigar. 2015-07-14 10:33:25 +01:00
Erik Johnston
b49a30a972 Capitalize contants 2015-07-14 10:20:31 +01:00
Erik Johnston
4624d6035e Docs 2015-07-14 10:19:07 +01:00
Erik Johnston
d5cc794598 Implement presets at room creation 2015-07-13 16:56:08 +01:00
Erik Johnston
5989637f37 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-13 13:50:57 +01:00
Erik Johnston
016c089f13 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/power_level_sanity 2015-07-13 13:48:13 +01:00
Erik Johnston
e5991af629 Comments 2015-07-13 13:30:43 +01:00
Erik Johnston
17bb9a7eb9 Remove commented out code 2015-07-10 14:07:57 +01:00
Erik Johnston
a5ea22d468 Sanitize power level checks 2015-07-10 14:05:38 +01:00
Erik Johnston
7e3b14fe78 You shouldn't be able to ban/kick users with higher power levels 2015-07-10 14:05:38 +01:00
Erik Johnston
532fcc997a Merge pull request #196 from matrix-org/erikj/room_history 
Add ability to restrict room history.
 2015-07-10 13:47:04 +01:00
Erik Johnston
0b3389bcd2 Merge pull request #194 from matrix-org/erikj/bulk_verify_sigs 
Implement bulk verify_signed_json API
 2015-07-10 13:46:53 +01:00
Erik Johnston
0d7f0febf4 Uniquely name unique constraint 2015-07-10 13:43:03 +01:00
Erik Johnston
b7cb37b189 Merge pull request #198 from matrix-org/markjh/client-end-to-end-key-management 
Client end to end key management API
 2015-07-10 13:36:17 +01:00
Mark Haines
a01097d60b Assume that each device for a user has only one of each type of key 2015-07-10 13:26:18 +01:00
Erik Johnston
f3049d0b81 Small tweaks to SAML2 configuration. 
- Add saml2 config docs to default config.
- Use existence of saml2 config to indicate if saml2 should be enabled.
 2015-07-10 10:50:14 +01:00
Erik Johnston
9158ad1abb Merge pull request #201 from EricssonResearch/msba/saml2-develop 
Integrate SAML2 basic authentication - uses pysaml2
 2015-07-10 10:25:56 +01:00
Erik Johnston
b5f0d73ea3 Add comment 2015-07-09 17:09:26 +01:00
Erik Johnston
ed88720952 Handle error slightly better 2015-07-09 16:14:46 +01:00
Erik Johnston
f0979afdb0 Remove spurious comment 2015-07-09 16:02:07 +01:00
Mark Haines
bf0d59ed30 Don't bother with a timeout for one time keys on the server. 2015-07-09 14:04:03 +01:00
Erik Johnston
c2d08ca62a Integer timestamps 2015-07-09 13:15:34 +01:00
Erik Johnston
4019b48aaa Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-09 11:55:52 +01:00
Erik Johnston
294dbd712f We don't want semicolons. 2015-07-09 11:47:24 +01:00
Erik Johnston
1af188209a Change format of receipts to allow inclusion of data 2015-07-09 11:39:30 +01:00
Muthu Subramanian
8cd34dfe95 Make SAML2 optional and add some references/comments 2015-07-09 13:34:47 +05:30
Muthu Subramanian
d2caa5351a code beautify 2015-07-09 12:58:15 +05:30
Matthew Hodgson
fb8d2862c1 remove the tls_certificate_chain_path param and simply support tls_certificate_path pointing to a file containing a chain of certificates 2015-07-09 00:45:41 +01:00
Matthew Hodgson
8ad2d2d1cb document tls_certificate_chain_path more clearly 2015-07-09 00:06:01 +01:00
Matthew Hodgson
f26a3df1bf oops, context.tls_certificate_chain_file() expects a file, not a certificate. 2015-07-08 21:33:02 +01:00
Matthew Hodgson
19fa3731ae typo 2015-07-08 18:53:41 +01:00
Matthew Hodgson
465acb0c6a *cough* 2015-07-08 18:30:59 +01:00
Matthew Hodgson
64afbe6ccd add new optional config for tls_certificate_chain_path for folks with intermediary SSL certs 2015-07-08 18:20:02 +01:00
Matthew Hodgson
04192ee05b typo 2015-07-08 17:49:15 +01:00
Mark Haines
8fb79eeea4 Only remove one time keys when new one time keys are added 2015-07-08 17:04:29 +01:00
Erik Johnston
ce9e2f84ad Add blist to dependencies 2015-07-08 15:41:59 +01:00
Erik Johnston
304343f4d7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-08 15:37:33 +01:00
Erik Johnston
af812b68dd Add a cache to fetching of receipt streams 2015-07-08 15:35:00 +01:00
Erik Johnston
d85ce8d89b Split receipt events up into one per room 2015-07-08 11:36:05 +01:00
Muthu Subramanian
f53bae0c19 code beautify 2015-07-08 16:05:46 +05:30
Muthu Subramanian
77c5db5977 code beautify 2015-07-08 16:05:20 +05:30
Muthu Subramanian
81682d0f82 Integrate SAML2 basic authentication - uses pysaml2 2015-07-08 15:36:54 +05:30
Erik Johnston
87311d1b8c Hook up receipts to v1 initialSync 2015-07-08 11:02:04 +01:00
Erik Johnston
ca041d5526 Wire together receipts and the notifer/federation 2015-07-07 15:25:30 +01:00
Erik Johnston
716e426933 Fix various typos 2015-07-07 10:55:31 +01:00
Erik Johnston
e8b2f6f8a1 Add a ReceiptServlet 2015-07-07 10:55:22 +01:00
Mark Haines
28ef344077 Merge branch 'mjark/missing_regex_group' into markjh/client-end-to-end-key-management 2015-07-06 18:48:27 +01:00
Mark Haines
2ef182ee93 Add client API for uploading and querying keys for end to end encryption 2015-07-06 18:47:57 +01:00
Mark Haines
b5770f8947 Add store for client end to end keys 2015-07-06 18:46:47 +01:00
Mark Haines
a7dcbfe430 Don't 500 if a group is missing from the regex 2015-07-06 16:47:17 +01:00
Erik Johnston
1a3255b507 Add m.room.history_visibility to newly created rooms' m.room.power_levels 2015-07-06 13:25:35 +01:00
Erik Johnston
fb47c3cfbe Rename key and values for m.room.history_visibility. Support 'invited' value 2015-07-06 13:05:52 +01:00
Erik Johnston
65e69dec8b Don't explode if we don't recognize one of the event_ids in the backfill request 2015-07-06 09:33:03 +01:00
Erik Johnston
c3e2600c67 Filter and redact events that the other server doesn't have permission to see during backfill 2015-07-03 17:52:57 +01:00
Erik Johnston
400894616d Respect m.room.history_visibility in v2_alpha sync API 2015-07-03 14:51:01 +01:00
Erik Johnston
12b83f1a0d If user supplies filename in URL when downloading from media repo, use that name in Content Disposition 2015-07-03 11:24:55 +01:00
Erik Johnston
00ab882ed6 Add m.room.history_visibility to list of auth events 2015-07-03 10:31:24 +01:00
Erik Johnston
41938afed8 Make v1 initial syncs respect room history ACL 2015-07-02 17:12:35 +01:00
Erik Johnston
1a60545626 Add basic impl for room history ACL on GET /messages client API 2015-07-02 16:20:10 +01:00
Erik Johnston
ac78e60de6 Add stream_id index 2015-07-02 13:18:41 +01:00
Erik Johnston
bd1236c0ee Consolidate duplicate code in notifier 2015-07-02 11:46:05 +01:00
Erik Johnston
ddf7979531 Add receipts_key to StreamToken 2015-07-02 11:45:44 +01:00
Erik Johnston
0862fed2a8 Add basic ReceiptHandler 2015-07-01 17:19:31 +01:00
Erik Johnston
80a61330ee Add basic storage functions for handling of receipts 2015-07-01 17:19:12 +01:00
Erik Johnston
480d720388 Bump changelog and version to v0.9.3 2015-07-01 15:12:32 +01:00
Erik Johnston
901f56fa63 Add tables for receipts 2015-06-30 15:29:47 +01:00
Erik Johnston
9beaedd164 Enforce ascii filenames for uploads 2015-06-30 10:31:59 +01:00
Erik Johnston
2124f668db Add Content-Disposition headers to media repo v1 downloads 2015-06-30 09:35:44 +01:00
Erik Johnston
f0dd568e16 Wait for previous attempts at fetching keys for a given server before trying to fetch more 2015-06-26 11:25:00 +01:00
Erik Johnston
b5f55a1d85 Implement bulk verify_signed_json API 2015-06-26 10:39:34 +01:00
Erik Johnston
5130d80d79 Add bulk insert events API 2015-06-25 17:29:34 +01:00
David Baker
6825eef955 Oops: underride rule had an identifier with override in it. 2015-06-23 14:26:14 +01:00
Erik Johnston
6924852592 Batch SELECTs in _get_auth_chain_ids_txn 2015-06-23 11:01:04 +01:00
Erik Johnston
9c72011fd7 Bumb version 2015-06-23 10:12:19 +01:00
Erik Johnston
2f556e0c55 Fix typo 2015-06-19 16:22:53 +01:00
Erik Johnston
7fa1363fb0 Merge pull request #192 from matrix-org/erikj/fix_log_context 
Fix log context when sending requests
 2015-06-19 16:21:40 +01:00
Erik Johnston
275dab6b55 Merge pull request #190 from matrix-org/erikj/syn-412 
Fix notifier leak
 2015-06-19 11:49:58 +01:00
Erik Johnston
a68abc79fd Add comment on cancellation of observers 2015-06-19 11:48:55 +01:00
Erik Johnston
653533a3da Fix log context when sending requests 2015-06-19 11:46:49 +01:00
Erik Johnston
9bf61ef97b Merge pull request #189 from matrix-org/erikj/room_init_sync 
Improve room init sync speed.
 2015-06-19 11:36:06 +01:00
Erik Johnston
0e58d19163 Merge pull request #187 from matrix-org/erikj/sanitize_logging 
Sanitize logging
 2015-06-19 11:35:59 +01:00
Erik Johnston
18968efa0a Remove stale debug lines 2015-06-19 10:18:02 +01:00
Erik Johnston
eb928c9f52 Add site_tag to logger 2015-06-19 10:16:48 +01:00
Erik Johnston
9d112f4440 Add IDs to outbound transactions 2015-06-19 10:13:03 +01:00
Eric Myhre
9e5a353663 Make upload dir a configurable path. 
Fixes SYN-425.

Signed-off-by: Eric Myhre <hash@exultant.us>
 2015-06-18 23:38:20 -05:00
Erik Johnston
6f6ebd216d PEP8 2015-06-18 17:00:32 +01:00
Erik Johnston
73513ececc Documentation 2015-06-18 16:15:10 +01:00
Erik Johnston
1f24c2e589 Don't bother proxying lookups on _NotificationListener to underlying deferred 2015-06-18 16:09:53 +01:00
Erik Johnston
22049ea700 Refactor the notifier.wait_for_events code to be clearer. Add _NotifierUserStream.new_listener that accpets a token to avoid races. 2015-06-18 15:49:24 +01:00
Erik Johnston
050ebccf30 Fix notifier leak 2015-06-18 11:36:26 +01:00
Kegan Dougal
d88e20cdb9 Fix bug where synapse was sending AS user queries incorrectly. 
Bug introduced in 92b20713d7
which reversed the comparison when checking if a user existed
in the users table. Added UTs to prevent this happening again.
 2015-06-17 17:26:03 +01:00
Erik Johnston
eceb554a2f Use another deferred list 2015-06-16 17:12:27 +01:00
Erik Johnston
b849a64f8d Use DeferredList 2015-06-16 17:03:24 +01:00
Erik Johnston
0460406298 Don't do unecessary db ops in presence.get_state 2015-06-16 16:59:38 +01:00
Paul "LeoNerd" Evans
9a3cd1c00d Correct -H SERVER_NAME in config-missing complaint message 2015-06-16 16:03:35 +01:00
Erik Johnston
fb7def3344 Remove access_token from synapse.rest.client.v1.transactions {get,store}_response logging 2015-06-16 10:09:43 +01:00
Erik Johnston
f13890ddce Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sanitize_logging 2015-06-15 18:26:24 +01:00
Erik Johnston
aaa749d366 Disable twisted access logging. Move access logging to SynapseRequest object 2015-06-15 18:18:05 +01:00
Erik Johnston
bc42ca121f Merge pull request #185 from matrix-org/erikj/listeners_config 
Change listener config.
 2015-06-15 18:05:58 +01:00
Erik Johnston
cee69441d3 Log more when we have processed the request 2015-06-15 17:11:44 +01:00
Erik Johnston
b5209c5744 Create SynapseRequest that overrides __repr__ to not print access_token 2015-06-15 16:37:04 +01:00
Erik Johnston
44c9102e7a Merge branch 'erikj/listeners_config' into erikj/sanitize_logging 2015-06-15 14:45:52 +01:00
Erik Johnston
7f7ec84d6f Bump version 2015-06-15 14:16:29 +01:00
Erik Johnston
f00f8346f1 Make http.server request logging more verbose, but redact access_tokens 2015-06-15 13:37:58 +01:00
Erik Johnston
83f119a84a Log requests and responses sent via http.client 2015-06-15 13:14:12 +01:00
Erik Johnston
9d0326baa6 Remove redundant newline 2015-06-15 11:27:29 +01:00
Erik Johnston
186f61a3ac Document listener config. Remove deprecated config options 2015-06-15 11:25:53 +01:00
Erik Johnston
6c01ceb8d0 Bump version 2015-06-13 12:22:14 +01:00
Erik Johnston
2eda996a63 Add a dummy.sql into delta/20 as pip isn't packinging the pushers.py 2015-06-13 12:21:58 +01:00
Erik Johnston
a005b7269a Add backwards compat support for metrics, manhole and webclient config options 2015-06-12 17:44:23 +01:00
Erik Johnston
942e39e87c PEP8 2015-06-12 17:13:54 +01:00
Erik Johnston
9c5fc81c2d Correctly handle x_forwaded listener option 2015-06-12 17:13:23 +01:00
Erik Johnston
fd2c07bfed Use config.listeners 2015-06-12 15:33:07 +01:00
Erik Johnston
c42ed47660 Fix up create_resource_tree 2015-06-12 11:52:52 +01:00
Erik Johnston
1e365e88bd Bump schema version 2015-06-11 15:50:39 +01:00
Erik Johnston
2d41dc0069 Bump version 2015-06-11 15:49:19 +01:00
Erik Johnston
f7f07dc517 Begin changing the config format 2015-06-11 15:48:52 +01:00
David Baker
b8690dd840 Catch any exceptions in the pusher loop. Use a lower timeout for pushers so we can see if they're actually still running. 2015-06-05 11:40:22 +01:00
David Baker
da84946de4 pep8 2015-06-04 16:43:45 +01:00
David Baker
63a7b3ad1e Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them. 2015-06-04 16:16:01 +01:00
Erik Johnston
5730b20c6d Merge pull request #175 from matrix-org/erikj/thumbnail_thread 
Thumbnail images on a seperate thread
 2015-06-03 17:26:56 +01:00
Erik Johnston
8047fd2434 Merge pull request #176 from matrix-org/erikj/backfill_auth 
Improve backfill.
 2015-06-03 17:25:37 +01:00
Erik Johnston
3bbd0d0e09 Merge pull request #180 from matrix-org/erikj/prev_state_context 
Don't needlessly compute prev_state
 2015-06-03 17:20:56 +01:00
Erik Johnston
9dda396baa Merge pull request #179 from matrix-org/erikj/state_group_outliers 
Don't compute EventContext for outliers.
 2015-06-03 17:20:40 +01:00
Erik Johnston
13ed3b9985 Merge pull request #178 from matrix-org/erikj/cache_state_groups 
Add cache to get_state_groups.
 2015-06-03 17:20:33 +01:00
Erik Johnston
bd2cf9d4bf Merge pull request #177 from matrix-org/erikj/content_repo_http_client 
SYN-403: Make content repository use its own http client.
 2015-06-03 17:20:27 +01:00
Erik Johnston
d4902a7ad0 Merge pull request #174 from matrix-org/erikj/compress_option 
Add config option to disable compression of http responses
 2015-06-03 17:18:17 +01:00
Erik Johnston
55bf90b9e4 Don't needlessly compute prev_state 2015-06-03 16:44:24 +01:00
Erik Johnston
53f0bf85d7 Comment 2015-06-03 16:43:40 +01:00
Erik Johnston
1c3d844e73 Don't needlessly compute context 2015-06-03 16:41:51 +01:00
Erik Johnston
0d7d9c37b6 Add cache to get_state_groups 2015-06-03 14:45:55 +01:00
Erik Johnston
d8866d7277 Caches should be bound to instances. 
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
 2015-06-03 14:45:17 +01:00
Erik Johnston
2ef2f6d593 SYN-403: Make content repository use its own http client. 2015-06-03 10:17:37 +01:00
Erik Johnston
3483b78d1a Log where a request came from in federation 2015-06-02 18:15:13 +01:00
Erik Johnston
d3ded420b1 Rephrase log line 2015-06-02 16:30:52 +01:00
Erik Johnston
22716774d5 Don't about JSON when warning about content tampering 2015-06-02 16:30:52 +01:00
Erik Johnston
5044e6c544 Thumbnail images on a seperate thread 2015-06-02 15:39:08 +01:00
Erik Johnston
09e23334de Add a timeout 2015-06-02 11:00:37 +01:00
Erik Johnston
02410e9239 Handle the fact we might be missing auth events 2015-06-02 10:58:35 +01:00
Erik Johnston
e552b78d50 Add some logging 2015-06-02 10:28:14 +01:00
Erik Johnston
fde0da6f19 Correctly look up auth_events 2015-06-02 10:19:38 +01:00
Erik Johnston
3f04a08a0c Don't process events we've already processed. Remember to process state events 2015-06-02 10:11:32 +01:00
Erik Johnston
4bbfbf898e Correctly pass in auth_events 2015-06-01 17:02:23 +01:00
Erik Johnston
6e17463228 Don't explode if we don't have the event 2015-06-01 16:39:43 +01:00
Erik Johnston
522f285f9b Add config option to disable compression of http responses 2015-06-01 13:36:30 +01:00
Mark Haines
b8d49be5a1 Merge branch 'develop' into markjh/twisted-15 
Conflicts:
	synapse/python_dependencies.py
 2015-06-01 10:56:05 +01:00
Mark Haines
90abdaf3bc Use Twisted-15.2.1, Use Agent.usingEndpointFactory rather than implement our own Agent 2015-06-01 10:51:50 +01:00
Mark Haines
d70c847b4f Merge pull request #170 from matrix-org/markjh/SYT-8-recaptcha 
Allow endpoint for verifying recaptcha to be configured
 2015-05-29 15:32:54 +01:00
Erik Johnston
d15f166093 Remove log line 2015-05-29 15:03:24 +01:00
Erik Johnston
ca580ef862 Don't copy twice 2015-05-29 15:02:55 +01:00
Mark Haines
784aaa53df Merge branch 'develop' into markjh/SYT-8-recaptcha 
Conflicts:
	synapse/handlers/auth.py
 2015-05-29 13:49:44 +01:00
Erik Johnston
8355b4d074 Bump syutil version 2015-05-29 13:08:43 +01:00
Erik Johnston
a7b65bdedf Add config option to turn off freezing events. Use new encode_json api and ujson.loads 2015-05-29 12:17:33 +01:00
Mark Haines
d94590ed48 Add config for setting the recaptcha verify api endpoint, so we can test it in sytest 2015-05-29 12:11:40 +01:00
Erik Johnston
afbd3b2fc4 SYN-395: Fix CAPTCHA, don't double decode json 2015-05-28 18:05:00 +01:00
Erik Johnston
79e37a7ecb Correctly pass connection pool parameter 2015-05-28 16:48:53 +01:00
Erik Johnston
0f118e55db Merge pull request #168 from matrix-org/erikj/conn_pool 
Make HTTP clients use connection pools.
 2015-05-28 16:03:56 +01:00
Erik Johnston
dd74436ffd Unused import 2015-05-28 15:47:20 +01:00
Erik Johnston
11f51e6ded Up maxPersistentPerHost count 2015-05-28 15:45:46 +01:00
Erik Johnston
086df80790 Add connection pooling to SimpleHttpClient 2015-05-28 15:43:21 +01:00
Erik Johnston
291e942332 Use connection pool for federation connections 2015-05-28 15:43:21 +01:00
Erik Johnston
31ade3b3e9 Remove a deep copy 2015-05-28 13:45:23 +01:00
Erik Johnston
36b3b75b21 Registration should be disabled by default 2015-05-28 11:01:34 +01:00
Mark Haines
554c63ca60 Iterate over the user_streams not the user_ids 2015-05-26 15:03:49 +01:00
Mark Haines
fff7905409 Merge branch 'bugs/SYN-390' into release-v0.9.1 2015-05-26 14:58:49 +01:00
Mark Haines
00dd207f60 Take a dict of the rule, not the rule list 2015-05-26 14:57:48 +01:00
Erik Johnston
2651fd5e24 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.9.1 2015-05-26 11:05:50 +01:00
Mark Haines
e7b25a649c Merge pull request #166 from matrix-org/bugs/SYN-390 
SYN-390: Don't modify the dictionary returned from the database here either
 2015-05-26 10:40:50 +01:00
Mark Haines
804b732aab SYN-390: Don't modify the dictionary returned from the database here either 2015-05-26 10:35:08 +01:00
Erik Johnston
45fffe8cbe Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.9.1 2015-05-26 10:22:41 +01:00
Erik Johnston
9ba3c1ede4 Merge pull request #165 from matrix-org/bugs/SYN-390 
SYN-390: Don't modify the dictionary returned from the data store
 2015-05-26 10:20:36 +01:00
Mark Haines
a0bebeda8b SYN-390: Don't modify the dictionary returned from the data store 2015-05-26 10:14:15 +01:00
Erik Johnston
27e093cbc1 Bump version 2015-05-22 17:03:37 +01:00
Mark Haines
d9f60e8dc8 Merge pull request #163 from matrix-org/markjh/presence_list_cache 
Add a cache for the presence list
 2015-05-22 17:02:23 +01:00
Mark Haines
0e42dfbe22 Merge pull request #164 from matrix-org/markjh/pusher_performance_2 
Add a cache for get_push rules for user, fix cache invalidation
 2015-05-22 17:01:56 +01:00
Erik Johnston
6eadbfbea0 Remove redundant for loop 2015-05-22 16:12:20 +01:00
Mark Haines
1a9a9abcc7 Add a cache for getting the presence list for a user 2015-05-22 16:11:17 +01:00
Erik Johnston
74b7de83ec Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes 2015-05-22 16:10:42 +01:00
Mark Haines
36317f3dad Merge pull request #156 from matrix-org/erikj/join_perf 
Make joining #matrix:matrix.org over federation quicker
 2015-05-22 16:09:54 +01:00
Mark Haines
052ac0c8d0 Merge pull request #159 from matrix-org/erikj/metrics_interface_config 
Enable changing the interface the metrics listener binds to
 2015-05-22 16:09:33 +01:00
Mark Haines
49a2c10279 Merge pull request #157 from matrix-org/markjh/presence_performance 
Improve presence performance in loadtest
 2015-05-22 16:04:40 +01:00
Mark Haines
5d53c14342 Merge pull request #160 from matrix-org/markjh/appservice_performance 
Make the appservice use 'users_in_room' rather than get_room_members …
 2015-05-22 16:04:22 +01:00
Mark Haines
106a3051b8 Remove spurious TODO comment 2015-05-22 15:53:03 +01:00
Erik Johnston
284f55a7fb Add doc strings 2015-05-22 15:18:04 +01:00
Erik Johnston
1ce1509989 s/metric_interface/metric_bind_host/ 2015-05-22 14:51:22 +01:00
Erik Johnston
8bb85c8c5a Update log line 2015-05-22 14:48:06 +01:00
Mark Haines
c8135f808b Remove unused import 2015-05-22 14:45:46 +01:00
Erik Johnston
b21d015c55 Log origin and stats of incoming transactions 2015-05-22 14:44:25 +01:00
Erik Johnston
e70e8e053e Add txn_id to some log lines 2015-05-22 14:44:02 +01:00
Erik Johnston
1b446a5d85 Log less lines at INFO level, but include more helpful information 2015-05-22 14:29:57 +01:00
Erik Johnston
59a0682f3e Enable changing the interface the metrics listener binds to 2015-05-22 13:13:07 +01:00
Mark Haines
b6adfc59f5 Invalidate the get_latest_event_ids_in_room cache when deleting from event_forward_extremities 2015-05-22 13:01:03 +01:00
Mark Haines
f43544eecc Make the appservice use 'users_in_room' rather than get_room_members since it is cached 2015-05-22 11:01:28 +01:00
Mark Haines
a04cde613e Add a cache for get_push rules for user, fix cache invalidation 2015-05-22 10:39:45 +01:00
Erik Johnston
4429e720ae Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-22 10:33:00 +01:00
Erik Johnston
f8c2cd129d Bump version 2015-05-21 17:03:30 +01:00
Mark Haines
2043527b9b Don't try to use a txn when not in one, remove spurious debug logging 2015-05-21 16:53:03 +01:00
Mark Haines
53447e9cd3 Add caches for things requested by the pushers 2015-05-21 16:41:39 +01:00
Mark Haines
d61ce3f670 Add a cache for get_current_state with state_key 2015-05-21 16:41:39 +01:00
Erik Johnston
a910984b58 Actually return something from lambda 2015-05-21 15:58:41 +01:00
Erik Johnston
e309b1045d Sort backfill events 2015-05-21 15:57:35 +01:00
Erik Johnston
0180bfe4aa Remove dead code 2015-05-21 15:53:41 +01:00
Erik Johnston
1f3d1d85a9 Only get non-state 2015-05-21 15:52:29 +01:00
Erik Johnston
39a3340f73 Skip events we've already seen 2015-05-21 15:48:56 +01:00
Erik Johnston
ae3bff3491 Correctly prepopulate queue 2015-05-21 15:46:07 +01:00
Erik Johnston
dc085ddf8c Don't prepopulate event_results 2015-05-21 15:44:05 +01:00
Erik Johnston
73d23c6ae8 Don't readd things that are already in event_results 2015-05-21 15:40:22 +01:00
Erik Johnston
6189d8e54d PriorityQueue gives lowest first 2015-05-21 15:38:08 +01:00
Erik Johnston
115ef3ddac Correctly capture Queue.Empty exception 2015-05-21 15:37:43 +01:00
Erik Johnston
4fb858d90a Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes 2015-05-21 15:25:54 +01:00
Mark Haines
88f1ea36ce Oops, get_rooms_for_user returns a namedtuple, not a room_id 2015-05-21 15:23:58 +01:00
Erik Johnston
c2633907c5 Merge branch 'erikj/join_perf' of github.com:matrix-org/synapse into erikj/backfill_fixes 2015-05-21 14:58:47 +01:00
Erik Johnston
ebfdd2eb5b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-21 14:54:52 +01:00
Erik Johnston
a551c5dad7 Merge pull request #155 from matrix-org/erikj/perf 
Bulk and batch retrieval of events.
 2015-05-21 14:54:40 +01:00
Erik Johnston
27e4b45c06 s/for events/for requests for events/ 2015-05-21 14:52:23 +01:00
Erik Johnston
ac5f2bf9db s/for events/for requests for events/ 2015-05-21 14:50:57 +01:00
Erik Johnston
80a167b1f0 Add comments 2015-05-21 11:19:04 +01:00
Mark Haines
7ae8afb7ef Removed unused 'is_visible' method 2015-05-20 14:48:11 +01:00
Erik Johnston
9118a92862 Split up _get_events into defer and txn versions 2015-05-20 13:27:16 +01:00
Mark Haines
e01b825cc9 Clean up the presence_list checking logic a bit 2015-05-20 13:21:59 +01:00
Erik Johnston
ab45e12d31 Make not return a deferred _get_event_from_row_txn 2015-05-20 13:07:19 +01:00
Erik Johnston
f407cbd2f1 PEP8 2015-05-20 13:02:01 +01:00
Erik Johnston
227f8ef031 Split out _get_event_from_row back into defer and _txn version 2015-05-20 13:00:57 +01:00
Erik Johnston
2bc60c55af Fix _get_backfill_events to return events in the correct order 2015-05-20 12:57:00 +01:00
Erik Johnston
20814fabdd Actually fetch state for new backwards extremeties when backfilling. 2015-05-20 11:59:02 +01:00
Erik Johnston
9084cdd70f Ensure event_results is a set 2015-05-19 16:34:31 +01:00
Erik Johnston
3a653515ec Add None check 2015-05-19 15:27:09 +01:00
Erik Johnston
aa729349dd Fix event_backwards_extrem insertion to ignore outliers 2015-05-19 15:27:00 +01:00
Erik Johnston
5b1631a4a9 Add a timeout param to get_event 2015-05-19 14:53:32 +01:00
Erik Johnston
291cba284b Handle the case when things return empty but non none things 2015-05-19 14:42:46 +01:00
Erik Johnston
253f76a0a5 Don't always hit get_server_verify_key_v1_direct 2015-05-19 14:42:38 +01:00
Erik Johnston
6837c5edab Handle the case when things return empty but non none things 2015-05-19 14:27:11 +01:00
Erik Johnston
7223129916 Don't apply new room join hack if depth > 5 2015-05-19 14:16:08 +01:00
Erik Johnston
5ae4a84211 Don't always hit get_server_verify_key_v1_direct 2015-05-19 13:43:34 +01:00
Erik Johnston
118a760719 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-19 13:20:29 +01:00
David Baker
19505e0392 Disable GZip encoding on static file resources as per comment 2015-05-19 13:20:25 +01:00
Erik Johnston
d3e09f12d0 SYN-383: Actually, we expect this value to be a dict 2015-05-19 13:12:41 +01:00
Erik Johnston
677be13ffc Revert accidental commit 2015-05-19 13:12:28 +01:00
Erik Johnston
350b88656a SYN-383: Actually, we expect this value to be a dict 2015-05-19 13:01:57 +01:00
Erik Johnston
9de94d5a4d Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-19 12:50:17 +01:00
Erik Johnston
2b7120e233 SYN-383: Handle the fact the server might not have signed things 2015-05-19 12:49:38 +01:00
Erik Johnston
8b256a7296 Don't reuse var names 2015-05-19 11:58:22 +01:00
Erik Johnston
62ccc6d95f Don't reuse var names 2015-05-19 11:58:04 +01:00
Erik Johnston
01858bcbf2 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-19 11:56:35 +01:00
Erik Johnston
2aeee2a905 SYN-383: Fix parsing of verify_keys and catching of _DefGen_Return 2015-05-19 11:56:18 +01:00
Erik Johnston
5e7883ec19 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf 2015-05-19 10:50:43 +01:00
Mark Haines
c6a03c46e6 SYN-383: Extract the response list from 'server_keys' in the response JSON as it might work better than iterating over the top level dict 2015-05-19 10:23:02 +01:00
Mark Haines
e4c65b338d Speed up the get_pagination_rows as well 2015-05-18 18:21:06 +01:00
Mark Haines
99914ec9f8 Merge pull request #152 from matrix-org/notifier_performance 
Notifier performance
 2015-05-18 17:49:59 +01:00
Erik Johnston
ef910a0358 Do work in parellel when joining a room 2015-05-18 17:17:04 +01:00
Mark Haines
591c4bf223 Cache the most recent serial for each room 2015-05-18 16:21:51 +01:00
Mark Haines
e1150cac4b Move updating the serial and state of the presence cache into a single function 2015-05-18 15:46:37 +01:00
Erik Johnston
165eb2dbe6 Comments and shuffle of functions 2015-05-18 15:18:41 +01:00
Mark Haines
880fb46de0 Merge branch 'notifier_performance' into markjh/presence_performance 2015-05-18 14:33:58 +01:00
Erik Johnston
65878a2319 Remove unused metric 2015-05-18 14:06:30 +01:00
Mark Haines
ad31fa3040 Don't bother sorting by the room_stream_ids, it shouldn't matter which order they are notified in 2015-05-18 14:04:58 +01:00
Erik Johnston
4d1b6f4ad1 Remove rejected events if we don't want rejected events 2015-05-18 14:03:46 +01:00
Mark Haines
0b0033c40b Merge branch 'develop' into notifier_performance 2015-05-18 13:50:01 +01:00
Mark Haines
755def8083 Add more doc string, reduce C+P boilerplate for getting room list 2015-05-18 13:46:47 +01:00
Mark Haines
1e90715a3d Make sure the notifier stream token goes forward when it is updated. Sort the pending events by the correct room_stream_id 2015-05-18 13:17:36 +01:00
Erik Johnston
131bdf9bb1 Merge branch 'erikj/events_move' of github.com:matrix-org/synapse into erikj/perf 2015-05-18 10:23:37 +01:00
Erik Johnston
10f1bdb9a2 Move get_events functions to storage.events 2015-05-18 10:21:40 +01:00
Erik Johnston
d5cea26d45 Remove pointless newline 2015-05-18 10:16:45 +01:00
Erik Johnston
c71176858b Newline, remove debug logging 2015-05-18 10:11:14 +01:00
Erik Johnston
f8bd4de87d Remove debug logging 2015-05-18 09:58:03 +01:00
Erik Johnston
c3b37abdfd PEP8 2015-05-15 16:59:58 +01:00
Erik Johnston
6c74fd62a0 Revert limiting of fetching, it didn't help perf. 2015-05-15 16:45:35 +01:00
Erik Johnston
9ff7f66a2b init j 2015-05-15 16:36:03 +01:00
Erik Johnston
70f272f71c Don't completely drain the list 2015-05-15 16:34:17 +01:00
Erik Johnston
8763dd80ef Don't fetch prev_content for current_state 2015-05-15 15:33:01 +01:00
Erik Johnston
807229f2f2 Err, defer.gatherResults ftw 2015-05-15 15:20:29 +01:00
Erik Johnston
acb12cc811 Make store.get_current_state fetch events asyncly 2015-05-15 15:20:05 +01:00
Erik Johnston
d62dee7eae Remove more debug logging 2015-05-15 15:06:37 +01:00
Erik Johnston
0f29cfabc3 Remove debug logging 2015-05-15 14:06:42 +01:00
Erik Johnston
e275a9c0d9 preserve log context 2015-05-15 11:54:51 +01:00
Erik Johnston
aa32bd38e4 Add a wait 2015-05-15 11:35:04 +01:00
Erik Johnston
372d4c6d7b Srsly. Don't use closures. Baaaaaad 2015-05-15 11:26:00 +01:00
Erik Johnston
575ec91d82 Correctly pass through params 2015-05-15 11:15:10 +01:00
Mark Haines
10be983f2c Merge pull request #153 from matrix-org/markjh/presence_docstring 
Add some doc strings for presence.
 2015-05-15 11:11:47 +01:00
Mark Haines
415b158ce2 More whitespace 2015-05-15 11:09:47 +01:00
Erik Johnston
de01438a57 Sort out error handling 2015-05-15 11:00:50 +01:00
Erik Johnston
a2c4f3f150 Fix daedlock 2015-05-15 10:54:04 +01:00
Mark Haines
0a4330cd5d Add some missed argument types, cleanup the whitespace a bit 2015-05-14 17:48:12 +01:00
Mark Haines
47ec693e29 More doc-strings 2015-05-14 17:07:02 +01:00
Erik Johnston
1d566edb81 Remove race condition 2015-05-14 16:54:35 +01:00
David Baker
6e1ad283cf Support gzip encoding for client, client v2 and web client resources (SYN-176). 2015-05-14 16:39:19 +01:00
Erik Johnston
ef3d8754f5 Call from right thread 2015-05-14 15:41:55 +01:00
Erik Johnston
142934084a Count and loop 2015-05-14 15:40:21 +01:00
Erik Johnston
96c5b9f87c Don't start up more fetch_events 2015-05-14 15:36:04 +01:00
Erik Johnston
7cd6a6f6cf Awful idea for speeding up fetching of events 2015-05-14 15:34:02 +01:00
Mark Haines
c5d1b4986b Remove unused arguments and doc PresenceHandler.push_update_to_clients 2015-05-14 14:59:31 +01:00
Erik Johnston
7f4105a5c9 Turn off preemptive transactions 2015-05-14 14:51:06 +01:00
Erik Johnston
f4d58deba1 PEP8 2015-05-14 14:45:42 +01:00
Erik Johnston
386b7330d2 Move from _base to events 2015-05-14 14:45:22 +01:00
Mark Haines
0ad1c67234 Add some doc-strings to notifier 2015-05-14 14:35:07 +01:00
Erik Johnston
7d6a1dae31 Jump out early 2015-05-14 14:27:58 +01:00
Erik Johnston
656223fbd3 Actually, we probably want to run this in a transaction 2015-05-14 14:26:35 +01:00
David Baker
67800f7626 Treat setting your display name to the empty string as removing it (SYN-186). 2015-05-14 14:19:59 +01:00
Erik Johnston
2f7f8e1c2b Preemptively jump into a transaction if we ask for get_prev_content 2015-05-14 14:17:36 +01:00
Mark Haines
4770cec7bc Merge pull request #150 from matrix-org/notifier_unify 
Make v1 and v2 client APIs interact with the notifier in the same way.
 2015-05-14 14:16:59 +01:00
Erik Johnston
e1e9f0c5b2 loop -> gatherResults 2015-05-14 13:58:49 +01:00
Erik Johnston
ab78a8926e Err, we probably want a bigger limit 2015-05-14 13:47:16 +01:00
Erik Johnston
f6f902d459 Move fetching of events into their own transactions 2015-05-14 13:45:48 +01:00
Erik Johnston
cdb3757942 Refactor _get_events 2015-05-14 13:31:55 +01:00
David Baker
92e1c8983d Disallow whitespace in aliases here too 2015-05-14 13:21:55 +01:00
David Baker
0c894e1ebd Throw error when creating room if alias contains whitespace #SYN-335 2015-05-14 13:11:28 +01:00
Mark Haines
084c365c3a Use the current token when timing out a notifier, make sure the user_id is a string in on_new_user_event 2015-05-14 12:03:26 +01:00
David Baker
c37a6e151f Make shared secret registration work again 2015-05-14 12:03:13 +01:00
Erik Johnston
36ea26c5c0 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/perf 2015-05-14 12:01:38 +01:00
David Baker
7c549dd557 Add ID generator for push_rules_enable to #resolve SYN-378 2015-05-14 11:44:03 +01:00
Mark Haines
899d4675dd Merge branch 'notifier_unify' into notifier_performance 2015-05-14 11:36:44 +01:00
Mark Haines
243c56e725 Merge branch 'develop' into notifier_unify 2015-05-14 11:36:23 +01:00
Mark Haines
3edd2d5c93 Fix v2 sync, update the last_notified_ms only if there was an active listener 2015-05-14 11:25:30 +01:00
Erik Johnston
4f1d984e56 Add index on events 2015-05-13 17:22:26 +01:00
Mark Haines
5e0c533672 Fix metric counter 2015-05-13 17:20:28 +01:00
Erik Johnston
968b01a91a Actually use async method 2015-05-13 17:02:46 +01:00
Erik Johnston
4071f29653 Fetch events from events_id in their own transactions 2015-05-13 16:59:41 +01:00
Mark Haines
f1b83d88a3 Discard unused NotifierUserStreams 2015-05-13 16:54:02 +01:00
Erik Johnston
a988361aea Typo 2015-05-13 15:44:15 +01:00
Erik Johnston
8888982db3 Don't insert None 2015-05-13 15:43:32 +01:00
Mark Haines
9af432257d Don't set a timer if there's already a result to return 2015-05-13 15:42:13 +01:00
Erik Johnston
cf706cc6ef Don't return None 2015-05-13 15:31:25 +01:00
Erik Johnston
5971d240d4 Limit batch size 2015-05-13 15:26:49 +01:00
Erik Johnston
ca4f458787 Fetch events in bulk 2015-05-13 15:13:42 +01:00
Mark Haines
df6db5c802 Don't bother checking for new events from a source if the stream token hasn't advanced for that source 2015-05-13 15:08:24 +01:00
Erik Johnston
6edff11a88 Don't fetch redaction and rejection stuff for each event, so we can use index only scan 2015-05-13 14:39:05 +01:00
Mark Haines
63878c0379 Don't bother checking for updates if the stream token hasn't advanced for a user 2015-05-13 13:42:21 +01:00
Erik Johnston
02590c3e1d Temp turn off checking for rejections and redactions 2015-05-13 11:31:28 +01:00
Erik Johnston
619a21812b defer.gatherResults loop 2015-05-13 11:29:03 +01:00
Erik Johnston
fec4485e28 Batch fetching of events for state groups 2015-05-13 11:22:42 +01:00
Erik Johnston
409bcc76bd Load events for state group seperately 2015-05-13 11:13:31 +01:00
Mark Haines
cffe6057fb Merge branch 'notifier_unify' into notifier_performance 
Conflicts:
	synapse/notifier.py
 2015-05-12 16:37:50 +01:00
Erik Johnston
80fd2b574c Don't talk to yourself when backfilling 2015-05-12 16:19:46 +01:00
Erik Johnston
e122685978 You need to call contextmanager 2015-05-12 16:12:37 +01:00
Mark Haines
d7b3ac46f8 Revert "Improvement to performance of presence event stream handling" 2015-05-12 15:44:21 +01:00
Mark Haines
4429e4bf24 Merge branch 'develop' into notifier_unify 
Conflicts:
	synapse/notifier.py
 2015-05-12 15:31:26 +01:00
Mark Haines
ec07dba29e Merge pull request #143 from matrix-org/erikj/SYN-375 
SYN-375 - Lots of unhandled deferred exceptions.
 2015-05-12 15:25:54 +01:00
Mark Haines
c167cbc9fd Merge pull request #147 from matrix-org/presence-performance 
Improvement to performance of presence event stream handling
 2015-05-12 15:24:54 +01:00
Mark Haines
a6fb2aa2a5 Merge pull request #144 from matrix-org/erikj/logging_context 
Preserving logging contexts
 2015-05-12 15:23:50 +01:00
Mark Haines
1fce36b111 Merge pull request #149 from matrix-org/erikj/backfill 
Backfill support
 2015-05-12 15:20:32 +01:00
Erik Johnston
8b28209c60 Err, delete the right stuff 2015-05-12 15:02:53 +01:00
Erik Johnston
30c72d377e Newlines 2015-05-12 14:47:40 +01:00
Erik Johnston
e4eddf9b36 We do actually want to delete rows out of event_backward_extremities 2015-05-12 14:47:23 +01:00
Erik Johnston
c1779a79bc Fix up _handle_prev_events to not try to insert duplicate rows 2015-05-12 14:41:50 +01:00
Erik Johnston
74850d7f75 Do state groups persistence /after/ checking if we have already persisted the event 2015-05-12 14:14:58 +01:00
Erik Johnston
07a1223156 s/backfil/backfill/ 2015-05-12 14:09:54 +01:00
Erik Johnston
0d31ad5101 Typos everywhere 2015-05-12 14:02:01 +01:00
Erik Johnston
a0dfffb33c And another typo. 2015-05-12 14:00:31 +01:00
Erik Johnston
6e5ac4a28f Err, gatherResults doesn't take a dict... 2015-05-12 13:58:14 +01:00
Erik Johnston
8022b27fc2 Make distributer.fire work as it did 2015-05-12 13:14:48 +01:00
Erik Johnston
95dedb866f Unwrap defer.gatherResults failures 2015-05-12 13:14:29 +01:00
Mark Haines
78672a9fd5 Merge branch 'notifier_unify' into notifier_performance 2015-05-12 13:11:54 +01:00
Erik Johnston
da6a7bbdde Merge branch 'develop' of github.com:matrix-org/synapse into erikj/logging_context 2015-05-12 13:10:42 +01:00
Mark Haines
2551b6645d Update the end_token correctly, otherwise the token doesn't advance and the client gets duplicate events 2015-05-12 11:54:18 +01:00
Mark Haines
5e4ba463b7 Merge branch 'develop' into notifier_unify 2015-05-12 11:41:53 +01:00
Mark Haines
5002056b16 SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called from the main thread after the transaction completes, not from database thread before the transaction completes. 2015-05-12 11:20:40 +01:00
Mark Haines
5c75adff95 Add a NotifierUserStream to hold all the notification listeners for a user 2015-05-12 11:00:37 +01:00
Erik Johnston
367382b575 Handle the case where the other side is unreachable when backfilling 2015-05-12 10:35:45 +01:00
Erik Johnston
4df11b5039 Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards' 2015-05-12 10:28:10 +01:00
Erik Johnston
84e6b4001f Initial hack at wiring together pagination and backfill 2015-05-11 18:01:31 +01:00
Erik Johnston
17653a5dfe Move storage.stream._StreamToken to types.RoomStreamToken 2015-05-11 18:01:01 +01:00
Mark Haines
e269c511f6 Don't bother passing the events to the notifier since it isn't using them 2015-05-11 15:01:51 +01:00
Mark Haines
5e3b254dc8 Use wait_for_events to implement 'get_events' 2015-05-11 14:37:33 +01:00
Erik Johnston
79b7154454 Merge pull request #146 from matrix-org/erikj/push_rules_fixes 
Fix 500 on push rule updates.
 2015-05-11 11:33:47 +01:00
Erik Johnston
4ef556f650 Bump version 2015-05-11 11:31:04 +01:00
Erik Johnston
b036596b75 Prefer to use _simple_*. 2015-05-11 11:24:01 +01:00
Erik Johnston
cd525c0f5a push_rules table expects an 'id' field 2015-05-11 11:24:01 +01:00
Erik Johnston
de87541862 Bump version 2015-05-10 10:51:08 +01:00
Erik Johnston
22d2f498fa Fix push rule bug: can't insert bool into small int column 2015-05-10 10:50:51 +01:00
Erik Johnston
2236ef6c92 Fix up leak. Add warnings. 2015-05-08 19:53:34 +01:00
Erik Johnston
da1aa07db5 Add some docs 2015-05-08 16:52:49 +01:00
Erik Johnston
4ac1941592 PEP8 2015-05-08 16:33:01 +01:00
Erik Johnston
476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Erik Johnston
fca28d243e Change the way we create observers to deferreds so that we don't get spammed by 'unhandled errors' 2015-05-08 16:28:08 +01:00
Erik Johnston
0cd1401f8d Bump version 2015-05-08 16:11:51 +01:00
Erik Johnston
724bb1e7d9 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-08 16:11:19 +01:00
Mark Haines
1c7912751e Drop the old table not the new table 2015-05-08 16:04:32 +01:00
Mark Haines
9d36eb4eab Rename unique constraint 2015-05-08 16:01:55 +01:00
Mark Haines
b0f71db3ff Remove unsigned 2015-05-08 15:59:51 +01:00
Mark Haines
84e1cacea4 Bump schema version 2015-05-08 15:58:14 +01:00
Mark Haines
6538d445e8 Make the timestamps in server_keys_json bigints 2015-05-08 15:55:17 +01:00
Erik Johnston
126d562576 Bump version 2015-05-08 13:29:37 +01:00
Paul "LeoNerd" Evans
45543028bb Use the presence cachemap ordering to early-abort the iteration loop 2015-05-07 22:40:10 +01:00
Paul "LeoNerd" Evans
f683b5de47 Store presence cachemap in an ordered dict, so that the newer serials will be at the end 2015-05-07 21:27:53 +01:00
Erik Johnston
db0dca2f6f Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-07 19:21:00 +01:00
Erik Johnston
a151693a3b Bump syweb version 2015-05-07 18:01:46 +01:00
Mark Haines
dfa98f911b revert accidental bcrypt gensalt round reduction from loadtesting 2015-05-07 17:45:42 +01:00
Mark Haines
ef8e8ebd91 pynacl-0.3.0 was released so we can finally start using it directly from pypi 2015-05-07 16:46:51 +01:00
David Baker
97a64f3ebe Merge branch 'develop' of github.com:matrix-org/synapse into develop 2015-05-07 09:33:42 +01:00
David Baker
b850c9fa04 Typo 2015-05-07 09:33:30 +01:00
Mark Haines
4a7a4a5b6c Optional profiling using cProfile 2015-05-06 17:08:00 +01:00
Mark Haines
b994fb2b96 Don't read from the config file before checking it exists 2015-05-06 12:56:47 +01:00
Erik Johnston
04c01882fc Bump version 2015-05-06 09:59:13 +01:00
Erik Johnston
0c4ac271ca Merge branch 'erikj/executemany' of github.com:matrix-org/synapse into erikj/SYN-371 2015-05-05 18:21:19 +01:00
Erik Johnston
0cf7e480b4 And use buffer(...) there as well 2015-05-05 18:20:01 +01:00
Erik Johnston
ed2584050f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemany 2015-05-05 18:15:20 +01:00
Erik Johnston
977338a7af Use buffer(...) when inserting into bytea column 2015-05-05 18:12:53 +01:00
Mark Haines
31049c4d72 Merge pull request #139 from matrix-org/bugs/SYN-369 
Fix race with cache invalidation. SYN-369
 2015-05-05 17:46:13 +01:00
Mark Haines
deb0237166 Add some doc-string 2015-05-05 17:45:11 +01:00
Mark Haines
e45b05647e Fix the --help option for synapse 2015-05-05 17:39:59 +01:00
Erik Johnston
3d5a955e08 Missed events are not outliers 2015-05-05 17:36:57 +01:00
Mark Haines
d18f37e026 Collect the invalidate callbacks on the transaction object rather than passing around a separate list 2015-05-05 17:32:21 +01:00
Erik Johnston
9951542393 Add a comment about the zip(*[zip(sorted(...),...)]) 2015-05-05 17:06:55 +01:00
Mark Haines
041b6cba61 SYN-369: Add comments to the sequence number logic in the cache 2015-05-05 16:32:44 +01:00
Mark Haines
63075118a5 Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access 2015-05-05 16:24:04 +01:00
Erik Johnston
531d7955fd Don't insert without deduplication. In this case we never actually use this table, so simply remove the insert entirely 2015-05-05 16:12:28 +01:00
Mark Haines
bfa4a7f8b0 Invalidate the room_member cache if the current state events updates 2015-05-05 15:43:49 +01:00
Mark Haines
d0fece8d3c Missing return for when the event was already persisted 2015-05-05 15:39:09 +01:00
Erik Johnston
bdcd7693c8 Fix indentation 2015-05-05 15:14:48 +01:00
Erik Johnston
43c2e8deae Add support for using executemany 2015-05-05 15:13:25 +01:00
Erik Johnston
1692dc019d Don't call 'encode_parameter' no-op 2015-05-05 15:00:30 +01:00
Mark Haines
a9aea68fd5 Invalidate the caches from the correct thread 2015-05-05 14:57:08 +01:00
Mark Haines
261d809a47 Sequence the modifications to the cache so that selects don't race with inserts 2015-05-05 14:13:50 +01:00
Erik Johnston
d9cc5de9e5 Correctly name transaction 2015-05-05 10:24:10 +01:00
Erik Johnston
b8940cd902 Remove some unused indexes 2015-05-01 16:14:25 +01:00
Erik Johnston
1942382246 Don't log enqueue_ 2015-05-01 16:14:25 +01:00
David Baker
eb9bd2d949 user_id now in user_threepids 2015-05-01 15:04:37 +01:00
Erik Johnston
2d386d7038 That wasn't a deferred 2015-05-01 14:41:25 +01:00
Erik Johnston
4ac2823b3c Remove inlineCallbacks from non-generator 2015-05-01 14:41:25 +01:00
Erik Johnston
22c7c5eb8f Typo 2015-05-01 14:41:25 +01:00
Erik Johnston
42c12c04f6 Remove some run_on_reactors 2015-05-01 14:41:25 +01:00
Erik Johnston
adb5b76ff5 Don't log all auth events every time we call auth.check 2015-05-01 14:41:25 +01:00
Mark Haines
3bcdf3664c Use the daemonize key from the config if it exists 2015-05-01 14:34:55 +01:00
David Baker
9eeb03c0dd Don't use self.execute: it's designed for fetching stuff 2015-05-01 14:21:25 +01:00
Mark Haines
7b50769eb9 Merge pull request #136 from matrix-org/markjh/config_cleanup 
Config restructuring.
 2015-05-01 14:04:39 +01:00
David Baker
7693f24792 No id field on user 2015-05-01 13:55:42 +01:00
Mark Haines
46a65c282f Allow generate-config to run against an existing config file to generate default keys 2015-05-01 13:54:38 +01:00
David Baker
92b20713d7 More missed get_user_by_id API changes 2015-05-01 13:45:54 +01:00
Erik Johnston
da4ed08739 One too many lens 2015-05-01 13:29:38 +01:00
Erik Johnston
9060dc6b59 Change public room list to use defer.gatherResults 2015-05-01 13:28:36 +01:00
David Baker
1fae1b3166 This api now no longer returns an array 2015-05-01 13:26:41 +01:00
Erik Johnston
80b4119279 Don't wait for storage of access_token 2015-05-01 13:14:05 +01:00
Erik Johnston
4011cf1c42 Cache latest_event_ids_in_room 2015-05-01 13:06:26 +01:00
Erik Johnston
657298cebd Don't lock user_ips table for upsert. 2015-05-01 10:46:48 +01:00
Erik Johnston
fabb7acd45 Fix bug where we reconnected to the database on every query. 2015-05-01 10:24:24 +01:00
Erik Johnston
23c639ff32 Split a storage function in two so that we don't have to do extra work. 2015-05-01 10:17:19 +01:00
Erik Johnston
8be5284e91 Remove pointless join 2015-04-30 18:49:26 +01:00
Erik Johnston
503e4d3d52 Fix broken SQL 2015-04-30 18:44:47 +01:00
Erik Johnston
00718ae7a9 Need more yield 2015-04-30 18:43:39 +01:00
Erik Johnston
0465560c1a Add missing param 2015-04-30 18:42:44 +01:00
Erik Johnston
61d05daab1 More join conditions 2015-04-30 18:41:05 +01:00
Erik Johnston
6ead27ddda Add more conditions on JOINs to make postgres go a little faster. 2015-04-30 18:32:03 +01:00
Mark Haines
50c87b8eed Allow "manhole" to be ommited from the config 2015-04-30 18:11:47 +01:00
Mark Haines
345995fcde Remove the ~, comment the lines instead 2015-04-30 18:10:19 +01:00
Mark Haines
62cebee8ee Update key.py 2015-04-30 17:54:01 +01:00
Mark Haines
95cbfee8ae Update metrics.py 2015-04-30 17:52:20 +01:00
Mark Haines
6ea9cf58be missing import 2015-04-30 17:21:21 +01:00
Erik Johnston
f383d5a801 Fix up get_current_state and get_room_name_and_aliases queries to parse events in transaction 2015-04-30 17:12:52 +01:00
Mark Haines
c95480963e read the pid_file from the config file in synctl 2015-04-30 17:12:15 +01:00
Mark Haines
2d4d2bbae4 Merge branch 'develop' into markjh/config_cleanup 
Conflicts:
	synapse/config/captcha.py
 2015-04-30 16:54:55 +01:00
Mark Haines
2f1348f339 Write a default log_config when generating config 2015-04-30 16:52:57 +01:00
Erik Johnston
69d4063651 Add get_rooms_for_user cache 2015-04-30 16:47:51 +01:00
David Baker
5b02f33451 Undo changes to logger config, ie. remove the access_log_file option: decision is to support this through log_config rather tan adding an option. 2015-04-30 16:21:39 +01:00
David Baker
054aa0d58c Do access log using python's logging stuff, just under a separate logger name 2015-04-30 16:21:38 +01:00
Erik Johnston
3c4c229788 Don't use sub queries, it makes postgres sad 2015-04-30 16:16:53 +01:00
Mark Haines
74aaacf82a Don't break when sizes or durations are given as integers 2015-04-30 16:04:02 +01:00
Mark Haines
29400b45b9 SYN-367: Use upsert rather than insert_or_replace 2015-04-30 15:21:31 +01:00
Mark Haines
c28f1d16f0 Add a random string to the auto generated key id 2015-04-30 15:13:14 +01:00
Mark Haines
265f30bd3f Allow --enable-registration to be passed on the commandline 2015-04-30 15:04:06 +01:00
Mark Haines
c9e62927f2 Use disable_registration keys if they are present 2015-04-30 14:34:09 +01:00
Erik Johnston
2366d28780 Don't needlessly join on state_events 2015-04-30 14:02:06 +01:00
David Baker
d89a9f7283 Add an access_log 
SYN-161 #resolve
 2015-04-30 13:58:13 +01:00
Mark Haines
1aa11cf7ce Allow multiple config files, set up a default config before applying the config files 2015-04-30 13:48:15 +01:00
David Baker
0c1b7f843b Unused import 2015-04-30 13:33:30 +01:00
David Baker
4b46fbec5b Doesn't look like this is used anymore 2015-04-30 12:04:08 +01:00
Erik Johnston
1d7702833d Make simple query rather than long one and then throw away half the results 2015-04-30 10:16:12 +01:00
Mark Haines
6b69ddd17a remove duplicate parse_size method 2015-04-30 04:26:29 +01:00
Mark Haines
d624e2a638 Manually generate the default config yaml, remove most of the commandline arguments for synapse anticipating that people will use the yaml instead. Simpify implementing config options by not requiring the classes to hit the super class 2015-04-30 04:24:44 +01:00
Erik Johnston
b1ca784aca Correctly decode, for sqlite and postgres, rows from pushers table 2015-04-29 19:41:14 +01:00
Erik Johnston
4a9dc5b2f5 pushkey' are also bytes. 2015-04-29 19:27:02 +01:00
Erik Johnston
0ade2712d1 Typo 2015-04-29 19:17:25 +01:00
Erik Johnston
50f96f256f Also remove yield from within lock in the other generator 2015-04-29 19:17:00 +01:00
Erik Johnston
d2d61a8288 Fix deadlock in id_generators. No idea why this was an actual deadlock. 2015-04-29 19:15:23 +01:00
Erik Johnston
3e71d13acf Also log when we've started pushers 2015-04-29 18:37:42 +01:00
Erik Johnston
e7a6edb0ee Revert previous 2015-04-29 18:37:30 +01:00
Erik Johnston
c27d6ad6b5 Only start pushers when synapse has fully started 2015-04-29 18:25:24 +01:00
Erik Johnston
46daf2d200 Start pushers on reactor thread 2015-04-29 18:22:20 +01:00
Erik Johnston
3864b3a8e6 Actually return rows 2015-04-29 18:07:36 +01:00
Erik Johnston
0618978238 Typo, args wrong way round 2015-04-29 18:04:35 +01:00
Erik Johnston
09177f4f2e Decode buffers in same thread 2015-04-29 18:03:42 +01:00
Erik Johnston
472be88674 We store pusher data as bytes 2015-04-29 17:43:46 +01:00
Erik Johnston
a6e62cf6d0 Fix off by one in presence token handling 2015-04-29 17:37:11 +01:00
David Baker
12d381bd5d Decode the data json in the storage layer (was moved but this part was missed) 2015-04-29 17:13:51 +01:00
David Baker
f8c30faf25 Oops, update the contraint too 2015-04-29 16:58:42 +01:00
David Baker
61cd5d9045 Be more postgressive 2015-04-29 16:57:14 +01:00
David Baker
fb95035a65 Be postgressive 2015-04-29 16:53:41 +01:00
David Baker
4669def000 Oops, forgot the schema delta file 2015-04-29 16:50:16 +01:00
Erik Johnston
0337eaf321 txn.execute doesn't return cursors 2015-04-29 16:43:39 +01:00
Erik Johnston
884fb88e28 txn.execute doesn't return cursors 2015-04-29 16:35:20 +01:00
Erik Johnston
d76c058eea Fix invalid SQL to work in postgres land 2015-04-29 16:30:25 +01:00
David Baker
9927170787 Accept camelcase + underscores in binding too 2015-04-29 15:57:09 +01:00