Matrix-Mirrors/anonymousland-synapse
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse.git synced 2024-12-21 09:34:18 -05:00
Code Issues Packages Projects Releases Wiki Activity
4,199 Commits 4 Branches 615 Tags 63 MiB
68b255c5a1
Commit Graph

2665 Commits

Author SHA1 Message Date
Mark Haines
ec07dba29e Merge pull request #143 from matrix-org/erikj/SYN-375 
SYN-375 - Lots of unhandled deferred exceptions.
 2015-05-12 15:25:54 +01:00
Mark Haines
c167cbc9fd Merge pull request #147 from matrix-org/presence-performance 
Improvement to performance of presence event stream handling
 2015-05-12 15:24:54 +01:00
Mark Haines
a6fb2aa2a5 Merge pull request #144 from matrix-org/erikj/logging_context 
Preserving logging contexts
 2015-05-12 15:23:50 +01:00
Mark Haines
1fce36b111 Merge pull request #149 from matrix-org/erikj/backfill 
Backfill support
 2015-05-12 15:20:32 +01:00
Erik Johnston
8b28209c60 Err, delete the right stuff 2015-05-12 15:02:53 +01:00
Erik Johnston
30c72d377e Newlines 2015-05-12 14:47:40 +01:00
Erik Johnston
e4eddf9b36 We do actually want to delete rows out of event_backward_extremities 2015-05-12 14:47:23 +01:00
Erik Johnston
c1779a79bc Fix up _handle_prev_events to not try to insert duplicate rows 2015-05-12 14:41:50 +01:00
Erik Johnston
74850d7f75 Do state groups persistence /after/ checking if we have already persisted the event 2015-05-12 14:14:58 +01:00
Erik Johnston
07a1223156 s/backfil/backfill/ 2015-05-12 14:09:54 +01:00
Erik Johnston
0d31ad5101 Typos everywhere 2015-05-12 14:02:01 +01:00
Erik Johnston
a0dfffb33c And another typo. 2015-05-12 14:00:31 +01:00
Erik Johnston
6e5ac4a28f Err, gatherResults doesn't take a dict... 2015-05-12 13:58:14 +01:00
Erik Johnston
8022b27fc2 Make distributer.fire work as it did 2015-05-12 13:14:48 +01:00
Erik Johnston
95dedb866f Unwrap defer.gatherResults failures 2015-05-12 13:14:29 +01:00
Mark Haines
78672a9fd5 Merge branch 'notifier_unify' into notifier_performance 2015-05-12 13:11:54 +01:00
Erik Johnston
da6a7bbdde Merge branch 'develop' of github.com:matrix-org/synapse into erikj/logging_context 2015-05-12 13:10:42 +01:00
Mark Haines
2551b6645d Update the end_token correctly, otherwise the token doesn't advance and the client gets duplicate events 2015-05-12 11:54:18 +01:00
Mark Haines
5e4ba463b7 Merge branch 'develop' into notifier_unify 2015-05-12 11:41:53 +01:00
Mark Haines
5002056b16 SYN-377: Make sure that the StreamIdGenerator.get_next.__exit__ is called from the main thread after the transaction completes, not from database thread before the transaction completes. 2015-05-12 11:20:40 +01:00
Mark Haines
5c75adff95 Add a NotifierUserStream to hold all the notification listeners for a user 2015-05-12 11:00:37 +01:00
Erik Johnston
367382b575 Handle the case where the other side is unreachable when backfilling 2015-05-12 10:35:45 +01:00
Erik Johnston
4df11b5039 Make get_current_token accept a direction parameter, which tells whether the source whether we want a token for going 'forwards' or 'backwards' 2015-05-12 10:28:10 +01:00
Erik Johnston
84e6b4001f Initial hack at wiring together pagination and backfill 2015-05-11 18:01:31 +01:00
Erik Johnston
17653a5dfe Move storage.stream._StreamToken to types.RoomStreamToken 2015-05-11 18:01:01 +01:00
Mark Haines
e269c511f6 Don't bother passing the events to the notifier since it isn't using them 2015-05-11 15:01:51 +01:00
Mark Haines
5e3b254dc8 Use wait_for_events to implement 'get_events' 2015-05-11 14:37:33 +01:00
Erik Johnston
79b7154454 Merge pull request #146 from matrix-org/erikj/push_rules_fixes 
Fix 500 on push rule updates.
 2015-05-11 11:33:47 +01:00
Erik Johnston
4ef556f650 Bump version 2015-05-11 11:31:04 +01:00
Erik Johnston
b036596b75 Prefer to use _simple_*. 2015-05-11 11:24:01 +01:00
Erik Johnston
cd525c0f5a push_rules table expects an 'id' field 2015-05-11 11:24:01 +01:00
Erik Johnston
de87541862 Bump version 2015-05-10 10:51:08 +01:00
Erik Johnston
22d2f498fa Fix push rule bug: can't insert bool into small int column 2015-05-10 10:50:51 +01:00
Erik Johnston
2236ef6c92 Fix up leak. Add warnings. 2015-05-08 19:53:34 +01:00
Erik Johnston
da1aa07db5 Add some docs 2015-05-08 16:52:49 +01:00
Erik Johnston
4ac1941592 PEP8 2015-05-08 16:33:01 +01:00
Erik Johnston
476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Erik Johnston
fca28d243e Change the way we create observers to deferreds so that we don't get spammed by 'unhandled errors' 2015-05-08 16:28:08 +01:00
Erik Johnston
0cd1401f8d Bump version 2015-05-08 16:11:51 +01:00
Erik Johnston
724bb1e7d9 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-08 16:11:19 +01:00
Mark Haines
1c7912751e Drop the old table not the new table 2015-05-08 16:04:32 +01:00
Mark Haines
9d36eb4eab Rename unique constraint 2015-05-08 16:01:55 +01:00
Mark Haines
b0f71db3ff Remove unsigned 2015-05-08 15:59:51 +01:00
Mark Haines
84e1cacea4 Bump schema version 2015-05-08 15:58:14 +01:00
Mark Haines
6538d445e8 Make the timestamps in server_keys_json bigints 2015-05-08 15:55:17 +01:00
Erik Johnston
126d562576 Bump version 2015-05-08 13:29:37 +01:00
Paul "LeoNerd" Evans
45543028bb Use the presence cachemap ordering to early-abort the iteration loop 2015-05-07 22:40:10 +01:00
Paul "LeoNerd" Evans
f683b5de47 Store presence cachemap in an ordered dict, so that the newer serials will be at the end 2015-05-07 21:27:53 +01:00
Erik Johnston
db0dca2f6f Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-07 19:21:00 +01:00
Erik Johnston
a151693a3b Bump syweb version 2015-05-07 18:01:46 +01:00
Mark Haines
dfa98f911b revert accidental bcrypt gensalt round reduction from loadtesting 2015-05-07 17:45:42 +01:00
Mark Haines
ef8e8ebd91 pynacl-0.3.0 was released so we can finally start using it directly from pypi 2015-05-07 16:46:51 +01:00
David Baker
97a64f3ebe Merge branch 'develop' of github.com:matrix-org/synapse into develop 2015-05-07 09:33:42 +01:00
David Baker
b850c9fa04 Typo 2015-05-07 09:33:30 +01:00
Mark Haines
4a7a4a5b6c Optional profiling using cProfile 2015-05-06 17:08:00 +01:00
Mark Haines
b994fb2b96 Don't read from the config file before checking it exists 2015-05-06 12:56:47 +01:00
Erik Johnston
04c01882fc Bump version 2015-05-06 09:59:13 +01:00
Erik Johnston
0c4ac271ca Merge branch 'erikj/executemany' of github.com:matrix-org/synapse into erikj/SYN-371 2015-05-05 18:21:19 +01:00
Erik Johnston
0cf7e480b4 And use buffer(...) there as well 2015-05-05 18:20:01 +01:00
Erik Johnston
ed2584050f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemany 2015-05-05 18:15:20 +01:00
Erik Johnston
977338a7af Use buffer(...) when inserting into bytea column 2015-05-05 18:12:53 +01:00
Mark Haines
31049c4d72 Merge pull request #139 from matrix-org/bugs/SYN-369 
Fix race with cache invalidation. SYN-369
 2015-05-05 17:46:13 +01:00
Mark Haines
deb0237166 Add some doc-string 2015-05-05 17:45:11 +01:00
Mark Haines
e45b05647e Fix the --help option for synapse 2015-05-05 17:39:59 +01:00
Erik Johnston
3d5a955e08 Missed events are not outliers 2015-05-05 17:36:57 +01:00
Mark Haines
d18f37e026 Collect the invalidate callbacks on the transaction object rather than passing around a separate list 2015-05-05 17:32:21 +01:00
Erik Johnston
9951542393 Add a comment about the zip(*[zip(sorted(...),...)]) 2015-05-05 17:06:55 +01:00
Mark Haines
041b6cba61 SYN-369: Add comments to the sequence number logic in the cache 2015-05-05 16:32:44 +01:00
Mark Haines
63075118a5 Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access 2015-05-05 16:24:04 +01:00
Erik Johnston
531d7955fd Don't insert without deduplication. In this case we never actually use this table, so simply remove the insert entirely 2015-05-05 16:12:28 +01:00
Mark Haines
bfa4a7f8b0 Invalidate the room_member cache if the current state events updates 2015-05-05 15:43:49 +01:00
Mark Haines
d0fece8d3c Missing return for when the event was already persisted 2015-05-05 15:39:09 +01:00
Erik Johnston
bdcd7693c8 Fix indentation 2015-05-05 15:14:48 +01:00
Erik Johnston
43c2e8deae Add support for using executemany 2015-05-05 15:13:25 +01:00
Erik Johnston
1692dc019d Don't call 'encode_parameter' no-op 2015-05-05 15:00:30 +01:00
Mark Haines
a9aea68fd5 Invalidate the caches from the correct thread 2015-05-05 14:57:08 +01:00
Mark Haines
261d809a47 Sequence the modifications to the cache so that selects don't race with inserts 2015-05-05 14:13:50 +01:00
Erik Johnston
d9cc5de9e5 Correctly name transaction 2015-05-05 10:24:10 +01:00
Erik Johnston
b8940cd902 Remove some unused indexes 2015-05-01 16:14:25 +01:00
Erik Johnston
1942382246 Don't log enqueue_ 2015-05-01 16:14:25 +01:00
David Baker
eb9bd2d949 user_id now in user_threepids 2015-05-01 15:04:37 +01:00
Erik Johnston
2d386d7038 That wasn't a deferred 2015-05-01 14:41:25 +01:00
Erik Johnston
4ac2823b3c Remove inlineCallbacks from non-generator 2015-05-01 14:41:25 +01:00
Erik Johnston
22c7c5eb8f Typo 2015-05-01 14:41:25 +01:00
Erik Johnston
42c12c04f6 Remove some run_on_reactors 2015-05-01 14:41:25 +01:00
Erik Johnston
adb5b76ff5 Don't log all auth events every time we call auth.check 2015-05-01 14:41:25 +01:00
Mark Haines
3bcdf3664c Use the daemonize key from the config if it exists 2015-05-01 14:34:55 +01:00
David Baker
9eeb03c0dd Don't use self.execute: it's designed for fetching stuff 2015-05-01 14:21:25 +01:00
Mark Haines
7b50769eb9 Merge pull request #136 from matrix-org/markjh/config_cleanup 
Config restructuring.
 2015-05-01 14:04:39 +01:00
David Baker
7693f24792 No id field on user 2015-05-01 13:55:42 +01:00
Mark Haines
46a65c282f Allow generate-config to run against an existing config file to generate default keys 2015-05-01 13:54:38 +01:00
David Baker
92b20713d7 More missed get_user_by_id API changes 2015-05-01 13:45:54 +01:00
Erik Johnston
da4ed08739 One too many lens 2015-05-01 13:29:38 +01:00
Erik Johnston
9060dc6b59 Change public room list to use defer.gatherResults 2015-05-01 13:28:36 +01:00
David Baker
1fae1b3166 This api now no longer returns an array 2015-05-01 13:26:41 +01:00
Erik Johnston
80b4119279 Don't wait for storage of access_token 2015-05-01 13:14:05 +01:00
Erik Johnston
4011cf1c42 Cache latest_event_ids_in_room 2015-05-01 13:06:26 +01:00
Erik Johnston
657298cebd Don't lock user_ips table for upsert. 2015-05-01 10:46:48 +01:00
Erik Johnston
fabb7acd45 Fix bug where we reconnected to the database on every query. 2015-05-01 10:24:24 +01:00
Erik Johnston
23c639ff32 Split a storage function in two so that we don't have to do extra work. 2015-05-01 10:17:19 +01:00
Erik Johnston
8be5284e91 Remove pointless join 2015-04-30 18:49:26 +01:00
Erik Johnston
503e4d3d52 Fix broken SQL 2015-04-30 18:44:47 +01:00
Erik Johnston
00718ae7a9 Need more yield 2015-04-30 18:43:39 +01:00
Erik Johnston
0465560c1a Add missing param 2015-04-30 18:42:44 +01:00
Erik Johnston
61d05daab1 More join conditions 2015-04-30 18:41:05 +01:00
Erik Johnston
6ead27ddda Add more conditions on JOINs to make postgres go a little faster. 2015-04-30 18:32:03 +01:00
Mark Haines
50c87b8eed Allow "manhole" to be ommited from the config 2015-04-30 18:11:47 +01:00
Mark Haines
345995fcde Remove the ~, comment the lines instead 2015-04-30 18:10:19 +01:00
Mark Haines
62cebee8ee Update key.py 2015-04-30 17:54:01 +01:00
Mark Haines
95cbfee8ae Update metrics.py 2015-04-30 17:52:20 +01:00
Mark Haines
6ea9cf58be missing import 2015-04-30 17:21:21 +01:00
Erik Johnston
f383d5a801 Fix up get_current_state and get_room_name_and_aliases queries to parse events in transaction 2015-04-30 17:12:52 +01:00
Mark Haines
c95480963e read the pid_file from the config file in synctl 2015-04-30 17:12:15 +01:00
Mark Haines
2d4d2bbae4 Merge branch 'develop' into markjh/config_cleanup 
Conflicts:
	synapse/config/captcha.py
 2015-04-30 16:54:55 +01:00
Mark Haines
2f1348f339 Write a default log_config when generating config 2015-04-30 16:52:57 +01:00
Erik Johnston
69d4063651 Add get_rooms_for_user cache 2015-04-30 16:47:51 +01:00
David Baker
5b02f33451 Undo changes to logger config, ie. remove the access_log_file option: decision is to support this through log_config rather tan adding an option. 2015-04-30 16:21:39 +01:00
David Baker
054aa0d58c Do access log using python's logging stuff, just under a separate logger name 2015-04-30 16:21:38 +01:00
Erik Johnston
3c4c229788 Don't use sub queries, it makes postgres sad 2015-04-30 16:16:53 +01:00
Mark Haines
74aaacf82a Don't break when sizes or durations are given as integers 2015-04-30 16:04:02 +01:00
Mark Haines
29400b45b9 SYN-367: Use upsert rather than insert_or_replace 2015-04-30 15:21:31 +01:00
Mark Haines
c28f1d16f0 Add a random string to the auto generated key id 2015-04-30 15:13:14 +01:00
Mark Haines
265f30bd3f Allow --enable-registration to be passed on the commandline 2015-04-30 15:04:06 +01:00
Mark Haines
c9e62927f2 Use disable_registration keys if they are present 2015-04-30 14:34:09 +01:00
Erik Johnston
2366d28780 Don't needlessly join on state_events 2015-04-30 14:02:06 +01:00
David Baker
d89a9f7283 Add an access_log 
SYN-161 #resolve
 2015-04-30 13:58:13 +01:00
Mark Haines
1aa11cf7ce Allow multiple config files, set up a default config before applying the config files 2015-04-30 13:48:15 +01:00
David Baker
0c1b7f843b Unused import 2015-04-30 13:33:30 +01:00
David Baker
4b46fbec5b Doesn't look like this is used anymore 2015-04-30 12:04:08 +01:00
Erik Johnston
1d7702833d Make simple query rather than long one and then throw away half the results 2015-04-30 10:16:12 +01:00
Mark Haines
6b69ddd17a remove duplicate parse_size method 2015-04-30 04:26:29 +01:00
Mark Haines
d624e2a638 Manually generate the default config yaml, remove most of the commandline arguments for synapse anticipating that people will use the yaml instead. Simpify implementing config options by not requiring the classes to hit the super class 2015-04-30 04:24:44 +01:00
Erik Johnston
b1ca784aca Correctly decode, for sqlite and postgres, rows from pushers table 2015-04-29 19:41:14 +01:00
Erik Johnston
4a9dc5b2f5 pushkey' are also bytes. 2015-04-29 19:27:02 +01:00
Erik Johnston
0ade2712d1 Typo 2015-04-29 19:17:25 +01:00
Erik Johnston
50f96f256f Also remove yield from within lock in the other generator 2015-04-29 19:17:00 +01:00
Erik Johnston
d2d61a8288 Fix deadlock in id_generators. No idea why this was an actual deadlock. 2015-04-29 19:15:23 +01:00
Erik Johnston
3e71d13acf Also log when we've started pushers 2015-04-29 18:37:42 +01:00
Erik Johnston
e7a6edb0ee Revert previous 2015-04-29 18:37:30 +01:00
Erik Johnston
c27d6ad6b5 Only start pushers when synapse has fully started 2015-04-29 18:25:24 +01:00
Erik Johnston
46daf2d200 Start pushers on reactor thread 2015-04-29 18:22:20 +01:00
Erik Johnston
3864b3a8e6 Actually return rows 2015-04-29 18:07:36 +01:00
Erik Johnston
0618978238 Typo, args wrong way round 2015-04-29 18:04:35 +01:00
Erik Johnston
09177f4f2e Decode buffers in same thread 2015-04-29 18:03:42 +01:00
Erik Johnston
472be88674 We store pusher data as bytes 2015-04-29 17:43:46 +01:00
Erik Johnston
a6e62cf6d0 Fix off by one in presence token handling 2015-04-29 17:37:11 +01:00
David Baker
12d381bd5d Decode the data json in the storage layer (was moved but this part was missed) 2015-04-29 17:13:51 +01:00
David Baker
f8c30faf25 Oops, update the contraint too 2015-04-29 16:58:42 +01:00
David Baker
61cd5d9045 Be more postgressive 2015-04-29 16:57:14 +01:00
David Baker
fb95035a65 Be postgressive 2015-04-29 16:53:41 +01:00
David Baker
4669def000 Oops, forgot the schema delta file 2015-04-29 16:50:16 +01:00
Erik Johnston
0337eaf321 txn.execute doesn't return cursors 2015-04-29 16:43:39 +01:00
Erik Johnston
884fb88e28 txn.execute doesn't return cursors 2015-04-29 16:35:20 +01:00
Erik Johnston
d76c058eea Fix invalid SQL to work in postgres land 2015-04-29 16:30:25 +01:00
David Baker
9927170787 Accept camelcase + underscores in binding too 2015-04-29 15:57:09 +01:00
David Baker
109c8aafd2 Fix includes 2015-04-29 15:45:44 +01:00
David Baker
b7788f80a3 Accept both camelcase and underscore threepid creds for transition 2015-04-29 15:41:29 +01:00
Erik Johnston
c8ed9bd278 pushers table requires an access_token 2015-04-29 15:33:27 +01:00
Mark Haines
f2d90d5c02 Fix whitespace 2015-04-29 14:53:23 +01:00
Mark Haines
845b0b2c97 Check requirements before doing anything else when running the homeserver 2015-04-29 14:52:42 +01:00
Mark Haines
c0036ced54 bump syutil to 0.0.6 2015-04-29 14:16:41 +01:00
Erik Johnston
970a9b9d2b We can't use REPLACE when upgrading databases now we have postgres 2015-04-29 13:55:44 +01:00
Erik Johnston
64991b0c8b Merge pull request #129 from matrix-org/key_distribution 
Key distribution v2
 2015-04-29 13:34:38 +01:00
Mark Haines
e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Mark Haines
1319905d7a Use a defer.gatherResults to collect results from the perspective servers 2015-04-29 13:31:14 +01:00
Mark Haines
a9549fdce3 Use bytea rather than BLOB 2015-04-29 13:16:09 +01:00
Mark Haines
4ad8b45155 Merge branch 'develop' into key_distribution 
Conflicts:
	synapse/config/homeserver.py
 2015-04-29 13:15:14 +01:00
Mark Haines
74874ffda7 Update the query format used by keyring to match current key v2 spec 2015-04-29 12:14:08 +01:00
Erik Johnston
cd0864121b Make postgres database error slightly more helpful 2015-04-29 12:12:25 +01:00
Erik Johnston
4932a7e2d9 Use __all__ instead of assert to stop pyflakes from warning about unused import in __init__ 2015-04-29 12:12:25 +01:00
Mark Haines
9894da6a29 Merge branch 'develop' into erikj/postgres_charset_check 2015-04-29 11:58:06 +01:00
Mark Haines
46d200a3a1 Implement minimum_valid_until_ts in the remote key resource 2015-04-29 11:57:26 +01:00
Erik Johnston
a08bf11138 Appease PEP8 2015-04-29 11:44:48 +01:00
Erik Johnston
204132a998 Check that postgres database has correct charset set 2015-04-29 11:42:28 +01:00
Mark Haines
f4c9ebbc34 Delete ugly commented out log line. 2015-04-29 11:07:13 +01:00
Matthew Hodgson
68c0603946 comment out ugly test logline 2015-04-29 00:14:44 +01:00
Erik Johnston
1783c7ca92 Ensure we never miss any presence updates 2015-04-28 17:24:24 +01:00
Erik Johnston
0126ef7f3c Fix typo 2015-04-28 17:23:53 +01:00
Erik Johnston
d98edb548a Ensure the serial returned by presence is always an integer 2015-04-28 17:20:32 +01:00
Erik Johnston
073b891ec1 Remove unused imports 2015-04-28 13:44:23 +01:00
Erik Johnston
327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston
18f8247701 Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit 2015-04-28 12:41:33 +01:00
Mark Haines
9182f87664 Merge pull request #126 from matrix-org/csauth 
Client / Server Auth Refactor
 2015-04-28 11:00:27 +01:00
Mark Haines
55e1bc8920 And don't bump the schema version unnecessarily 2015-04-28 10:54:15 +01:00
Mark Haines
55fcf62e9c Merge pull request #133 from matrix-org/invite_power_level 
Invite power level
 2015-04-28 10:50:34 +01:00
Mark Haines
b96c133034 Add server_keys.sql to the current delta rather than creating a new delta 2015-04-28 10:50:00 +01:00
Mark Haines
252e6f6869 Merge pull request #130 from matrix-org/server_rename_check 
Fix for SYN-266
 2015-04-28 10:39:31 +01:00
Mark Haines
0bc71103e1 Output vim style mode markers into the yaml config file 2015-04-28 10:17:10 +01:00
Mark Haines
f8b865264a Merge branch 'develop' into key_distribution 
Conflicts:
	synapse/crypto/keyring.py
 2015-04-27 18:29:32 +01:00
Paul "LeoNerd" Evans
38432d8c25 Merge branch 'develop' into invite_power_level 2015-04-27 17:09:25 +01:00
Erik Johnston
42b7139dec Remove unused import 2015-04-27 15:59:56 +01:00
Erik Johnston
1ef66cc3bd Move database configuration into config module 2015-04-27 15:57:43 +01:00
Erik Johnston
416a3e6c4f Ensure check_same_thread is enabled for sqlite3 2015-04-27 15:44:30 +01:00
Erik Johnston
8558e1ec73 Make get_max_token into inlineCallbacks so that the lock works. 2015-04-27 15:19:44 +01:00
Erik Johnston
6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
Erik Johnston
2c70849dc3 Fix newlines 2015-04-27 14:38:29 +01:00
Erik Johnston
0a016b0525 Pull inner function out. 2015-04-27 14:37:24 +01:00
Erik Johnston
e701aec2d1 Implement locks using create_observer for fetching media and server keys 2015-04-27 14:20:26 +01:00
David Baker
412ece18e7 Add commentage. 2015-04-27 14:08:45 +01:00
Erik Johnston
1c82fbd2eb Implement create_observer. 
`create_observer` takes a deferred and create a new deferred that
*observers* the original deferred. Any callbacks added to the observing
deferred will *not* affect the origin deferred.
 2015-04-27 13:59:37 +01:00
Erik Johnston
2732be83d9 Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible. 2015-04-27 13:22:30 +01:00
Erik Johnston
e4c4664d73 Handle the fact that postgres databases can be restarted from under us 2015-04-27 12:40:49 +01:00
David Baker
03c4f0ed67 pep8 2015-04-27 12:36:59 +01:00
David Baker
f1acb9fd40 logging args 2015-04-27 11:56:34 +01:00
David Baker
8a5be236e0 pep8 2015-04-27 11:49:18 +01:00
David Baker
df75914791 pep8 2015-04-27 11:48:33 +01:00
David Baker
b02e1006b9 Run database check before daemonizing, at the cost of database hygiene. 2015-04-27 11:46:00 +01:00
David Baker
f8152f2708 rename db method to be more informative 2015-04-27 10:16:26 +01:00
David Baker
2f475bd5d5 pep8 2015-04-24 18:15:07 +01:00
David Baker
a7b51f4539 Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266 2015-04-24 18:11:21 +01:00
Mark Haines
288702170d Add config for setting the perspective servers 2015-04-24 17:01:34 +01:00
David Baker
1bac74b9ae Change to https for ID server communication 2015-04-24 14:48:49 +01:00
David Baker
ed83638668 Make one-to-one rule an underride otherwise bings don't work in one-to-one wrooms. Likewise a couple of other rules. 2015-04-24 14:27:17 +01:00
David Baker
7ac8a60c6f More underscores 2015-04-24 11:44:27 +01:00
Mark Haines
c253b14f6e Merge branch 'develop' into key_distribution 2015-04-24 11:29:46 +01:00
Mark Haines
bdcb23ca25 Fix spelling 2015-04-24 11:29:19 +01:00
Mark Haines
b2c2dc8940 Merge branch 'develop' into key_distribution 2015-04-24 11:28:10 +01:00
Mark Haines
869dc94cbb Call the super classes when generating config 2015-04-24 11:27:56 +01:00
David Baker
a218619626 Use underscores instead of camelcase for id server stuff 2015-04-24 11:27:38 +01:00
Mark Haines
b1e68add19 Add a config file for perspective servers 2015-04-24 11:26:19 +01:00
Mark Haines
31e262e6b4 Copyright notice 2015-04-24 10:36:51 +01:00
Mark Haines
eede182df7 Merge branch 'develop' into key_distribution 2015-04-24 10:35:49 +01:00
Mark Haines
4e2f8b8722 Copyright notices 2015-04-24 10:35:29 +01:00
Mark Haines
c8c710eca7 Move the key related config parser into a separate file 2015-04-24 10:22:22 +01:00
Mark Haines
149ed9f151 Better help for the old-signing-key option 2015-04-24 10:07:55 +01:00
David Baker
f7a79a37be pep8 2015-04-24 09:42:37 +01:00
David Baker
6532b6e607 Merge branch 'develop' into csauth 
Conflicts:
	synapse/http/server.py
 2015-04-24 09:37:54 +01:00
David Baker
74270defda No commas here, otherwise our error string constants become tuples. 2015-04-24 09:28:57 +01:00
Paul "LeoNerd" Evans
e1e5e53127 Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again 2015-04-23 19:01:37 +01:00
Paul "LeoNerd" Evans
b3bda8a75f Don't let the remote offline serial list grow arbitrarily large 2015-04-23 18:40:47 +01:00
Paul "LeoNerd" Evans
8a785c3006 Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261) 2015-04-23 18:40:19 +01:00
Paul "LeoNerd" Evans
191f7f09ce Generate presence event-stream JSON structures directly 2015-04-23 18:27:25 +01:00
David Baker
03eb4adc6e Dedicated error code for failed 3pid auth verification 2015-04-23 18:20:17 +01:00
Mark Haines
4bbf7156ef Update to match the specification for key/v2 2015-04-23 16:39:13 +01:00
David Baker
0eb61a3d16 Remove ultimately unused feature of saving params from the first call in the session: it's probably too open to abuse. 2015-04-23 14:44:12 +01:00
David Baker
a2c10d37d7 Add an error code to 'missing token' response. 2015-04-23 13:23:44 +01:00
David Baker
2e0d9219b9 Remove now-redundant email config 2015-04-23 11:45:29 +01:00
Mark Haines
f30d47c876 Implement remote key lookup api 2015-04-22 14:21:08 +01:00
Paul "LeoNerd" Evans
a16eaa0c33 Neater fetching of user's auth level in a room - squash to int() at access time (SYN-353) 2015-04-22 14:20:04 +01:00
Paul "LeoNerd" Evans
f43063158a Appease pep8 2015-04-22 13:12:11 +01:00
Paul "LeoNerd" Evans
2808c040ef Also remember to check 'invite' level for changes 2015-04-21 21:13:14 +01:00
Paul "LeoNerd" Evans
48b6ee2b67 Create an 'invite' powerlevel when making new rooms 2015-04-21 21:07:35 +01:00
Paul "LeoNerd" Evans
bc41f0398f Initial implementation of an 'invite' power_level 2015-04-21 20:56:08 +01:00
Paul "LeoNerd" Evans
d3309933f5 Much neater fetching of defined powerlevels from m.room.power_levels state event 2015-04-21 20:53:23 +01:00
Paul "LeoNerd" Evans
b568c0231c Remove debugging print statement accidentally committed 2015-04-21 20:21:14 +01:00
Paul "LeoNerd" Evans
3a7d7a3f22 Sanitise a user's powerlevel to an int() before numerical comparison, because otherwise Python is "helpful" with it (SYN-351) 2015-04-21 20:18:29 +01:00
Mark Haines
3ba522bb23 Merge branch 'develop' into key_distribution 2015-04-21 17:10:25 +01:00
Mark Haines
6080830bef Bump syutil version to 0.0.5 2015-04-21 17:04:06 +01:00
Mark Haines
8b183781cb Merge pull request #128 from matrix-org/unify_http_wrappers 
Unify http wrappers
 2015-04-21 17:01:04 +01:00
Mark Haines
812a99100b Set a version_string in BaseMediaResource so that the request_handler wrapper works 2015-04-21 16:43:58 +01:00
Mark Haines
1967650bc4 Combine the request wrappers in rest/media/v1 and http/server into a single wrapper decorator 2015-04-21 16:35:53 +01:00
Mark Haines
1ebff9736b Split out the JsonResource request logging and error handling into a separate wrapper function 2015-04-21 16:07:20 +01:00
Mark Haines
24d21887ed SYN-350: Don't ratelimit the individual events generated during room creation 2015-04-21 14:14:19 +01:00
Mark Haines
db8d4e8dd6 Merge branch 'develop' into key_distribution 2015-04-20 16:24:21 +01:00
Mark Haines
2f9157b427 Implement v2 key lookup 2015-04-20 16:23:47 +01:00
David Baker
91c8f828e1 pep8 2015-04-17 19:56:04 +01:00
David Baker
8db6832db8 Password reset, finally. 2015-04-17 19:53:47 +01:00
David Baker
117f35ac4a Add endpoint to get threepids from server 2015-04-17 17:20:18 +01:00
David Baker
4eea5cf6c2 pep8 2015-04-17 16:46:45 +01:00
David Baker
f96ab9d18d make add3pid servlet work 2015-04-17 16:44:49 +01:00
Erik Johnston
865398b4a9 Revert needless change to rest.profile 2015-04-17 16:14:51 +01:00
Erik Johnston
e3417bbbe0 Revert needless change to storage.profile 2015-04-17 16:13:14 +01:00
Erik Johnston
2492efb162 Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-17 16:08:42 +01:00
Erik Johnston
4a5990ff8f Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-04-17 16:08:28 +01:00
Erik Johnston
5e7a90316d Update --database-path metavar to SQLITE_DATABASE_PATH 2015-04-17 16:08:18 +01:00
David Baker
0b1a8500a2 just the once would probably be fine 2015-04-17 13:53:54 +01:00
David Baker
cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
David Baker
bf5e54f255 Register the 3pid servlet 2015-04-17 13:44:55 +01:00
David Baker
94e1e58b4d password -> account servlet and add start of an 'add 3pid' endpoint 2015-04-17 13:44:12 +01:00
Erik Johnston
ced39d019f Bump version 2015-04-17 13:25:16 +01:00
David Baker
83b554437e Need to yield the username check, otherwise very very weird things happen. 2015-04-17 12:57:25 +01:00
Erik Johnston
6ba2e3df4e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-17 11:22:31 +01:00
Erik Johnston
0ec346d942 Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea 2015-04-17 11:10:20 +01:00
David Baker
4cd5fb13a3 Oops, left debugging in. 2015-04-16 20:03:13 +01:00
David Baker
ea1776f556 Return user ID in use error straight away 2015-04-16 19:56:44 +01:00
Erik Johnston
e1c0970c11 PEP8 2015-04-16 11:18:45 +01:00
Erik Johnston
b8092fbc82 Go back to storing JSON in TEXT 2015-04-16 11:17:52 +01:00
Erik Johnston
bc9e69e160 Move encoding and decoding of JSON into storage layer 2015-04-16 11:01:09 +01:00
Paul "LeoNerd" Evans
f2cf37518b Filter typing nofication events to only those rooms the requesting user is a member of (SYN-328) 2015-04-15 23:34:16 +01:00
Paul "LeoNerd" Evans
0268d40281 Have TypingNotificationEventSource.get_new_events_for_user() return a deferred, for consistency and extensibility 2015-04-15 23:09:35 +01:00
Paul "LeoNerd" Evans
399b5add58 Neater implementation of membership change auth checks, ensuring we can't forget to check if the calling user is a member of the room 2015-04-15 18:40:23 +01:00
Paul "LeoNerd" Evans
e6e130b9ba Ensure that non-room-members cannot ban others, even if they do have enough powerlevel (SYN-343) 2015-04-15 18:07:33 +01:00
David Baker
766bd8e880 Dummy login so we can do the first POST request to get login flows without it just succeeding 2015-04-15 17:14:25 +01:00
Erik Johnston
ffad75bd62 Remove mysql/maria support 2015-04-15 17:00:50 +01:00
Mark Haines
a429515bdd Add methods for storing and retrieving the raw key json 2015-04-15 16:58:35 +01:00
Mark Haines
8d761134c2 Fail quicker for 4xx responses in the key client, optional hit a different API path 2015-04-15 16:57:58 +01:00
Erik Johnston
cf04cedf21 Change full_schemas/11 to work with postgres 2015-04-15 16:53:47 +01:00
Erik Johnston
5b31afcbd1 Remove debug logging 2015-04-15 16:27:04 +01:00
Erik Johnston
6e91f14d09 Add missing yield 2015-04-15 16:25:07 +01:00
Erik Johnston
ed26e4012b pushers table requires a unique id. 2015-04-15 16:24:14 +01:00
Erik Johnston
806f380a8b Make LruCache thread safe, as its used for event cache 2015-04-15 16:07:59 +01:00
David Baker
a19b739909 Regstration with email in v2 2015-04-15 15:50:38 +01:00
Erik Johnston
a5c72780e6 Don't pass in removed flag 2015-04-15 15:13:22 +01:00
Erik Johnston
e19f794fee Change from exception to warn 2015-04-15 15:12:57 +01:00
Erik Johnston
d5ff9effcf Don't wait on federation_handler.handle_new_event 2015-04-15 15:05:57 +01:00
Erik Johnston
e845434028 Remove run_on_reactor()s 2015-04-15 15:05:45 +01:00
Erik Johnston
4af32a2817 Postgres does not allow you to continue using a cursor after a DB exception has been raised, so move _simple_insert or_ignore flag out of transaction 2015-04-15 14:51:21 +01:00
Erik Johnston
bc6cef823f Do more parellelization for initialSync 2015-04-15 14:21:59 +01:00
Erik Johnston
1ec6fa98c9 Parellelize initial sync 2015-04-15 14:17:16 +01:00
Erik Johnston
25d2914fba Turn off persistance of sent_transactions.response_json 2015-04-15 14:09:35 +01:00
Erik Johnston
cce5d057d3 Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses 2015-04-15 11:43:25 +01:00
Erik Johnston
6606f7c659 Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-15 10:27:20 +01:00
Erik Johnston
a971fa9d58 Use try..finally in contextlib.contextmanager 2015-04-15 10:25:43 +01:00
Erik Johnston
ded4128965 Use True for True rather than 1 2015-04-15 10:24:24 +01:00
Erik Johnston
f9e12f79ca Add missing yield in storage func 2015-04-15 10:24:07 +01:00
Erik Johnston
c756dfeb14 Correctly identify deadlocks 2015-04-15 10:23:42 +01:00
Mark Haines
32e14d8181 Return a sha256 fingerprint rather than the entire tls certificate 2015-04-14 19:10:09 +01:00
Mark Haines
88cb06e996 Update syutil version to 0.0.4 2015-04-14 16:18:17 +01:00
Mark Haines
d488463fa3 Add a version 2 of the key server api 2015-04-14 16:04:52 +01:00
Erik Johnston
127fad17dd Add postgres database engine 2015-04-14 14:50:29 +01:00
Erik Johnston
5a95cd4442 Rename user_ips.user -> user_id 2015-04-14 13:54:09 +01:00
Erik Johnston
58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Mark Haines
3cbc286d06 Move server key api into rest/key/v1 2015-04-14 13:28:11 +01:00
Erik Johnston
3c741682e5 Correctly increment the _next_id initially 2015-04-14 09:54:44 +01:00
Erik Johnston
86fc9b617c For backwards compat, make state_groups.id have a type of int, not varchar 2015-04-13 17:03:49 +01:00
Erik Johnston
93937b2b31 Remove duplicate rows 2015-04-10 18:47:03 +01:00
Erik Johnston
c5365dee56 Use case sensitive collations 2015-04-10 18:46:33 +01:00
Erik Johnston
4103b1c470 DROP indexes before recreating them 2015-04-10 18:46:09 +01:00
Erik Johnston
4d5b098626 Use LONGBLOB and TEXT for arbitary length rows 2015-04-10 18:45:09 +01:00
Erik Johnston
7ed2ec3061 Handle the fact that in sqlite binary data might be stored as unicode or bytes 2015-04-10 13:41:54 +01:00
Erik Johnston
ce797ad373 Bump schema version 2015-04-10 11:22:30 +01:00
Erik Johnston
7e863c51e6 Use unsigned bigint 2015-04-10 11:22:04 +01:00
Erik Johnston
0f12772e32 SQLite wants INTEGER and not BIGINT for primary keys 2015-04-10 11:16:09 +01:00
Erik Johnston
d5d4281647 Update full_schemas/16 to match delta files. Add delta/16 scripts 2015-04-10 10:59:46 +01:00
Erik Johnston
cda4a6f93f Revert non-trivial changes to upgrade scripts 2015-04-10 10:19:50 +01:00
Erik Johnston
e2722f58ee Fix schema again 2015-04-10 10:16:29 +01:00
Erik Johnston
a1665c5094 Revert non-trivial schema changes and move them to a new schema version. 2015-04-10 10:05:44 +01:00
Erik Johnston
2ded344620 Remove unused import 2015-04-09 13:46:06 +01:00
Erik Johnston
9707acfc40 Remove spurious spaces 2015-04-09 13:45:20 +01:00
Erik Johnston
8bf285e082 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-09 13:13:26 +01:00
Mark Haines
1280a47fc6 Add comment 2015-04-09 11:42:21 +01:00
Mark Haines
23d285ad57 Unset the timer in the timeout callback so that we don't try to cancel it if it has been called 2015-04-09 11:41:50 +01:00
Erik Johnston
8ad0f4912e
Stream ordering and out of order insertions. 
Handle the fact that events can be persisted out of order, and so to get
the "current max" stream token becomes non trivial - as we need to make
sure that *all* stream tokens less than the current max have also
successfully been persisted.
 2015-04-09 11:41:36 +01:00
Mark Haines
6f9dea7483 SYN-339: Cancel the notifier timeout when the notifier fires 2015-04-09 11:07:20 +01:00
Erik Johnston
279a547a8b Use generic db exceptions rather than sqlite3 specific ones 2015-04-08 16:53:48 +01:00
Mark Haines
83f5125d52 Merge pull request #114 from matrix-org/improve_get_event_cache 
Improve get event cache
 2015-04-08 16:50:27 +01:00
Erik Johnston
a43b40449b Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-08 16:46:56 +01:00
Erik Johnston
9cef051ce2 Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-04-08 16:43:17 +01:00
Erik Johnston
45131b2bca Bump version 2015-04-08 16:35:12 +01:00
Erik Johnston
ccda401dbf SYN-338: Fix typo that caused the cache to throw an exception in some instances 2015-04-08 16:34:23 +01:00
Erik Johnston
3887350e47 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-08 14:11:07 +01:00
Erik Johnston
19234cc6c3 typo 2015-04-08 14:10:06 +01:00
Erik Johnston
e8f1521605 Don't yield on notifying all listeners 2015-04-08 14:08:30 +01:00
Mark Haines
605941ee26 Merge pull request #117 from matrix-org/notifier-leak 
Fix a notifier leak
 2015-04-08 14:02:37 +01:00
Erik Johnston
5bc41fe9f8 Move comment into docstring 2015-04-08 14:01:22 +01:00
Erik Johnston
638be5a6b9 Factor out loops into '_discard_if_notified' 2015-04-08 13:58:32 +01:00
Erik Johnston
830d07db82 Also perform paranoia checks in 'on_new_user_event' 2015-04-08 13:40:20 +01:00
Erik Johnston
65f5e4e3e4 Add paranoia checks to make sure that we evict stale NotificationListeners when we are about to process them 2015-04-08 13:33:38 +01:00
Erik Johnston
07d4041709 Fix bug where we didn't inform the NotificataionListeners about new rooms they have been subscribed to. This meant that the listeners didn't clean themselves up fully from all the dicts 2015-04-08 13:33:38 +01:00
Erik Johnston
c1b34af441 Move database timer logging to seperate logger 2015-04-08 13:31:19 +01:00
Erik Johnston
9a05795619 Retry transaction, not SQL query 2015-04-08 13:11:28 +01:00
Erik Johnston
24d8134ac1 Fix maria engine to correctly recognize deadlocks 2015-04-08 13:10:54 +01:00
Erik Johnston
d5e7e6b9b6 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:17:22 +01:00
Erik Johnston
0775c62469 Fix --enable-registration flag to work if you don't give a value 2015-04-07 18:16:23 +01:00
Erik Johnston
38928c6609 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:08:38 +01:00
Erik Johnston
4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Kegan Dougal
ae8ff92e05 Fix a bug which causes a send event level of 0 to not be honoured. 
Caused by a bad if check, which incorrectly executes for both 0 and None,
when None was the original intent.
 2015-04-07 15:48:20 +01:00
Erik Johnston
49d6aa1394 Retry on deadlock 2015-04-07 15:28:37 +01:00
Erik Johnston
0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00
Erik Johnston
6bc9edd8b2 Fix prepare_sqlite3_database's convert_param_style 2015-04-07 12:13:58 +01:00
Erik Johnston
05a35d62b6 Bump database version 2015-04-07 12:10:15 +01:00
Erik Johnston
8574bf62dc Add index to presence table 2015-04-07 12:09:36 +01:00
Erik Johnston
0af5f5efaf Don't use multiple UNIQUE constraints; it will cause deadlocks 2015-04-07 12:08:35 +01:00
Erik Johnston
c8d3f6486d Implement or_ignore flag on inserts 2015-04-07 12:06:22 +01:00
Erik Johnston
304111afd0 Don't use AUTOINCREMENT, use an in memory version 2015-04-07 12:05:36 +01:00
Erik Johnston
d0e444a648 Explicitly name the __main__ module logger 2015-04-07 12:04:02 +01:00
David Baker
4eb6d66b45 Add app service auth back in to v2 register 2015-04-02 17:51:19 +01:00
David Baker
6b59650753 Throw sensible errors on not-json when allowing empty body 2015-04-02 17:45:16 +01:00
David Baker
41cd778d66 pep8 2015-04-02 17:06:17 +01:00
David Baker
70a84f17f3 Add shared secret auth into register v2 and switch the script over. 2015-04-02 17:01:29 +01:00
Erik Johnston
779f7b0f44 Fix unicode support 2015-04-02 10:06:22 +01:00
Paul "LeoNerd" Evans
ef1e019840 Appease pep8 2015-04-01 19:17:38 +01:00
Paul "LeoNerd" Evans
5583e29513 Report process open filehandles in metrics 2015-04-01 19:15:23 +01:00
David Baker
c5bf0343e8 Explain how I justified to myself making JsonResource not always send JSON. 2015-04-01 15:13:14 +01:00
Erik Johnston
e24c32e6f3 Fix SQLite support 2015-04-01 15:09:51 +01:00
David Baker
e9c908ebc0 Completely replace fallback auth for C/S V2: 
* Now only the auth part goes to fallback, not the whole operation
 * Auth fallback is a normal API endpoint, not a static page
 * Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
 2015-04-01 15:05:30 +01:00
Erik Johnston
9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal
813e54bd5b Fix more AS sender ID thinkos. 
Specifically, the ASes own user ID wasn't being treated as 'exclusive' so
a human could nab it. Also, the HS would needlessly send user queries to the
AS for its own user ID.
 2015-04-01 14:05:24 +01:00
Kegan Dougal
9fa8bda099 Merge branch 'develop' into application-services-registration-script 2015-04-01 10:19:17 +01:00
David Baker
f129ee1e18 Make docs a bit more true 2015-03-31 18:25:10 +01:00
Kegan Dougal
09cbff174a Fix thinko whereby events *for the AS specifically* were not passed on. 
This was caused by not explicitly checking the service.sender field. This
has now been fixed and a regression test has been added.
 2015-03-31 16:44:45 +01:00
David Baker
d18e7779ca Grammar and deduplication 2015-03-31 14:40:02 +01:00
Kegan Dougal
5e88a09a42 Add same user_id char checks as registration. 2015-03-31 14:00:25 +01:00
Kegan Dougal
cf1fa59f4b Use a sender localpart instead of a user ID. 
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
 2015-03-31 13:48:03 +01:00
Kegan Dougal
3470cb36a8 Pyflakes 2015-03-31 13:03:31 +01:00
Kegan Dougal
c217504949 Edit SQL schema to use string IDs not ints. Use token as ID. Update tests. 2015-03-31 12:07:56 +01:00
Kegan Dougal
b59aa74556 Fix tests and missing returns on deferreds. 2015-03-31 11:35:45 +01:00
Kegan Dougal
d33ae65efc Remove more reg/unreg methods. Read config not database for cache. 2015-03-31 11:00:00 +01:00
David Baker
9f642a93ec pep8 2015-03-31 09:50:44 +01:00
Kegan Dougal
e7887e37a8 Remove appservice REST servlets 2015-03-31 09:32:40 +01:00
Kegan Dougal
af853a4cdb Add AppServiceConfig 2015-03-31 09:22:31 +01:00
David Baker
59bf16eddc New registration for C/S API v2. Only ReCAPTCHA working currently. 2015-03-30 18:13:10 +01:00
Matthew Hodgson
8366fde82f turn --disable-registration into --enable-registration, given the default is for registration to be disabled by default now. this is backwards incompatible by removing the old --disable-registration arg, but makes for a much more intuitive arg 2015-03-30 12:01:09 -04:00
Paul "LeoNerd" Evans
3e420aebd8 Revert "Add another @cached wrapper, this time on get_presence_state()" 
This reverts commit ff1fa0fbf8.
 2015-03-27 16:16:58 +00:00
Paul "LeoNerd" Evans
ff1fa0fbf8 Add another @cached wrapper, this time on get_presence_state() 2015-03-27 15:57:16 +00:00
David Baker
6f4f7e4e22 pep8 2015-03-26 14:12:06 +00:00
David Baker
a32e876ef4 Delete pushers when changing password 2015-03-26 13:40:16 +00:00
Paul "LeoNerd" Evans
a198894bf7 Appease pep8 2015-03-26 11:53:58 +00:00
Kegan Dougal
32206dde3f Fixes from PR comments 2015-03-26 10:11:52 +00:00
Kegan Dougal
4edcbcee3b Merge branch 'develop' into application-services-txn-reliability 
Conflicts:
	synapse/storage/__init__.py
 2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans
953e40f9dc Implement the main getEvent cache using Cache() instead of a custom application of LruCache; also unify its two-level structure into just one 2015-03-25 19:12:16 +00:00
David Baker
df4c12c762 pep8 blank lines 2015-03-25 19:08:17 +00:00
David Baker
c1a256cc4c Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API. 2015-03-25 19:06:22 +00:00
Paul "LeoNerd" Evans
f173d40a32 Use FrozenEvent's reject_reason to decide whether to return it; don't include allow_rejected in the main getEvents cache key 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
1b988b051b Store the rejected reason in (Frozen)Event structs 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
033a517feb Indirect invalidations of _get_event_cache via a helper method to keep all uses of the cache lexically within one .py file 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
9ba6487b3f Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict() 2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans
d6b3ea75d4 Implement the 'key in dict' test for LruCache() 2015-03-25 19:04:59 +00:00
Erik Johnston
0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
David Baker
9aa0224cdf unused import 2015-03-24 17:25:59 +00:00
David Baker
c7023f2155 1) Pushers are now associated with an access token 
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
 2015-03-24 17:24:15 +00:00
Erik Johnston
0ba393924a Escape non printing ascii character 2015-03-24 16:31:52 +00:00
Erik Johnston
f488293d96 Don't reinsert into event_edges 2015-03-24 16:20:26 +00:00
Erik Johnston
1aa44939fc Fix bugs in transactions storage 2015-03-24 16:20:05 +00:00
Erik Johnston
5a447098dd Don't use room hosts table 2015-03-24 16:19:24 +00:00
Erik Johnston
9e98f1022a Don't order by rowid 2015-03-24 16:19:01 +00:00
Erik Johnston
9115421ace Use _simple_upsert 2015-03-24 16:17:39 +00:00
David Baker
d19e79ecc9 Make deleting other access tokens when you change your password actually work 2015-03-24 15:33:48 +00:00
Paul "LeoNerd" Evans
ed008e85a8 Reduce activity timer granularity to avoid too many quick updates (SYN-247) 2015-03-23 17:25:50 +00:00
Erik Johnston
6e7131f02f Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable. 2015-03-23 15:38:56 +00:00
Erik Johnston
9a7f496298 Sanitize RoomMemberStore 2015-03-23 15:29:04 +00:00
David Baker
78adccfaf4 pep8 / pyflakes 2015-03-23 14:23:51 +00:00
David Baker
d98660a60d Implement password changing (finally) along with a start on making client/server auth more general. 2015-03-23 14:20:28 +00:00
Erik Johnston
d5272b1d2c Use 'update or insert' rather than on 'conflict replace' 2015-03-23 14:02:34 +00:00
Erik Johnston
278149f533 Sanitize TransactionStore 2015-03-23 13:43:21 +00:00
Paul "LeoNerd" Evans
72d8406409 Put a cache on get_aliases_for_room 2015-03-20 19:21:13 +00:00
Paul "LeoNerd" Evans
a63b4f7101 Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale 2015-03-20 18:25:49 +00:00
Paul "LeoNerd" Evans
0f86312c4c Pull out the cache logic from the @cached wrapper into its own class we can reuse 2015-03-20 18:25:42 +00:00
Paul "LeoNerd" Evans
b1022ed8b5 func(*EXPR) is valid Python syntax, really... 2015-03-20 17:47:45 +00:00
Erik Johnston
f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston
80cd08c190 PEP8 2015-03-20 16:03:25 +00:00
Erik Johnston
9517f4da4d Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement 2015-03-20 16:02:47 +00:00
Erik Johnston
dc0c989ef4 Give sensible names for '_simple_...' transactions 2015-03-20 15:59:18 +00:00
Paul "LeoNerd" Evans
ceb61daa70 Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream 2015-03-20 15:44:06 +00:00
Erik Johnston
fce0114005 Start removing Tables 2015-03-20 15:05:44 +00:00
Erik Johnston
7e282a53a5 Tidy up _simple_... methods 2015-03-20 15:05:10 +00:00
Paul "LeoNerd" Evans
91cb46191d Allow @cached-wrapped functions to have more or fewer than 1 argument; assert on the total count of them though 2015-03-20 14:59:45 +00:00
Erik Johnston
87db64b839 Rearrange storage modules 2015-03-20 14:11:38 +00:00
Erik Johnston
cb8162d3d1 Rearrange storage modules 2015-03-20 13:52:56 +00:00
Erik Johnston
d288d273e1 Generate transaction id in code 2015-03-20 10:57:44 +00:00
Erik Johnston
d4f50f3ae5 decode_result takes an iterable 2015-03-20 10:57:26 +00:00
Erik Johnston
455579ca90 Make database selection configurable 2015-03-20 10:55:55 +00:00
Erik Johnston
0d0610870d Fix up schemas some more 2015-03-20 10:55:31 +00:00
Erik Johnston
56f2d31676 Bump version 2015-03-19 17:48:33 +00:00
Erik Johnston
c178e4e6ca Add missing servlet to list 2015-03-19 17:48:21 +00:00
Erik Johnston
d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Erik Johnston
58ed393235 Remove redundant key 2015-03-19 15:12:05 +00:00
Erik Johnston
fae059cc18 Fix up schemas to work with mariadb 2015-03-19 13:42:39 +00:00
Erik Johnston
0dcb145c7e Bump version 2015-03-19 11:26:03 +00:00
Erik Johnston
6cc046302f Bump version 2015-03-18 11:41:00 +00:00
Erik Johnston
ed4d44d833 Merge pull request #109 from matrix-org/default_registration 
Disable registration by default. Add script to register new users.
 2015-03-18 11:38:52 +00:00
Erik Johnston
f88db7ac0b Factor out user id validation checks 2015-03-18 11:34:18 +00:00
Erik Johnston
57976f646f Do more validation of incoming request 2015-03-18 11:30:04 +00:00
Erik Johnston
bb24609158 Clean out event_forward_extremities table when the server rejoins the room 2015-03-18 11:19:47 +00:00
Erik Johnston
89036579ed Update schema to work with mariadb 2015-03-18 11:18:49 +00:00
Paul "LeoNerd" Evans
93978c5e2b @cached() annotate get_user_by_token() - achieves a minor DB performance improvement 2015-03-17 17:24:51 +00:00
Paul "LeoNerd" Evans
1489521ee5 Be polite and ensure we use @functools.wraps() when creating a function decorator 2015-03-17 17:19:22 +00:00
David Baker
6d33f97703 pep8 2015-03-17 11:53:55 +00:00
David Baker
7564dac8cb Wire up the webclient option 
It existed but was hardcoded to True.
Give it an underscore for consistency.
Also don't pull in syweb unless we're actually using the web client.
 2015-03-17 12:45:37 +01:00
Paul "LeoNerd" Evans
3f7a31d366 Add a DistributionMetric to HTTP request/response processing time in the server 2015-03-16 18:31:29 +00:00
Paul "LeoNerd" Evans
be170b1426 Add a metric for the scheduling latency of SQL queries 2015-03-16 17:21:59 +00:00
Erik Johnston
cd2539ab2a Merge pull request #110 from matrix-org/fix_ban 
Fix ban
 2015-03-16 15:36:52 +00:00
Kegan Dougal
f0d6f724a2 Set the service ID as soon as it is known. 2015-03-16 15:24:32 +00:00
Erik Johnston
f1d2b94e0b Copy dict of context.current_state before changing it. 2015-03-16 15:13:05 +00:00
Erik Johnston
857810d2dd Revert incorrect changes to where we send events 2015-03-16 15:12:47 +00:00
Kegan Dougal
d04fa1f712 Implement ServiceQueuer with tests. 2015-03-16 14:03:16 +00:00
Erik Johnston
c2c9471cba Don't block waiting on waking up all the listeners when sending an event. 2015-03-16 13:16:37 +00:00
Kegan Dougal
6279285b2a Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer. 2015-03-16 13:15:40 +00:00
Erik Johnston
8bad40701b Comment. 2015-03-16 13:13:07 +00:00
Erik Johnston
250e143084 Use 403 instead of 400 2015-03-16 13:11:42 +00:00
Erik Johnston
b2e6ee5b43 Remove concept of context.auth_events, instead use context.current_state 2015-03-16 13:06:23 +00:00
Kegan Dougal
c9c444f562 Wrap polling/retry blocks in try/excepts to avoid sending to other ASes breaking permanently should an error occur. 2015-03-16 10:38:02 +00:00
Kegan Dougal
835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal
f9232c7917 Merge branch 'develop' into application-services-txn-reliability 
Conflicts:
	synapse/storage/appservice.py
 2015-03-16 10:09:15 +00:00
Erik Johnston
758d114cbc Send all membership events to the remote homeserver 2015-03-16 00:27:59 +00:00
Erik Johnston
ea8590cf66 Make context.auth_events grap auth events from current state. Otherwise auth is wrong. 2015-03-16 00:18:08 +00:00
Erik Johnston
ab8229479b Respect ban membership 2015-03-16 00:17:25 +00:00
Matthew Hodgson
c8032aec17 actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0' 2015-03-14 00:12:20 +00:00
Paul Evans
e731d30d90 Merge pull request #108 from matrix-org/metrics 
Metrics
 2015-03-13 17:31:10 +00:00
Erik Johnston
98a3825614 Allow enabling of registration with --disable-registration false 2015-03-13 16:49:18 +00:00
Erik Johnston
9266cb0a22 PEP8 2015-03-13 15:26:00 +00:00
Erik Johnston
dea236e4fa Add missing commas 2015-03-13 15:24:03 +00:00
Erik Johnston
69135f59aa Implement registering with shared secret. 2015-03-13 15:23:37 +00:00
Erik Johnston
58367a9da2 Disable registration by default 2015-03-13 12:59:45 +00:00
Erik Johnston
58247c8b4b Also bump dependency link version 2015-03-13 11:39:57 +00:00
Matthew Hodgson
f55bd3f94b bump dep on syweb 0.6.5 2015-03-12 18:56:53 +00:00
Paul "LeoNerd" Evans
e90002ca1d Merge remote-tracking branch 'origin/develop' into metrics 2015-03-12 16:55:25 +00:00
David Baker
bbb010a30f More sacrifices to the pep8 gods. 2015-03-12 16:53:12 +00:00
Paul "LeoNerd" Evans
05a056a409 Appease pyflakes 2015-03-12 16:45:05 +00:00
Paul "LeoNerd" Evans
0eb7e6b9a8 Delete unused import of NOT_READY_YET 2015-03-12 16:39:52 +00:00
Paul "LeoNerd" Evans
128cf2daf7 Appease pep8 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b98b4c135d Option to serve metrics from their own localhost-only TCP port instead of muxed on the main listener 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
a2cdd11d4a Fold the slightly-odd bind_port/secure_port/etc.. logic into SynapseHomeServer.start_listening() 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
e0214a263b Build MetricsResource as a specific HomeServer dependency 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
e75fa8bbbf Bugfix to sql_txn_timer increment - add only the per-TXN duration, not the total time ever spent since boot 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
c782e893ec Neater metrics from TransactionQueue 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
89ac1fa8ba Add a counter to track total number of events served by the notifier 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
2e4f0b2bd7 Replace the @metrics.counted annotations in federation with specifically-written counters and distributions 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
c1cdd7954d Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
63cb7ece62 Rename the timer metrics exported by synapse.storage to append _time, so the meaning of ':total' is clearer 2015-03-12 16:24:51 +00:00