Mark Haines
deb0237166
Add some doc-string
2015-05-05 17:45:11 +01:00
Mark Haines
e45b05647e
Fix the --help option for synapse
2015-05-05 17:39:59 +01:00
Erik Johnston
3d5a955e08
Missed events are not outliers
2015-05-05 17:36:57 +01:00
Mark Haines
d18f37e026
Collect the invalidate callbacks on the transaction object rather than passing around a separate list
2015-05-05 17:32:21 +01:00
Erik Johnston
9951542393
Add a comment about the zip(*[zip(sorted(...),...)])
2015-05-05 17:06:55 +01:00
Mark Haines
041b6cba61
SYN-369: Add comments to the sequence number logic in the cache
2015-05-05 16:32:44 +01:00
Mark Haines
63075118a5
Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access
2015-05-05 16:24:04 +01:00
Erik Johnston
531d7955fd
Don't insert without deduplication. In this case we never actually use this table, so simply remove the insert entirely
2015-05-05 16:12:28 +01:00
Mark Haines
bfa4a7f8b0
Invalidate the room_member cache if the current state events updates
2015-05-05 15:43:49 +01:00
Mark Haines
d0fece8d3c
Missing return for when the event was already persisted
2015-05-05 15:39:09 +01:00
Erik Johnston
bdcd7693c8
Fix indentation
2015-05-05 15:14:48 +01:00
Erik Johnston
43c2e8deae
Add support for using executemany
2015-05-05 15:13:25 +01:00
Erik Johnston
1692dc019d
Don't call 'encode_parameter' no-op
2015-05-05 15:00:30 +01:00
Mark Haines
a9aea68fd5
Invalidate the caches from the correct thread
2015-05-05 14:57:08 +01:00
Mark Haines
261d809a47
Sequence the modifications to the cache so that selects don't race with inserts
2015-05-05 14:13:50 +01:00
Erik Johnston
d9cc5de9e5
Correctly name transaction
2015-05-05 10:24:10 +01:00
Erik Johnston
b8940cd902
Remove some unused indexes
2015-05-01 16:14:25 +01:00
Erik Johnston
1942382246
Don't log enqueue_
2015-05-01 16:14:25 +01:00
David Baker
eb9bd2d949
user_id now in user_threepids
2015-05-01 15:04:37 +01:00
Erik Johnston
2d386d7038
That wasn't a deferred
2015-05-01 14:41:25 +01:00
Erik Johnston
4ac2823b3c
Remove inlineCallbacks from non-generator
2015-05-01 14:41:25 +01:00
Erik Johnston
22c7c5eb8f
Typo
2015-05-01 14:41:25 +01:00
Erik Johnston
42c12c04f6
Remove some run_on_reactors
2015-05-01 14:41:25 +01:00
Erik Johnston
adb5b76ff5
Don't log all auth events every time we call auth.check
2015-05-01 14:41:25 +01:00
Mark Haines
3bcdf3664c
Use the daemonize key from the config if it exists
2015-05-01 14:34:55 +01:00
David Baker
9eeb03c0dd
Don't use self.execute: it's designed for fetching stuff
2015-05-01 14:21:25 +01:00
Mark Haines
7b50769eb9
Merge pull request #136 from matrix-org/markjh/config_cleanup
...
Config restructuring.
2015-05-01 14:04:39 +01:00
David Baker
7693f24792
No id field on user
2015-05-01 13:55:42 +01:00
Mark Haines
46a65c282f
Allow generate-config to run against an existing config file to generate default keys
2015-05-01 13:54:38 +01:00
David Baker
92b20713d7
More missed get_user_by_id API changes
2015-05-01 13:45:54 +01:00
Erik Johnston
da4ed08739
One too many lens
2015-05-01 13:29:38 +01:00
Erik Johnston
9060dc6b59
Change public room list to use defer.gatherResults
2015-05-01 13:28:36 +01:00
David Baker
1fae1b3166
This api now no longer returns an array
2015-05-01 13:26:41 +01:00
Erik Johnston
80b4119279
Don't wait for storage of access_token
2015-05-01 13:14:05 +01:00
Erik Johnston
4011cf1c42
Cache latest_event_ids_in_room
2015-05-01 13:06:26 +01:00
Erik Johnston
657298cebd
Don't lock user_ips table for upsert.
2015-05-01 10:46:48 +01:00
Erik Johnston
fabb7acd45
Fix bug where we reconnected to the database on every query.
2015-05-01 10:24:24 +01:00
Erik Johnston
23c639ff32
Split a storage function in two so that we don't have to do extra work.
2015-05-01 10:17:19 +01:00
Erik Johnston
8be5284e91
Remove pointless join
2015-04-30 18:49:26 +01:00
Erik Johnston
503e4d3d52
Fix broken SQL
2015-04-30 18:44:47 +01:00
Erik Johnston
00718ae7a9
Need more yield
2015-04-30 18:43:39 +01:00
Erik Johnston
0465560c1a
Add missing param
2015-04-30 18:42:44 +01:00
Erik Johnston
61d05daab1
More join conditions
2015-04-30 18:41:05 +01:00
Erik Johnston
6ead27ddda
Add more conditions on JOINs to make postgres go a little faster.
2015-04-30 18:32:03 +01:00
Mark Haines
50c87b8eed
Allow "manhole" to be ommited from the config
2015-04-30 18:11:47 +01:00
Mark Haines
345995fcde
Remove the ~, comment the lines instead
2015-04-30 18:10:19 +01:00
Mark Haines
62cebee8ee
Update key.py
2015-04-30 17:54:01 +01:00
Mark Haines
95cbfee8ae
Update metrics.py
2015-04-30 17:52:20 +01:00
Mark Haines
6ea9cf58be
missing import
2015-04-30 17:21:21 +01:00
Erik Johnston
f383d5a801
Fix up get_current_state and get_room_name_and_aliases queries to parse events in transaction
2015-04-30 17:12:52 +01:00
Mark Haines
c95480963e
read the pid_file from the config file in synctl
2015-04-30 17:12:15 +01:00
Mark Haines
2d4d2bbae4
Merge branch 'develop' into markjh/config_cleanup
...
Conflicts:
synapse/config/captcha.py
2015-04-30 16:54:55 +01:00
Mark Haines
2f1348f339
Write a default log_config when generating config
2015-04-30 16:52:57 +01:00
Erik Johnston
69d4063651
Add get_rooms_for_user cache
2015-04-30 16:47:51 +01:00
David Baker
5b02f33451
Undo changes to logger config, ie. remove the access_log_file option: decision is to support this through log_config rather tan adding an option.
2015-04-30 16:21:39 +01:00
David Baker
054aa0d58c
Do access log using python's logging stuff, just under a separate logger name
2015-04-30 16:21:38 +01:00
Erik Johnston
3c4c229788
Don't use sub queries, it makes postgres sad
2015-04-30 16:16:53 +01:00
Mark Haines
74aaacf82a
Don't break when sizes or durations are given as integers
2015-04-30 16:04:02 +01:00
Mark Haines
29400b45b9
SYN-367: Use upsert rather than insert_or_replace
2015-04-30 15:21:31 +01:00
Mark Haines
c28f1d16f0
Add a random string to the auto generated key id
2015-04-30 15:13:14 +01:00
Mark Haines
265f30bd3f
Allow --enable-registration to be passed on the commandline
2015-04-30 15:04:06 +01:00
Mark Haines
c9e62927f2
Use disable_registration keys if they are present
2015-04-30 14:34:09 +01:00
Erik Johnston
2366d28780
Don't needlessly join on state_events
2015-04-30 14:02:06 +01:00
David Baker
d89a9f7283
Add an access_log
...
SYN-161 #resolve
2015-04-30 13:58:13 +01:00
Mark Haines
1aa11cf7ce
Allow multiple config files, set up a default config before applying the config files
2015-04-30 13:48:15 +01:00
David Baker
0c1b7f843b
Unused import
2015-04-30 13:33:30 +01:00
David Baker
4b46fbec5b
Doesn't look like this is used anymore
2015-04-30 12:04:08 +01:00
Erik Johnston
1d7702833d
Make simple query rather than long one and then throw away half the results
2015-04-30 10:16:12 +01:00
Mark Haines
6b69ddd17a
remove duplicate parse_size method
2015-04-30 04:26:29 +01:00
Mark Haines
d624e2a638
Manually generate the default config yaml, remove most of the commandline arguments for synapse anticipating that people will use the yaml instead. Simpify implementing config options by not requiring the classes to hit the super class
2015-04-30 04:24:44 +01:00
Erik Johnston
b1ca784aca
Correctly decode, for sqlite and postgres, rows from pushers table
2015-04-29 19:41:14 +01:00
Erik Johnston
4a9dc5b2f5
pushkey' are also bytes.
2015-04-29 19:27:02 +01:00
Erik Johnston
0ade2712d1
Typo
2015-04-29 19:17:25 +01:00
Erik Johnston
50f96f256f
Also remove yield from within lock in the other generator
2015-04-29 19:17:00 +01:00
Erik Johnston
d2d61a8288
Fix deadlock in id_generators. No idea why this was an actual deadlock.
2015-04-29 19:15:23 +01:00
Erik Johnston
3e71d13acf
Also log when we've started pushers
2015-04-29 18:37:42 +01:00
Erik Johnston
e7a6edb0ee
Revert previous
2015-04-29 18:37:30 +01:00
Erik Johnston
c27d6ad6b5
Only start pushers when synapse has fully started
2015-04-29 18:25:24 +01:00
Erik Johnston
46daf2d200
Start pushers on reactor thread
2015-04-29 18:22:20 +01:00
Erik Johnston
3864b3a8e6
Actually return rows
2015-04-29 18:07:36 +01:00
Erik Johnston
0618978238
Typo, args wrong way round
2015-04-29 18:04:35 +01:00
Erik Johnston
09177f4f2e
Decode buffers in same thread
2015-04-29 18:03:42 +01:00
Erik Johnston
472be88674
We store pusher data as bytes
2015-04-29 17:43:46 +01:00
Erik Johnston
a6e62cf6d0
Fix off by one in presence token handling
2015-04-29 17:37:11 +01:00
David Baker
12d381bd5d
Decode the data json in the storage layer (was moved but this part was missed)
2015-04-29 17:13:51 +01:00
David Baker
f8c30faf25
Oops, update the contraint too
2015-04-29 16:58:42 +01:00
David Baker
61cd5d9045
Be more postgressive
2015-04-29 16:57:14 +01:00
David Baker
fb95035a65
Be postgressive
2015-04-29 16:53:41 +01:00
David Baker
4669def000
Oops, forgot the schema delta file
2015-04-29 16:50:16 +01:00
Erik Johnston
0337eaf321
txn.execute doesn't return cursors
2015-04-29 16:43:39 +01:00
Erik Johnston
884fb88e28
txn.execute doesn't return cursors
2015-04-29 16:35:20 +01:00
Erik Johnston
d76c058eea
Fix invalid SQL to work in postgres land
2015-04-29 16:30:25 +01:00
David Baker
9927170787
Accept camelcase + underscores in binding too
2015-04-29 15:57:09 +01:00
David Baker
109c8aafd2
Fix includes
2015-04-29 15:45:44 +01:00
David Baker
b7788f80a3
Accept both camelcase and underscore threepid creds for transition
2015-04-29 15:41:29 +01:00
Erik Johnston
c8ed9bd278
pushers table requires an access_token
2015-04-29 15:33:27 +01:00
Mark Haines
f2d90d5c02
Fix whitespace
2015-04-29 14:53:23 +01:00
Mark Haines
845b0b2c97
Check requirements before doing anything else when running the homeserver
2015-04-29 14:52:42 +01:00
Mark Haines
c0036ced54
bump syutil to 0.0.6
2015-04-29 14:16:41 +01:00
Erik Johnston
970a9b9d2b
We can't use REPLACE when upgrading databases now we have postgres
2015-04-29 13:55:44 +01:00
Erik Johnston
64991b0c8b
Merge pull request #129 from matrix-org/key_distribution
...
Key distribution v2
2015-04-29 13:34:38 +01:00
Mark Haines
e26a3d8d9e
bump database schema version
2015-04-29 13:32:32 +01:00
Mark Haines
1319905d7a
Use a defer.gatherResults to collect results from the perspective servers
2015-04-29 13:31:14 +01:00
Mark Haines
a9549fdce3
Use bytea rather than BLOB
2015-04-29 13:16:09 +01:00
Mark Haines
4ad8b45155
Merge branch 'develop' into key_distribution
...
Conflicts:
synapse/config/homeserver.py
2015-04-29 13:15:14 +01:00
Mark Haines
74874ffda7
Update the query format used by keyring to match current key v2 spec
2015-04-29 12:14:08 +01:00
Erik Johnston
cd0864121b
Make postgres database error slightly more helpful
2015-04-29 12:12:25 +01:00
Erik Johnston
4932a7e2d9
Use __all__ instead of assert to stop pyflakes from warning about unused import in __init__
2015-04-29 12:12:25 +01:00
Mark Haines
9894da6a29
Merge branch 'develop' into erikj/postgres_charset_check
2015-04-29 11:58:06 +01:00
Mark Haines
46d200a3a1
Implement minimum_valid_until_ts in the remote key resource
2015-04-29 11:57:26 +01:00
Erik Johnston
a08bf11138
Appease PEP8
2015-04-29 11:44:48 +01:00
Erik Johnston
204132a998
Check that postgres database has correct charset set
2015-04-29 11:42:28 +01:00
Mark Haines
f4c9ebbc34
Delete ugly commented out log line.
2015-04-29 11:07:13 +01:00
Matthew Hodgson
68c0603946
comment out ugly test logline
2015-04-29 00:14:44 +01:00
Erik Johnston
1783c7ca92
Ensure we never miss any presence updates
2015-04-28 17:24:24 +01:00
Erik Johnston
0126ef7f3c
Fix typo
2015-04-28 17:23:53 +01:00
Erik Johnston
d98edb548a
Ensure the serial returned by presence is always an integer
2015-04-28 17:20:32 +01:00
Erik Johnston
073b891ec1
Remove unused imports
2015-04-28 13:44:23 +01:00
Erik Johnston
327ca883ec
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-28 13:39:42 +01:00
Erik Johnston
18f8247701
Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit
2015-04-28 12:41:33 +01:00
Mark Haines
9182f87664
Merge pull request #126 from matrix-org/csauth
...
Client / Server Auth Refactor
2015-04-28 11:00:27 +01:00
Mark Haines
55e1bc8920
And don't bump the schema version unnecessarily
2015-04-28 10:54:15 +01:00
Mark Haines
55fcf62e9c
Merge pull request #133 from matrix-org/invite_power_level
...
Invite power level
2015-04-28 10:50:34 +01:00
Mark Haines
b96c133034
Add server_keys.sql to the current delta rather than creating a new delta
2015-04-28 10:50:00 +01:00
Mark Haines
252e6f6869
Merge pull request #130 from matrix-org/server_rename_check
...
Fix for SYN-266
2015-04-28 10:39:31 +01:00
Mark Haines
0bc71103e1
Output vim style mode markers into the yaml config file
2015-04-28 10:17:10 +01:00
Mark Haines
f8b865264a
Merge branch 'develop' into key_distribution
...
Conflicts:
synapse/crypto/keyring.py
2015-04-27 18:29:32 +01:00
Paul "LeoNerd" Evans
38432d8c25
Merge branch 'develop' into invite_power_level
2015-04-27 17:09:25 +01:00
Erik Johnston
42b7139dec
Remove unused import
2015-04-27 15:59:56 +01:00
Erik Johnston
1ef66cc3bd
Move database configuration into config module
2015-04-27 15:57:43 +01:00
Erik Johnston
416a3e6c4f
Ensure check_same_thread is enabled for sqlite3
2015-04-27 15:44:30 +01:00
Erik Johnston
8558e1ec73
Make get_max_token into inlineCallbacks so that the lock works.
2015-04-27 15:19:44 +01:00
Erik Johnston
6f8e2d517e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-27 14:41:40 +01:00
Erik Johnston
2c70849dc3
Fix newlines
2015-04-27 14:38:29 +01:00
Erik Johnston
0a016b0525
Pull inner function out.
2015-04-27 14:37:24 +01:00
Erik Johnston
e701aec2d1
Implement locks using create_observer for fetching media and server keys
2015-04-27 14:20:26 +01:00
David Baker
412ece18e7
Add commentage.
2015-04-27 14:08:45 +01:00
Erik Johnston
1c82fbd2eb
Implement create_observer.
...
`create_observer` takes a deferred and create a new deferred that
*observers* the original deferred. Any callbacks added to the observing
deferred will *not* affect the origin deferred.
2015-04-27 13:59:37 +01:00
Erik Johnston
2732be83d9
Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible.
2015-04-27 13:22:30 +01:00
Erik Johnston
e4c4664d73
Handle the fact that postgres databases can be restarted from under us
2015-04-27 12:40:49 +01:00
David Baker
03c4f0ed67
pep8
2015-04-27 12:36:59 +01:00
David Baker
f1acb9fd40
logging args
2015-04-27 11:56:34 +01:00
David Baker
8a5be236e0
pep8
2015-04-27 11:49:18 +01:00
David Baker
df75914791
pep8
2015-04-27 11:48:33 +01:00
David Baker
b02e1006b9
Run database check before daemonizing, at the cost of database hygiene.
2015-04-27 11:46:00 +01:00
David Baker
f8152f2708
rename db method to be more informative
2015-04-27 10:16:26 +01:00
David Baker
2f475bd5d5
pep8
2015-04-24 18:15:07 +01:00
David Baker
a7b51f4539
Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266
2015-04-24 18:11:21 +01:00
Mark Haines
288702170d
Add config for setting the perspective servers
2015-04-24 17:01:34 +01:00
David Baker
1bac74b9ae
Change to https for ID server communication
2015-04-24 14:48:49 +01:00
David Baker
ed83638668
Make one-to-one rule an underride otherwise bings don't work in one-to-one wrooms. Likewise a couple of other rules.
2015-04-24 14:27:17 +01:00
David Baker
7ac8a60c6f
More underscores
2015-04-24 11:44:27 +01:00
Mark Haines
c253b14f6e
Merge branch 'develop' into key_distribution
2015-04-24 11:29:46 +01:00
Mark Haines
bdcb23ca25
Fix spelling
2015-04-24 11:29:19 +01:00
Mark Haines
b2c2dc8940
Merge branch 'develop' into key_distribution
2015-04-24 11:28:10 +01:00
Mark Haines
869dc94cbb
Call the super classes when generating config
2015-04-24 11:27:56 +01:00
David Baker
a218619626
Use underscores instead of camelcase for id server stuff
2015-04-24 11:27:38 +01:00
Mark Haines
b1e68add19
Add a config file for perspective servers
2015-04-24 11:26:19 +01:00
Mark Haines
31e262e6b4
Copyright notice
2015-04-24 10:36:51 +01:00
Mark Haines
eede182df7
Merge branch 'develop' into key_distribution
2015-04-24 10:35:49 +01:00
Mark Haines
4e2f8b8722
Copyright notices
2015-04-24 10:35:29 +01:00
Mark Haines
c8c710eca7
Move the key related config parser into a separate file
2015-04-24 10:22:22 +01:00
Mark Haines
149ed9f151
Better help for the old-signing-key option
2015-04-24 10:07:55 +01:00
David Baker
f7a79a37be
pep8
2015-04-24 09:42:37 +01:00
David Baker
6532b6e607
Merge branch 'develop' into csauth
...
Conflicts:
synapse/http/server.py
2015-04-24 09:37:54 +01:00
David Baker
74270defda
No commas here, otherwise our error string constants become tuples.
2015-04-24 09:28:57 +01:00
Paul "LeoNerd" Evans
e1e5e53127
Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again
2015-04-23 19:01:37 +01:00
Paul "LeoNerd" Evans
b3bda8a75f
Don't let the remote offline serial list grow arbitrarily large
2015-04-23 18:40:47 +01:00
Paul "LeoNerd" Evans
8a785c3006
Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261)
2015-04-23 18:40:19 +01:00
Paul "LeoNerd" Evans
191f7f09ce
Generate presence event-stream JSON structures directly
2015-04-23 18:27:25 +01:00
David Baker
03eb4adc6e
Dedicated error code for failed 3pid auth verification
2015-04-23 18:20:17 +01:00
Mark Haines
4bbf7156ef
Update to match the specification for key/v2
2015-04-23 16:39:13 +01:00
David Baker
0eb61a3d16
Remove ultimately unused feature of saving params from the first call in the session: it's probably too open to abuse.
2015-04-23 14:44:12 +01:00
David Baker
a2c10d37d7
Add an error code to 'missing token' response.
2015-04-23 13:23:44 +01:00
David Baker
2e0d9219b9
Remove now-redundant email config
2015-04-23 11:45:29 +01:00
Mark Haines
f30d47c876
Implement remote key lookup api
2015-04-22 14:21:08 +01:00
Paul "LeoNerd" Evans
a16eaa0c33
Neater fetching of user's auth level in a room - squash to int() at access time (SYN-353)
2015-04-22 14:20:04 +01:00
Paul "LeoNerd" Evans
f43063158a
Appease pep8
2015-04-22 13:12:11 +01:00
Paul "LeoNerd" Evans
2808c040ef
Also remember to check 'invite' level for changes
2015-04-21 21:13:14 +01:00
Paul "LeoNerd" Evans
48b6ee2b67
Create an 'invite' powerlevel when making new rooms
2015-04-21 21:07:35 +01:00
Paul "LeoNerd" Evans
bc41f0398f
Initial implementation of an 'invite' power_level
2015-04-21 20:56:08 +01:00
Paul "LeoNerd" Evans
d3309933f5
Much neater fetching of defined powerlevels from m.room.power_levels state event
2015-04-21 20:53:23 +01:00
Paul "LeoNerd" Evans
b568c0231c
Remove debugging print statement accidentally committed
2015-04-21 20:21:14 +01:00
Paul "LeoNerd" Evans
3a7d7a3f22
Sanitise a user's powerlevel to an int() before numerical comparison, because otherwise Python is "helpful" with it (SYN-351)
2015-04-21 20:18:29 +01:00
Mark Haines
3ba522bb23
Merge branch 'develop' into key_distribution
2015-04-21 17:10:25 +01:00
Mark Haines
6080830bef
Bump syutil version to 0.0.5
2015-04-21 17:04:06 +01:00
Mark Haines
8b183781cb
Merge pull request #128 from matrix-org/unify_http_wrappers
...
Unify http wrappers
2015-04-21 17:01:04 +01:00
Mark Haines
812a99100b
Set a version_string in BaseMediaResource so that the request_handler wrapper works
2015-04-21 16:43:58 +01:00
Mark Haines
1967650bc4
Combine the request wrappers in rest/media/v1 and http/server into a single wrapper decorator
2015-04-21 16:35:53 +01:00
Mark Haines
1ebff9736b
Split out the JsonResource request logging and error handling into a separate wrapper function
2015-04-21 16:07:20 +01:00
Mark Haines
24d21887ed
SYN-350: Don't ratelimit the individual events generated during room creation
2015-04-21 14:14:19 +01:00
Mark Haines
db8d4e8dd6
Merge branch 'develop' into key_distribution
2015-04-20 16:24:21 +01:00
Mark Haines
2f9157b427
Implement v2 key lookup
2015-04-20 16:23:47 +01:00
David Baker
91c8f828e1
pep8
2015-04-17 19:56:04 +01:00
David Baker
8db6832db8
Password reset, finally.
2015-04-17 19:53:47 +01:00
David Baker
117f35ac4a
Add endpoint to get threepids from server
2015-04-17 17:20:18 +01:00
David Baker
4eea5cf6c2
pep8
2015-04-17 16:46:45 +01:00
David Baker
f96ab9d18d
make add3pid servlet work
2015-04-17 16:44:49 +01:00
Erik Johnston
865398b4a9
Revert needless change to rest.profile
2015-04-17 16:14:51 +01:00
Erik Johnston
e3417bbbe0
Revert needless change to storage.profile
2015-04-17 16:13:14 +01:00
Erik Johnston
2492efb162
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-17 16:08:42 +01:00
Erik Johnston
4a5990ff8f
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-04-17 16:08:28 +01:00
Erik Johnston
5e7a90316d
Update --database-path metavar to SQLITE_DATABASE_PATH
2015-04-17 16:08:18 +01:00
David Baker
0b1a8500a2
just the once would probably be fine
2015-04-17 13:53:54 +01:00
David Baker
cb03fafdf1
Merge branch 'develop' into csauth
2015-04-17 13:51:10 +01:00
David Baker
bf5e54f255
Register the 3pid servlet
2015-04-17 13:44:55 +01:00
David Baker
94e1e58b4d
password -> account servlet and add start of an 'add 3pid' endpoint
2015-04-17 13:44:12 +01:00
Erik Johnston
ced39d019f
Bump version
2015-04-17 13:25:16 +01:00
David Baker
83b554437e
Need to yield the username check, otherwise very very weird things happen.
2015-04-17 12:57:25 +01:00
Erik Johnston
6ba2e3df4e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-17 11:22:31 +01:00
Erik Johnston
0ec346d942
Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea
2015-04-17 11:10:20 +01:00
David Baker
4cd5fb13a3
Oops, left debugging in.
2015-04-16 20:03:13 +01:00
David Baker
ea1776f556
Return user ID in use error straight away
2015-04-16 19:56:44 +01:00
Erik Johnston
e1c0970c11
PEP8
2015-04-16 11:18:45 +01:00
Erik Johnston
b8092fbc82
Go back to storing JSON in TEXT
2015-04-16 11:17:52 +01:00
Erik Johnston
bc9e69e160
Move encoding and decoding of JSON into storage layer
2015-04-16 11:01:09 +01:00
Paul "LeoNerd" Evans
f2cf37518b
Filter typing nofication events to only those rooms the requesting user is a member of (SYN-328)
2015-04-15 23:34:16 +01:00
Paul "LeoNerd" Evans
0268d40281
Have TypingNotificationEventSource.get_new_events_for_user() return a deferred, for consistency and extensibility
2015-04-15 23:09:35 +01:00
Paul "LeoNerd" Evans
399b5add58
Neater implementation of membership change auth checks, ensuring we can't forget to check if the calling user is a member of the room
2015-04-15 18:40:23 +01:00
Paul "LeoNerd" Evans
e6e130b9ba
Ensure that non-room-members cannot ban others, even if they do have enough powerlevel (SYN-343)
2015-04-15 18:07:33 +01:00
David Baker
766bd8e880
Dummy login so we can do the first POST request to get login flows without it just succeeding
2015-04-15 17:14:25 +01:00
Erik Johnston
ffad75bd62
Remove mysql/maria support
2015-04-15 17:00:50 +01:00
Mark Haines
a429515bdd
Add methods for storing and retrieving the raw key json
2015-04-15 16:58:35 +01:00
Mark Haines
8d761134c2
Fail quicker for 4xx responses in the key client, optional hit a different API path
2015-04-15 16:57:58 +01:00
Erik Johnston
cf04cedf21
Change full_schemas/11 to work with postgres
2015-04-15 16:53:47 +01:00
Erik Johnston
5b31afcbd1
Remove debug logging
2015-04-15 16:27:04 +01:00
Erik Johnston
6e91f14d09
Add missing yield
2015-04-15 16:25:07 +01:00
Erik Johnston
ed26e4012b
pushers table requires a unique id.
2015-04-15 16:24:14 +01:00
Erik Johnston
806f380a8b
Make LruCache thread safe, as its used for event cache
2015-04-15 16:07:59 +01:00
David Baker
a19b739909
Regstration with email in v2
2015-04-15 15:50:38 +01:00
Erik Johnston
a5c72780e6
Don't pass in removed flag
2015-04-15 15:13:22 +01:00
Erik Johnston
e19f794fee
Change from exception to warn
2015-04-15 15:12:57 +01:00
Erik Johnston
d5ff9effcf
Don't wait on federation_handler.handle_new_event
2015-04-15 15:05:57 +01:00
Erik Johnston
e845434028
Remove run_on_reactor()s
2015-04-15 15:05:45 +01:00
Erik Johnston
4af32a2817
Postgres does not allow you to continue using a cursor after a DB exception has been raised, so move _simple_insert or_ignore flag out of transaction
2015-04-15 14:51:21 +01:00
Erik Johnston
bc6cef823f
Do more parellelization for initialSync
2015-04-15 14:21:59 +01:00
Erik Johnston
1ec6fa98c9
Parellelize initial sync
2015-04-15 14:17:16 +01:00
Erik Johnston
25d2914fba
Turn off persistance of sent_transactions.response_json
2015-04-15 14:09:35 +01:00
Erik Johnston
cce5d057d3
Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses
2015-04-15 11:43:25 +01:00
Erik Johnston
6606f7c659
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-15 10:27:20 +01:00
Erik Johnston
a971fa9d58
Use try..finally in contextlib.contextmanager
2015-04-15 10:25:43 +01:00
Erik Johnston
ded4128965
Use True for True rather than 1
2015-04-15 10:24:24 +01:00
Erik Johnston
f9e12f79ca
Add missing yield in storage func
2015-04-15 10:24:07 +01:00
Erik Johnston
c756dfeb14
Correctly identify deadlocks
2015-04-15 10:23:42 +01:00
Mark Haines
32e14d8181
Return a sha256 fingerprint rather than the entire tls certificate
2015-04-14 19:10:09 +01:00
Mark Haines
88cb06e996
Update syutil version to 0.0.4
2015-04-14 16:18:17 +01:00
Mark Haines
d488463fa3
Add a version 2 of the key server api
2015-04-14 16:04:52 +01:00
Erik Johnston
127fad17dd
Add postgres database engine
2015-04-14 14:50:29 +01:00
Erik Johnston
5a95cd4442
Rename user_ips.user -> user_id
2015-04-14 13:54:09 +01:00
Erik Johnston
58d8339966
Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS.
2015-04-14 13:53:20 +01:00
Mark Haines
3cbc286d06
Move server key api into rest/key/v1
2015-04-14 13:28:11 +01:00
Erik Johnston
3c741682e5
Correctly increment the _next_id initially
2015-04-14 09:54:44 +01:00
Erik Johnston
86fc9b617c
For backwards compat, make state_groups.id have a type of int, not varchar
2015-04-13 17:03:49 +01:00
Erik Johnston
93937b2b31
Remove duplicate rows
2015-04-10 18:47:03 +01:00
Erik Johnston
c5365dee56
Use case sensitive collations
2015-04-10 18:46:33 +01:00
Erik Johnston
4103b1c470
DROP indexes before recreating them
2015-04-10 18:46:09 +01:00
Erik Johnston
4d5b098626
Use LONGBLOB and TEXT for arbitary length rows
2015-04-10 18:45:09 +01:00
Erik Johnston
7ed2ec3061
Handle the fact that in sqlite binary data might be stored as unicode or bytes
2015-04-10 13:41:54 +01:00
Erik Johnston
ce797ad373
Bump schema version
2015-04-10 11:22:30 +01:00
Erik Johnston
7e863c51e6
Use unsigned bigint
2015-04-10 11:22:04 +01:00
Erik Johnston
0f12772e32
SQLite wants INTEGER and not BIGINT for primary keys
2015-04-10 11:16:09 +01:00
Erik Johnston
d5d4281647
Update full_schemas/16 to match delta files. Add delta/16 scripts
2015-04-10 10:59:46 +01:00
Erik Johnston
cda4a6f93f
Revert non-trivial changes to upgrade scripts
2015-04-10 10:19:50 +01:00
Erik Johnston
e2722f58ee
Fix schema again
2015-04-10 10:16:29 +01:00
Erik Johnston
a1665c5094
Revert non-trivial schema changes and move them to a new schema version.
2015-04-10 10:05:44 +01:00
Erik Johnston
2ded344620
Remove unused import
2015-04-09 13:46:06 +01:00
Erik Johnston
9707acfc40
Remove spurious spaces
2015-04-09 13:45:20 +01:00
Erik Johnston
8bf285e082
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-09 13:13:26 +01:00
Mark Haines
1280a47fc6
Add comment
2015-04-09 11:42:21 +01:00
Mark Haines
23d285ad57
Unset the timer in the timeout callback so that we don't try to cancel it if it has been called
2015-04-09 11:41:50 +01:00
Erik Johnston
8ad0f4912e
Stream ordering and out of order insertions.
...
Handle the fact that events can be persisted out of order, and so to get
the "current max" stream token becomes non trivial - as we need to make
sure that *all* stream tokens less than the current max have also
successfully been persisted.
2015-04-09 11:41:36 +01:00
Mark Haines
6f9dea7483
SYN-339: Cancel the notifier timeout when the notifier fires
2015-04-09 11:07:20 +01:00
Erik Johnston
279a547a8b
Use generic db exceptions rather than sqlite3 specific ones
2015-04-08 16:53:48 +01:00
Mark Haines
83f5125d52
Merge pull request #114 from matrix-org/improve_get_event_cache
...
Improve get event cache
2015-04-08 16:50:27 +01:00
Erik Johnston
a43b40449b
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-08 16:46:56 +01:00
Erik Johnston
9cef051ce2
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-04-08 16:43:17 +01:00
Erik Johnston
45131b2bca
Bump version
2015-04-08 16:35:12 +01:00
Erik Johnston
ccda401dbf
SYN-338: Fix typo that caused the cache to throw an exception in some instances
2015-04-08 16:34:23 +01:00
Erik Johnston
3887350e47
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-08 14:11:07 +01:00
Erik Johnston
19234cc6c3
typo
2015-04-08 14:10:06 +01:00
Erik Johnston
e8f1521605
Don't yield on notifying all listeners
2015-04-08 14:08:30 +01:00
Mark Haines
605941ee26
Merge pull request #117 from matrix-org/notifier-leak
...
Fix a notifier leak
2015-04-08 14:02:37 +01:00
Erik Johnston
5bc41fe9f8
Move comment into docstring
2015-04-08 14:01:22 +01:00
Erik Johnston
638be5a6b9
Factor out loops into '_discard_if_notified'
2015-04-08 13:58:32 +01:00
Erik Johnston
830d07db82
Also perform paranoia checks in 'on_new_user_event'
2015-04-08 13:40:20 +01:00
Erik Johnston
65f5e4e3e4
Add paranoia checks to make sure that we evict stale NotificationListeners when we are about to process them
2015-04-08 13:33:38 +01:00
Erik Johnston
07d4041709
Fix bug where we didn't inform the NotificataionListeners about new rooms they have been subscribed to. This meant that the listeners didn't clean themselves up fully from all the dicts
2015-04-08 13:33:38 +01:00
Erik Johnston
c1b34af441
Move database timer logging to seperate logger
2015-04-08 13:31:19 +01:00
Erik Johnston
9a05795619
Retry transaction, not SQL query
2015-04-08 13:11:28 +01:00
Erik Johnston
24d8134ac1
Fix maria engine to correctly recognize deadlocks
2015-04-08 13:10:54 +01:00
Erik Johnston
d5e7e6b9b6
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:17:22 +01:00
Erik Johnston
0775c62469
Fix --enable-registration flag to work if you don't give a value
2015-04-07 18:16:23 +01:00
Erik Johnston
38928c6609
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:08:38 +01:00
Erik Johnston
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
Kegan Dougal
ae8ff92e05
Fix a bug which causes a send event level of 0 to not be honoured.
...
Caused by a bad if check, which incorrectly executes for both 0 and None,
when None was the original intent.
2015-04-07 15:48:20 +01:00
Erik Johnston
49d6aa1394
Retry on deadlock
2015-04-07 15:28:37 +01:00
Erik Johnston
0bfa78b39b
PEP8
2015-04-07 12:16:05 +01:00
Erik Johnston
6bc9edd8b2
Fix prepare_sqlite3_database's convert_param_style
2015-04-07 12:13:58 +01:00
Erik Johnston
05a35d62b6
Bump database version
2015-04-07 12:10:15 +01:00
Erik Johnston
8574bf62dc
Add index to presence table
2015-04-07 12:09:36 +01:00
Erik Johnston
0af5f5efaf
Don't use multiple UNIQUE constraints; it will cause deadlocks
2015-04-07 12:08:35 +01:00
Erik Johnston
c8d3f6486d
Implement or_ignore flag on inserts
2015-04-07 12:06:22 +01:00
Erik Johnston
304111afd0
Don't use AUTOINCREMENT, use an in memory version
2015-04-07 12:05:36 +01:00
Erik Johnston
d0e444a648
Explicitly name the __main__ module logger
2015-04-07 12:04:02 +01:00
David Baker
4eb6d66b45
Add app service auth back in to v2 register
2015-04-02 17:51:19 +01:00
David Baker
6b59650753
Throw sensible errors on not-json when allowing empty body
2015-04-02 17:45:16 +01:00
David Baker
41cd778d66
pep8
2015-04-02 17:06:17 +01:00
David Baker
70a84f17f3
Add shared secret auth into register v2 and switch the script over.
2015-04-02 17:01:29 +01:00
Erik Johnston
779f7b0f44
Fix unicode support
2015-04-02 10:06:22 +01:00
Paul "LeoNerd" Evans
ef1e019840
Appease pep8
2015-04-01 19:17:38 +01:00
Paul "LeoNerd" Evans
5583e29513
Report process open filehandles in metrics
2015-04-01 19:15:23 +01:00
David Baker
c5bf0343e8
Explain how I justified to myself making JsonResource not always send JSON.
2015-04-01 15:13:14 +01:00
Erik Johnston
e24c32e6f3
Fix SQLite support
2015-04-01 15:09:51 +01:00
David Baker
e9c908ebc0
Completely replace fallback auth for C/S V2:
...
* Now only the auth part goes to fallback, not the whole operation
* Auth fallback is a normal API endpoint, not a static page
* Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
2015-04-01 15:05:30 +01:00
Erik Johnston
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
Kegan Dougal
813e54bd5b
Fix more AS sender ID thinkos.
...
Specifically, the ASes own user ID wasn't being treated as 'exclusive' so
a human could nab it. Also, the HS would needlessly send user queries to the
AS for its own user ID.
2015-04-01 14:05:24 +01:00
Kegan Dougal
9fa8bda099
Merge branch 'develop' into application-services-registration-script
2015-04-01 10:19:17 +01:00
David Baker
f129ee1e18
Make docs a bit more true
2015-03-31 18:25:10 +01:00
Kegan Dougal
09cbff174a
Fix thinko whereby events *for the AS specifically* were not passed on.
...
This was caused by not explicitly checking the service.sender field. This
has now been fixed and a regression test has been added.
2015-03-31 16:44:45 +01:00
David Baker
d18e7779ca
Grammar and deduplication
2015-03-31 14:40:02 +01:00
Kegan Dougal
5e88a09a42
Add same user_id char checks as registration.
2015-03-31 14:00:25 +01:00
Kegan Dougal
cf1fa59f4b
Use a sender localpart instead of a user ID.
...
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
2015-03-31 13:48:03 +01:00
Kegan Dougal
3470cb36a8
Pyflakes
2015-03-31 13:03:31 +01:00
Kegan Dougal
c217504949
Edit SQL schema to use string IDs not ints. Use token as ID. Update tests.
2015-03-31 12:07:56 +01:00
Kegan Dougal
b59aa74556
Fix tests and missing returns on deferreds.
2015-03-31 11:35:45 +01:00
Kegan Dougal
d33ae65efc
Remove more reg/unreg methods. Read config not database for cache.
2015-03-31 11:00:00 +01:00
David Baker
9f642a93ec
pep8
2015-03-31 09:50:44 +01:00
Kegan Dougal
e7887e37a8
Remove appservice REST servlets
2015-03-31 09:32:40 +01:00
Kegan Dougal
af853a4cdb
Add AppServiceConfig
2015-03-31 09:22:31 +01:00
David Baker
59bf16eddc
New registration for C/S API v2. Only ReCAPTCHA working currently.
2015-03-30 18:13:10 +01:00
Matthew Hodgson
8366fde82f
turn --disable-registration into --enable-registration, given the default is for registration to be disabled by default now. this is backwards incompatible by removing the old --disable-registration arg, but makes for a much more intuitive arg
2015-03-30 12:01:09 -04:00
Paul "LeoNerd" Evans
3e420aebd8
Revert "Add another @cached wrapper, this time on get_presence_state()"
...
This reverts commit ff1fa0fbf8
2015-03-27 16:16:58 +00:00
Paul "LeoNerd" Evans
ff1fa0fbf8
Add another @cached wrapper, this time on get_presence_state()
2015-03-27 15:57:16 +00:00
David Baker
6f4f7e4e22
pep8
2015-03-26 14:12:06 +00:00
David Baker
a32e876ef4
Delete pushers when changing password
2015-03-26 13:40:16 +00:00
Paul "LeoNerd" Evans
a198894bf7
Appease pep8
2015-03-26 11:53:58 +00:00
Kegan Dougal
32206dde3f
Fixes from PR comments
2015-03-26 10:11:52 +00:00
Kegan Dougal
4edcbcee3b
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans
953e40f9dc
Implement the main getEvent cache using Cache() instead of a custom application of LruCache; also unify its two-level structure into just one
2015-03-25 19:12:16 +00:00
David Baker
df4c12c762
pep8 blank lines
2015-03-25 19:08:17 +00:00
David Baker
c1a256cc4c
Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API.
2015-03-25 19:06:22 +00:00
Paul "LeoNerd" Evans
f173d40a32
Use FrozenEvent's reject_reason to decide whether to return it; don't include allow_rejected in the main getEvents cache key
2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
1b988b051b
Store the rejected reason in (Frozen)Event structs
2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
033a517feb
Indirect invalidations of _get_event_cache via a helper method to keep all uses of the cache lexically within one .py file
2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans
9ba6487b3f
Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict()
2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans
d6b3ea75d4
Implement the 'key in dict' test for LruCache()
2015-03-25 19:04:59 +00:00
Erik Johnston
0e8f5095c7
Fix unicode database support
2015-03-25 17:15:20 +00:00
David Baker
9aa0224cdf
unused import
2015-03-24 17:25:59 +00:00
David Baker
c7023f2155
1) Pushers are now associated with an access token
...
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
2015-03-24 17:24:15 +00:00
Erik Johnston
0ba393924a
Escape non printing ascii character
2015-03-24 16:31:52 +00:00
Erik Johnston
f488293d96
Don't reinsert into event_edges
2015-03-24 16:20:26 +00:00
Erik Johnston
1aa44939fc
Fix bugs in transactions storage
2015-03-24 16:20:05 +00:00
Erik Johnston
5a447098dd
Don't use room hosts table
2015-03-24 16:19:24 +00:00
Erik Johnston
9e98f1022a
Don't order by rowid
2015-03-24 16:19:01 +00:00
Erik Johnston
9115421ace
Use _simple_upsert
2015-03-24 16:17:39 +00:00
David Baker
d19e79ecc9
Make deleting other access tokens when you change your password actually work
2015-03-24 15:33:48 +00:00
Paul "LeoNerd" Evans
ed008e85a8
Reduce activity timer granularity to avoid too many quick updates (SYN-247)
2015-03-23 17:25:50 +00:00
Erik Johnston
6e7131f02f
Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable.
2015-03-23 15:38:56 +00:00
Erik Johnston
9a7f496298
Sanitize RoomMemberStore
2015-03-23 15:29:04 +00:00
David Baker
78adccfaf4
pep8 / pyflakes
2015-03-23 14:23:51 +00:00
David Baker
d98660a60d
Implement password changing (finally) along with a start on making client/server auth more general.
2015-03-23 14:20:28 +00:00
Erik Johnston
d5272b1d2c
Use 'update or insert' rather than on 'conflict replace'
2015-03-23 14:02:34 +00:00
Erik Johnston
278149f533
Sanitize TransactionStore
2015-03-23 13:43:21 +00:00
Paul "LeoNerd" Evans
72d8406409
Put a cache on get_aliases_for_room
2015-03-20 19:21:13 +00:00
Paul "LeoNerd" Evans
a63b4f7101
Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale
2015-03-20 18:25:49 +00:00
Paul "LeoNerd" Evans
0f86312c4c
Pull out the cache logic from the @cached wrapper into its own class we can reuse
2015-03-20 18:25:42 +00:00
Paul "LeoNerd" Evans
b1022ed8b5
func(*EXPR) is valid Python syntax, really...
2015-03-20 17:47:45 +00:00
Erik Johnston
f6583796fe
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-03-20 16:31:48 +00:00
Erik Johnston
80cd08c190
PEP8
2015-03-20 16:03:25 +00:00
Erik Johnston
9517f4da4d
Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement
2015-03-20 16:02:47 +00:00
Erik Johnston
dc0c989ef4
Give sensible names for '_simple_...' transactions
2015-03-20 15:59:18 +00:00
Paul "LeoNerd" Evans
ceb61daa70
Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream
2015-03-20 15:44:06 +00:00
Erik Johnston
fce0114005
Start removing Tables
2015-03-20 15:05:44 +00:00
Erik Johnston
7e282a53a5
Tidy up _simple_... methods
2015-03-20 15:05:10 +00:00
Paul "LeoNerd" Evans
91cb46191d
Allow @cached-wrapped functions to have more or fewer than 1 argument; assert on the total count of them though
2015-03-20 14:59:45 +00:00
Erik Johnston
87db64b839
Rearrange storage modules
2015-03-20 14:11:38 +00:00
Erik Johnston
cb8162d3d1
Rearrange storage modules
2015-03-20 13:52:56 +00:00
Erik Johnston
d288d273e1
Generate transaction id in code
2015-03-20 10:57:44 +00:00
Erik Johnston
d4f50f3ae5
decode_result takes an iterable
2015-03-20 10:57:26 +00:00
Erik Johnston
455579ca90
Make database selection configurable
2015-03-20 10:55:55 +00:00
Erik Johnston
0d0610870d
Fix up schemas some more
2015-03-20 10:55:31 +00:00
Erik Johnston
56f2d31676
Bump version
2015-03-19 17:48:33 +00:00
Erik Johnston
c178e4e6ca
Add missing servlet to list
2015-03-19 17:48:21 +00:00
Erik Johnston
d7a0496f3e
Convert storage layer to be mysql compatible
2015-03-19 15:59:48 +00:00
Erik Johnston
58ed393235
Remove redundant key
2015-03-19 15:12:05 +00:00
Erik Johnston
fae059cc18
Fix up schemas to work with mariadb
2015-03-19 13:42:39 +00:00
Erik Johnston
0dcb145c7e
Bump version
2015-03-19 11:26:03 +00:00
Erik Johnston
6cc046302f
Bump version
2015-03-18 11:41:00 +00:00
Erik Johnston
ed4d44d833
Merge pull request #109 from matrix-org/default_registration
...
Disable registration by default. Add script to register new users.
2015-03-18 11:38:52 +00:00
Erik Johnston
f88db7ac0b
Factor out user id validation checks
2015-03-18 11:34:18 +00:00
Erik Johnston
57976f646f
Do more validation of incoming request
2015-03-18 11:30:04 +00:00
Erik Johnston
bb24609158
Clean out event_forward_extremities table when the server rejoins the room
2015-03-18 11:19:47 +00:00
Erik Johnston
89036579ed
Update schema to work with mariadb
2015-03-18 11:18:49 +00:00
Paul "LeoNerd" Evans
93978c5e2b
@cached() annotate get_user_by_token() - achieves a minor DB performance improvement
2015-03-17 17:24:51 +00:00
Paul "LeoNerd" Evans
1489521ee5
Be polite and ensure we use @functools.wraps() when creating a function decorator
2015-03-17 17:19:22 +00:00
David Baker
6d33f97703
pep8
2015-03-17 11:53:55 +00:00
David Baker
7564dac8cb
Wire up the webclient option
...
It existed but was hardcoded to True.
Give it an underscore for consistency.
Also don't pull in syweb unless we're actually using the web client.
2015-03-17 12:45:37 +01:00
Paul "LeoNerd" Evans
3f7a31d366
Add a DistributionMetric to HTTP request/response processing time in the server
2015-03-16 18:31:29 +00:00
Paul "LeoNerd" Evans
be170b1426
Add a metric for the scheduling latency of SQL queries
2015-03-16 17:21:59 +00:00
Erik Johnston
cd2539ab2a
Merge pull request #110 from matrix-org/fix_ban
...
Fix ban
2015-03-16 15:36:52 +00:00
Kegan Dougal
f0d6f724a2
Set the service ID as soon as it is known.
2015-03-16 15:24:32 +00:00
Erik Johnston
f1d2b94e0b
Copy dict of context.current_state before changing it.
2015-03-16 15:13:05 +00:00
Erik Johnston
857810d2dd
Revert incorrect changes to where we send events
2015-03-16 15:12:47 +00:00
Kegan Dougal
d04fa1f712
Implement ServiceQueuer with tests.
2015-03-16 14:03:16 +00:00
Erik Johnston
c2c9471cba
Don't block waiting on waking up all the listeners when sending an event.
2015-03-16 13:16:37 +00:00
Kegan Dougal
6279285b2a
Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer.
2015-03-16 13:15:40 +00:00
Erik Johnston
8bad40701b
Comment.
2015-03-16 13:13:07 +00:00
Erik Johnston
250e143084
Use 403 instead of 400
2015-03-16 13:11:42 +00:00
Erik Johnston
b2e6ee5b43
Remove concept of context.auth_events, instead use context.current_state
2015-03-16 13:06:23 +00:00
Kegan Dougal
c9c444f562
Wrap polling/retry blocks in try/excepts to avoid sending to other ASes breaking permanently should an error occur.
2015-03-16 10:38:02 +00:00
Kegan Dougal
835e01fc70
Minor PR comment tweaks.
2015-03-16 10:16:59 +00:00
Kegan Dougal
f9232c7917
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Erik Johnston
758d114cbc
Send all membership events to the remote homeserver
2015-03-16 00:27:59 +00:00
Erik Johnston
ea8590cf66
Make context.auth_events grap auth events from current state. Otherwise auth is wrong.
2015-03-16 00:18:08 +00:00
Erik Johnston
ab8229479b
Respect ban membership
2015-03-16 00:17:25 +00:00
Matthew Hodgson
c8032aec17
actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0'
2015-03-14 00:12:20 +00:00
Paul Evans
e731d30d90
Merge pull request #108 from matrix-org/metrics
...
Metrics
2015-03-13 17:31:10 +00:00
Erik Johnston
98a3825614
Allow enabling of registration with --disable-registration false
2015-03-13 16:49:18 +00:00
Erik Johnston
9266cb0a22
PEP8
2015-03-13 15:26:00 +00:00
Erik Johnston
dea236e4fa
Add missing commas
2015-03-13 15:24:03 +00:00
Erik Johnston
69135f59aa
Implement registering with shared secret.
2015-03-13 15:23:37 +00:00
Erik Johnston
58367a9da2
Disable registration by default
2015-03-13 12:59:45 +00:00
Erik Johnston
58247c8b4b
Also bump dependency link version
2015-03-13 11:39:57 +00:00
Matthew Hodgson
f55bd3f94b
bump dep on syweb 0.6.5
2015-03-12 18:56:53 +00:00
Paul "LeoNerd" Evans
e90002ca1d
Merge remote-tracking branch 'origin/develop' into metrics
2015-03-12 16:55:25 +00:00
David Baker
bbb010a30f
More sacrifices to the pep8 gods.
2015-03-12 16:53:12 +00:00
Paul "LeoNerd" Evans
05a056a409
Appease pyflakes
2015-03-12 16:45:05 +00:00
Paul "LeoNerd" Evans
0eb7e6b9a8
Delete unused import of NOT_READY_YET
2015-03-12 16:39:52 +00:00
Paul "LeoNerd" Evans
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b98b4c135d
Option to serve metrics from their own localhost-only TCP port instead of muxed on the main listener
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
a2cdd11d4a
Fold the slightly-odd bind_port/secure_port/etc.. logic into SynapseHomeServer.start_listening()
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
e0214a263b
Build MetricsResource as a specific HomeServer dependency
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
e75fa8bbbf
Bugfix to sql_txn_timer increment - add only the per-TXN duration, not the total time ever spent since boot
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
c782e893ec
Neater metrics from TransactionQueue
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
89ac1fa8ba
Add a counter to track total number of events served by the notifier
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
2e4f0b2bd7
Replace the @metrics.counted annotations in federation with specifically-written counters and distributions
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
c1cdd7954d
Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
63cb7ece62
Rename the timer metrics exported by synapse.storage to append _time, so the meaning of ':total' is clearer
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
493e3fa0ca
Don't forbid '_' in metric basenames any more, to allow things like foo_time
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f1fbe3e09f
Rename TimerMetric to DistributionMetric; as it could count more than just time
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
642f725fd7
Pretend the 'getEvent' cache is just another cache in the set of all the others for metric
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
cbc0406be8
Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
1748605c5d
Count incoming HTTP requests per servlet that responds
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
4d661ec0f3
Remember to emit final linefeed from /metrics page, or Prometheus gets upset
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
0e847540c3
Prometheus needs "escaped" label values
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
22b37b75db
Kill unused CounterMetric.fetch() method
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b0cf867319
Use _ instead of . as a metric namespacing separator, for Prometheus
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
0b96bb793e
Have all @metrics.counted use a single metric name vectored on the method name, rather than a brand new scalar counter per counted method
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b3a0179d64
Bugfix to rendering output of vectored TimerMetrics
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
399689dcc7
Provide some process resource usage metrics
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
fa319a5786
Add TimerMetrics to shadow the PerformanceCounters in synapse.storage; with the view to eventually replacing them entirely
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
6d146e15df
Put some gauge metrics on the number of notifier listeners, and notified-on objects (users, rooms, appservices)
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
25187ab674
Collect per-SQL-verb timer stats on query execution time
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
f52acf3b12
Neater register_* methods on overall Metrics container
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
a99d6edc05
Neater implementation of metric render methods by pulling out 'render' as a base method that calls self.render_item
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
72625f2f4d
Initial hack at a TimerMetric; for storing counts + duration accumulators
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e1a7e3564f
Delete a couple of TODO markers of monitoring stats now done
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
094803cf82
Put vector gauges on transaction queue pending PDU and EDU dicts
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e9c4b0d178
Ensure that /_synapse/metrics response is UTF-8 encoded
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
23ab0c68c2
Implement vector CallbackMetrics
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
849300bc73
Neater introspection methods on BaseMetric so that subclasses don't need to touch self.keys directly
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
8664599af7
Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e02cc249da
Ensure that exceptions while rendering individual metrics don't stop others from being rendered anyway - especially useful for CallbackMetric
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
59c448f074
Add a scalar gauge metric on the size of the presence user cachemap
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
d8caa5454d
Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
b0cdf097f4
Sprinkle some CacheCounterMetrics around the synapse.storage layer
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
ce8b5769f7
Create the concept of a cachecounter metric; generating two counters specific to caches
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
7d72e44eb9
Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
c53ec53d80
Pull out all uses of the underlying HTTP user agent .request() method into a single wrapper function, to make adding metrics easier
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
9470412316
Initial attempt at sprinkling some @metrics.counted decorations around the federation code
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
a594087f06
Have the MetricsResource actually render metric counters
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
74bc42cfdd
An initial implementation of a 'metrics' instance, similar to a 'logger' for keeping counter stats on method calls
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
120b689284
Delete pointless (and unreachable) __init__ method from FederationClient
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e7420a3bef
Initial tiny attempt at (vectorable) counter metrics
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
e07fc62833
A trivial 'hello world'-style resource on /_synapse/metrics, with optional commandline flag
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
5b6e11d560
Commandline option to enable metrics system
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
211c14c391
No need to explicitly pass 'web_client' in to create_resource_tree as it can be found via config
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
ad5701f50f
Expose 'config' as a real HomeServer dependency key
2015-03-12 16:24:50 +00:00
David Baker
c92fdf88a3
Log the matching push rule.
2015-03-11 22:17:31 +00:00
Paul "LeoNerd" Evans
a7a28f85ae
Appease pep8
2015-03-11 17:32:43 +00:00
Paul "LeoNerd" Evans
59a5f012cc
Also give _execute() a description
2015-03-11 17:19:17 +00:00
Paul "LeoNerd" Evans
099e4b88d8
Add a description to storage layer's _execute_and_decode()
2015-03-11 17:08:57 +00:00
David Baker
cdb2e045ee
Again, underscore, not hyphen
2015-03-11 14:22:35 +00:00
David Baker
465354ffde
'false' is not False
2015-03-11 11:24:50 +00:00
David Baker
83b1e7fb3c
PEP8 blank lines
2015-03-11 10:01:17 +00:00
David Baker
04f8478aaa
Add the master push rule for the break-my-push button. Allow server default rules to be disabled by default.
2015-03-10 17:26:25 +00:00
David Baker
8916acbc13
These aren't defined for redacted events so don't crash
2015-03-10 11:21:37 +00:00
Erik Johnston
abaf47bbb6
Fix bug in logging.
2015-03-10 10:28:29 +00:00
Erik Johnston
045afd6b61
in_thread takes no arguments
2015-03-10 10:19:03 +00:00
Erik Johnston
98b867f7b7
Fix bug in logging.
2015-03-10 10:16:09 +00:00
Kegan Dougal
db1fbc6c6f
Fix remaining scheduler bugs. Add more informative logging.
2015-03-10 10:04:20 +00:00
Erik Johnston
e84fe3599b
Merge pull request #105 from matrix-org/erikj-perf
...
Add a Twisted Service to synapse.app.homeserver
2015-03-10 10:02:26 +00:00
Erik Johnston
c37eceeb9e
Split out the 'run' from 'setup'
2015-03-10 09:58:33 +00:00
Erik Johnston
b8a6692657
Add documentation. When starting via twistd respect soft_file_limit config option.
2015-03-10 09:39:42 +00:00
Kegan Dougal
7e0bba555c
Remove unused import
2015-03-09 17:48:37 +00:00
Kegan Dougal
04c9751f24
Bug fixes whilst putting it all together
2015-03-09 17:45:41 +00:00
Erik Johnston
019422ebba
Merge pull request #101 from matrix-org/neaten-federation-servlets
...
Neaten federation servlets
2015-03-09 17:39:06 +00:00
Kegan Dougal
b98cd03193
Use event IDs instead of dumping event content in the txns table.
2015-03-09 17:25:20 +00:00
Kegan Dougal
21fd84dcb8
Use seconds; start gluing in the AS scheduler into the AS handler.
2015-03-09 17:01:19 +00:00
