Erik Johnston
5238960850
Bump CHANGES and version
2016-07-27 17:33:09 +01:00
Richard van der Hoff
ccec25e2c6
key upload tweaks
...
1. Add v2_alpha URL back in, since things seem to be using it.
2. Don't reject the request if the device_id in the upload request fails to
match that in the access_token.
2016-07-27 16:41:06 +01:00
Mark Haines
29b25d59c6
Merge branch 'develop' into markjh/verify
...
Conflicts:
synapse/crypto/keyring.py
2016-07-27 15:11:02 +01:00
Mark Haines
884b800899
Merge pull request #955 from matrix-org/markjh/only_from2
...
Add a couple more checks to the keyring
2016-07-27 15:08:22 +01:00
Mark Haines
09d31815b4
Merge pull request #954 from matrix-org/markjh/even_more_fixes
...
Fix a couple of bugs in the transaction and keyring code
2016-07-27 15:08:14 +01:00
Mark Haines
fe1b369946
Clean up verify_json_objects_for_server
2016-07-27 14:10:43 +01:00
Richard van der Hoff
26cb0efa88
SQL syntax fix
2016-07-27 12:30:22 +01:00
Richard van der Hoff
d47115ff8b
Delete e2e keys on device delete
2016-07-27 12:24:52 +01:00
Richard van der Hoff
2e3d90d67c
Make the device id on e2e key upload optional
...
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.
For active access_tokens without an associated device_id, we ought to register
the device in the devices table.
Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
Mark Haines
a4b06b619c
Add a couple more checks to the keyring
2016-07-26 19:50:11 +01:00
Mark Haines
c63b1697f4
Merge pull request #952 from matrix-org/markjh/more_fixes
...
Check if the user is banned when handling 3pid invites
2016-07-26 19:20:56 +01:00
Mark Haines
87ffd21b29
Fix a couple of bugs in the transaction and keyring code
2016-07-26 19:19:08 +01:00
Richard van der Hoff
eb359eced4
Add create_requester
function
...
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
Mark Haines
c824b29e77
Check if the user is banned when handling 3pid invites
2016-07-26 16:39:14 +01:00
Richard van der Hoff
33d7776473
Fix typo
2016-07-26 13:32:15 +01:00
Richard van der Hoff
9ad8d9b17c
Merge branch 'develop' into rav/delete_refreshtoken_on_delete_device
2016-07-26 13:29:46 +01:00
Richard van der Hoff
5b1825ba5b
Merge pull request #951 from matrix-org/rav/flake8
...
Fix flake8 noise
2016-07-26 13:27:54 +01:00
Mark Haines
9c4cf83259
Merge pull request #948 from matrix-org/markjh/auth_fixes
...
Don't add rejections to the state_group, persist all rejections
2016-07-26 13:22:57 +01:00
Richard van der Hoff
05e7e5e972
Fix flake8 violation
...
Apparently flake8 v3 puts the error on a different line to v2. Easiest way to
make sure that happens is by putting the whole statement on one line :)
2016-07-26 11:59:08 +01:00
Richard van der Hoff
8e02494166
Delete refresh tokens when deleting devices
2016-07-26 11:10:37 +01:00
Mark Haines
a6f06ce3e2
Fix how push_actions are redacted.
2016-07-26 11:05:39 +01:00
David Baker
d34e9f93b7
Merge pull request #949 from matrix-org/rav/update_devices
...
Implement updates and deletes for devices
2016-07-26 10:49:55 +01:00
Mark Haines
efeb6176c1
Don't add rejected events if we've seen them befrore. Add some comments to explain what the code is doing mechanically
2016-07-26 10:49:52 +01:00
Matthew Hodgson
242c52d607
typo
2016-07-26 10:09:25 +02:00
Richard van der Hoff
012b4c1913
Implement updating devices
...
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
Richard van der Hoff
436bffd15f
Implement deleting devices
2016-07-26 07:35:48 +01:00
Mark Haines
1b3c3e6d68
Only update the events and event_json tables for rejected events
2016-07-25 18:44:30 +01:00
Richard van der Hoff
33d08e8433
Log when adding listeners
2016-07-25 17:22:15 +01:00
Mark Haines
8f7f4cb92b
Don't add the events to forward extremities if the event is rejected
2016-07-25 17:13:37 +01:00
Mark Haines
2623cec874
Don't add rejections to the state_group, persist all rejections
2016-07-25 16:12:16 +01:00
David Baker
4fcdf7b4b2
Merge pull request #946 from matrix-org/dbkr/log_recaptcha_hostname
...
Log the hostname the reCAPTCHA was completed on
2016-07-25 16:10:39 +01:00
Mark Haines
955ef1f06c
fix: defer.returnValue takes one argument
2016-07-25 16:04:45 +01:00
Richard van der Hoff
2ee4c9ee02
background updates: fix assert again
2016-07-25 16:01:46 +01:00
Richard van der Hoff
9dbd903f41
background updates: Fix assertion to do something
2016-07-25 14:05:23 +01:00
Richard van der Hoff
465117d7ca
Fix background_update tests
...
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
David Baker
7ed58bb347
Use get to avoid KeyErrors
2016-07-22 17:18:50 +01:00
David Baker
dad2da7e54
Log the hostname the reCAPTCHA was completed on
...
This could be useful information to have in the logs. Also comment about how & why we don't verify the hostname.
2016-07-22 17:00:56 +01:00
Richard van der Hoff
363786845b
PEP8
2016-07-22 13:21:07 +01:00
Richard van der Hoff
ec5717caf5
Create index on user_ips in the background
...
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.
I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
Erik Johnston
d26b660aa6
Cache getPeer
2016-07-21 17:38:51 +01:00
Erik Johnston
aede7248ab
Split out a FederationReader process
2016-07-21 17:37:44 +01:00
Richard van der Hoff
55abbe1850
make /devices return a list
...
Turns out I specced this to return a list of devices rather than a dict of them
2016-07-21 15:57:28 +01:00
David Baker
2c28e25bda
Merge pull request #943 from matrix-org/rav/get_device_api
...
Implement GET /device/{deviceId}
2016-07-21 13:41:42 +01:00
David Baker
1e6e370b76
Merge pull request #942 from matrix-org/rav/fix_register_deviceid
...
Preserve device_id from first call to /register
2016-07-21 13:16:31 +01:00
Richard van der Hoff
1c3c202b96
Fix PEP8 errors
2016-07-21 13:15:15 +01:00
Richard van der Hoff
406f7aa0f6
Implement GET /device/{deviceId}
2016-07-21 12:00:29 +01:00
Richard van der Hoff
c445f5fec7
storage/client_ips: remove some dead code
2016-07-21 11:58:47 +01:00
Erik Johnston
cf94a78872
Set host not path
2016-07-21 11:45:53 +01:00
Richard van der Hoff
1a64dffb00
Preserve device_id from first call to /register
...
device_id may only be passed in the first call to /register, so make sure we
fish it out of the register `params` rather than the body of the final call.
2016-07-21 11:34:16 +01:00
Erik Johnston
081e5d55e6
Send the correct host header when fetching keys
2016-07-21 11:14:54 +01:00
Erik Johnston
248e6770ca
Cache federation state responses
2016-07-21 10:30:12 +01:00
Richard van der Hoff
7314bf4682
Merge branch 'develop' into rav/get_devices_api
...
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff
e9e3eaa67d
Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips
...
Record device_id in client_ips
2016-07-20 17:38:45 +01:00
Erik Johnston
d36b1d849d
Don't explode if we have no snapshots yet
2016-07-20 16:59:52 +01:00
David Baker
742056be0d
Merge pull request #937 from matrix-org/rav/register_device_on_register
...
Register a device_id in the /v2/register flow.
2016-07-20 16:51:27 +01:00
Richard van der Hoff
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff
ec041b335e
Record device_id in client_ips
...
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
Richard van der Hoff
053e83dafb
More doc-comments
...
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
Richard van der Hoff
b97a1356b1
Register a device_id in the /v2/register flow.
...
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.
Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
Erik Johnston
b73dc0ef4d
Merge pull request #936 from matrix-org/erikj/log_rss
...
Add metrics for psutil derived memory usage
2016-07-20 16:32:38 +01:00
Erik Johnston
66868119dc
Add metrics for psutil derived memory usage
2016-07-20 16:00:21 +01:00
Erik Johnston
aba0b2a39b
Merge pull request #935 from matrix-org/erikj/backfill_notifs
...
Don't notify pusher pool for backfilled events
2016-07-20 13:39:16 +01:00
Erik Johnston
57dca35692
Don't notify pusher pool for backfilled events
2016-07-20 13:25:06 +01:00
Richard van der Hoff
c68518dfbb
Merge pull request #933 from matrix-org/rav/type_annotations
...
Type annotations
2016-07-20 12:26:32 +01:00
David Baker
e967bc86e7
Merge pull request #932 from matrix-org/rav/register_refactor
...
Further registration refactoring
2016-07-20 11:03:33 +01:00
Erik Johnston
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Erik Johnston
f91faf09b3
Comment
2016-07-20 10:18:09 +01:00
Richard van der Hoff
3413f1e284
Type annotations
...
Add some type annotations to help PyCharm (in particular) to figure out the
types of a bunch of things.
2016-07-19 18:56:16 +01:00
Richard van der Hoff
40cbffb2d2
Further registration refactoring
...
* `RegistrationHandler.appservice_register` no longer issues an access token:
instead it is left for the caller to do it. (There are two of these, one in
`synapse/rest/client/v1/register.py`, which now simply calls
`AuthHandler.issue_access_token`, and the other in
`synapse/rest/client/v2_alpha/register.py`, which is covered below).
* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
access_tokens into `_create_registration_details`. This means that the normal
flow no longer needs to call `AuthHandler.issue_access_token`; the
shared-secret flow can tell `RegistrationHandler.register` not to generate a
token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
David Baker
b9e997f561
Merge pull request #931 from matrix-org/rav/refactor_register
...
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
Richard van der Hoff
9a7a77a22a
Merge pull request #929 from matrix-org/rav/support_deviceid_in_login
...
Add device_id support to /login
2016-07-19 15:53:04 +01:00
Richard van der Hoff
8f6281ab0c
Don't bind email unless threepid contains expected fields
2016-07-19 15:50:01 +01:00
Richard van der Hoff
0da0d0a29d
rest/client/v2_alpha/register.py: Refactor flow somewhat.
...
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.
The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
Richard van der Hoff
022b9176fe
schema fix
...
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
Mark Haines
c41d52a042
Summary line
2016-07-19 10:28:27 +01:00
Mark Haines
7e554aac86
Update docstring on Handlers.
...
To indicate it is deprecated.
2016-07-19 10:20:58 +01:00
Richard van der Hoff
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Richard van der Hoff
93efcb8526
Merge pull request #928 from matrix-org/rav/refactor_login
...
Refactor login flow
2016-07-18 16:12:35 +01:00
Richard van der Hoff
dcfd71aa4c
Refactor login flow
...
Make sure that we have the canonical user_id *before* calling
get_login_tuple_for_user_id.
Replace login_with_password with a method which just validates the password,
and have the caller call get_login_tuple_for_user_id. This brings the password
flow into line with the other flows, and will give us a place to register the
device_id if necessary.
2016-07-18 15:23:54 +01:00
Erik Johnston
fca90b3445
Merge pull request #924 from matrix-org/erikj/purge_history
...
Fix /purge_history bug
2016-07-18 15:11:11 +01:00
Mark Haines
a292454aa1
Merge pull request #925 from matrix-org/markjh/auth_fix
...
Fix 500 ISE when sending alias event without a state_key
2016-07-18 15:04:47 +01:00
Erik Johnston
4f81edbd4f
Merge pull request #927 from Half-Shot/develop
...
Fall back to 'username' if 'user' is not given for appservice registration.
2016-07-18 10:44:56 +01:00
Richard van der Hoff
6344db659f
Fix a doc-comment
...
The `store` in a handler is a generic DataStore, not just an events.StateStore.
2016-07-18 09:48:10 +01:00
Will Hunt
511a52afc8
Use body.get to check for 'user'
2016-07-16 18:44:08 +01:00
Will Hunt
e885e2a623
Fall back to 'username' if 'user' is not given for appservice reg.
2016-07-16 18:33:48 +01:00
Mark Haines
d137e03231
Fix 500 ISE when sending alias event without a state_key
2016-07-15 18:58:25 +01:00
Erik Johnston
f52565de50
Fix /purge_history bug
...
This was caused by trying to insert duplicate backward extremeties
2016-07-15 14:23:15 +01:00
Erik Johnston
a2d288c6a9
Merge pull request #923 from matrix-org/erikj/purge_history
...
Various purge_history fixes
2016-07-15 13:23:29 +01:00
Erik Johnston
bd7c51921d
Merge pull request #919 from matrix-org/erikj/auth_fix
...
Various auth.py fixes.
2016-07-15 11:38:33 +01:00
Erik Johnston
978fa53cc2
Pull out min stream_ordering from ex_outlier_stream
2016-07-15 10:22:30 +01:00
Erik Johnston
eec9609e96
event_backwards_extremeties may not be empty
2016-07-15 10:22:09 +01:00
Erik Johnston
9e1b43bcbf
Comment
2016-07-15 09:29:54 +01:00
Erik Johnston
a3036ac37e
Merge pull request #921 from matrix-org/erikj/account_deactivate
...
Feature: Add an /account/deactivate endpoint
2016-07-14 17:25:15 +01:00
Erik Johnston
ebdafd8114
Check sender signed event
2016-07-14 17:03:24 +01:00
Erik Johnston
a98d215204
Add filter param to /messages API
2016-07-14 16:30:56 +01:00
Erik Johnston
d554ca5e1d
Add support for filters in paginate_room_events
2016-07-14 15:59:04 +01:00
Erik Johnston
209e04fa11
Merge pull request #918 from negzi/bugfix_for_token_expiry
...
Bug fix: expire invalid access tokens
2016-07-14 15:51:52 +01:00
Erik Johnston
e5142f65a6
Add 'contains_url' to filter
2016-07-14 15:35:48 +01:00
Erik Johnston
b64aa6d687
Add sender and contains_url field to events table
2016-07-14 15:35:43 +01:00
Erik Johnston
848d3bf2e1
Add hs object
2016-07-14 10:25:52 +01:00
Erik Johnston
b55c770271
Only accept password auth
2016-07-14 10:00:38 +01:00
Erik Johnston
d543b72562
Add an /account/deactivate endpoint
2016-07-14 09:56:53 +01:00
Negar Fazeli
0136a522b1
Bug fix: expire invalid access tokens
2016-07-13 15:00:37 +02:00
Erik Johnston
2cb758ac75
Check if alias event's state_key matches sender's domain
2016-07-13 13:12:25 +01:00
Erik Johnston
560c71c735
Check creation event's room_id domain matches sender's
2016-07-13 13:07:19 +01:00
David Baker
a37ee2293c
Merge pull request #915 from matrix-org/dbkr/more_requesttokens
...
Add requestToken endpoints
2016-07-13 11:51:46 +01:00
David Baker
c55ad2e375
be more pythonic
2016-07-12 14:15:10 +01:00
David Baker
aaa9d9f0e1
on_OPTIONS isn't neccessary
2016-07-12 14:13:14 +01:00
David Baker
75fa7f6b3c
Remove other debug logging
2016-07-12 14:08:57 +01:00
David Baker
a5db0026ed
Separate out requestTokens to separate handlers
2016-07-11 09:57:07 +01:00
David Baker
9c491366c5
Oops, remove debug logging
2016-07-11 09:07:40 +01:00
David Baker
385aec4010
Implement https://github.com/matrix-org/matrix-doc/pull/346/files
2016-07-08 17:42:48 +01:00
Mark Haines
dfde67a6fe
Add a comment explaining allow_none
2016-07-08 15:57:06 +01:00
Mark Haines
10c843fcfb
Ensure that the guest user is in the database when upgrading accounts
2016-07-08 15:15:55 +01:00
Erik Johnston
58930da52b
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-07-08 14:11:37 +01:00
Erik Johnston
0870588c20
Merge branch 'hotfixes-v0.16.1'
2016-07-08 13:22:32 +01:00
Erik Johnston
f90cf150e2
Bump version and changelog
2016-07-07 16:33:00 +01:00
Erik Johnston
067596d341
Fix bug where we did not correctly explode when multiple user_ids were set in macaroon
2016-07-07 16:22:24 +01:00
Erik Johnston
b92e7955be
Comment
2016-07-07 11:42:15 +01:00
Erik Johnston
c98e1479bd
Return 400 rather than 500
2016-07-07 11:41:07 +01:00
Erik Johnston
67f2c901ea
Add rest servlet. Fix SQL.
2016-07-06 15:56:59 +01:00
Erik Johnston
eef7778af9
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/test2
2016-07-06 14:50:22 +01:00
Erik Johnston
a17e7caeb7
Merge branch 'erikj/shared_secret' into erikj/test2
2016-07-06 14:46:31 +01:00
Erik Johnston
f0c06ac65c
Merge pull request #909 from matrix-org/erikj/shared_secret
...
Add an admin option to shared secret registration (breaks backwards compat)
2016-07-06 14:08:51 +01:00
Erik Johnston
76b18df3d9
Check that there are no null bytes in user and passsword
2016-07-06 11:17:53 +01:00
Erik Johnston
0da24cac8b
Add null separator to hmac
2016-07-06 11:05:16 +01:00
Kent Shikama
8d9a884cee
Update password config comment
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-06 12:18:19 +09:00
Erik Johnston
be3548f7e1
Remove spurious txn
2016-07-05 17:46:51 +01:00
Erik Johnston
4adf93e0f7
Fix for postgres
2016-07-05 17:34:25 +01:00
Erik Johnston
651faee698
Add an admin option to shared secret registration
2016-07-05 17:30:22 +01:00
Erik Johnston
caf33b2d9b
Protect password when registering using shared secret
2016-07-05 17:18:19 +01:00
Erik Johnston
8f8798bc0d
Add ReadWriteLock for pagination and history prune
2016-07-05 15:30:25 +01:00
Erik Johnston
7335f0adda
Add ReadWriteLock
2016-07-05 15:23:17 +01:00
David Baker
ef535178ff
Merge pull request #904 from matrix-org/dbkr/register_email_no_untrusted_id_server
...
requestToken update
2016-07-05 15:13:34 +01:00
Mark Haines
04dee11e97
Merge pull request #906 from matrix-org/markjh/faster_events_around
...
Use a query that postgresql optimises better for get_events_around
2016-07-05 14:48:34 +01:00
Mark Haines
dd2ccee27d
Fix typo
2016-07-05 14:06:07 +01:00
Mark Haines
b6b0132ac7
Make get_events_around more efficient on sqlite3
2016-07-05 13:55:18 +01:00
Kent Shikama
252ee2d979
Remove default password pepper string
2016-07-05 19:15:51 +09:00
Kent Shikama
14362bf359
Fix password config
2016-07-05 19:12:53 +09:00
Kent Shikama
1ee2584307
Fix pep8
2016-07-05 19:01:00 +09:00
Kent Shikama
507b8bb091
Add comment to prompt changing of pepper
2016-07-05 18:42:35 +09:00
Mark Haines
d44d11d864
Use true/false for boolean parameter inclusive to avoid potential for sqli, and possibly make the code clearer
2016-07-05 10:39:13 +01:00
Erik Johnston
2d21d43c34
Add purge_history API
2016-07-05 10:28:51 +01:00
Mark Haines
0fb76c71ac
Use different SQL for postgres and sqlite3 for when using multicolumn indexes
2016-07-04 19:44:55 +01:00
Kent Shikama
8bdaf5f7af
Add pepper to password hashing
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-05 02:13:52 +09:00
Erik Johnston
a67bf0b074
Add storage function to purge history for a room
2016-07-04 16:02:50 +01:00
Mark Haines
f18d7546c6
Use a query that postgresql optimises better for get_events_around
2016-07-04 15:48:25 +01:00
Kent Shikama
bb069079bb
Fix style violations
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-04 22:07:11 +09:00
Kent Shikama
2e5a31f197
Use .get() instead of [] to access password_hash
2016-07-04 22:00:13 +09:00
Kent Shikama
fc8007dbec
Optionally include password hash in createUser endpoint
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-03 15:08:15 +09:00
David Baker
be8be535f7
requestToken update
...
Don't send requestToken request to untrusted ID servers
Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.
2016-06-30 17:51:28 +01:00
Erik Johnston
f328d95cef
Feature: Add deactivate account admin API
...
Allows server admins to "deactivate" accounts, which:
- Revokes all access tokens
- Removes all threepids
- Removes password
The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Erik Johnston
f52cb4cd78
Remove race
2016-06-29 15:24:50 +01:00
Mark Haines
6783534a0f
Merge pull request #886 from matrix-org/markjh/async_commit
...
Optionally make committing to postgres asynchronous.
2016-06-29 15:21:58 +01:00
Erik Johnston
a70688445d
Implement purge_media_cache admin API
2016-06-29 14:57:59 +01:00
Erik Johnston
314b146b2e
Track approximate last access time for remote media
2016-06-29 11:41:20 +01:00
David Baker
db74dcda5b
Merge pull request #894 from matrix-org/dbkr/push_room_naming
...
Use similar naming we use in email notifs for push
2016-06-28 10:12:24 +01:00
Matthew Hodgson
63bb8f0df9
remove vector.im from default secondary DS list
2016-06-27 13:13:33 +04:00
David Baker
0b640aa56b
even more pep8
2016-06-24 11:47:11 +01:00
David Baker
aa3a4944d5
more pep8
2016-06-24 11:45:23 +01:00
David Baker
46b7362304
pep8
2016-06-24 11:44:57 +01:00
David Baker
870c45913e
Use similar naming we use in email notifs for push
...
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
Mark Haines
05f1a4596a
Merge branch 'master' into develop
2016-06-23 11:17:48 +01:00
David Baker
b5fb7458d5
Actually we need to order these properly
...
otherwise we'll end up returning the wrong 20
2016-06-22 18:07:14 +01:00
David Baker
f73fdb04a6
Style
2016-06-22 17:51:40 +01:00
David Baker
3a4120e49a
Put most recent 20 messages in notif
...
Fixes https://github.com/vector-im/vector-web/issues/1648
2016-06-22 17:47:18 +01:00
Martin Weinelt
0a32208e5d
Rework ldap integration with ldap3
...
Use the pure-python ldap3 library, which eliminates the need for a
system dependency.
Offer both a `search` and `simple_bind` mode, for more sophisticated
ldap scenarios.
- `search` tries to find a matching DN within the `user_base` while
employing the `user_filter`, then tries the bind when a single
matching DN was found.
- `simple_bind` tries the bind against a specific DN by combining the
localpart and `user_base`
Offer support for STARTTLS on a plain connection.
The configuration was changed to reflect these new possibilities.
Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
2016-06-22 17:51:59 +02:00
Mark Haines
774f3a692c
Merge pull request #889 from matrix-org/markjh/synctl_workers
...
Optionally start or stop workers in synctl.
2016-06-21 17:58:19 +01:00
Mark Haines
5cc7564c5c
Optionally start or stop workers in synctl.
...
Optionally start or stop an individual worker by passing -w with
the path to the worker config.
Optionally start or stop every worker and the main synapse by
passing -a with a path to a directory containing worker configs.
The "-w" is intended to be used to bounce individual workers proceses.
THe "-a" is intended for when you want to restart all the workers
simultaneuously, for example when performing database upgrades.
2016-06-21 16:38:05 +01:00
Mark Haines
0fe0b0eeb6
Merge pull request #888 from matrix-org/markjh/content_repo
...
Remove the legacy v0 content upload API.
2016-06-21 14:01:01 +01:00
Mark Haines
13e334506c
Remove the legacy v0 content upload API.
...
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
2016-06-21 11:47:39 +01:00
Mark Haines
d5fb561709
Optionally make committing to postgres asynchronous.
...
Useful when running tests when you don't care whether the server
will lose data that it claims that it has committed.
2016-06-20 17:53:38 +01:00
Erik Johnston
d8ec81cc31
Merge pull request #879 from matrix-org/erikj/linearize_fed_server
...
Linearize some federation endpoints based on (origin, room_id)
2016-06-20 17:34:29 +01:00
Erik Johnston
bc72d381b2
Merge branch 'release-v0.16.1' of github.com:matrix-org/synapse
2016-06-20 14:18:04 +01:00
Erik Johnston
4d362a61ea
Bump version and changelog
2016-06-20 14:17:42 +01:00
Mark Haines
41e4b2efea
Add the create_profile method back since the tests use it
2016-06-17 19:20:47 +01:00
Mark Haines
0c13d45522
Add a comment on why we don't create a profile for upgrading users
2016-06-17 19:18:53 +01:00
Mark Haines
9f1800fba8
Remove registered_users from the distributor.
...
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.
This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
2016-06-17 19:14:16 +01:00
Erik Johnston
8f4a9bbc16
Linearize some federation endpoints based on (origin, room_id)
2016-06-17 16:43:45 +01:00
Erik Johnston
9ba2bf1570
Merge pull request #878 from matrix-org/erikj/ujson
...
Disable responding with canonical json for federation
2016-06-17 16:22:12 +01:00
Erik Johnston
120c238705
Disable responding with canonical json for federation
2016-06-17 16:10:37 +01:00
Erik Johnston
1c1f633b13
Merge pull request #877 from matrix-org/erikj/frozen_default
...
Turn use_frozen_events off by default
2016-06-17 15:33:55 +01:00
Erik Johnston
6660f37558
Merge pull request #876 from matrix-org/erikj/sign_own
...
Only re-sign our own events
2016-06-17 15:23:20 +01:00
Erik Johnston
3e41de05cc
Turn use_frozen_events off by default
2016-06-17 15:11:22 +01:00
Erik Johnston
2884712ca7
Only re-sign our own events
2016-06-17 14:47:33 +01:00
Mark Haines
ded01c3bf6
Fix `KeyError: 'msgtype'
. Use
.get
`
...
Fixes a key error where the mailer tried to get the ``msgtype`` of an
event that was missing a ``msgtype``.
```
File "synapse/push/mailer.py", line 264, in get_notif_vars
File "synapse/push/mailer.py", line 285, in get_message_vars
File ".../frozendict/__init__.py", line 10, in __getitem__
return self.__dict[key]
KeyError: 'msgtype'
```
2016-06-17 13:49:16 +01:00
Mark Haines
8c75040c25
Fix setting gc thresholds in the workers
2016-06-17 11:48:12 +01:00
Mark Haines
a352b68acf
Use worker_ prefixes for worker config, use existing support for multiple config files
2016-06-16 17:29:50 +01:00
Mark Haines
364d616792
Access the event_cache_size directly from the server object.
...
This means that the workers can override the event_cache_size
directly without clobbering the value in the main synapse config.
2016-06-16 12:53:15 +01:00
Mark Haines
bde13833cb
Access replication_url from the worker config directly
2016-06-16 12:44:40 +01:00
Mark Haines
80a1bc7db5
Comment on what's going on in clobber_with_worker_config
2016-06-16 11:29:45 +01:00
Mark Haines
f1f70bf4b5
Merge remote-tracking branch 'origin/develop' into markjh/worker_config
2016-06-16 11:20:17 +01:00
Mark Haines
dbb5a39b64
Add worker config module
2016-06-16 11:09:15 +01:00
Mark Haines
885ee861f7
Inline the synchrotron and pusher configs into the main config
2016-06-16 11:06:12 +01:00
Erik Johnston
486b9a6a2d
Merge pull request #873 from vt0r/bugfix/bcrypt-utf8-encode
...
Fix TypeError in call to bcrypt.hashpw
2016-06-16 10:12:17 +01:00
Salvatore LaMendola
ed5f43a55a
Fix TypeError in call to bcrypt.hashpw
...
- At the very least, this TypeError caused logins to fail on my own
running instance of Synapse, and the simple (explicit) UTF-8
conversion resolved login errors for me.
Signed-off-by: Salvatore LaMendola <salvatore.lamendola@gmail.com>
2016-06-16 00:43:42 -04:00
Erik Johnston
09a17f965c
Line lengths
2016-06-15 16:58:12 +01:00
Erik Johnston
1e9026e484
Handle floats as img widths
2016-06-15 16:58:05 +01:00
Erik Johnston
a60169ea09
Handle og props with not content
2016-06-15 16:57:48 +01:00
Erik Johnston
0ef0655b83
Bump version and changelog
2016-06-15 15:50:48 +01:00
Erik Johnston
d41a1a91d3
Linearize fetching of gaps on incoming events
...
This potentially stops the server from doing multiple requests for the
same data.
2016-06-15 15:16:14 +01:00
Richard van der Hoff
15bf3e3376
Merge pull request #870 from matrix-org/rav/work_around_tls_bug
...
Work around TLS bug in twisted
2016-06-15 13:28:13 +01:00
Erik Johnston
b31c49d676
Correctly mark backfilled events as backfilled
2016-06-15 10:59:08 +01:00
Richard van der Hoff
255c229f23
Work around TLS bug in twisted
...
Wrap up twisted's FileBodyProducer to work around
https://twistedmatrix.com/trac/ticket/8473 . Hopefully this fixes
https://matrix.org/jira/browse/SYN-700 .
2016-06-15 10:39:08 +01:00
Erik Johnston
d12134ce37
Merge pull request #868 from matrix-org/erikj/invalid_id
...
Make get_domain_from_id throw SynapseError on invalid ID
2016-06-14 15:09:07 +01:00
Erik Johnston
36e2aade87
Make get_domain_from_id throw SynapseError on invalid ID
2016-06-14 13:25:29 +01:00
Matthew Hodgson
33546b58aa
point to the CAPTCHA docs
2016-06-12 23:11:29 +01:00
Mark Haines
41b35412bf
Merge pull request #863 from matrix-org/markjh/load_config
...
Add function to load config without generating it
2016-06-10 11:12:32 +01:00
Mark Haines
7dbb473339
Add function to load config without generating it
...
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.
The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.
As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
2016-06-09 18:50:38 +01:00
Erik Johnston
16a8884233
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-06-09 16:57:33 +01:00
Erik Johnston
a31befbcd0
Bump version and changelog
2016-06-09 13:23:41 +01:00
Erik Johnston
5ee5b655b2
Merge pull request #862 from matrix-org/erikj/media_remote_error
...
502 on /thumbnail when can't contact remote server
2016-06-09 13:11:02 +01:00
Erik Johnston
0a43219a27
Merge pull request #860 from negzi/bug_fix_get_or_create_user
...
Fix a bug caused by a change in auth_handler function
2016-06-09 11:40:12 +01:00
Erik Johnston
eba4ff1bcb
502 on /thumbnail when can't contact remote server
2016-06-09 11:29:43 +01:00
Erik Johnston
95f305c35a
Remove redundant exception log in /events
2016-06-09 11:15:04 +01:00
Negar Fazeli
6e7dc7c7dd
Fix a bug caused by a change in auth_handler function
...
Fix the relevant unit test cases
2016-06-08 23:22:39 +02:00
Erik Johnston
b7fbc9bd95
Merge pull request #859 from matrix-org/erikj/public_room_performance
...
Pull full state for each room all at once
2016-06-08 16:36:06 +01:00
Erik Johnston
919a2c74f6
Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop
2016-06-08 16:26:26 +01:00
Erik Johnston
81c07a32fd
Pull full state for each room all at once
2016-06-08 15:51:49 +01:00
Erik Johnston
690029d1a3
Don't make rooms visibile by default
2016-06-08 14:47:42 +01:00
Erik Johnston
d88faf92d1
Fix up federation PublicRoomList
2016-06-08 14:39:31 +01:00
Erik Johnston
958c968d02
Merge pull request #856 from matrix-org/erikj/fed_pub_rooms
...
Enable auth on /publicRoom endpoints
2016-06-08 14:36:09 +01:00
Erik Johnston
746b2f5657
Merge pull request #854 from matrix-org/erikj/federation_logging
...
Add some logging for when servers ask for missing events
2016-06-08 14:31:41 +01:00
Erik Johnston
efeabd3180
Log user that is making /publicRooms calls
2016-06-08 14:23:15 +01:00
Erik Johnston
1fd6eb695d
Enable auth on federation PublicRoomList
2016-06-08 14:15:18 +01:00
Erik Johnston
17aab5827a
Add some logging for when servers ask for missing events
2016-06-08 11:55:31 +01:00
Erik Johnston
1a815fb04f
Don't hit DB for noop replications queries
2016-06-08 11:33:30 +01:00
Erik Johnston
bab916bccc
Bump version and changelog to v0.16.0-rc2
2016-06-08 11:05:45 +01:00
Erik Johnston
e0fda29f94
Merge pull request #850 from matrix-org/erikj/gc_threshold
...
Add gc_threshold to pusher and synchrotron
2016-06-08 10:05:56 +01:00
Erik Johnston
0f2165ccf4
Don't track total objects as its too expensive to calculate
2016-06-07 17:00:45 +01:00
Erik Johnston
18f0cc7d99
Record some more GC metrics
2016-06-07 16:55:49 +01:00
Erik Johnston
2d1d1025fa
Add gc_threshold to pusher and synchrotron
2016-06-07 16:26:25 +01:00
Erik Johnston
b01e71e719
Merge pull request #849 from matrix-org/erikj/gc_threshold
...
Allow setting of gc.set_thresholds
2016-06-07 16:07:54 +01:00
Erik Johnston
dded389ac1
Allow setting of gc.set_thresholds
2016-06-07 15:45:56 +01:00
Mark Haines
0b2158719c
Remove dead code.
...
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.
The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.
The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston
48e65099b5
Also record number of unreachable objects
2016-06-07 13:40:22 +01:00
Erik Johnston
75331c5fca
Change the way we do stats
2016-06-07 13:33:13 +01:00
Erik Johnston
8c966fbd51
Merge pull request #771 from matrix-org/erikj/gc_tick
...
Manually run GC on reactor tick.
2016-06-07 13:18:36 +01:00
Mark Haines
88625db05f
Notify users for events in rooms they join.
...
Change how the notifier updates the map from room_id to user streams on
receiving a join event. Make it update the map when it notifies for the
join event, rather than using the "user_joined_room" distributor signal
2016-06-07 11:33:36 +01:00
Erik Johnston
84379062f9
Fix AS retries, but with correct ordering
2016-06-07 10:24:50 +01:00
Erik Johnston
310197bab5
Fix AS retries
2016-06-07 09:34:50 +01:00
Mark Haines
b0932b34cb
Merge pull request #845 from matrix-org/markjh/synchrotron_presence
...
Fix a KeyError in the synchrotron presence
2016-06-06 16:52:27 +01:00
Mark Haines
4a5bbb1941
Fix a KeyError in the synchrotron presence
2016-06-06 16:37:12 +01:00
Mark Haines
87f60e7053
Merge pull request #844 from matrix-org/markjh/yield_on_sleep
...
Yield on the sleeps intended to backoff replication
2016-06-06 16:20:54 +01:00
Mark Haines
5ef84da4f1
Yield on the sleeps intended to backoff replication
2016-06-06 16:05:28 +01:00
Erik Johnston
216a05b3e3
.values() returns list of sets
2016-06-06 16:00:09 +01:00
Erik Johnston
96dc600579
Fix typos
2016-06-06 15:44:41 +01:00
Erik Johnston
377eb480ca
Fire after 30s not 8h
2016-06-06 15:14:21 +01:00
Erik Johnston
e4134c5e13
Merge pull request #841 from matrix-org/erikj/event_counter
...
Add metric counter for number of persisted events
2016-06-06 14:17:40 +01:00
Erik Johnston
7aa778fba9
Add metric counter for number of persisted events
2016-06-06 11:58:09 +01:00
Erik Johnston
70aee0717c
Add events to cache when we persist them
2016-06-06 11:34:53 +01:00
Erik Johnston
3210f4c385
Merge pull request #836 from matrix-org/erikj/change_event_cache
...
Change the way we cache events
2016-06-03 18:47:46 +01:00
Erik Johnston
cffe46408f
Don't rely on options when inserting event into cache
2016-06-03 18:25:21 +01:00
Mark Haines
ac9716f154
Fix spelling
2016-06-03 18:10:00 +01:00
Mark Haines
8f79084bd4
Add get_presence_list_accepted to the broken caches in synchrotron
2016-06-03 18:03:40 +01:00
Erik Johnston
10ea3f46ba
Change the way we cache events
2016-06-03 17:57:50 +01:00
Erik Johnston
f6be734be9
Merge pull request #835 from matrix-org/erikj/get_event_txn
...
Remove event fetching from DB threads
2016-06-03 17:30:00 +01:00
Erik Johnston
05e01f21d7
Remove event fetching from DB threads
2016-06-03 17:22:13 +01:00
David Baker
72c4d482e9
3rd time lucky: we'd already calculated it above
2016-06-03 16:39:50 +01:00
David Baker
fbf608decb
Oops, we're using the dict form
2016-06-03 16:38:39 +01:00
David Baker
06d40c8b98
Add substitutions to email notif From
2016-06-03 16:31:23 +01:00
Erik Johnston
5f88549f4a
Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop
2016-06-03 16:27:24 +01:00
Mark Haines
ca457f594e
Merge pull request #831 from matrix-org/markjh/synchrotronII
...
Add a separate process that can handle /sync requests
2016-06-03 16:06:25 +01:00
Erik Johnston
21961c93c7
Bump changelog and version
2016-06-03 15:33:14 +01:00
Mark Haines
48340e4f13
Clear the list of ongoing syncs on shutdown
2016-06-03 15:02:27 +01:00
Erik Johnston
fcbc282f56
Merge branch 'release-v0.15.0' of github.com:matrix-org/synapse into release-v0.16.0
2016-06-03 15:02:04 +01:00
Erik Johnston
51773bcbaf
Merge pull request #832 from matrix-org/erikj/presence_coount
...
Change def of small delta in presence stream. Add metrics.
2016-06-03 14:57:00 +01:00
Mark Haines
da491e75b2
Appease flake8
2016-06-03 14:56:36 +01:00
Mark Haines
0b3c80a234
Use ClientIpStore to record client ips
2016-06-03 14:55:01 +01:00
Mark Haines
dd6f62ed99
Merge branch 'develop' into markjh/synchrotronII
2016-06-03 14:51:33 +01:00
Mark Haines
eef541a291
Move insert_client_ip to a separate class
2016-06-03 14:42:35 +01:00
Mark Haines
80aade3805
Send updates to the syncing users every ten seconds or immediately if they've just come online
2016-06-03 14:24:19 +01:00
Erik Johnston
ab116bdb0c
Fix typo
2016-06-03 14:03:42 +01:00
Erik Johnston
4ce84a1acd
Change metric style
2016-06-03 13:49:16 +01:00
Erik Johnston
a7ff5a1770
Presence metrics. Change def of small delta
2016-06-03 13:40:55 +01:00
Matthew Hodgson
8d740132f4
Merge branch 'develop' into matthew/brand-from-header
2016-06-03 12:14:18 +01:00
Erik Johnston
3b096c5f5c
Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop
2016-06-03 12:00:33 +01:00
Erik Johnston
43b7f371f5
Merge pull request #830 from matrix-org/erikj/metrics_perf
...
Change CacheMetrics to be quicker
2016-06-03 11:57:29 +01:00
Mark Haines
abb151f3c9
Add a separate process that can handle /sync requests
2016-06-03 11:57:26 +01:00
Erik Johnston
4982b28868
Merge pull request #829 from matrix-org/erikj/poke_notifier
...
Poke notifier on next reactor tick
2016-06-03 11:52:10 +01:00
Erik Johnston
d06f2a229e
Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room
...
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:50:30 +01:00
Erik Johnston
58a224a651
Pull out update_results_dict
2016-06-03 11:47:07 +01:00
Mark Haines
20eccd84d4
Merge pull request #827 from matrix-org/markjh/more_slaved_methods
...
Add methods to events, account data and receipt slaves
2016-06-03 11:46:21 +01:00
Erik Johnston
722472b48c
Merge pull request #825 from matrix-org/erikj/cache_push_rules
...
Load push rules in storage layer so that they get cached
2016-06-03 11:44:32 +01:00
Erik Johnston
73c7112433
Change CacheMetrics to be quicker
...
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
Mark Haines
b09f348530
Merge pull request #824 from matrix-org/markjh/slaved_presence_store
...
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
Erik Johnston
4c04222fa5
Poke notifier on next reactor tick
2016-06-03 11:24:16 +01:00
Erik Johnston
ccb56fc24b
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:20:23 +01:00
Mark Haines
81cf449daa
Add methods to events, account data and receipt slaves
...
Adds the methods needed by /sync to the slaved events,
account data and receipt stores.
2016-06-03 11:19:27 +01:00
Erik Johnston
e043ede4a2
Small optimisation to CacheListDescriptor
2016-06-03 11:19:22 +01:00
Mark Haines
b821c839dc
Merge pull request #823 from matrix-org/markjh/more_slaved_stores
...
Add slaved stores for filters, tokens, and push rules
2016-06-03 11:17:43 +01:00
Erik Johnston
597013caa5
Make cachedList go a bit faster
2016-06-03 11:13:29 +01:00
Erik Johnston
6a0afa582a
Load push rules in storage layer, so that they get cached
2016-06-03 11:10:00 +01:00
Mark Haines
3ae915b27e
Add a slaved store for presence
2016-06-03 11:05:53 +01:00
Erik Johnston
59f2d73522
Remove unnecessary sets
2016-06-03 11:05:45 +01:00
Erik Johnston
9c26b390a2
Only get local users
2016-06-03 11:04:31 +01:00
Mark Haines
f88d747f79
Add a comment explaining why the filter cache doesn't need exipiring
2016-06-03 11:03:10 +01:00
Erik Johnston
065e739d6e
Merge pull request #811 from matrix-org/erikj/state_users_in_room
...
Use state to calculate get_users_in_room
2016-06-03 10:58:27 +01:00
Erik Johnston
696d7c5937
Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room
...
Add get_users_with_read_receipts_in_room cache
2016-06-03 10:58:24 +01:00
Mark Haines
0eae075723
Add slaved stores for filters, tokens, and push rules
2016-06-03 10:58:03 +01:00
Matthew Hodgson
79d1f072f4
brand the email from header
2016-06-02 21:34:40 +01:00
David Baker
6bb9aacf9d
Merge pull request #821 from matrix-org/dbkr/email_unsubscribe
...
Email unsubscribe links that don't require logging in
2016-06-02 17:44:55 +01:00
David Baker
745ddb4dd0
peppate
2016-06-02 17:38:41 +01:00
David Baker
7a5a5f2df2
Merge pull request #820 from matrix-org/dbkr/email_notif_string_fmt_error
...
Fix error in email notification string formatting
2016-06-02 17:26:06 +01:00
David Baker
1f31cc37f8
Working unsubscribe links going straight to the HS
...
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
Matthew Hodgson
2675c1e40e
add some branding debugging
2016-06-02 17:21:12 +01:00
David Baker
c71177f285
Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe
2016-06-02 17:20:56 +01:00
David Baker
07a5559916
Fix error in email notification string formatting
2016-06-02 17:17:16 +01:00
Mark Haines
56d15a0530
Store the typing users as user_id strings. ( #819 )
...
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
David Baker
812b5de0fe
Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe
2016-06-02 15:33:28 +01:00
Mark Haines
80f34d7b57
Fix setting the _clock in SQLBaseStore
2016-06-02 15:23:56 +01:00
Mark Haines
661a540dd1
Deduplicate presence entries in sync ( #818 )
2016-06-02 15:20:28 +01:00
Mark Haines
70599ce925
Allow external processes to mark a user as syncing. ( #812 )
...
* Add infrastructure to the presence handler to track sync requests in external processes
* Expire stale entries for dead external processes
* Add an http endpoint for making users as syncing
Add some docstrings and comments.
* Fixes
2016-06-02 15:20:15 +01:00
David Baker
fb2193cc63
Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler
...
Split out the auth handler
2016-06-02 14:31:35 +01:00
Erik Johnston
356f13c069
Disable INCLUDE_ALL_UNREAD_NOTIFS
2016-06-02 14:07:38 +01:00
Erik Johnston
02ac463dbf
Merge pull request #800 from matrix-org/erikj/sync_refactor
...
Refactor SyncHandler
2016-06-02 14:02:13 +01:00
Matthew Hodgson
c5af1b6b00
Merge pull request #814 from matrix-org/matthew/3pid_invite_auth
...
special case m.room.third_party_invite event auth to match invites,
2016-06-02 13:47:40 +01:00
David Baker
3a3fb2f6f9
Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe
2016-06-02 13:35:25 +01:00
David Baker
4a10510cd5
Split out the auth handler
2016-06-02 13:31:45 +01:00
Matthew Hodgson
f84b89f0c6
if an email pusher specifies a brand param, use it
2016-06-02 13:29:48 +01:00
David Baker
a15ad60849
Email unsubscribing that may in theory, work
...
Were it not for that fact that you can't use the base handler in the pusher because it pulls in the world. Comitting while I fix that on a different branch.
2016-06-02 11:44:15 +01:00
David Baker
e793866398
Use user_id in email greeting if display name is null
2016-06-02 09:41:13 +01:00
Matthew Hodgson
aaa70e26a2
special case m.room.third_party_invite event auth to match invites, otherwise they get out of sync and you get https://github.com/vector-im/vector-web/issues/1208
2016-06-01 22:13:47 +01:00
Erik Johnston
a04a2d043c
Merge pull request #807 from matrix-org/erikj/push_rules_cache
...
Ensure we always return boolean in push rules
2016-06-01 18:07:48 +01:00
Erik Johnston
0f06b496d1
Merge pull request #806 from matrix-org/erikj/hash_cache
...
Cache get_event_reference_hashes
2016-06-01 18:07:42 +01:00
David Baker
e0deeff23e
Fix room list spidering
2016-06-01 17:58:58 +01:00
David Baker
991af8b0d6
WIP on unsubscribing email notifs without logging in
2016-06-01 17:40:52 +01:00
David Baker
00c487a8db
Merge pull request #808 from matrix-org/dbkr/room_list_spider
...
Add secondary_directory_servers option to fetch room list from other servers
2016-06-01 15:32:52 +01:00
Erik Johnston
c8285564a3
Use state to calculate get_users_in_room
2016-06-01 15:25:25 +01:00
David Baker
d60eed0710
Limit number of notifications in an email notification
2016-06-01 11:45:43 +01:00
Erik Johnston
43db0d9f6a
Add get_users_with_read_receipts_in_room cache
2016-06-01 10:54:32 +01:00
David Baker
8e539f13c0
Merge remote-tracking branch 'origin/develop' into dbkr/room_list_spider
2016-06-01 09:54:36 +01:00
David Baker
6ecb2ca4ec
pep8
2016-06-01 09:48:55 +01:00
David Baker
2a449fec4d
Add cache to remote room lists
...
Poll for updates from remote servers, waiting for the poll if there's no cache entry.
2016-05-31 18:27:23 +01:00
David Baker
6ca4d3ae9a
Add vector.im to default secondary_directory_servers and add comment explaining it's not a permanent solution
2016-05-31 17:24:50 +01:00
Erik Johnston
dea9f20f8c
Force boolean
2016-05-31 17:24:30 +01:00
David Baker
963e3ed282
Apparently I am not permitted to have two blank lines here
2016-05-31 17:22:53 +01:00
David Baker
d240796ded
Basic, un-cached support for secondary_directory_servers
2016-05-31 17:20:07 +01:00
Mark Haines
c8c5bf950a
Fix synapse/storage/schema/delta/30/as_users.py
2016-05-31 17:10:40 +01:00
Erik Johnston
c9ca285d33
Merge pull request #805 from matrix-org/erikj/push_rules_cache
...
Fix GET /push_rules
2016-05-31 16:42:21 +01:00
Erik Johnston
1d4ee854e2
Fix typo
2016-05-31 15:45:53 +01:00
Erik Johnston
cca0093fa9
Change fix
2016-05-31 15:44:08 +01:00
Erik Johnston
4efa389299
Fix GET /push_rules
2016-05-31 15:37:53 +01:00
Erik Johnston
aefd2d1cbc
Cache get_event_reference_hashes
2016-05-31 15:32:32 +01:00
Erik Johnston
10de8c2631
Merge pull request #804 from matrix-org/erikj/push_rules_cache
...
Add caches to bulk_get_push_rules*
2016-05-31 15:04:40 +01:00
Mark Haines
c626fc576a
Move the AS handler out of the Handlers object.
...
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
Erik Johnston
e5b0bbcd33
Add caches to bulk_get_push_rules*
2016-05-31 13:46:58 +01:00
David Baker
70ecb415f5
Fix c+p fail
2016-05-31 12:00:54 +01:00
David Baker
e1625d62a8
Add federation room list servlet
2016-05-31 11:55:57 +01:00
David Baker
887c6e6f05
Split out the room list handler
...
So I can use it from federation bits without pulling in all the handlers.
2016-05-31 11:05:16 +01:00
Erik Johnston
85b992f621
Fix to allow start with postgres
2016-05-27 10:44:44 +01:00
Erik Johnston
cc84f7cb8e
Send down correct error response if user not found
2016-05-27 10:35:15 +01:00
David Baker
209ba4d024
Merge pull request #795 from matrix-org/dbkr/delete_push_actions_after_a_month
...
Only delete push actions after 30 days
2016-05-24 16:22:13 +01:00
David Baker
b007ee4606
Check for presence of 'avatar_url' key
2016-05-24 15:12:05 +01:00
David Baker
cbf8d146ac
Merge pull request #799 from matrix-org/matthew/quieter-email-notifs
...
Tune email notifs to make them quieter:
2016-05-24 14:30:51 +01:00
Erik Johnston
faad233ea6
Change short circuit path
2016-05-24 14:27:19 +01:00
Erik Johnston
6900303997
Don't send down all ephemeral events
2016-05-24 11:44:55 +01:00
David Baker
37b7e84620
Include the ts the notif was received at
2016-05-24 11:33:32 +01:00
Erik Johnston
1c5ed2a19b
Only work out newly_joined_users for incremental sync
2016-05-24 11:21:34 +01:00
Erik Johnston
b08ad0389e
Only include non-offline presence in initial sync
2016-05-24 11:15:05 +01:00
Erik Johnston
be2c677386
Spell builder correctly
2016-05-24 10:53:03 +01:00
Erik Johnston
79bea8ab9a
Inline function. Make load_filtered_recents private
2016-05-24 10:22:24 +01:00
Erik Johnston
84f94e4cbb
Add comments
2016-05-24 10:14:53 +01:00
Erik Johnston
137e6a4557
Shuffle things room
2016-05-24 09:50:55 +01:00
Matthew Hodgson
680f1d9387
catch thinko in presentable names
2016-05-23 22:55:11 +01:00
Matthew Hodgson
cb8a321bdd
fix NPE in room ordering
2016-05-23 22:54:56 +01:00
Matthew Hodgson
88ea5ab2c3
consistency is the better part of valour
2016-05-23 19:33:45 +01:00
Matthew Hodgson
989bdc9e56
Tune email notifs to make them quieter:
...
* After initial 10 minute window, only alert every 24h for room notifs
* Reset room state after 6h of idleness
* Synchronise throttles for messages sent in the same notif, so the 24 hourly notifs 'line up'
* Fix the email subjects to say what triggered the notification
* Order the rooms in reverse activity order in the email, so the 'reason' room should always come first
2016-05-23 19:24:11 +01:00
Negi Fazeli
6fe04ffef2
Fix set profile error with Requester.
...
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
2016-05-23 19:50:28 +02:00
David Baker
b791a530da
Actually make the 'read' flag correct
2016-05-23 18:48:02 +01:00
David Baker
a24bc5b2dc
Add GET /notifications API
2016-05-23 18:33:51 +01:00
Erik Johnston
c0c79ef444
Add back concurrently_execute
2016-05-23 18:21:27 +01:00
Erik Johnston
b5605dfecc
Refactor SyncHandler
2016-05-23 18:08:18 +01:00
David Baker
31b5395ab6
Remove debug logging
2016-05-23 16:32:01 +01:00
David Baker
c2da3406fc
Oops, missing comma
2016-05-20 18:03:31 +01:00
David Baker
ccffb0965d
Remove stale line
2016-05-20 17:59:10 +01:00
David Baker
18d68bfee4
Handle empty events table
2016-05-20 17:58:09 +01:00
David Baker
d4503e25ed
Make deleting push actions more efficient
...
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
David Baker
149fa411e2
Only delete push actions after 30 days
2016-05-20 15:25:12 +01:00
Kegan Dougal
332d7e9b97
Allow clients to specify a server_name to avoid 'No known servers'
...
Multiple server_names are supported via ?server_name=foo&server_name=bar
2016-05-19 13:50:52 +01:00
Matthew Hodgson
6fb51eaf7b
Merge pull request #793 from matrix-org/matthew/one-push-badge-per-convo
...
increment badge count per missed convo, not per msg
2016-05-18 13:56:38 +01:00
Matthew Hodgson
e837df6adb
increment badge count per missed convo, not per msg
2016-05-18 11:53:25 +01:00
Erik Johnston
42368ea8db
Add desc to get_presence_for_users
2016-05-18 11:38:10 +01:00
Mark Haines
0cb441fedd
Move typing handler out of the Handlers object
2016-05-17 15:58:46 +01:00
Mark Haines
6a30a0bfd3
Move the functions for parsing app service config
2016-05-17 11:28:58 +01:00
Mark Haines
523d5bcd0b
Merge remote-tracking branch 'origin/develop' into markjh/liberate_sync_handler
2016-05-17 10:43:58 +01:00
Matthew Hodgson
43e1e0489c
Merge pull request #786 from matrix-org/matthew/email_notifs_tuning
...
tune email notifs, fix CSS a bit, and add debugging details
2016-05-17 10:43:24 +01:00
Mark Haines
526bf8126f
Remove unused get_joined_rooms_for_user
2016-05-17 10:20:51 +01:00
Mark Haines
425e6b4983
Merge branch 'develop' into markjh/member_cleanup
2016-05-17 10:13:16 +01:00
Mark Haines
b153f5b150
Merge pull request #787 from matrix-org/markjh/liberate_presence_handler
...
Move the presence handler out of the Handlers object
2016-05-17 10:09:43 +01:00
Mark Haines
3fd8a07fca
Merge pull request #788 from matrix-org/markjh/domian
...
Spell "domain" correctly
2016-05-17 10:09:34 +01:00
Mark Haines
f68eea808a
Move SyncHandler out of the Handlers object
2016-05-16 20:19:26 +01:00
Mark Haines
53e171f345
Merge branch 'markjh/liberate_presence_handler' into markjh/liberate_sync_handler
2016-05-16 20:08:32 +01:00
Mark Haines
80cb9becd8
Remove get_joined_rooms_for_user from RoomMemberHandler
2016-05-16 20:06:55 +01:00
Mark Haines
816df9f267
get_room_members is unused now
2016-05-16 19:51:43 +01:00
Mark Haines
821306120a
Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room
2016-05-16 19:48:07 +01:00
Mark Haines
1a3a2002ff
Spell "domain" correctly
...
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Mark Haines
e168abbcff
Don't inherit PresenceHandler from BaseHandler, remove references to self.hs from presence handler
2016-05-16 19:08:40 +01:00
Matthew Hodgson
cbd2adc95e
tune email notifs, fix CSS a bit, and add debugging details
2016-05-16 18:58:38 +01:00
Mark Haines
3b86ecfa79
Move the presence handler out of the Handlers object
2016-05-16 18:56:37 +01:00
David Baker
647781ca56
Fix emailpusher import
...
Try importing at the root level rather than conditionally importing, as per comment
2016-05-16 18:41:32 +01:00
Erik Johnston
c39f305067
os.environ requires a string
2016-05-16 17:21:30 +01:00
Erik Johnston
678c8a7f1e
Merge pull request #785 from matrix-org/erikj/cache_factor_synctly
...
Make synctl read a cache factor from config file
2016-05-16 17:07:35 +01:00
Erik Johnston
c5c5a7403b
Make synctl read a cache factor from config file
2016-05-16 17:01:57 +01:00
Mark Haines
eb79110beb
Clean up the blacklist/whitelist handling.
...
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
2016-05-16 13:03:59 +01:00
Mark Haines
dd95eb4cb5
Merge branch 'develop' into matthew/preview_url_ip_whitelist
2016-05-16 12:59:41 +01:00
Erik Johnston
60d53f9e95
Count number of GC collects
2016-05-16 09:34:42 +01:00
Matthew Hodgson
782471b7e1
fix matrix.to URLs
2016-05-13 17:50:16 +01:00
Mark Haines
0466454b00
Assert that stream replicated stream positions are ints
2016-05-13 17:33:44 +01:00
Mark Haines
b3f29dc1e5
Manually expire broken caches like the who_forgot_in_room
2016-05-13 17:16:27 +01:00
Mark Haines
f03ddc98ec
Use the SlavedAccountDataStore
2016-05-13 17:01:28 +01:00
Mark Haines
1f71f386f6
Merge branch 'develop' into dbkr/email_notifs_on_pusher
2016-05-13 16:59:56 +01:00
Mark Haines
206eb9fd94
Shift some of the state_group methods into the SlavedEventStore
2016-05-13 16:58:14 +01:00
Erik Johnston
7d6e89ed22
Add a comment
2016-05-13 16:31:08 +01:00
Mark Haines
21018c2c13
Merge pull request #783 from matrix-org/markjh/slave_account_data
...
Add a slaved datastore for account data
2016-05-13 15:56:04 +01:00
Mark Haines
b7381d5338
Allow receipts for events we haven't seen in the db
2016-05-13 15:46:41 +01:00
Mark Haines
3abab26458
Add a slaved datastore for account data
2016-05-13 15:34:06 +01:00
Erik Johnston
99b5a2e560
Merge pull request #741 from negzi/create_user_with_expiry
...
Create user with expiry
2016-05-13 14:46:53 +01:00
Erik Johnston
ba5c616ff4
Merge pull request #778 from matrix-org/erikj/add_pusher
...
Fixup add_pusher
2016-05-13 14:43:23 +01:00
Erik Johnston
0c11c1be88
Spelling
2016-05-13 14:42:25 +01:00
Erik Johnston
e00e8f2166
Merge pull request #769 from matrix-org/erikj/push_actions_delete
...
Delete old pushers
2016-05-13 14:41:36 +01:00
Erik Johnston
fd8e921b6e
Merge pull request #779 from matrix-org/erikj/receipts
...
Use tree cache for get_linearized_receipts_for_room
2016-05-13 14:41:21 +01:00
Erik Johnston
c9aff0736c
Remove topics table
2016-05-13 14:40:38 +01:00
Negi Fazeli
40aa6e8349
Create user with expiry
...
- Add unittests for client, api and handler
Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
Mark Haines
9295fa30a8
Annotate the removed indicies with why they were removed.
2016-05-13 14:16:57 +01:00
Erik Johnston
5e50058473
Remove unused indices
...
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
Mark Haines
cdda850ce1
Merge pull request #781 from matrix-org/markjh/replication_problems
...
Fix a bug in replication that was causing the pusher to tight loop
2016-05-13 12:05:19 +01:00
Mark Haines
0e792e7903
Log the stream IDs in an order that makes sense
2016-05-13 11:54:44 +01:00
Mark Haines
3547e66bc6
Make sure we advance our stream position
2016-05-13 11:53:00 +01:00
Erik Johnston
6da7f39d95
Use tree cache for get_linearized_receipts_for_room
2016-05-13 11:41:23 +01:00
David Baker
b5e646a18c
Make email notifs work on the pusher synapse
...
Plus general bugfix to email notif code
2016-05-13 11:36:50 +01:00
Erik Johnston
13d37c3c56
Fixup add_pusher
2016-05-13 11:25:02 +01:00
Mark Haines
a458a40337
missed a spot
2016-05-12 18:19:58 +01:00
Mark Haines
7e23476814
move filter_events_for_client out of base handler
2016-05-11 13:42:37 +01:00
Mark Haines
1620578b13
Shuffle when we get the signing_key attribute.
...
Wait until we sign a message to get the signing key from the homeserver
config. This means that the message handler can be created without
having a signing key in the config which means that separate processes
like the pusher that don't send messages and don't need to sign them can
still access the handlers.
2016-05-11 12:20:57 +01:00
Erik Johnston
108434e53d
Merge pull request #775 from matrix-org/erikj/password_hash
...
Correctly handle NULL password hashes from the database
2016-05-11 12:18:13 +01:00
Erik Johnston
1400bb1663
Correctly handle NULL password hashes from the database
2016-05-11 12:06:02 +01:00
Mark Haines
458a435114
Fix typo
2016-05-11 10:35:33 +01:00
Mark Haines
30057b1e15
Move _create_new_client_event and handle_new_client_event out of base handler
2016-05-11 09:09:20 +01:00
David Baker
ae1af262f6
Pass through _get_state_group_for_events
2016-05-10 19:18:03 +02:00
David Baker
90afc07f39
StateStore, not EventsStore
2016-05-10 19:10:46 +02:00
David Baker
89b5ef7c4b
Cached functions must be accessed through the dict
2016-05-10 19:05:22 +02:00
David Baker
35b6e6d2a8
Pass though _get_state_group_for_events
2016-05-10 18:56:40 +02:00
David Baker
3367e65476
Pass through get_state_groups
2016-05-10 18:53:15 +02:00
David Baker
0c4ccdcb83
Also pass through get_profile_displayname
2016-05-10 18:51:14 +02:00
David Baker
f28643cea9
Uncommit accidentally commited edit to cipher list
2016-05-10 18:44:32 +02:00
David Baker
5f46be19a7
Pass through get_events to pusher too
2016-05-10 18:43:40 +02:00
David Baker
d46b18a00f
Pass through _get_event_txn
2016-05-10 18:27:06 +02:00
Matrix
3b1930e8ec
unbreak schema
2016-05-10 16:42:37 +01:00
Matthew Hodgson
fe97b81c09
Merge pull request #759 from matrix-org/dbkr/email_notifs
...
Send email notifications for missed messages
2016-05-10 16:30:05 +02:00
David Baker
997db04648
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-05-10 14:40:19 +02:00
David Baker
c00b484eff
More consistent config naming
2016-05-10 14:39:16 +02:00
David Baker
94040b0798
Add config option to not send email notifs for new users
2016-05-10 14:34:53 +02:00
David Baker
e04b1d6b0a
Make pep8 happy
2016-05-10 14:23:16 +02:00
Matthew Hodgson
5599608887
Switch from CSS to Table layout for HTML mails so they work in Outlook aka Word
...
Remove templates-vector and theme templates with variables instead
Switch to matrix.to URLs by default for links
2016-05-10 00:14:48 +02:00
Erik Johnston
c9eb6dfc1b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user
2016-05-09 13:21:06 +01:00
Erik Johnston
3f84da139c
Merge pull request #773 from matrix-org/erikj/get_domian_from_id
...
Add and use get_domain_from_id
2016-05-09 13:21:00 +01:00
Erik Johnston
def64d6ef3
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user
2016-05-09 13:05:09 +01:00
Erik Johnston
100e2c42f6
Merge pull request #764 from matrix-org/erikj/replication_logging
...
Add some log information at returned replication streams
2016-05-09 11:18:00 +01:00
Erik Johnston
8715731559
Merge pull request #772 from matrix-org/erikj/get_user_cache
...
Add cache to get_user_by_id
2016-05-09 11:12:11 +01:00
Erik Johnston
34b3af3363
Merge pull request #770 from matrix-org/erikj/transaction_reactor
...
Run transaction queue on reactor
2016-05-09 11:12:06 +01:00
Erik Johnston
08dfa8eee2
Add and use get_domian_from_id
2016-05-09 10:36:03 +01:00
Erik Johnston
1f1dee94f6
Manually run GC on reactor tick.
...
This also adds a metric for amount of time spent in GC.
2016-05-09 10:13:25 +01:00
Erik Johnston
f6ebaf4a32
Run transaction queue on reactor
...
This ensures that any CPU work that happens doesn't block message
sending.
2016-05-09 10:10:06 +01:00
Erik Johnston
4ea762c1a2
Add cache to get_user_by_id
2016-05-09 10:08:21 +01:00
Erik Johnston
012cb5416c
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/push_actions_delete
2016-05-06 15:59:20 +01:00
Erik Johnston
fcb2c3f0db
Remove unused import
2016-05-06 15:47:40 +01:00
Erik Johnston
fd85b167ec
Pull loop one level up
2016-05-06 15:38:42 +01:00
Erik Johnston
b6e0be701e
Queue events for persistence
2016-05-06 14:31:44 +01:00
Erik Johnston
d13459636f
Pull prev txn from in memory
2016-05-06 11:30:55 +01:00
Erik Johnston
1d275dba69
Don't needlessly enter transaction
2016-05-06 11:25:58 +01:00
Erik Johnston
56b5e83e36
Reduce database inserts when sending transactions
2016-05-06 11:20:18 +01:00
Matthew Hodgson
53ca739f1f
better mail subject lines
2016-05-05 15:55:44 +01:00
Matthew Hodgson
81c2176cba
fix layout; handle app naming in synapse, not jinja
2016-05-05 15:54:29 +01:00
Mark Haines
573ef3f1c9
Rename openid/token to openid/request_token
2016-05-05 15:15:00 +01:00
Erik Johnston
8940281d1b
Don't warn
2016-05-05 15:10:03 +01:00
Mark Haines
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
Erik Johnston
5d8a93a10e
Add some log information at returned replication streams
2016-05-05 10:29:21 +01:00
Erik Johnston
1f0f5ffa1e
Add bulk fetch storage API
2016-05-05 10:03:15 +01:00
Matthew Hodgson
634efb65f1
pep8
2016-05-05 02:10:57 +01:00
Matthew Hodgson
ce81ccb063
handle fragments correctly on mxc URLs.
...
switch to vector.im permalinks as matrix.to isn't ready yet.
merge overlapping notifications together.
give one message of context after a notification (in the unlikely event it exists, but it's possible thanks to throttling).
include name of app in mail templates
2016-05-05 02:00:33 +01:00
Matthew Hodgson
1cf5c379cb
spell out emailpusher full path
2016-05-05 01:59:39 +01:00
Erik Johnston
fee1118a20
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user
2016-05-04 19:08:27 +01:00
Erik Johnston
97b9141245
Merge pull request #762 from matrix-org/erikj/report_event
...
Add /report endpoint
2016-05-04 19:05:49 +01:00
Erik Johnston
fcd1eb642d
Add primary key
2016-05-04 16:51:51 +01:00
Erik Johnston
8e6a163f27
Add timestamp and auto incrementing ID
2016-05-04 15:19:12 +01:00
David Baker
39d0a99972
Include no context
...
until we can de-dup between the context and other notifs
2016-05-04 14:52:49 +01:00
David Baker
9ef05a12c3
Add date header & message id
2016-05-04 14:52:10 +01:00
David Baker
80be396464
Correct SQL statement for postgres
...
In standard sql, join binds tighter than comma, so we were joining on the wrong table. Postgres follows the standard (apparently).
2016-05-04 13:19:59 +01:00
Erik Johnston
5650e38e7d
Move event_id to path
2016-05-04 13:19:39 +01:00
David Baker
8cc82aad87
Add db functions used for email to the pusher app
2016-05-04 11:47:59 +01:00
David Baker
de22001ab5
pep8
2016-05-04 11:41:35 +01:00
Matthew Hodgson
f1026418ea
copyright
2016-05-04 11:38:01 +01:00
Matthew Hodgson
17cbf773b9
fix assorted typos in default config
2016-05-04 11:38:01 +01:00
Erik Johnston
984d4a2c0f
Add /report endpoint
2016-05-04 11:28:10 +01:00
David Baker
e6bffa4475
Unused import
2016-05-04 11:26:58 +01:00
David Baker
92f0f3d21d
Catch all exceptions when creating a pusher
2016-05-04 11:24:07 +01:00
Erik Johnston
a438a6d2bc
Implement basic ignore user
2016-05-04 10:16:46 +01:00
Erik Johnston
7ea3b4118d
Merge pull request #757 from matrix-org/erikj/event_auth_typo
...
Fix typo in event_auth servlet path
2016-05-03 14:23:15 +01:00
Erik Johnston
183f23f10d
Delete old pushers
2016-05-03 14:22:33 +01:00
Matthew Hodgson
792def4928
add a url_preview_ip_range_whitelist config param so we can whitelist the matrix.org IP space
2016-05-01 12:44:24 +01:00
David Baker
2df75de505
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-04-29 20:28:47 +01:00
David Baker
b084e4d963
Add constant for throttle multiplier
2016-04-29 20:14:55 +01:00
David Baker
35b7b8e4bc
Remove unused function
2016-04-29 20:10:34 +01:00
David Baker
6b9b6a9169
Remove unused arg
2016-04-29 20:02:52 +01:00
David Baker
c7c75e87dc
Docstring
2016-04-29 19:47:35 +01:00
David Baker
b0a1036d93
Use explicit join
2016-04-29 19:28:56 +01:00
David Baker
8f99cd5996
Oops, actually specify the user id
2016-04-29 19:27:03 +01:00
David Baker
60f44c098d
Remove unnecessary if
2016-04-29 19:17:10 +01:00
David Baker
50ad8005e4
Put spaces at start of line
2016-04-29 19:16:15 +01:00
David Baker
83618d719a
Try imports in config
2016-04-29 19:13:52 +01:00
David Baker
e7a76b5123
Use the constant
2016-04-29 19:10:45 +01:00
David Baker
29c8cf8db8
Avoid vars
builtin
2016-04-29 19:09:28 +01:00
David Baker
d3da5294e8
Use named parameter format
2016-04-29 19:04:40 +01:00
David Baker
765f2b8446
Default enable email notifs to False
2016-04-29 14:46:18 +01:00
David Baker
311b5ce051
pep8
2016-04-29 14:37:30 +01:00
David Baker
3facde2536
Remove rather pointless get function
2016-04-29 14:36:45 +01:00
David Baker
4364ea1272
Stop processing notifs once we've sent a mail
2016-04-29 14:31:27 +01:00
David Baker
4b0c3a3270
Correct public_baseurl default
2016-04-29 14:30:15 +01:00
David Baker
5048455965
Nicer get() shorthand
2016-04-29 14:27:40 +01:00
David Baker
6c8957be7f
Remove redundant docstring
2016-04-29 14:25:28 +01:00
David Baker
18ce88bd2d
Correct default template and add text template
2016-04-29 14:24:25 +01:00
David Baker
40d40e470d
Send mail notifs with a plaintext part too
2016-04-29 13:56:21 +01:00
David Baker
dc2c527ce9
Fix password reset
...
Default requester to None, otherwise it isn't defined when resetting using email auth
2016-04-29 12:07:54 +01:00
Erik Johnston
62b51b8452
Fix typo in event_auth servlet path
2016-04-29 12:00:51 +01:00
David Baker
b2c04da8dc
Add an email pusher for new users
...
If they registered with an email address and email notifs are enabled on the HS
2016-04-29 11:43:57 +01:00
David Baker
ec9cbe847d
pep8 newline
2016-04-29 10:07:30 +01:00
David Baker
acded821c4
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-04-29 10:05:20 +01:00
David Baker
e800ee2f63
May as well always include room link
2016-04-28 17:28:27 +01:00
David Baker
cc0874cf71
Put back real delay before mailing
2016-04-28 17:00:40 +01:00
David Baker
68f8fc2f14
Support file messages & fix plain text
2016-04-28 16:59:57 +01:00
David Baker
4845c7359d
Support image notifs
2016-04-28 15:55:53 +01:00
Mark Haines
351b50a887
Fix more typos in per-request metrics
2016-04-28 15:29:46 +01:00
David Baker
60f86fc876
pep8
2016-04-28 15:16:30 +01:00
David Baker
937c407eef
Only import email pusher if email notifs are on
2016-04-28 15:12:14 +01:00
Mark Haines
dcfc10b129
Fix typo in request metrics
2016-04-28 15:11:06 +01:00
Matthew Hodgson
aebd0c9717
fix typo
2016-04-28 15:09:25 +01:00
Mark Haines
1a12766e3b
Add a comment explaining why automatic metric reporting is disabled for JsonResource
2016-04-28 12:31:26 +01:00
Mark Haines
6037349512
Check if report_metrics is True
2016-04-28 12:26:07 +01:00
David Baker
ebbabc4986
Handle room invites in email notifs
2016-04-28 11:49:36 +01:00
Mark Haines
8d7ad44331
Report per request metrics for all of the things using request_handler
2016-04-28 10:57:49 +01:00
David Baker
9dba1b668c
Linkify plain text messages too
2016-04-28 10:55:08 +01:00
David Baker
424a7f48f8
Run filter_events_for_client
...
so we don't accidentally mail out events people shouldn't see
2016-04-27 17:50:49 +01:00
David Baker
4ed1e45869
Make html messages work
2016-04-27 17:18:51 +01:00
Mark Haines
21d188bf95
Merge pull request #755 from matrix-org/markjh/right_direction
...
Fix backfill replication to advance the stream correctly
2016-04-27 15:54:48 +01:00
Mark Haines
8a65666454
Fix backfill replication to advance the stream correctly
2016-04-27 15:38:43 +01:00
David Baker
8781083960
Better grammar for multiple messages in a room
...
Say who the messages are from if there's no room name, otherwise it's a bit nonsensical
2016-04-27 15:30:41 +01:00
David Baker
fa12209c1b
Hopefully all remaining bits for email notifs
...
Add public facing base url to the server so synapse knows what URL to use when converting mxc to http urls for use in emails
2016-04-27 15:09:55 +01:00
Mark Haines
871357d539
Check that somethign has happend before running the selects
2016-04-27 11:54:13 +01:00
Mark Haines
71df327190
Actually start the pusher daemon
2016-04-26 17:07:09 +01:00
Mark Haines
c9eab73f2a
Fix typo in default pusher config
2016-04-26 17:06:18 +01:00
Mark Haines
b80b93ea0f
Add a log context to the daemonized pusher
2016-04-26 15:57:28 +01:00
Mark Haines
6df5a6a833
Optionally daemonize the pusher
2016-04-26 15:37:41 +01:00
Erik Johnston
5164ccc3e5
Bump changelog and version
2016-04-26 11:26:32 +01:00
Erik Johnston
3306cf45ca
Merge pull request #750 from matrix-org/erikj/jwt_optional
...
Make pyjwt dependency optional
2016-04-26 11:07:22 +01:00
Mark Haines
9c417c54d4
Add a couple of update methods to the PusherSlaveStore
2016-04-26 10:45:02 +01:00
David Baker
7b4715bad7
More variable calculation for email notifs
...
Include name of the person we're sending to and add summary text at the top giving an overview of what's happened.
2016-04-25 18:27:04 +01:00
Mark Haines
f15e9e8de4
Remove the uncomments from the comments
2016-04-25 17:56:24 +01:00
Mark Haines
72e2fafa20
Add a metrics listener and a ssh listener to the pusher
2016-04-25 17:34:25 +01:00
Mark Haines
f22f46f4f9
Move the listenTCP call outside the manhole function
2016-04-25 14:59:21 +01:00
David Baker
290f125a13
Typo
2016-04-25 14:42:59 +01:00
Erik Johnston
52ecbc2843
Make pyjwt dependency optional
2016-04-25 14:30:15 +01:00
David Baker
05e49ffbdf
No we don't: it's just the display name
2016-04-22 18:44:17 +01:00
David Baker
bd0f9c2065
Actually do UTF8 correctly
2016-04-22 18:42:00 +01:00
David Baker
c5b3c6e101
Sort member events
...
So names of people in a room are given in order
2016-04-22 18:33:36 +01:00
David Baker
83bf65297a
Mime part is binary so encode it first.
...
Doesn't get character enocind right yet but makes it not error.
2016-04-22 18:31:47 +01:00
David Baker
e8701e64b9
Implement group-of-people names
2016-04-22 17:28:42 +01:00
David Baker
c553797c4f
No inlineCallbacks necessary on this
2016-04-22 17:27:54 +01:00
Mark Haines
5905f36f05
Split out setting up the manhole to a separate file
2016-04-22 17:09:15 +01:00
Mark Haines
62607d5452
Merge branch 'develop' into markjh/split_out_site.py
...
Conflicts:
synapse/app/homeserver.py
2016-04-22 16:26:57 +01:00
Mark Haines
e856036f4c
Move SynapseSite to its own file
2016-04-22 16:09:55 +01:00
Mark Haines
9e7aa98c22
Split out create_resource_tree to a separate file
2016-04-22 15:40:51 +01:00
Mark Haines
2022ae0fb9
Merge pull request #746 from matrix-org/markjh/split_out_pusher
...
Optionally split out the pushers into a separate process
2016-04-22 11:34:08 +01:00
Erik Johnston
64ec3493c1
Merge pull request #745 from matrix-org/erikj/search-index
...
Optimise event_search in postgres
2016-04-22 11:23:49 +01:00
Erik Johnston
183cacac90
Simplify query and handle finishing correctly
2016-04-22 10:01:57 +01:00
David Baker
c10ed26c30
Flesh out email templating
...
Mostly WIP porting the room name calculation logic from the web client so our room names in the email mirror the clients.
2016-04-21 19:19:07 +01:00
Erik Johnston
ae571810f2
Order NULLs first
2016-04-21 18:14:18 +01:00
Erik Johnston
3ddbb1687c
Fix query
2016-04-21 18:02:36 +01:00
Erik Johnston
8fae3d7b1e
Use special UPDATE syntax
2016-04-21 18:01:49 +01:00
Erik Johnston
b57dcb4b51
Typo
2016-04-21 17:49:00 +01:00
Erik Johnston
26db18bc90
Need to do _background_update_progress_txn in actual transaction
2016-04-21 17:45:56 +01:00
Erik Johnston
b9675ef6e6
Merge pull request #687 from nikriek/jwt-fix
...
Fix issues with JWT login
2016-04-21 17:42:25 +01:00
Erik Johnston
e395eb1108
Update progress when creating index
2016-04-21 17:39:24 +01:00
Erik Johnston
3b0fa77f50
Fix SQL statement
2016-04-21 17:37:42 +01:00
Erik Johnston
129e403487
Create index must be on a conn
2016-04-21 17:35:51 +01:00
Mark Haines
a3ac837599
Optionally split out the pushers into a separate process
2016-04-21 17:22:37 +01:00
Mark Haines
78741cf025
Merge pull request #743 from matrix-org/markjh/slave_pushers
...
Replicate the pushers
2016-04-21 17:21:29 +01:00
Erik Johnston
51bb339ab2
Create index concurrently
2016-04-21 17:16:11 +01:00
Erik Johnston
b743c1237e
Add missing run_upgrade
2016-04-21 17:12:04 +01:00
Mark Haines
31719ad124
Merge pull request #744 from matrix-org/markjh/replication_remove_pusher
...
Add a replication endpoint for deleting pushers
2016-04-21 17:10:49 +01:00
Niklas Riekenbrauck
565c2edb0a
Fix issues with JWT login
2016-04-21 18:10:48 +02:00
Erik Johnston
c877f0f034
Optimise event_search in postgres
2016-04-21 16:56:14 +01:00
Mark Haines
cfe1ff4bdb
Add a replication endpoint for deleting pushers
2016-04-21 16:33:05 +01:00
Mark Haines
d4823efad9
Replicate the pushers
2016-04-21 16:18:00 +01:00
Mark Haines
c0d8e0eb63
Replicate push actions
2016-04-21 15:25:58 +01:00
David Baker
2ed0adb075
Generate mails from a template
2016-04-20 18:35:29 +01:00
David Baker
05adc6c2de
more pep8
2016-04-20 13:02:45 +01:00
David Baker
f63bd4ff47
Send a rather basic email notif
...
Also pep8 fixes
2016-04-20 13:02:01 +01:00
Erik Johnston
5bbc321588
Always use state cache entry if it exists
...
Also check if the resolved state matches an existing state group.
2016-04-20 11:49:10 +01:00
Erik Johnston
4cf4320593
Add some logging to state resolve_events
2016-04-20 11:06:02 +01:00
Erik Johnston
eab47ea1e5
Merge pull request #739 from matrix-org/erikj/cache_get_state_groups_for_groups
...
Add cache to _get_state_groups_from_groups
2016-04-19 17:37:19 +01:00
Mark Haines
f52dd35ac3
Merge pull request #738 from matrix-org/markjh/slaved_receipts
...
Add a slaved receipts store
2016-04-19 17:31:59 +01:00
Erik Johnston
61c7edfd34
Add cache to _get_state_groups_from_groups
2016-04-19 17:22:03 +01:00
Mark Haines
5bbd424ee0
Add a slaved receipts store
2016-04-19 17:14:08 +01:00
Erik Johnston
6ac40f7b65
Merge pull request #737 from matrix-org/erikj/spider_ssl_factory
...
Use tls_server_context_factory for SpiderEndpoint
2016-04-19 16:22:05 +01:00
Erik Johnston
f505575f69
Make InsecureInterceptableContextFactory work with SpiderEndpoint
2016-04-19 16:08:14 +01:00
Mark Haines
e99365f601
Replicate get_invited_rooms_for_user
2016-04-19 15:22:14 +01:00
David Baker
e2a01455af
Add single instance & logging stuff
...
Copy the stuff over from http pusher that prevents multiple instances of process running at once and sets up logging and measure blocks.
2016-04-19 14:52:58 +01:00
Erik Johnston
e8884e5e9c
Add self.media_repo to PreviewUrlResource
2016-04-19 14:51:34 +01:00
Erik Johnston
a7001c311b
_make_dirs was moved to MediaRepository
2016-04-19 14:49:31 +01:00
Erik Johnston
9181e2f4c7
Add store to PreviewUrlResource
2016-04-19 14:48:24 +01:00
Erik Johnston
fb76a81ff7
Reorder imports
2016-04-19 14:45:05 +01:00
David Baker
07d765209d
First bits of emailpusher
...
Mostly logic of when to send an email
2016-04-19 14:24:36 +01:00
Erik Johnston
0c93df89b6
Move MediaRepository to media_repository module
2016-04-19 11:31:43 +01:00
Erik Johnston
43f0941e8f
Split out BaseMediaResource into MediaRepository
...
This is so that a single MediaRepository can be shared across all
resources, rather than having a "copy" per resource.
In particular this allows us to guard against both the thumbnail and
download resource triggering a download of remote content at the same
time.
2016-04-19 11:24:59 +01:00
Erik Johnston
eb8619e256
Create log context in Measure if one doesn't exist
2016-04-18 16:08:32 +01:00
Erik Johnston
4ef7a25c10
Merge pull request #733 from matrix-org/erikj/make_member_timeout
...
Lower timeout for make_membership_event
2016-04-18 15:08:05 +01:00
Erik Johnston
3727a15764
Merge pull request #732 from matrix-org/erikj/login
...
Simplify _check_password
2016-04-18 15:07:57 +01:00
Matthew Hodgson
aaabbd3e9e
explicitly pass in the charset from Content-Type to lxml to fix cyrillic woes better
2016-04-15 14:32:25 +01:00
Matthew Hodgson
84f9cac4d0
fix cyrillic URL previews by hardcoding all page decoding to UTF-8 for now, rather than relying on lxml's heuristics which seem to get it wrong
2016-04-15 13:20:08 +01:00
Erik Johnston
914f1eafac
Lower timeout for make_membership_event
...
Calls to make_membership_event are done in response to client requests,
and so should not be retried over long timeframes.
2016-04-15 11:22:23 +01:00
Erik Johnston
6fd2f685fe
Simplify _check_password
2016-04-15 11:17:18 +01:00
Erik Johnston
cb9c465707
Use SynapseError 504 for Timeout errors
2016-04-15 10:21:32 +01:00
Mark Haines
3c79bdd7a0
Fix check_password rather than inverting the meaning of _check_local_password ( #730 )
2016-04-14 19:00:21 +01:00
David Baker
a4c56bf67b
Merge pull request #729 from matrix-org/dbkr/fix_login_nonexistent_user
...
Fix login to error for nonexistent users
2016-04-14 18:46:45 +01:00
David Baker
4c1b32d7e2
Fix login to error for nonexistent users
...
Fixes SYN-680
2016-04-14 18:28:42 +01:00
Matthew Hodgson
f78b479118
fix urlparse import thinko breaking tiny URLs
2016-04-14 15:23:55 +01:00
Kegan Dougal
83776d6219
Make v2_alpha reg follow the AS API specification
...
The spec is clear the key should be 'user' not 'username' and this is indeed
the case for v1. This is not true for v2_alpha though, which is what this
commit is fixing.
2016-04-14 14:52:26 +01:00
Matthew Hodgson
bd77216d06
comment out 2c838f6459
due to risk of https://en.wikipedia.org/wiki/Billion_laughs attacks - thanks @torhve
2016-04-14 14:39:24 +01:00
Erik Johnston
9ae64c9910
Measure push action generator
2016-04-14 13:42:22 +01:00
Erik Johnston
b42ad359e9
Merge pull request #725 from matrix-org/dbkr/push_only_joined
...
Don't push for everyone who ever sent an RR to the room
2016-04-14 12:05:13 +01:00
David Baker
757e2c79b4
Don't push for everyone who ever sent an RR to the room
2016-04-14 12:02:50 +01:00
Erik Johnston
86e9bbc74e
Add missing yield
2016-04-14 11:56:52 +01:00
Erik Johnston
e40f25ebe1
Rename log context
2016-04-14 11:54:14 +01:00
Erik Johnston
2ae91a9e2f
Make send_badge private
2016-04-14 11:37:50 +01:00
Erik Johnston
d213d69fe3
Add desc arg
2016-04-14 11:36:23 +01:00
Erik Johnston
56da835eaf
Add necessary logging contexts
2016-04-14 11:33:50 +01:00
Erik Johnston
96bcfb29c7
Add index
2016-04-14 11:26:33 +01:00
Erik Johnston
7be1065b8f
Add extra Measure
2016-04-14 11:26:15 +01:00
Erik Johnston
a2546b9082
Fix query for get_unread_push_actions_for_user_in_range
2016-04-14 11:08:31 +01:00
Erik Johnston
ceeb5b909f
Merge pull request #721 from matrix-org/erikj/spider
...
Sanitize the optional dependencies for spider API
2016-04-14 09:59:29 +01:00
David Baker
43a89cca8e
Merge pull request #722 from matrix-org/dbkr/only_unread_event_actions
...
Only return unread notifications
2016-04-13 14:54:26 +01:00
Erik Johnston
f338bf9257
Give install requirements
2016-04-13 14:33:48 +01:00
David Baker
767fc0b739
pep8
2016-04-13 14:23:27 +01:00
David Baker
54d08c8868
Only return unread notifications
...
Make get_unread_push_actions_for_user_in_range only return unread event actions, being more true to its name. Done in two separate sql queries to get actions after a read receipt and those in a room wiht no receipt at all. SQL queries by Erik.
2016-04-13 14:16:45 +01:00
Erik Johnston
5880bc5417
Merge pull request #718 from matrix-org/erikj/public_room_list
...
Don't return empty public rooms
2016-04-13 14:07:26 +01:00
Erik Johnston
f613a3e332
Merge pull request #720 from matrix-org/erikj/auth_chec
...
Don't auto log failed auth checks
2016-04-13 14:07:23 +01:00
Erik Johnston
bfe586843f
Add back in helpful description for missing url_preview_ip_range_blacklist
2016-04-13 13:52:57 +01:00
Erik Johnston
d0633e6dbe
Sanitize the optional dependencies for spider API
2016-04-13 13:38:09 +01:00
Erik Johnston
0f2ca8cde1
Measure Auth.check
2016-04-13 11:15:59 +01:00
Erik Johnston
c53f9d561e
Don't auto log failed auth checks
2016-04-13 11:11:46 +01:00
David Baker
65141161f6
Unused member variable
2016-04-12 16:25:26 +01:00
Erik Johnston
72f454b752
Don't return empty public rooms
2016-04-12 16:06:18 +01:00
David Baker
d33d623f0d
Merge pull request #716 from matrix-org/dbkr/get_pushers
...
Add get endpoint for pushers
2016-04-12 14:40:37 +01:00
David Baker
7984ffdc6a
Unneccessarywhitespaceisunnecessary
2016-04-12 13:55:57 +01:00
David Baker
c1267d04c5
Oops, forgot the desc.
2016-04-12 13:55:32 +01:00
David Baker
a04c076b7f
Make the /set part mandatory
2016-04-12 13:54:41 +01:00
David Baker
44891b4a0a
Tidy up get_pusher functions
...
Decodes pushers rows on the main thread rather than the db thread and uses _simple_select_list. Also do the same to the function I copied and factor out the duplication into a helper function.
2016-04-12 13:47:17 +01:00
David Baker
7b39bcdaae
Mis-named function
2016-04-12 13:35:08 +01:00
David Baker
d937f342bb
Split into separate servlet classes
2016-04-12 13:33:30 +01:00
Erik Johnston
c48465dbaa
More comments
2016-04-12 12:48:30 +01:00
Erik Johnston
8be1a37909
More comments
2016-04-12 12:04:19 +01:00
Erik Johnston
d3d0be4167
Don't append to unused list
2016-04-12 11:59:00 +01:00
Erik Johnston
762ada1e07
Add back backfilled parameter that was removed
2016-04-12 11:58:04 +01:00
Erik Johnston
0d3da210f0
Add comment
2016-04-12 11:54:41 +01:00
Erik Johnston
cccf86dd05
Check if we've already backfilled events
2016-04-12 11:19:32 +01:00
David Baker
8a76094965
Add get endpoint for pushers
...
As per https://github.com/matrix-org/matrix-doc/pull/308
2016-04-11 18:00:03 +01:00
Mark Haines
790f5848b2
Fix the rule_id for .m.rule.invite_for_me ( #715 )
2016-04-11 16:10:39 +01:00
Mark Haines
82d7eea7e3
Move the versionstring code out of app.homeserver into util
2016-04-11 14:57:09 +01:00
David Baker
2547dffccc
Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
...
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
David Baker
9bb041791c
Run unsafe proces in a loop until we've caught up
...
and wrap unsafe process in a try block
2016-04-11 12:48:30 +01:00
Erik Johnston
17515bae14
PEP8
2016-04-11 11:02:50 +01:00
Matthew Hodgson
4bd3d25218
Merge pull request #688 from matrix-org/matthew/preview_urls
...
URL previewing support
2016-04-11 10:40:29 +01:00
Matthew Hodgson
5ffacc5e84
fix typos and needless try/except from PR review
2016-04-11 10:39:16 +01:00
Matthew Hodgson
83b2f83da0
actually throw meaningful errors
2016-04-08 21:36:59 +01:00
Mark Haines
b36270b5e1
Fix pep8 warning
2016-04-08 19:52:23 +01:00
Matthew Hodgson
6ff7a79308
move local_media_repository_url_cache.sql to schema v31
2016-04-08 19:09:02 +01:00
Matthew Hodgson
af582b66bb
fix typo
2016-04-08 19:08:47 +01:00
Matthew Hodgson
2460d904bd
fix error checking for new SQL
2016-04-08 19:04:29 +01:00
Matthew Hodgson
1ccabe2965
more PR feedback
2016-04-08 18:58:08 +01:00
Matthew Hodgson
fb83f6a1fc
fix SQL based on PR feedback
2016-04-08 18:55:38 +01:00
Matthew Hodgson
dafef5a688
Add url_preview_enabled config option to turn on/off preview_url endpoint. defaults to off.
...
Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered.
Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered.
Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist
Add commentary and generally address PR feedback
2016-04-08 18:37:15 +01:00
David Baker
d96a070a3a
Actually check if we;re processing
2016-04-08 16:49:39 +01:00
David Baker
ed3979df5f
Fix invite pushes
...
* If the event is an invite event, add the invitee to list of user we run push rules for (if they have a pusher etc)
* Move invite_for_me to be higher prio than member events otherwise member events matches them
* Spell override right
2016-04-08 15:29:59 +01:00
Erik Johnston
79fc4ff6f9
Merge pull request #677 from matrix-org/erikj/dns_cache
...
Read from DNS cache if within TTL
2016-04-08 14:09:56 +01:00
David Baker
7b6d519482
Make sure max stream ordering only increases
2016-04-08 14:08:16 +01:00
David Baker
52d1008661
Unsafe process should call itself if the max has changed
2016-04-08 14:06:54 +01:00
Erik Johnston
96bd8ff57c
Merge pull request #707 from matrix-org/markjh/remove_changed_presencelike_data
...
changed_presencelike_data isn't observed anywhere so can be removed
2016-04-08 14:04:54 +01:00
David Baker
ce3fe52498
Comment why unsafe process is unsafe
2016-04-08 14:02:38 +01:00
Mark Haines
7e2f971c08
Remove some unused functions ( #711 )
...
* Remove some unused functions
* get_room_events_stream is only used in tests
* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
Mark Haines
b9ee5650b0
Move all the wrapper functions for distributor.fire
...
Move the functions inside the distributor and import them
where needed. This reduces duplication and makes it possible
for flake8 to detect when the functions aren't used in a
given file.
2016-04-08 11:01:38 +01:00
Mark Haines
caef337587
changed_presencelike_data isn't observed anywhere in synapse so can be removed
2016-04-08 10:37:19 +01:00
Mark Haines
b4a5002a6e
Merge pull request #708 from matrix-org/markjh/remove_collect_presencelike_data
...
Call profile handler get_displayname directly
2016-04-08 09:51:36 +01:00
Mark Haines
86be915cce
Call profile handler get_displayname directly rather than using collect_presencelike_data
2016-04-07 18:11:49 +01:00
David Baker
d9f38561c8
Literally a dictionary
2016-04-07 17:45:01 +01:00
David Baker
4836864f56
generate id in the main thread
2016-04-07 17:38:48 +01:00
David Baker
a4a31fa8dc
Only pass in what we need
2016-04-07 17:37:19 +01:00
Erik Johnston
f942980c0b
Merge pull request #701 from DoubleMalt/ldap-auth
...
Add LDAP authentication
2016-04-07 17:35:28 +01:00
David Baker
3fb35cbd6f
Oops, inequality fail
2016-04-07 17:33:37 +01:00
David Baker
15e0f1696f
Wrap process in a flag so we don't process whist already processing.
2016-04-07 17:31:08 +01:00
Matthew Hodgson
d6e7333ae4
Merge branch 'develop' into matthew/preview_urls
2016-04-07 17:26:44 +01:00
David Baker
6ec02e9ecf
indenting
2016-04-07 17:24:05 +01:00
David Baker
25cd5bb697
defer.gatherResults rather than doing all the pokes in series
2016-04-07 17:22:14 +01:00
David Baker
fa129ce5b5
Add measure blocks
2016-04-07 17:12:29 +01:00
David Baker
e1e042f2a1
Add comments on min_stream_id
...
saying that the min stream id won't be completely accurate all the time
2016-04-07 17:09:36 +01:00
Mark Haines
ceb599e789
Add tests for redactions
2016-04-07 16:52:07 +01:00
Mark Haines
8c82b06904
Merge pull request #704 from matrix-org/markh/slaveIII
...
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 16:49:34 +01:00
David Baker
05d044aac3
pep8
2016-04-07 16:45:38 +01:00
David Baker
2d5c693fd3
Fix port script for changes merged from develop
2016-04-07 16:43:54 +01:00
Mark Haines
57fa1801c3
Add sensible __eq__ operators inside the tests.
...
Rather than adding them globally. This limits the changes to only
affect the tests.
2016-04-07 16:41:37 +01:00
Erik Johnston
a294b04bf0
Merge pull request #700 from matrix-org/erikj/deduplicate_joins
...
Deduplicate membership changes
2016-04-07 16:35:40 +01:00
David Baker
9c99ab4572
Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions
2016-04-07 16:35:22 +01:00
David Baker
d549fdfa22
Remove code that's now been obsoleted or moved elsewhere
2016-04-07 16:31:38 +01:00
Erik Johnston
95ac3078da
Rename things
2016-04-07 16:07:16 +01:00
David Baker
92e3071623
Send badge count pushes.
...
Also fix bugs with retrying.
2016-04-07 15:39:53 +01:00
Erik Johnston
ee5aef6c72
Log contexts and squash things together
2016-04-07 15:34:21 +01:00
Erik Johnston
639cd07d6d
Add comment
2016-04-07 14:24:12 +01:00
Erik Johnston
af03ecf352
Deduplicate joins
2016-04-07 14:19:02 +01:00
Mark Haines
60ec9793fb
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 13:17:56 +01:00
Erik Johnston
a28d066732
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache
2016-04-07 11:11:17 +01:00
Erik Johnston
1ef0365670
Set profile information when joining rooms remotely
2016-04-07 09:42:52 +01:00
Richard van der Hoff
87a30890a3
Merge pull request #699 from matrix-org/rav/show_own_leave_event
...
Let users see their own leave events
2016-04-06 17:57:06 +01:00
Christoph Witzany
ed4d18f516
fix check for failed authentication
2016-04-06 18:30:11 +02:00
Christoph Witzany
9c62fcdb68
remove line
2016-04-06 18:23:46 +02:00
Christoph Witzany
27a0c21c38
make tests for ldap more specific to not be fooled by Mocks
2016-04-06 18:23:46 +02:00
Christoph Witzany
3555a659ec
output ldap version for info and to pacify pep8
2016-04-06 18:23:46 +02:00
Christoph Witzany
4c5e8adf8b
conditionally import ldap
2016-04-06 18:23:46 +02:00
Christoph Witzany
875ed05bdc
fix pep8
2016-04-06 18:23:46 +02:00
Christoph Witzany
67f3a50e9a
fix exception handling
2016-04-06 18:23:46 +02:00
Christoph Witzany
afff321e9a
code style
2016-04-06 18:23:46 +02:00
Christoph Witzany
8f0e47fae8
cleanup
2016-04-06 18:23:45 +02:00
Christoph Witzany
823b8be4b7
add tls property and twist my head around twisted
2016-04-06 18:23:45 +02:00
Christoph Witzany
92767dd703
add tls property
2016-04-06 18:23:45 +02:00
Christoph Witzany
7b9319b1c8
move LDAP authentication to AuthenticationHandler
2016-04-06 18:23:45 +02:00
Christoph Witzany
3d95405e5f
Introduce LDAP authentication
2016-04-06 18:23:45 +02:00
David Baker
0fd1cd2400
pep8
2016-04-06 16:50:47 +01:00
Mark Haines
6bfec56796
Test that room membership is replicated
2016-04-06 16:20:13 +01:00
Mark Haines
e815763b7f
Merge pull request #697 from matrix-org/markjh/slaveI
...
Add a slaved events store class
2016-04-06 16:19:25 +01:00
David Baker
7e2c89a37f
Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
...
* Make badges work again
* Remove old, unused code
2016-04-06 15:42:15 +01:00
Richard van der Hoff
1e05637e37
Let users see their own leave events
...
... otherwise clients get confused.
Fixes https://matrix.org/jira/browse/SYN-662 ,
https://github.com/vector-im/vector-web/issues/368
2016-04-06 15:36:19 +01:00
Erik Johnston
b713934b2e
Merge pull request #698 from matrix-org/erikj/port_script_fix
...
Don't require config to create database
2016-04-06 14:32:45 +01:00
Mark Haines
75fb9ac1be
Add a slaved events store class
...
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
2016-04-06 14:18:35 +01:00
Erik Johnston
8aab9d87fa
Don't require config to create database
2016-04-06 14:15:45 +01:00
Mark Haines
7d11f825aa
Merge pull request #694 from matrix-org/markjh/caches
...
Move _get_cache_dict into the SQLBaseStore
2016-04-06 13:21:25 +01:00
Mark Haines
87f2dec8d4
Make the cache objects be per instance rather than being global
2016-04-06 13:08:05 +01:00
Mark Haines
a1e0d316ea
Move _get_cache_dict into the SQLBaseStore
2016-04-06 13:05:19 +01:00
Mark Haines
2e308a3a38
Merge pull request #692 from matrix-org/markjh/replicate_reshuffle
...
Separate generating the replication response...
2016-04-05 13:23:36 +01:00
Erik Johnston
c2b429ab24
Merge pull request #693 from matrix-org/erikj/backfill_self
...
Don't backfill from self
2016-04-05 13:04:36 +01:00
Erik Johnston
6222ae51ce
Don't backfill from self
2016-04-05 12:56:29 +01:00
Erik Johnston
b29f98377d
Merge pull request #691 from matrix-org/erikj/member
...
Fix stuck invites
2016-04-05 12:44:39 +01:00
Mark Haines
1d4deff25a
Separate generating the replication response...
...
from doing the http request parsing to make it easier
to write unit tests for replication.
2016-04-05 11:23:57 +01:00
Erik Johnston
df727f2126
Fix stuck invites
...
If rejecting a remote invite fails with an error response don't fail
the entire request; instead mark the invite as locally rejected.
This fixes the bug where users can get stuck invites which they can
neither accept nor reject.
2016-04-05 11:13:24 +01:00
Erik Johnston
7a77f8b6d5
Merge pull request #690 from matrix-org/erikj/member
...
Store invites in a separate table.
2016-04-05 09:12:27 +01:00
Erik Johnston
0c53d750e7
Docs and indents
2016-04-04 18:02:48 +01:00
Erik Johnston
92ab45a330
Add upgrade path, rename table
2016-04-04 17:07:43 +01:00
Erik Johnston
3d76b7cb2b
Store invites in a separate table.
2016-04-04 16:30:15 +01:00
Erik Johnston
bf14883a04
Merge pull request #689 from matrix-org/erikj/member
...
Do checks for memberships before creating events
2016-04-04 11:56:40 +01:00
Matthew Hodgson
9f7dc2bef7
Merge branch 'develop' into matthew/preview_urls
2016-04-04 00:38:21 +01:00
Matthew Hodgson
cf51c4120e
report image size (bytewise) in OG meta
2016-04-03 23:57:05 +01:00
Matthew Hodgson
0834b152fb
char encoding
2016-04-03 12:59:27 +01:00
Matthew Hodgson
8b98a7e8c3
pep8
2016-04-03 12:56:29 +01:00
Matthew Hodgson
eab4d462f8
fix etag typing error. fix timestamp typing error
2016-04-03 02:02:46 +01:00
Matthew Hodgson
c3916462f6
rebase all image URLs
2016-04-03 01:33:12 +01:00
Matthew Hodgson
110780b18b
remove stale todo
2016-04-03 00:48:31 +01:00
Matthew Hodgson
b09e29a03c
Ensure only one download for a given URL is active at a time
2016-04-03 00:47:40 +01:00
Matthew Hodgson
7426c86eb8
add a persistent cache of URL lookups, and fix up the in-memory one to work
2016-04-03 00:31:57 +01:00
Matthew Hodgson
d1b154a10f
support gzip compression, and don't pass through error msgs
2016-04-02 03:06:39 +01:00
Matthew Hodgson
9377157961
how was _respond_default_thumbnail ever meant to work?
2016-04-02 02:31:45 +01:00
Matthew Hodgson
2c838f6459
pass back SVGs as their own thumbnails
2016-04-02 02:30:07 +01:00
Matthew Hodgson
5037ee0d37
handle missing dimensions without crashing
2016-04-02 02:29:57 +01:00
Matthew Hodgson
b26e8604f1
make meta comparisons case insensitive
2016-04-02 01:35:44 +01:00
Matthew Hodgson
5fd07da764
refactor calc_og; spider image URLs; fix xpath; add a (broken) expiringcache; loads of other fixes
2016-04-02 00:35:49 +01:00
Erik Johnston
d76d89323c
Use computed prev event ids
2016-04-01 17:39:32 +01:00
Erik Johnston
aa82cb38e9
Remove state hack from _create_new_client_event
2016-04-01 16:36:54 +01:00
Mark Haines
89e6839a48
Merge pull request #686 from matrix-org/markjh/doc_strings
...
Use google style doc strings.
2016-04-01 16:20:09 +01:00
Erik Johnston
c906f30661
Do checks for memberships before creating events
2016-04-01 16:17:32 +01:00
Mark Haines
2a37467fa1
Use google style doc strings.
...
pycharm supports them so there is no need to use the other format.
Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
2016-04-01 16:12:07 +01:00
Mark Haines
f2b916534b
Merge pull request #684 from matrix-org/markjh/backfill_id_gen
...
Use a stream id generator for backfilled ids
2016-04-01 15:13:14 +01:00
Mark Haines
9bc5b4c663
Assert that the step != 0
2016-04-01 15:08:20 +01:00
Mark Haines
35b5c4ba1b
use google style doc strings
2016-04-01 15:07:01 +01:00
Erik Johnston
3f4eb4c924
Comment
2016-04-01 14:15:27 +01:00
Erik Johnston
8d73cd502b
Add concurrently_execute function
2016-04-01 14:06:00 +01:00
Mark Haines
a2866e2e6a
Rename direction to step, apply checks consistently
2016-04-01 13:50:54 +01:00
Mark Haines
e36bfbab38
Use a stream id generator for backfilled ids
2016-04-01 13:29:05 +01:00
Erik Johnston
35bb465b86
Filter rooms list before chunking
2016-04-01 13:14:53 +01:00
Mark Haines
c42f46ab7d
Merge pull request #682 from matrix-org/markjh/fix_invalidate
...
Fix the invalidation of the names and aliases cache
2016-04-01 10:52:29 +01:00
Mark Haines
7753fc6570
Fix the invalidation of the names and aliases cache
2016-04-01 10:34:51 +01:00
Matthew Hodgson
c60b751694
fix assorted redirect, unicode and screenscraping bugs
2016-04-01 02:17:48 +01:00
Matthew Hodgson
683e564815
handle spidered relative images correctly
2016-03-31 23:52:58 +01:00
Mark Haines
dc4c1579d4
Remove outlier parameter from compute_event_context
...
Use event.internal_metadata.is_outlier instead.
2016-03-31 15:32:24 +01:00
Mark Haines
03e406eefc
Merge pull request #680 from matrix-org/markjh/remove_is_new_state
...
Remove the is_new_state argument to persist event.
2016-03-31 15:14:48 +01:00
Matthew Hodgson
72550c3803
prevent choking on invalid utf-8, and handle image thumbnailing smarter
2016-03-31 15:14:14 +01:00
Mark Haines
5d06929169
Move the check for backfilled outside the for loop
2016-03-31 15:09:09 +01:00
Mark Haines
76503f95ed
Remove the is_new_state argument to persist event.
...
Move the checks for whether an event is new state inside persist
event itself.
This was harder than expected because there wasn't enough information
passed to persist event to correctly handle invites from remote servers
for new rooms.
2016-03-31 15:00:42 +01:00
Erik Johnston
fe95943305
Merge pull request #679 from matrix-org/erikj/member
...
Split out RoomMemberHandler
2016-03-31 14:45:57 +01:00
Matthew Hodgson
bb9a2ca87c
synthesise basig OG metadata from pages lacking it
2016-03-31 14:15:09 +01:00
Erik Johnston
d35780eda0
Split out RoomMemberHandler
2016-03-31 13:08:45 +01:00
Matthew Hodgson
0d3d7de6fc
sync in changes from matrixfederationclient
2016-03-31 12:42:27 +01:00
Mark Haines
62e395f0e3
Merge pull request #676 from matrix-org/markjh/replicate_stateIII
...
Add replication streams for ex outliers and current state resets
2016-03-31 11:20:57 +01:00
Erik Johnston
5260db7663
Line length
2016-03-31 10:49:27 +01:00