Erik Johnston
4c3eb14d68
Increase batching of sent transaction inserts
...
This should further reduce the number of individual inserts,
transactions and updates that are required for keeping sent_transactions
up to date.
2016-10-31 16:07:45 +00:00
Mark Haines
82e71a259c
Bump changelog and version
2016-10-28 11:16:05 +01:00
Mark Haines
490b97d3e7
Merge branch 'develop' into release-v0.18.2
2016-10-28 11:12:31 +01:00
Paul "LeoNerd" Evans
1cc22da600
Set up the process collector during metrics __init__; that way all split-process workers have it
2016-10-27 18:09:34 +01:00
Paul "LeoNerd" Evans
aac13b1f9a
Pass the Metrics group into the process collector instead of having it find its own one; this avoids it needing to import from synapse.metrics
2016-10-27 18:08:15 +01:00
Paul "LeoNerd" Evans
ccc1a3d54d
Allow creation of a 'subspace' within a Metrics object, returning another one
2016-10-27 18:07:34 +01:00
Erik Johnston
665e53524e
Bump changelog and version
2016-10-27 14:52:47 +01:00
Erik Johnston
a9111786f9
Use most recently added binding, not most recently seen user.
2016-10-27 14:32:45 +01:00
Erik Johnston
1fc1bc2a51
Fix user_threepids schema delta
...
The delta `37/user_threepids.sql` aimed to update all the email
addresses to be lower case, however duplicate emails may exist in the
table already.
This commit adds a step where the delta moves the duplicate emails to a
new `medium` `email_old`. Only the most recently used account keeps the
binding intact. We move rather than delete so that we retain some record
of which emails were associated with which account.
2016-10-27 14:14:44 +01:00
Erik Johnston
ab731d8f8e
Bump changelog and version
2016-10-27 11:07:02 +01:00
Mark Haines
177f104432
Merge pull request #1098 from matrix-org/markjh/bearer_token
...
Allow clients to supply access_tokens as headers
2016-10-25 17:33:15 +01:00
Erik Johnston
22fbf86e4f
Merge branch 'release-v0.18.2' of github.com:matrix-org/synapse into develop
2016-10-25 15:09:46 +01:00
Erik Johnston
855645c719
Bump version and changelog
2016-10-25 11:16:16 +01:00
Erik Johnston
2ef617bc06
Fix infinite typing bug
...
There's a bug somewhere that causes typing notifications to not be timed
out properly. By adding a paranoia timer and using correct inequalities
notifications should stop being stuck, even if it the root cause hasn't
been fixed.
2016-10-24 15:51:22 +01:00
Erik Johnston
b6800a8ecd
Actually use the new function
2016-10-24 13:39:49 +01:00
Erik Johnston
d04e2ff3a4
Fix incredubly slow back pagination query
...
If a client didn't specify a from token when paginating backwards
synapse would attempt to query the (global) maximum topological token.
This a) doesn't make much sense since they're room specific and b) there
are no indices that lets postgres do this efficiently.
2016-10-24 13:35:51 +01:00
Paul Evans
a842fed418
Merge pull request #1177 from matrix-org/paul/standard-metric-names
...
Standardise prometheus metrics
2016-10-21 13:06:19 +01:00
Luke Barnard
e01a1bc92d
Merge pull request #1175 from matrix-org/luke/feature-configurable-as-rate-limiting
...
Allow Configurable Rate Limiting Per AS
2016-10-20 16:21:10 +01:00
Luke Barnard
6fdd31915b
Style
2016-10-20 13:53:15 +01:00
Luke Barnard
07caa749bf
Closing brace on following line
2016-10-20 12:07:16 +01:00
Luke Barnard
f09db236b1
as_user->app_service, less redundant comments, better positioned comments
2016-10-20 12:04:54 +01:00
Luke Barnard
8bfd01f619
flake8
2016-10-20 11:52:46 +01:00
Luke Barnard
1b17d1a106
Use real AS object by passing it through the requester
...
This means synapse does not have to check if the AS is interested, but instead it effectively re-uses what it already knew about the requesting user
2016-10-20 11:43:05 +01:00
Paul "LeoNerd" Evans
b01aaadd48
Split callback metric lambda functions down onto their own lines to keep line lengths under 90
2016-10-19 18:26:13 +01:00
Paul "LeoNerd" Evans
1071c7d963
Adjust code for <100 char line limit
2016-10-19 18:23:25 +01:00
Paul "LeoNerd" Evans
6453d03edd
Cut the raw /proc/self/stat line up into named fields at collection time
2016-10-19 18:21:40 +01:00
Paul "LeoNerd" Evans
3ae48a1f99
Move the process metrics collector code into its own file
2016-10-19 18:10:24 +01:00
Paul "LeoNerd" Evans
4cedd53224
A slightly neater way to manage metric collector functions
2016-10-19 17:54:09 +01:00
Paul "LeoNerd" Evans
5663137e03
appease pep8
2016-10-19 16:09:42 +01:00
Paul "LeoNerd" Evans
b202531be6
Also guard /proc/self/fds-related code with a suitable psuedoconstant
2016-10-19 15:37:41 +01:00
Paul "LeoNerd" Evans
1b179455fc
Guard registration of process-wide metrics by existence of the requisite /proc entries
2016-10-19 15:34:38 +01:00
Paul "LeoNerd" Evans
981f852d54
Add standard process_start_time_seconds metric
2016-10-19 15:05:22 +01:00
Paul "LeoNerd" Evans
def63649df
Add standard process_max_fds metric
2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
06f1ad1625
Add standard process_open_fds metric
2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
95fc70216d
Add standard process_*_memory_bytes metrics
2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
9b0316c75a
Use /proc/self/stat to generate the new process_cpu_*_seconds_total metrics
2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
03c2720940
Export CPU usage metrics also under prometheus-standard metric name
2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
b21b9dbc37
Callback metric values might not just be integers - allow floats
2016-10-19 15:05:15 +01:00
Erik Johnston
78c083f159
Merge pull request #1164 from pik/error-codes
...
Clarify Error codes for GET /filter/
2016-10-19 14:26:17 +01:00
Erik Johnston
3aa8925091
Merge pull request #1176 from matrix-org/erikj/eager_ratelimit_check
...
Check whether to ratelimit sooner to avoid work
2016-10-19 14:25:52 +01:00
Erik Johnston
f2f74ffce6
Comment
2016-10-19 14:21:28 +01:00
David Baker
0108ed8ae6
Latest delta is now 37
2016-10-19 11:40:35 +01:00
David Baker
a7f48320b1
Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive
2016-10-19 11:28:56 +01:00
David Baker
df2a616c7b
Convert emails to lowercase when storing
...
And db migration sql to convert existing addresses.
2016-10-19 11:13:55 +01:00
Erik Johnston
550308c7a1
Check whether to ratelimit sooner to avoid work
2016-10-19 10:45:24 +01:00
pik
e8b1d2a452
Refactor test_filter to use real DataStore
...
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
Luke Barnard
5b54d51d1e
Allow Configurable Rate Limiting Per AS
...
This adds a flag loaded from the registration file of an AS that will determine whether or not its users are rate limited (by ratelimit in _base.py). Needed for IRC bridge reasons - see https://github.com/matrix-org/matrix-appservice-irc/issues/240 .
2016-10-18 17:04:09 +01:00
Erik Johnston
8ca05b5755
Fix push notifications for a single unread message
2016-10-18 10:57:33 +01:00
Erik Johnston
f0ca088280
Reduce redundant database work in email pusher
...
Update the last stream ordering if the
`get_unread_push_actions_for_user_in_range_for_email` returns no new
push actions. This reduces the range that it needs to check next
iteration.
2016-10-18 10:52:47 +01:00
Erik Johnston
b95dbdcba4
Bump version and changelog
2016-10-17 15:59:12 +01:00
Erik Johnston
6942d68247
Bump schema version
2016-10-17 11:17:45 +01:00
Erik Johnston
b59994b454
Remove TODO
2016-10-17 11:17:02 +01:00
Erik Johnston
816988baaa
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_auth
2016-10-17 11:10:37 +01:00
Erik Johnston
2869a29fd7
Drop some unused indices
2016-10-17 11:08:19 +01:00
Erik Johnston
a68ade6ed3
Merge pull request #1162 from larroy/master
...
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-14 21:42:55 +01:00
David Baker
29c5922021
Revert part of 6207399
...
older sqlite doesn't support indexes on expressions, lets just
store things lowercase in the db
2016-10-14 16:20:24 +01:00
Alexander Maznev
d9350b0db8
Error codes for filters
...
* add tests
Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
2016-10-14 10:18:28 -05:00
David Baker
bcb1245a2d
Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive
2016-10-14 15:10:38 +01:00
David Baker
62073992c5
Make password reset email field case insensitive
2016-10-14 13:56:53 +01:00
Erik Johnston
6f7540ada4
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fix_email_notifs
2016-10-14 10:22:43 +01:00
Erik Johnston
1d107d8484
Fix email push notifs being dropped
...
A lot of email push notifications were failing to be sent due to an
exception being thrown along one of the (many) paths. This was due to a
change where we moved from pulling out the full state for each room, but
rather pulled out the event ids for the state and separately loaded the
full events when needed.
2016-10-13 13:40:38 +01:00
Richard van der Hoff
f7aed3d7a2
Merge pull request #1168 from matrix-org/rav/ui_auth_on_device_delete
...
User-interactive auth on delete device
2016-10-13 09:38:41 +01:00
Richard van der Hoff
9009143fb9
Handle delete device requests with no body
...
We should probably return a 401 rather than a 400 for existing clients that
don't know they have to do the UIA dance to delete a device.
2016-10-12 18:47:28 +01:00
Richard van der Hoff
fbd3866bc6
User-interactive auth on delete device
2016-10-12 16:16:31 +01:00
Mark Haines
9e18e0b1cb
Merge pull request #1167 from matrix-org/markjh/fingerprints
...
Add config option for adding additional TLS fingerprints
2016-10-12 15:27:44 +01:00
Mark Haines
c61ddeedac
Explain how long the servers can cache the TLS fingerprints for
2016-10-12 14:48:24 +01:00
Mark Haines
0af6213019
Improve comment formatting
2016-10-12 14:45:13 +01:00
Erik Johnston
35e2cc8b52
Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth
...
Implement pluggable password auth
2016-10-12 11:41:20 +01:00
Mark Haines
6e9f3ab415
Add config option for adding additional TLS fingerprints
2016-10-11 19:14:46 +01:00
Erik Johnston
3061dac53e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/replication_noop
2016-10-11 14:08:29 +01:00
Erik Johnston
668f91d707
Fix check of wrong variable
2016-10-11 13:57:22 +01:00
Richard van der Hoff
0061e8744f
Merge pull request #1166 from matrix-org/rav/grandfather_broken_riot_signup
...
Work around email-spamming Riot bug
2016-10-11 11:58:58 +01:00
Richard van der Hoff
fa74fcf512
Work around email-spamming Riot bug
...
5d9546f9https://github.com/matrix-org/matrix-doc/pull/397 ).
That change exposed a bug in Riot which would make it request a new validation
token (and send a new email) each time it got a 401 with a `flows` parameter
(see https://github.com/vector-im/vector-web/issues/2447 and the fix at
https://github.com/matrix-org/matrix-react-sdk/pull/510 ).
To preserve compatibility with broken versions of Riot, grandfather in the old
behaviour for the email validation stage.
2016-10-11 11:34:40 +01:00
Erik Johnston
a2f2516199
Merge pull request #1157 from Rugvip/nolimit
...
Remove rate limiting from app service senders and fix get_or_create_user requester
2016-10-11 11:20:54 +01:00
Erik Johnston
a940618c94
Merge pull request #1150 from Rugvip/state_key
...
api/auth: fix for not being allowed to set your own state_key
2016-10-11 11:19:55 +01:00
Pedro Larroy
c57f871184
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-08 23:55:20 +02:00
Richard van der Hoff
8681aff4f1
Merge pull request #1160 from matrix-org/rav/401_on_password_fail
...
Interactive Auth: Return 401 from for incorrect password
2016-10-07 10:57:43 +01:00
Richard van der Hoff
5d9546f9f4
Interactive Auth: Return 401 from for incorrect password
...
This requires a bit of fettling, because I want to return a helpful error
message too but we don't want to distinguish between unknown user and invalid
password. To avoid hardcoding the error message into 15 places in the code,
I've had to refactor a few methods to return None instead of throwing.
Fixes https://matrix.org/jira/browse/SYN-744
2016-10-07 00:00:00 +01:00
Patrik Oldsberg
7b5546d077
rest/client/v1/register: use the correct requester in createUser
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 22:12:32 +02:00
Richard van der Hoff
5d34e32d42
Merge pull request #1159 from matrix-org/rav/uia_fallback_postmessage
...
window.postmessage for Interactive Auth fallback
2016-10-06 19:56:43 +01:00
Richard van der Hoff
f382117852
window.postmessage for Interactive Auth fallback
...
If you're a webapp running the fallback in an iframe, you can't set set a
window.onAuthDone function. Let's post a message back to window.opener instead.
2016-10-06 18:16:59 +01:00
Patrik Oldsberg
3de7c8a4d0
handlers/profile: added admin override for set_displayname and set_avatar_url
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 15:24:59 +02:00
Patrik Oldsberg
2ff2d36b80
handers: do not ratelimit app service senders
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 15:24:59 +02:00
Patrik Oldsberg
9bfc617791
storage/appservice: make appservice methods only relying on the cache synchronous
2016-10-06 15:24:59 +02:00
Erik Johnston
4285be791d
Bump changelog and version
2016-10-05 14:40:38 +01:00
Erik Johnston
850b103b36
Implement pluggable password auth
...
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
2016-10-03 10:36:40 +01:00
Patrik Oldsberg
24a70e19c7
api/auth: fix for not being allowed to set your own state_key
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-09-30 13:08:25 +02:00
Erik Johnston
04aa2f2863
Bump version and changelog
2016-09-30 10:34:57 +01:00
Martin Weinelt
3027ea22b0
Restructure ldap authentication
...
- properly parse return values of ldap bind() calls
- externalize authentication methods
- change control flow to be more error-resilient
- unbind ldap connections in many places
- improve log messages and loglevels
2016-09-29 15:30:08 +01:00
Erik Johnston
5875a65253
Merge pull request #1145 from matrix-org/erikj/fix_reindex
...
Fix background reindex of origin_server_ts
2016-09-29 13:53:48 +01:00
Erik Johnston
9040c9ffa1
Fix background reindex of origin_server_ts
...
The storage function `_get_events_txn` was removed everywhere except
from this background reindex. The function was removed due to it being
(almost) completely unused while also being large and complex.
Therefore, instead of resurrecting `_get_events_txn` we manually
reimplement the bits that are needed directly.
2016-09-27 11:23:49 +01:00
Erik Johnston
4974147aa3
Remove duplication
2016-09-27 09:27:54 +01:00
Erik Johnston
13122e5e24
Remove unused variable
2016-09-27 09:21:51 +01:00
Erik Johnston
cf3e1cc200
Fix perf of fetching state in SQLite
2016-09-26 17:16:24 +01:00
Erik Johnston
a38d46249e
Merge pull request #1140 from matrix-org/erikj/typing_fed_timeout
...
Time out typing over federation
2016-09-26 11:24:14 +01:00
Erik Johnston
748d8fdc7b
Reduce DB hits for replication
...
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
2016-09-23 16:49:21 +01:00
Erik Johnston
655891d179
Move FEDERATION_PING_INTERVAL timer. Update log line
2016-09-23 15:43:34 +01:00
Erik Johnston
4225a97f4e
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-09-23 15:36:59 +01:00
Erik Johnston
22578545a0
Time out typing over federation
2016-09-23 14:00:52 +01:00
Erik Johnston
667fcd54e8
Merge pull request #1136 from matrix-org/erikj/fix_signed_3pid
...
Allow invites via 3pid to bypass sender sig check
2016-09-22 13:41:49 +01:00
