Matrix-Mirrors/forked-synapse
1
0
Fork 0
mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00
Code Issues Packages Projects Releases Wiki Activity
11,145 Commits 5 Branches 723 Tags 124 MiB
75d4986a8c
Commit Graph

8113 Commits

Author SHA1 Message Date
Amber Brown
3ef5cd74a6 update to more consistently use seconds in any metrics or logging 2018-05-28 19:39:27 +10:00
Amber Brown
5c40ce3777 invalid syntax :( 2018-05-28 19:16:09 +10:00
Amber Brown
357c74a50f add comment about why unreg 2018-05-28 19:14:41 +10:00
Amber Brown
a2eb5db4a0 update metrics to be in seconds 2018-05-28 19:10:27 +10:00
Amber Brown
754826a830 Merge remote-tracking branch 'origin/develop' into 3218-official-prom 2018-05-28 18:57:23 +10:00
Ruben Barkow
08ea5fe635
add link to thorough instruction how to configure consent 2018-05-25 23:19:55 +02:00
Richard van der Hoff
66bdae986f Fix default for send_server_notice_to_guests 
bool("False") == True...
 2018-05-25 11:42:05 +01:00
Richard van der Hoff
ba1b163590 Avoid sending consent notice to guest users 
we think it makes sense not to send the notices to guest users.
 2018-05-25 11:36:43 +01:00
Richard van der Hoff
41921ac01b
Merge pull request #3287 from matrix-org/rav/allow_leaving_server_notices_room 
Let users leave the server notice room after joining
 2018-05-25 11:15:45 +01:00
Richard van der Hoff
757ed27258 Let users leave the server notice room after joining 
They still can't reject invites, but we let them leave it.
 2018-05-25 11:07:21 +01:00
Adrian Tschira
4ee4450d66 fix recursion error 2018-05-24 21:44:10 +02:00
Amber Brown
9c36c150e7
Merge pull request #3283 from NotAFile/py3-state 
py3-ize state.py
 2018-05-24 14:23:33 -05:00
Amber Brown
cc1349c06a
Merge pull request #3279 from NotAFile/py3-more-iteritems 
more six iteritems
 2018-05-24 14:23:13 -05:00
Adrian Tschira
0e61705661 py3-ize state.py 2018-05-24 20:59:00 +02:00
Adrian Tschira
dd068ca979 remaining isintance fixes 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-24 20:55:08 +02:00
Adrian Tschira
17a70cf6e9 Misc. py3 fixes 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-24 20:20:33 +02:00
Adrian Tschira
6c16a4ec1b more iteritems 2018-05-24 20:19:06 +02:00
Amber Brown
7ea07c7305
Merge pull request #3278 from NotAFile/py3-storage-base 
Py3 storage/_base.py
 2018-05-24 13:08:09 -05:00
Amber Brown
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4 
replace some iteritems with six
 2018-05-24 13:04:07 -05:00
Amber Brown
c4fb15a06c
Merge pull request #3246 from NotAFile/py3-repr-string 
use repr, not str
 2018-05-24 13:00:20 -05:00
Amber Brown
36501068d8
Merge pull request #3247 from NotAFile/py3-misc 
Misc Python3 fixes
 2018-05-24 12:58:37 -05:00
Amber Brown
2aff6eab6d
Merge pull request #3245 from NotAFile/batch-iter 
Add batch_iter to utils
 2018-05-24 12:54:12 -05:00
Adrian Tschira
095292304f Py3 storage/_base.py 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-24 18:24:12 +02:00
David Baker
77a23e2e05 Merge remote-tracking branch 'origin/develop' into dbkr/unbind 2018-05-24 16:20:53 +01:00
David Baker
ecc4b88bd1
Merge pull request #3277 from matrix-org/dbkr/remove_from_user_dir 
Remove users from user directory on deactivate
 2018-05-24 16:12:12 +01:00
Erik Johnston
46345187cc
Merge pull request #3243 from NotAFile/py3-six-3 
Replace some more comparisons with six
 2018-05-24 16:08:57 +01:00
Neil Johnson
037c6db85d Merge branch 'master' into develop 2018-05-24 16:03:44 +01:00
David Baker
7a1af504d7 Remove users from user directory on deactivate 2018-05-24 15:59:58 +01:00
Erik Johnston
f72d5a44d5
Merge pull request #3261 from matrix-org/erikj/pagination_fixes 
Fix federation backfill bugs
 2018-05-24 14:52:03 +01:00
Erik Johnston
68399fc4de
Merge pull request #3267 from matrix-org/erikj/iter_filter 
Use iter* methods for _filter_events_for_server
 2018-05-24 14:44:57 +01:00
Neil Johnson
91d95a1d8e bump version 2018-05-24 14:05:07 +01:00
David Baker
9700d15611 pep8 2018-05-24 11:23:15 +01:00
David Baker
a21a41bad7 comment 2018-05-24 11:19:59 +01:00
David Baker
b3bff53178 Unbind 3pids when they're deleted too 2018-05-24 11:08:05 +01:00
Richard van der Hoff
6abcb5d22d
Merge pull request #3273 from matrix-org/rav/server_notices_avatar_url 
Allow overriding the server_notices user's avatar
 2018-05-24 10:31:43 +01:00
Amber Brown
389dac2c15 pepeightttt 2018-05-23 13:08:59 -05:00
Amber Brown
472a5ec4e2 add back CPU metrics 2018-05-23 13:03:56 -05:00
Amber Brown
e987079037 fixes 2018-05-23 13:03:51 -05:00
Richard van der Hoff
9bf4b2bda3 Allow overriding the server_notices user's avatar 
probably should have done this in the first place, like @turt2live suggested.
 2018-05-23 17:43:30 +01:00
Richard van der Hoff
a0b3946fe2
Merge branch 'release-v0.30.0' into rav/localpart_in_consent_uri 2018-05-23 16:06:03 +01:00
Richard van der Hoff
2f7008d4eb
Merge pull request #3271 from matrix-org/rav/consent_uri_in_messages 
Support for putting %(consent_uri)s in messages
 2018-05-23 16:04:30 +01:00
Richard van der Hoff
dda40fb55d fix typo 2018-05-23 15:30:26 +01:00
Richard van der Hoff
3ff6f50eac Use the localpart in the consent uri 
... because it's shorter.
 2018-05-23 15:28:23 +01:00
Richard van der Hoff
82191b08f6 Support for putting %(consent_uri)s in messages 
Make it possible to put the URI in the error message and the server notice that
get sent by the server
 2018-05-23 15:24:31 +01:00
David Baker
2c7866d664 Hit the 3pid unbind endpoint on deactivation 2018-05-23 14:38:56 +01:00
Richard van der Hoff
321f02d263 Block attempts to send server notices to remote users 2018-05-23 14:30:47 +01:00
Neil Johnson
3601a240aa bump version and changelog 2018-05-23 12:03:23 +01:00
Erik Johnston
5aaa3189d5 s/values/itervalues/ 2018-05-23 10:13:05 +01:00
Erik Johnston
0a4bca4134 Use iter* methods for _filter_events_for_server 2018-05-23 10:04:23 +01:00
Amber Brown
b6063631c3 more cleanup 2018-05-22 17:36:20 -05:00
Amber Brown
53cc2cde1f cleanup 2018-05-22 17:32:57 -05:00
Amber Brown
071206304d cleanup pep8 errors 2018-05-22 16:54:22 -05:00
Amber Brown
85ba83eb51 fixes 2018-05-22 16:28:23 -05:00
Amber Brown
228f1f584e fix the test failures 2018-05-22 15:02:38 -05:00
Erik Johnston
e85b5a0ff7 Use iter* methods 2018-05-22 19:02:48 +01:00
Erik Johnston
586b66b197 Fix that states is a dict of dicts 2018-05-22 19:02:36 +01:00
Erik Johnston
a17e901f4d Remove unused string formatting param 2018-05-22 18:24:32 +01:00
Erik Johnston
5494c1d71e Don't support limitless pagination 
The pagination storage function supported not specifiying a limit on the
number of events returned. This was triggered when using the search or
context API with a limit of zero, which the storage function took to
mean not being limited.
 2018-05-22 18:15:21 +01:00
Neil Johnson
d8cb7225d2 daily user type phone home stats 2018-05-22 18:09:09 +01:00
hera
ad2823ee27 fix synchrotron 2018-05-22 17:47:42 +01:00
Richard van der Hoff
08bfc48abf custom error code for not leaving server notices room 2018-05-22 17:27:27 +01:00
David Baker
0a078026ea comment typo 2018-05-22 17:14:06 +01:00
Amber Brown
8f5a688d42 cleanups, self-registration 2018-05-22 10:56:03 -05:00
Amber Brown
a8990fa2ec Merge remote-tracking branch 'origin/develop' into 3218-official-prom 2018-05-22 10:50:26 -05:00
Erik Johnston
cb2a2ad791 get_domains_from_state returns list of tuples 2018-05-22 16:23:39 +01:00
Richard van der Hoff
08a14b32ae
Merge pull request #3262 from matrix-org/rav/has_already_consented 
Add a 'has_consented' template var to consent forms
 2018-05-22 15:35:10 +01:00
Richard van der Hoff
7b36d06a69 Add a 'has_consented' template var to consent forms 
fixes #3260
 2018-05-22 14:58:34 +01:00
Richard van der Hoff
669400e22f Enable auto-escaping for the consent templates 
... to reduce the risk of somebody introducing an html injection attack...
 2018-05-22 14:58:34 +01:00
Richard van der Hoff
b5b2d5d64b Fix dependency on jinja2 
Delay the import of ConsentResource, so that we can get away without jinja2 if
people don't have the consent resource enabled.

Fixes #3259
 2018-05-22 14:03:45 +01:00
Richard van der Hoff
a5e2941aad Reject attempts to send event before privacy consent is given 
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252) if consent is not yet
given.
 2018-05-22 12:00:47 +01:00
Richard van der Hoff
8aeb529262
Merge pull request #3236 from matrix-org/rav/consent_notice 
Send users a server notice about consent
 2018-05-22 11:56:09 +01:00
Richard van der Hoff
8810685df9 Stub out ServerNoticesSender on the workers 
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
 2018-05-22 11:54:51 +01:00
Richard van der Hoff
d5dca9a04f Move consent config parsing into ConsentConfig 
turns out we need to reuse this, so it's better in the config class.
 2018-05-22 11:54:51 +01:00
Richard van der Hoff
9ea219c514 Send users a server notice about consent 
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
 2018-05-22 11:54:51 +01:00
Richard van der Hoff
d14d7b8fdc Rename 'version' param on user consent config 
we're going to use it for the version we require too.
 2018-05-22 11:54:51 +01:00
Erik Johnston
7948ecf234 Comment 2018-05-22 11:39:43 +01:00
Erik Johnston
020377a550 Fix logcontext resource usage tracking 2018-05-22 11:16:07 +01:00
Erik Johnston
13a8dfba0d
Merge pull request #3252 from matrix-org/erikj/in_flight_requests 
Add in flight request metrics
 2018-05-22 09:54:30 +01:00
Erik Johnston
c435b0b441 Don't store context 2018-05-22 09:34:26 +01:00
Erik Johnston
fb2806b186 Move in_flight_requests_count to be a callback metric 2018-05-22 09:31:53 +01:00
Amber Brown
fcc525b0b7 rest of the changes 2018-05-21 19:48:57 -05:00
Amber Brown
df9f72d9e5 replacing portions 2018-05-21 19:47:37 -05:00
Amber Brown
c60e0d5e02 don't need the resource portion 2018-05-21 17:03:20 -05:00
Amber Brown
02c1d29133 look at the Prometheus metrics instead 2018-05-21 17:02:20 -05:00
Amber Brown
f258deffcb remove old metrics libs 2018-05-21 17:01:15 -05:00
Richard van der Hoff
413482f578
Merge pull request #3255 from matrix-org/rav/fix_transactions 
Stop the transaction cache caching failures
 2018-05-21 17:58:08 +01:00
Richard van der Hoff
6e1cb54a05 Fix logcontext leak in HttpTransactionCache 
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
 2018-05-21 16:58:20 +01:00
Richard van der Hoff
6d6e7288fe Stop the transaction cache caching failures 
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
 2018-05-21 16:49:59 +01:00
Erik Johnston
dfa70adc33 Add in flight request metrics 
This tracks CPU and DB usage while requests are in flight, rather than
when we write the response.
 2018-05-21 16:23:06 +01:00
Adrian Tschira
933bf2dd35 replace some iteritems with six 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-19 17:59:26 +02:00
Adrian Tschira
d9fe2b2d9d Replace some more comparisons with six 
plus a bonus b"" string I missed last time

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-19 17:56:31 +02:00
Adrian Tschira
45b55e23d3 Add batch_iter to utils 
There's a frequent idiom I noticed where an iterable is split up into a
number of chunks/batches. Unfortunately that method does not work with
iterators like dict.keys() in python3. This implementation works with
iterators.

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-19 17:48:30 +02:00
Adrian Tschira
dcc235b47d use stand-in value if maxint is not available 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-19 17:35:44 +02:00
Adrian Tschira
73cbdef5f7 fix py3 intern and remove unnecessary py3 encode 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-19 17:35:31 +02:00
Adrian Tschira
aafb0f6b0d py3-ize url preview 2018-05-19 17:35:20 +02:00
Adrian Tschira
b932b4ea25 use repr, not str 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-19 17:28:42 +02:00
Neil Johnson
644aac5f73 Tighter filtering for user_daily_visits 2018-05-18 17:10:35 +01:00
Neil Johnson
ef466b3a13 fix psql compatability bug 2018-05-18 15:51:21 +01:00
Neil Johnson
2725223f08 Merge branch 'master' into develop 2018-05-18 14:07:50 +01:00
Richard van der Hoff
6d9dc67139
Merge pull request #3232 from matrix-org/rav/server_notices_room 
Infrastructure for a server notices room
 2018-05-18 11:28:04 +01:00
Richard van der Hoff
ed3125b0a1
Merge pull request #3235 from matrix-org/rav/fix_receipts_deferred 
Fix error in handling receipts
 2018-05-18 11:23:11 +01:00
Richard van der Hoff
67af392712
Merge pull request #3233 from matrix-org/rav/remove_dead_code 
Remove unused `update_external_syncs`
 2018-05-18 11:22:43 +01:00
Richard van der Hoff
011e1f4010 Better docstrings 2018-05-18 11:22:12 +01:00
Richard van der Hoff
26305788fe Make sure we reject attempts to invite the notices user 2018-05-18 11:18:39 +01:00
Richard van der Hoff
d10707c810 Replace inline docstrings with "Attributes" in class docstring 2018-05-18 11:00:55 +01:00
Erik Johnston
fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token 
Make purge_history operate on tokens
 2018-05-18 10:35:23 +01:00
Richard van der Hoff
8b1c856d81 Fix error in handling receipts 
Fixes an error which has been happening ever since #2158 (v0.21.0-rc1):

> TypeError: argument of type 'ObservableDeferred' is not iterable

fixes #3234
 2018-05-18 09:15:35 +01:00
Neil Johnson
6958459b50 bump version, change log 2018-05-17 21:35:07 +01:00
Richard van der Hoff
88d3405332 fix missing yield for server_notices_room 2018-05-17 18:33:45 +01:00
Richard van der Hoff
d43d480d86 Remove unused update_external_syncs 
This method isn't used anywhere. Burninate it.
 2018-05-17 18:22:19 +01:00
Richard van der Hoff
fed62e21ad Infrastructure for a server notices room 
Server Notices use a special room which the user can't dismiss. They are
created on demand when some other bit of the code calls send_notice.

(This doesn't actually do much yet becuse we don't call send_notice anywhere)
 2018-05-17 17:58:25 +01:00
Richard van der Hoff
f8a1e76d64
Merge pull request #3225 from matrix-org/rav/move_creation_handler 
Move RoomCreationHandler out of synapse.handlers.Handlers
 2018-05-17 17:56:42 +01:00
Erik Johnston
f7906203f6
Merge pull request #3212 from matrix-org/erikj/epa_stream 
Use stream rather depth ordering for push actions
 2018-05-17 12:01:21 +01:00
Richard van der Hoff
c46367d0d7 Move RoomCreationHandler out of synapse.handlers.Handlers 
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.

Also fix the docstrings on create_room.
 2018-05-17 09:08:42 +01:00
Neil Johnson
85b8acdeb4 Merge branch 'master' into develop 2018-05-16 16:48:10 +01:00
Neil Johnson
3c099219e0 bump version and changelog for 0.29.0 2018-05-16 15:44:59 +01:00
Erik Johnston
680530cc7f Clarify comment 2018-05-16 11:47:29 +01:00
Erik Johnston
43e6e82c4d Comments 2018-05-16 11:13:31 +01:00
Neil Johnson
dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics 
user visit data
 2018-05-16 10:09:24 +00:00
Erik Johnston
c945af8799 Move and rename variable 2018-05-16 10:52:06 +01:00
Neil Johnson
be11a02c4f remove empty line 2018-05-16 10:45:40 +01:00
Neil Johnson
a2204cc9cc remove unused method recurring_user_daily_visit_stats 2018-05-16 09:47:20 +01:00
Neil Johnson
31c2502ca8 style and further contraining query 2018-05-16 09:46:43 +01:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler 
ConsentResource to gather policy consent from users
 2018-05-16 07:19:18 +01:00
Neil Johnson
c92a8aa578 pep8 2018-05-15 17:31:11 +01:00
Neil Johnson
05ac15ae82 Limit query load of generate_user_daily_visits 
The aim is to keep track of when it was last called and only query from that point in time
 2018-05-15 17:01:33 +01:00
Erik Johnston
5f27ed75ad Make purge_history operate on tokens 
As we're soon going to change how topological_ordering works
 2018-05-15 16:23:50 +01:00
Erik Johnston
37dbee6490 Use events_to_purge table rather than token 2018-05-15 16:23:47 +01:00
Richard van der Hoff
47815edcfa ConsentResource to gather policy consent from users 
Hopefully there are enough comments and docs in this that it makes sense on its
own.
 2018-05-15 15:11:59 +01:00
Neil Johnson
aea80a0118 v0.29.0-rc1: bump version and change log 2018-05-14 15:50:57 +01:00
Neil Johnson
f077e97914 instead of inserting user daily visit data at the end of the day, instead insert incrementally through the day 2018-05-14 13:50:58 +01:00
David Baker
8cbbfd16fb
Merge pull request #3201 from matrix-org/dbkr/leave_rooms_on_deactivate 
Part user from rooms on account deactivate
 2018-05-14 11:31:48 +01:00
Neil Johnson
977765bde2 Merge branch 'develop' of https://github.com/matrix-org/synapse into cohort_analytics 2018-05-14 09:31:42 +01:00
Richard van der Hoff
c25d7ba12e
Merge pull request #3208 from matrix-org/rav/more_refactor_request_handler 
Set Server header in SynapseRequest
 2018-05-11 16:07:47 +01:00
Erik Johnston
6406b70aeb Use stream rather depth ordering for push actions 
This simplifies things as it is, but will also allow us to change the
way we traverse topologically without having to update the way push
actions work.
 2018-05-11 15:30:11 +01:00
Richard van der Hoff
23e2dfe940
Merge pull request #3209 from damir-manapov/master 
transaction_id, destination defined twice
 2018-05-11 00:35:13 +01:00
Damir Manapov
db18d854cd transaction_id, destination twice 2018-05-10 22:13:31 +03:00
Richard van der Hoff
318711e139 Set Server header in SynapseRequest 
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).

This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.

Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
 2018-05-10 18:50:27 +01:00
David Baker
6b49628e3b Catch failure to part user from room 2018-05-10 12:23:53 +01:00
David Baker
217bc53c98 Many docstrings 2018-05-10 12:20:40 +01:00
Richard van der Hoff
645cb4bf06 Remove redundant request_handler decorator 
This is needless complexity; we might as well use the wrapper directly.

Also rename wrap_request_handler->wrap_json_request_handler.
 2018-05-10 12:19:53 +01:00
Richard van der Hoff
09f570b935 Factor wrap_request_handler_with_logging out of wrap_request_handler 
... so that it can be used on non-JSON endpoints
 2018-05-10 12:19:52 +01:00
Richard van der Hoff
9589a1925e Remove include_metrics param 
The metrics are now available via the request, so this is redundant and can go
away at last.
 2018-05-10 12:19:52 +01:00
Richard van der Hoff
49e5a613f1 Move outgoing_responses_counter handling to RequestMetrics 
it's much neater there.
 2018-05-10 12:19:52 +01:00
Richard van der Hoff
b8700dd7d0 Bump requests_counter in wrapped_request_handler 
less magic
 2018-05-10 12:19:52 +01:00
Richard van der Hoff
c6f730282c Move RequestMetrics handling into SynapseRequest.processing() 
It fits quite nicely here, and opens the path to getting rid of the
"include_metrics" mess.
 2018-05-10 12:19:51 +01:00
Richard van der Hoff
09b29f9c4a Make RequestMetrics take a raw time rather than a clock 
... which is going to make it easier to move around.
 2018-05-10 12:18:52 +01:00
David Baker
4d298506dd Oops, don't call function passed to run_in_background 2018-05-10 11:57:13 +01:00
Richard van der Hoff
8460e48d06 Move request_id management into SynapseRequest 2018-05-10 11:48:17 +01:00
Richard van der Hoff
18e144fe08 Move RequestsMetrics to its own file 
This is useful in its own right, because server.py is full of stuff; but more
importantly, I want to do some refactoring that will cause a circular reference
as it is.
 2018-05-09 19:55:03 +01:00
Erik Johnston
bfe1f73855
Merge pull request #3199 from matrix-org/erikj/pagination_sync 
Refactor sync APIs to reuse pagination API
 2018-05-09 16:16:56 +01:00
Erik Johnston
5adb75bcba
Merge pull request #3198 from matrix-org/erikj/fixup_return_pagination 
Refactor get_recent_events_for_room return type
 2018-05-09 16:07:14 +01:00
Erik Johnston
d26bec8a43 Add comment to sync as to why code path is split 2018-05-09 15:56:07 +01:00
Erik Johnston
fcf55f2255 Fix returned token is no longer a tuple 2018-05-09 15:43:00 +01:00
Erik Johnston
7ce98804ff Fix up comment 2018-05-09 15:42:39 +01:00
Erik Johnston
cddf91c8b9 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_membership_change 2018-05-09 15:32:07 +01:00
Erik Johnston
9896dab8f6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fixup_return_pagination 2018-05-09 15:31:33 +01:00
Erik Johnston
1e5280b7d0
Merge pull request #3196 from matrix-org/erikj/pagination_return 
Refactor pagination DB API to return concrete type
 2018-05-09 15:27:17 +01:00
Erik Johnston
75552d2148 Update comments 2018-05-09 15:15:38 +01:00
David Baker
294e9a0c9b Prefix internal functions 2018-05-09 15:10:37 +01:00
David Baker
46df23f581 Add the schema file 2018-05-09 15:07:54 +01:00
David Baker
52281e4c54 Indent fail 2018-05-09 15:06:16 +01:00
David Baker
7e8726b8fb Part deactivated users in the background 
One room at a time so we don't take out the whole server with leave
events, and restart at server restart.
 2018-05-09 14:54:28 +01:00
Erik Johnston
c0e08dc45b Remove unused code path from member change DB func 
The function is never called without a from_key, so we can remove all
the handling for that scenario.
 2018-05-09 14:31:32 +01:00
Erik Johnston
0461ef01b7
Merge pull request #3195 from matrix-org/erikj/pagination_refactor 
Refactor recent events func to use pagination func
 2018-05-09 14:12:24 +01:00
Erik Johnston
e2accd7f1d Refactor sync APIs to reuse pagination API 
The sync API often returns events in a topological rather than stream
ordering, e.g. when the user joined the room or on initial sync. When
this happens we can reuse existing pagination storage functions.
 2018-05-09 13:43:39 +01:00
Erik Johnston
e5ab9cd24b Don't unnecessarily require token to be stream token 
This allows calling the `get_recent_event_ids_for_room` function in more
situations.
 2018-05-09 11:58:35 +01:00
Erik Johnston
c4af4c24ca Refactor get_recent_events_for_room return type 
There is no reason to return a tuple of tokens when the last token is
always the token passed as an argument. Changing it makes it consistent
with other storage APIs
 2018-05-09 11:55:34 +01:00
Erik Johnston
05e0a2462c Refactor pagination DB API to return concrete type 
This makes it easier to document what is being returned by the storage
functions and what some functions expect as arguments.
 2018-05-09 11:34:24 +01:00
Erik Johnston
7dd13415db Remove unused from_token param 2018-05-09 10:58:16 +01:00
Erik Johnston
27cf170558 Refactor recent events func to use pagination func 
This also removes a cache that is unlikely to ever get hit.
 2018-05-09 10:55:55 +01:00
Erik Johnston
1aeb5e28a9
Merge pull request #3193 from matrix-org/erikj/pagination_refactor 
Refactor /context to reuse pagination storage functions
 2018-05-09 10:22:38 +01:00
Erik Johnston
23ec51c94c Fix up comments and make function private 2018-05-09 09:55:19 +01:00
Erik Johnston
696f532453 Reuse existing pagination code for context API 2018-05-08 16:20:19 +01:00
Erik Johnston
3e6d306e94 Parse tokens before calling DB function 2018-05-08 16:18:58 +01:00
Erik Johnston
274b8c6025 Only fetch required fields from database 2018-05-08 16:15:25 +01:00
Erik Johnston
06c0d0ed08 Split paginate_room_events storage function 2018-05-08 16:14:26 +01:00
David Baker
bf98fa0864 Part user from rooms on account deactivate 
This implements this very crudely: this probably isn't viable
because parting a user from all their rooms could take a long time,
and if the HS gets restarted in that time the process will be
aborted.
 2018-05-08 15:58:35 +01:00
Richard van der Hoff
678e649b78
Merge pull request #3190 from mujx/notif-token-fix 
notifications: Convert next_token to string according to the spec
 2018-05-08 13:54:47 +01:00
Konstantinos Sideris
88868b2839 notifications: Convert next_token to string according to the spec 
Currently the parameter is serialized as an integer.

Signed-off-by: Konstantinos Sideris <sideris.konstantin@gmail.com>
 2018-05-05 12:55:02 +03:00
Erik Johnston
6d8ec3462d Note that label values can be anything 2018-05-03 16:25:05 +01:00
Erik Johnston
95b6912045 Fix metrics that have integer value labels 2018-05-03 15:51:04 +01:00
Richard van der Hoff
093d8c415a Merge remote-tracking branch 'origin/develop' into rav/warn_on_logcontext_fail 2018-05-03 14:59:29 +01:00
Richard van der Hoff
0ba609dc6f
Merge pull request #3183 from matrix-org/rav/moar_logcontext_leaks 
Fix logcontext leaks in rate limiter
 2018-05-03 14:58:13 +01:00
Richard van der Hoff
2117f84323
Merge pull request #3182 from Half-Shot/hs/fix-twisted-shutdown 
Fix 'Unhandled Error' logs with Twisted 18.4
 2018-05-03 12:40:11 +01:00
Richard van der Hoff
a7fe62f0cb Fix logcontext leaks in rate limiter 2018-05-03 12:31:59 +01:00
Will Hunt
2e7a94c36b Don't abortConnection() if the transport connection has already closed. 2018-05-03 12:31:47 +01:00
Richard van der Hoff
a2aaa9cb3c
Merge pull request #3178 from matrix-org/rav/fix_request_timeouts 
fix http request timeout code
 2018-05-03 11:33:26 +01:00
Erik Johnston
0a3b51c420
Merge pull request #3141 from matrix-org/erikj/fixup_state 
Refactor event storage to prepare for changes in state calculations
 2018-05-03 10:39:20 +01:00
Erik Johnston
31c7c29d43 Fix up grammar 2018-05-03 10:38:58 +01:00
Richard van der Hoff
902673e356
Merge pull request #3161 from NotAFile/remove-v1auth 
Make Client-Server API return 403 for invalid token
 2018-05-03 10:10:57 +01:00
Erik Johnston
53a5fdf312
Merge pull request #3175 from matrix-org/erikj/escape_metric_values 
Escape label values in prometheus metrics
 2018-05-03 10:01:04 +01:00
Richard van der Hoff
1dfd650348 add missing param to cancelled_to_request_timed_out_error 
This gets two arguments, not one.
 2018-05-02 22:42:36 +01:00
Erik Johnston
a41117c63b Make _escape_character take MatchObject 2018-05-02 17:27:27 +01:00
Erik Johnston
32015e1109 Escape label values in prometheus metrics 2018-05-02 16:52:42 +01:00
Richard van der Hoff
3a42aed9a1
Merge pull request #3170 from matrix-org/rav/more_logcontext_leaks 
Fix a class of logcontext leaks
 2018-05-02 16:45:51 +01:00
Richard van der Hoff
415c6b672e Merge branch 'develop' into rav/more_logcontext_leaks 2018-05-02 16:16:01 +01:00
Richard van der Hoff
be31adb036 Fix logcontext leak in media repo 
Make FileResponder.write_to_consumer uphold the logcontext contract
 2018-05-02 16:14:50 +01:00
Richard van der Hoff
f22e7cda2c Fix a class of logcontext leaks 
So, it turns out that if you have a first `Deferred` `D1`, you can add a
callback which returns another `Deferred` `D2`, and `D2` must then complete
before any further callbacks on `D1` will execute (and later callbacks on `D1`
get the *result* of `D2` rather than `D2` itself).

So, `D1` might have `called=True` (as in, it has started running its
callbacks), but any new callbacks added to `D1` won't get run until `D2`
completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield`
will 'block'.

In conclusion: some of our assumptions in `logcontext` were invalid. We need to
make sure that we don't optimise out the logcontext juggling when this
situation happens. Fortunately, it is easy to detect by checking `D1.paused`.
 2018-05-02 11:58:00 +01:00
Richard van der Hoff
a8d8bf92e0
Merge pull request #3168 from matrix-org/rav/fix_logformatter 
Fix incorrect reference to StringIO
 2018-05-02 10:03:36 +01:00