Amber Brown
a61738b316
Remove run_on_reactor ( #3395 )
2018-06-14 18:27:37 +10:00
Amber Brown
0fde1896cd
Merge pull request #3389 from turt2live/travis/name_metrics
...
Use the correct flag (enable_metrics) when warning about an incorrect metrics setup
2018-06-13 23:50:10 +10:00
Michael Telatynski
94700e55fa
if inviter_display_name == ""||None then default to inviter MXID
...
to prevent email invite from "None"
2018-06-13 10:31:01 +01:00
Travis Ralston
45768d1640
Use the RegistryProxy for appservices too
...
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-06-12 12:55:48 -06:00
Travis Ralston
12285a1a76
The flag is named enable_metrics, not collect_metrics
...
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-06-12 12:51:31 -06:00
Richard van der Hoff
b6faef2ad7
Filter out erased messages
...
Redact any messges sent by erased users.
2018-06-12 09:53:18 +01:00
Richard van der Hoff
f1023ebf4b
mark accounts as erased when requested
2018-06-12 09:53:18 +01:00
Richard van der Hoff
3ff8a619f5
UserErasureStore
...
to store which users have been erased
2018-06-12 09:52:22 +01:00
Richard van der Hoff
9fc5b74b24
simplify get_persisted_pdu
...
it doesn't make much sense to use get_persisted_pdu on the receive path: just
get the event straight from the store.
2018-06-12 09:51:31 +01:00
Richard van der Hoff
bd348f0af6
remove dead filter_events_for_clients
...
This is only used by filter_events_for_client, so we can simplify the whole
thing by just doing one user at a time, and removing a dead storage function to
boot.
2018-06-12 09:51:31 +01:00
Richard van der Hoff
eb32b2ca20
Optimise state_group_cache update
...
(1) matrix-org-hotfixes has removed the intern calls; let's do the same here.
(2) remove redundant iteritems() so we can used an optimised db update.
2018-06-11 22:56:11 +01:00
David Baker
187a546bff
Merge pull request #3276 from matrix-org/dbkr/unbind
...
Remove email addresses / phone numbers from ID servers when they're removed from synapse
2018-06-11 16:02:00 +01:00
Matthew Hodgson
d6cc369205
fix idiotic typo in state res
2018-06-11 14:43:55 +01:00
Matthew Hodgson
c96d882a02
Merge branch 'develop' into matthew/filter_members
2018-06-10 12:26:14 +03:00
Neil Johnson
ed5a0780a4
Merge branch 'master' into develop
2018-06-08 15:47:11 +01:00
Neil Johnson
1032393dfb
Changes in synapse v0.31.1 (2018-06-08)
...
=======================================
v0.31.1 fixes a security bug in the ``get_missing_events`` federation API
where event visibility rules were not applied correctly.
We are not aware of it being actively exploited but please upgrade asap.
Bug Fixes:
* Fix event filtering in get_missing_events handler (PR #3371 )
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEETQ1YthIGLQRddG54CTxDAAxPS/QFAlsalP8ACgkQCTxDAAxP
S/RADw/+NeDu0LjVpS5Uc4ElHgRBuFSm6l2i4z8rZBBlKSYnuq0Em4WMvLloi/JF
iAvTOYE7OjmF+gNvmsdH1N7hc1lKdQ2gAlpvaQR/5Qz9NtOVmM3WPZxS7n5jZHvD
hVSxeO9+GQOwK7rJorqrrsnWHQt0OkLHV6WThFdrgZb1JjWCUDTvw+Hei2uMX2aq
y2mkMG4TLStHwMvL2qw0h+hFtXywXI796qJR73ZxbEn24YD+kOXeEVkIFi2LT0Pj
cgkg7WWT32JD43/ioumLupZuhCmpRyxn4fi5gIKpXe5kiLsxOdApzNQSwmoJ+WA8
7zlrWY+0QDN4pbA5ESLitWSWAT50Ul//uM4nwmM4xEBPHdljXvKyHPsaSCeKLvT9
RT8pc41TQAqSshlXF8zgIAtStnF3oGel3EBBl1mmM9Un1ULnBFwWDZhlIm+ZZGhJ
MWoAWNG7j8AQuy0BTUAUr76x7t+/cdSqDuyVl1GO1tbDh0DUWoHZGXCUKrAXnn2T
SbiFigwOLvEADbvkW7L9Je9CVOi2V5Pg/32X9O8YMEiSz+j5PQEiGefyVh/I/QvV
Ha/atRpZF2OZ+XUOO5DLZMP/XCXpVgvHuskzfU6LvvVQCXgExuJhRD1PrRGeBaWr
zjJW+rmY+VeredqX7QkTB3XOGLMLGJfx2FeSMb+j0w3a7/iFj+Q=
=J80S
-----END PGP SIGNATURE-----
Merge tag 'v0.31.1'
Changes in synapse v0.31.1 (2018-06-08)
=======================================
v0.31.1 fixes a security bug in the ``get_missing_events`` federation API
where event visibility rules were not applied correctly.
We are not aware of it being actively exploited but please upgrade asap.
Bug Fixes:
* Fix event filtering in get_missing_events handler (PR #3371 )
2018-06-08 15:46:18 +01:00
Neil Johnson
0eb4722932
changelog a bump version
2018-06-08 15:21:46 +01:00
Richard van der Hoff
c6b1441c52
Fix event filtering in get_missing_events handler
2018-06-08 14:15:31 +01:00
David Baker
0e505b1913
Merge pull request #3372 from matrix-org/rav/better_verification_logging
...
Try to log more helpful info when a sig verification fails
2018-06-08 13:32:02 +01:00
Richard van der Hoff
e82db24a0e
Try to log more helpful info when a sig verification fails
...
Firstly, don't swallow the reason for the failure
Secondly, don't assume all exceptions are verification failures
Thirdly, log a bit of info about the key being used if debug is enabled
2018-06-08 12:13:08 +01:00
Richard van der Hoff
0834b49c6a
Fix event filtering in get_missing_events handler
2018-06-08 11:34:46 +01:00
Richard van der Hoff
8503dd0047
Remove event re-signing hacks
...
These "temporary fixes" have been here three and a half years, and I can't find
any events in the matrix.org database where the calculated signature differs
from what's in the db. It's time for them to go away.
2018-06-07 16:08:29 +01:00
Will Hunt
13d211edc1
Merge pull request #3344 from Half-Shot/hs/as-metrics
...
Add metrics to track appservice transactions
2018-06-07 11:37:12 +01:00
Richard van der Hoff
1152f495a0
Merge pull request #3363 from matrix-org/rav/fix_purge
...
Fix event-purge-by-ts admin API
2018-06-07 11:34:46 +01:00
Richard van der Hoff
e2acf536d4
Merge pull request #3355 from matrix-org/rav/fix_federation_backfill
...
Fix federation backfill from sqlite servers
2018-06-07 10:18:01 +01:00
Amber Brown
0160f6601f
Merge pull request #3356 from matrix-org/rav/add_missing_attr_dep
...
Make sure that attr is installed
2018-06-07 16:33:30 +10:00
Richard van der Hoff
f4caf3f83d
fix log
2018-06-07 00:26:38 +01:00
Richard van der Hoff
0546715c18
Fix event-purge-by-ts admin API
...
This got completely broken in 0.30.
Fixes #3300 .
2018-06-07 00:15:49 +01:00
Richard van der Hoff
57e3f923d2
Add missing dependency on attr
...
We've rcently added a dep on `attr`. I don't know why the CI didn't pick this
up, but we should make it explicit anyway.
2018-06-06 17:12:41 +01:00
Richard van der Hoff
d3a8c9c55e
Fix sql error in _get_state_groups_from_groups
...
If this was called with a `(type, None)` entry in types (which is supposed to
return all state of type `type`), it would explode with a sql error.
2018-06-06 14:19:01 +01:00
Neil Johnson
fef6c2cdcc
Merge branch 'master' into develop
2018-06-06 12:28:15 +01:00
Neil Johnson
752b7b32ed
Changes in synapse v0.31.0 (2018-06-06)
...
======================================
Most notable change from v0.30.0 is to switch to python prometheus library to improve system
stats reporting. WARNING this changes a number of prometheus metrics in a
backwards-incompatible manner. For more details, see
`docs/metrics-howto.rst <docs/metrics-howto.rst#removal-of-deprecated-metrics--time-based-counters-becoming-histograms-in-0310>`_.
Bug Fixes:
* Fix metric documentation tables (PR #3341 )
* Fix LaterGuage error handling (694968fb7e7fd2#3256 , #3274 )
* Let users leave the server notice room after joining (PR #3287 )
Changes:
* daily user type phone home stats (PR #3264 )
* Use iter* methods for _filter_events_for_server (PR #3267 )
* Docs on consent bits (PR #3268 )
* Remove users from user directory on deactivate (PR #3277 )
* Avoid sending consent notice to guest users (PR #3288 )
* disable CPUMetrics if no /proc/self/stat (PR #3299 )
* Add local and loopback IPv6 addresses to url_preview_ip_range_blacklist (PR #3312 ) Thanks to @thegcat!
* Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy (PR #3307 )
* Add private IPv6 addresses to example config for url preview blacklist (PR #3317 ) Thanks to @thegcat!
* Reduce stuck read-receipts: ignore depth when updating (PR #3318 )
* Put python's logs into Trial when running unit tests (PR #3319 )
Changes, python 3 migration:
* Replace some more comparisons with six (PR #3243 ) Thanks to @NotAFile!
* replace some iteritems with six (PR #3244 ) Thanks to @NotAFile!
* Add batch_iter to utils (PR #3245 ) Thanks to @NotAFile!
* use repr, not str (PR #3246 ) Thanks to @NotAFile!
* Misc Python3 fixes (PR #3247 ) Thanks to @NotAFile!
* Py3 storage/_base.py (PR #3278 ) Thanks to @NotAFile!
* more six iteritems (PR #3279 ) Thanks to @NotAFile!
* More Misc. py3 fixes (PR #3280 ) Thanks to @NotAFile!
* remaining isintance fixes (PR #3281 ) Thanks to @NotAFile!
* py3-ize state.py (PR #3283 ) Thanks to @NotAFile!
* extend tox testing for py3 to avoid regressions (PR #3302 ) Thanks to @krombel!
* use memoryview in py3 (PR #3303 ) Thanks to @NotAFile!
Bugs:
* Fix federation backfill bugs (PR #3261 )
* federation: fix LaterGauge usage (PR #3328 ) Thanks to @intelfx!
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEETQ1YthIGLQRddG54CTxDAAxPS/QFAlsXxJAACgkQCTxDAAxP
S/SJTg//Wtr+Qop9LJh2/leAYXpyqW6P7Ftak0w3aJ3KL3+tYg32yYNoRADCqbp3
LkrHu8MwbZagHjRUyEWNfDk4jbfq5fwh0JVGmYuUKhG9aF0HYyytKkbW79YzuhdQ
dfHj9x0xSBOUvgt/husloZSDy0VHC6uyQSAFgFDyHS2y7RPAiGstqLGByv0ciZOk
pO7TdjkUQcx4Ps7Wgip31NuHy3GY2int6f540pUXoZHLXs7RkfqS2cpF9Z/sTXJ4
xDLiY7uYNsTcCblwqaiijY5c90xwRB2vLs5CJdKFgyB6PNgg/2wHJqP/WHHEj+F8
BoSm3Ts7NXQf23pP9CXICe7vXX3J+ruOnC7FOSRobr6KGjn6DUrIZxo1ZepTwpp9
DIq+1eOFKKjwLQM3Jdi8WBCP63LhYXrTZxreke3jpwdcD7oIO9v6/e9J1gU5xHWa
Izg+YnWn1JLfq/X8T7YTZddUXGGPkH5i6LZKKkyY8u7LkJ4WR7syuAceUzkOOIAq
UWO0uEV7IiLnZzZGTtNIlEqtuklmVQTm6bvAgTPabai2JQyngFpH5M/5mPpVSiLV
QRLwaM56c+5GGZJWef8vxdGeYn+8rFI/UUniJ7358kLJF2IHsxlZu8J0ZZO2HWI2
ze5Kz0AWRzXLhWzq62Qb2dsiGySrZ7hng1tDxLak2IiusY+9SjM=
=Mz9U
-----END PGP SIGNATURE-----
Merge tag 'v0.31.0'
Changes in synapse v0.31.0 (2018-06-06)
======================================
Most notable change from v0.30.0 is to switch to python prometheus library to improve system
stats reporting. WARNING this changes a number of prometheus metrics in a
backwards-incompatible manner. For more details, see
`docs/metrics-howto.rst <docs/metrics-howto.rst#removal-of-deprecated-metrics--time-based-counters-becoming-histograms-in-0310>`_.
Bug Fixes:
* Fix metric documentation tables (PR #3341 )
* Fix LaterGuage error handling (694968fb7e7fd2#3256 , #3274 )
* Let users leave the server notice room after joining (PR #3287 )
Changes:
* daily user type phone home stats (PR #3264 )
* Use iter* methods for _filter_events_for_server (PR #3267 )
* Docs on consent bits (PR #3268 )
* Remove users from user directory on deactivate (PR #3277 )
* Avoid sending consent notice to guest users (PR #3288 )
* disable CPUMetrics if no /proc/self/stat (PR #3299 )
* Add local and loopback IPv6 addresses to url_preview_ip_range_blacklist (PR #3312 ) Thanks to @thegcat!
* Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy (PR #3307 )
* Add private IPv6 addresses to example config for url preview blacklist (PR #3317 ) Thanks to @thegcat!
* Reduce stuck read-receipts: ignore depth when updating (PR #3318 )
* Put python's logs into Trial when running unit tests (PR #3319 )
Changes, python 3 migration:
* Replace some more comparisons with six (PR #3243 ) Thanks to @NotAFile!
* replace some iteritems with six (PR #3244 ) Thanks to @NotAFile!
* Add batch_iter to utils (PR #3245 ) Thanks to @NotAFile!
* use repr, not str (PR #3246 ) Thanks to @NotAFile!
* Misc Python3 fixes (PR #3247 ) Thanks to @NotAFile!
* Py3 storage/_base.py (PR #3278 ) Thanks to @NotAFile!
* more six iteritems (PR #3279 ) Thanks to @NotAFile!
* More Misc. py3 fixes (PR #3280 ) Thanks to @NotAFile!
* remaining isintance fixes (PR #3281 ) Thanks to @NotAFile!
* py3-ize state.py (PR #3283 ) Thanks to @NotAFile!
* extend tox testing for py3 to avoid regressions (PR #3302 ) Thanks to @krombel!
* use memoryview in py3 (PR #3303 ) Thanks to @NotAFile!
Bugs:
* Fix federation backfill bugs (PR #3261 )
* federation: fix LaterGauge usage (PR #3328 ) Thanks to @intelfx!
2018-06-06 12:27:33 +01:00
Neil Johnson
61134debdc
bump version and changelog
2018-06-06 11:26:21 +01:00
Richard van der Hoff
ad459a106c
Merge pull request #3349 from t3chguy/redact_as_request_token
...
Redact AS tokens in log (fixes to #3327 )
2018-06-06 10:58:07 +01:00
Michael Telatynski
592c162516
also redact __str__ of ApplicationService used for logging
2018-06-06 10:35:29 +01:00
Michael Telatynski
330432031b
redact_uri in two missed log paths
2018-06-06 10:25:48 +01:00
David Baker
bf54c1cf6c
pep8
2018-06-06 10:15:33 +01:00
David Baker
3e4bc4488c
More doc fixes
2018-06-06 09:44:10 +01:00
Richard van der Hoff
592ee217a3
Merge commit 'b7e7fd2' into release-v0.31.0
2018-06-06 07:02:02 +01:00
Ivan Shapovalov
c88d50aa8f
federation/send_queue.py: fix usage of sortedcontainers.SortedDict
2018-06-06 02:45:18 +03:00
Richard van der Hoff
e316407b5d
Merge pull request #3327 from t3chguy/redact_as_request_token
...
Strip `access_token` from outgoing requests
2018-06-05 19:08:46 +01:00
Michael Telatynski
e6cbf47773
factor out uri redaction into a method on http
2018-06-05 18:31:40 +01:00
David Baker
607bd27c83
fix pep8
2018-06-05 18:10:35 +01:00
David Baker
d62162bbec
doc fixes
2018-06-05 18:09:13 +01:00
Richard van der Hoff
522bd3c8a3
Merge remote-tracking branch 'origin/master' into develop
2018-06-05 17:42:49 +01:00
Will Hunt
d6e3c2c79b
Let's try labels instead of label, that might work
2018-06-05 17:30:45 +01:00
Amber Brown
f7869f8f8b
Port to sortedcontainers (with tests!) ( #3332 )
2018-06-06 00:13:57 +10:00
Will Hunt
604cff1a06
Add metrics to track appservice transactions
2018-06-05 13:21:30 +01:00
Richard van der Hoff
f29b41fde9
Merge pull request #3324 from matrix-org/rav/remove_dead_method
...
Remove was_forgotten_at
2018-06-04 18:18:08 +01:00
Richard van der Hoff
28b0490dfd
Merge pull request #3334 from matrix-org/rav/cache_factor_override
...
Cache factor override system for specific caches
2018-06-04 17:19:33 +01:00
Richard van der Hoff
b7e7fd2d0e
Fix replication metrics
...
fix bug introduced in #3256
2018-06-04 16:23:05 +01:00
Neil Johnson
244ab974e7
bump version and changelog
2018-06-04 16:09:58 +01:00
Richard van der Hoff
694968fa81
Hopefully, fix LaterGuage error handling
2018-06-04 15:59:14 +01:00
Erik Johnston
042eedfa2b
Add hacky cache factor override system
2018-06-04 15:39:28 +01:00
David Baker
c5930d513a
Docstring
2018-06-04 12:05:58 +01:00
David Baker
6a29e815fc
Fix comment
2018-06-04 12:01:23 +01:00
David Baker
e44150a6de
Missing yield
2018-06-04 12:01:13 +01:00
David Baker
f731e42baf
docstring
2018-06-04 12:00:51 +01:00
Matthew Hodgson
28f09fcdd5
Merge branch 'develop' into matthew/filter_members
2018-06-04 00:09:17 +03:00
Matthew Hodgson
5f6122fe10
more comments
2018-06-04 00:08:52 +03:00
Ivan Shapovalov
7d9d75e4e8
federation/send_queue.py: fix usage of LaterGauge
...
Fixes a startup crash due to commit df9f72d9e5
2018-06-03 14:16:17 +03:00
Michael Telatynski
09503126df
Strip access_token from outgoing requests using existing regex
2018-06-02 23:25:13 +01:00
Richard van der Hoff
c1f4118bb6
Remove was_forgotten_at
...
This is unused. IT MUST DIE!!!1
̧̪͈̱̹̳͖͙H̵̰̤̰͕̖e̛ ͚͉̗̼̞w̶̩̥͉̮h̩̺̪̩͘ͅọ͎͉̟ ̜̩͔̦̘ͅW̪̫̩̣̲͔̳a͏͔̳͖i͖͜t͓̤̠͓͙s̘̰̩̥̙̝ͅ ̲̠̬̥Be̡̙̫̦h̰̩i̛̫͙͔̭̤̗̲n̳͞d̸ ͎̻͘T̛͇̝̲̹̠̗ͅh̫̦̝ͅe̩̫͟ ͓͖̼W͕̳͎͚̙̥ą̙l̘͚̺͔͞ͅl̳͍̙̤̤̮̳.̢
̟̺̜̙͉Z̤̲̙̙͎̥̝A͎̣͔̙͘L̥̻̗̳̻̳̳͢G͉̖̯͓̞̩̦O̹̹̺!̙͈͎̞̬ *
2018-06-01 18:21:49 +01:00
Richard van der Hoff
a9e97dcd65
Merge pull request #3317 from thegcat/feature/3312-add_ipv6_to_blacklist_example_config
...
Add private IPv6 addresses to example config for url preview blacklist
2018-06-01 14:45:14 +01:00
Neil Johnson
71477f3317
Merge pull request #3264 from matrix-org/neil/sign-up-stats
...
daily user type phone home stats
2018-06-01 13:42:01 +00:00
Richard van der Hoff
9f797a24a4
Handle RRs which arrive before their events
2018-06-01 14:01:43 +01:00
Richard van der Hoff
857e6fd8b6
Ignore depth when updating read-receipts
...
Order read receipts by stream ordering instead of depth
2018-06-01 12:18:11 +01:00
Felix Schäfer
4ef76f3ac4
Add private IPv6 addresses to preview blacklist #3312
...
The added addresses are expected to be local or loopback addresses and
shouldn't be spidered for previews.
Signed-off-by: Felix Schäfer <felix@thegcat.net>
2018-06-01 12:18:35 +02:00
Neil Johnson
4986b084f8
remove unnecessary INSERT
2018-06-01 10:50:40 +01:00
Richard van der Hoff
7e15410f02
Enforce the specified API for report_event
...
as per
https://matrix.org/docs/spec/client_server/unstable.html#post-matrix-client-r0-rooms-roomid-report-eventid
2018-05-31 18:17:11 +01:00
Amber Brown
febe0ec8fd
Run Prometheus on a different port, optionally. ( #3274 )
2018-05-31 19:04:50 +10:00
Amber Brown
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
Richard van der Hoff
219c2a322b
remove trailing whitespace
2018-05-30 19:42:19 +01:00
Richard van der Hoff
2e4be8bfd9
fix english and wrap comment
2018-05-30 19:24:12 +01:00
Amber Brown
872cf43516
Merge pull request #3303 from NotAFile/py3-memoryview
...
use memoryview in py3
2018-05-30 12:51:42 +10:00
Amber Brown
debff7ae09
Merge pull request #3281 from NotAFile/py3-six-isinstance
...
remaining isintance fixes
2018-05-30 12:44:46 +10:00
Richard van der Hoff
34b85df7f5
Update some comments and docstrings in SyncHandler
2018-05-29 22:31:18 +01:00
Richard van der Hoff
4a9cbdbc15
Exempt AS-registered users from doing gdpr
2018-05-29 19:54:32 +01:00
Neil Johnson
ab0ef31dc7
create users index on creation_ts
2018-05-29 17:51:08 +01:00
Neil Johnson
558f3d376a
create index in background
2018-05-29 17:47:55 +01:00
Neil Johnson
c379acd4fd
bump version
2018-05-29 17:47:28 +01:00
Adrian Tschira
4b9d0cde97
add remaining memoryview changes
2018-05-29 17:42:43 +02:00
Adrian Tschira
7873cde526
pep8
2018-05-29 17:35:55 +02:00
Adrian Tschira
1afafb3497
use memoryview in py3
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-29 17:14:34 +02:00
Matthew Hodgson
9bbb9f5556
add lazy_load_members to the filter json schema
2018-05-29 04:26:10 +01:00
Matthew Hodgson
8df7bad839
pep8
2018-05-29 02:49:36 +01:00
Matthew Hodgson
ff1bc0a279
pep8
2018-05-29 02:32:15 +01:00
Matthew Hodgson
0a240ad36e
disable CPUMetrics if no /proc/self/stat
...
fixes build on macOS again
2018-05-29 02:23:30 +01:00
Matthew Hodgson
b69ff33d9e
disable CPUMetrics if no /proc/self/stat
...
fixes build on macOS again
2018-05-29 02:22:27 +01:00
Matthew Hodgson
a6c8f7c875
add pydoc
2018-05-29 01:09:55 +01:00
Matthew Hodgson
7a6df013cc
merge develop
2018-05-29 00:25:22 +01:00
Amber Brown
57ad76fa4a
fix up tests
2018-05-28 19:51:53 +10:00
Amber Brown
3ef5cd74a6
update to more consistently use seconds in any metrics or logging
2018-05-28 19:39:27 +10:00
Amber Brown
5c40ce3777
invalid syntax :(
2018-05-28 19:16:09 +10:00
Amber Brown
357c74a50f
add comment about why unreg
2018-05-28 19:14:41 +10:00
Amber Brown
a2eb5db4a0
update metrics to be in seconds
2018-05-28 19:10:27 +10:00
Amber Brown
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
Ruben Barkow
08ea5fe635
add link to thorough instruction how to configure consent
2018-05-25 23:19:55 +02:00
Richard van der Hoff
66bdae986f
Fix default for send_server_notice_to_guests
...
bool("False") == True...
2018-05-25 11:42:05 +01:00
Richard van der Hoff
ba1b163590
Avoid sending consent notice to guest users
...
we think it makes sense not to send the notices to guest users.
2018-05-25 11:36:43 +01:00
Richard van der Hoff
41921ac01b
Merge pull request #3287 from matrix-org/rav/allow_leaving_server_notices_room
...
Let users leave the server notice room after joining
2018-05-25 11:15:45 +01:00
Richard van der Hoff
757ed27258
Let users leave the server notice room after joining
...
They still can't reject invites, but we let them leave it.
2018-05-25 11:07:21 +01:00
Adrian Tschira
4ee4450d66
fix recursion error
2018-05-24 21:44:10 +02:00
Amber Brown
9c36c150e7
Merge pull request #3283 from NotAFile/py3-state
...
py3-ize state.py
2018-05-24 14:23:33 -05:00
Amber Brown
cc1349c06a
Merge pull request #3279 from NotAFile/py3-more-iteritems
...
more six iteritems
2018-05-24 14:23:13 -05:00
Adrian Tschira
0e61705661
py3-ize state.py
2018-05-24 20:59:00 +02:00
Adrian Tschira
dd068ca979
remaining isintance fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:55:08 +02:00
Adrian Tschira
17a70cf6e9
Misc. py3 fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:20:33 +02:00
Adrian Tschira
6c16a4ec1b
more iteritems
2018-05-24 20:19:06 +02:00
Amber Brown
7ea07c7305
Merge pull request #3278 from NotAFile/py3-storage-base
...
Py3 storage/_base.py
2018-05-24 13:08:09 -05:00
Amber Brown
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4
...
replace some iteritems with six
2018-05-24 13:04:07 -05:00
Amber Brown
c4fb15a06c
Merge pull request #3246 from NotAFile/py3-repr-string
...
use repr, not str
2018-05-24 13:00:20 -05:00
Amber Brown
36501068d8
Merge pull request #3247 from NotAFile/py3-misc
...
Misc Python3 fixes
2018-05-24 12:58:37 -05:00
Amber Brown
2aff6eab6d
Merge pull request #3245 from NotAFile/batch-iter
...
Add batch_iter to utils
2018-05-24 12:54:12 -05:00
Adrian Tschira
095292304f
Py3 storage/_base.py
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 18:24:12 +02:00
David Baker
77a23e2e05
Merge remote-tracking branch 'origin/develop' into dbkr/unbind
2018-05-24 16:20:53 +01:00
David Baker
ecc4b88bd1
Merge pull request #3277 from matrix-org/dbkr/remove_from_user_dir
...
Remove users from user directory on deactivate
2018-05-24 16:12:12 +01:00
Erik Johnston
46345187cc
Merge pull request #3243 from NotAFile/py3-six-3
...
Replace some more comparisons with six
2018-05-24 16:08:57 +01:00
Neil Johnson
037c6db85d
Merge branch 'master' into develop
2018-05-24 16:03:44 +01:00
David Baker
7a1af504d7
Remove users from user directory on deactivate
2018-05-24 15:59:58 +01:00
Erik Johnston
f72d5a44d5
Merge pull request #3261 from matrix-org/erikj/pagination_fixes
...
Fix federation backfill bugs
2018-05-24 14:52:03 +01:00
Erik Johnston
68399fc4de
Merge pull request #3267 from matrix-org/erikj/iter_filter
...
Use iter* methods for _filter_events_for_server
2018-05-24 14:44:57 +01:00
Neil Johnson
91d95a1d8e
bump version
2018-05-24 14:05:07 +01:00
David Baker
9700d15611
pep8
2018-05-24 11:23:15 +01:00
David Baker
a21a41bad7
comment
2018-05-24 11:19:59 +01:00
David Baker
b3bff53178
Unbind 3pids when they're deleted too
2018-05-24 11:08:05 +01:00
Richard van der Hoff
6abcb5d22d
Merge pull request #3273 from matrix-org/rav/server_notices_avatar_url
...
Allow overriding the server_notices user's avatar
2018-05-24 10:31:43 +01:00
Amber Brown
389dac2c15
pepeightttt
2018-05-23 13:08:59 -05:00
Amber Brown
472a5ec4e2
add back CPU metrics
2018-05-23 13:03:56 -05:00
Amber Brown
e987079037
fixes
2018-05-23 13:03:51 -05:00
Richard van der Hoff
9bf4b2bda3
Allow overriding the server_notices user's avatar
...
probably should have done this in the first place, like @turt2live suggested.
2018-05-23 17:43:30 +01:00
Richard van der Hoff
a0b3946fe2
Merge branch 'release-v0.30.0' into rav/localpart_in_consent_uri
2018-05-23 16:06:03 +01:00
Richard van der Hoff
2f7008d4eb
Merge pull request #3271 from matrix-org/rav/consent_uri_in_messages
...
Support for putting %(consent_uri)s in messages
2018-05-23 16:04:30 +01:00
Richard van der Hoff
dda40fb55d
fix typo
2018-05-23 15:30:26 +01:00
Richard van der Hoff
3ff6f50eac
Use the localpart in the consent uri
...
... because it's shorter.
2018-05-23 15:28:23 +01:00
Richard van der Hoff
82191b08f6
Support for putting %(consent_uri)s in messages
...
Make it possible to put the URI in the error message and the server notice that
get sent by the server
2018-05-23 15:24:31 +01:00
David Baker
2c7866d664
Hit the 3pid unbind endpoint on deactivation
2018-05-23 14:38:56 +01:00
Richard van der Hoff
321f02d263
Block attempts to send server notices to remote users
2018-05-23 14:30:47 +01:00
Neil Johnson
3601a240aa
bump version and changelog
2018-05-23 12:03:23 +01:00
Erik Johnston
5aaa3189d5
s/values/itervalues/
2018-05-23 10:13:05 +01:00
Erik Johnston
0a4bca4134
Use iter* methods for _filter_events_for_server
2018-05-23 10:04:23 +01:00
Amber Brown
b6063631c3
more cleanup
2018-05-22 17:36:20 -05:00
Amber Brown
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
Amber Brown
071206304d
cleanup pep8 errors
2018-05-22 16:54:22 -05:00
Amber Brown
85ba83eb51
fixes
2018-05-22 16:28:23 -05:00
Amber Brown
228f1f584e
fix the test failures
2018-05-22 15:02:38 -05:00
Erik Johnston
e85b5a0ff7
Use iter* methods
2018-05-22 19:02:48 +01:00
Erik Johnston
586b66b197
Fix that states is a dict of dicts
2018-05-22 19:02:36 +01:00
Erik Johnston
a17e901f4d
Remove unused string formatting param
2018-05-22 18:24:32 +01:00
Erik Johnston
5494c1d71e
Don't support limitless pagination
...
The pagination storage function supported not specifiying a limit on the
number of events returned. This was triggered when using the search or
context API with a limit of zero, which the storage function took to
mean not being limited.
2018-05-22 18:15:21 +01:00
Neil Johnson
d8cb7225d2
daily user type phone home stats
2018-05-22 18:09:09 +01:00
hera
ad2823ee27
fix synchrotron
2018-05-22 17:47:42 +01:00
Richard van der Hoff
08bfc48abf
custom error code for not leaving server notices room
2018-05-22 17:27:27 +01:00
David Baker
0a078026ea
comment typo
2018-05-22 17:14:06 +01:00
Amber Brown
8f5a688d42
cleanups, self-registration
2018-05-22 10:56:03 -05:00
Amber Brown
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
Erik Johnston
cb2a2ad791
get_domains_from_state returns list of tuples
2018-05-22 16:23:39 +01:00
Richard van der Hoff
08a14b32ae
Merge pull request #3262 from matrix-org/rav/has_already_consented
...
Add a 'has_consented' template var to consent forms
2018-05-22 15:35:10 +01:00
Richard van der Hoff
7b36d06a69
Add a 'has_consented' template var to consent forms
...
fixes #3260
2018-05-22 14:58:34 +01:00
Richard van der Hoff
669400e22f
Enable auto-escaping for the consent templates
...
... to reduce the risk of somebody introducing an html injection attack...
2018-05-22 14:58:34 +01:00
Richard van der Hoff
b5b2d5d64b
Fix dependency on jinja2
...
Delay the import of ConsentResource, so that we can get away without jinja2 if
people don't have the consent resource enabled.
Fixes #3259
2018-05-22 14:03:45 +01:00
Richard van der Hoff
a5e2941aad
Reject attempts to send event before privacy consent is given
...
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252 ) if consent is not yet
given.
2018-05-22 12:00:47 +01:00
Richard van der Hoff
8aeb529262
Merge pull request #3236 from matrix-org/rav/consent_notice
...
Send users a server notice about consent
2018-05-22 11:56:09 +01:00
Richard van der Hoff
8810685df9
Stub out ServerNoticesSender on the workers
...
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
2018-05-22 11:54:51 +01:00
Richard van der Hoff
d5dca9a04f
Move consent config parsing into ConsentConfig
...
turns out we need to reuse this, so it's better in the config class.
2018-05-22 11:54:51 +01:00
Richard van der Hoff
9ea219c514
Send users a server notice about consent
...
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
2018-05-22 11:54:51 +01:00
Richard van der Hoff
d14d7b8fdc
Rename 'version' param on user consent config
...
we're going to use it for the version we require too.
2018-05-22 11:54:51 +01:00
Erik Johnston
7948ecf234
Comment
2018-05-22 11:39:43 +01:00
Erik Johnston
020377a550
Fix logcontext resource usage tracking
2018-05-22 11:16:07 +01:00
Erik Johnston
13a8dfba0d
Merge pull request #3252 from matrix-org/erikj/in_flight_requests
...
Add in flight request metrics
2018-05-22 09:54:30 +01:00
Erik Johnston
c435b0b441
Don't store context
2018-05-22 09:34:26 +01:00
Erik Johnston
fb2806b186
Move in_flight_requests_count to be a callback metric
2018-05-22 09:31:53 +01:00
Amber Brown
fcc525b0b7
rest of the changes
2018-05-21 19:48:57 -05:00
Amber Brown
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
Amber Brown
c60e0d5e02
don't need the resource portion
2018-05-21 17:03:20 -05:00
Amber Brown
02c1d29133
look at the Prometheus metrics instead
2018-05-21 17:02:20 -05:00
Amber Brown
f258deffcb
remove old metrics libs
2018-05-21 17:01:15 -05:00
Richard van der Hoff
413482f578
Merge pull request #3255 from matrix-org/rav/fix_transactions
...
Stop the transaction cache caching failures
2018-05-21 17:58:08 +01:00
Richard van der Hoff
6e1cb54a05
Fix logcontext leak in HttpTransactionCache
...
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
2018-05-21 16:58:20 +01:00
Richard van der Hoff
6d6e7288fe
Stop the transaction cache caching failures
...
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
2018-05-21 16:49:59 +01:00
Erik Johnston
dfa70adc33
Add in flight request metrics
...
This tracks CPU and DB usage while requests are in flight, rather than
when we write the response.
2018-05-21 16:23:06 +01:00
Adrian Tschira
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
Adrian Tschira
d9fe2b2d9d
Replace some more comparisons with six
...
plus a bonus b"" string I missed last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
Adrian Tschira
45b55e23d3
Add batch_iter to utils
...
There's a frequent idiom I noticed where an iterable is split up into a
number of chunks/batches. Unfortunately that method does not work with
iterators like dict.keys() in python3. This implementation works with
iterators.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:48:30 +02:00
Adrian Tschira
dcc235b47d
use stand-in value if maxint is not available
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:44 +02:00
Adrian Tschira
73cbdef5f7
fix py3 intern and remove unnecessary py3 encode
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:31 +02:00
Adrian Tschira
aafb0f6b0d
py3-ize url preview
2018-05-19 17:35:20 +02:00
Adrian Tschira
b932b4ea25
use repr, not str
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:28:42 +02:00
Neil Johnson
644aac5f73
Tighter filtering for user_daily_visits
2018-05-18 17:10:35 +01:00
Neil Johnson
ef466b3a13
fix psql compatability bug
2018-05-18 15:51:21 +01:00
Neil Johnson
2725223f08
Merge branch 'master' into develop
2018-05-18 14:07:50 +01:00
Richard van der Hoff
6d9dc67139
Merge pull request #3232 from matrix-org/rav/server_notices_room
...
Infrastructure for a server notices room
2018-05-18 11:28:04 +01:00
Richard van der Hoff
ed3125b0a1
Merge pull request #3235 from matrix-org/rav/fix_receipts_deferred
...
Fix error in handling receipts
2018-05-18 11:23:11 +01:00
Richard van der Hoff
67af392712
Merge pull request #3233 from matrix-org/rav/remove_dead_code
...
Remove unused `update_external_syncs`
2018-05-18 11:22:43 +01:00
Richard van der Hoff
011e1f4010
Better docstrings
2018-05-18 11:22:12 +01:00
Richard van der Hoff
26305788fe
Make sure we reject attempts to invite the notices user
2018-05-18 11:18:39 +01:00
Richard van der Hoff
d10707c810
Replace inline docstrings with "Attributes" in class docstring
2018-05-18 11:00:55 +01:00
Erik Johnston
fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token
...
Make purge_history operate on tokens
2018-05-18 10:35:23 +01:00
Richard van der Hoff
8b1c856d81
Fix error in handling receipts
...
Fixes an error which has been happening ever since #2158 (v0.21.0-rc1):
> TypeError: argument of type 'ObservableDeferred' is not iterable
fixes #3234
2018-05-18 09:15:35 +01:00
Neil Johnson
6958459b50
bump version, change log
2018-05-17 21:35:07 +01:00
Richard van der Hoff
88d3405332
fix missing yield for server_notices_room
2018-05-17 18:33:45 +01:00
Richard van der Hoff
d43d480d86
Remove unused update_external_syncs
...
This method isn't used anywhere. Burninate it.
2018-05-17 18:22:19 +01:00
Richard van der Hoff
fed62e21ad
Infrastructure for a server notices room
...
Server Notices use a special room which the user can't dismiss. They are
created on demand when some other bit of the code calls send_notice.
(This doesn't actually do much yet becuse we don't call send_notice anywhere)
2018-05-17 17:58:25 +01:00
Richard van der Hoff
f8a1e76d64
Merge pull request #3225 from matrix-org/rav/move_creation_handler
...
Move RoomCreationHandler out of synapse.handlers.Handlers
2018-05-17 17:56:42 +01:00
Erik Johnston
f7906203f6
Merge pull request #3212 from matrix-org/erikj/epa_stream
...
Use stream rather depth ordering for push actions
2018-05-17 12:01:21 +01:00
Richard van der Hoff
c46367d0d7
Move RoomCreationHandler out of synapse.handlers.Handlers
...
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.
Also fix the docstrings on create_room.
2018-05-17 09:08:42 +01:00
Neil Johnson
85b8acdeb4
Merge branch 'master' into develop
2018-05-16 16:48:10 +01:00
Neil Johnson
3c099219e0
bump version and changelog for 0.29.0
2018-05-16 15:44:59 +01:00
Erik Johnston
680530cc7f
Clarify comment
2018-05-16 11:47:29 +01:00
Erik Johnston
43e6e82c4d
Comments
2018-05-16 11:13:31 +01:00
Neil Johnson
dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics
...
user visit data
2018-05-16 10:09:24 +00:00
Erik Johnston
c945af8799
Move and rename variable
2018-05-16 10:52:06 +01:00
Neil Johnson
be11a02c4f
remove empty line
2018-05-16 10:45:40 +01:00
Neil Johnson
a2204cc9cc
remove unused method recurring_user_daily_visit_stats
2018-05-16 09:47:20 +01:00
Neil Johnson
31c2502ca8
style and further contraining query
2018-05-16 09:46:43 +01:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler
...
ConsentResource to gather policy consent from users
2018-05-16 07:19:18 +01:00
Neil Johnson
c92a8aa578
pep8
2018-05-15 17:31:11 +01:00
Neil Johnson
05ac15ae82
Limit query load of generate_user_daily_visits
...
The aim is to keep track of when it was last called and only query from that point in time
2018-05-15 17:01:33 +01:00
Erik Johnston
5f27ed75ad
Make purge_history operate on tokens
...
As we're soon going to change how topological_ordering works
2018-05-15 16:23:50 +01:00
Erik Johnston
37dbee6490
Use events_to_purge table rather than token
2018-05-15 16:23:47 +01:00
Richard van der Hoff
47815edcfa
ConsentResource to gather policy consent from users
...
Hopefully there are enough comments and docs in this that it makes sense on its
own.
2018-05-15 15:11:59 +01:00
Neil Johnson
aea80a0118
v0.29.0-rc1: bump version and change log
2018-05-14 15:50:57 +01:00
Neil Johnson
f077e97914
instead of inserting user daily visit data at the end of the day, instead insert incrementally through the day
2018-05-14 13:50:58 +01:00
David Baker
8cbbfd16fb
Merge pull request #3201 from matrix-org/dbkr/leave_rooms_on_deactivate
...
Part user from rooms on account deactivate
2018-05-14 11:31:48 +01:00
Neil Johnson
977765bde2
Merge branch 'develop' of https://github.com/matrix-org/synapse into cohort_analytics
2018-05-14 09:31:42 +01:00
Richard van der Hoff
c25d7ba12e
Merge pull request #3208 from matrix-org/rav/more_refactor_request_handler
...
Set Server header in SynapseRequest
2018-05-11 16:07:47 +01:00
Erik Johnston
6406b70aeb
Use stream rather depth ordering for push actions
...
This simplifies things as it is, but will also allow us to change the
way we traverse topologically without having to update the way push
actions work.
2018-05-11 15:30:11 +01:00
Richard van der Hoff
23e2dfe940
Merge pull request #3209 from damir-manapov/master
...
transaction_id, destination defined twice
2018-05-11 00:35:13 +01:00
Damir Manapov
db18d854cd
transaction_id, destination twice
2018-05-10 22:13:31 +03:00
Richard van der Hoff
318711e139
Set Server header in SynapseRequest
...
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).
This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.
Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
2018-05-10 18:50:27 +01:00
David Baker
6b49628e3b
Catch failure to part user from room
2018-05-10 12:23:53 +01:00
David Baker
217bc53c98
Many docstrings
2018-05-10 12:20:40 +01:00
Richard van der Hoff
645cb4bf06
Remove redundant request_handler decorator
...
This is needless complexity; we might as well use the wrapper directly.
Also rename wrap_request_handler->wrap_json_request_handler.
2018-05-10 12:19:53 +01:00
Richard van der Hoff
09f570b935
Factor wrap_request_handler_with_logging out of wrap_request_handler
...
... so that it can be used on non-JSON endpoints
2018-05-10 12:19:52 +01:00
Richard van der Hoff
9589a1925e
Remove include_metrics param
...
The metrics are now available via the request, so this is redundant and can go
away at last.
2018-05-10 12:19:52 +01:00
Richard van der Hoff
49e5a613f1
Move outgoing_responses_counter handling to RequestMetrics
...
it's much neater there.
2018-05-10 12:19:52 +01:00
Richard van der Hoff
b8700dd7d0
Bump requests_counter in wrapped_request_handler
...
less magic
2018-05-10 12:19:52 +01:00
Richard van der Hoff
c6f730282c
Move RequestMetrics handling into SynapseRequest.processing()
...
It fits quite nicely here, and opens the path to getting rid of the
"include_metrics" mess.
2018-05-10 12:19:51 +01:00
Richard van der Hoff
09b29f9c4a
Make RequestMetrics take a raw time rather than a clock
...
... which is going to make it easier to move around.
2018-05-10 12:18:52 +01:00
David Baker
4d298506dd
Oops, don't call function passed to run_in_background
2018-05-10 11:57:13 +01:00
Richard van der Hoff
8460e48d06
Move request_id management into SynapseRequest
2018-05-10 11:48:17 +01:00
Richard van der Hoff
18e144fe08
Move RequestsMetrics to its own file
...
This is useful in its own right, because server.py is full of stuff; but more
importantly, I want to do some refactoring that will cause a circular reference
as it is.
2018-05-09 19:55:03 +01:00
Erik Johnston
bfe1f73855
Merge pull request #3199 from matrix-org/erikj/pagination_sync
...
Refactor sync APIs to reuse pagination API
2018-05-09 16:16:56 +01:00
Erik Johnston
5adb75bcba
Merge pull request #3198 from matrix-org/erikj/fixup_return_pagination
...
Refactor get_recent_events_for_room return type
2018-05-09 16:07:14 +01:00
Erik Johnston
d26bec8a43
Add comment to sync as to why code path is split
2018-05-09 15:56:07 +01:00
Erik Johnston
fcf55f2255
Fix returned token is no longer a tuple
2018-05-09 15:43:00 +01:00
Erik Johnston
7ce98804ff
Fix up comment
2018-05-09 15:42:39 +01:00
Erik Johnston
cddf91c8b9
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_membership_change
2018-05-09 15:32:07 +01:00
Erik Johnston
9896dab8f6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fixup_return_pagination
2018-05-09 15:31:33 +01:00
Erik Johnston
1e5280b7d0
Merge pull request #3196 from matrix-org/erikj/pagination_return
...
Refactor pagination DB API to return concrete type
2018-05-09 15:27:17 +01:00
Erik Johnston
75552d2148
Update comments
2018-05-09 15:15:38 +01:00
David Baker
294e9a0c9b
Prefix internal functions
2018-05-09 15:10:37 +01:00
David Baker
46df23f581
Add the schema file
2018-05-09 15:07:54 +01:00
David Baker
52281e4c54
Indent fail
2018-05-09 15:06:16 +01:00
David Baker
7e8726b8fb
Part deactivated users in the background
...
One room at a time so we don't take out the whole server with leave
events, and restart at server restart.
2018-05-09 14:54:28 +01:00
Erik Johnston
c0e08dc45b
Remove unused code path from member change DB func
...
The function is never called without a from_key, so we can remove all
the handling for that scenario.
2018-05-09 14:31:32 +01:00
Erik Johnston
0461ef01b7
Merge pull request #3195 from matrix-org/erikj/pagination_refactor
...
Refactor recent events func to use pagination func
2018-05-09 14:12:24 +01:00
Erik Johnston
e2accd7f1d
Refactor sync APIs to reuse pagination API
...
The sync API often returns events in a topological rather than stream
ordering, e.g. when the user joined the room or on initial sync. When
this happens we can reuse existing pagination storage functions.
2018-05-09 13:43:39 +01:00
Erik Johnston
e5ab9cd24b
Don't unnecessarily require token to be stream token
...
This allows calling the `get_recent_event_ids_for_room` function in more
situations.
2018-05-09 11:58:35 +01:00
Erik Johnston
c4af4c24ca
Refactor get_recent_events_for_room return type
...
There is no reason to return a tuple of tokens when the last token is
always the token passed as an argument. Changing it makes it consistent
with other storage APIs
2018-05-09 11:55:34 +01:00
Erik Johnston
05e0a2462c
Refactor pagination DB API to return concrete type
...
This makes it easier to document what is being returned by the storage
functions and what some functions expect as arguments.
2018-05-09 11:34:24 +01:00
Erik Johnston
7dd13415db
Remove unused from_token param
2018-05-09 10:58:16 +01:00
Erik Johnston
27cf170558
Refactor recent events func to use pagination func
...
This also removes a cache that is unlikely to ever get hit.
2018-05-09 10:55:55 +01:00
Erik Johnston
1aeb5e28a9
Merge pull request #3193 from matrix-org/erikj/pagination_refactor
...
Refactor /context to reuse pagination storage functions
2018-05-09 10:22:38 +01:00
Erik Johnston
23ec51c94c
Fix up comments and make function private
2018-05-09 09:55:19 +01:00
Erik Johnston
696f532453
Reuse existing pagination code for context API
2018-05-08 16:20:19 +01:00
Erik Johnston
3e6d306e94
Parse tokens before calling DB function
2018-05-08 16:18:58 +01:00
Erik Johnston
274b8c6025
Only fetch required fields from database
2018-05-08 16:15:25 +01:00
Erik Johnston
06c0d0ed08
Split paginate_room_events storage function
2018-05-08 16:14:26 +01:00
David Baker
bf98fa0864
Part user from rooms on account deactivate
...
This implements this very crudely: this probably isn't viable
because parting a user from all their rooms could take a long time,
and if the HS gets restarted in that time the process will be
aborted.
2018-05-08 15:58:35 +01:00
Richard van der Hoff
678e649b78
Merge pull request #3190 from mujx/notif-token-fix
...
notifications: Convert next_token to string according to the spec
2018-05-08 13:54:47 +01:00
Konstantinos Sideris
88868b2839
notifications: Convert next_token to string according to the spec
...
Currently the parameter is serialized as an integer.
Signed-off-by: Konstantinos Sideris <sideris.konstantin@gmail.com>
2018-05-05 12:55:02 +03:00
Erik Johnston
6d8ec3462d
Note that label values can be anything
2018-05-03 16:25:05 +01:00
Erik Johnston
95b6912045
Fix metrics that have integer value labels
2018-05-03 15:51:04 +01:00
Richard van der Hoff
093d8c415a
Merge remote-tracking branch 'origin/develop' into rav/warn_on_logcontext_fail
2018-05-03 14:59:29 +01:00
Richard van der Hoff
0ba609dc6f
Merge pull request #3183 from matrix-org/rav/moar_logcontext_leaks
...
Fix logcontext leaks in rate limiter
2018-05-03 14:58:13 +01:00
Richard van der Hoff
2117f84323
Merge pull request #3182 from Half-Shot/hs/fix-twisted-shutdown
...
Fix 'Unhandled Error' logs with Twisted 18.4
2018-05-03 12:40:11 +01:00
Richard van der Hoff
a7fe62f0cb
Fix logcontext leaks in rate limiter
2018-05-03 12:31:59 +01:00
Will Hunt
2e7a94c36b
Don't abortConnection() if the transport connection has already closed.
2018-05-03 12:31:47 +01:00
Richard van der Hoff
a2aaa9cb3c
Merge pull request #3178 from matrix-org/rav/fix_request_timeouts
...
fix http request timeout code
2018-05-03 11:33:26 +01:00
Erik Johnston
0a3b51c420
Merge pull request #3141 from matrix-org/erikj/fixup_state
...
Refactor event storage to prepare for changes in state calculations
2018-05-03 10:39:20 +01:00
Erik Johnston
31c7c29d43
Fix up grammar
2018-05-03 10:38:58 +01:00
Richard van der Hoff
902673e356
Merge pull request #3161 from NotAFile/remove-v1auth
...
Make Client-Server API return 403 for invalid token
2018-05-03 10:10:57 +01:00
Erik Johnston
53a5fdf312
Merge pull request #3175 from matrix-org/erikj/escape_metric_values
...
Escape label values in prometheus metrics
2018-05-03 10:01:04 +01:00
Richard van der Hoff
1dfd650348
add missing param to cancelled_to_request_timed_out_error
...
This gets two arguments, not one.
2018-05-02 22:42:36 +01:00
Erik Johnston
a41117c63b
Make _escape_character take MatchObject
2018-05-02 17:27:27 +01:00
Erik Johnston
32015e1109
Escape label values in prometheus metrics
2018-05-02 16:52:42 +01:00
Richard van der Hoff
3a42aed9a1
Merge pull request #3170 from matrix-org/rav/more_logcontext_leaks
...
Fix a class of logcontext leaks
2018-05-02 16:45:51 +01:00
Richard van der Hoff
415c6b672e
Merge branch 'develop' into rav/more_logcontext_leaks
2018-05-02 16:16:01 +01:00
Richard van der Hoff
be31adb036
Fix logcontext leak in media repo
...
Make FileResponder.write_to_consumer uphold the logcontext contract
2018-05-02 16:14:50 +01:00
Richard van der Hoff
f22e7cda2c
Fix a class of logcontext leaks
...
So, it turns out that if you have a first `Deferred` `D1`, you can add a
callback which returns another `Deferred` `D2`, and `D2` must then complete
before any further callbacks on `D1` will execute (and later callbacks on `D1`
get the *result* of `D2` rather than `D2` itself).
So, `D1` might have `called=True` (as in, it has started running its
callbacks), but any new callbacks added to `D1` won't get run until `D2`
completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield`
will 'block'.
In conclusion: some of our assumptions in `logcontext` were invalid. We need to
make sure that we don't optimise out the logcontext juggling when this
situation happens. Fortunately, it is easy to detect by checking `D1.paused`.
2018-05-02 11:58:00 +01:00
Richard van der Hoff
a8d8bf92e0
Merge pull request #3168 from matrix-org/rav/fix_logformatter
...
Fix incorrect reference to StringIO
2018-05-02 10:03:36 +01:00
Richard van der Hoff
e482f8cd85
Fix incorrect reference to StringIO
...
This was introduced in 4f2f5171
2018-05-02 09:12:26 +01:00
Neil Johnson
2414178ed6
Merge branch 'master' into develop
2018-05-01 18:53:56 +01:00
Neil Johnson
8570bb84cc
Update __init__.py
...
bump version
2018-05-01 18:22:53 +01:00
Richard van der Hoff
ca7211104e
Merge branch 'release-v0.28.1' into develop
2018-05-01 18:16:57 +01:00
Richard van der Hoff
d5eee5d601
Merge commit '33f469b' into release-v0.28.1
2018-05-01 18:14:18 +01:00
Richard van der Hoff
d858f3bd4e
Miscellaneous fixes to python_dependencies
...
* add some doc about wtf this thing does
* pin Twisted to < 18.4
* add explicit dep on six (fixes #3089 )
2018-05-01 18:13:54 +01:00
Richard van der Hoff
33f469ba19
Apply some limits to depth to counter abuse
...
* When creating a new event, cap its depth to 2^63 - 1
* When receiving events, reject any without a sensible depth
As per https://docs.google.com/document/d/1I3fi2S-XnpO45qrpCsowZv8P8dHcNZ4fsBsbOW7KABI
2018-05-01 17:54:19 +01:00
Neil Johnson
dd1a832419
remove user agent from data model, will just join on user_ips
2018-05-01 12:13:49 +01:00
