Amber Brown
99800de63d
try and clean up
2018-06-27 11:40:27 +01:00
Amber Brown
f03a5d1a17
pep8
2018-06-27 11:38:14 +01:00
Amber Brown
a7ecf34b70
cleanups
2018-06-27 11:36:03 +01:00
Amber Brown
77078d6c8e
handle federation not telling us about prev_events
2018-06-27 11:27:32 +01:00
Matthew Hodgson
8057489b26
Revert "Try to not use as much CPU in the StreamChangeCache"
2018-06-26 18:09:01 +01:00
Matthew Hodgson
d91efb06cf
Merge pull request #3451 from matrix-org/hawkowl/sorteddict-api
...
Try to not use as much CPU in the cache
2018-06-26 17:49:55 +01:00
Amber Brown
1202508067
fixes
2018-06-26 17:29:01 +01:00
Amber Brown
bd3d329c88
fixes
2018-06-26 17:28:12 +01:00
Amber Brown
abfe4b2957
try and make loading items from the cache faster
2018-06-26 17:25:34 +01:00
David Baker
028490afd4
Fix error on deleting users pending deactivation
...
Use simple_delete instead of simple_delete_one as commented
2018-06-26 10:52:52 +01:00
Matthew Hodgson
c7f6b420ae
Merge pull request #3448 from matrix-org/matthew/gdpr-deactivate-admin-api
...
add GDPR erase param to deactivate API
2018-06-26 10:43:14 +01:00
Matthew Hodgson
9570aa82eb
update doc for deactivate API
2018-06-26 10:42:50 +01:00
Matthew Hodgson
1e788db430
add GDPR erase param to deactivate API
2018-06-26 10:26:54 +01:00
Amber Brown
1d62c4a127
Merge pull request #3438 from turt2live/travis/dont-print-access-tokens-in-logs
...
Stop including access tokens in warnings in the log
2018-06-26 09:55:55 +01:00
Erik Johnston
d72fb9a448
Merge pull request #3442 from matrix-org/matthew/allow-unconsented-parts
...
allow non-consented users to still part rooms (to let us autopart them)
2018-06-25 20:14:18 +01:00
Erik Johnston
df48f7ef37
Actually fix it
2018-06-25 20:03:41 +01:00
Erik Johnston
a0e8a53c6d
Comment
2018-06-25 19:57:38 +01:00
Erik Johnston
7bdc5c8fa3
Fix bug with assuming wrong type
2018-06-25 19:56:02 +01:00
Erik Johnston
ea7a9c0483
Add fast path to _filter_events_for_server
...
Most rooms have a trivial history visibility like "shared" or
"world_readable", especially large rooms, so lets not bother getting the
full membership of those rooms in that case.
2018-06-25 19:49:13 +01:00
Matthew Hodgson
0269367f18
allow non-consented users to still part rooms (to let us autopart them)
2018-06-25 17:56:10 +01:00
Matthew Hodgson
784189b1f4
typos
2018-06-25 17:37:16 +01:00
Matthew Hodgson
6eb861b67f
typo
2018-06-25 17:37:16 +01:00
Erik Johnston
947fea67cb
Need to pass reactor to endpoint fac
2018-06-25 15:22:57 +01:00
Erik Johnston
33fdcfa957
Merge pull request #3441 from matrix-org/erikj/redo_erasure
...
Fix user erasure and re-enable
2018-06-25 14:37:01 +01:00
Erik Johnston
eb50c44eaf
Add UserErasureWorkerStore to workers
2018-06-25 14:22:24 +01:00
Amber Brown
07cad26d65
Remove all global reactor imports & pass it around explicitly ( #3424 )
2018-06-25 14:08:28 +01:00
Erik Johnston
244484bf3c
Revert "Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility""
...
This reverts commit 1d009013b3
.
2018-06-25 13:42:55 +01:00
Travis Ralston
ec1e799e17
Don't print invalid access tokens in the logs
...
Tokens shouldn't be appearing the logs, valid or invalid.
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-06-24 12:17:01 -06:00
Richard van der Hoff
1d009013b3
Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"
...
This reverts commit ce0d911156
, reversing
changes made to b4a5d767a9
.
2018-06-22 16:35:10 +01:00
Richard van der Hoff
516f884176
Merge pull request #3435 from matrix-org/rav/fix_event_push_actions_tablescan
...
Fix event_push_actions tablescan when reinserting events
2018-06-22 15:57:22 +01:00
Mark Haines
9850f66abe
Deleting from event_push_actions needs to use an index
2018-06-22 15:54:48 +01:00
Erik Johnston
28ddc6cfbe
Also log number of events for serach context
2018-06-22 15:42:11 +01:00
Erik Johnston
4b4cec3989
Add some logging to search queries
2018-06-22 15:42:11 +01:00
Richard van der Hoff
200e11c5bf
Merge pull request #3432 from matrix-org/rav/joined_hosts_cache_non_iterable
...
Make _get_joined_hosts_cache cache non-iterable
2018-06-22 15:18:51 +01:00
Erik Johnston
f8272813a9
Make _get_joined_hosts_cache cache non-iterable
2018-06-22 15:12:26 +01:00
Richard van der Hoff
1d7ad11747
Merge pull request #3430 from matrix-org/rav/configurable_push_action_rotation
...
Make push actions rotation configurable
2018-06-22 15:07:31 +01:00
Erik Johnston
ce0d911156
Merge pull request #3431 from matrix-org/rav/erasure_visibility
...
Support hiding events from deleted users
2018-06-22 15:06:44 +01:00
Erik Johnston
b4a5d767a9
Merge pull request #3428 from matrix-org/erikj/persisted_pdu
...
Simplify get_persisted_pdu
2018-06-22 14:47:43 +01:00
Erik Johnston
f79abda87f
Merge pull request #3427 from matrix-org/erikj/remove_filters
...
remove dead filter_events_for_clients
2018-06-22 14:45:42 +01:00
Erik Johnston
75dc3ddeab
Make push actions rotation configurable
2018-06-22 14:44:37 +01:00
Richard van der Hoff
43e02c409d
Disable partial state group caching for wildcard lookups
...
When _get_state_for_groups is given a wildcard filter, just do a complete
lookup. Hopefully this will give us the best of both worlds by not filling up
the ram if we only need one or two keys, but also making the cache still work
for the federation reader usecase.
2018-06-22 11:52:07 +01:00
Richard van der Hoff
240f192523
Merge pull request #3382 from matrix-org/rav/optimise_state_groups
...
Optimise state_group_cache update
2018-06-22 11:20:20 +01:00
Richard van der Hoff
70e6501913
Merge pull request #3419 from matrix-org/rav/events_per_request
...
Log number of events fetched from DB
2018-06-22 11:17:56 +01:00
Richard van der Hoff
0495fe0035
Indirect evt_count updates via method call
...
so that we can stub it for the sentinel and not have a billion failing UTs
2018-06-22 10:42:28 +01:00
Amber Brown
9a685d60ae
Merge pull request #3418 from matrix-org/rav/fix_metric_desc
...
Fix description of "python_gc_time" metric
2018-06-22 09:43:26 +01:00
Amber Brown
77ac14b960
Pass around the reactor explicitly ( #3385 )
2018-06-22 09:37:10 +01:00
Richard van der Hoff
cbbfaa4be8
Fix description of "python_gc_time" metric
2018-06-21 10:02:42 +01:00
Amber Brown
c2eff937ac
Populate synapse_federation_client_sent_pdu_destinations:count again ( #3386 )
2018-06-21 09:39:58 +01:00
Amber Brown
99b77aa829
Fix tcp protocol metrics naming ( #3410 )
2018-06-21 09:39:27 +01:00
Richard van der Hoff
b088aafcae
Log number of events fetched from DB
...
When we finish processing a request, log the number of events we fetched from
the database to handle it.
[I'm trying to figure out which requests are responsible for large amounts of
event cache churn. It may turn out to be more helpful to add counts to the
prometheus per-request/block metrics, but that is an extension to this code
anyway.]
2018-06-21 06:15:03 +01:00
Richard van der Hoff
aff3d76920
Merge pull request #3416 from matrix-org/rav/restart_indicator
...
Write a clear restart indicator in logs
2018-06-20 18:14:08 +01:00
Richard van der Hoff
02bfc581f8
Merge pull request #3399 from costacruise/master
...
Add error code to room creation error
2018-06-20 17:26:25 +01:00
Richard van der Hoff
245d53d32a
Write a clear restart indicator in logs
...
I'm fed up with never being able to find the point a server restarted in the
logs.
2018-06-20 15:33:14 +01:00
Amber Brown
f6c4d74f96
Fix inflight requests metric (incorrect name & traceback) ( #3413 )
2018-06-20 11:18:57 +01:00
Matthew Hodgson
ccfdaf68be
spell gauge correctly
2018-06-16 07:10:34 +01:00
Richard van der Hoff
9a793f861c
Merge branch 'master' into develop
2018-06-14 16:36:01 +01:00
Richard van der Hoff
53969e1960
SECURITY UPDATE: Prevent unauthorised users from setting state events in a room
...
when there is no `m.room.power_levels` event in force in the room. (PR #3397 )
Discussion around the Matrix Spec change proposal for this change can be
followed at https://github.com/matrix-org/matrix-doc/issues/1304 .
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJbIop9AAoJEIofk9V1tejV9lsIAJVH0l5dXROmy1KH/zt16AUA
CXa6Vv4Vyo6hKad/fZ81OZVRr5ChK/TvbIJVn/SA/muCfdoIFdxhT8eo/pXzO2UW
zReuLsDhAg+gSvpNus37oWj2FVsAE1HYDZ60lfaapAdZnkFit68d5DQZjO6nZHHA
YUXcU3GUwj0ZYuUzFzYKMLu6uNNasNkN8h6SS2lF7Bm4JaKDW+mFMfCyJwdIVSEh
BGhHoVpXdxFysD9s6Mwxqrz3KKg1Jtp7idDkk0x2S2Eh+gxyiDQQokv0oQ3+0+HG
sgy5Iz2t2CkpS02/j+LOvAZljTmnD0bXu3srGR+25StsoDFP038Am3bfQwtD190=
=9jsT
-----END PGP SIGNATURE-----
Merge tag 'v0.31.2'
SECURITY UPDATE: Prevent unauthorised users from setting state events in a room
when there is no `m.room.power_levels` event in force in the room. (PR #3397 )
Discussion around the Matrix Spec change proposal for this change can be
followed at https://github.com/matrix-org/matrix-doc/issues/1304 .
2018-06-14 16:35:33 +01:00
Richard van der Hoff
7e1c616452
v0.31.2
2018-06-14 16:24:32 +01:00
Richard van der Hoff
61ab08a197
Merge pull request #3397 from matrix-org/rav/adjust_auth_rules
...
Adjust event auth rules when there is no PL event
2018-06-14 16:09:13 +01:00
Michael Wagner
19cd3120ec
Add error code to room creation error
...
This error code is mentioned in the documentation at https://matrix.org/docs/api/client-server/#!/Room32creation/createRoom
2018-06-14 14:08:40 +02:00
Richard van der Hoff
5c9afd6f80
Make default state_default 50
...
Make it so that, before there is a power-levels event in the room, you need a
power level of at least 50 to send state.
Partially addresses https://github.com/matrix-org/matrix-doc/issues/1192
2018-06-14 12:38:09 +01:00
Richard van der Hoff
52423607bd
Clarify interface for event_auth
...
stop pretending that it returns a boolean, which just almost gave me a heart
attack.
2018-06-14 12:26:17 +01:00
Amber Brown
f116f32ace
add a last seen metric ( #3396 )
2018-06-14 20:26:59 +10:00
Richard van der Hoff
557b686eac
Refactor get_send_level to take a power_levels event
...
it makes it easier for me to reason about
2018-06-14 11:26:27 +01:00
Amber Brown
a61738b316
Remove run_on_reactor ( #3395 )
2018-06-14 18:27:37 +10:00
Amber Brown
0fde1896cd
Merge pull request #3389 from turt2live/travis/name_metrics
...
Use the correct flag (enable_metrics) when warning about an incorrect metrics setup
2018-06-13 23:50:10 +10:00
Michael Telatynski
94700e55fa
if inviter_display_name == ""||None then default to inviter MXID
...
to prevent email invite from "None"
2018-06-13 10:31:01 +01:00
Travis Ralston
45768d1640
Use the RegistryProxy for appservices too
...
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-06-12 12:55:48 -06:00
Travis Ralston
12285a1a76
The flag is named enable_metrics, not collect_metrics
...
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-06-12 12:51:31 -06:00
Richard van der Hoff
b6faef2ad7
Filter out erased messages
...
Redact any messges sent by erased users.
2018-06-12 09:53:18 +01:00
Richard van der Hoff
f1023ebf4b
mark accounts as erased when requested
2018-06-12 09:53:18 +01:00
Richard van der Hoff
3ff8a619f5
UserErasureStore
...
to store which users have been erased
2018-06-12 09:52:22 +01:00
Richard van der Hoff
9fc5b74b24
simplify get_persisted_pdu
...
it doesn't make much sense to use get_persisted_pdu on the receive path: just
get the event straight from the store.
2018-06-12 09:51:31 +01:00
Richard van der Hoff
bd348f0af6
remove dead filter_events_for_clients
...
This is only used by filter_events_for_client, so we can simplify the whole
thing by just doing one user at a time, and removing a dead storage function to
boot.
2018-06-12 09:51:31 +01:00
Richard van der Hoff
eb32b2ca20
Optimise state_group_cache update
...
(1) matrix-org-hotfixes has removed the intern calls; let's do the same here.
(2) remove redundant iteritems() so we can used an optimised db update.
2018-06-11 22:56:11 +01:00
David Baker
187a546bff
Merge pull request #3276 from matrix-org/dbkr/unbind
...
Remove email addresses / phone numbers from ID servers when they're removed from synapse
2018-06-11 16:02:00 +01:00
Matthew Hodgson
d6cc369205
fix idiotic typo in state res
2018-06-11 14:43:55 +01:00
Matthew Hodgson
c96d882a02
Merge branch 'develop' into matthew/filter_members
2018-06-10 12:26:14 +03:00
Vincent Breitmoser
b800834351
add note that the affinity package is required for the cpu_affinity setting
2018-06-09 22:50:29 +02:00
Neil Johnson
ed5a0780a4
Merge branch 'master' into develop
2018-06-08 15:47:11 +01:00
Neil Johnson
1032393dfb
Changes in synapse v0.31.1 (2018-06-08)
...
=======================================
v0.31.1 fixes a security bug in the ``get_missing_events`` federation API
where event visibility rules were not applied correctly.
We are not aware of it being actively exploited but please upgrade asap.
Bug Fixes:
* Fix event filtering in get_missing_events handler (PR #3371 )
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEETQ1YthIGLQRddG54CTxDAAxPS/QFAlsalP8ACgkQCTxDAAxP
S/RADw/+NeDu0LjVpS5Uc4ElHgRBuFSm6l2i4z8rZBBlKSYnuq0Em4WMvLloi/JF
iAvTOYE7OjmF+gNvmsdH1N7hc1lKdQ2gAlpvaQR/5Qz9NtOVmM3WPZxS7n5jZHvD
hVSxeO9+GQOwK7rJorqrrsnWHQt0OkLHV6WThFdrgZb1JjWCUDTvw+Hei2uMX2aq
y2mkMG4TLStHwMvL2qw0h+hFtXywXI796qJR73ZxbEn24YD+kOXeEVkIFi2LT0Pj
cgkg7WWT32JD43/ioumLupZuhCmpRyxn4fi5gIKpXe5kiLsxOdApzNQSwmoJ+WA8
7zlrWY+0QDN4pbA5ESLitWSWAT50Ul//uM4nwmM4xEBPHdljXvKyHPsaSCeKLvT9
RT8pc41TQAqSshlXF8zgIAtStnF3oGel3EBBl1mmM9Un1ULnBFwWDZhlIm+ZZGhJ
MWoAWNG7j8AQuy0BTUAUr76x7t+/cdSqDuyVl1GO1tbDh0DUWoHZGXCUKrAXnn2T
SbiFigwOLvEADbvkW7L9Je9CVOi2V5Pg/32X9O8YMEiSz+j5PQEiGefyVh/I/QvV
Ha/atRpZF2OZ+XUOO5DLZMP/XCXpVgvHuskzfU6LvvVQCXgExuJhRD1PrRGeBaWr
zjJW+rmY+VeredqX7QkTB3XOGLMLGJfx2FeSMb+j0w3a7/iFj+Q=
=J80S
-----END PGP SIGNATURE-----
Merge tag 'v0.31.1'
Changes in synapse v0.31.1 (2018-06-08)
=======================================
v0.31.1 fixes a security bug in the ``get_missing_events`` federation API
where event visibility rules were not applied correctly.
We are not aware of it being actively exploited but please upgrade asap.
Bug Fixes:
* Fix event filtering in get_missing_events handler (PR #3371 )
2018-06-08 15:46:18 +01:00
Neil Johnson
0eb4722932
changelog a bump version
2018-06-08 15:21:46 +01:00
Richard van der Hoff
c6b1441c52
Fix event filtering in get_missing_events handler
2018-06-08 14:15:31 +01:00
David Baker
0e505b1913
Merge pull request #3372 from matrix-org/rav/better_verification_logging
...
Try to log more helpful info when a sig verification fails
2018-06-08 13:32:02 +01:00
Richard van der Hoff
e82db24a0e
Try to log more helpful info when a sig verification fails
...
Firstly, don't swallow the reason for the failure
Secondly, don't assume all exceptions are verification failures
Thirdly, log a bit of info about the key being used if debug is enabled
2018-06-08 12:13:08 +01:00
Richard van der Hoff
0834b49c6a
Fix event filtering in get_missing_events handler
2018-06-08 11:34:46 +01:00
Richard van der Hoff
8503dd0047
Remove event re-signing hacks
...
These "temporary fixes" have been here three and a half years, and I can't find
any events in the matrix.org database where the calculated signature differs
from what's in the db. It's time for them to go away.
2018-06-07 16:08:29 +01:00
Will Hunt
13d211edc1
Merge pull request #3344 from Half-Shot/hs/as-metrics
...
Add metrics to track appservice transactions
2018-06-07 11:37:12 +01:00
Richard van der Hoff
1152f495a0
Merge pull request #3363 from matrix-org/rav/fix_purge
...
Fix event-purge-by-ts admin API
2018-06-07 11:34:46 +01:00
Richard van der Hoff
e2acf536d4
Merge pull request #3355 from matrix-org/rav/fix_federation_backfill
...
Fix federation backfill from sqlite servers
2018-06-07 10:18:01 +01:00
Amber Brown
0160f6601f
Merge pull request #3356 from matrix-org/rav/add_missing_attr_dep
...
Make sure that attr is installed
2018-06-07 16:33:30 +10:00
Richard van der Hoff
f4caf3f83d
fix log
2018-06-07 00:26:38 +01:00
Richard van der Hoff
0546715c18
Fix event-purge-by-ts admin API
...
This got completely broken in 0.30.
Fixes #3300 .
2018-06-07 00:15:49 +01:00
Richard van der Hoff
57e3f923d2
Add missing dependency on attr
...
We've rcently added a dep on `attr`. I don't know why the CI didn't pick this
up, but we should make it explicit anyway.
2018-06-06 17:12:41 +01:00
Richard van der Hoff
d3a8c9c55e
Fix sql error in _get_state_groups_from_groups
...
If this was called with a `(type, None)` entry in types (which is supposed to
return all state of type `type`), it would explode with a sql error.
2018-06-06 14:19:01 +01:00
Neil Johnson
fef6c2cdcc
Merge branch 'master' into develop
2018-06-06 12:28:15 +01:00
Neil Johnson
752b7b32ed
Changes in synapse v0.31.0 (2018-06-06)
...
======================================
Most notable change from v0.30.0 is to switch to python prometheus library to improve system
stats reporting. WARNING this changes a number of prometheus metrics in a
backwards-incompatible manner. For more details, see
`docs/metrics-howto.rst <docs/metrics-howto.rst#removal-of-deprecated-metrics--time-based-counters-becoming-histograms-in-0310>`_.
Bug Fixes:
* Fix metric documentation tables (PR #3341 )
* Fix LaterGuage error handling (694968f
)
* Fix replication metrics (b7e7fd2
)
Changes in synapse v0.31.0-rc1 (2018-06-04)
==========================================
Features:
* Switch to the Python Prometheus library (PR #3256 , #3274 )
* Let users leave the server notice room after joining (PR #3287 )
Changes:
* daily user type phone home stats (PR #3264 )
* Use iter* methods for _filter_events_for_server (PR #3267 )
* Docs on consent bits (PR #3268 )
* Remove users from user directory on deactivate (PR #3277 )
* Avoid sending consent notice to guest users (PR #3288 )
* disable CPUMetrics if no /proc/self/stat (PR #3299 )
* Add local and loopback IPv6 addresses to url_preview_ip_range_blacklist (PR #3312 ) Thanks to @thegcat!
* Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy (PR #3307 )
* Add private IPv6 addresses to example config for url preview blacklist (PR #3317 ) Thanks to @thegcat!
* Reduce stuck read-receipts: ignore depth when updating (PR #3318 )
* Put python's logs into Trial when running unit tests (PR #3319 )
Changes, python 3 migration:
* Replace some more comparisons with six (PR #3243 ) Thanks to @NotAFile!
* replace some iteritems with six (PR #3244 ) Thanks to @NotAFile!
* Add batch_iter to utils (PR #3245 ) Thanks to @NotAFile!
* use repr, not str (PR #3246 ) Thanks to @NotAFile!
* Misc Python3 fixes (PR #3247 ) Thanks to @NotAFile!
* Py3 storage/_base.py (PR #3278 ) Thanks to @NotAFile!
* more six iteritems (PR #3279 ) Thanks to @NotAFile!
* More Misc. py3 fixes (PR #3280 ) Thanks to @NotAFile!
* remaining isintance fixes (PR #3281 ) Thanks to @NotAFile!
* py3-ize state.py (PR #3283 ) Thanks to @NotAFile!
* extend tox testing for py3 to avoid regressions (PR #3302 ) Thanks to @krombel!
* use memoryview in py3 (PR #3303 ) Thanks to @NotAFile!
Bugs:
* Fix federation backfill bugs (PR #3261 )
* federation: fix LaterGauge usage (PR #3328 ) Thanks to @intelfx!
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEETQ1YthIGLQRddG54CTxDAAxPS/QFAlsXxJAACgkQCTxDAAxP
S/SJTg//Wtr+Qop9LJh2/leAYXpyqW6P7Ftak0w3aJ3KL3+tYg32yYNoRADCqbp3
LkrHu8MwbZagHjRUyEWNfDk4jbfq5fwh0JVGmYuUKhG9aF0HYyytKkbW79YzuhdQ
dfHj9x0xSBOUvgt/husloZSDy0VHC6uyQSAFgFDyHS2y7RPAiGstqLGByv0ciZOk
pO7TdjkUQcx4Ps7Wgip31NuHy3GY2int6f540pUXoZHLXs7RkfqS2cpF9Z/sTXJ4
xDLiY7uYNsTcCblwqaiijY5c90xwRB2vLs5CJdKFgyB6PNgg/2wHJqP/WHHEj+F8
BoSm3Ts7NXQf23pP9CXICe7vXX3J+ruOnC7FOSRobr6KGjn6DUrIZxo1ZepTwpp9
DIq+1eOFKKjwLQM3Jdi8WBCP63LhYXrTZxreke3jpwdcD7oIO9v6/e9J1gU5xHWa
Izg+YnWn1JLfq/X8T7YTZddUXGGPkH5i6LZKKkyY8u7LkJ4WR7syuAceUzkOOIAq
UWO0uEV7IiLnZzZGTtNIlEqtuklmVQTm6bvAgTPabai2JQyngFpH5M/5mPpVSiLV
QRLwaM56c+5GGZJWef8vxdGeYn+8rFI/UUniJ7358kLJF2IHsxlZu8J0ZZO2HWI2
ze5Kz0AWRzXLhWzq62Qb2dsiGySrZ7hng1tDxLak2IiusY+9SjM=
=Mz9U
-----END PGP SIGNATURE-----
Merge tag 'v0.31.0'
Changes in synapse v0.31.0 (2018-06-06)
======================================
Most notable change from v0.30.0 is to switch to python prometheus library to improve system
stats reporting. WARNING this changes a number of prometheus metrics in a
backwards-incompatible manner. For more details, see
`docs/metrics-howto.rst <docs/metrics-howto.rst#removal-of-deprecated-metrics--time-based-counters-becoming-histograms-in-0310>`_.
Bug Fixes:
* Fix metric documentation tables (PR #3341 )
* Fix LaterGuage error handling (694968f
)
* Fix replication metrics (b7e7fd2
)
Changes in synapse v0.31.0-rc1 (2018-06-04)
==========================================
Features:
* Switch to the Python Prometheus library (PR #3256 , #3274 )
* Let users leave the server notice room after joining (PR #3287 )
Changes:
* daily user type phone home stats (PR #3264 )
* Use iter* methods for _filter_events_for_server (PR #3267 )
* Docs on consent bits (PR #3268 )
* Remove users from user directory on deactivate (PR #3277 )
* Avoid sending consent notice to guest users (PR #3288 )
* disable CPUMetrics if no /proc/self/stat (PR #3299 )
* Add local and loopback IPv6 addresses to url_preview_ip_range_blacklist (PR #3312 ) Thanks to @thegcat!
* Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy (PR #3307 )
* Add private IPv6 addresses to example config for url preview blacklist (PR #3317 ) Thanks to @thegcat!
* Reduce stuck read-receipts: ignore depth when updating (PR #3318 )
* Put python's logs into Trial when running unit tests (PR #3319 )
Changes, python 3 migration:
* Replace some more comparisons with six (PR #3243 ) Thanks to @NotAFile!
* replace some iteritems with six (PR #3244 ) Thanks to @NotAFile!
* Add batch_iter to utils (PR #3245 ) Thanks to @NotAFile!
* use repr, not str (PR #3246 ) Thanks to @NotAFile!
* Misc Python3 fixes (PR #3247 ) Thanks to @NotAFile!
* Py3 storage/_base.py (PR #3278 ) Thanks to @NotAFile!
* more six iteritems (PR #3279 ) Thanks to @NotAFile!
* More Misc. py3 fixes (PR #3280 ) Thanks to @NotAFile!
* remaining isintance fixes (PR #3281 ) Thanks to @NotAFile!
* py3-ize state.py (PR #3283 ) Thanks to @NotAFile!
* extend tox testing for py3 to avoid regressions (PR #3302 ) Thanks to @krombel!
* use memoryview in py3 (PR #3303 ) Thanks to @NotAFile!
Bugs:
* Fix federation backfill bugs (PR #3261 )
* federation: fix LaterGauge usage (PR #3328 ) Thanks to @intelfx!
2018-06-06 12:27:33 +01:00
Neil Johnson
61134debdc
bump version and changelog
2018-06-06 11:26:21 +01:00
Richard van der Hoff
ad459a106c
Merge pull request #3349 from t3chguy/redact_as_request_token
...
Redact AS tokens in log (fixes to #3327 )
2018-06-06 10:58:07 +01:00
Michael Telatynski
592c162516
also redact __str__ of ApplicationService used for logging
2018-06-06 10:35:29 +01:00
Michael Telatynski
330432031b
redact_uri in two missed log paths
2018-06-06 10:25:48 +01:00
David Baker
bf54c1cf6c
pep8
2018-06-06 10:15:33 +01:00
David Baker
3e4bc4488c
More doc fixes
2018-06-06 09:44:10 +01:00
Richard van der Hoff
592ee217a3
Merge commit 'b7e7fd2' into release-v0.31.0
2018-06-06 07:02:02 +01:00
Ivan Shapovalov
c88d50aa8f
federation/send_queue.py: fix usage of sortedcontainers.SortedDict
2018-06-06 02:45:18 +03:00
Richard van der Hoff
e316407b5d
Merge pull request #3327 from t3chguy/redact_as_request_token
...
Strip `access_token` from outgoing requests
2018-06-05 19:08:46 +01:00
Michael Telatynski
e6cbf47773
factor out uri redaction into a method on http
2018-06-05 18:31:40 +01:00
David Baker
607bd27c83
fix pep8
2018-06-05 18:10:35 +01:00
David Baker
d62162bbec
doc fixes
2018-06-05 18:09:13 +01:00
Richard van der Hoff
522bd3c8a3
Merge remote-tracking branch 'origin/master' into develop
2018-06-05 17:42:49 +01:00
Will Hunt
d6e3c2c79b
Let's try labels instead of label, that might work
2018-06-05 17:30:45 +01:00
Amber Brown
f7869f8f8b
Port to sortedcontainers (with tests!) ( #3332 )
2018-06-06 00:13:57 +10:00
Will Hunt
604cff1a06
Add metrics to track appservice transactions
2018-06-05 13:21:30 +01:00
Richard van der Hoff
f29b41fde9
Merge pull request #3324 from matrix-org/rav/remove_dead_method
...
Remove was_forgotten_at
2018-06-04 18:18:08 +01:00
Richard van der Hoff
28b0490dfd
Merge pull request #3334 from matrix-org/rav/cache_factor_override
...
Cache factor override system for specific caches
2018-06-04 17:19:33 +01:00
Richard van der Hoff
b7e7fd2d0e
Fix replication metrics
...
fix bug introduced in #3256
2018-06-04 16:23:05 +01:00
Neil Johnson
244ab974e7
bump version and changelog
2018-06-04 16:09:58 +01:00
Richard van der Hoff
694968fa81
Hopefully, fix LaterGuage error handling
2018-06-04 15:59:14 +01:00
Erik Johnston
042eedfa2b
Add hacky cache factor override system
2018-06-04 15:39:28 +01:00
David Baker
c5930d513a
Docstring
2018-06-04 12:05:58 +01:00
David Baker
6a29e815fc
Fix comment
2018-06-04 12:01:23 +01:00
David Baker
e44150a6de
Missing yield
2018-06-04 12:01:13 +01:00
David Baker
f731e42baf
docstring
2018-06-04 12:00:51 +01:00
Matthew Hodgson
28f09fcdd5
Merge branch 'develop' into matthew/filter_members
2018-06-04 00:09:17 +03:00
Matthew Hodgson
5f6122fe10
more comments
2018-06-04 00:08:52 +03:00
Ivan Shapovalov
7d9d75e4e8
federation/send_queue.py: fix usage of LaterGauge
...
Fixes a startup crash due to commit df9f72d9e5
"replacing portions".
2018-06-03 14:16:17 +03:00
Michael Telatynski
09503126df
Strip access_token
from outgoing requests using existing regex
2018-06-02 23:25:13 +01:00
Richard van der Hoff
c1f4118bb6
Remove was_forgotten_at
...
This is unused. IT MUST DIE!!!1
̧̪͈̱̹̳͖͙H̵̰̤̰͕̖e̛ ͚͉̗̼̞w̶̩̥͉̮h̩̺̪̩͘ͅọ͎͉̟ ̜̩͔̦̘ͅW̪̫̩̣̲͔̳a͏͔̳͖i͖͜t͓̤̠͓͙s̘̰̩̥̙̝ͅ ̲̠̬̥Be̡̙̫̦h̰̩i̛̫͙͔̭̤̗̲n̳͞d̸ ͎̻͘T̛͇̝̲̹̠̗ͅh̫̦̝ͅe̩̫͟ ͓͖̼W͕̳͎͚̙̥ą̙l̘͚̺͔͞ͅl̳͍̙̤̤̮̳.̢
̟̺̜̙͉Z̤̲̙̙͎̥̝A͎̣͔̙͘L̥̻̗̳̻̳̳͢G͉̖̯͓̞̩̦O̹̹̺!̙͈͎̞̬ *
2018-06-01 18:21:49 +01:00
Richard van der Hoff
a9e97dcd65
Merge pull request #3317 from thegcat/feature/3312-add_ipv6_to_blacklist_example_config
...
Add private IPv6 addresses to example config for url preview blacklist
2018-06-01 14:45:14 +01:00
Neil Johnson
71477f3317
Merge pull request #3264 from matrix-org/neil/sign-up-stats
...
daily user type phone home stats
2018-06-01 13:42:01 +00:00
Richard van der Hoff
9f797a24a4
Handle RRs which arrive before their events
2018-06-01 14:01:43 +01:00
Richard van der Hoff
857e6fd8b6
Ignore depth when updating read-receipts
...
Order read receipts by stream ordering instead of depth
2018-06-01 12:18:11 +01:00
Felix Schäfer
4ef76f3ac4
Add private IPv6 addresses to preview blacklist #3312
...
The added addresses are expected to be local or loopback addresses and
shouldn't be spidered for previews.
Signed-off-by: Felix Schäfer <felix@thegcat.net>
2018-06-01 12:18:35 +02:00
Neil Johnson
4986b084f8
remove unnecessary INSERT
2018-06-01 10:50:40 +01:00
Richard van der Hoff
7e15410f02
Enforce the specified API for report_event
...
as per
https://matrix.org/docs/spec/client_server/unstable.html#post-matrix-client-r0-rooms-roomid-report-eventid
2018-05-31 18:17:11 +01:00
Amber Brown
febe0ec8fd
Run Prometheus on a different port, optionally. ( #3274 )
2018-05-31 19:04:50 +10:00
Amber Brown
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
Richard van der Hoff
219c2a322b
remove trailing whitespace
2018-05-30 19:42:19 +01:00
Richard van der Hoff
2e4be8bfd9
fix english and wrap comment
2018-05-30 19:24:12 +01:00
Amber Brown
872cf43516
Merge pull request #3303 from NotAFile/py3-memoryview
...
use memoryview in py3
2018-05-30 12:51:42 +10:00
Amber Brown
debff7ae09
Merge pull request #3281 from NotAFile/py3-six-isinstance
...
remaining isintance fixes
2018-05-30 12:44:46 +10:00
Richard van der Hoff
34b85df7f5
Update some comments and docstrings in SyncHandler
2018-05-29 22:31:18 +01:00
Richard van der Hoff
4a9cbdbc15
Exempt AS-registered users from doing gdpr
2018-05-29 19:54:32 +01:00
Neil Johnson
ab0ef31dc7
create users index on creation_ts
2018-05-29 17:51:08 +01:00
Neil Johnson
558f3d376a
create index in background
2018-05-29 17:47:55 +01:00
Neil Johnson
c379acd4fd
bump version
2018-05-29 17:47:28 +01:00
Adrian Tschira
4b9d0cde97
add remaining memoryview changes
2018-05-29 17:42:43 +02:00
Adrian Tschira
7873cde526
pep8
2018-05-29 17:35:55 +02:00
Adrian Tschira
1afafb3497
use memoryview in py3
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-29 17:14:34 +02:00
Matthew Hodgson
9bbb9f5556
add lazy_load_members to the filter json schema
2018-05-29 04:26:10 +01:00
Matthew Hodgson
8df7bad839
pep8
2018-05-29 02:49:36 +01:00
Matthew Hodgson
ff1bc0a279
pep8
2018-05-29 02:32:15 +01:00
Matthew Hodgson
0a240ad36e
disable CPUMetrics if no /proc/self/stat
...
fixes build on macOS again
2018-05-29 02:23:30 +01:00
Matthew Hodgson
b69ff33d9e
disable CPUMetrics if no /proc/self/stat
...
fixes build on macOS again
2018-05-29 02:22:27 +01:00
Matthew Hodgson
a6c8f7c875
add pydoc
2018-05-29 01:09:55 +01:00
Matthew Hodgson
7a6df013cc
merge develop
2018-05-29 00:25:22 +01:00
Amber Brown
57ad76fa4a
fix up tests
2018-05-28 19:51:53 +10:00
Amber Brown
3ef5cd74a6
update to more consistently use seconds in any metrics or logging
2018-05-28 19:39:27 +10:00
Amber Brown
5c40ce3777
invalid syntax :(
2018-05-28 19:16:09 +10:00
Amber Brown
357c74a50f
add comment about why unreg
2018-05-28 19:14:41 +10:00
Amber Brown
a2eb5db4a0
update metrics to be in seconds
2018-05-28 19:10:27 +10:00
Amber Brown
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
Ruben Barkow
08ea5fe635
add link to thorough instruction how to configure consent
2018-05-25 23:19:55 +02:00
Richard van der Hoff
66bdae986f
Fix default for send_server_notice_to_guests
...
bool("False") == True...
2018-05-25 11:42:05 +01:00
Richard van der Hoff
ba1b163590
Avoid sending consent notice to guest users
...
we think it makes sense not to send the notices to guest users.
2018-05-25 11:36:43 +01:00
Richard van der Hoff
41921ac01b
Merge pull request #3287 from matrix-org/rav/allow_leaving_server_notices_room
...
Let users leave the server notice room after joining
2018-05-25 11:15:45 +01:00
Richard van der Hoff
757ed27258
Let users leave the server notice room after joining
...
They still can't reject invites, but we let them leave it.
2018-05-25 11:07:21 +01:00
Adrian Tschira
4ee4450d66
fix recursion error
2018-05-24 21:44:10 +02:00
Amber Brown
9c36c150e7
Merge pull request #3283 from NotAFile/py3-state
...
py3-ize state.py
2018-05-24 14:23:33 -05:00
Amber Brown
cc1349c06a
Merge pull request #3279 from NotAFile/py3-more-iteritems
...
more six iteritems
2018-05-24 14:23:13 -05:00
Adrian Tschira
0e61705661
py3-ize state.py
2018-05-24 20:59:00 +02:00
Adrian Tschira
dd068ca979
remaining isintance fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:55:08 +02:00
Adrian Tschira
17a70cf6e9
Misc. py3 fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:20:33 +02:00
Adrian Tschira
6c16a4ec1b
more iteritems
2018-05-24 20:19:06 +02:00
Amber Brown
7ea07c7305
Merge pull request #3278 from NotAFile/py3-storage-base
...
Py3 storage/_base.py
2018-05-24 13:08:09 -05:00
Amber Brown
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4
...
replace some iteritems with six
2018-05-24 13:04:07 -05:00
Amber Brown
c4fb15a06c
Merge pull request #3246 from NotAFile/py3-repr-string
...
use repr, not str
2018-05-24 13:00:20 -05:00
Amber Brown
36501068d8
Merge pull request #3247 from NotAFile/py3-misc
...
Misc Python3 fixes
2018-05-24 12:58:37 -05:00
Amber Brown
2aff6eab6d
Merge pull request #3245 from NotAFile/batch-iter
...
Add batch_iter to utils
2018-05-24 12:54:12 -05:00
Adrian Tschira
095292304f
Py3 storage/_base.py
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 18:24:12 +02:00
David Baker
77a23e2e05
Merge remote-tracking branch 'origin/develop' into dbkr/unbind
2018-05-24 16:20:53 +01:00
David Baker
ecc4b88bd1
Merge pull request #3277 from matrix-org/dbkr/remove_from_user_dir
...
Remove users from user directory on deactivate
2018-05-24 16:12:12 +01:00
Erik Johnston
46345187cc
Merge pull request #3243 from NotAFile/py3-six-3
...
Replace some more comparisons with six
2018-05-24 16:08:57 +01:00
Neil Johnson
037c6db85d
Merge branch 'master' into develop
2018-05-24 16:03:44 +01:00
David Baker
7a1af504d7
Remove users from user directory on deactivate
2018-05-24 15:59:58 +01:00
Erik Johnston
f72d5a44d5
Merge pull request #3261 from matrix-org/erikj/pagination_fixes
...
Fix federation backfill bugs
2018-05-24 14:52:03 +01:00
Erik Johnston
68399fc4de
Merge pull request #3267 from matrix-org/erikj/iter_filter
...
Use iter* methods for _filter_events_for_server
2018-05-24 14:44:57 +01:00
Neil Johnson
91d95a1d8e
bump version
2018-05-24 14:05:07 +01:00
David Baker
9700d15611
pep8
2018-05-24 11:23:15 +01:00
David Baker
a21a41bad7
comment
2018-05-24 11:19:59 +01:00
David Baker
b3bff53178
Unbind 3pids when they're deleted too
2018-05-24 11:08:05 +01:00
Richard van der Hoff
6abcb5d22d
Merge pull request #3273 from matrix-org/rav/server_notices_avatar_url
...
Allow overriding the server_notices user's avatar
2018-05-24 10:31:43 +01:00
Amber Brown
389dac2c15
pepeightttt
2018-05-23 13:08:59 -05:00
Amber Brown
472a5ec4e2
add back CPU metrics
2018-05-23 13:03:56 -05:00
Amber Brown
e987079037
fixes
2018-05-23 13:03:51 -05:00
Richard van der Hoff
9bf4b2bda3
Allow overriding the server_notices user's avatar
...
probably should have done this in the first place, like @turt2live suggested.
2018-05-23 17:43:30 +01:00
Richard van der Hoff
a0b3946fe2
Merge branch 'release-v0.30.0' into rav/localpart_in_consent_uri
2018-05-23 16:06:03 +01:00
Richard van der Hoff
2f7008d4eb
Merge pull request #3271 from matrix-org/rav/consent_uri_in_messages
...
Support for putting %(consent_uri)s in messages
2018-05-23 16:04:30 +01:00
Richard van der Hoff
dda40fb55d
fix typo
2018-05-23 15:30:26 +01:00
Richard van der Hoff
3ff6f50eac
Use the localpart in the consent uri
...
... because it's shorter.
2018-05-23 15:28:23 +01:00
Richard van der Hoff
82191b08f6
Support for putting %(consent_uri)s in messages
...
Make it possible to put the URI in the error message and the server notice that
get sent by the server
2018-05-23 15:24:31 +01:00
David Baker
2c7866d664
Hit the 3pid unbind endpoint on deactivation
2018-05-23 14:38:56 +01:00
Richard van der Hoff
321f02d263
Block attempts to send server notices to remote users
2018-05-23 14:30:47 +01:00
Neil Johnson
3601a240aa
bump version and changelog
2018-05-23 12:03:23 +01:00
Erik Johnston
5aaa3189d5
s/values/itervalues/
2018-05-23 10:13:05 +01:00
Erik Johnston
0a4bca4134
Use iter* methods for _filter_events_for_server
2018-05-23 10:04:23 +01:00
Amber Brown
b6063631c3
more cleanup
2018-05-22 17:36:20 -05:00
Amber Brown
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
Amber Brown
071206304d
cleanup pep8 errors
2018-05-22 16:54:22 -05:00
Amber Brown
85ba83eb51
fixes
2018-05-22 16:28:23 -05:00
Amber Brown
228f1f584e
fix the test failures
2018-05-22 15:02:38 -05:00
Erik Johnston
e85b5a0ff7
Use iter* methods
2018-05-22 19:02:48 +01:00
Erik Johnston
586b66b197
Fix that states is a dict of dicts
2018-05-22 19:02:36 +01:00
Erik Johnston
a17e901f4d
Remove unused string formatting param
2018-05-22 18:24:32 +01:00
Erik Johnston
5494c1d71e
Don't support limitless pagination
...
The pagination storage function supported not specifiying a limit on the
number of events returned. This was triggered when using the search or
context API with a limit of zero, which the storage function took to
mean not being limited.
2018-05-22 18:15:21 +01:00
Neil Johnson
d8cb7225d2
daily user type phone home stats
2018-05-22 18:09:09 +01:00
hera
ad2823ee27
fix synchrotron
2018-05-22 17:47:42 +01:00
Richard van der Hoff
08bfc48abf
custom error code for not leaving server notices room
2018-05-22 17:27:27 +01:00
David Baker
0a078026ea
comment typo
2018-05-22 17:14:06 +01:00
Amber Brown
8f5a688d42
cleanups, self-registration
2018-05-22 10:56:03 -05:00
Amber Brown
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
Erik Johnston
cb2a2ad791
get_domains_from_state returns list of tuples
2018-05-22 16:23:39 +01:00
Richard van der Hoff
08a14b32ae
Merge pull request #3262 from matrix-org/rav/has_already_consented
...
Add a 'has_consented' template var to consent forms
2018-05-22 15:35:10 +01:00
Richard van der Hoff
7b36d06a69
Add a 'has_consented' template var to consent forms
...
fixes #3260
2018-05-22 14:58:34 +01:00
Richard van der Hoff
669400e22f
Enable auto-escaping for the consent templates
...
... to reduce the risk of somebody introducing an html injection attack...
2018-05-22 14:58:34 +01:00
Richard van der Hoff
b5b2d5d64b
Fix dependency on jinja2
...
Delay the import of ConsentResource, so that we can get away without jinja2 if
people don't have the consent resource enabled.
Fixes #3259
2018-05-22 14:03:45 +01:00
Richard van der Hoff
a5e2941aad
Reject attempts to send event before privacy consent is given
...
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252 ) if consent is not yet
given.
2018-05-22 12:00:47 +01:00
Richard van der Hoff
8aeb529262
Merge pull request #3236 from matrix-org/rav/consent_notice
...
Send users a server notice about consent
2018-05-22 11:56:09 +01:00
Richard van der Hoff
8810685df9
Stub out ServerNoticesSender on the workers
...
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
2018-05-22 11:54:51 +01:00
Richard van der Hoff
d5dca9a04f
Move consent config parsing into ConsentConfig
...
turns out we need to reuse this, so it's better in the config class.
2018-05-22 11:54:51 +01:00
Richard van der Hoff
9ea219c514
Send users a server notice about consent
...
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
2018-05-22 11:54:51 +01:00
Richard van der Hoff
d14d7b8fdc
Rename 'version' param on user consent config
...
we're going to use it for the version we require too.
2018-05-22 11:54:51 +01:00
Erik Johnston
7948ecf234
Comment
2018-05-22 11:39:43 +01:00
Erik Johnston
020377a550
Fix logcontext resource usage tracking
2018-05-22 11:16:07 +01:00
Erik Johnston
13a8dfba0d
Merge pull request #3252 from matrix-org/erikj/in_flight_requests
...
Add in flight request metrics
2018-05-22 09:54:30 +01:00
Erik Johnston
c435b0b441
Don't store context
2018-05-22 09:34:26 +01:00
Erik Johnston
fb2806b186
Move in_flight_requests_count to be a callback metric
2018-05-22 09:31:53 +01:00
Amber Brown
fcc525b0b7
rest of the changes
2018-05-21 19:48:57 -05:00
Amber Brown
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
Amber Brown
c60e0d5e02
don't need the resource portion
2018-05-21 17:03:20 -05:00
Amber Brown
02c1d29133
look at the Prometheus metrics instead
2018-05-21 17:02:20 -05:00
Amber Brown
f258deffcb
remove old metrics libs
2018-05-21 17:01:15 -05:00
Richard van der Hoff
413482f578
Merge pull request #3255 from matrix-org/rav/fix_transactions
...
Stop the transaction cache caching failures
2018-05-21 17:58:08 +01:00
Richard van der Hoff
6e1cb54a05
Fix logcontext leak in HttpTransactionCache
...
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
2018-05-21 16:58:20 +01:00
Richard van der Hoff
6d6e7288fe
Stop the transaction cache caching failures
...
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
2018-05-21 16:49:59 +01:00
Erik Johnston
dfa70adc33
Add in flight request metrics
...
This tracks CPU and DB usage while requests are in flight, rather than
when we write the response.
2018-05-21 16:23:06 +01:00
Adrian Tschira
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
Adrian Tschira
d9fe2b2d9d
Replace some more comparisons with six
...
plus a bonus b"" string I missed last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
Adrian Tschira
45b55e23d3
Add batch_iter to utils
...
There's a frequent idiom I noticed where an iterable is split up into a
number of chunks/batches. Unfortunately that method does not work with
iterators like dict.keys() in python3. This implementation works with
iterators.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:48:30 +02:00
Adrian Tschira
dcc235b47d
use stand-in value if maxint is not available
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:44 +02:00
Adrian Tschira
73cbdef5f7
fix py3 intern and remove unnecessary py3 encode
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:31 +02:00
Adrian Tschira
aafb0f6b0d
py3-ize url preview
2018-05-19 17:35:20 +02:00
Adrian Tschira
b932b4ea25
use repr, not str
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:28:42 +02:00
Neil Johnson
644aac5f73
Tighter filtering for user_daily_visits
2018-05-18 17:10:35 +01:00
Neil Johnson
ef466b3a13
fix psql compatability bug
2018-05-18 15:51:21 +01:00
Neil Johnson
2725223f08
Merge branch 'master' into develop
2018-05-18 14:07:50 +01:00
Richard van der Hoff
6d9dc67139
Merge pull request #3232 from matrix-org/rav/server_notices_room
...
Infrastructure for a server notices room
2018-05-18 11:28:04 +01:00
Richard van der Hoff
ed3125b0a1
Merge pull request #3235 from matrix-org/rav/fix_receipts_deferred
...
Fix error in handling receipts
2018-05-18 11:23:11 +01:00
Richard van der Hoff
67af392712
Merge pull request #3233 from matrix-org/rav/remove_dead_code
...
Remove unused `update_external_syncs`
2018-05-18 11:22:43 +01:00
Richard van der Hoff
011e1f4010
Better docstrings
2018-05-18 11:22:12 +01:00
Richard van der Hoff
26305788fe
Make sure we reject attempts to invite the notices user
2018-05-18 11:18:39 +01:00
Richard van der Hoff
d10707c810
Replace inline docstrings with "Attributes" in class docstring
2018-05-18 11:00:55 +01:00
Erik Johnston
fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token
...
Make purge_history operate on tokens
2018-05-18 10:35:23 +01:00
Richard van der Hoff
8b1c856d81
Fix error in handling receipts
...
Fixes an error which has been happening ever since #2158 (v0.21.0-rc1):
> TypeError: argument of type 'ObservableDeferred' is not iterable
fixes #3234
2018-05-18 09:15:35 +01:00
Neil Johnson
6958459b50
bump version, change log
2018-05-17 21:35:07 +01:00
Richard van der Hoff
88d3405332
fix missing yield for server_notices_room
2018-05-17 18:33:45 +01:00
Richard van der Hoff
d43d480d86
Remove unused update_external_syncs
...
This method isn't used anywhere. Burninate it.
2018-05-17 18:22:19 +01:00
Richard van der Hoff
fed62e21ad
Infrastructure for a server notices room
...
Server Notices use a special room which the user can't dismiss. They are
created on demand when some other bit of the code calls send_notice.
(This doesn't actually do much yet becuse we don't call send_notice anywhere)
2018-05-17 17:58:25 +01:00
Richard van der Hoff
f8a1e76d64
Merge pull request #3225 from matrix-org/rav/move_creation_handler
...
Move RoomCreationHandler out of synapse.handlers.Handlers
2018-05-17 17:56:42 +01:00
Erik Johnston
f7906203f6
Merge pull request #3212 from matrix-org/erikj/epa_stream
...
Use stream rather depth ordering for push actions
2018-05-17 12:01:21 +01:00
Richard van der Hoff
c46367d0d7
Move RoomCreationHandler out of synapse.handlers.Handlers
...
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.
Also fix the docstrings on create_room.
2018-05-17 09:08:42 +01:00
Neil Johnson
85b8acdeb4
Merge branch 'master' into develop
2018-05-16 16:48:10 +01:00
Neil Johnson
3c099219e0
bump version and changelog for 0.29.0
2018-05-16 15:44:59 +01:00
Erik Johnston
680530cc7f
Clarify comment
2018-05-16 11:47:29 +01:00
Erik Johnston
43e6e82c4d
Comments
2018-05-16 11:13:31 +01:00
Neil Johnson
dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics
...
user visit data
2018-05-16 10:09:24 +00:00
Erik Johnston
c945af8799
Move and rename variable
2018-05-16 10:52:06 +01:00
Neil Johnson
be11a02c4f
remove empty line
2018-05-16 10:45:40 +01:00
Neil Johnson
a2204cc9cc
remove unused method recurring_user_daily_visit_stats
2018-05-16 09:47:20 +01:00
Neil Johnson
31c2502ca8
style and further contraining query
2018-05-16 09:46:43 +01:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler
...
ConsentResource to gather policy consent from users
2018-05-16 07:19:18 +01:00
Neil Johnson
c92a8aa578
pep8
2018-05-15 17:31:11 +01:00
Neil Johnson
05ac15ae82
Limit query load of generate_user_daily_visits
...
The aim is to keep track of when it was last called and only query from that point in time
2018-05-15 17:01:33 +01:00
Erik Johnston
5f27ed75ad
Make purge_history operate on tokens
...
As we're soon going to change how topological_ordering works
2018-05-15 16:23:50 +01:00
Erik Johnston
37dbee6490
Use events_to_purge table rather than token
2018-05-15 16:23:47 +01:00
Richard van der Hoff
47815edcfa
ConsentResource to gather policy consent from users
...
Hopefully there are enough comments and docs in this that it makes sense on its
own.
2018-05-15 15:11:59 +01:00
Neil Johnson
aea80a0118
v0.29.0-rc1: bump version and change log
2018-05-14 15:50:57 +01:00
Neil Johnson
f077e97914
instead of inserting user daily visit data at the end of the day, instead insert incrementally through the day
2018-05-14 13:50:58 +01:00
David Baker
8cbbfd16fb
Merge pull request #3201 from matrix-org/dbkr/leave_rooms_on_deactivate
...
Part user from rooms on account deactivate
2018-05-14 11:31:48 +01:00
Neil Johnson
977765bde2
Merge branch 'develop' of https://github.com/matrix-org/synapse into cohort_analytics
2018-05-14 09:31:42 +01:00
Richard van der Hoff
c25d7ba12e
Merge pull request #3208 from matrix-org/rav/more_refactor_request_handler
...
Set Server header in SynapseRequest
2018-05-11 16:07:47 +01:00
Erik Johnston
6406b70aeb
Use stream rather depth ordering for push actions
...
This simplifies things as it is, but will also allow us to change the
way we traverse topologically without having to update the way push
actions work.
2018-05-11 15:30:11 +01:00
Richard van der Hoff
23e2dfe940
Merge pull request #3209 from damir-manapov/master
...
transaction_id, destination defined twice
2018-05-11 00:35:13 +01:00
Damir Manapov
db18d854cd
transaction_id, destination twice
2018-05-10 22:13:31 +03:00
Richard van der Hoff
318711e139
Set Server header in SynapseRequest
...
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).
This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.
Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
2018-05-10 18:50:27 +01:00
David Baker
6b49628e3b
Catch failure to part user from room
2018-05-10 12:23:53 +01:00
David Baker
217bc53c98
Many docstrings
2018-05-10 12:20:40 +01:00
Richard van der Hoff
645cb4bf06
Remove redundant request_handler decorator
...
This is needless complexity; we might as well use the wrapper directly.
Also rename wrap_request_handler->wrap_json_request_handler.
2018-05-10 12:19:53 +01:00
Richard van der Hoff
09f570b935
Factor wrap_request_handler_with_logging out of wrap_request_handler
...
... so that it can be used on non-JSON endpoints
2018-05-10 12:19:52 +01:00
Richard van der Hoff
9589a1925e
Remove include_metrics param
...
The metrics are now available via the request, so this is redundant and can go
away at last.
2018-05-10 12:19:52 +01:00