Ivan Shapovalov
7d9d75e4e8
federation/send_queue.py: fix usage of LaterGauge
...
Fixes a startup crash due to commit df9f72d9e5
"replacing portions".
2018-06-03 14:16:17 +03:00
Michael Telatynski
09503126df
Strip access_token
from outgoing requests using existing regex
2018-06-02 23:25:13 +01:00
Richard van der Hoff
c1f4118bb6
Remove was_forgotten_at
...
This is unused. IT MUST DIE!!!1
̧̪͈̱̹̳͖͙H̵̰̤̰͕̖e̛ ͚͉̗̼̞w̶̩̥͉̮h̩̺̪̩͘ͅọ͎͉̟ ̜̩͔̦̘ͅW̪̫̩̣̲͔̳a͏͔̳͖i͖͜t͓̤̠͓͙s̘̰̩̥̙̝ͅ ̲̠̬̥Be̡̙̫̦h̰̩i̛̫͙͔̭̤̗̲n̳͞d̸ ͎̻͘T̛͇̝̲̹̠̗ͅh̫̦̝ͅe̩̫͟ ͓͖̼W͕̳͎͚̙̥ą̙l̘͚̺͔͞ͅl̳͍̙̤̤̮̳.̢
̟̺̜̙͉Z̤̲̙̙͎̥̝A͎̣͔̙͘L̥̻̗̳̻̳̳͢G͉̖̯͓̞̩̦O̹̹̺!̙͈͎̞̬ *
2018-06-01 18:21:49 +01:00
Richard van der Hoff
a9e97dcd65
Merge pull request #3317 from thegcat/feature/3312-add_ipv6_to_blacklist_example_config
...
Add private IPv6 addresses to example config for url preview blacklist
2018-06-01 14:45:14 +01:00
Neil Johnson
71477f3317
Merge pull request #3264 from matrix-org/neil/sign-up-stats
...
daily user type phone home stats
2018-06-01 13:42:01 +00:00
Richard van der Hoff
9f797a24a4
Handle RRs which arrive before their events
2018-06-01 14:01:43 +01:00
Richard van der Hoff
857e6fd8b6
Ignore depth when updating read-receipts
...
Order read receipts by stream ordering instead of depth
2018-06-01 12:18:11 +01:00
Felix Schäfer
4ef76f3ac4
Add private IPv6 addresses to preview blacklist #3312
...
The added addresses are expected to be local or loopback addresses and
shouldn't be spidered for previews.
Signed-off-by: Felix Schäfer <felix@thegcat.net>
2018-06-01 12:18:35 +02:00
Neil Johnson
4986b084f8
remove unnecessary INSERT
2018-06-01 10:50:40 +01:00
Amber Brown
febe0ec8fd
Run Prometheus on a different port, optionally. ( #3274 )
2018-05-31 19:04:50 +10:00
Amber Brown
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
Richard van der Hoff
219c2a322b
remove trailing whitespace
2018-05-30 19:42:19 +01:00
Richard van der Hoff
2e4be8bfd9
fix english and wrap comment
2018-05-30 19:24:12 +01:00
Amber Brown
872cf43516
Merge pull request #3303 from NotAFile/py3-memoryview
...
use memoryview in py3
2018-05-30 12:51:42 +10:00
Amber Brown
debff7ae09
Merge pull request #3281 from NotAFile/py3-six-isinstance
...
remaining isintance fixes
2018-05-30 12:44:46 +10:00
Richard van der Hoff
34b85df7f5
Update some comments and docstrings in SyncHandler
2018-05-29 22:31:18 +01:00
Richard van der Hoff
4a9cbdbc15
Exempt AS-registered users from doing gdpr
2018-05-29 19:54:32 +01:00
Neil Johnson
ab0ef31dc7
create users index on creation_ts
2018-05-29 17:51:08 +01:00
Neil Johnson
558f3d376a
create index in background
2018-05-29 17:47:55 +01:00
Neil Johnson
c379acd4fd
bump version
2018-05-29 17:47:28 +01:00
Adrian Tschira
4b9d0cde97
add remaining memoryview changes
2018-05-29 17:42:43 +02:00
Adrian Tschira
7873cde526
pep8
2018-05-29 17:35:55 +02:00
Adrian Tschira
1afafb3497
use memoryview in py3
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-29 17:14:34 +02:00
Matthew Hodgson
ff1bc0a279
pep8
2018-05-29 02:32:15 +01:00
Matthew Hodgson
0a240ad36e
disable CPUMetrics if no /proc/self/stat
...
fixes build on macOS again
2018-05-29 02:23:30 +01:00
Amber Brown
57ad76fa4a
fix up tests
2018-05-28 19:51:53 +10:00
Amber Brown
3ef5cd74a6
update to more consistently use seconds in any metrics or logging
2018-05-28 19:39:27 +10:00
Amber Brown
5c40ce3777
invalid syntax :(
2018-05-28 19:16:09 +10:00
Amber Brown
357c74a50f
add comment about why unreg
2018-05-28 19:14:41 +10:00
Amber Brown
a2eb5db4a0
update metrics to be in seconds
2018-05-28 19:10:27 +10:00
Amber Brown
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
Ruben Barkow
08ea5fe635
add link to thorough instruction how to configure consent
2018-05-25 23:19:55 +02:00
Richard van der Hoff
66bdae986f
Fix default for send_server_notice_to_guests
...
bool("False") == True...
2018-05-25 11:42:05 +01:00
Richard van der Hoff
ba1b163590
Avoid sending consent notice to guest users
...
we think it makes sense not to send the notices to guest users.
2018-05-25 11:36:43 +01:00
Richard van der Hoff
41921ac01b
Merge pull request #3287 from matrix-org/rav/allow_leaving_server_notices_room
...
Let users leave the server notice room after joining
2018-05-25 11:15:45 +01:00
Richard van der Hoff
757ed27258
Let users leave the server notice room after joining
...
They still can't reject invites, but we let them leave it.
2018-05-25 11:07:21 +01:00
Adrian Tschira
4ee4450d66
fix recursion error
2018-05-24 21:44:10 +02:00
Amber Brown
9c36c150e7
Merge pull request #3283 from NotAFile/py3-state
...
py3-ize state.py
2018-05-24 14:23:33 -05:00
Amber Brown
cc1349c06a
Merge pull request #3279 from NotAFile/py3-more-iteritems
...
more six iteritems
2018-05-24 14:23:13 -05:00
Adrian Tschira
0e61705661
py3-ize state.py
2018-05-24 20:59:00 +02:00
Adrian Tschira
dd068ca979
remaining isintance fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:55:08 +02:00
Adrian Tschira
17a70cf6e9
Misc. py3 fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:20:33 +02:00
Adrian Tschira
6c16a4ec1b
more iteritems
2018-05-24 20:19:06 +02:00
Amber Brown
7ea07c7305
Merge pull request #3278 from NotAFile/py3-storage-base
...
Py3 storage/_base.py
2018-05-24 13:08:09 -05:00
Amber Brown
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4
...
replace some iteritems with six
2018-05-24 13:04:07 -05:00
Amber Brown
c4fb15a06c
Merge pull request #3246 from NotAFile/py3-repr-string
...
use repr, not str
2018-05-24 13:00:20 -05:00
Amber Brown
36501068d8
Merge pull request #3247 from NotAFile/py3-misc
...
Misc Python3 fixes
2018-05-24 12:58:37 -05:00
Amber Brown
2aff6eab6d
Merge pull request #3245 from NotAFile/batch-iter
...
Add batch_iter to utils
2018-05-24 12:54:12 -05:00
Adrian Tschira
095292304f
Py3 storage/_base.py
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 18:24:12 +02:00
David Baker
77a23e2e05
Merge remote-tracking branch 'origin/develop' into dbkr/unbind
2018-05-24 16:20:53 +01:00
David Baker
ecc4b88bd1
Merge pull request #3277 from matrix-org/dbkr/remove_from_user_dir
...
Remove users from user directory on deactivate
2018-05-24 16:12:12 +01:00
Erik Johnston
46345187cc
Merge pull request #3243 from NotAFile/py3-six-3
...
Replace some more comparisons with six
2018-05-24 16:08:57 +01:00
Neil Johnson
037c6db85d
Merge branch 'master' into develop
2018-05-24 16:03:44 +01:00
David Baker
7a1af504d7
Remove users from user directory on deactivate
2018-05-24 15:59:58 +01:00
Erik Johnston
f72d5a44d5
Merge pull request #3261 from matrix-org/erikj/pagination_fixes
...
Fix federation backfill bugs
2018-05-24 14:52:03 +01:00
Erik Johnston
68399fc4de
Merge pull request #3267 from matrix-org/erikj/iter_filter
...
Use iter* methods for _filter_events_for_server
2018-05-24 14:44:57 +01:00
Neil Johnson
91d95a1d8e
bump version
2018-05-24 14:05:07 +01:00
David Baker
9700d15611
pep8
2018-05-24 11:23:15 +01:00
David Baker
a21a41bad7
comment
2018-05-24 11:19:59 +01:00
David Baker
b3bff53178
Unbind 3pids when they're deleted too
2018-05-24 11:08:05 +01:00
Richard van der Hoff
6abcb5d22d
Merge pull request #3273 from matrix-org/rav/server_notices_avatar_url
...
Allow overriding the server_notices user's avatar
2018-05-24 10:31:43 +01:00
Amber Brown
389dac2c15
pepeightttt
2018-05-23 13:08:59 -05:00
Amber Brown
472a5ec4e2
add back CPU metrics
2018-05-23 13:03:56 -05:00
Amber Brown
e987079037
fixes
2018-05-23 13:03:51 -05:00
Richard van der Hoff
9bf4b2bda3
Allow overriding the server_notices user's avatar
...
probably should have done this in the first place, like @turt2live suggested.
2018-05-23 17:43:30 +01:00
Richard van der Hoff
a0b3946fe2
Merge branch 'release-v0.30.0' into rav/localpart_in_consent_uri
2018-05-23 16:06:03 +01:00
Richard van der Hoff
2f7008d4eb
Merge pull request #3271 from matrix-org/rav/consent_uri_in_messages
...
Support for putting %(consent_uri)s in messages
2018-05-23 16:04:30 +01:00
Richard van der Hoff
dda40fb55d
fix typo
2018-05-23 15:30:26 +01:00
Richard van der Hoff
3ff6f50eac
Use the localpart in the consent uri
...
... because it's shorter.
2018-05-23 15:28:23 +01:00
Richard van der Hoff
82191b08f6
Support for putting %(consent_uri)s in messages
...
Make it possible to put the URI in the error message and the server notice that
get sent by the server
2018-05-23 15:24:31 +01:00
David Baker
2c7866d664
Hit the 3pid unbind endpoint on deactivation
2018-05-23 14:38:56 +01:00
Richard van der Hoff
321f02d263
Block attempts to send server notices to remote users
2018-05-23 14:30:47 +01:00
Neil Johnson
3601a240aa
bump version and changelog
2018-05-23 12:03:23 +01:00
Erik Johnston
5aaa3189d5
s/values/itervalues/
2018-05-23 10:13:05 +01:00
Erik Johnston
0a4bca4134
Use iter* methods for _filter_events_for_server
2018-05-23 10:04:23 +01:00
Amber Brown
b6063631c3
more cleanup
2018-05-22 17:36:20 -05:00
Amber Brown
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
Amber Brown
071206304d
cleanup pep8 errors
2018-05-22 16:54:22 -05:00
Amber Brown
85ba83eb51
fixes
2018-05-22 16:28:23 -05:00
Amber Brown
228f1f584e
fix the test failures
2018-05-22 15:02:38 -05:00
Erik Johnston
e85b5a0ff7
Use iter* methods
2018-05-22 19:02:48 +01:00
Erik Johnston
586b66b197
Fix that states is a dict of dicts
2018-05-22 19:02:36 +01:00
Erik Johnston
a17e901f4d
Remove unused string formatting param
2018-05-22 18:24:32 +01:00
Erik Johnston
5494c1d71e
Don't support limitless pagination
...
The pagination storage function supported not specifiying a limit on the
number of events returned. This was triggered when using the search or
context API with a limit of zero, which the storage function took to
mean not being limited.
2018-05-22 18:15:21 +01:00
Neil Johnson
d8cb7225d2
daily user type phone home stats
2018-05-22 18:09:09 +01:00
hera
ad2823ee27
fix synchrotron
2018-05-22 17:47:42 +01:00
Richard van der Hoff
08bfc48abf
custom error code for not leaving server notices room
2018-05-22 17:27:27 +01:00
David Baker
0a078026ea
comment typo
2018-05-22 17:14:06 +01:00
Amber Brown
8f5a688d42
cleanups, self-registration
2018-05-22 10:56:03 -05:00
Amber Brown
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
Erik Johnston
cb2a2ad791
get_domains_from_state returns list of tuples
2018-05-22 16:23:39 +01:00
Richard van der Hoff
08a14b32ae
Merge pull request #3262 from matrix-org/rav/has_already_consented
...
Add a 'has_consented' template var to consent forms
2018-05-22 15:35:10 +01:00
Richard van der Hoff
7b36d06a69
Add a 'has_consented' template var to consent forms
...
fixes #3260
2018-05-22 14:58:34 +01:00
Richard van der Hoff
669400e22f
Enable auto-escaping for the consent templates
...
... to reduce the risk of somebody introducing an html injection attack...
2018-05-22 14:58:34 +01:00
Richard van der Hoff
b5b2d5d64b
Fix dependency on jinja2
...
Delay the import of ConsentResource, so that we can get away without jinja2 if
people don't have the consent resource enabled.
Fixes #3259
2018-05-22 14:03:45 +01:00
Richard van der Hoff
a5e2941aad
Reject attempts to send event before privacy consent is given
...
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252 ) if consent is not yet
given.
2018-05-22 12:00:47 +01:00
Richard van der Hoff
8aeb529262
Merge pull request #3236 from matrix-org/rav/consent_notice
...
Send users a server notice about consent
2018-05-22 11:56:09 +01:00
Richard van der Hoff
8810685df9
Stub out ServerNoticesSender on the workers
...
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
2018-05-22 11:54:51 +01:00
Richard van der Hoff
d5dca9a04f
Move consent config parsing into ConsentConfig
...
turns out we need to reuse this, so it's better in the config class.
2018-05-22 11:54:51 +01:00
Richard van der Hoff
9ea219c514
Send users a server notice about consent
...
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
2018-05-22 11:54:51 +01:00
Richard van der Hoff
d14d7b8fdc
Rename 'version' param on user consent config
...
we're going to use it for the version we require too.
2018-05-22 11:54:51 +01:00
Erik Johnston
7948ecf234
Comment
2018-05-22 11:39:43 +01:00
Erik Johnston
020377a550
Fix logcontext resource usage tracking
2018-05-22 11:16:07 +01:00
Erik Johnston
13a8dfba0d
Merge pull request #3252 from matrix-org/erikj/in_flight_requests
...
Add in flight request metrics
2018-05-22 09:54:30 +01:00
Erik Johnston
c435b0b441
Don't store context
2018-05-22 09:34:26 +01:00
Erik Johnston
fb2806b186
Move in_flight_requests_count to be a callback metric
2018-05-22 09:31:53 +01:00
Amber Brown
fcc525b0b7
rest of the changes
2018-05-21 19:48:57 -05:00
Amber Brown
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
Amber Brown
c60e0d5e02
don't need the resource portion
2018-05-21 17:03:20 -05:00
Amber Brown
02c1d29133
look at the Prometheus metrics instead
2018-05-21 17:02:20 -05:00
Amber Brown
f258deffcb
remove old metrics libs
2018-05-21 17:01:15 -05:00
Richard van der Hoff
413482f578
Merge pull request #3255 from matrix-org/rav/fix_transactions
...
Stop the transaction cache caching failures
2018-05-21 17:58:08 +01:00
Richard van der Hoff
6e1cb54a05
Fix logcontext leak in HttpTransactionCache
...
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
2018-05-21 16:58:20 +01:00
Richard van der Hoff
6d6e7288fe
Stop the transaction cache caching failures
...
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
2018-05-21 16:49:59 +01:00
Erik Johnston
dfa70adc33
Add in flight request metrics
...
This tracks CPU and DB usage while requests are in flight, rather than
when we write the response.
2018-05-21 16:23:06 +01:00
Adrian Tschira
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
Adrian Tschira
d9fe2b2d9d
Replace some more comparisons with six
...
plus a bonus b"" string I missed last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
Adrian Tschira
45b55e23d3
Add batch_iter to utils
...
There's a frequent idiom I noticed where an iterable is split up into a
number of chunks/batches. Unfortunately that method does not work with
iterators like dict.keys() in python3. This implementation works with
iterators.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:48:30 +02:00
Adrian Tschira
dcc235b47d
use stand-in value if maxint is not available
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:44 +02:00
Adrian Tschira
73cbdef5f7
fix py3 intern and remove unnecessary py3 encode
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:31 +02:00
Adrian Tschira
aafb0f6b0d
py3-ize url preview
2018-05-19 17:35:20 +02:00
Adrian Tschira
b932b4ea25
use repr, not str
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:28:42 +02:00
Neil Johnson
644aac5f73
Tighter filtering for user_daily_visits
2018-05-18 17:10:35 +01:00
Neil Johnson
ef466b3a13
fix psql compatability bug
2018-05-18 15:51:21 +01:00
Neil Johnson
2725223f08
Merge branch 'master' into develop
2018-05-18 14:07:50 +01:00
Richard van der Hoff
6d9dc67139
Merge pull request #3232 from matrix-org/rav/server_notices_room
...
Infrastructure for a server notices room
2018-05-18 11:28:04 +01:00
Richard van der Hoff
ed3125b0a1
Merge pull request #3235 from matrix-org/rav/fix_receipts_deferred
...
Fix error in handling receipts
2018-05-18 11:23:11 +01:00
Richard van der Hoff
67af392712
Merge pull request #3233 from matrix-org/rav/remove_dead_code
...
Remove unused `update_external_syncs`
2018-05-18 11:22:43 +01:00
Richard van der Hoff
011e1f4010
Better docstrings
2018-05-18 11:22:12 +01:00
Richard van der Hoff
26305788fe
Make sure we reject attempts to invite the notices user
2018-05-18 11:18:39 +01:00
Richard van der Hoff
d10707c810
Replace inline docstrings with "Attributes" in class docstring
2018-05-18 11:00:55 +01:00
Erik Johnston
fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token
...
Make purge_history operate on tokens
2018-05-18 10:35:23 +01:00
Richard van der Hoff
8b1c856d81
Fix error in handling receipts
...
Fixes an error which has been happening ever since #2158 (v0.21.0-rc1):
> TypeError: argument of type 'ObservableDeferred' is not iterable
fixes #3234
2018-05-18 09:15:35 +01:00
Neil Johnson
6958459b50
bump version, change log
2018-05-17 21:35:07 +01:00
Richard van der Hoff
88d3405332
fix missing yield for server_notices_room
2018-05-17 18:33:45 +01:00
Richard van der Hoff
d43d480d86
Remove unused update_external_syncs
...
This method isn't used anywhere. Burninate it.
2018-05-17 18:22:19 +01:00
Richard van der Hoff
fed62e21ad
Infrastructure for a server notices room
...
Server Notices use a special room which the user can't dismiss. They are
created on demand when some other bit of the code calls send_notice.
(This doesn't actually do much yet becuse we don't call send_notice anywhere)
2018-05-17 17:58:25 +01:00
Richard van der Hoff
f8a1e76d64
Merge pull request #3225 from matrix-org/rav/move_creation_handler
...
Move RoomCreationHandler out of synapse.handlers.Handlers
2018-05-17 17:56:42 +01:00
Erik Johnston
f7906203f6
Merge pull request #3212 from matrix-org/erikj/epa_stream
...
Use stream rather depth ordering for push actions
2018-05-17 12:01:21 +01:00
Richard van der Hoff
c46367d0d7
Move RoomCreationHandler out of synapse.handlers.Handlers
...
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.
Also fix the docstrings on create_room.
2018-05-17 09:08:42 +01:00
Neil Johnson
85b8acdeb4
Merge branch 'master' into develop
2018-05-16 16:48:10 +01:00
Neil Johnson
3c099219e0
bump version and changelog for 0.29.0
2018-05-16 15:44:59 +01:00
Erik Johnston
680530cc7f
Clarify comment
2018-05-16 11:47:29 +01:00
Erik Johnston
43e6e82c4d
Comments
2018-05-16 11:13:31 +01:00
Neil Johnson
dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics
...
user visit data
2018-05-16 10:09:24 +00:00
Erik Johnston
c945af8799
Move and rename variable
2018-05-16 10:52:06 +01:00
Neil Johnson
be11a02c4f
remove empty line
2018-05-16 10:45:40 +01:00
Neil Johnson
a2204cc9cc
remove unused method recurring_user_daily_visit_stats
2018-05-16 09:47:20 +01:00
Neil Johnson
31c2502ca8
style and further contraining query
2018-05-16 09:46:43 +01:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler
...
ConsentResource to gather policy consent from users
2018-05-16 07:19:18 +01:00