Erik Johnston
d14e94bae4
Merge pull request #4716 from matrix-org/erikj/pusher_logging
...
Fix up pusher logging a bit
2019-02-22 15:29:16 +00:00
Erik Johnston
b82c9cf462
Add missing return
2019-02-22 15:27:40 +00:00
Erik Johnston
f2891d2487
Correctly handle PusherConfigException
2019-02-22 15:18:19 +00:00
Erik Johnston
a164134a53
Drop logging level of creating a pusher
2019-02-22 14:48:06 +00:00
Erik Johnston
1d9df51ff1
Correctly handle null data in HttpPusher
2019-02-22 14:47:48 +00:00
Erik Johnston
80467bbac3
Fix state cache invalidation on workers
2019-02-22 14:38:14 +00:00
Erik Johnston
7b288826b7
Fix backfill storing incorrect state for events
2019-02-22 11:33:51 +00:00
Richard van der Hoff
e07384c4e1
Add prometheus metrics for number of badge update pushes. ( #4709 )
...
We're counting the number of push notifications, but not the number of badges;
I'd like to see if they are significant.
2019-02-22 10:57:15 +00:00
Richard van der Hoff
0abb094f1a
bail out early in on_new_receipts if no pushers ( #4706 )
2019-02-21 17:51:21 +00:00
Richard van der Hoff
6d65659b62
Run push_receipts_to_remotes as background job ( #4707 )
...
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
2019-02-21 17:50:30 +00:00
Erik Johnston
b9d6756b14
Merge pull request #4263 from rkfg/develop
...
Prevent crash on pagination.
2019-02-21 17:42:15 +00:00
Erik Johnston
71b625d808
Stop backpaginating when events not visible
2019-02-20 18:14:12 +00:00
Erik Johnston
82fca11fc1
Merge pull request #4694 from matrix-org/erikj/fix_sentry_config_format
...
Fixup generated metrics config
2019-02-20 14:13:38 +00:00
Richard van der Hoff
82ca6d1f9f
Add metrics for number of outgoing EDUs, by type ( #4695 )
2019-02-20 14:13:14 +00:00
Erik Johnston
3d672fec51
Fixup generated metrics config
2019-02-20 13:39:37 +00:00
Amber Brown
a06614bd2a
UPSERT many functionality ( #4644 )
2019-02-20 23:03:30 +11:00
Richard van der Hoff
b2200a8690
Logging improvements for the pusher ( #4691 )
2019-02-20 11:36:50 +00:00
Richard van der Hoff
c88bc53903
Fix TaskStopped exceptions when outbound requests time out ( #4690 )
2019-02-20 11:35:52 +00:00
Richard van der Hoff
86920ac266
Merge pull request #4678 from matrix-org/rav/tls_install_instructions
...
Attempt to clarify installation/config instructions
2019-02-20 11:10:33 +00:00
Erik Johnston
dbdc565dfd
Fix registration on workers ( #4682 )
...
* Move RegistrationHandler init to HomeServer
* Move post registration actions to RegistrationHandler
* Add post regisration replication endpoint
* Newsfile
2019-02-20 18:47:31 +11:00
Richard van der Hoff
5f9bdf90fe
Attempt to make default config more consistent
...
The general idea here is that config examples should just have a hash and no
extraneous whitespace, both to make it easier for people who don't understand
yaml, and to make the examples stand out from the comments.
2019-02-19 13:54:29 +00:00
Erik Johnston
c003450057
Merge pull request #4671 from matrix-org/erikj/state_cache_invalidation
...
Batch cache invalidation over replication
2019-02-19 13:14:30 +00:00
Erik Johnston
62175a20e5
Docs
2019-02-19 11:38:40 +00:00
Erik Johnston
1bb35e3a83
Use itertools
2019-02-19 11:34:40 +00:00
Erik Johnston
bc8fa1509d
Documentation
2019-02-19 11:24:59 +00:00
Brendan Abolivier
a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated
...
Support .well-known delegation when issuing certificates through ACME
2019-02-19 11:15:38 +00:00
Brendan Abolivier
5a707a2f9a
Improve config documentation
2019-02-19 10:59:26 +00:00
Brendan Abolivier
a8626901cd
Fetch ACME domain into an instance member
2019-02-19 10:54:33 +00:00
Erik Johnston
32590b7139
Merge pull request #4670 from matrix-org/erikj/register_login_split
...
Split /login into client_reader
2019-02-19 10:46:22 +00:00
Richard van der Hoff
7c70b8f8a6
Try and make TLS federation client code faster ( #4674 )
...
* fix to use makeContext so that we don't need to rebuild the certificateoptions each time
2019-02-19 10:19:16 +00:00
Andrew Morgan
968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade
...
Transfer bans on room upgrade
2019-02-19 10:16:42 +00:00
Amber Brown
561eebe170
fix to use makeContext so that we don't need to rebuild the certificateoptions each time
2019-02-19 16:18:05 +11:00
Andrew Morgan
34ac75ce2c
lint
2019-02-18 18:23:37 +00:00
Erik Johnston
a9b5ea6fc1
Batch cache invalidation over replication
...
Currently whenever the current state changes in a room invalidate a lot
of caches, which cause *a lot* of traffic over replication. Instead,
lets batch up all those invalidations and send a single poke down
the replication streams.
Hopefully this will reduce load on the master process by substantially
reducing traffic.
2019-02-18 17:53:31 +00:00
Andrew Morgan
f8b9ca53ce
Move member event processing and changelog fix
2019-02-18 17:28:52 +00:00
Erik Johnston
d154f5a055
Merge pull request #4632 from matrix-org/erikj/basic_sentry
...
Add basic optional sentry.io integration
2019-02-18 17:22:45 +00:00
Erik Johnston
4cc4400b4d
Split /login into client_reader
2019-02-18 17:19:01 +00:00
Erik Johnston
fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split
...
Split out registration to worker
2019-02-18 17:18:06 +00:00
Erik Johnston
459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions
...
Correctly handle HttpResponseException when handling device updates
2019-02-18 16:55:43 +00:00
Erik Johnston
d328a93b51
Fixup error handling and message
2019-02-18 16:53:56 +00:00
Erik Johnston
af691e415c
Move register_device into handler
2019-02-18 16:49:38 +00:00
Erik Johnston
028267acd2
Merge pull request #4669 from matrix-org/erikj/log_exception
...
Cleanup top level request exception logging
2019-02-18 16:41:41 +00:00
Erik Johnston
d08bac4136
Merge pull request #4651 from matrix-org/matthew/well-known-cors
...
set CORS on .well-known URI to unbreak modular
2019-02-18 16:38:11 +00:00
Erik Johnston
c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers
...
Fix kicking guest users in worker mode
2019-02-18 15:55:48 +00:00
Erik Johnston
092b541401
Fixup
2019-02-18 15:52:26 +00:00
Brendan Abolivier
45bb55c6de
Use a configuration parameter to give the domain to generate a certificate for
2019-02-18 15:46:23 +00:00
Erik Johnston
8b9ae6d3a6
Update docs
2019-02-18 15:26:13 +00:00
Erik Johnston
94960cef03
pep8
2019-02-18 15:24:13 +00:00
Erik Johnston
fe725f7e45
Cleanup top level request exception logging
...
Firstly, we always logged that the request was being handled via
`JsonResource._async_render`, so we change that to use the servlet name
we add to the request.
Secondly, we pass the exception information to the logger rather than
formatting it manually. This makes it consistent with other exception
logging, allwoing logging hooks and formatters to access the exception
information.
2019-02-18 15:11:04 +00:00
Matthew Hodgson
04dad5ac16
fix missig import
2019-02-18 15:59:23 +01:00
Erik Johnston
e07cc31cb8
Correctly handle HttpResponseException
2019-02-18 14:55:09 +00:00
Brendan Abolivier
68a53f825f
Merge branch 'develop' into babolivier/acme-delegated
2019-02-18 14:52:23 +00:00
Erik Johnston
32e54b472a
Fix kicking guest users in worker mode
...
When guest_access changes from allowed to forbidden all local guest
users should be kicked from the room. This did not happen when
revocation was received from federation on a worker.
Presumably broken in #4141
2019-02-18 14:16:07 +00:00
Andrew Morgan
915421065b
Membership events are done later
2019-02-18 14:02:09 +00:00
Erik Johnston
d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions
...
Correctly handle RequestSendFailed exceptions
2019-02-18 13:53:24 +00:00
Andrew Morgan
9caab0c364
Transfer bans on room upgrade
2019-02-18 13:52:51 +00:00
Erik Johnston
dc5efc92a8
Fixup
2019-02-18 13:52:49 +00:00
Erik Johnston
eb2b8523ae
Split out registration to worker
...
This allows registration to be handled by a worker, though the actual
write to the database still happens on master.
Note: due to the in-memory session map all registration requests must be
handled by the same worker.
2019-02-18 12:12:57 +00:00
Andrew Morgan
6d02a13d81
Typo in info log
...
Co-Authored-By: babolivier <contact@brendanabolivier.com>
2019-02-18 11:36:34 +00:00
Juuso "Linda" Lapinlampi
68d2869c8d
config: Remove a repeated word from a logger warning
...
The warning for missing macaroon_secret_key was "missing missing".
2019-02-15 22:24:53 -07:00
Erik Johnston
bd4505f765
Merge pull request #4647 from matrix-org/erikj/add_room_publishing_rules
...
Add configurable room list publishing rules
2019-02-15 22:11:01 +00:00
Brendan Abolivier
f86b695cbd
Various cosmetics to make TravisCI happy
2019-02-15 12:29:34 +00:00
Brendan Abolivier
af8a2f679b
Remove unused import
2019-02-15 12:27:43 +00:00
Brendan Abolivier
1895d14e12
Support .well-known delegation when issuing certificates through ACME
2019-02-15 12:05:08 +00:00
Erik Johnston
b99c532c1c
Move defaults up into code
2019-02-15 10:53:39 +00:00
Erik Johnston
02c729d6b0
Hoist up checks to reduce overall work
2019-02-15 10:20:02 +00:00
Erik Johnston
02c46acc6a
Fixup comments
2019-02-15 10:17:13 +00:00
Matthew Hodgson
bfcefbb230
tabs
2019-02-14 19:53:12 +00:00
Matthew Hodgson
6f47bc3fb2
set CORS on .well-known URI to unbreak modular
...
otherwise a riot/web running on foo.riot.im can't query
the .well-known on foo.modular.im...
2019-02-14 19:51:41 +00:00
Erik Johnston
8e32f26cb8
Clarify comments
2019-02-14 18:21:24 +00:00
Erik Johnston
cb12a37708
Clarify and fix behaviour when there are multiple aliases
2019-02-14 18:16:32 +00:00
Erik Johnston
f61b2068e6
Only fetch aliases when publishing rooms
2019-02-14 18:08:09 +00:00
Erik Johnston
f666fe36d7
Fixup comments
2019-02-14 18:07:24 +00:00
Richard van der Hoff
bf4fd14806
Merge branch 'master' into develop
2019-02-14 17:34:09 +00:00
Richard van der Hoff
f595d6ac57
0.99.1.1
2019-02-14 17:20:02 +00:00
Richard van der Hoff
f311018823
Fix errors in acme provisioning ( #4648 )
...
* Better logging for errors on startup
* Fix "TypeError: '>' not supported" when starting without an existing
certificate
* Fix a bug where an existing certificate would be reprovisoned every day
2019-02-14 17:10:36 +00:00
Erik Johnston
eaf4d11af9
Add configurable room list publishing rules
...
This allows specifying who and what is allowed to be published onto the
public room list
2019-02-14 16:02:23 +00:00
Richard van der Hoff
b02465b9db
Merge branch 'master' into develop
2019-02-14 14:42:03 +00:00
Richard van der Hoff
06cd757ae7
0.99.1
2019-02-14 14:24:24 +00:00
Erik Johnston
7fc1196a36
Correctly handle RequestSendFailed exceptions
...
This mainly reduces the number of exceptions we log.
2019-02-14 14:01:04 +00:00
Erik Johnston
6cb415b63f
Fixup comments and add warning
2019-02-13 16:15:11 +00:00
Richard van der Hoff
c6e75c9f2d
Merge pull request #4450 from 14mRh4X0r/fix-dependency-message
...
Fix error message for optional dependencies
2019-02-13 16:12:49 +00:00
Richard van der Hoff
3bc238629e
0.99.1rc2
2019-02-13 14:46:18 +00:00
Richard van der Hoff
c1dfd6a18a
Merge remote-tracking branch 'origin/release-v0.99.1' into develop
2019-02-13 14:27:45 +00:00
Erik Johnston
309f3bb322
Update synapse/app/_base.py
...
Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com>
2019-02-13 13:24:27 +00:00
Amber Brown
bb4fd8f927
Run black
on user directory code ( #4635 )
2019-02-13 23:05:32 +11:00
Richard van der Hoff
767686af48
Use listen_tcp
for the replication listener
...
Fixes the "can't listen on 0.0.0.0" error. Also makes it more consistent with
what we do elsewhere.
2019-02-13 11:59:04 +00:00
Richard van der Hoff
2a5a15aff8
Improve logging around listening services
...
I wanted to bring listen_tcp into line with listen_ssl in terms of returning a
list of ports, and wanted to check that was a safe thing to do - hence the
logging in `refresh_certificate`.
Also, pull the 'Synapse now listening' message up to homeserver.py, because it
was being duplicated everywhere else.
2019-02-13 11:58:54 +00:00
Richard van der Hoff
e3a0300431
Special-case the default bind_addresses for metrics listener
...
turns out it doesn't really support ipv6, so let's hack around that by only
listening on ipv4 by default.
2019-02-13 11:48:56 +00:00
Erik Johnston
93f7d2df3e
Comments
2019-02-12 16:03:40 +00:00
Erik Johnston
6a8f902edb
Raise an appropriate error message if sentry_sdk missing
2019-02-12 16:01:41 +00:00
Erik Johnston
ef2228c890
Basic sentry integration
2019-02-12 13:55:58 +00:00
Erik Johnston
d2fa7b7e99
Update changelog and version
2019-02-12 13:22:25 +00:00
Erik Johnston
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
...
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
Erik Johnston
cf82338930
Merge pull request #4627 from matrix-org/erikj/user_ips_analyze
...
Analyze user_ips before running deduplication
2019-02-12 13:05:09 +00:00
Erik Johnston
3df8fcca25
Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe
...
Reduce user_ips bloat during dedupe background update
2019-02-12 13:02:58 +00:00
Erik Johnston
495ea92350
Fix pep8
2019-02-12 12:40:42 +00:00
Erik Johnston
483ba85c7a
Analyze user_ips before running deduplication
...
Due to the table locks taken out by the naive upsert, the table
statistics may be out of date. During deduplication it is important that
the correct index is used as otherwise a full table scan may be
incorrectly used, which can end up thrashing the database badly.
2019-02-12 11:55:27 +00:00
Erik Johnston
362d80b770
Reduce user_ips bloat during dedupe background update
...
The background update to remove duplicate rows naively deleted and
reinserted the duplicates. For large tables with a large number of
duplicates this causes a lot of bloat (with postgres), as the inserted
rows are appended to the table, since deleted rows will not be
overwritten until a VACUUM has happened.
This should hopefully also help ensure that the query in the last batch
uses the correct index, as inserting a large number of new rows without
analyzing will upset the query planner.
2019-02-12 11:39:34 +00:00
Erik Johnston
3c03c37883
Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings
...
fix self-signed cert notice from generate-config
2019-02-12 11:24:45 +00:00
Richard van der Hoff
a4ce91396b
Disable TLS by default ( #4614 )
2019-02-12 10:52:08 +00:00
Richard van der Hoff
32b781bfe2
Fix error when loading cert if tls is disabled ( #4618 )
...
If TLS is disabled, it should not be an error if no cert is given.
Fixes #4554 .
2019-02-12 10:51:31 +00:00
Richard van der Hoff
dfc846a316
fix self-signed cert notice from generate-config
...
fixes #4620
2019-02-12 10:37:59 +00:00
Richard van der Hoff
0ca2908653
fix tests
2019-02-11 22:01:27 +00:00
Richard van der Hoff
4fddf8fc77
Infer no_tls from presence of TLS listeners
...
Rather than have to specify `no_tls` explicitly, infer whether we need to load
the TLS keys etc from whether we have any TLS-enabled listeners.
2019-02-11 21:39:14 +00:00
Richard van der Hoff
15272f837c
Merge branch 'rav/no_create_server_contexts_if_no_tls' into rav/tls_cert/work
2019-02-11 21:34:19 +00:00
Richard van der Hoff
9645728619
Don't create server contexts when TLS is disabled
...
we aren't going to use them anyway.
2019-02-11 21:32:01 +00:00
Richard van der Hoff
be794c7cf7
Merge branch 'rav/tls_config_logging_fixes' into rav/tls_cert/work
2019-02-11 21:16:00 +00:00
Richard van der Hoff
2129dd1a02
Fail cleanly if listener config lacks a 'port'
...
... otherwise we would fail with a mysterious KeyError or something later.
2019-02-11 21:15:01 +00:00
Richard van der Hoff
086f6f27d4
Logging improvements around TLS certs
...
Log which file we're reading keys and certs from, and refactor the code a bit
in preparation for other work
2019-02-11 21:02:06 +00:00
Richard van der Hoff
5d27730a73
Move ClientTLSOptionsFactory init out of refresh_certificates ( #4611 )
...
It's nothing to do with refreshing the certificates. No idea why it was here.
2019-02-11 18:03:30 +00:00
Erik Johnston
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating
...
add updating of backup versions
2019-02-11 13:45:49 +00:00
Richard van der Hoff
24b7f3916d
Clean up default listener configuration ( #4586 )
...
Rearrange the comments to try to clarify them, and expand on what some of it
means.
Use a sensible default 'bind_addresses' setting.
For the insecure port, only bind to localhost, and enable x_forwarded, since
apparently it's for use behind a load-balancer.
2019-02-11 12:50:30 +00:00
Andrew Morgan
a126f86eec
Transfer Server ACLs on room upgrade
2019-02-11 11:30:37 +00:00
Amber Brown
6e2a5aa050
ACME Reprovisioning ( #4522 )
2019-02-11 10:36:26 +00:00
Amber Brown
4ffd10f46d
Be tolerant of blank TLS fingerprints config ( #4589 )
2019-02-11 10:04:27 +00:00
Erik Johnston
b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
...
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
Valentin Anger
2dc2b6e9f1
Allow "unavailable" presence status for /sync ( #4592 )
...
* Allow "unavailable" presence status for /sync
Closes #3772 , closes #3779
Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com>
* Add changelog for PR 4592
2019-02-08 21:09:56 +00:00
Richard van der Hoff
56710c7df5
Fix 'no unique or exclusion constraint' error ( #4591 )
...
Add more tables to the list of tables which need a background update to
complete before we can upsert into them, which fixes a race against the
background updates.
2019-02-08 18:30:46 +00:00
Amber Brown
9cd33d2f4b
Deduplicate some code in synapse.app ( #4567 )
2019-02-08 17:25:57 +00:00
Hubert Chathi
afae8442b5
make sure version is in body and wrap in linearizer queue
...
also add tests
2019-02-08 01:32:45 -05:00
Hubert Chathi
d9e424bf64
re-try to make isort happy
2019-02-06 22:18:41 -05:00
Hubert Chathi
9ff620a518
fix import to make isort happy
2019-02-06 21:32:52 -05:00
Hubert Chathi
8248637173
add new endpoint to update backup versions
2019-02-06 17:57:10 -05:00
Hubert Chathi
664c81e8b7
return proper error codes for some 404s
2019-02-06 17:47:22 -05:00
Richard van der Hoff
2475434080
Merge branch 'master' into develop
2019-02-05 18:44:49 +00:00
Richard van der Hoff
3bd9daf4b8
v0.99.0
2019-02-05 18:33:02 +00:00
Andrew Morgan
627ecd358e
Filter user directory state query to a subset of state events ( #4462 )
...
* Filter user directory state query to a subset of state events
* Add changelog
2019-02-05 12:16:28 +00:00
Richard van der Hoff
bf1e4d96ad
Fix default ACME config for py2 ( #4564 )
...
Fixes #4559
2019-02-05 11:37:33 +00:00
Richard van der Hoff
d7e27a1f08
fix typo in config comments ( #4557 )
2019-02-05 11:32:45 +00:00
Richard van der Hoff
30fd2f89db
0.99.0rc4
2019-02-01 15:52:28 +00:00
Richard van der Hoff
ef43a03fc5
Merge pull request #4546 from matrix-org/rav/silence_critical_error_from_federation
...
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
2019-02-01 14:37:22 +00:00
Richard van der Hoff
f8db967d5a
Merge remote-tracking branch 'origin/release-v0.99.0' into develop
2019-02-01 13:20:15 +00:00
Richard van der Hoff
fa794980ec
Merge pull request #4544 from matrix-org/rav/skip_invalid_well_known
...
Treat an invalid .well-known the same as an absent one
2019-02-01 13:18:36 +00:00
Richard van der Hoff
e9779a6f8f
Fix b'ab' noise in logs
2019-02-01 12:34:31 +00:00
Richard van der Hoff
9763a73af0
Merge branch 'release-v0.99.0' into develop
2019-02-01 12:30:22 +00:00
Richard van der Hoff
f0ba34f581
Fix noisy "twisted.internet.task.TaskStopped" errors in logs
...
Fixes #4003
2019-02-01 12:22:57 +00:00
Richard van der Hoff
8a21b03fba
Treat an invalid .well-known the same as an absent one
...
... basically, carry on and fall back to SRV etc.
2019-02-01 11:37:31 +00:00
Richard van der Hoff
3c8a41140e
Cache failures to parse .well-known
...
Also add a Measure block around the .well-known fetch
2019-02-01 00:37:52 +00:00
Richard van der Hoff
24d59c7568
better logging for federation connections
2019-01-31 23:18:20 +00:00
Richard van der Hoff
d428b46346
Update federation routing logic to check .well-known before SRV
2019-01-31 23:14:18 +00:00
Andrew Morgan
c45fd0dda0
Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable
...
Copy over non-federatable trait on room upgrade
2019-01-31 20:28:37 +00:00
Richard van der Hoff
625385d684
Merge branch 'release-v0.99.0' into develop
2019-01-31 18:43:20 +00:00
Richard van der Hoff
85129d7068
v0.99.0rc3
2019-01-31 18:35:38 +00:00
Andrew Morgan
d239f67c25
Raise an exception instead of returning None
2019-01-31 18:34:15 +00:00
Richard van der Hoff
07dfe148de
Add some debug for membership syncing issues ( #4538 )
...
I can't figure out what's going on with #4422 and #4436 ; perhaps this will help.
2019-01-31 18:30:40 +00:00
Andrew Morgan
3ed3cb4339
New function for getting room's create event
2019-01-31 18:21:39 +00:00
Andrew Morgan
bbb97a35fd
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable
2019-01-31 18:11:00 +00:00
Richard van der Hoff
e707e7b38d
Fix infinite loop when an event is redacted in a v3 room ( #4535 )
2019-01-31 15:34:17 +00:00
Andrew Morgan
563f6a832b
Reject large transactions on federation ( #4513 )
...
* Reject large transactions on federation
* Add changelog
* lint
* Simplify large transaction handling
2019-01-31 11:44:04 +00:00
Andrew Morgan
fb50934b8f
lint
2019-01-31 11:34:45 +00:00
Richard van der Hoff
6fba9fd20c
Merge remote-tracking branch 'origin/release-v0.99.0' into develop
2019-01-30 17:02:32 +00:00
Andrew Morgan
d621c5562e
Copy over non-federatable trait on room upgrade
2019-01-30 16:33:51 +00:00
Richard van der Hoff
b8b898666e
v0.99.0rc2
2019-01-30 16:31:07 +00:00
Matthew Hodgson
ad7ac8853c
by default include m.room.encryption on invites ( #3902 )
...
* by default include m.room.encryption on invites
* fix constant
* changelog
2019-01-30 16:26:13 +00:00
Richard van der Hoff
c74b96755c
Merge remote-tracking branch 'origin/develop' into release-v0.99.0
2019-01-30 16:23:28 +00:00
Neil Johnson
81b7e7eed3
Update constants.py
...
remove trailing ,
2019-01-30 16:11:36 +00:00
Erik Johnston
ed8c5e4cda
Fix remote invite rejections not comming down sync
...
This was broken in PR #4405 , commit 886e5ac
, where we changed remote
rejections to be outliers.
The fix is to explicitly add the leave event in when we know its an out
of band invite. We can't always add the event as if the server is/was in
the room there might be more events to send down the sync than just the
leave.
2019-01-30 15:46:27 +00:00
Richard van der Hoff
a5d0c771a3
0.99.0rc1
2019-01-30 15:11:18 +00:00
Erik Johnston
6587b0b89b
Merge pull request #4472 from matrix-org/neilj/room_capabilities
...
Server capabilities support
2019-01-30 14:26:56 +00:00
Erik Johnston
a4f52a33fe
Fix replication for room v3 ( #4523 )
...
* Fix replication for room v3
We were not correctly quoting the path fragments over http replication,
which meant that it exploded when the event IDs had a slash in them
* Newsfile
2019-01-30 14:19:52 +00:00
Richard van der Hoff
7615a8ced1
ACME config cleanups ( #4525 )
...
* Handle listening for ACME requests on IPv6 addresses
the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.
* Clean up the default ACME config
make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.
* newsfile
2019-01-30 14:17:55 +00:00
Erik Johnston
e87d7a4b0f
Raise ConfigError instead
2019-01-30 12:48:09 +00:00
Erik Johnston
270f212a2a
_listener_http should return a list
2019-01-30 12:14:50 +00:00
Richard van der Hoff
a79034aedf
Merge pull request #4521 from matrix-org/rav/fed_routing/cleanups
...
Tiny .well-known fixes
2019-01-30 11:47:24 +00:00
Richard van der Hoff
c7b24ac3d0
Follow redirects on .well-known ( #4520 )
2019-01-30 11:43:33 +00:00
Amber Brown
f6813919e8
SIGHUP for TLS cert reloading ( #4495 )
2019-01-30 11:00:02 +00:00
Richard van der Hoff
09a1a6b55e
fix exception text
2019-01-30 10:58:52 +00:00
Richard van der Hoff
928c50b59a
Also jitter the invalid cache period
2019-01-30 10:58:52 +00:00
Neil Johnson
b37e8c9572
Merge branch 'neilj/room_capabilities' of github.com:matrix-org/synapse into neilj/room_capabilities
2019-01-30 10:56:47 +00:00
Neil Johnson
f834d98402
isort
2019-01-30 10:55:42 +00:00
Richard van der Hoff
bc5f6e1797
Add a caching layer to .well-known responses ( #4516 )
2019-01-30 10:55:25 +00:00
Amber Brown
3f189c902e
Fix flake8 ( #4519 )
2019-01-30 10:53:17 +00:00
Neil Johnson
ee4df7fd7a
Merge branch 'develop' into neilj/room_capabilities
2019-01-30 10:28:08 +00:00
Neil Johnson
c5a0f82cca
define room dispositions for use in exposing room capabilities
2019-01-30 10:24:24 +00:00
Neil Johnson
9c850d9d5e
formatting and use constants where available
2019-01-30 10:23:26 +00:00
Neil Johnson
2f46804055
Populate default room version from Constants
2019-01-30 09:39:10 +00:00
Neil Johnson
c7837dce24
reflect that rooms v3 is a stable room version
2019-01-30 09:33:30 +00:00
Richard van der Hoff
457fbfaf22
Merge pull request #4486 from xperimental/workaround-4216
...
Implement workaround for login error.
2019-01-30 07:06:11 +00:00
Robert Jacob
2a7f0b8953
Implement workaround for login error.
...
Signed-off-by: Robert Jacob <xperimental@solidproject.de>
2019-01-30 01:06:39 +01:00
Erik Johnston
0b24d58e05
No vdh tests!
2019-01-29 23:11:48 +00:00
Erik Johnston
a1b0e1879b
Enable room version v3
2019-01-29 23:09:10 +00:00
Erik Johnston
67b82f1336
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah
2019-01-29 22:58:38 +00:00
Erik Johnston
afeea319df
Fixup comment
2019-01-29 22:55:29 +00:00
Erik Johnston
7740eddd04
Merge pull request #4514 from matrix-org/erikj/remove_event_id
...
Remove usages of event ID's domain
2019-01-29 22:54:25 +00:00
Erik Johnston
ff2f65d737
Update comment
2019-01-29 22:35:36 +00:00
Erik Johnston
655ce037fd
check event format version not room version
2019-01-29 22:33:43 +00:00
Erik Johnston
f46a818ce5
kill vdh test some more
2019-01-29 22:02:58 +00:00
Erik Johnston
a696c48133
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah
2019-01-29 22:00:33 +00:00
Erik Johnston
c21b7cbc09
Update synapse/storage/events_worker.py
2019-01-29 21:53:48 +00:00
Erik Johnston
4db252c073
Check redaction state when event is pulled out of the database
2019-01-29 21:48:36 +00:00
Erik Johnston
6d23ec2111
Fix typo
2019-01-29 21:45:53 +00:00
Erik Johnston
b5d510ad64
Remove unused arg
2019-01-29 21:45:28 +00:00
Erik Johnston
6f9cdc2d47
Merge pull request #4483 from matrix-org/erikj/event_v2
...
Implement event format V2
2019-01-29 21:40:00 +00:00
Erik Johnston
47e2dd1994
Drop vdh support
2019-01-29 21:24:34 +00:00
Erik Johnston
38590a4870
Add docstring
2019-01-29 21:22:47 +00:00
Richard van der Hoff
82165eeb05
Update synapse/storage/events_worker.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-01-29 21:14:39 +00:00
Andrew Morgan
03b086647f
Merge pull request #4512 from matrix-org/anoa/consent_dir
...
Check consent dir path on startup
2019-01-29 20:08:18 +00:00
Erik Johnston
84af577356
Implement event format v2
2019-01-29 18:06:11 +00:00
Neil Johnson
e4bef9d470
rework format of change password capability
2019-01-29 18:04:56 +00:00
Erik Johnston
610f0830b0
Don't assert an event must have an event ID
2019-01-29 18:02:26 +00:00
Erik Johnston
840068bd78
Only check event ID domain for signatures for V1 events
...
In future version events won't have an event ID, so we won't be able to
do this check.
2019-01-29 18:02:02 +00:00
Erik Johnston
8e3d34e3c5
Use event origin for filtering incoming events
...
We only process events sent to us from a server if the event ID matches
the server, to help guard against federation storms. We replace this
with a check against the event origin.
2019-01-29 16:57:00 +00:00
Erik Johnston
55d9024835
Use snder and not event ID domain to check if ours
...
The transaction queue only sends out events that we generate. This was
done by checking domain of event ID, but that can no longer be used.
Instead, we may as well use the sender field.
2019-01-29 16:54:23 +00:00
Richard van der Hoff
cc2d650ef7
Relax requirement for a content-type on .well-known ( #4511 )
2019-01-29 16:49:17 +00:00
Erik Johnston
b1fffca345
Remove event ID usage when checking if new room
...
The event ID is changing, so we can no longer get the domain from it. On
the other hand, the check is unnecessary.
2019-01-29 16:15:02 +00:00
Erik Johnston
770b823445
Only check event IDs domain signed event for V1 and V2
...
Since newer versions of events don't have the same format for event ID.
2019-01-29 16:15:00 +00:00
Travis Ralston
d02c5ccb11
Merge pull request #4498 from matrix-org/travis/fix-docs-public_baseurl
...
Don't recommend :8448 to people on public_baseurl
2019-01-29 09:06:16 -07:00
Neil Johnson
f03b3a7a3a
support change_password in capabilities end-point
2019-01-29 15:58:37 +00:00
Andrew Morgan
e65a17b26f
Check consent dir path on startup
2019-01-29 15:30:33 +00:00
Amber Brown
6bd4374636
Do not generate self-signed TLS certificates by default. ( #4509 )
2019-01-29 14:09:10 +00:00
Erik Johnston
b8d75ef53e
Merge pull request #4481 from matrix-org/erikj/event_builder
...
Refactor event building into EventBuilder
2019-01-29 14:07:23 +00:00
Richard van der Hoff
99e36d5e24
Implement MSC1708 (.well-known lookups for server routing) ( #4489 )
2019-01-29 13:53:02 +00:00
Erik Johnston
b82a76c384
Finish comment...
2019-01-29 13:50:59 +00:00
Erik Johnston
2562319821
Merge pull request #4510 from matrix-org/erikj/fixup_compute_event_signature
...
Fixup calls to `comput_event_signature`
2019-01-29 13:35:19 +00:00
Erik Johnston
5891a6edc8
Correctly set context.app_service
2019-01-29 12:09:10 +00:00
Erik Johnston
fb99dae9c8
Don't set event_id twice
2019-01-29 12:08:23 +00:00
Erik Johnston
64c1bd1d21
Remove dead function
2019-01-29 12:06:28 +00:00
Erik Johnston
7709d2bd16
Implement rechecking of redactions
2019-01-29 11:56:20 +00:00
Erik Johnston
7a3ec5b022
Add RoomVersions.V3 constant, without enabling it
...
We add the constant, but don't add it to the known room versions. This
lets us start adding V3 logic, but the servers will never join or create
V3 rooms
2019-01-29 11:55:33 +00:00
Erik Johnston
6598992b01
Fixup calls to comput_event_signature
...
We currently pass FrozenEvent instead of `dict` to
`compute_event_signature`, which works by accident due to `dict(event)`
producing the correct result.
This fixes PR #4493 commit 855a151
2019-01-29 11:41:58 +00:00
Neil Johnson
a124025dab
enforce auth for capabilities endpoint
2019-01-29 11:37:56 +00:00
Erik Johnston
ff37acb8ce
Merge pull request #4496 from matrix-org/erikj/invite_fallback
...
Implement fallback for V2 invite API
2019-01-29 11:28:23 +00:00
Erik Johnston
5180f12bae
Replace usage of builder.user_id with builder.sender
...
`.user_id` is proxed to `.sender` in FrozenEvent, so this has no
functional change
2019-01-29 11:18:38 +00:00
Erik Johnston
be47cfa9c9
Refactor event building into EventBuilder
...
This is so that everything is done in one place, making it easier to
change the event format based on room version
2019-01-29 11:13:00 +00:00
Erik Johnston
554ca58ea1
Make add_hashes_and_signatures operate on dicts
2019-01-29 11:12:38 +00:00
Amber Brown
f815bd7feb
Make linearizer more quiet ( #4507 )
2019-01-29 11:05:31 +00:00
Erik Johnston
073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
...
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
Erik Johnston
40638ae7f5
Remove duplicate checks
2019-01-29 10:37:40 +00:00
Erik Johnston
9fa3c6ffa3
Fix up error messages
2019-01-29 10:36:46 +00:00
Erik Johnston
28efc80723
Fold validate into validate_new
2019-01-29 10:34:49 +00:00
Erik Johnston
b6b73a0bcf
Fix receiving events from federation via a worker
...
This bug was introduced in PR #4470 , commit 678a92cb56
2019-01-29 10:30:26 +00:00
Neil Johnson
327b992e17
register capabilities servlet
2019-01-29 10:28:35 +00:00
Amber Brown
94fb63e44f
Fix typo in upserts code ( #4505 )
...
* fix obvious problem :|
* changelog
2019-01-29 10:04:23 +00:00
Erik Johnston
17709f8f9c
Merge pull request #4493 from matrix-org/erikj/refactor_event_signing
...
Refactor event signing to work on dicts
2019-01-29 09:52:54 +00:00
Erik Johnston
f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version
...
Pass through room version to event auth
2019-01-28 20:09:38 +00:00
Travis Ralston
6901ac7e9d
Don't recommend :8448 to people on public_baseurl
2019-01-28 12:15:22 -07:00
Erik Johnston
d414f30019
Implement fallback for V2 invite API
...
If the room version is either 1 or 2 then a server should retry failed
`/v2/invite` requests with the v1 API
2019-01-28 17:33:25 +00:00
Erik Johnston
d758d5310e
Correctly use default room version if none is set
2019-01-28 17:26:39 +00:00
Amber Brown
5d976c0c7c
Fix worker TLS ( #4492 )
...
* load cert
* changelog
* fix
2019-01-28 17:18:33 +00:00
Erik Johnston
b872c7b1b4
Split up event validation between event and builder
...
The validator was being run on the EventBuilder objects, and so the
validator only checked a subset of fields. With the upcoming
EventBuilder refactor even fewer fields will be there to validate.
To get around this we split the validation into those that can be run
against an EventBuilder and those run against a fully fledged event.
2019-01-28 17:00:14 +00:00
Erik Johnston
855a151015
Refactor event signing to work on dicts
...
This is in preparation for making EventBuilder format agnostic, which
means event signing should be done against the event dict rather than
the EventBuilder object.
2019-01-28 16:42:10 +00:00
Amber Brown
7072fe3084
Fix UPSERTs on SQLite 3.24+ ( #4477 )
2019-01-28 15:43:32 +00:00
Andrew Morgan
f0e96ab66a
Change return syntax in doc string
2019-01-28 14:09:45 +00:00
Andrew Morgan
4026d555fa
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-28 14:08:24 +00:00
Andrew Morgan
1ce463963d
Reuse predecessor method
2019-01-28 14:08:18 +00:00
Richard van der Hoff
3bd0f1a4a3
docstrings for _RoutingResult
2019-01-28 12:43:09 +00:00
Richard van der Hoff
0fd5b3b53e
Handle IP literals explicitly
...
We don't want to be doing .well-known lookups on these guys.
2019-01-28 10:34:30 +00:00
Richard van der Hoff
51958df766
MatrixFederationAgent: factor out routing logic
...
This is going to get too big and unmanageable.
2019-01-28 10:34:30 +00:00
Richard van der Hoff
d840019192
Fix idna and ipv6 literal handling in MatrixFederationAgent ( #4487 )
...
Turns out that the library does a better job of parsing URIs than our
reinvented wheel. Who knew.
There are two things going on here. The first is that, unlike
parse_server_name, URI.fromBytes will strip off square brackets from IPv6
literals, which means that it is valid input to ClientTLSOptionsFactory and
HostnameEndpoint.
The second is that we stay in `bytes` throughout (except for the argument to
ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up
with idna-encoded values being held in `unicode` variables. TBH it probably
would have been ok but it made the tests fragile.
2019-01-28 09:56:59 +00:00
Erik Johnston
ae2a957dba
Pass through room version to event auth
2019-01-25 18:31:41 +00:00
Neil Johnson
893107be78
backout v3
2019-01-25 17:27:36 +00:00
Erik Johnston
b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
...
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
Erik Johnston
431e485914
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sqlite_native_upsert
2019-01-25 14:11:17 +00:00
Richard van der Hoff
4a3f138832
Fix quoting for allowed_local_3pids example config ( #4476 )
...
If you use double-quotes here, you have to escape your backslashes. It's much
easier with single-quotes.
(Note that the existing double-backslashes are already interpreted by python's
""" parsing.)
2019-01-25 13:57:52 +00:00
Richard van der Hoff
8520bc3109
Fix Host header sent by MatrixFederationAgent ( #4468 )
...
Move the Host header logic down here so that (a) it is used if we reuse the
agent elsewhere, and (b) we can mess about with it with .well-known.
2019-01-25 12:38:16 +00:00
Andrew Morgan
9244a3089e
Fixes
2019-01-25 11:48:38 +00:00
Andrew Morgan
da0d2219d2
Clean up direct_rooms access
2019-01-25 11:37:12 +00:00
Andrew Morgan
8265995498
Use python magic
2019-01-25 11:26:06 +00:00
Andrew Morgan
0b3fd1401f
Don't require sqlite3 when using postgres ( #4466 )
2019-01-25 22:25:02 +11:00
Andrew Morgan
c4cdafa81f
Destructure account data tuple before use
2019-01-25 11:24:28 +00:00
Andrew Morgan
516456b763
Remove unnecessary null check
2019-01-25 11:22:14 +00:00
Andrew Morgan
6f3fda79ce
Move room_tag declaration to be closer to its use
2019-01-25 11:21:25 +00:00
Neil Johnson
95f871fc0d
Support room version capabilities in CS API (MSC1804)
2019-01-25 11:16:29 +00:00
Andrew Morgan
821b65aeb5
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-25 11:09:53 +00:00
Andrew Morgan
0862d35b8e
Move tag and direct state copying into separate function
2019-01-25 11:09:34 +00:00
Neil Johnson
53ef4da8c2
track unstable room v3
2019-01-25 11:04:11 +00:00
Andrew Morgan
b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
...
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
Erik Johnston
1953067136
Disable native upserts for sqlite, as they don't work
2019-01-25 10:46:49 +00:00
Erik Johnston
678a92cb56
Replace missed usages of FrozenEvent
2019-01-25 10:32:30 +00:00
Erik Johnston
a50cf929c1
Require event format version to parse or create events
2019-01-25 10:32:19 +00:00
Erik Johnston
62514bb81b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813
2019-01-25 10:07:08 +00:00
Erik Johnston
edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions
...
Add room_version param to get_pdu
2019-01-24 18:47:15 +00:00
Erik Johnston
80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
...
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
Erik Johnston
664b7a2920
Merge pull request #4437 from matrix-org/erikj/event_format_version_v2
...
Add support for persisting event format versions
2019-01-24 18:32:30 +00:00
Erik Johnston
d148c43050
Review comments
2019-01-24 18:31:23 +00:00
Erik Johnston
26f44164c8
Review comments
2019-01-24 18:28:00 +00:00
Erik Johnston
5ee1f997a8
Update make_membership_event docs
2019-01-24 18:08:08 +00:00
Erik Johnston
9139b87be4
Remove unecessary setting of outlier bit
2019-01-24 18:04:02 +00:00
Erik Johnston
b8082a5445
Use term 'out of band membership' instead
2019-01-24 17:33:19 +00:00
Andrew Morgan
a383289b0d
Merge branch 'anoa/full_search_upgraded_rooms' of github.com:matrix-org/synapse into anoa/full_search_upgraded_rooms
2019-01-24 17:23:51 +00:00
Andrew Morgan
e1781b043b
Remove redundant create event None check
2019-01-24 17:23:39 +00:00
Richard van der Hoff
03c85335d1
Apply suggestions from code review
...
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:22:09 +00:00
Richard van der Hoff
8ea509a935
Update synapse/api/filtering.py
...
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:21:35 +00:00
Andrew Morgan
50c396a7ee
Merge pull request #4461 from matrix-org/anoa/room_dir_quick_fix
...
Add a 60s timeout to filtered room directory queries
2019-01-24 15:24:35 +00:00
Andrew Morgan
075ff3ede9
Change default timeout value from 0 to None
2019-01-24 15:10:22 +00:00
Andrew Morgan
5b1dc94083
Use self.clock instead of datetime
2019-01-24 14:59:50 +00:00
Andrew Morgan
a2d85144e5
isort
2019-01-24 14:22:26 +00:00
Richard van der Hoff
4a6e863843
Merge pull request #4464 from matrix-org/rav/fix_srv_lookup
...
MatrixFederationAgent: Look up the right SRV record
2019-01-24 13:57:51 +00:00
Richard van der Hoff
f4697b5ec1
Fix UnboundLocalError in post_urlencoded_get_json ( #4460 )
...
This could cause exceptions if the id server returned 4xx responses.
2019-01-24 13:38:29 +00:00
Richard van der Hoff
afd69a0920
Look up the right SRV record
2019-01-24 13:31:43 +00:00
Neil Johnson
10b89d5c2e
Merge pull request #4435 from matrix-org/neilj/fix_threepid_auth_check
...
Neilj/fix threepid auth check
2019-01-24 13:02:50 +00:00
Andrew Morgan
5541645e80
lint
2019-01-24 12:45:32 +00:00
Andrew Morgan
068aa1d228
Time out filtered room dir queries after 60s
2019-01-24 12:44:27 +00:00
Neil Johnson
5c41b22359
Merge pull request #4458 from matrix-org/dbkr/public_baseurl_doc
...
Clarify docs for public_baseurl
2019-01-24 12:36:09 +00:00
Amber Brown
0e27501ee5
Fix UPSERT check ( #4459 )
2019-01-24 22:57:41 +11:00
David Baker
92d8a068ad
Clarify docs for public_baseurl
...
This is leading to problems with people upgrading to clients that
support MSC1730 because people have this misconfigured, so try
to make the docs completely unambiguous.
2019-01-24 10:52:06 +00:00
Amber Brown
58f6c48183
Use native UPSERTs where possible ( #4306 )
2019-01-24 21:31:54 +11:00
Richard van der Hoff
97fd29c019
Don't send IP addresses as SNI ( #4452 )
...
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files ; let's do the same here.
2019-01-24 09:34:44 +00:00
Erik Johnston
7c288c2250
Clarify the invite flows
2019-01-23 20:07:47 +00:00
Erik Johnston
07f62da55a
Remove unnecessary '_sign_event'
2019-01-23 20:04:57 +00:00
Erik Johnston
886e5acc76
Store rejected remote invite events as outliers
...
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.
In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
2019-01-23 20:04:57 +00:00
Willem Mulder
d528406cb8
Fix error message for optional dependencies
...
Signed-off-by: Willem Mulder <willemmaster@hotmail.com>
2019-01-23 18:44:57 +01:00
Erik Johnston
6a41d2a187
Add room_version param to get_pdu
...
When we add new event format we'll need to know the event format or room
version when parsing events.
2019-01-23 17:19:58 +00:00
Erik Johnston
67cd4dad81
Implement MSC 1813 - Add room version to make APIs
...
We also implement `make_membership_event` converting the returned
room version to an event format version.
2019-01-23 16:51:46 +00:00
Andrew Morgan
6b90ae6efc
Merge pull request #4445 from matrix-org/anoa/user_dir_develop_backport
...
Add metric for user dir current event stream position
2019-01-23 15:58:40 +00:00
Richard van der Hoff
a0ae475219
Merge pull request #4428 from matrix-org/rav/matrix_federation_agent
...
Move SRV magic into an Agent-like thing
2019-01-23 15:50:25 +00:00
Andrew Morgan
82a92ba535
Add metric for user dir current event stream position
2019-01-23 15:34:47 +00:00
Erik Johnston
be1065af59
isort
2019-01-23 11:48:16 +00:00
Erik Johnston
c5a296b10c
Add support for persisting event format versions
...
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.
We specify the format version rather than the room version as:
1. We don't necessarily know the room version, existing events may be
either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
events in the same room reported to be of different versions, which
sounds annoying.
2019-01-23 11:30:01 +00:00
Jason Robinson
6f680241bd
Fix flake8 issues
...
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:53:48 +02:00
Erik Johnston
90743c9d89
Fixup removal of duplicate user_ips
rows ( #4432 )
...
* Remove unnecessary ORDER BY clause
* Add logging
* Newsfile
2019-01-23 19:45:18 +11:00
Amber Brown
6129e52f43
Support ACME for certificate provisioning ( #4384 )
2019-01-23 19:39:06 +11:00
Jason Robinson
0516dc4d85
Remove openid resource from default config
...
Instead document it commented out.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson
d39b7b6d38
Document servlet_groups
parameters
...
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson
82e13662c0
Split federation OpenID userinfo endpoint out of the federation resource
...
This allows the OpenID userinfo endpoint to be active even if the
federation resource is not active. The OpenID userinfo endpoint
is called by integration managers to verify user actions using the
client API OpenID access token. Without this verification, the
integration manager cannot know that the access token is valid.
The OpenID userinfo endpoint will be loaded in the case that either
"federation" or "openid" resource is defined. The new "openid"
resource is defaulted to active in default configuration.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson
a17bac171f
Make SynapseHomeServer _http_listener use self.get_reactor()
...
For all the homeserver classes, only the FrontendProxyServer passes
its reactor when doing the http listen. Looking at previous PR's looks
like this was introduced to make it possible to write a test, otherwise
when you try to run a test with the test homeserver it tries to
do a real bind to a port. Passing the reactor that the homeserver
is instantiated with should probably be the right thing to do anyway?
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson
4f8f41c824
Make FederationReaderServer _http_listen use self.get_reactor()
...
For all the homeserver classes, only the FrontendProxyServer passes
its reactor when doing the http listen. Looking at previous PR's looks
like this was introduced to make it possible to write a test, otherwise
when you try to run a test with the test homeserver it tries to
do a real bind to a port. Passing the reactor that the homeserver
is instantiated with should probably be the right thing to do anyway?
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Jason Robinson
899e60be80
Add parameterized Python module to test dependencies
...
Allows running parameterized tests. BSD license.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Richard van der Hoff
7021784d46
put resolve_service in an object
...
this makes it easier to stub things out for tests.
2019-01-22 20:35:12 +00:00
Richard van der Hoff
53a327b4d5
Require that service_name be a byte string
...
it is only ever a bytes now, so let's enforce that.
2019-01-22 20:35:12 +00:00
Richard van der Hoff
fe212bbe4a
Kill off matrix_federation_endpoint
...
this thing is now redundant.
2019-01-22 20:34:35 +00:00
Richard van der Hoff
7871146667
Make MatrixFederationClient use MatrixFederationAgent
...
... instead of the matrix_federation_endpoint
2019-01-22 20:34:35 +00:00
Richard van der Hoff
44be7513bf
MatrixFederationAgent
...
Pull the magic that is currently in matrix_federation_endpoint and friends into
an agent-like thing
2019-01-22 20:34:35 +00:00
Neil Johnson
c99c2d58d7
move guard out of is_threepid_reserved and into register.py
2019-01-22 17:47:00 +00:00
Neil Johnson
d619b113ed
Fix None guard in config.server.is_threepid_reserved
2019-01-22 16:52:29 +00:00
Erik Johnston
12699a701f
Merge pull request #4434 from matrix-org/erikj/fix_user_ips_dedup
...
Fix bug when removing duplicate rows from user_ips
2019-01-22 16:51:57 +00:00
Erik Johnston
7f503f83b9
Refactor to rewrite the SQL instead
2019-01-22 16:31:05 +00:00
Neil Johnson
388c164aea
Merge pull request #4423 from matrix-org/neilj/disable_msisdn_on_registration
...
Config option to disable requesting MSISDN on registration
2019-01-22 16:23:08 +00:00
Erik Johnston
1c9704f8ab
Don't shadow params
2019-01-22 16:20:33 +00:00
Andrew Morgan
766a172b99
lint
2019-01-22 13:51:40 +00:00
Erik Johnston
2557531f0f
Fix bug when removing duplicate rows from user_ips
...
This was caused by accidentally overwritting a `last_seen` variable
in a for loop, causing the wrong value to be written to the progress
table. The result of which was that we didn't scan sections of the table
when searching for duplicates, and so some duplicates did not get
deleted.
2019-01-22 13:33:46 +00:00
Andrew Morgan
117bc94cd2
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-22 13:16:51 +00:00
Andrew Morgan
277e50462d
Do not return in a deferred function
2019-01-22 12:40:26 +00:00
Andrew Morgan
c4875d8c76
Prevent duplicate room IDs in m.direct
2019-01-22 12:13:46 +00:00
Andrew Morgan
c433f61091
Ensure new filter is actually created
2019-01-22 12:06:36 +00:00
Andrew Morgan
c9bfb058d8
Fix a bug with single-room search searching all rooms
...
* Create a new method for getting predecessor rooms
* Remove formatting change
2019-01-22 12:00:41 +00:00
Andrew Morgan
8086a5c05e
Fix comments
2019-01-22 11:16:23 +00:00
Richard van der Hoff
6bfa735a69
Make key fetches use regular federation client ( #4426 )
...
All this magic is redundant.
2019-01-22 11:04:20 +00:00
Andrew Morgan
48951f437f
Join logic covers both room creator and arbitrary users
2019-01-22 11:00:04 +00:00
Andrew Morgan
8c85f0833d
tags, m.direct copying over correctly
2019-01-22 11:00:04 +00:00
Andrew Morgan
25d64a846a
Fix typos
2019-01-22 11:00:04 +00:00
Andrew Morgan
ea8903fcc9
Migrating dm and room tags work for migrator
2019-01-22 11:00:04 +00:00
Andrew Morgan
887ca93a1b
Prevent crash on user who doesn't have any direct rooms
2019-01-22 11:00:04 +00:00
Andrew Morgan
4ff6d22245
Preserve DM status of a room on upgrade
...
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-22 11:00:04 +00:00
Andrew Morgan
75942af1db
Fix typo
2019-01-22 11:00:04 +00:00
Richard van der Hoff
33a55289cb
Refactor and bugfix for resove_service ( #4427 )
2019-01-22 10:59:27 +00:00
Amber Brown
23b0813599
Require ECDH key exchange & remove dh_params ( #4429 )
...
* remove dh_params and set better cipher string
2019-01-22 21:58:50 +11:00
Neil Johnson
1b53cc3cb4
fix line length
2019-01-21 15:17:20 +00:00
Neil Johnson
5349262302
Config option to disable requesting MSISDN on registration
2019-01-21 14:59:37 +00:00
Erik Johnston
35e1d67b4e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_v2_invite_server
2019-01-21 14:04:19 +00:00
Erik Johnston
5f54765587
Merge pull request #4390 from matrix-org/erikj/versioned_fed_apis
...
Add groundwork for new versions of federation APIs
2019-01-21 11:44:05 +00:00
Andrew Morgan
702c4b750c
Migrate encryption state on room upgrade ( #4411 )
...
* Migrate encryption state on room upgrade
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
* Add changelog file
2019-01-21 20:42:58 +11:00
Erik Johnston
25dd56ace3
Fix race when persisting create event ( #4404 )
...
* Fix race when persisting create event
When persisting a chunk of DAG it is sometimes requried to do a state
resolution, which requires knowledge of the room version. If this
happens while we're persisting the create event then we need to use that
event rather than attempting to look it up in the database.
2019-01-18 23:17:04 +11:00
Richard van der Hoff
de6888e7ce
Remove redundant WrappedConnection ( #4409 )
...
* Remove redundant WrappedConnection
The matrix federation client uses an HTTP connection pool, which times out its
idle HTTP connections, so there is no need for any of this business.
2019-01-18 23:07:38 +11:00
Andrew Morgan
df3a661e4a
Search for messages across predecessor rooms
...
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-18 11:19:20 +00:00
Richard van der Hoff
676cf2ee26
Fix incorrect logcontexts after a Deferred was cancelled ( #4407 )
2019-01-17 14:00:23 +00:00
Richard van der Hoff
9feb5d0b71
sign_request -> build_auth_headers ( #4408 )
...
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
Richard van der Hoff
3982a6ee07
Changing macaroon_secret_key no longer logs you out ( #4387 )
2019-01-16 23:14:41 +00:00
Richard van der Hoff
05e1296649
don't store more remote device lists if they have more than 1K devices ( #4397 )
2019-01-16 23:14:11 +00:00
Andrej Shadura
64cf6788d9
Depend on pymacaroons >= 0.13.0 instead on pymacaroons-pynacl
...
Since 0.13.0, pymacaroons works correctly with pynacl, so there
isn’t any more reason to depend on an outdated pynacl fork.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
2019-01-16 10:54:41 +01:00
Andrej Shadura
fab948120f
Use msgpack instead of msgpack-python
...
The package msgpack-python has been deprecated.
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
2019-01-16 10:53:49 +01:00
Neil Johnson
9ec56d6935
ALL_USER_TYPES should be a tuple
2019-01-15 14:38:15 +00:00
Erik Johnston
4a4d2e17bc
Add /v2/invite federation API
2019-01-15 13:22:44 +00:00
Erik Johnston
bb63e7ca4f
Add groundwork for new versions of federation APIs
2019-01-15 11:14:34 +00:00
Amber Brown
a35c66a00b
Remove duplicates in the user_ips table and add an index ( #4370 )
2019-01-12 06:21:50 +11:00
Amber Brown
522dada206
Merge remote-tracking branch 'origin/master' into develop
2019-01-11 02:22:48 +11:00
Amber Brown
6dc06c3775
version
2019-01-11 01:56:37 +11:00
Richard van der Hoff
e0910d0145
Merge branch rav/macaroon_key_fix_0.34 into rav/macaroon_key_fix_0.34.1
...
Fixes #4371
2019-01-10 14:12:50 +00:00
Richard van der Hoff
ba41aeed6a
Revert "Fix macaroon_secret_key fallback logic"
...
This is already fixed in 0.34.1, by 59f93bb
This reverts commit efc522c55e
.
2019-01-10 14:09:26 +00:00
Richard van der Hoff
aa70d24125
Merge branch 'rav/macaroon_key_fix' into rav/macaroon_key_fix_0.34
2019-01-10 12:58:33 +00:00
Richard van der Hoff
efc522c55e
Fix macaroon_secret_key fallback logic
2019-01-10 12:57:27 +00:00
Richard van der Hoff
566947ff34
Skip macaroon check for access tokens in the db
2019-01-10 12:57:21 +00:00
Richard van der Hoff
353f2407b7
Fix fallback to signing key for macaroon-secret-key
2019-01-10 12:42:56 +00:00
Richard van der Hoff
d566e6b17a
Merge branch 'master' into develop
2019-01-09 15:00:46 +00:00
Richard van der Hoff
998f5225c1
0.34.1
2019-01-09 14:53:54 +00:00
Erik Johnston
55c3e853c5
Merge pull request #4368 from matrix-org/erikj/better_errors
...
Fixup docstrings for matrixfederationclient
2019-01-09 11:39:28 +00:00
Erik Johnston
484867d35d
Merge branch 'develop' into release-v0.34.1
2019-01-09 11:38:13 +00:00
Amber Brown
7960c26fda
Fix adding new rows instead of updating them if one of the key values is a NULL in upserts. ( #4369 )
2019-01-09 22:26:25 +11:00
Erik Johnston
34ea14139d
Fixup docstrings for matrixfederationclient
2019-01-09 09:25:59 +00:00
Erik Johnston
d422570e9b
Use RequestSendFailed when fail to parse content type headers
2019-01-08 14:22:18 +00:00
Erik Johnston
1371d5b798
Don't log stack traces for HTTP error responses
2019-01-08 12:28:30 +00:00
Richard van der Hoff
ba85786d71
0.34.1rc1
2019-01-08 11:15:34 +00:00
Erik Johnston
b970cb0e96
Refactor request sending to have better excpetions ( #4358 )
...
* Correctly retry and back off if we get a HTTPerror response
* Refactor request sending to have better excpetions
MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).
This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
2019-01-08 11:04:28 +00:00
Amber Brown
1dcb086f33
Fix synapse.config.__main__ on python 3 ( #4356 )
2019-01-08 10:03:09 +11:00
Travis Ralston
bc1fa8cd01
Add GET account data routes ( #4303 )
...
As per https://github.com/matrix-org/matrix-doc/issues/1339
2019-01-07 10:27:54 +00:00
Richard van der Hoff
b7c0218812
Check jinja version for consent resource ( #4327 )
...
* Raise a ConfigError if an invalid resource is specified
* Require Jinja 2.9 for the consent resource
* changelog
2019-01-07 10:14:31 +00:00
Michael Telatynski
a27e501b09
fix the check for whether is_url
to match all the other ones in codebase ( #3405 )
...
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2019-01-06 21:43:25 +00:00
Matthew Hodgson
27128145e6
fix NPE in /messages by checking if all events were filtered out ( #4330 )
2019-01-02 23:37:39 +00:00
Neil Johnson
84b6fae1f5
Ensure synchrotrons can access is_support_user in the storage layer
2019-01-02 10:19:59 +00:00
Amber Brown
d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console ( #4334 )
2018-12-29 23:12:30 +11:00
Will Hunt
0708f437cc
Log roomid along with Unknown room ( #4297 )
2018-12-24 10:49:42 +01:00
Richard van der Hoff
cd018e3543
Merge pull request #4307 from matrix-org/erikj/v2_rooms
...
Add v2 room version
2018-12-24 10:48:59 +01:00
Richard van der Hoff
f85676cc93
Return well_known in /login response ( #4319 )
...
... as per MSC1730.
2018-12-24 10:44:33 +01:00
Richard van der Hoff
9c2af7b2c5
Add a script to generate a clean config file ( #4315 )
2018-12-22 02:04:57 +11:00
Richard van der Hoff
f3561f8d86
Fix indentation in default config ( #4313 )
...
These settings are not supposed to be under 'listeners'.
2018-12-22 02:01:43 +11:00
Amber Brown
ea6abf6724
Fix IP URL previews on Python 3 ( #4215 )
2018-12-22 01:56:13 +11:00
Amber Brown
c26f49a664
Make the dependencies more like a standard Python project and hook up the optional dependencies to setuptools ( #4298 )
2018-12-22 01:37:26 +11:00
Richard van der Hoff
0b26feb422
Merge branch 'master' into develop
2018-12-20 16:48:47 +00:00
Richard van der Hoff
6dd7271243
Prepare 0.34 release
2018-12-20 12:07:15 +00:00