Commit Graph

539 Commits

Author SHA1 Message Date
Richard van der Hoff
495975e231 Optimisation for filter_events_for_server
We're better off hashing just the event_id than the whole ((type, state_key),
event_id) tuple - so use a dict instead of a set.

Also, iteritems > items.
2018-10-08 13:46:52 +01:00
Erik Johnston
8164f6daf3 Newsfile 2018-10-05 11:25:09 +01:00
Richard van der Hoff
c6dbd216e6
Merge pull request #3995 from matrix-org/rav/no_deextrem_outliers
Fix bug in forward_extremity update logic
2018-10-04 16:27:05 +01:00
Amber Brown
d86794325f Merge branch 'master' into develop 2018-10-04 22:41:52 +10:00
Amber Brown
92faeb2a3f changelog 2018-10-04 22:38:10 +10:00
Richard van der Hoff
a59d899668 Pin to prometheus_client<0.4 to avoid renaming all of our metrics 2018-10-03 17:20:15 +01:00
Erik Johnston
8935ec5a93
Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetion
Fix handling of rejected threepid invites
2018-10-03 14:47:41 +01:00
Erik Johnston
81e2813948
Merge pull request #3996 from matrix-org/erikj/fix_bg_iteration
Fix exception in background metrics collection
2018-10-03 14:14:38 +01:00
Erik Johnston
93a8603904 Newsfile 2018-10-03 11:59:05 +01:00
Erik Johnston
6e0c66f651 Newsfile 2018-10-03 11:40:02 +01:00
Erik Johnston
c69faf8c4a Newsfile 2018-10-03 11:29:44 +01:00
Richard van der Hoff
2a4ea3baa8 fix newsfile name 2018-10-03 07:09:25 +01:00
Richard van der Hoff
ae61ade891 Fix bug in forward_extremity update logic
An event does not stop being a forward_extremity just because an outlier or
rejected event refers to it.
2018-10-02 23:36:08 +01:00
Amber Brown
ccce6f26a6 changelog 2018-10-03 02:16:21 +10:00
Erik Johnston
25baf3b2ac
Merge pull request #3991 from matrix-org/erikj/fix_pop_retry_cache
Fix bug when invalidating destination retry timings
2018-10-02 16:42:28 +01:00
Amber Brown
058a4c665e
Remove Jenkins & other old dev junk (#3988) 2018-10-03 00:59:11 +10:00
Erik Johnston
8f614f842d Newsfile 2018-10-02 15:54:31 +01:00
Richard van der Hoff
2b8d28b095
Merge pull request #3960 from matrix-org/rav/fix_missing_create_event_error
Fix error handling for missing auth_event
2018-10-02 13:57:54 +01:00
Amber Brown
7232917f12
Disable frozen dicts by default (#3987) 2018-10-02 22:53:47 +10:00
Erik Johnston
eeb755d17b Newsfile 2018-10-02 12:06:25 +01:00
Richard van der Hoff
8c41b0ca66
Merge pull request #3989 from matrix-org/rav/better_stacktraces
Avoid reraise, to improve stacktraces
2018-10-02 10:47:38 +01:00
Erik Johnston
bc29946809
Merge pull request #3986 from matrix-org/erikj/fix_sync_with_redacted_state
Fix lazy loaded sync with rejected state events
2018-10-02 11:38:35 +02:00
Richard van der Hoff
8174c6725b Avoid reraise, to improve stacktraces 2018-10-01 18:50:34 +01:00
Richard van der Hoff
5908a8f6f5 Merge branch 'develop' into rav/fix_missing_create_event_error 2018-10-01 17:16:20 +01:00
Amber Brown
abdc141c2b
Update instructions to point to pip install (#3985) 2018-10-02 01:08:38 +10:00
Richard van der Hoff
b5b93f45d5
Merge pull request #3968 from matrix-org/rav/fix_federation_errors
Fix exceptions when handling incoming transactions
2018-10-01 15:54:24 +01:00
Amber Brown
6e05fd032c
Fix userconsent on Python 3 (#3938) 2018-10-02 00:11:58 +10:00
Erik Johnston
5fa27eac78 Newsfile 2018-10-01 14:24:41 +01:00
Erik Johnston
8f5c23d0cd
Merge pull request #3933 from matrix-org/erikj/destination_retry_cache
Add a five minute cache to get_destination_retry_timings
2018-10-01 14:20:30 +02:00
Richard van der Hoff
53c5fa4e6c
Further reduce the size of the docker image (#3972)
Rewrite the dockerfile as a multistage build: this means we can get rid of a whole load of cruft which we don't need.
2018-10-01 12:29:17 +01:00
Bruno Windels
9a8bbc9a59 add --no-admin flag to registration script (#3836) 2018-09-28 13:36:56 +01:00
Richard van der Hoff
3deaad2fb4
Merge pull request #3964 from matrix-org/rav/remove_localhost_checks
remove spurious federation checks on localhost
2018-09-28 13:35:47 +01:00
Richard van der Hoff
26557c9db4
Merge pull request #3980 from matrix-org/rav/remove_broken_cache_call
Remove redundant call to start_get_pdu_cache
2018-09-28 13:13:19 +01:00
Jan Christian Grünhage
5304449738 build python 3 docker images on circle CI (#3976) 2018-09-28 12:36:35 +01:00
Richard van der Hoff
19475cf337 Remove redundant call to start_get_pdu_cache
I think this got forgotten in #3932. We were getting away with it because it
was the last call in this function.
2018-09-28 12:01:23 +01:00
Richard van der Hoff
9c8cec5dab Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache 2018-09-28 10:51:09 +01:00
Richard van der Hoff
f094f715cf Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors 2018-09-27 15:18:21 +01:00
Richard van der Hoff
36c62a67c4
Merge pull request #3794 from matrix-org/erikj/faster_typing
Improve performance of getting typing updates for replication
2018-09-27 15:14:26 +01:00
Richard van der Hoff
b5c976347b
Merge pull request #3946 from matrix-org/michaelkaye/automate_docker_hub_upload
Build and push docker image to hub automatically
2018-09-27 15:07:41 +01:00
Richard van der Hoff
e1e3e77bfd
Merge pull request #3967 from matrix-org/rav/federation_handler_cleanups
Clarifications in FederationHandler
2018-09-27 15:06:59 +01:00
Amber Brown
6de3884e5e
Merge pull request #3965 from matrix-org/rav/notify_app_services_bg_process
Run notify_app_services as a bg process
2018-09-27 23:40:30 +10:00
Amber Brown
a512e637ac
Merge pull request #3970 from schnuffle/develop-py3
Replaced all occurences of e.message with str(e)
2018-09-27 23:39:45 +10:00
Amber Brown
b3064532d0
Run our oldest supported configuration in CI (#3952) 2018-09-27 23:21:54 +10:00
Amber Brown
861c063ebc
Update 3970.bugfix 2018-09-27 22:47:01 +10:00
Schnuffle
a873896096 Added changelog fragment 2018-09-27 14:01:44 +02:00
Richard van der Hoff
51d33d5178
Merge pull request #3961 from matrix-org/neilj/lock_mau_upserts
fix #3854 MAU transaction errors
2018-09-27 12:31:37 +01:00
Michael Kaye
74bbdd0412 Do the changelog.d dance 2018-09-27 12:01:43 +01:00
Richard van der Hoff
948a6d8776 changelog 2018-09-27 11:37:39 +01:00
Richard van der Hoff
ad8e137062 changelog 2018-09-27 11:31:51 +01:00
Richard van der Hoff
92abd3d6d6
Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2
Logging improvements
2018-09-27 11:27:46 +01:00
Richard van der Hoff
4a15a3e4d5
Include eventid in log lines when processing incoming federation transactions (#3959)
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.

This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
2018-09-27 11:25:34 +01:00
Richard van der Hoff
ae6ad4cf41
docstrings and unittests for storage.state (#3958)
I spent ages trying to figure out how I was going mad...
2018-09-27 11:22:25 +01:00
Travis Ralston
2a7b3439de Changelog 2018-09-26 13:51:34 -06:00
Richard van der Hoff
e70b4ce069 Logging improvements
Some logging tweaks to help with debugging incoming federation transactions
2018-09-26 17:36:14 +01:00
Richard van der Hoff
3c37c7e45b changelog 2018-09-26 17:01:30 +01:00
Richard van der Hoff
0c4a99ea2d changelog 2018-09-26 16:54:54 +01:00
Richard van der Hoff
ab59f3d8da changelog 2018-09-26 16:52:24 +01:00
Neil Johnson
d514608b5c towncrier 2018-09-26 16:19:53 +01:00
Amber Brown
d4e0861ff9
Reduce the load on our CI (#3957)
* changelog

* reduce circleci config

* plus a handy script

* fix regex
2018-09-27 00:23:21 +10:00
Richard van der Hoff
a5e70b31a1 changelog 2018-09-26 14:41:14 +01:00
Richard van der Hoff
4e8276a34a
Merge pull request #3956 from matrix-org/rav/fix_expiring_cache_len
Fix ExpiringCache.__len__ to be accurate
2018-09-26 13:10:13 +01:00
Richard van der Hoff
5b4028fa78 Merge branch 'rav/fix_expiring_cache_len' into erikj/destination_retry_cache 2018-09-26 12:55:53 +01:00
Amber Brown
66a1d57adb
Merge pull request #3948 from matrix-org/rav/no_symlink_synctl
Move synctl into top dir to avoid a symlink
2018-09-26 21:41:58 +10:00
Amber Brown
c2185f14d7
Merge pull request #3924 from matrix-org/rav/clean_up_on_receive_pdu
Comments and interface cleanup for on_receive_pdu
2018-09-26 21:41:26 +10:00
Richard van der Hoff
8ac9fa7375 changelog 2018-09-26 12:35:10 +01:00
Richard van der Hoff
f65163627f
Merge pull request #3911 from matrix-org/jcgruenhage/docker-support-python3
make python 3 work in the docker container
2018-09-25 15:18:09 +01:00
Richard van der Hoff
a1cd37390f Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache 2018-09-25 12:03:54 +01:00
Richard van der Hoff
4c3e7eeec5
Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_caches
Fix some instances of ExpiringCache not expiring cache items
2018-09-25 12:02:57 +01:00
Jérémy Farnaud
6cf261930a added "media-src: 'self'" to CSP for resources (#3578)
Synapse doesn’t allow for media resources to be played directly from
Chrome. It is a problem for users on other networks (e.g. IRC)
communicating with Matrix users through a gateway. The gateway sends
them the raw URL for the resource when a Matrix user uploads a video
and the video cannot be played directly in Chrome using that URL.

Chrome argues it is not authorized to play the video because of the
Content Security Policy. Chrome checks for the "media-src" policy which
is missing, and defauts to the "default-src" policy which is "none".

As Synapse already sends "object-src: 'self'" I thought it wouldn’t be
a problem to add "media-src: 'self'" to the CSP to fix this problem.
2018-09-25 11:55:02 +01:00
Richard van der Hoff
94f7befc31
Merge pull request #3925 from matrix-org/erikj/fix_producers_unregistered
Fix spurious exceptions when client closes conncetion
2018-09-25 11:52:06 +01:00
Richard van der Hoff
bd469adaa9 changelog 2018-09-25 11:22:17 +01:00
Richard van der Hoff
45a1053d44 changelog 2018-09-25 10:45:34 +01:00
Matthew Hodgson
787d22ed6c
Only lazy load self-members on initial sync
Given we have disabled lazy loading for incr syncs in #3840, we can make self-LL more efficient by only doing it on initial sync.  Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
2018-09-25 00:49:26 +01:00
Amber Brown
04eed80a73 Merge branch 'master' into develop 2018-09-24 23:42:25 +10:00
Erik Johnston
5230bc1471 Newsfile 2018-09-21 15:05:37 +01:00
Erik Johnston
d3f80cbc9c Newsfile 2018-09-21 14:24:39 +01:00
Erik Johnston
ad53a5497d
Merge pull request #3927 from matrix-org/erikj/handle_background_errors
Handle exceptions thrown by background tasks
2018-09-21 09:26:30 +01:00
Matthew Hodgson
a2ddaa90f2
Always LL ourselves if we're in a room to simplify clients (#3916)
Should fix https://github.com/vector-im/riot-web/issues/7209
2018-09-20 21:21:54 +01:00
Erik Johnston
168491c412 Newsfile 2018-09-20 16:16:52 +01:00
Jan Christian Grünhage
3af7a95a9d add changelog 2018-09-20 14:55:11 +02:00
Erik Johnston
13f6f1624b Newsfile 2018-09-20 13:52:09 +01:00
Neil Johnson
23b53b4ef8
Merge pull request #3868 from matrix-org/neilj/fix_room_invite_mail_links
Neilj/fix room invite mail links
2018-09-20 13:32:38 +01:00
Richard van der Hoff
6bb9a4b1d6 changelog 2018-09-20 13:15:20 +01:00
Amber Brown
1f3f5fcf52
Fix client IPs being broken on Python 3 (#3908) 2018-09-20 20:14:34 +10:00
Erik Johnston
3fd68d533b
Merge pull request #3914 from matrix-org/erikj/remove_retry_cache
Remove get_destination_retry_timings cache
2018-09-20 10:54:49 +01:00
Amber Brown
741571cf22
Add a way to run tests in PostgreSQL in Docker (#3699) 2018-09-20 18:12:45 +10:00
Amber Brown
aeca5a5ed5
Add a regression test for logging on failed connections (#3912) 2018-09-20 16:28:18 +10:00
Richard van der Hoff
642199570c
Improve the logging when handling a federation transaction (#3904)
Let's try to rationalise the logging that happens when we are processing an
incoming transaction, to make it easier to figure out what is going wrong when
they take ages. In particular:

- make everything start with a [room_id event_id] prefix
- make sure we log a warning when catching exceptions rather than just turning
  them into other, more cryptic, exceptions.
2018-09-19 17:28:18 +01:00
Erik Johnston
ce846bb620 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/faster_typing 2018-09-19 15:08:36 +01:00
Erik Johnston
1a24d4effa Newsfile 2018-09-19 15:03:05 +01:00
Erik Johnston
bbab6ebfd9 Fix up changelog and remove spurious comment 2018-09-19 14:45:14 +01:00
Erik Johnston
83ee5592a5 Newsfile 2018-09-19 14:22:59 +01:00
Erik Johnston
80d2d50f47 Fixup 2018-09-19 11:19:47 +01:00
Erik Johnston
6bbe3d5732 Newsfile 2018-09-19 10:43:13 +01:00
Richard van der Hoff
05b9937cd7 update changelog for #3909 2018-09-19 09:17:54 +01:00
Amber Brown
47c02e6332
Merge pull request #3909 from turt2live/travis/fix-logging-1
Fix matrixfederationclient.py logging: Destination is a string
2018-09-19 18:14:47 +10:00
Amber Brown
3f0d8e6b09
Remove documentation referencing Cygwin (#3873) 2018-09-19 18:14:30 +10:00
Amber Brown
3d6b24fb1b
Merge pull request #3907 from matrix-org/rav/set_sni_to_server_name
Set SNI to the server_name, not whatever was in the SRV record
2018-09-19 17:59:33 +10:00
Amber Brown
f773ecbd61
Merge pull request #3903 from matrix-org/rav/increase_get_missing_events_timeout
Bump timeout on get_missing_events request
2018-09-19 17:57:48 +10:00
Travis Ralston
13b31a9baf Changelog 2018-09-18 15:30:25 -06:00