Amber Brown
df758e155d
Use <meta> tags to discover the per-page encoding of html previews ( #4183 )
2018-11-15 11:05:08 -06:00
Neil Johnson
bf648c37e7
release 0.33.9rc1
2018-11-14 11:45:52 +00:00
Richard van der Hoff
c1efcd7c6a
Add a test for the public T&Cs form
2018-11-14 10:46:27 +00:00
Richard van der Hoff
83a5f459aa
Fix an internal server error when viewing the public privacy policy
2018-11-14 10:21:07 +00:00
David Baker
0869566ad3
Merge pull request #4113 from matrix-org/dbkr/e2e_backup_versions_are_numbers
...
Make e2e backup versions numeric in the DB
2018-11-14 07:55:48 +00:00
David Baker
bca3b91c2d
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backup_versions_are_numbers
2018-11-09 18:35:02 +00:00
Erik Johnston
db5a1c059a
Merge pull request #4166 from matrix-org/erikj/drop_unknown_events
...
Drop incoming events from federation for unknown rooms
2018-11-09 17:59:34 +00:00
Erik Johnston
dc59ad5334
Remove hack to support rejoining rooms
2018-11-09 14:58:09 +00:00
David Baker
d44dea0223
pep8
2018-11-09 14:38:31 +00:00
David Baker
4f93abd62d
add docs
2018-11-09 13:25:38 +00:00
Erik Johnston
30dd27afff
Simplify to always drop events if server isn't in the room
2018-11-09 11:36:45 +00:00
Richard van der Hoff
3cecf5340d
Update synapse/federation/federation_server.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2018-11-09 11:28:25 +00:00
Richard van der Hoff
9bce065a53
Update synapse/federation/federation_server.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2018-11-09 11:28:22 +00:00
David Baker
d3fa6194f7
Remove unnecessary str()
2018-11-09 11:11:31 +00:00
Erik Johnston
7b22421a7b
Merge pull request #4164 from matrix-org/erikj/fix_device_comparison
...
Fix noop checks when updating device keys
2018-11-08 14:37:20 +00:00
Erik Johnston
5ebed18692
Lets convert bytes to unicode instead
2018-11-08 12:33:13 +00:00
Erik Johnston
b1a22b24ab
Fix noop checks when updating device keys
...
Clients often reupload their device keys (for some reason) so its
important for the server to check for no-ops before sending out device
list update notifications.
The check is broken in python 3 due to the fact comparing bytes and
unicode always fails, and that we write bytes to the DB but get unicode
when we read.
2018-11-08 12:18:38 +00:00
Erik Johnston
9417986f77
Drop PDUs of unknown rooms
...
When we receive events over federation we will need to know the room
version to be able to correctly handle them, e.g. once we start changing
event formats. Currently, we attempt to handle events in unknown rooms.
2018-11-08 12:11:20 +00:00
Richard van der Hoff
0a1fc52971
fix parse_string docstring
2018-11-08 11:12:29 +00:00
hera
2b075fb03a
Fix encoding error for consent form on python3
...
The form was rendering this as "b'01234....'".
-- richvdh
2018-11-08 11:05:39 +00:00
Amber Brown
b3708830b8
Fix URL preview bugs (type error when loading cache from db, content-type including quotes) ( #4157 )
2018-11-08 01:37:43 +11:00
Travis Ralston
0f5e51f726
Add config variables for enabling terms auth and the policy name ( #4142 )
...
So people can still collect consent the old way if they want to.
2018-11-06 10:32:34 +00:00
Hubert Chathi
f1087106cf
handle empty backups according to latest spec proposal ( #4123 )
...
fixes #4056
2018-11-05 17:59:29 -05:00
Amber Brown
efdcbbe46b
Tests for user consent resource ( #4140 )
2018-11-06 05:53:44 +11:00
Erik Johnston
bc80b3f454
Add helpers for getting prev and auth events ( #4139 )
...
* Add helpers for getting prev and auth events
This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
Amber Brown
0467384d2f
Set the encoding to UTF8 in the default logconfig ( #4138 )
2018-11-03 02:28:07 +11:00
Erik Johnston
90d713b8c6
Merge pull request #4137 from matrix-org/erikj/clean_up_events
...
Clean up event accesses and tests
2018-11-02 14:12:49 +00:00
Erik Johnston
b86d05a279
Clean up event accesses and tests
...
This is in preparation to refactor FrozenEvent to support different
event formats for different room versions
2018-11-02 13:44:14 +00:00
Amber Brown
cb7a6b2379
Fix typing being reset causing infinite syncs ( #4127 )
2018-11-03 00:19:23 +11:00
Richard van der Hoff
efb9343c8c
Merge pull request #4132 from matrix-org/rav/fix_device_list_locking
...
Fix locked upsert on device_lists_remote_cache
2018-11-02 10:50:53 +00:00
Richard van der Hoff
00f12e00f8
Merge pull request #4133 from matrix-org/travis/fix-terms-auth
...
Fix logic error that prevented guests from seeing the privacy policy
2018-11-02 10:50:43 +00:00
Erik Johnston
b199534518
Merge pull request #4135 from matrix-org/erikj/fix_state_res_none
...
Fix None exception in state res v2
2018-11-02 10:45:57 +00:00
Richard van der Hoff
350f654e7b
Add unique indexes to a couple of tables
...
The indexes on device_lists_remote_extremeties can be unique, and they
therefore should, to ensure that the db remains consistent.
2018-11-02 10:36:13 +00:00
Richard van der Hoff
50e328d1e7
Remove redundant database locks for device list updates
...
We can rely on the application-level per-user linearizer.
2018-11-02 10:36:13 +00:00
Erik Johnston
54aec35867
Fix None exception in state res v2
2018-11-02 10:29:19 +00:00
Travis Ralston
642505abc3
Fix logic error that prevented guests from seeing the privacy policy
2018-11-01 16:48:32 -06:00
Travis Ralston
c68aab1536
Merge pull request #4004 from matrix-org/travis/login-terms
...
Add m.login.terms to the registration flow
2018-11-01 11:03:38 -06:00
Erik Johnston
1b21e771d0
Merge pull request #4128 from matrix-org/erikj/state_res_v2_version
...
Add STATE_V2_TEST room version
2018-11-01 13:17:57 +00:00
Erik Johnston
b3dd6fa981
Add STATE_V2_TEST room version
2018-11-01 11:43:46 +00:00
Amber Brown
073d400b84
Merge branch 'master' into develop
2018-11-01 21:32:12 +11:00
Amber Brown
aa98e38896
version
2018-11-01 21:28:35 +11:00
Travis Ralston
a8d41c6aff
Include a version query string arg for the consent route
2018-10-31 13:19:28 -06:00
Travis Ralston
d1e7b9c44c
Merge branch 'develop' into travis/login-terms
2018-10-31 13:15:14 -06:00
Richard van der Hoff
1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades
...
Attempt to move room aliases on room upgrades
2018-10-31 17:52:18 +00:00
Richard van der Hoff
0f8591a5a8
Avoid else clause on exception for clarity
2018-10-31 15:43:57 +00:00
Richard van der Hoff
94c7fadc98
Attempt to move room aliases on room upgrades
2018-10-31 15:43:57 +00:00
Richard van der Hoff
9b827c40ca
Log some bits about event creation ( #4121 )
...
I found these helpful in debugging my room upgrade tests.
2018-10-31 15:42:23 +00:00
Amber Brown
916efc8249
Remove fetching keys via the deprecated v1 kex method ( #4120 )
2018-10-31 23:14:39 +11:00
Amber Brown
f79f454485
Remove deprecated v1 key exchange endpoint ( #4119 )
2018-10-31 22:29:02 +11:00
Amber Brown
67c1924899
version bump
2018-10-31 06:27:05 +11:00
Amber Brown
086e1a8f3e
Fix search 500ing ( #4122 )
2018-10-31 06:26:46 +11:00
Amber Brown
3bade14ec0
Fix search 500ing ( #4122 )
2018-10-31 04:33:41 +11:00
Amber Brown
2e223a8c22
Remove the unused /pull federation API ( #4118 )
2018-10-31 04:24:59 +11:00
Erik Johnston
0794504bce
Merge pull request #4006 from matrix-org/erikj/purge_state_groups
...
Delete unreferenced state groups during purge
2018-10-30 16:58:22 +00:00
Amber Brown
0dce9e1379
Write some tests for the email pusher ( #4095 )
2018-10-30 23:55:43 +11:00
David Baker
e0934acdbb
Cast to int here too
2018-10-30 11:12:23 +00:00
David Baker
12941f5f8b
Cast bacjup version to int when querying
2018-10-30 11:01:07 +00:00
David Baker
2f0f911c52
Convert version back to a string
2018-10-30 10:35:18 +00:00
David Baker
64fa557f80
Try & make it work on postgres
2018-10-30 09:51:04 +00:00
David Baker
563f9b61b1
Make e2e backup versions numeric in the DB
...
We were doing max(version) which does not do what we wanted
on a column of type TEXT.
2018-10-29 21:01:22 +00:00
Erik Johnston
88e5ffe6fe
Deduplicate device updates sent over replication
...
We currently send several kHz of device list updates over replication
occisonally, which often causes the replications streams to lag and then
get dropped.
A lot of those updates will actually be duplicates, since we don't send
e.g. device_ids across replication, so let's deduplicate it when we pull
them out of the database.
2018-10-29 17:34:34 +00:00
Erik Johnston
a163b748a5
Don't truncate command name in metrics
2018-10-29 17:34:21 +00:00
Erik Johnston
ad88460e0d
Move _find_unreferenced_groups
2018-10-29 14:24:19 +00:00
Erik Johnston
664b192a3b
Fix set operations thinko
2018-10-29 14:21:43 +00:00
Erik Johnston
f4f223aa44
Don't make temporary list
2018-10-29 14:01:49 +00:00
Erik Johnston
b2399f6281
Make SQL a bit cleaner
2018-10-29 14:01:11 +00:00
Amber Brown
4cd1c9f2ff
Delete the disused & unspecced identicon functionality ( #4106 )
2018-10-29 23:57:24 +11:00
Richard van der Hoff
7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar
...
Remember to copy the avatar on room upgrades
2018-10-29 12:49:21 +00:00
Richard van der Hoff
56ca578f77
Merge pull request #4099 from matrix-org/rav/upgrade_odd_pls
...
Better handling of odd PLs during room upgrades
2018-10-29 12:48:51 +00:00
Richard van der Hoff
bf33eed609
Merge pull request #4091 from matrix-org/rav/room_version_upgrades
...
Room version upgrade support
2018-10-29 12:47:20 +00:00
Amber Brown
c4b3698a80
Make the replication logger quieter ( #4108 )
2018-10-29 22:59:44 +11:00
Amber Brown
2b791865c4
version bump
2018-10-29 21:52:52 +11:00
Richard van der Hoff
db24d7f15e
Better handling of odd PLs during room upgrades
...
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
2018-10-27 00:54:26 +01:00
Richard van der Hoff
5caf79b312
Remember to copy the avatar on room upgrades
2018-10-26 23:56:40 +01:00
Richard van der Hoff
54bbe71867
optimise state copying
2018-10-26 22:51:34 +01:00
Richard van der Hoff
193cadc988
Address review comments
...
Improve comments, get old room state from the context we already have
2018-10-26 17:10:30 +01:00
Erik Johnston
03e634dad4
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups
2018-10-26 16:22:45 +01:00
Amber Brown
77d70a7646
Port register_new_matrix_user to Python 3 and add tests ( #4085 )
2018-10-26 22:05:22 +11:00
Richard van der Hoff
e6babc27d5
restrict PLs in old room
2018-10-25 19:18:25 +01:00
Richard van der Hoff
3a263bf3ae
copy state
2018-10-25 19:18:25 +01:00
Richard van der Hoff
1b9f253e20
preserve PLs
2018-10-25 19:10:24 +01:00
Richard van der Hoff
4cda300058
preserve room visibility
2018-10-25 19:10:24 +01:00
Richard van der Hoff
0f7d1c9906
Basic initial support for room upgrades
...
Currently just creates a new, empty, room, and sends a tombstone in the old
room.
2018-10-25 19:10:24 +01:00
Richard van der Hoff
e1948175ee
Allow power_level_content_override=None for _send_events_for_new_room
2018-10-25 19:10:24 +01:00
Richard van der Hoff
7f7b2cd3de
Make room_member_handler a member of RoomCreationHandler
...
... to save passing it into `_send_events_for_new_room`
2018-10-25 19:10:18 +01:00
Richard van der Hoff
871c4abfec
Factor _generate_room_id out of create_room
...
we're going to need this for room upgrades.
2018-10-25 18:23:09 +01:00
Erik Johnston
cb53ce9d64
Refactor state group lookup to reduce DB hits ( #4011 )
...
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
2018-10-25 17:49:55 +01:00
Erik Johnston
c85e063302
Merge pull request #4051 from matrix-org/erikj/alias_disallow_list
...
Add config option to control alias creation
2018-10-25 17:04:59 +01:00
Neil Johnson
95ad128851
Merge pull request #4081 from matrix-org/neilj/fix_mau_init
...
fix race condiftion in calling initialise_reserved_users
2018-10-25 16:33:40 +01:00
Neil Johnson
fcbd488e9a
add new line
2018-10-25 16:13:43 +01:00
Erik Johnston
b94a43d5b5
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list
2018-10-25 15:25:31 +01:00
Erik Johnston
e5481b22aa
Use allow/deny
2018-10-25 15:25:21 +01:00
Neil Johnson
f8fe98812b
improve comments
2018-10-25 14:58:59 +01:00
Neil Johnson
f7f487e14c
Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin
2018-10-25 14:40:06 +01:00
Travis Ralston
54def42c19
Merge branch 'develop' into travis/login-terms
2018-10-24 13:22:59 -06:00
Neil Johnson
ea69a84bbb
fix style inconsistencies
2018-10-24 17:18:08 +01:00
Neil Johnson
663d9db8e7
commit transaction before closing
2018-10-24 17:17:30 +01:00
Neil Johnson
07126e43a4
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init
2018-10-24 16:25:39 +01:00
Neil Johnson
9ec2186586
isort
2018-10-24 16:09:21 +01:00
Neil Johnson
9532caf6ef
remove trailing whiter space
2018-10-24 16:08:25 +01:00
Richard van der Hoff
94a49e0636
fix tuple
...
Co-Authored-By: neilisfragile <neil@matrix.org>
2018-10-24 14:39:23 +01:00
Richard van der Hoff
3ad359e5be
Merge remote-tracking branch 'origin/develop' into rav/fix_event_filter_validation
2018-10-24 11:23:49 +01:00
Richard van der Hoff
7328039117
Merge pull request #4082 from matrix-org/rav/fix_pep8
...
Fix a number of flake8 errors
2018-10-24 11:23:35 +01:00
Erik Johnston
3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase
...
Add v2 state resolution algorithm
2018-10-24 11:12:12 +01:00
Richard van der Hoff
7e07d25ed6
Allow backslashes in event field filters
...
Fixes a bug introduced in https://github.com/matrix-org/synapse/pull/1783 which
meant that single backslashes were not allowed in event field filters.
The intention here is to allow single-backslashes, but disallow
double-backslashes.
2018-10-24 11:11:24 +01:00
Richard van der Hoff
ef771cc4c2
Fix a number of flake8 errors
...
Broadly three things here:
* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
them
* use `r""` for strings which include backslashes
Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
2018-10-24 10:39:03 +01:00
Erik Johnston
47a9ba435d
Use match rather than search
2018-10-24 09:54:57 +01:00
Richard van der Hoff
e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts
...
Clean up the way logcontexts and threads work in the pushers
2018-10-24 09:53:57 +01:00
Erik Johnston
dacbeb2e03
Comment
2018-10-24 09:47:49 +01:00
Erik Johnston
810715f79a
Rename resolve_events_with_factory
2018-10-24 09:44:22 +01:00
Erik Johnston
cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation
...
Clean up room alias creation
2018-10-24 09:40:27 +01:00
Richard van der Hoff
c573794b22
Fix start_pushers vs _start_pushers confusion
2018-10-24 09:24:55 +01:00
Richard van der Hoff
e564306e31
sanity-check the is_processing flag
...
... and rename it, for even more sanity
2018-10-24 09:23:33 +01:00
Neil Johnson
a67d8ace9b
remove errant exception and style
2018-10-23 17:44:39 +01:00
Travis Ralston
43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests
...
Handle HttpResponseException more safely for federated groups
2018-10-23 10:41:04 -06:00
Travis Ralston
3e704822be
Comments help
2018-10-23 10:25:31 -06:00
Neil Johnson
329d18b39c
remove white space
2018-10-23 15:27:20 +01:00
Neil Johnson
6105c6101f
fix race condiftion in calling initialise_reserved_users
2018-10-23 15:24:58 +01:00
Richard van der Hoff
b3f6dddad2
Give some more things logcontexts ( #4077 )
2018-10-23 14:29:17 +01:00
Richard van der Hoff
5c445114d3
Correctly account for cpu usage by background threads ( #4074 )
...
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.
While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.
Fixes #4064
2018-10-23 13:12:32 +01:00
Richard van der Hoff
026cd91ac8
Run PusherPool.start as a background process
...
We don't do anything with the result, so this is needed to give this code a
logcontext.
2018-10-22 16:12:11 +01:00
Richard van der Hoff
f749607c91
Make on_started synchronous too
...
This brings it into line with on_new_notifications and on_new_receipts. It
requires a little bit of hoop-jumping in EmailPusher to load the throttle
params before the first loop.
2018-10-22 16:12:11 +01:00
Richard van der Hoff
e7a16c6210
Remove redundant run_as_background_process() from pusherpool
...
`on_new_notifications` and `on_new_receipts` in `HttpPusher` and `EmailPusher`
now always return synchronously, so we can remove the `defer.gatherResults` on
their results, and the `run_as_background_process` wrappers can be removed too
because the PusherPool methods will now complete quickly enough.
2018-10-22 16:12:11 +01:00
Richard van der Hoff
c7273c11bc
Give pushers their own background logcontext
...
Each pusher has its own loop which runs for as long as it has work to do. This
should run in its own background thread with its own logcontext, as other
similar loops elsewhere in the system do - which means that CPU usage is
consistently attributed to that loop, rather than to whatever request happened
to start the loop.
2018-10-22 16:12:11 +01:00
Richard van der Hoff
5110f4e425
move get_all_pushers call down
...
simplifies the interface to _start_pushers
2018-10-22 16:12:11 +01:00
Richard van der Hoff
04277d0ed8
Factor PusherPool._start_pusher out of _start_pushers
...
... and use it from start_pusher_by_id. This mostly simplifies
start_pusher_by_id.
2018-10-22 16:12:11 +01:00
Richard van der Hoff
3e8b02c939
Rename _refresh_pusher
...
This is public (or at least, called from outside the class), so ought to have a
better name.
2018-10-22 16:12:11 +01:00
Richard van der Hoff
7aea00069c
Merge pull request #4076 from matrix-org/rav/fix_init_logcontexts
...
Run MAU queries as background processes
2018-10-22 14:46:59 +01:00
Richard van der Hoff
911db96658
Merge pull request #4073 from matrix-org/rav/require_psutil
...
Make psutil an explicit dependency
2018-10-22 12:33:21 +01:00
Richard van der Hoff
a6f421e812
Run MAU queries as background processes
...
Fixes #3820
2018-10-20 02:14:35 +01:00
Amber Brown
e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) ( #4068 )
2018-10-20 11:16:55 +11:00
Richard van der Hoff
e5b52d0f94
Make psutil an explicit dependency
...
As of #4027 , we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992 .
Fixes #4062 .
2018-10-19 21:51:15 +01:00
Richard van der Hoff
f62c597d14
Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065
...
Calculate absolute path for email templates
2018-10-19 16:33:54 +01:00
Erik Johnston
67f7b9cb50
pep8
2018-10-19 16:06:59 +01:00
Erik Johnston
056f099126
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups
2018-10-19 15:48:59 +01:00
Erik Johnston
47a9da28ca
Batch process handling state groups
2018-10-19 15:48:15 +01:00
Richard van der Hoff
cc325c7069
Calculate absolute path for email templates
2018-10-19 14:01:59 +01:00
Amber Brown
e404ba9aac
Fix manhole on py3 (pt 2) ( #4067 )
2018-10-19 22:26:00 +11:00
Amber Brown
b69216f768
Make the metrics less racy ( #4061 )
2018-10-19 21:45:45 +11:00
Erik Johnston
9fafdfa97d
Anchor returned regex to start and end of string
2018-10-19 10:22:45 +01:00
Erik Johnston
084046456e
Add config option to control alias creation
2018-10-19 10:22:45 +01:00
Erik Johnston
74e7617083
Clean up room alias creation
2018-10-19 10:11:56 +01:00
Amber Brown
a36b0ec195
make a bytestring
2018-10-19 09:24:00 +11:00
Travis Ralston
49a044aa5f
Merge branch 'develop' into travis/login-terms
2018-10-18 09:57:58 -06:00
Richard van der Hoff
c632bc8654
Merge branch 'master' into develop
2018-10-18 15:07:03 +01:00
Richard van der Hoff
926da4dda8
0.33.7
2018-10-18 14:57:32 +01:00
Richard van der Hoff
c7d0f34a3c
v0.33.7rc2
2018-10-17 17:40:19 +01:00
Richard van der Hoff
0fd2321629
Fix incorrect truncation in get_missing_events
...
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-17 17:35:26 +01:00
Richard van der Hoff
c8f2c19991
Put the warning blob at the top of the file
2018-10-17 16:56:22 +01:00
Richard van der Hoff
1519572961
Ship the email templates as package_data
...
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.
In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
2018-10-17 16:46:02 +01:00
Erik Johnston
f6a0a02a62
Fix bug where we raised StopIteration in a generator
...
This made python 3.7 unhappy
2018-10-17 16:10:52 +01:00
Will Hunt
d6a7797dd1
Fix roomlist since tokens on Python 3 ( #4046 )
...
Thanks @Half-Shot !!!
2018-10-17 23:04:55 +11:00
Neil Johnson
c6584f4b5f
clean up config error logic and imports
2018-10-17 11:36:41 +01:00
Richard van der Hoff
fc0f13dd03
Fix incorrect truncation in get_missing_events
...
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
Erik Johnston
4a28d3d36f
Update event_auth table for rejected events
2018-10-16 16:28:42 +01:00
Erik Johnston
15133477ee
Fix up use of resolve_events_with_factory
2018-10-16 16:28:42 +01:00
Erik Johnston
6bd856caa2
User event.sender rather than alias event.user_id
2018-10-16 16:16:13 +01:00
Erik Johnston
e238013c44
Add v2 state res algorithm.
...
We hook this up to the vdh test room version.
2018-10-16 16:16:13 +01:00
Richard van der Hoff
b8a5b0097c
Various cleanups in the federation client code ( #4031 )
...
- Improve logging: log things in the right order, include destination and txids
in all log lines, don't log successful responses twice
- Fix the docstring on TransportLayerClient.send_transaction
- Don't use treq.request, which is overcomplicated for our purposes: just use a
twisted.web.client.Agent.
- simplify the logic for setting up the bodyProducer
- fix bytes/str confusions
2018-10-16 10:44:49 +01:00
Travis Ralston
a8ed93a4b5
pep8
2018-10-15 16:10:29 -06:00
Travis Ralston
442734ff9e
Ensure the terms params are actually provided
2018-10-15 14:56:13 -06:00
Travis Ralston
762a0982aa
Python is hard
2018-10-15 14:46:09 -06:00
Travis Ralston
f293d124b6
Merge branch 'develop' into travis/login-terms
2018-10-15 14:44:32 -06:00
Amber Brown
f726f2dc6c
version bump
2018-10-15 22:21:45 +11:00
David Baker
03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups
...
E2E backups
2018-10-15 10:19:43 +01:00
Ivan Shapovalov
06bc8d2fe5
synapse/app: frontend_proxy.py: actually make workers work on py3
2018-10-14 20:08:39 +03:00
Ivan Shapovalov
fb216a22db
synapse/visibility.py: fix SyntaxError on py3.7
2018-10-14 20:08:17 +03:00
Neil Johnson
1ccafb0c5e
no need to join room if creator
2018-10-13 21:14:21 +01:00
Travis Ralston
dd99db846d
Update login terms structure for the proposed language support
2018-10-12 18:03:27 -06:00
Travis Ralston
5119818e9d
Rely on the lack of ?u to represent public access
...
also general cleanup
2018-10-12 18:03:17 -06:00
Travis Ralston
7ede650956
Merge branch 'develop' into travis/login-terms
2018-10-12 16:24:07 -06:00
Travis Ralston
164f8e4843
isort
2018-10-12 15:11:59 -06:00
Travis Ralston
7bb651de6a
More sane handling of group errors and pep8
2018-10-12 14:53:30 -06:00
Travis Ralston
e3586f7c06
Merge branch 'develop' into travis/fix-federated-group-requests
2018-10-12 14:49:58 -06:00
Neil Johnson
a2bfb778c8
improve auto room join logic, comments and tests
2018-10-12 18:17:36 +01:00
David Baker
a45f2c3a00
missed one
2018-10-12 14:33:55 +01:00
Amber Brown
381d2cfdf0
Make workers work on Py3 ( #4027 )
2018-10-13 00:14:08 +11:00
Erik Johnston
67a1e315cc
Fix up comments
2018-10-12 13:49:48 +01:00
David Baker
8c0ff0287a
Linting soothes the savage PEP8 monster
2018-10-12 13:47:43 +01:00
David Baker
306361b31b
Misc PR feedback bits
2018-10-12 11:48:56 +01:00
David Baker
bddfad253a
Don't mangle exceptions
2018-10-12 11:48:02 +01:00
David Baker
86ef9760a7
Split /room_keys/version into 2 servlets
2018-10-12 11:35:08 +01:00
David Baker
83e72bb2f0
PR feedback pt. 1
2018-10-12 11:26:18 +01:00
Richard van der Hoff
8ddd0f273c
Comments on get_all_new_events_stream
...
just some docstrings to clarify the behaviour here
2018-10-12 09:55:41 +01:00
Erik Johnston
7e561b5c1a
Add description to counter metric
2018-10-10 11:41:15 +01:00
Erik Johnston
3cbe8331e6
Track number of non-empty sync responses instead
2018-10-10 11:23:17 +01:00
Erik Johnston
395276b405
Append _total to metric and fix up spelling
2018-10-10 09:24:39 +01:00
David Baker
b8d9e108be
Fix mergefail
2018-10-09 18:04:21 +01:00
Erik Johnston
bdc27d6716
Add metric to count lazy member sync requests
2018-10-09 14:17:52 +01:00
David Baker
d3464ce708
isort
2018-10-09 10:33:59 +01:00
Erik Johnston
9eb1a79100
Merge pull request #4008 from matrix-org/erikj/log_looping_exceptions
...
Log looping call exceptions
2018-10-09 10:12:10 +01:00
David Baker
dc045ef202
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
2018-10-09 10:05:02 +01:00
David Baker
f4a4dbcad1
Apparently this blank line is Very Important
2018-10-09 09:47:04 +01:00
David Baker
0c905ee015
be python3 compatible
2018-10-09 09:39:13 +01:00
Erik Johnston
6982320572
Remove unnecessary extra function call layer
2018-10-08 14:06:19 +01:00
Richard van der Hoff
495975e231
Optimisation for filter_events_for_server
...
We're better off hashing just the event_id than the whole ((type, state_key),
event_id) tuple - so use a dict instead of a set.
Also, iteritems > items.
2018-10-08 13:46:52 +01:00
Erik Johnston
8a1817f0d2
Use errback pattern and catch async failures
2018-10-08 13:29:47 +01:00
David Baker
497444f1fd
Don't reuse backup versions
...
Since we don't actually delete the keys, just mark the versions
as deleted in the db rather than actually deleting them, then we
won't reuse versions.
Fixes https://github.com/vector-im/riot-web/issues/7448
2018-10-05 15:08:36 +01:00
Erik Johnston
f7199e8734
Log looping call exceptions
...
If a looping call function errors, then it kills the loop entirely.
Currently it throws away the exception logs, so we should make it
actually log them.
Fixes #3929
2018-10-05 11:24:12 +01:00
Neil Johnson
ed82043efb
Merge branch 'develop' into matthew/autocreate_autojoin
2018-10-04 17:26:59 +01:00
Neil Johnson
2dadc092b8
move logic into register, fix room alias localpart bug, tests
2018-10-04 17:00:27 +01:00
Richard van der Hoff
c6dbd216e6
Merge pull request #3995 from matrix-org/rav/no_deextrem_outliers
...
Fix bug in forward_extremity update logic
2018-10-04 16:27:05 +01:00
Erik Johnston
4917ff5523
Add state_group index to event_to_state_groups
...
This is needed to efficiently check for unreferenced state groups during
purge.
2018-10-04 16:03:08 +01:00
Erik Johnston
17d585753f
Delete unreferened state groups during purge
2018-10-04 16:03:06 +01:00
Amber Brown
d86794325f
Merge branch 'master' into develop
2018-10-04 22:41:52 +10:00
Amber Brown
dd59dfc51f
full version
2018-10-04 22:37:55 +10:00
Travis Ralston
537d0b7b36
Use a flag rather than a new route for the public policy
...
This also means that the template now has optional parameters, which will need to be documented somehow.
2018-10-03 17:50:11 -06:00
Travis Ralston
f9d34a763c
Auto-consent to the privacy policy if the user registered with terms
2018-10-03 17:39:45 -06:00
Travis Ralston
dfcad5fad5
Make the terms flow requried
2018-10-03 17:39:00 -06:00
Travis Ralston
3099d96dba
Flesh out the fallback auth for terms
2018-10-03 17:39:00 -06:00
Travis Ralston
149c4f1765
Supply params for terms auth stage
...
As per https://github.com/matrix-org/matrix-doc/pull/1692
2018-10-03 15:57:42 -06:00
Travis Ralston
fd99787162
Incorporate Dave's work for GDPR login flows
...
As per https://github.com/vector-im/riot-web/issues/7168#issuecomment-419996117
2018-10-03 15:57:42 -06:00
Richard van der Hoff
a59d899668
Pin to prometheus_client<0.4 to avoid renaming all of our metrics
2018-10-03 17:20:15 +01:00
Erik Johnston
8935ec5a93
Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetion
...
Fix handling of rejected threepid invites
2018-10-03 14:47:41 +01:00
Erik Johnston
81e2813948
Merge pull request #3996 from matrix-org/erikj/fix_bg_iteration
...
Fix exception in background metrics collection
2018-10-03 14:14:38 +01:00
Erik Johnston
52e6e815be
Sanitise error messages when user doesn't have permission to invite
2018-10-03 14:13:07 +01:00
Erik Johnston
69e857853f
Fix handling of rejected threepid invites
2018-10-03 11:57:30 +01:00
Erik Johnston
495a9d06bb
Fix exception handling in fetching remote profiles
2018-10-03 11:34:30 +01:00
Erik Johnston
7c570bff74
Fix exception in background metrics collection
...
We attempted to iterate through a list on a separate thread without
doing the necessary copying.
2018-10-03 11:28:01 +01:00
Richard van der Hoff
9693625e55
actually exclude outliers
2018-10-03 10:19:41 +01:00
Richard van der Hoff
3e39783d5d
remove debugging
2018-10-02 23:44:14 +01:00
Richard van der Hoff
ae61ade891
Fix bug in forward_extremity update logic
...
An event does not stop being a forward_extremity just because an outlier or
rejected event refers to it.
2018-10-02 23:36:08 +01:00
Amber Brown
da8f82008d
version
2018-10-03 02:15:48 +10:00
Erik Johnston
7258d081a5
Fix bug when invalidating destination retry timings
2018-10-02 15:47:57 +01:00
Richard van der Hoff
2b8d28b095
Merge pull request #3960 from matrix-org/rav/fix_missing_create_event_error
...
Fix error handling for missing auth_event
2018-10-02 13:57:54 +01:00
Amber Brown
7232917f12
Disable frozen dicts by default ( #3987 )
2018-10-02 22:53:47 +10:00
Erik Johnston
334e075dd8
Fix error when logging incomplete requests
...
If a connection is lost before a request is read from Request, Twisted
sets `method` (and `uri`) attributes to dummy values. These dummy values
have incorrect types (i.e. they're not bytes), and so things like
`__repr__` would raise an exception.
To fix this we had a helper method to return the method with a
consistent type.
2018-10-02 12:06:22 +01:00
Richard van der Hoff
8c41b0ca66
Merge pull request #3989 from matrix-org/rav/better_stacktraces
...
Avoid reraise, to improve stacktraces
2018-10-02 10:47:38 +01:00
Erik Johnston
bc29946809
Merge pull request #3986 from matrix-org/erikj/fix_sync_with_redacted_state
...
Fix lazy loaded sync with rejected state events
2018-10-02 11:38:35 +02:00
Richard van der Hoff
8174c6725b
Avoid reraise, to improve stacktraces
2018-10-01 18:50:34 +01:00
Richard van der Hoff
5908a8f6f5
Merge branch 'develop' into rav/fix_missing_create_event_error
2018-10-01 17:16:20 +01:00
Richard van der Hoff
b5b93f45d5
Merge pull request #3968 from matrix-org/rav/fix_federation_errors
...
Fix exceptions when handling incoming transactions
2018-10-01 15:54:24 +01:00
Amber Brown
6e05fd032c
Fix userconsent on Python 3 ( #3938 )
2018-10-02 00:11:58 +10:00
Erik Johnston
82f922b4af
Fix lazy loaded sync with rejected state events
...
In particular, we assume that the name and canonical alias events in
the state have not been rejected. In practice this may not be the case
(though we should probably think about fixing that) so lets ensure that
we gracefully handle that case, rather than 404'ing the sync request
like we do now.
2018-10-01 14:19:40 +01:00
Erik Johnston
8f5c23d0cd
Merge pull request #3933 from matrix-org/erikj/destination_retry_cache
...
Add a five minute cache to get_destination_retry_timings
2018-10-01 14:20:30 +02:00
Erik Johnston
4f3e3ac192
Correctly match 'dict.pop' api
2018-10-01 12:25:27 +01:00
Erik Johnston
8ea887856c
Don't update eviction metrics on explicit removal
2018-10-01 12:00:58 +01:00
Matthew Hodgson
23b6a0537f
emit room aliases event
2018-09-29 02:19:37 +01:00
Matthew Hodgson
5b68f29f48
fix thinkos
2018-09-29 02:14:40 +01:00
Matthew Hodgson
8f646f2d04
fix UTs
2018-09-28 15:37:28 +01:00
Richard van der Hoff
3deaad2fb4
Merge pull request #3964 from matrix-org/rav/remove_localhost_checks
...
remove spurious federation checks on localhost
2018-09-28 13:35:47 +01:00
Richard van der Hoff
965154d60a
Fix complete fail to do the right thing
2018-09-28 12:45:54 +01:00
Richard van der Hoff
19475cf337
Remove redundant call to start_get_pdu_cache
...
I think this got forgotten in #3932 . We were getting away with it because it
was the last call in this function.
2018-09-28 12:01:23 +01:00
Richard van der Hoff
9c8cec5dab
Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache
2018-09-28 10:51:09 +01:00
Matthew Hodgson
07340cdaca
untested stab at autocreating autojoin rooms
2018-09-28 01:42:53 +01:00
Richard van der Hoff
f094f715cf
Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors
2018-09-27 15:18:21 +01:00
Richard van der Hoff
36c62a67c4
Merge pull request #3794 from matrix-org/erikj/faster_typing
...
Improve performance of getting typing updates for replication
2018-09-27 15:14:26 +01:00
Richard van der Hoff
e1e3e77bfd
Merge pull request #3967 from matrix-org/rav/federation_handler_cleanups
...
Clarifications in FederationHandler
2018-09-27 15:06:59 +01:00
Amber Brown
6de3884e5e
Merge pull request #3965 from matrix-org/rav/notify_app_services_bg_process
...
Run notify_app_services as a bg process
2018-09-27 23:40:30 +10:00
Amber Brown
a512e637ac
Merge pull request #3970 from schnuffle/develop-py3
...
Replaced all occurences of e.message with str(e)
2018-09-27 23:39:45 +10:00