Michael Kaye
f85b9842f0
Don't encode object as UTF-8 string if not needed.
...
I believe that string formatting ~10-15 sized events will
take a proportion of CPU time.
2019-10-24 18:08:45 +01:00
Michael Kaye
39266a9c9f
Make user/room stats log line less verbose.
2019-10-24 17:55:53 +01:00
Erik Johnston
3aa2a90556
Move schema delta files to the correct data store.
...
They were put in the global schema delta directory due to a bad merge.
2019-10-24 16:45:03 +01:00
Andrew Morgan
8831b04a53
1.5.0rc1
2019-10-24 12:47:58 +01:00
Neil Johnson
2794b79052
Option to suppress resource exceeded alerting ( #6173 )
...
The expected use case is to suppress MAU limiting on small instances
2019-10-24 11:48:46 +01:00
Andrew Morgan
92e88a71d3
Cleanup extra quotes from IDEs ( #6236 )
2019-10-23 16:49:05 +01:00
Erik Johnston
3ca4c7c516
Use new EventPersistenceStore
2019-10-23 16:15:03 +01:00
Erik Johnston
73cf63784b
Add DataStores and Storage classes.
2019-10-23 16:15:03 +01:00
Hubert Chathi
dc2cd6f79d
move get_e2e_cross_signing_key to EndToEndKeyWorkerStore so it works with workers
2019-10-23 09:13:47 -04:00
Erik Johnston
22a9847670
Move persist_events out from main data store.
...
This is in preparation for splitting out of state_groups_state from the
main store into it own one, as persisting events depends on calculating
state.
2019-10-23 13:29:44 +01:00
Andrew Morgan
409c62b27b
Add config linting script that checks for bool casing ( #6203 )
...
Add a linting script that enforces all boolean values in the default config be lowercase.
This has annoyed me for a while so I decided to fix it.
2019-10-23 13:22:54 +01:00
Hubert Chathi
480eac30eb
black
2019-10-22 22:37:16 -04:00
Hubert Chathi
404e8c8532
vendor-prefix the EDU name until MSC1756 is merged into the spec
2019-10-22 22:33:23 -04:00
Hubert Chathi
1fabf82d50
update to work with newer code, and fix formatting
2019-10-22 21:44:58 -04:00
Hubert Chathi
41ad35b523
add missing param
2019-10-22 19:06:29 -04:00
Hubert Chathi
cfdb84422d
make black happy
2019-10-22 19:06:06 -04:00
Hubert Chathi
a1aaf3eea6
don't crash if the user doesn't have cross-signing keys
2019-10-22 19:04:37 -04:00
Hubert Chathi
8d3542a64e
implement federation parts of cross-signing
2019-10-22 19:04:35 -04:00
Erik Johnston
c17efdc01c
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores
2019-10-22 18:44:29 +01:00
Erik Johnston
23d62eded2
Clean up prepare_database.py a bit and add comments
2019-10-22 18:43:31 +01:00
Erik Johnston
6cc497f99b
Delete background_update table creation in main data_store
2019-10-22 18:02:50 +01:00
Erik Johnston
1bbc5444a8
Move README into synapse/storage/__init__.py
2019-10-22 17:59:31 +01:00
Hubert Chathi
2761731634
Merge pull request #5726 from matrix-org/uhoreg/e2e_cross-signing2-part2
...
Cross-signing [3/4] -- uploading signatures edition
2019-10-22 17:05:06 +01:00
Erik Johnston
acf47c7698
Add a basic README to synapse.storage
2019-10-22 11:55:46 +01:00
Erik Johnston
bb6264be0b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores
2019-10-22 10:41:18 +01:00
Erik Johnston
ffd24545bb
Fix schema management to work with multiple data stores.
2019-10-21 16:08:40 +01:00
Erik Johnston
c66a06ac6b
Move storage classes into a main "data store".
...
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
2019-10-21 16:05:06 +01:00
Richard van der Hoff
93eaeec75a
Remove Auth.check method ( #6217 )
...
This method was somewhat redundant, and confusing.
2019-10-18 19:43:36 +02:00
Hubert Chathi
36adfaedab
Merge branch 'develop' into cross-signing_sig_upload
2019-10-18 18:34:42 +01:00
Hubert Chathi
125eb45e19
fix doc strings
2019-10-18 16:56:16 +01:00
Hubert Chathi
0a9d22808c
Merge branch 'uhoreg/e2e_cross-signing_merged' into develop
2019-10-18 15:57:40 +01:00
Richard van der Hoff
2b22faded7
Merge remote-tracking branch 'origin/develop' into rav/event_auth/1
2019-10-18 12:35:13 +01:00
Richard van der Hoff
560c122267
Fix logging config for the docker image ( #6197 )
...
Turns out that loggers that are instantiated before the config is loaded get
turned off.
Also bring the logging config that is generated by --generate-config into line.
Fixes #6194 .
2019-10-18 13:34:33 +02:00
Hubert Chathi
cfc28325a6
Merge branch 'develop' into uhoreg/e2e_cross-signing_merged
2019-10-18 11:39:40 +01:00
Brendan Abolivier
1ee97cbd01
Merge branch 'master' into develop
2019-10-18 11:00:43 +01:00
Hubert Chathi
f0f6a2b360
use the right function for when we're already in runInteraction
2019-10-18 10:56:54 +01:00
Erik Johnston
d98029ea89
Merge pull request #6196 from matrix-org/erikj/await
...
Move rest/admin to use async/await.
2019-10-18 11:53:02 +02:00
Brendan Abolivier
41b9faed16
1.4.1
2019-10-18 10:15:12 +01:00
Richard van der Hoff
80003dfcd5
Merge remote-tracking branch 'origin/develop' into rav/event_auth/1
2019-10-18 10:11:40 +01:00
Erik Johnston
5859a5c569
Fix presence timeouts when synchrotron restarts. ( #6212 )
...
* Fix presence timeouts when synchrotron restarts.
Handling timeouts would fail if there was an external process that had
timed out, e.g. a synchrotron restarting. This was due to a couple of
variable name typoes.
Fixes #3715 .
2019-10-18 06:42:26 +01:00
Richard van der Hoff
1ba359a11f
rip out some unreachable code
...
The only possible rejection reason is AUTH_ERROR, so all of this is unreachable.
2019-10-17 21:45:59 +01:00
Richard van der Hoff
70d9759cdf
Synapse 1.4.1rc1 (2019-10-17)
...
=============================
Bugfixes
--------
- Fix bug where redacted events were sometimes incorrectly censored in the database, breaking APIs that attempted to fetch such events. ([\#6185](https://github.com/matrix-org/synapse/issues/6185 ), [5b0e99485b0e9948eahttps://github.com/matrix-org/synapse/issues/6185 ), [5b0e99485b0e9948ea
2019-10-17 17:03:02 +01:00
Richard van der Hoff
71cd3fed66
1.4.1rc1
2019-10-17 16:40:56 +01:00
Erik Johnston
5b0e9948ea
Do the update as a background index
2019-10-17 16:31:42 +01:00
Erik Johnston
a2bb50c2eb
Merge pull request #6185 from matrix-org/erikj/fix_censored_evnets
...
Fix inserting bytes as text in `censor_redactions`
2019-10-17 16:29:07 +01:00
Hubert Chathi
06fc66c81e
Merge pull request #6193 from matrix-org/uhoreg/interpret_device_key_in_storage
...
make storage layer in charge of interpreting the device key data
2019-10-11 15:19:06 -04:00
Hubert Chathi
132b251e29
expand on comment
2019-10-11 14:24:52 -04:00
Erik Johnston
2e97a4c197
Port synapse/rest/client/_base.py to async/await
2019-10-11 15:29:26 +01:00
Erik Johnston
3c2d6c708c
Add maybe_awaitable and fix __init__ bugs
2019-10-11 15:26:09 +01:00
Hubert Chathi
691dd67fcd
Merge pull request #6189 from matrix-org/uhoreg/e2e_backup_optional_version
...
make version optional in body of e2e backup version update
2019-10-11 10:11:59 -04:00
Erik Johnston
6f5c6c8f60
Merge pull request #6195 from matrix-org/erikj/opentracing_preview_url
...
Trace non-JSON APIs, /media, /key etc
2019-10-11 13:23:52 +01:00
Valérian Rousset
be9b55e0d2
cas: support setting display name ( #6114 )
...
Now, the CAS server can return an attribute stating what's the desired displayname, instead of using the username directly.
2019-10-11 12:33:12 +01:00
Erik Johnston
dfbb62c28d
Port synaps/rest/client/users.py to async/await
2019-10-11 12:21:40 +01:00
Erik Johnston
f95325e22a
Port synaps/rest/client/server_notice_servlet.py to async/await
2019-10-11 12:21:40 +01:00
Erik Johnston
281f887090
Port synaps/rest/client/media.py to async/await
2019-10-11 12:21:40 +01:00
Erik Johnston
fca3a541e7
Port rest/admin/__init__.py to async/await
2019-10-11 12:17:52 +01:00
Erik Johnston
f3ceaf4323
Trace non-JSON APIs, /media, /key etc
2019-10-11 11:58:52 +01:00
Neil Johnson
a0d0ba7862
Fix MAU reaping where reserved users are specified. ( #6168 )
2019-10-11 09:38:26 +01:00
Hubert Chathi
7a0dce9259
make sure we actually return something
2019-10-10 20:31:30 -04:00
Hubert Chathi
4908fb3b30
make storage layer in charge of interpreting the device key data
2019-10-10 19:15:30 -04:00
Erik Johnston
83d86106a8
Merge pull request #6156 from matrix-org/erikj/postgres_any
...
Use Postgres ANY for selecting many values.
2019-10-10 16:41:36 +01:00
Erik Johnston
b54b1e759a
Fix SQLite take 2
2019-10-10 16:19:40 +01:00
Erik Johnston
afb6d9d53b
Fix SQLite
2019-10-10 15:55:41 +01:00
Erik Johnston
3bc687508f
Remove add_in_list_sql_clause
2019-10-10 15:35:46 +01:00
Erik Johnston
ca3e01e50d
Fix store_url_cache using bytes
2019-10-10 14:52:29 +01:00
Erik Johnston
9d06fb9cb1
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/postgres_any
2019-10-10 14:38:21 +01:00
Erik Johnston
8bc529c04d
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/disable_sql_bytes
2019-10-10 14:30:51 +01:00
Erik Johnston
933034e2fe
Merge pull request #6127 from matrix-org/erikj/patch_inner
...
Add more log context checks when patching inlineCallbacks
2019-10-10 13:47:50 +01:00
Erik Johnston
fe1c1e6c28
Fixup comments
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-10-10 13:17:19 +01:00
Erik Johnston
b161786c14
Replace IN usage with helper funcs
2019-10-10 13:15:49 +01:00
Erik Johnston
b4fbf71187
Add helper funcs to use postgres ANY
...
This means that we can write queries with `col = ANY(?)`, which helps
postgres.
2019-10-10 13:15:24 +01:00
Erik Johnston
91f43dca39
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/disable_sql_bytes
2019-10-10 13:10:57 +01:00
werner291
b5b03b7079
Add domain validation when creating room with list of invitees ( #6121 )
2019-10-10 13:05:48 +01:00
Michael Telatynski
9a84d74417
before fulfilling a group invite,check if user is already joined/invited ( #3436 )
...
Fixes vector-im/riot-web#5645
2019-10-10 13:03:44 +01:00
krombel
2efd050c9d
send 404 as http-status when filter-id is unknown to the server ( #2380 )
...
This fixed the weirdness of 400 vs 404 as http status code in the case
the filter id is not known by the server.
As e.g. matrix-js-sdk expects 404 to catch this situation this leads
to unwanted behaviour.
2019-10-10 12:59:55 +01:00
Erik Johnston
59e0ed8306
Fix py3.5
2019-10-10 12:47:07 +01:00
Erik Johnston
c349e3ebaf
Fix py3.5
2019-10-10 12:29:38 +01:00
Erik Johnston
f735aeec65
sort
2019-10-10 12:20:29 +01:00
Erik Johnston
941edad583
Appease mypy
2019-10-10 12:15:17 +01:00
Erik Johnston
791a8c559b
Add coments
2019-10-10 11:53:57 +01:00
Erik Johnston
9970f955ce
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/patch_inner
2019-10-10 11:51:50 +01:00
Richard van der Hoff
a139420a3c
Fix races in room stats (and other) updates. ( #6187 )
...
Hopefully this will fix the occasional failures we were seeing in the room directory.
The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8.
We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
2019-10-10 11:29:01 +01:00
Richard van der Hoff
562b4e51dd
Rewrite the user_filter migration again ( #6184 )
...
you can't plausibly ALTER TABLE in sqlite, so we create the new table with the
right schema to start with.
2019-10-10 11:28:23 +01:00
Erik Johnston
128d5948c4
Fix packaging
2019-10-10 11:16:26 +01:00
Erik Johnston
ec0596f2ab
Log correct context
2019-10-10 11:11:38 +01:00
Erik Johnston
3e4272961a
Test for sentinel commit
2019-10-10 10:58:32 +01:00
Erik Johnston
1d6dd1c294
Move patch_inline_callbacks into synapse/
2019-10-10 10:53:06 +01:00
Andrew Morgan
da815c1f69
Move tag/push rules room upgrade checking ealier ( #6155 )
...
It turns out that _local_membership_update doesn't run when you join a new, remote room. It only runs if you're joining a room that your server already knows about. This would explain #4703 and #5295 and why the transfer would work in testing and some rooms, but not others. This would especially hit single-user homeservers.
The check has been moved to right after the room has been joined, and works much more reliably. (Though it may still be a bit awkward of a place).
2019-10-10 10:06:45 +01:00
Amber Brown
f743108a94
Refactor HomeserverConfig so it can be typechecked ( #6137 )
2019-10-10 09:39:35 +01:00
Hubert Chathi
4535a07f4a
make version optional in body of e2e backup version update
...
to agree with latest version of the MSC
2019-10-09 17:54:03 -04:00
Erik Johnston
5c1f886c75
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/patch_inner
2019-10-09 16:52:21 +01:00
Erik Johnston
7f18b3d526
Do the update as a background index
2019-10-09 16:03:24 +01:00
Erik Johnston
def5413480
Merge pull request #6185 from matrix-org/erikj/fix_censored_evnets
...
Fix inserting bytes as text in `censor_redactions`
2019-10-09 15:39:13 +01:00
Brendan Abolivier
59d6290ed9
Merge pull request #6178 from matrix-org/babolivier/factor_out_bg_updates
...
Factor out backgroung updates
2019-10-09 12:29:01 +01:00
Erik Johnston
1d3858371e
Disable bytes usage with postgres
...
More often than not passing bytes to `txn.execute` is a bug (where we
meant to pass a string) that just happens to work if `BYTEA_OUTPUT` is
set to `ESCAPE`. However, this is a bit of a footgun so we want to
instead error when this happens, and force using `bytearray` if we
actually want to use bytes.
2019-10-08 16:28:57 +01:00
Erik Johnston
e7631d84e6
Fix existing hex encoded json values in DB
2019-10-08 16:18:43 +01:00
Erik Johnston
ced4784592
Fix inserting bytes as text
2019-10-08 16:18:43 +01:00
Brendan Abolivier
c69324ffb5
Fix RegistrationStore
2019-10-08 14:48:33 +01:00
Brendan Abolivier
b1c0a4ceb3
Cleanup client_ips
2019-10-08 14:38:14 +01:00
Brendan Abolivier
8f1b385acc
Don't end up with 4 classes in registration
2019-10-08 14:36:33 +01:00
