Matrix-Mirrors/forked-synapse
1
0
Fork 0
mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00
Code Issues Packages Projects Releases Wiki Activity
10,856 Commits 5 Branches 723 Tags 124 MiB
e987079037
Commit Graph

840 Commits

Author SHA1 Message Date
Erik Johnston
770b2252ca s/_create_new_client_event/create_new_client_event/ 2018-02-06 16:40:30 +00:00
Erik Johnston
3d33eef6fc
Store state groups separately from events (#2784) 
* Split state group persist into seperate storage func

* Add per database engine code for state group id gen

* Move store_state_group to StateReadStore

This allows other workers to use it, and so resolve state.

* Hook up store_state_group

* Fix tests

* Rename _store_mult_state_groups_txn

* Rename StateGroupReadStore

* Remove redundant _have_persisted_state_group_txn

* Update comments

* Comment compute_event_context

* Set start val for state_group_id_seq

... otherwise we try to recreate old state groups

* Update comments

* Don't store state for outliers

* Update comment

* Update docstring as state groups are ints
 2018-02-06 14:31:24 +00:00
Erik Johnston
25c0a020f4 Updates tests 2018-02-05 16:01:48 +00:00
Richard van der Hoff
db91e72ade
Merge pull request #2844 from matrix-org/rav/evicted_metrics 
montoring metrics for number of cache evictions
 2018-02-05 16:34:36 +01:00
Richard van der Hoff
bc496df192 report metrics on number of cache evictions 2018-02-05 15:34:01 +00:00
Erik Johnston
a1beca0e25 Fix broken unit test for media storage 2018-02-05 12:44:03 +00:00
Erik Johnston
1f881e0746
Merge pull request #2791 from matrix-org/erikj/media_storage_refactor 
Ensure media is in local cache before thumbnailing
 2018-02-05 11:28:52 +00:00
Richard van der Hoff
6da4c4d3bd Factor out resolve_state_groups to a separate handler 
We extract the storage-independent bits of the state group resolution out to a
separate functiom, and stick it in a new handler, in preparation for its use
from the storage layer.
 2018-02-01 12:40:04 +00:00
Richard van der Hoff
d1f3490e75 Add tests for user directory search 2018-01-27 17:21:57 +00:00
Richard van der Hoff
88b9c5cbf0 Make it possible to run tests against postgres 2018-01-27 17:15:24 +00:00
Richard van der Hoff
d7eacc4f87 Create dbpool as normal in tests 
... instead of creating our own special SQLiteMemoryDbPool, whose purpose was a
bit of a mystery.

For some reason this makes one of the tests run slightly slower, so bump the
sleep(). Sorry.
 2018-01-27 17:15:15 +00:00
Richard van der Hoff
b178eca261 Run on_new_connection for unit tests 
Configure the connectionpool used for unit tests to run the `on_new_connection`
function.
 2018-01-27 17:06:04 +00:00
Richard van der Hoff
6e9bf67f18 Remove unused/bitrotted MemoryDataStore 
This isn't used, and looks thoroughly bitrotted.
 2018-01-26 00:35:15 +00:00
Richard van der Hoff
2b91846497 Remove spurious unittest.DEBUG 2018-01-26 00:34:27 +00:00
Richard van der Hoff
b2da272b77
Merge pull request #2821 from matrix-org/rav/matthew_test_fixes 
Matthew's fixes to the unit tests
 2018-01-22 20:43:56 +00:00
Matthew Hodgson
ab9f844aaf
Add federation_domain_whitelist option (#2820) 
Add federation_domain_whitelist

gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
 2018-01-22 19:11:18 +01:00
Richard van der Hoff
5c431f421c Matthew's fixes to the unit tests 
Extracted from https://github.com/matrix-org/synapse/pull/2820
 2018-01-22 16:46:16 +00:00
Matthew Hodgson
d84f65255e
Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid 
add registrations_require_3pid and allow_local_3pids
 2018-01-22 13:57:22 +00:00
Matthew Hodgson
81d037dbd8 mock registrations_require_3pid 2018-01-19 00:28:08 +00:00
Erik Johnston
ce4f66133e Add unit tests 2018-01-18 16:32:11 +00:00
Erik Johnston
b6dc7044a9
Merge pull request #2804 from matrix-org/erikj/file_consumer 
Add decent impl of a FileConsumer
 2018-01-18 16:31:33 +00:00
Erik Johnston
1432f7ccd5 Move test stuff to tests 2018-01-18 11:57:57 +00:00
Erik Johnston
bc67e7d260 Add decent impl of a FileConsumer 
Twisted core doesn't have a general purpose one, so we need to write one
ourselves.

Features:
- All writing happens in background thread
- Supports both push and pull producers
- Push producers get paused if the consumer falls behind
 2018-01-17 16:43:03 +00:00
Richard van der Hoff
44a498418c Optimise LoggingContext creation and copying 
It turns out that the only thing we use the __dict__ of LoggingContext for is
`request`, and given we create lots of LoggingContexts and then copy them every
time we do a db transaction or log line, using the __dict__ seems a bit
redundant. Let's try to optimise things by making the request attribute
explicit.
 2018-01-16 15:49:42 +00:00
Richard van der Hoff
a767f06e3f
Merge pull request #2765 from matrix-org/rav/fix_room_uts 
Fix flaky test_rooms UTs
 2018-01-09 14:21:03 +00:00
Richard van der Hoff
f8fa5ae4af enable twisted delayedcall debugging in UTs 2018-01-09 12:06:45 +00:00
Richard van der Hoff
374c4d4ced Remove dead code 
pointless function is pointless
 2018-01-09 12:06:45 +00:00
Richard van der Hoff
142fb0a7d4 Disable user_directory updates for UTs 
Fix flakiness in the UTs caused by the user_directory being updated in the
background
 2018-01-09 12:06:45 +00:00
Richard van der Hoff
0211464ba2 Fix broken config UTs 
https://github.com/matrix-org/synapse/pull/2755 broke log-config generation,
which in turn broke the unit tests.
 2018-01-09 11:28:33 +00:00
Matthew Hodgson
33cb7ef0b7
Merge pull request #2723 from matrix-org/matthew/search-all-local-users 
Add all local users to the user_directory and optionally search them
 2017-12-05 11:09:47 +00:00
Richard van der Hoff
d5f9fb06b0 Refactor UI auth implementation 
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
 2017-12-05 09:40:05 +00:00
Matthew Hodgson
7b86c1fdcd try make tests work a bit more... 2017-12-04 17:10:03 +00:00
Matthew Hodgson
95f8a713dc erik told me to 2017-12-04 16:56:25 +00:00
Luke Barnard
ab1dc84779 Add extra space before inline comment 2017-11-16 18:22:40 +00:00
Luke Barnard
e836bdf734 Fix tests 2017-11-16 18:14:39 +00:00
Richard van der Hoff
63ef607f1f Fix tests for Store.__init__ update 
Fix the test to pass the right number of args to the Store constructors
 2017-11-13 10:46:08 +00:00
Ilya Zhuravlev
8a4a0ddea6 Fix appservice tests to account for new behavior of notify_interested_services 2017-11-02 23:19:57 +03:00
Richard van der Hoff
02237ce725 Fix tests for refresh_token removal 2017-11-01 10:19:42 +00:00
Richard van der Hoff
9ded00f221 fix tests 2017-10-31 14:21:13 +00:00
Richard van der Hoff
eaaabc6c4f replace 'except:' with 'except Exception:' 
what could possibly go wrong
 2017-10-23 15:52:32 +01:00
Richard van der Hoff
1135193dfd Validate group ids when parsing 
May as well do it whenever we parse a Group ID. We check the sigil and basic
structure here so it makes sense to check the grammar in the same place.
 2017-10-21 00:30:39 +01:00
Richard van der Hoff
dbdfd8967d Merge pull request #2546 from matrix-org/rav/remove_dead_event_injector 
Remove dead class
 2017-10-17 13:21:22 +01:00
Richard van der Hoff
b8e40d146f Merge pull request #2547 from matrix-org/rav/test_make_deferred_yieldable 
Add some tests for make_deferred_yieldable
 2017-10-17 13:21:09 +01:00
Richard van der Hoff
a6ad8148b9 Fix name of test_logcontext 
The file under test is logcontext.py, not log_context.py
 2017-10-17 10:53:34 +01:00
Richard van der Hoff
5b5f35ccc0 Add some tests for make_deferred_yieldable 2017-10-17 10:52:31 +01:00
Richard van der Hoff
9b714abf35 Remove dead class 
This isn't used anywhere.
 2017-10-17 10:43:36 +01:00
David Baker
33122c5a1b Fix test 2017-10-17 10:39:50 +01:00
Richard van der Hoff
cc794d60e7 Merge pull request #2532 from matrix-org/rav/fix_linearizer 
Fix stackoverflow and logcontexts from linearizer
 2017-10-11 17:29:32 +01:00
Richard van der Hoff
4fad8efbfb Fix stackoverflow and logcontexts from linearizer 
1. make it not blow out the stack when there are more than 50 things waiting
   for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505.

2. Make it not mess up the log contexts.
 2017-10-11 15:05:05 +01:00
David Baker
27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
Richard van der Hoff
f65e31d22f Do an AAAA lookup on SRV record targets (#2462) 
Support SRV records which point at AAAA records, as well as A records.

Fixes https://github.com/matrix-org/synapse/issues/2405
 2017-09-22 20:26:47 +01:00
Richard van der Hoff
72472456d8 Add some more tests for Keyring 2017-09-20 01:32:42 +01:00
Richard van der Hoff
9864efa532 Fix concurrent server_key requests (#2458) 
Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.
 2017-09-19 23:25:44 +01:00
Richard van der Hoff
aa620d09a0 Add a config option to block all room invites (#2457) 
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
 2017-09-19 16:08:14 +01:00
Erik Johnston
b5feaa5a49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged 2017-09-19 11:07:45 +01:00
Richard van der Hoff
290777b3d9 Clean up and document handling of logcontexts in Keyring (#2452) 
I'm still unclear on what the intended behaviour for
`verify_json_objects_for_server` is, but at least I now understand the
behaviour of most of the things it calls...
 2017-09-18 18:31:01 +01:00
Erik Johnston
bf81f3cf2c Split out profile handler to fix tests 2017-08-25 14:34:56 +01:00
Erik Johnston
c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston
ed9a7f5436 Merge pull request #2309 from matrix-org/erikj/user_ip_repl 
Fix up user_ip replication commands
 2017-07-06 14:33:14 +01:00
Erik Johnston
29a4066a4d Update test 2017-07-04 10:21:25 +01:00
Erik Johnston
8c23221666 Fix up 2017-06-27 15:53:45 +01:00
Erik Johnston
a0a561ae85 Fix up client ips to read from pending data 2017-06-27 14:46:12 +01:00
Erik Johnston
052c5d19d5 Merge pull request #2281 from matrix-org/erikj/phone_home_stats 
Fix phone home stats
 2017-06-15 12:46:23 +01:00
Erik Johnston
93e7a38370 Remove unhelpful test 2017-06-15 09:30:54 +01:00
Erik Johnston
23da638360 Fix typing tests 2017-05-26 10:02:04 +01:00
Erik Johnston
11c2a3655f Only load jinja2 templates once 
Instead of every time a new email pusher is created, as loading jinja2
templates is slow.
 2017-05-22 17:48:58 +01:00
Erik Johnston
bbfe4e996c Make get_state_groups_from_groups faster. 
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.

By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
 2017-05-17 15:12:15 +01:00
Richard van der Hoff
de042b3b88 Do some logging when one-time-keys get claimed 
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
 2017-05-09 19:04:56 +01:00
Richard van der Hoff
a7e9d8762d Allow clients to upload one-time-keys with new sigs 
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).

Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
 2017-05-09 19:04:56 +01:00
Erik Johnston
3e5a62ecd8 Add more granular event send metrics 2017-05-02 14:23:26 +01:00
Erik Johnston
85a0d6c7ab Remove test of replication resource 2017-04-11 10:59:27 +01:00
Erik Johnston
a5c401bd12 Merge pull request #2097 from matrix-org/erikj/repl_tcp_client 
Move to using TCP replication
 2017-04-05 09:36:21 +01:00
Erik Johnston
3a1f3f8388 Change slave storage to use new replication interface 
As the TCP replication uses a slightly different API and streams than
the HTTP replication.

This breaks HTTP replication.
 2017-04-03 15:34:19 +01:00
Erik Johnston
9cee0ce7db Merge pull request #2075 from matrix-org/erikj/cache_speed 
Speed up cached function access
 2017-03-31 10:10:56 +01:00
Erik Johnston
350333a09a Merge pull request #2076 from matrix-org/erikj/as_perf 
Make AS's faster
 2017-03-31 09:43:10 +01:00
Erik Johnston
eefd9fee81 Fix up tests 2017-03-30 14:14:46 +01:00
Richard van der Hoff
f9b4bb05e0 Fix the logcontext handling in the cache wrappers (#2077) 
The cache wrappers had a habit of leaking the logcontext into the reactor while
the lookup function was running, and then not restoring it correctly when the
lookup function had completed. It's all the fault of
`preserve_context_over_{fn,deferred}` which are basically a bit broken.
 2017-03-30 13:22:24 +01:00
Erik Johnston
650f0e69f2 Compile the regex's used in ASes 2017-03-28 13:27:21 +01:00
Erik Johnston
57cfa513f5 Merge pull request #2054 from matrix-org/erikj/user_iter_cursor 
Reduce some CPU work on DB threads
 2017-03-24 11:40:08 +00:00
Richard van der Hoff
9397edb28b Merge pull request #2050 from matrix-org/rav/federation_backoff 
push federation retry limiter down to matrixfederationclient
 2017-03-23 22:27:01 +00:00
Richard van der Hoff
06ce7335e9 Merge pull request #2052 from matrix-org/rav/time_bound_deferred 
Fix time_bound_deferred to throw the right exception
 2017-03-23 22:22:54 +00:00
Erik Johnston
00957d1aa4 User Cursor.__iter__ instead of fetchall 
This prevents unnecessary construction of lists
 2017-03-23 17:53:49 +00:00
pik
566641a0b5 use jsonschema.FormatChecker for RoomID and UserID strings 
* use a valid filter in rest/client/v2_alpha test

Signed-off-by: pik <alexander.maznev@gmail.com>
 2017-03-23 11:42:41 -03:00
pik
acafcf1c5b Add valid filter tests, flake8, fix typo 
Signed-off-by: pik <alexander.maznev@gmail.com>
 2017-03-23 11:42:10 -03:00
pik
e56c79c114 check_valid_filter using JSONSchema 
* add invalid filter tests

Signed-off-by: pik <alexander.maznev@gmail.com>
 2017-03-23 11:42:07 -03:00
Richard van der Hoff
0bfea9a2be fix tests 2017-03-23 13:20:08 +00:00
Richard van der Hoff
b88a323ffb Fix time_bound_deferred to throw the right exception 
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred
would throw a CancelledError when the deferred timed out, which was rather
confusing.
 2017-03-23 12:07:11 +00:00
Richard van der Hoff
4bd597d9fc push federation retry limiter down to matrixfederationclient 
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
 2017-03-23 09:28:46 +00:00
Richard van der Hoff
95f21c7a66 Fix caching of remote servers' signature keys 
The `@cached` decorator on `KeyStore._get_server_verify_key` was missing
its `num_args` parameter, which meant that it was returning the wrong key for
any server which had more than one recorded key.

By way of a fix, change the default for `num_args` to be *all* arguments. To
implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.
 2017-03-22 15:11:30 +00:00
Erik Johnston
37a187bfab Merge pull request #2033 from matrix-org/erikj/repl_speed 
Don't send the full event json over replication
 2017-03-21 13:11:15 +00:00
Erik Johnston
737f283a07 Fix unit test 2017-03-20 14:03:58 +00:00
Richard van der Hoff
f40c2db05a Stop preserve_fn leaking context into the reactor 
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.

Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
 2017-03-18 00:07:43 +00:00
Will Hunt
c144292373 Modify test_user_sync so it doesn't look at last_active_ts over last_user_sync_ts 2017-03-15 15:38:57 +00:00
Erik Johnston
095b45c165 Aggregate event push actions 2017-02-14 13:39:41 +00:00
Erik Johnston
51adaac953 Fix email push in pusher worker 
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
 2017-02-02 10:53:36 +00:00
Erik Johnston
692daf6f54 Remote membership tests for replication 
This is because it now relies of the caches stream, which only works on
postgres. We are trying to test with sqlite.
 2017-01-31 16:10:16 +00:00
Erik Johnston
c7a26b7c32 Fix unit tests 2017-01-30 17:11:24 +00:00
Erik Johnston
b3e1f2aa7a Fix unit tests 2017-01-26 17:16:24 +00:00
Erik Johnston
c974116f19 Implement device key caching over federation 2017-01-26 16:07:24 +00:00
Erik Johnston
2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00