Neil Johnson
0ddf486724
expose number of real reserved users
2018-09-12 11:58:52 +01:00
Richard van der Hoff
0b07f02e19
Make sure that we close db connections opened during init
...
We should explicitly close any db connections we open, because failing to do so
can block other transactions as per
https://github.com/matrix-org/synapse/issues/3682 .
Let's also try to factor out some of the boilerplate by having server classes
define their datastore class rather than duplicating the whole of `setup`.
2018-08-28 13:39:49 +01:00
Erik Johnston
764030cf63
Merge pull request #3659 from matrix-org/erikj/split_profiles
...
Allow profile updates to happen on workers
2018-08-22 11:35:55 +01:00
Richard van der Hoff
3cef867cc1
Merge pull request #3709 from matrix-org/rav/logcontext_for_replication_commands
...
Logcontexts for replication command handlers
2018-08-17 16:22:07 +01:00
Richard van der Hoff
c144252a8c
Merge pull request #3710 from matrix-org/rav/logcontext_for_pusher_updates
...
Fix logcontexts for running pushers
2018-08-17 16:21:49 +01:00
Amber Brown
c334ca67bb
Integrate presence from hotfixes ( #3694 )
2018-08-18 01:08:45 +10:00
Erik Johnston
ab822a2d1f
Add some fixmes
2018-08-17 15:31:50 +01:00
Erik Johnston
91cdb6de08
Call UserDirectoryHandler methods directly
...
Turns out that the user directory handling is fairly racey as a bunch
of stuff assumes that the processing happens on master, which it doesn't
when there is a synapse.app.user_dir worker. So lets just call the
function directly until we actually get round to fixing it, since it
doesn't make the situation any worse.
2018-08-17 15:26:13 +01:00
Erik Johnston
782689bd40
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles
2018-08-17 14:15:48 +01:00
Neil Johnson
521d369e7a
remove errant yield
2018-08-17 10:12:11 +01:00
Richard van der Hoff
0e8d78f6aa
Logcontexts for replication command handlers
...
Run the handlers for replication commands as background processes. This should
improve the visibility in our metrics, and reduce the number of "running db
transaction from sentinel context" warnings.
Ideally it means converting the things that fire off deferreds into the night
into things that actually return a Deferred when they are done. I've made a bit
of a stab at this, but it will probably be leaky.
2018-08-17 00:43:43 +01:00
Richard van der Hoff
66f7dc8c87
Fix logcontexts for running pushers
...
First of all, avoid resetting the logcontext before running the pushers, to fix
the "Starting db txn 'get_all_updated_receipts' from sentinel context" warning.
Instead, give them their own "background process" logcontexts.
2018-08-17 00:32:39 +01:00
Neil Johnson
bcfeb44afe
call reap on start up and fix under reaping bug
2018-08-16 22:55:32 +01:00
Erik Johnston
7d5b1a60a3
Fix inbound federation on reader worker
...
Inbound federation requires calculating push, which in turn relies on
having access to account data.
2018-08-16 15:32:20 +01:00
Erik Johnston
ef184caf30
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation
2018-08-15 14:25:46 +01:00
Erik Johnston
773db62a22
Rename slave TransactionStore to SlaveTransactionStore
2018-08-15 14:17:06 +01:00
Neil Johnson
e5962f845c
pep8
2018-08-14 16:36:14 +01:00
Neil Johnson
e7d091fb86
combine mau metrics into one group
2018-08-14 16:26:55 +01:00
Amber Brown
b37c472419
Rename async to async_helpers because async
is a keyword on Python 3.7 ( #3678 )
2018-08-10 23:50:21 +10:00
Richard van der Hoff
3c0213a217
Merge pull request #3439 from vojeroen/send_sni_for_federation_requests
...
send SNI for federation requests
2018-08-10 12:23:54 +01:00
Erik Johnston
484a0ebdfc
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles
2018-08-09 10:16:29 +01:00
Erik Johnston
5785b93711
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation
2018-08-09 10:16:16 +01:00
Neil Johnson
e8eba2b4e3
implement reserved users for mau limits
2018-08-07 17:49:43 +01:00
Erik Johnston
495cb100d1
Allow profile changes to happen on workers
2018-08-07 10:50:26 +01:00
Erik Johnston
96a9a29645
Pull in necessary stores in federation_reader
2018-08-06 15:23:57 +01:00
Erik Johnston
e26dbd82ef
Add replication APIs for persisting federation events
2018-08-06 15:02:28 +01:00
Neil Johnson
950807d93a
fix caching and tests
2018-08-03 13:49:53 +01:00
Neil Johnson
c0affa7b4f
update generate_monthly_active_users, and reap_monthly_active_users
2018-08-02 23:03:01 +01:00
Neil Johnson
d766f26de9
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker
2018-08-01 17:49:41 +01:00
Neil Johnson
b7f203a566
count_monthly_users is now async
2018-08-01 16:17:42 +01:00
Neil Johnson
2c54f1c225
remove need to plot limit_usage_by_mau
2018-08-01 11:46:59 +01:00
Neil Johnson
c507fa15ce
only need to loop if mau limiting is enabled
2018-08-01 10:20:42 +01:00
Neil Johnson
6ef983ce5c
api into monthly_active_users table
2018-07-31 16:36:24 +01:00
Neil Johnson
df2235e7fa
coding style
2018-07-31 13:16:20 +01:00
Neil Johnson
21276ff846
remove errant logging
2018-07-30 22:42:12 +01:00
Neil Johnson
9b13817e06
factor out metrics from __init__ to app/homeserver
2018-07-30 22:07:07 +01:00
Richard van der Hoff
7041cd872b
Merge branch 'develop' into send_sni_for_federation_requests
2018-07-27 09:17:11 +01:00
Richard van der Hoff
03751a6420
Fix some looping_call calls which were broken in #3604
...
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604 .
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
Richard van der Hoff
371da42ae4
Wrap a number of things that run in the background
...
This will reduce the number of "Starting db connection from sentinel context"
warnings, and will help with our metrics.
2018-07-25 09:41:12 +01:00
Erik Johnston
8b8c4f34a3
Replace usage of get_current_toke with StreamToken.START
...
This allows us to handle /context/ requests on the client_reader worker
without having to pull in all the various stream handlers (e.g.
precence, typing, pushers etc). The only thing the token gets used for
is pagination, and that ignores everything but the room portion of the
token.
2018-07-24 16:49:17 +01:00
Erik Johnston
3188973857
Pull out did_forget to worker store
2018-07-24 16:49:14 +01:00
Erik Johnston
0b0b24cb82
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move
2018-07-23 13:21:15 +01:00
Amber Brown
3132b89f12
Make the rest of the .iterwhatever go away ( #3562 )
2018-07-21 15:47:18 +10:00
Erik Johnston
9b596177ae
Add some room read only APIs to client_reader
2018-07-18 15:33:03 +01:00
Jeroen
505530f36a
Merge remote-tracking branch 'upstream/develop' into send_sni_for_federation_requests
...
# Conflicts:
# synapse/crypto/context_factory.py
2018-07-14 20:24:46 +02:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Jeroen
3d605853c8
send SNI for federation requests
2018-06-24 22:38:43 +02:00
Richard van der Hoff
245d53d32a
Write a clear restart indicator in logs
...
I'm fed up with never being able to find the point a server restarted in the
logs.
2018-06-20 15:33:14 +01:00
Amber Brown
0fde1896cd
Merge pull request #3389 from turt2live/travis/name_metrics
...
Use the correct flag (enable_metrics) when warning about an incorrect metrics setup
2018-06-13 23:50:10 +10:00
Travis Ralston
45768d1640
Use the RegistryProxy for appservices too
...
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-06-12 12:55:48 -06:00
Travis Ralston
12285a1a76
The flag is named enable_metrics, not collect_metrics
...
Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-06-12 12:51:31 -06:00
Erik Johnston
042eedfa2b
Add hacky cache factor override system
2018-06-04 15:39:28 +01:00
Neil Johnson
71477f3317
Merge pull request #3264 from matrix-org/neil/sign-up-stats
...
daily user type phone home stats
2018-06-01 13:42:01 +00:00
Amber Brown
febe0ec8fd
Run Prometheus on a different port, optionally. ( #3274 )
2018-05-31 19:04:50 +10:00
Amber Brown
85ba83eb51
fixes
2018-05-22 16:28:23 -05:00
Neil Johnson
d8cb7225d2
daily user type phone home stats
2018-05-22 18:09:09 +01:00
Amber Brown
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
Richard van der Hoff
b5b2d5d64b
Fix dependency on jinja2
...
Delay the import of ConsentResource, so that we can get away without jinja2 if
people don't have the consent resource enabled.
Fixes #3259
2018-05-22 14:03:45 +01:00
Amber Brown
02c1d29133
look at the Prometheus metrics instead
2018-05-21 17:02:20 -05:00
Neil Johnson
dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics
...
user visit data
2018-05-16 10:09:24 +00:00
Neil Johnson
a2204cc9cc
remove unused method recurring_user_daily_visit_stats
2018-05-16 09:47:20 +01:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler
...
ConsentResource to gather policy consent from users
2018-05-16 07:19:18 +01:00
Neil Johnson
c92a8aa578
pep8
2018-05-15 17:31:11 +01:00
Neil Johnson
05ac15ae82
Limit query load of generate_user_daily_visits
...
The aim is to keep track of when it was last called and only query from that point in time
2018-05-15 17:01:33 +01:00
Richard van der Hoff
47815edcfa
ConsentResource to gather policy consent from users
...
Hopefully there are enough comments and docs in this that it makes sense on its
own.
2018-05-15 15:11:59 +01:00
Neil Johnson
f077e97914
instead of inserting user daily visit data at the end of the day, instead insert incrementally through the day
2018-05-14 13:50:58 +01:00
Neil Johnson
977765bde2
Merge branch 'develop' of https://github.com/matrix-org/synapse into cohort_analytics
2018-05-14 09:31:42 +01:00
Richard van der Hoff
318711e139
Set Server header in SynapseRequest
...
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).
This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.
Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
2018-05-10 18:50:27 +01:00
Neil Johnson
5917562b60
10 mins seems more reasonable that every minute
2018-05-01 12:12:22 +01:00
Richard van der Hoff
fc149b4eeb
Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background
2018-04-27 14:31:23 +01:00
Richard van der Hoff
2a13af23bc
Use run_in_background in preference to preserve_fn
...
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
Richard van der Hoff
9255a6cb17
Improve exception handling for background processes
...
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.
This is unsatisfactory for a number of reasons:
- logging on garbage collection is best-effort and may happen some time after
the error, if at all
- it can be hard to figure out where the error actually happened.
- it is logged as a scary CRITICAL error which (a) I always forget to grep for
and (b) it's not really CRITICAL if a background process we don't care about
fails.
So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Neil Johnson
617bf40924
Generate user daily stats
2018-04-25 17:37:29 +01:00
Adrian Tschira
36c59ce669
Use six.itervalues in some places
...
There's more where that came from
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:39:43 +02:00
Richard van der Hoff
87478c5a60
Merge pull request #3061 from NotAFile/add-some-byte-strings
...
Add b prefixes to some strings that are bytes in py3
2018-04-09 23:54:05 +01:00
Richard van der Hoff
15e8ed874f
more verbosity in synctl
2018-04-06 09:28:36 +01:00
Neil Johnson
68b0ee4e8d
Merge pull request #3041 from matrix-org/r30_stats
...
R30 stats
2018-04-05 15:37:37 +01:00
Neil Johnson
0e5f479fc0
Review comments
...
Use iteritems over item to loop over dict
formatting
2018-04-05 12:16:46 +01:00
Jan Christian Grünhage
e4570c53dd
phone home cache size configurations
2018-04-04 16:46:58 +01:00
Adrian Tschira
6168351877
Add b prefixes to some strings that are bytes in py3
...
This has no effect on python2
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-04 13:48:51 +02:00
Richard van der Hoff
725a72ec5a
Merge pull request #3000 from NotAFile/change-except-style
...
Replace old style error catching with 'as' keyword
2018-04-04 10:45:22 +01:00
Neil Johnson
86932be2cb
Support multi client R30 for psql
2018-03-28 14:36:53 +01:00
Michael Kaye
33f6195d9a
Handle review comments
2018-03-28 14:25:25 +01:00
Neil Johnson
792d340572
rename stat to future proof
2018-03-28 12:25:02 +01:00
Michael Kaye
4ceaa7433a
As daemonizing will make a new process, defer call to init.
2018-03-28 12:19:01 +01:00
Neil Johnson
a32d2548d9
query and call for r30 stats
2018-03-28 10:39:13 +01:00
Michael Kaye
ef520d8d0e
Include coarse CPU and Memory use in stats callbacks.
...
This requires the psutil module, and is still opt-in based on the report_stats
config option.
2018-03-27 17:56:03 +01:00
Matthew Hodgson
8cbbfaefc1
404 correctly on missing paths via NoResource
...
fixes https://github.com/matrix-org/synapse/issues/2043 and https://github.com/matrix-org/synapse/issues/2029
2018-03-23 10:32:50 +00:00
NotAFile
2cc9f76bc3
replace old style error catching with 'as' keyword
...
This is both easier to read and compatible with python3 (not that that
matters)
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-03-15 16:11:17 +01:00
Erik Johnston
4f28018c83
Register membership/state servlets in event_creator
2018-03-14 14:30:06 +00:00
Erik Johnston
95cb401ae0
Merge pull request #2978 from matrix-org/erikj/refactor_replication_layer
...
Remove ReplicationLayer and user Client/Server directly
2018-03-13 15:45:08 +00:00
Erik Johnston
cb9f8e527c
s/replication_client/federation_client/
2018-03-13 13:26:52 +00:00
Erik Johnston
d023ecb810
Don't build handlers on workers unnecessarily
2018-03-13 11:08:10 +00:00
Erik Johnston
265b993b8a
Split replication layer into two
2018-03-13 10:55:47 +00:00
Erik Johnston
28e973ac11
Calculate push actions on worker
2018-02-28 18:02:30 +00:00
Erik Johnston
493e25d554
Move storage functions for push calculations
...
This will allow push actions for an event to be calculated on workers.
2018-02-27 13:58:16 +00:00
Erik Johnston
8ec2e638be
Add event_creator worker
2018-02-07 10:32:32 +00:00
Erik Johnston
24dd73028a
Add replication http endpoint for event sending
2018-02-07 10:32:32 +00:00
Richard van der Hoff
25adde9a04
Factor out get_db_conn to HomeServer base class
...
This function is identical to all subclasses, so we may as well push it up to
the base class to reduce duplication (and make use of it in the tests)
2018-01-26 00:56:49 +00:00
Erik Johnston
f4d93ae424
Actually make it work
2018-01-12 10:39:27 +00:00
Erik Johnston
f68e4cf690
Refactor
2018-01-12 10:11:12 +00:00
Erik Johnston
7cd34512d8
When using synctl with workers, don't start the main synapse automatically
2018-01-11 11:37:39 +00:00
Silke
df0f602796
Implement listen_tcp method in remaining workers
...
Signed-off-by: Silke <silke@slxh.eu>
2017-12-18 20:00:42 +01:00
Silke
26cd3f5690
Remove logger argument and do not catch replication listener
...
Signed-off-by: Silke <silke@slxh.eu>
2017-12-18 20:00:42 +01:00
Silke Hofstra
ed48ecc58c
Add methods for listening on multiple addresses
...
Add listen_tcp and listen_ssl which implement Twisted's reactor.listenTCP
and reactor.listenSSL for multiple addresses.
Signed-off-by: Silke Hofstra <silke@slxh.eu>
2017-12-17 13:15:48 +01:00
Silke Hofstra
37d1a90025
Allow binds to both :: and 0.0.0.0
...
Binding on 0.0.0.0 when :: is specified in the bind_addresses is now allowed.
This causes a warning explaining the behaviour.
Configuration changed to match.
See #2232
Signed-off-by: Silke Hofstra <silke@slxh.eu>
2017-12-17 13:10:31 +01:00
Richard van der Hoff
795b0849f3
Add a comment which might save some confusion
2017-11-24 00:34:56 +00:00
Richard van der Hoff
7f14f0ae38
Remove dead sync_callback
...
This is never used; let's remove it to stop confusing things.
2017-11-24 00:32:04 +00:00
Richard van der Hoff
68ca864141
Add config option to disable media_repo on main synapse
...
... to stop us doing the cache cleanup jobs on the master.
2017-11-22 16:20:27 +00:00
Richard van der Hoff
e1fd4751de
Build MediaRepositoryResource as a homeserver dependency
...
This avoids the scenario where we have four different PreviewUrlResources
configured on a single app, each of which have their own caches and cache
clearing jobs.
2017-11-22 16:19:49 +00:00
David Baker
d4553818a0
Merge pull request #2627 from matrix-org/rav/custom_rest_endpoints
...
Add a hook for custom rest endpoints
2017-11-02 15:18:37 +00:00
Richard van der Hoff
fcdfc911ee
Add a hook for custom rest endpoints
...
Let the user specify custom modules which can be used for implementing extra
endpoints.
2017-11-02 14:36:55 +00:00
Richard van der Hoff
6650a07ede
Factor out _configure_named_resource
...
This was a bit of a code vomit, so let's factor it out to preserve some sanity
2017-11-02 14:33:37 +00:00
Richard van der Hoff
8b56977b6f
Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header
...
Front-end proxy: pass through auth header
2017-10-27 11:01:50 +01:00
Richard van der Hoff
54a2525133
Front-end proxy: pass through auth header
...
So that access-token-in-an-auth-header works.
2017-10-26 18:19:01 +01:00
Richard van der Hoff
0a5866bec9
Support /keys/upload on /r0 as well as /unstable
...
(So that we can stop riot relying on it in /unstable)
2017-10-26 18:18:23 +01:00
Richard van der Hoff
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Erik Johnston
535cc49f27
Merge pull request #2466 from matrix-org/erikj/groups_merged
...
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Erik Johnston
ea87cb1ba5
Make 'affinity' package optional
2017-10-02 18:03:59 +01:00
Erik Johnston
3fed5bb25f
Move quit_with_error
2017-10-02 17:59:34 +01:00
Erik Johnston
b5feaa5a49
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged
2017-09-19 11:07:45 +01:00
Richard van der Hoff
692250c6be
Fix user_dir startup
...
Add missing parameter to _base.start_worker_reactor
2017-08-16 15:11:29 +01:00
Richard van der Hoff
10d8b701a1
Allow configuration of CPU affinity
...
Make it possible to set the CPU affinity in the config file, so that we don't
need to remember to do it manually every time.
2017-08-15 17:08:28 +01:00
Richard van der Hoff
543c794a76
Factor out common application start
...
We have 10 copies of this code, and I don't really want to update each one
separately.
2017-08-15 17:04:40 +01:00
Erik Johnston
2cc998fed8
Fix replication. And notify
2017-07-20 17:13:18 +01:00
Erik Johnston
d4d12daed9
Include registration and as stores in frontend proxy
2017-07-07 18:36:45 +01:00
Erik Johnston
1fc4a962e4
Add a frontend proxy
2017-07-07 18:19:46 +01:00
Erik Johnston
78cefd78d6
Make workers report to master for user ip updates
2017-06-27 14:58:10 +01:00
Erik Johnston
8177563ebe
Fix for workers
2017-06-21 13:57:49 +01:00
Erik Johnston
dfeca6cf40
Merge pull request #2286 from matrix-org/erikj/split_out_user_dir
...
Split out user directory to a separate process
2017-06-16 13:01:19 +01:00
Erik Johnston
6aa5bc8635
Initial worker impl
2017-06-16 11:47:11 +01:00
Erik Johnston
5ddd199870
Typo
2017-06-15 10:49:10 +01:00
Erik Johnston
4b461a6931
Add some more stats
2017-06-15 09:39:39 +01:00
Erik Johnston
617304b2cf
Fix phone home stats
2017-06-14 19:47:15 +01:00
Erik Johnston
f87b287291
Merge pull request #2127 from APwhitehat/alreadystarted
...
print something legible if synapse already running
2017-04-27 15:46:53 +01:00
Matthew Wolff
d4040e9e28
Queried CONDITIONAL_REQUIREMENTS
2017-04-18 16:19:48 -05:00
Matthew Wolff
54f9a4cb59
Fixed travis build failure
...
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
2017-04-17 01:38:27 -05:00
Matthew Wolff
8e780b113d
web_server_root documentation fix
...
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
2017-04-17 00:49:11 -05:00
Anant Prakash
c1f52a321d
synctl.py: Check if synapse is already running
2017-04-13 18:00:02 +05:30
Erik Johnston
247c736b9b
Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl
...
Reduce federation replication traffic
2017-04-12 11:07:13 +01:00
Erik Johnston
9c712a366f
Move get_presence_list_* to SlaveStore
2017-04-11 16:07:33 +01:00
Erik Johnston
414522aed5
Move get_interested_parties
2017-04-11 15:33:26 +01:00
Erik Johnston
28a4649785
Remove HTTP replication APIs
2017-04-11 09:52:11 +01:00
Erik Johnston
29574fd5b3
Reduce federation presence replication traffic
...
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00
Erik Johnston
449d1297ca
Fix up federation SendQueue and document types
2017-04-07 11:48:33 +01:00
Erik Johnston
54f59bd7d4
Merge pull request #2107 from HarHarLinks/patch-1
...
fix typo in synctl help
2017-04-07 09:54:37 +01:00
Erik Johnston
98ce212093
Merge pull request #2103 from matrix-org/erikj/no-double-encode
...
Don't double encode replication data
2017-04-07 09:39:52 +01:00
Kim Brose
8a1137ceab
fix typo in synctl help
2017-04-06 17:10:20 +02:00
Erik Johnston
391712a4f9
Comment
2017-04-06 13:35:00 +01:00
Erik Johnston
dbf87282d3
Docs
2017-04-06 13:11:21 +01:00
Erik Johnston
b5cb6347a4
Don't immediately notify the master about users whose syncs have gone away
2017-04-05 13:25:40 +01:00
Erik Johnston
96b9b6c127
Don't double json encode typing replication data
2017-04-05 11:34:20 +01:00
Erik Johnston
f10ce8944b
Don't double json encode federation replication data
2017-04-05 11:10:28 +01:00
Erik Johnston
a5c401bd12
Merge pull request #2097 from matrix-org/erikj/repl_tcp_client
...
Move to using TCP replication
2017-04-05 09:36:21 +01:00
Erik Johnston
d1d5362267
Add comment
2017-04-04 16:41:03 +01:00
Erik Johnston
ac66e11f2b
Add the appropriate amount of preserve_fn
2017-04-04 15:22:54 +01:00
Erik Johnston
3376f16012
Shuffle and comment synchrotron presence
2017-04-04 11:14:16 +01:00
Erik Johnston
6ce6bbedcb
Move where we ack federation
2017-04-04 11:02:44 +01:00
Erik Johnston
62b89daac6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server
2017-04-04 09:46:16 +01:00
Erik Johnston
36c28bc467
Update all the workers and master to use TCP replication
2017-04-03 15:35:52 +01:00
Erik Johnston
8a240e4f9c
Merge pull request #2078 from APwhitehat/assertuserfriendly
...
add user friendly report of assertion error in synctl.py
2017-03-31 14:41:49 +01:00
Erik Johnston
3ba2859e0c
Add tcp replication listener type and hook it up
2017-03-30 13:31:10 +01:00
Anant Prakash
6cdca71079
synctl.py: wait for synapse to stop before restarting ( #2020 )
2017-03-29 19:20:13 +01:00
Anant Prakash
305d16d612
add user friendly report of assertion error in synctl.py
...
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2017-03-29 20:41:39 +05:30
Erik Johnston
48e7697911
Add slave transaction store
2017-03-24 13:05:30 +00:00
Erik Johnston
37a187bfab
Merge pull request #2033 from matrix-org/erikj/repl_speed
...
Don't send the full event json over replication
2017-03-21 13:11:15 +00:00
Erik Johnston
aac6d1fc9b
PEP8
2017-03-20 13:47:56 +00:00
Richard van der Hoff
067b00d49d
Run the reactor with the sentinel logcontext
...
This fixes a class of 'Unexpected logcontext' messages, which were happening
because the logcontext was somewhat arbitrarily swapping between the sentinel
and the `run` logcontext.
2017-03-18 00:07:43 +00:00
Erik Johnston
61f471f779
Don't send the full event json over replication
2017-03-17 15:50:01 +00:00
Erik Johnston
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
Richard van der Hoff
d84bd51e95
Refactor logger config for workers
...
- to make it easier to add more config options.
2017-03-10 15:34:01 +00:00
Mark Haines
3a46280ca3
Add db functions needed for room initial sync to slave
2017-02-13 11:16:53 +00:00
Erik Johnston
4ac363a168
Remove debug logging
2017-01-30 14:10:12 +00:00
Erik Johnston
252b503fc8
Hook device list updates to replication
2017-01-27 14:31:35 +00:00
Erik Johnston
edd6cdfc9a
Restore default bind address
2017-01-10 17:21:41 +00:00
Erik Johnston
8a0fddfd73
Remove spurious for..else..
2017-01-10 16:30:53 +00:00
Johannes Löthberg
702c020e58
Fix check for bind_address
...
The empty string is a valid setting for the bind_address option, so
explicitly check for None here instead.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-20 01:37:50 +01:00
Johannes Löthberg
7dfd70fc83
Add support for specifying multiple bind addresses
...
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 21:51:56 +01:00
Erik Johnston
4d9b5c60f9
Comment
2016-11-23 11:11:41 +00:00
Erik Johnston
4c79a63fd7
Explicit federation ack
2016-11-23 10:40:44 +00:00
Erik Johnston
50934ce460
Comments
2016-11-21 16:55:23 +00:00
Erik Johnston
7c9cdb2245
Store federation stream positions in the database
2016-11-21 11:33:08 +00:00
Erik Johnston
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
Erik Johnston
ed787cf09e
Hook up the send queue and create a federation sender worker
2016-11-16 17:34:44 +00:00
Erik Johnston
24772ba56e
Respect use_frozen_dicts option in workers
2016-11-08 11:07:18 +00:00
Paul "LeoNerd" Evans
1cc22da600
Set up the process collector during metrics __init__; that way all split-process workers have it
2016-10-27 18:09:34 +01:00
Paul "LeoNerd" Evans
aac13b1f9a
Pass the Metrics group into the process collector instead of having it find its own one; this avoids it needing to import from synapse.metrics
2016-10-27 18:08:15 +01:00
Paul "LeoNerd" Evans
3ae48a1f99
Move the process metrics collector code into its own file
2016-10-19 18:10:24 +01:00
Erik Johnston
a68ade6ed3
Merge pull request #1162 from larroy/master
...
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-14 21:42:55 +01:00
Pedro Larroy
c57f871184
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-08 23:55:20 +02:00
Erik Johnston
748d8fdc7b
Reduce DB hits for replication
...
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
2016-09-23 16:49:21 +01:00
Erik Johnston
87528f0756
Support /initialSync in synchrotron worker
2016-09-21 11:46:28 +01:00
Matthew Hodgson
883df2e983
fix logger for client_reader worker
2016-09-17 14:12:04 +01:00
Erik Johnston
4be85281f9
Enable state caches on workers
2016-09-15 14:31:22 +01:00
Erik Johnston
e0b6db29ed
Split out public room list into a worker process
2016-09-14 14:42:51 +01:00
Erik Johnston
44330a21e9
Comment
2016-09-09 17:22:07 +01:00
Mark Haines
7ed5acacf4
Fix up the calls to the notifier for device messages
2016-09-01 18:08:40 +01:00
Erik Johnston
873f870e5a
Add new direct message storage functions to slave store
2016-08-30 09:40:32 +01:00
Mark Haines
689f4cb914
Update comment
2016-08-22 16:17:31 +01:00
Mark Haines
c8f9b45bc2
Add usage stats to prometheus monitoring
2016-08-22 15:34:38 +01:00