Matrix-Mirrors/anonymousland-synapse
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse.git synced 2024-12-14 19:44:18 -05:00
Code Issues Packages Projects Releases Wiki Activity
21,957 Commits 4 Branches 615 Tags 63 MiB
08a881dba4
Commit Graph

912 Commits

Author SHA1 Message Date
Neil Johnson
d8cb7225d2 daily user type phone home stats 2018-05-22 18:09:09 +01:00
Amber Brown
a8990fa2ec Merge remote-tracking branch 'origin/develop' into 3218-official-prom 2018-05-22 10:50:26 -05:00
Richard van der Hoff
b5b2d5d64b Fix dependency on jinja2 
Delay the import of ConsentResource, so that we can get away without jinja2 if
people don't have the consent resource enabled.

Fixes #3259
 2018-05-22 14:03:45 +01:00
Amber Brown
02c1d29133 look at the Prometheus metrics instead 2018-05-21 17:02:20 -05:00
Neil Johnson
dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics 
user visit data
 2018-05-16 10:09:24 +00:00
Neil Johnson
a2204cc9cc remove unused method recurring_user_daily_visit_stats 2018-05-16 09:47:20 +01:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler 
ConsentResource to gather policy consent from users
 2018-05-16 07:19:18 +01:00
Neil Johnson
c92a8aa578 pep8 2018-05-15 17:31:11 +01:00
Neil Johnson
05ac15ae82 Limit query load of generate_user_daily_visits 
The aim is to keep track of when it was last called and only query from that point in time
 2018-05-15 17:01:33 +01:00
Richard van der Hoff
47815edcfa ConsentResource to gather policy consent from users 
Hopefully there are enough comments and docs in this that it makes sense on its
own.
 2018-05-15 15:11:59 +01:00
Neil Johnson
f077e97914 instead of inserting user daily visit data at the end of the day, instead insert incrementally through the day 2018-05-14 13:50:58 +01:00
Neil Johnson
977765bde2 Merge branch 'develop' of https://github.com/matrix-org/synapse into cohort_analytics 2018-05-14 09:31:42 +01:00
Richard van der Hoff
318711e139 Set Server header in SynapseRequest 
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).

This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.

Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
 2018-05-10 18:50:27 +01:00
Neil Johnson
5917562b60 10 mins seems more reasonable that every minute 2018-05-01 12:12:22 +01:00
Richard van der Hoff
fc149b4eeb Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background 2018-04-27 14:31:23 +01:00
Richard van der Hoff
2a13af23bc Use run_in_background in preference to preserve_fn 
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
 2018-04-27 12:55:51 +01:00
Richard van der Hoff
9255a6cb17 Improve exception handling for background processes 
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
 2018-04-27 11:07:40 +01:00
Neil Johnson
617bf40924 Generate user daily stats 2018-04-25 17:37:29 +01:00
Adrian Tschira
36c59ce669 Use six.itervalues in some places 
There's more where that came from

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-15 20:39:43 +02:00
Richard van der Hoff
87478c5a60
Merge pull request #3061 from NotAFile/add-some-byte-strings 
Add b prefixes to some strings that are bytes in py3
 2018-04-09 23:54:05 +01:00
Richard van der Hoff
15e8ed874f more verbosity in synctl 2018-04-06 09:28:36 +01:00
Neil Johnson
68b0ee4e8d
Merge pull request #3041 from matrix-org/r30_stats 
R30 stats
 2018-04-05 15:37:37 +01:00
Neil Johnson
0e5f479fc0 Review comments 
Use iteritems over item to loop over dict
formatting
 2018-04-05 12:16:46 +01:00
Jan Christian Grünhage
e4570c53dd phone home cache size configurations 2018-04-04 16:46:58 +01:00
Adrian Tschira
6168351877 Add b prefixes to some strings that are bytes in py3 
This has no effect on python2

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-04 13:48:51 +02:00
Richard van der Hoff
725a72ec5a
Merge pull request #3000 from NotAFile/change-except-style 
Replace old style error catching with 'as' keyword
 2018-04-04 10:45:22 +01:00
Neil Johnson
86932be2cb Support multi client R30 for psql 2018-03-28 14:36:53 +01:00
Michael Kaye
33f6195d9a Handle review comments 2018-03-28 14:25:25 +01:00
Neil Johnson
792d340572 rename stat to future proof 2018-03-28 12:25:02 +01:00
Michael Kaye
4ceaa7433a As daemonizing will make a new process, defer call to init. 2018-03-28 12:19:01 +01:00
Neil Johnson
a32d2548d9 query and call for r30 stats 2018-03-28 10:39:13 +01:00
Michael Kaye
ef520d8d0e Include coarse CPU and Memory use in stats callbacks. 
This requires the psutil module, and is still opt-in based on the report_stats
config option.
 2018-03-27 17:56:03 +01:00
Matthew Hodgson
8cbbfaefc1 404 correctly on missing paths via NoResource 
fixes https://github.com/matrix-org/synapse/issues/2043 and https://github.com/matrix-org/synapse/issues/2029
 2018-03-23 10:32:50 +00:00
NotAFile
2cc9f76bc3 replace old style error catching with 'as' keyword 
This is both easier to read and compatible with python3 (not that that
matters)

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-03-15 16:11:17 +01:00
Erik Johnston
4f28018c83 Register membership/state servlets in event_creator 2018-03-14 14:30:06 +00:00
Erik Johnston
95cb401ae0
Merge pull request #2978 from matrix-org/erikj/refactor_replication_layer 
Remove ReplicationLayer and user Client/Server directly
 2018-03-13 15:45:08 +00:00
Erik Johnston
cb9f8e527c s/replication_client/federation_client/ 2018-03-13 13:26:52 +00:00
Erik Johnston
d023ecb810 Don't build handlers on workers unnecessarily 2018-03-13 11:08:10 +00:00
Erik Johnston
265b993b8a Split replication layer into two 2018-03-13 10:55:47 +00:00
Erik Johnston
28e973ac11 Calculate push actions on worker 2018-02-28 18:02:30 +00:00
Erik Johnston
493e25d554 Move storage functions for push calculations 
This will allow push actions for an event to be calculated on workers.
 2018-02-27 13:58:16 +00:00
Erik Johnston
8ec2e638be Add event_creator worker 2018-02-07 10:32:32 +00:00
Erik Johnston
24dd73028a Add replication http endpoint for event sending 2018-02-07 10:32:32 +00:00
Richard van der Hoff
25adde9a04 Factor out get_db_conn to HomeServer base class 
This function is identical to all subclasses, so we may as well push it up to
the base class to reduce duplication (and make use of it in the tests)
 2018-01-26 00:56:49 +00:00
Erik Johnston
f4d93ae424 Actually make it work 2018-01-12 10:39:27 +00:00
Erik Johnston
f68e4cf690 Refactor 2018-01-12 10:11:12 +00:00
Erik Johnston
7cd34512d8 When using synctl with workers, don't start the main synapse automatically 2018-01-11 11:37:39 +00:00
Silke
df0f602796 Implement listen_tcp method in remaining workers 
Signed-off-by: Silke <silke@slxh.eu>
 2017-12-18 20:00:42 +01:00
Silke
26cd3f5690 Remove logger argument and do not catch replication listener 
Signed-off-by: Silke <silke@slxh.eu>
 2017-12-18 20:00:42 +01:00
Silke Hofstra
ed48ecc58c Add methods for listening on multiple addresses 
Add listen_tcp and listen_ssl which implement Twisted's reactor.listenTCP
and reactor.listenSSL for multiple addresses.

Signed-off-by: Silke Hofstra <silke@slxh.eu>
 2017-12-17 13:15:48 +01:00
Silke Hofstra
37d1a90025 Allow binds to both :: and 0.0.0.0 
Binding on 0.0.0.0 when :: is specified in the bind_addresses is now allowed.
This causes a warning explaining the behaviour.
Configuration changed to match.

See #2232

Signed-off-by: Silke Hofstra <silke@slxh.eu>
 2017-12-17 13:10:31 +01:00
Richard van der Hoff
795b0849f3 Add a comment which might save some confusion 2017-11-24 00:34:56 +00:00
Richard van der Hoff
7f14f0ae38 Remove dead sync_callback 
This is never used; let's remove it to stop confusing things.
 2017-11-24 00:32:04 +00:00
Richard van der Hoff
68ca864141 Add config option to disable media_repo on main synapse 
... to stop us doing the cache cleanup jobs on the master.
 2017-11-22 16:20:27 +00:00
Richard van der Hoff
e1fd4751de Build MediaRepositoryResource as a homeserver dependency 
This avoids the scenario where we have four different PreviewUrlResources
configured on a single app, each of which have their own caches and cache
clearing jobs.
 2017-11-22 16:19:49 +00:00
David Baker
d4553818a0
Merge pull request #2627 from matrix-org/rav/custom_rest_endpoints 
Add a hook for custom rest endpoints
 2017-11-02 15:18:37 +00:00
Richard van der Hoff
fcdfc911ee Add a hook for custom rest endpoints 
Let the user specify custom modules which can be used for implementing extra
endpoints.
 2017-11-02 14:36:55 +00:00
Richard van der Hoff
6650a07ede Factor out _configure_named_resource 
This was a bit of a code vomit, so let's factor it out to preserve some sanity
 2017-11-02 14:33:37 +00:00
Richard van der Hoff
8b56977b6f Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header 
Front-end proxy: pass through auth header
 2017-10-27 11:01:50 +01:00
Richard van der Hoff
54a2525133 Front-end proxy: pass through auth header 
So that access-token-in-an-auth-header works.
 2017-10-26 18:19:01 +01:00
Richard van der Hoff
0a5866bec9 Support /keys/upload on /r0 as well as /unstable 
(So that we can stop riot relying on it in /unstable)
 2017-10-26 18:18:23 +01:00
Richard van der Hoff
eaaabc6c4f replace 'except:' with 'except Exception:' 
what could possibly go wrong
 2017-10-23 15:52:32 +01:00
Erik Johnston
535cc49f27 Merge pull request #2466 from matrix-org/erikj/groups_merged 
Initial Group Implementation
 2017-10-11 13:20:07 +01:00
Erik Johnston
ea87cb1ba5 Make 'affinity' package optional 2017-10-02 18:03:59 +01:00
Erik Johnston
3fed5bb25f Move quit_with_error 2017-10-02 17:59:34 +01:00
Erik Johnston
b5feaa5a49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged 2017-09-19 11:07:45 +01:00
Richard van der Hoff
692250c6be Fix user_dir startup 
Add missing parameter to _base.start_worker_reactor
 2017-08-16 15:11:29 +01:00
Richard van der Hoff
10d8b701a1 Allow configuration of CPU affinity 
Make it possible to set the CPU affinity in the config file, so that we don't
need to remember to do it manually every time.
 2017-08-15 17:08:28 +01:00
Richard van der Hoff
543c794a76 Factor out common application start 
We have 10 copies of this code, and I don't really want to update each one
separately.
 2017-08-15 17:04:40 +01:00
Erik Johnston
2cc998fed8 Fix replication. And notify 2017-07-20 17:13:18 +01:00
Erik Johnston
d4d12daed9 Include registration and as stores in frontend proxy 2017-07-07 18:36:45 +01:00
Erik Johnston
1fc4a962e4 Add a frontend proxy 2017-07-07 18:19:46 +01:00
Erik Johnston
78cefd78d6 Make workers report to master for user ip updates 2017-06-27 14:58:10 +01:00
Erik Johnston
8177563ebe Fix for workers 2017-06-21 13:57:49 +01:00
Erik Johnston
dfeca6cf40 Merge pull request #2286 from matrix-org/erikj/split_out_user_dir 
Split out user directory to a separate process
 2017-06-16 13:01:19 +01:00
Erik Johnston
6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston
5ddd199870 Typo 2017-06-15 10:49:10 +01:00
Erik Johnston
4b461a6931 Add some more stats 2017-06-15 09:39:39 +01:00
Erik Johnston
617304b2cf Fix phone home stats 2017-06-14 19:47:15 +01:00
Erik Johnston
f87b287291 Merge pull request #2127 from APwhitehat/alreadystarted 
print something legible if synapse already running
 2017-04-27 15:46:53 +01:00
Matthew Wolff
d4040e9e28
Queried CONDITIONAL_REQUIREMENTS 2017-04-18 16:19:48 -05:00
Matthew Wolff
54f9a4cb59
Fixed travis build failure 
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
 2017-04-17 01:38:27 -05:00
Matthew Wolff
8e780b113d
web_server_root documentation fix 
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
 2017-04-17 00:49:11 -05:00
Anant Prakash
c1f52a321d synctl.py: Check if synapse is already running 2017-04-13 18:00:02 +05:30
Erik Johnston
247c736b9b Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl 
Reduce federation replication traffic
 2017-04-12 11:07:13 +01:00
Erik Johnston
9c712a366f Move get_presence_list_* to SlaveStore 2017-04-11 16:07:33 +01:00
Erik Johnston
414522aed5 Move get_interested_parties 2017-04-11 15:33:26 +01:00
Erik Johnston
28a4649785 Remove HTTP replication APIs 2017-04-11 09:52:11 +01:00
Erik Johnston
29574fd5b3 Reduce federation presence replication traffic 
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
 2017-04-10 16:48:30 +01:00
Erik Johnston
449d1297ca Fix up federation SendQueue and document types 2017-04-07 11:48:33 +01:00
Erik Johnston
54f59bd7d4 Merge pull request #2107 from HarHarLinks/patch-1 
fix typo in synctl help
 2017-04-07 09:54:37 +01:00
Erik Johnston
98ce212093 Merge pull request #2103 from matrix-org/erikj/no-double-encode 
Don't double encode replication data
 2017-04-07 09:39:52 +01:00
Kim Brose
8a1137ceab fix typo in synctl help 2017-04-06 17:10:20 +02:00
Erik Johnston
391712a4f9 Comment 2017-04-06 13:35:00 +01:00
Erik Johnston
dbf87282d3 Docs 2017-04-06 13:11:21 +01:00
Erik Johnston
b5cb6347a4 Don't immediately notify the master about users whose syncs have gone away 2017-04-05 13:25:40 +01:00
Erik Johnston
96b9b6c127 Don't double json encode typing replication data 2017-04-05 11:34:20 +01:00
Erik Johnston
f10ce8944b Don't double json encode federation replication data 2017-04-05 11:10:28 +01:00
Erik Johnston
a5c401bd12 Merge pull request #2097 from matrix-org/erikj/repl_tcp_client 
Move to using TCP replication
 2017-04-05 09:36:21 +01:00
Erik Johnston
d1d5362267 Add comment 2017-04-04 16:41:03 +01:00
Erik Johnston
ac66e11f2b Add the appropriate amount of preserve_fn 2017-04-04 15:22:54 +01:00
Erik Johnston
3376f16012 Shuffle and comment synchrotron presence 2017-04-04 11:14:16 +01:00
Erik Johnston
6ce6bbedcb Move where we ack federation 2017-04-04 11:02:44 +01:00
Erik Johnston
62b89daac6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server 2017-04-04 09:46:16 +01:00
Erik Johnston
36c28bc467 Update all the workers and master to use TCP replication 2017-04-03 15:35:52 +01:00
Erik Johnston
8a240e4f9c Merge pull request #2078 from APwhitehat/assertuserfriendly 
add user friendly report of assertion error in synctl.py
 2017-03-31 14:41:49 +01:00
Erik Johnston
3ba2859e0c Add tcp replication listener type and hook it up 2017-03-30 13:31:10 +01:00
Anant Prakash
6cdca71079 synctl.py: wait for synapse to stop before restarting (#2020) 2017-03-29 19:20:13 +01:00
Anant Prakash
305d16d612 add user friendly report of assertion error in synctl.py 
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
 2017-03-29 20:41:39 +05:30
Erik Johnston
48e7697911 Add slave transaction store 2017-03-24 13:05:30 +00:00
Erik Johnston
37a187bfab Merge pull request #2033 from matrix-org/erikj/repl_speed 
Don't send the full event json over replication
 2017-03-21 13:11:15 +00:00
Erik Johnston
aac6d1fc9b PEP8 2017-03-20 13:47:56 +00:00
Richard van der Hoff
067b00d49d Run the reactor with the sentinel logcontext 
This fixes a class of 'Unexpected logcontext' messages, which were happening
because the logcontext was somewhat arbitrarily swapping between the sentinel
and the `run` logcontext.
 2017-03-18 00:07:43 +00:00
Erik Johnston
61f471f779 Don't send the full event json over replication 2017-03-17 15:50:01 +00:00
Erik Johnston
6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Richard van der Hoff
d84bd51e95 Refactor logger config for workers 
- to make it easier to add more config options.
 2017-03-10 15:34:01 +00:00
Mark Haines
3a46280ca3 Add db functions needed for room initial sync to slave 2017-02-13 11:16:53 +00:00
Erik Johnston
4ac363a168 Remove debug logging 2017-01-30 14:10:12 +00:00
Erik Johnston
252b503fc8 Hook device list updates to replication 2017-01-27 14:31:35 +00:00
Erik Johnston
edd6cdfc9a Restore default bind address 2017-01-10 17:21:41 +00:00
Erik Johnston
8a0fddfd73 Remove spurious for..else.. 2017-01-10 16:30:53 +00:00
Johannes Löthberg
702c020e58 Fix check for bind_address 
The empty string is a valid setting for the bind_address option, so
explicitly check for None here instead.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
 2016-12-20 01:37:50 +01:00
Johannes Löthberg
7dfd70fc83 Add support for specifying multiple bind addresses 
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
 2016-12-18 21:51:56 +01:00
Erik Johnston
4d9b5c60f9 Comment 2016-11-23 11:11:41 +00:00
Erik Johnston
4c79a63fd7 Explicit federation ack 2016-11-23 10:40:44 +00:00
Erik Johnston
50934ce460 Comments 2016-11-21 16:55:23 +00:00
Erik Johnston
7c9cdb2245 Store federation stream positions in the database 2016-11-21 11:33:08 +00:00
Erik Johnston
f8ee66250a Handle sending events and device messages over federation 2016-11-17 15:48:04 +00:00
Erik Johnston
ed787cf09e Hook up the send queue and create a federation sender worker 2016-11-16 17:34:44 +00:00
Erik Johnston
24772ba56e Respect use_frozen_dicts option in workers 2016-11-08 11:07:18 +00:00
Paul "LeoNerd" Evans
1cc22da600 Set up the process collector during metrics __init__; that way all split-process workers have it 2016-10-27 18:09:34 +01:00
Paul "LeoNerd" Evans
aac13b1f9a Pass the Metrics group into the process collector instead of having it find its own one; this avoids it needing to import from synapse.metrics 2016-10-27 18:08:15 +01:00
Paul "LeoNerd" Evans
3ae48a1f99 Move the process metrics collector code into its own file 2016-10-19 18:10:24 +01:00
Erik Johnston
a68ade6ed3 Merge pull request #1162 from larroy/master 
Use sys.executable instead of hardcoded python. fixes #1161
 2016-10-14 21:42:55 +01:00
Pedro Larroy
c57f871184 Use sys.executable instead of hardcoded python. fixes #1161 2016-10-08 23:55:20 +02:00
Erik Johnston
748d8fdc7b Reduce DB hits for replication 
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.

This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
 2016-09-23 16:49:21 +01:00
Erik Johnston
87528f0756 Support /initialSync in synchrotron worker 2016-09-21 11:46:28 +01:00
Matthew Hodgson
883df2e983 fix logger for client_reader worker 2016-09-17 14:12:04 +01:00
Erik Johnston
4be85281f9 Enable state caches on workers 2016-09-15 14:31:22 +01:00
Erik Johnston
e0b6db29ed Split out public room list into a worker process 2016-09-14 14:42:51 +01:00
Erik Johnston
44330a21e9 Comment 2016-09-09 17:22:07 +01:00
Mark Haines
7ed5acacf4 Fix up the calls to the notifier for device messages 2016-09-01 18:08:40 +01:00
Erik Johnston
873f870e5a Add new direct message storage functions to slave store 2016-08-30 09:40:32 +01:00
Mark Haines
689f4cb914 Update comment 2016-08-22 16:17:31 +01:00
Mark Haines
c8f9b45bc2 Add usage stats to prometheus monitoring 2016-08-22 15:34:38 +01:00
Erik Johnston
bcbd74dc5b Remove log lines 2016-08-18 15:52:10 +01:00
Erik Johnston
07229bbdae Add appservice worker 2016-08-18 14:59:55 +01:00
Mark Haines
403ecd8a2c Missed a s/federation reader/media repository/ in a log message 2016-08-18 10:26:15 +01:00
Mark Haines
396624864a Add a media repository worker 2016-08-18 09:38:42 +01:00
Erik Johnston
784a2d4f2c Remove broken cache stuff 2016-08-15 11:25:48 +01:00
Erik Johnston
64e7e11853 Implement cache replication stream 2016-08-15 11:16:45 +01:00
Erik Johnston
4e1cebd56f Make synchrotron accept /events 2016-08-12 15:31:44 +01:00
Will Hunt
5b5148b7ec Synced up synchrotron set_state with PresenceHandler set_state 2016-08-11 11:48:30 +01:00
Erik Johnston
24f36469bc Add federation /version API 2016-08-05 16:36:07 +01:00
Erik Johnston
5aa024e501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader 2016-07-29 11:24:56 +01:00
Erik Johnston
ec8b217722 Add destination retry to slave store 2016-07-28 17:35:53 +01:00
Erik Johnston
76b89d0edb Add slace storage functions for public room list 2016-07-28 17:03:40 +01:00
Richard van der Hoff
05e7e5e972 Fix flake8 violation 
Apparently flake8 v3 puts the error on a different line to v2. Easiest way to
make sure that happens is by putting the whole statement on one line :)
 2016-07-26 11:59:08 +01:00
Erik Johnston
aede7248ab Split out a FederationReader process 2016-07-21 17:37:44 +01:00
Erik Johnston
66868119dc Add metrics for psutil derived memory usage 2016-07-20 16:00:21 +01:00
Mark Haines
05f1a4596a Merge branch 'master' into develop 2016-06-23 11:17:48 +01:00
Mark Haines
5cc7564c5c Optionally start or stop workers in synctl. 
Optionally start or stop an individual worker by passing -w with
the path to the worker config.
Optionally start or stop every worker and the main synapse by
passing -a with a path to a directory containing worker configs.

The "-w" is intended to be used to bounce individual workers proceses.
THe "-a" is intended for when you want to restart all the workers
simultaneuously, for example when performing database upgrades.
 2016-06-21 16:38:05 +01:00
Mark Haines
13e334506c Remove the legacy v0 content upload API. 
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
 2016-06-21 11:47:39 +01:00
Mark Haines
8c75040c25 Fix setting gc thresholds in the workers 2016-06-17 11:48:12 +01:00
Mark Haines
a352b68acf Use worker_ prefixes for worker config, use existing support for multiple config files 2016-06-16 17:29:50 +01:00
Mark Haines
364d616792 Access the event_cache_size directly from the server object. 
This means that the workers can override the event_cache_size
directly without clobbering the value in the main synapse config.
 2016-06-16 12:53:15 +01:00
Mark Haines
bde13833cb Access replication_url from the worker config directly 2016-06-16 12:44:40 +01:00
Mark Haines
885ee861f7 Inline the synchrotron and pusher configs into the main config 2016-06-16 11:06:12 +01:00
Mark Haines
7dbb473339 Add function to load config without generating it 
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.

The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.

As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
 2016-06-09 18:50:38 +01:00
Erik Johnston
2d1d1025fa Add gc_threshold to pusher and synchrotron 2016-06-07 16:26:25 +01:00
Erik Johnston
dded389ac1 Allow setting of gc.set_thresholds 2016-06-07 15:45:56 +01:00
Mark Haines
b0932b34cb Merge pull request #845 from matrix-org/markjh/synchrotron_presence 
Fix a KeyError in the synchrotron presence
 2016-06-06 16:52:27 +01:00
Mark Haines
4a5bbb1941 Fix a KeyError in the synchrotron presence 2016-06-06 16:37:12 +01:00
Mark Haines
5ef84da4f1 Yield on the sleeps intended to backoff replication 2016-06-06 16:05:28 +01:00
Mark Haines
ac9716f154 Fix spelling 2016-06-03 18:10:00 +01:00
Mark Haines
8f79084bd4 Add get_presence_list_accepted to the broken caches in synchrotron 2016-06-03 18:03:40 +01:00
Mark Haines
48340e4f13 Clear the list of ongoing syncs on shutdown 2016-06-03 15:02:27 +01:00
Mark Haines
da491e75b2 Appease flake8 2016-06-03 14:56:36 +01:00
Mark Haines
0b3c80a234 Use ClientIpStore to record client ips 2016-06-03 14:55:01 +01:00
Mark Haines
80aade3805 Send updates to the syncing users every ten seconds or immediately if they've just come online 2016-06-03 14:24:19 +01:00
Mark Haines
abb151f3c9 Add a separate process that can handle /sync requests 2016-06-03 11:57:26 +01:00
David Baker
1f31cc37f8 Working unsubscribe links going straight to the HS 
and authed by macaroons that let you delete pushers and nothing else
 2016-06-02 17:21:31 +01:00
Erik Johnston
c39f305067 os.environ requires a string 2016-05-16 17:21:30 +01:00
Erik Johnston
c5c5a7403b Make synctl read a cache factor from config file 2016-05-16 17:01:57 +01:00
Mark Haines
b3f29dc1e5 Manually expire broken caches like the who_forgot_in_room 2016-05-13 17:16:27 +01:00
Mark Haines
f03ddc98ec Use the SlavedAccountDataStore 2016-05-13 17:01:28 +01:00
Mark Haines
206eb9fd94 Shift some of the state_group methods into the SlavedEventStore 2016-05-13 16:58:14 +01:00
David Baker
b5e646a18c Make email notifs work on the pusher synapse 
Plus general bugfix to email notif code
 2016-05-13 11:36:50 +01:00
David Baker
ae1af262f6 Pass through _get_state_group_for_events 2016-05-10 19:18:03 +02:00
David Baker
90afc07f39 StateStore, not EventsStore 2016-05-10 19:10:46 +02:00
David Baker
89b5ef7c4b Cached functions must be accessed through the dict 2016-05-10 19:05:22 +02:00
David Baker
35b6e6d2a8 Pass though _get_state_group_for_events 2016-05-10 18:56:40 +02:00
David Baker
3367e65476 Pass through get_state_groups 2016-05-10 18:53:15 +02:00
David Baker
0c4ccdcb83 Also pass through get_profile_displayname 2016-05-10 18:51:14 +02:00
David Baker
9ef05a12c3 Add date header & message id 2016-05-04 14:52:10 +01:00
David Baker
8cc82aad87 Add db functions used for email to the pusher app 2016-05-04 11:47:59 +01:00
Mark Haines
71df327190 Actually start the pusher daemon 2016-04-26 17:07:09 +01:00
Mark Haines
c9eab73f2a Fix typo in default pusher config 2016-04-26 17:06:18 +01:00
Mark Haines
b80b93ea0f Add a log context to the daemonized pusher 2016-04-26 15:57:28 +01:00
Mark Haines
6df5a6a833 Optionally daemonize the pusher 2016-04-26 15:37:41 +01:00
Mark Haines
9c417c54d4 Add a couple of update methods to the PusherSlaveStore 2016-04-26 10:45:02 +01:00
Mark Haines
f15e9e8de4 Remove the uncomments from the comments 2016-04-25 17:56:24 +01:00
Mark Haines
72e2fafa20 Add a metrics listener and a ssh listener to the pusher 2016-04-25 17:34:25 +01:00
Mark Haines
f22f46f4f9 Move the listenTCP call outside the manhole function 2016-04-25 14:59:21 +01:00
Mark Haines
5905f36f05 Split out setting up the manhole to a separate file 2016-04-22 17:09:15 +01:00
Mark Haines
62607d5452 Merge branch 'develop' into markjh/split_out_site.py 
Conflicts:
	synapse/app/homeserver.py
 2016-04-22 16:26:57 +01:00
Mark Haines
e856036f4c Move SynapseSite to its own file 2016-04-22 16:09:55 +01:00
Mark Haines
9e7aa98c22 Split out create_resource_tree to a separate file 2016-04-22 15:40:51 +01:00
Mark Haines
a3ac837599 Optionally split out the pushers into a separate process 2016-04-21 17:22:37 +01:00
Mark Haines
82d7eea7e3 Move the versionstring code out of app.homeserver into util 2016-04-11 14:57:09 +01:00
Erik Johnston
8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Mark Haines
239badea9b Use syntax that works on both py2.7 and py3 2016-03-07 20:13:10 +00:00
Daniel Wagner-Hall
95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids 
Mark AS users with their AS's ID
 2016-03-03 13:56:28 +00:00
Mark Haines
60a0f81c7a Add a /replication API for extracting the updates that happened on 
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
 2016-03-01 14:49:41 +00:00
Daniel Wagner-Hall
763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston
2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Daniel Wagner-Hall
737c4223ef Host /media/r0 as well as /media/v1 2016-02-05 10:47:46 +00:00
Daniel Wagner-Hall
6a9f1209df Error if macaroon key is missing from config 
Currently we store all access tokens in the DB, and fall back to that
check if we can't validate the macaroon, so our fallback works here, but
for guests, their macaroons don't get persisted, so we don't get to
find them in the database. Each restart, we generate a new ephemeral
key, so guests lose access after each server restart.

I tried to fix up the config stuff to be less insane, but gave up, so
instead I bolt on yet another piece of custom one-off insanity.

Also, add some basic tests for config generation and loading.
 2016-02-05 01:58:23 +00:00
Erik Johnston
f8aae79a72 Simplify get_rooms 2016-02-03 13:24:35 +00:00
Daniel Wagner-Hall
d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston
167d1df699 Merge pull request #534 from matrix-org/erikj/setup 
Add a Homeserver.setup method
 2016-01-28 14:45:36 +00:00
Erik Johnston
7ed2bbeb11 Clean up a bit. Add comment 2016-01-28 14:32:05 +00:00
Erik Johnston
a6477d5933 Remove chdir 2016-01-28 09:19:55 +00:00
Erik Johnston
fa90c180ee Merge branch 'develop' of github.com:matrix-org/synapse into erikj/setup 2016-01-27 17:37:33 +00:00
Erik Johnston
9fda8b5193 Don't turn on profiling 2016-01-26 18:27:23 +00:00
Erik Johnston
87f9477b10 Add a Homeserver.setup method. 
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
 2016-01-26 15:51:06 +00:00
Erik Johnston
9959d9ece8 Remove redundated BaseHomeServer 2016-01-26 13:52:29 +00:00
Erik Johnston
44b4fc5f50 Use compiled regex 2016-01-13 11:47:32 +00:00
Erik Johnston
8f8b884430 Don't log urlencoded access_tokens 2016-01-08 17:48:08 +00:00
Erik Johnston
33bef689c1 Turn on core dumps 2016-01-07 15:34:30 +00:00
Erik Johnston
5727922106 Merge pull request #473 from matrix-org/erikj/ssh_manhole 
Change manhole to use ssh
 2016-01-07 14:36:16 +00:00
Erik Johnston
c9ae1d1ee5 Change manhole to use ssh 2016-01-07 13:59:02 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall
11a974da21 Add /_matrix/versions to report supported versions 2016-01-06 18:08:52 +00:00
Daniel Wagner-Hall
bf32922e5a Log when starting stats reporting 2016-01-06 14:13:34 +00:00
Daniel Wagner-Hall
797691f908 Log on stats scheduling 2016-01-06 14:04:27 +00:00
Daniel Wagner-Hall
7a8ba4c9a0 Actually host r0 and unstable prefixes 2015-12-08 15:26:52 +00:00
Mark Haines
6a5ff5f223 Track the time spent in the database per request. 
and track the number of transactions that request started.
 2015-12-07 17:56:11 +00:00
Mark Haines
d6059bdd2a Fix warnings 2015-12-04 11:34:23 +00:00
Mark Haines
a412b9a465 Run the background updates when starting synapse. 2015-11-10 15:50:58 +00:00
Erik Johnston
fa1cf5ef34 Move static folder into synapse 
This is because otherwise it won't get picked up by python packaging.

This also fixes the problem where the "static" folder was found if
synapse wasn't started from that directory.
 2015-10-26 15:37:44 +00:00
Mark Haines
9020860479 Only turn on the twisted deferred debugging if full_twisted_stacktraces is set in the config 2015-10-13 17:50:44 +01:00
Mark Haines
7639c3d9e5 Bounce all deferreds through the reactor to make debugging easier. 
If all deferreds wait a reactor tick before resolving then there is
always a chance to add an errback to the deferred so that stacktraces
get reported, rather than being discarded.
 2015-10-13 17:13:04 +01:00
Erik Johnston
17c80c8a3d rename schema_prepare to prepare_database 2015-10-13 13:56:22 +01:00
Erik Johnston
ec398af41c Expose error more nicely 2015-10-13 11:43:43 +01:00
Erik Johnston
0a4b7226fc Don't change cwd in synctl 2015-10-01 09:21:36 +01:00
Mark Haines
301141515a Merge pull request #288 from matrix-org/markjh/unused_definitions 
Remove some of the unused definitions from synapse
 2015-09-28 14:22:44 +01:00
Daniel Wagner-Hall
f87a11e0fd Fix restart 2015-09-24 21:59:38 +00:00
Daniel Wagner-Hall
76328b85f6 Allow config file path to be configurable in in synctl 
Also, allow it to be run from directories other than the synapse directory
 2015-09-24 21:50:20 +00:00
Mark Haines
7a5818ed81 Note that GzipFile was removed in comment that referenced it 2015-09-22 18:27:22 +01:00
Mark Haines
184ba0968a synapse/app/homeserver.py:GzipFile was unused 2015-09-22 18:25:30 +01:00
Daniel Wagner-Hall
b6e0303c83 Catch stats-reporting errors 2015-09-22 13:34:29 +01:00
Daniel Wagner-Hall
7213588083 Implement configurable stats reporting 
SYN-287

This requires that HS owners either opt in or out of stats reporting.

When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
 2015-09-22 12:57:40 +01:00
Mark Haines
0b5c9adeb5 Merge pull request #267 from matrix-org/markjh/missing_requirements 
Print an example "pip install" line for a missing requirement
 2015-09-18 18:52:08 +01:00
Daniel Wagner-Hall
2c746382e0 Merge branch 'daniel/insecureclient' into develop 2015-09-09 14:27:30 +01:00
Daniel Wagner-Hall
6485f03d91 Fix random formatting 2015-09-09 13:05:00 +01:00
Daniel Wagner-Hall
81a93ddcc8 Allow configuration to ignore invalid SSL certs 
This will be useful for sytest, and sytest only, hence the aggressive
config key name.
 2015-09-09 12:02:07 +01:00
Mark Haines
73e4ad4b8b Merge branch 'master' into develop 
Conflicts:
	setup.py
 2015-09-07 15:06:46 +01:00
Erik Johnston
1002bbd732 Change log level to info 2015-09-03 09:51:01 +01:00
Mark Haines
3cdfd37d95 Print an example "pip install" line for a missing requirement 2015-09-01 16:47:26 +01:00
Matthew Hodgson
d71af2ee12 don't log the whole DB config (including postgres password...) 2015-08-29 22:23:21 +01:00
Erik Johnston
d33f31d741 Print the correct pip install line when failing due to lack of matrix-angular-sdk 2015-08-25 15:33:23 +01:00
Erik Johnston
37403ab06c Update the log message 2015-08-25 14:19:09 +01:00
Erik Johnston
d9088c923f Remove dependency on matrix-angular-sdk 2015-08-25 13:34:50 +01:00
Erik Johnston
86cef6a91b Allow specifying a directory to host a web client from 2015-08-25 12:01:23 +01:00
Paul "LeoNerd" Evans
efe60d5e8c Only print the pidfile path on startup if requested by a commandline flag 2015-08-07 16:36:42 +01:00
Erik Johnston
0e58d19163 Merge pull request #187 from matrix-org/erikj/sanitize_logging 
Sanitize logging
 2015-06-19 11:35:59 +01:00
Erik Johnston
eb928c9f52 Add site_tag to logger 2015-06-19 10:16:48 +01:00
Eric Myhre
9e5a353663 Make upload dir a configurable path. 
Fixes SYN-425.

Signed-off-by: Eric Myhre <hash@exultant.us>
 2015-06-18 23:38:20 -05:00
Erik Johnston
aaa749d366 Disable twisted access logging. Move access logging to SynapseRequest object 2015-06-15 18:18:05 +01:00
Erik Johnston
cee69441d3 Log more when we have processed the request 2015-06-15 17:11:44 +01:00
Erik Johnston
b5209c5744 Create SynapseRequest that overrides __repr__ to not print access_token 2015-06-15 16:37:04 +01:00
Erik Johnston
a005b7269a Add backwards compat support for metrics, manhole and webclient config options 2015-06-12 17:44:23 +01:00
Erik Johnston
942e39e87c PEP8 2015-06-12 17:13:54 +01:00
Erik Johnston
9c5fc81c2d Correctly handle x_forwaded listener option 2015-06-12 17:13:23 +01:00
Erik Johnston
fd2c07bfed Use config.listeners 2015-06-12 15:33:07 +01:00
Erik Johnston
c42ed47660 Fix up create_resource_tree 2015-06-12 11:52:52 +01:00
Erik Johnston
522f285f9b Add config option to disable compression of http responses 2015-06-01 13:36:30 +01:00
Erik Johnston
a7b65bdedf Add config option to turn off freezing events. Use new encode_json api and ujson.loads 2015-05-29 12:17:33 +01:00
Erik Johnston
1ce1509989 s/metric_interface/metric_bind_host/ 2015-05-22 14:51:22 +01:00
Erik Johnston
8bb85c8c5a Update log line 2015-05-22 14:48:06 +01:00
Erik Johnston
59a0682f3e Enable changing the interface the metrics listener binds to 2015-05-22 13:13:07 +01:00
David Baker
19505e0392 Disable GZip encoding on static file resources as per comment 2015-05-19 13:20:25 +01:00
Erik Johnston
677be13ffc Revert accidental commit 2015-05-19 13:12:28 +01:00
Erik Johnston
2b7120e233 SYN-383: Handle the fact the server might not have signed things 2015-05-19 12:49:38 +01:00
David Baker
6e1ad283cf Support gzip encoding for client, client v2 and web client resources (SYN-176). 2015-05-14 16:39:19 +01:00
Mark Haines
4a7a4a5b6c Optional profiling using cProfile 2015-05-06 17:08:00 +01:00
Mark Haines
b994fb2b96 Don't read from the config file before checking it exists 2015-05-06 12:56:47 +01:00
Mark Haines
6ea9cf58be missing import 2015-04-30 17:21:21 +01:00
Mark Haines
c95480963e read the pid_file from the config file in synctl 2015-04-30 17:12:15 +01:00
Mark Haines
2d4d2bbae4 Merge branch 'develop' into markjh/config_cleanup 
Conflicts:
	synapse/config/captcha.py
 2015-04-30 16:54:55 +01:00
David Baker
054aa0d58c Do access log using python's logging stuff, just under a separate logger name 2015-04-30 16:21:38 +01:00
David Baker
d89a9f7283 Add an access_log 
SYN-161 #resolve
 2015-04-30 13:58:13 +01:00
Mark Haines
6b69ddd17a remove duplicate parse_size method 2015-04-30 04:26:29 +01:00
Mark Haines
d624e2a638 Manually generate the default config yaml, remove most of the commandline arguments for synapse anticipating that people will use the yaml instead. Simpify implementing config options by not requiring the classes to hit the super class 2015-04-30 04:24:44 +01:00
Erik Johnston
e7a6edb0ee Revert previous 2015-04-29 18:37:30 +01:00
Erik Johnston
c27d6ad6b5 Only start pushers when synapse has fully started 2015-04-29 18:25:24 +01:00
Mark Haines
f2d90d5c02 Fix whitespace 2015-04-29 14:53:23 +01:00
Mark Haines
845b0b2c97 Check requirements before doing anything else when running the homeserver 2015-04-29 14:52:42 +01:00
Erik Johnston
64991b0c8b Merge pull request #129 from matrix-org/key_distribution 
Key distribution v2
 2015-04-29 13:34:38 +01:00
Mark Haines
4ad8b45155 Merge branch 'develop' into key_distribution 
Conflicts:
	synapse/config/homeserver.py
 2015-04-29 13:15:14 +01:00
Erik Johnston
cd0864121b Make postgres database error slightly more helpful 2015-04-29 12:12:25 +01:00
Erik Johnston
204132a998 Check that postgres database has correct charset set 2015-04-29 11:42:28 +01:00
Erik Johnston
073b891ec1 Remove unused imports 2015-04-28 13:44:23 +01:00
Erik Johnston
327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston
42b7139dec Remove unused import 2015-04-27 15:59:56 +01:00
Erik Johnston
1ef66cc3bd Move database configuration into config module 2015-04-27 15:57:43 +01:00
Erik Johnston
416a3e6c4f Ensure check_same_thread is enabled for sqlite3 2015-04-27 15:44:30 +01:00
David Baker
df75914791 pep8 2015-04-27 11:48:33 +01:00
David Baker
b02e1006b9 Run database check before daemonizing, at the cost of database hygiene. 2015-04-27 11:46:00 +01:00
David Baker
f8152f2708 rename db method to be more informative 2015-04-27 10:16:26 +01:00
David Baker
a7b51f4539 Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266 2015-04-24 18:11:21 +01:00
Erik Johnston
ffad75bd62 Remove mysql/maria support 2015-04-15 17:00:50 +01:00
Erik Johnston
6606f7c659 Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-15 10:27:20 +01:00
Mark Haines
d488463fa3 Add a version 2 of the key server api 2015-04-14 16:04:52 +01:00
Erik Johnston
58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Mark Haines
3cbc286d06 Move server key api into rest/key/v1 2015-04-14 13:28:11 +01:00
Erik Johnston
c5365dee56 Use case sensitive collations 2015-04-10 18:46:33 +01:00
Erik Johnston
4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston
0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00
Erik Johnston
d0e444a648 Explicitly name the __main__ module logger 2015-04-07 12:04:02 +01:00
Erik Johnston
779f7b0f44 Fix unicode support 2015-04-02 10:06:22 +01:00
Erik Johnston
e24c32e6f3 Fix SQLite support 2015-04-01 15:09:51 +01:00
Erik Johnston
9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal
3470cb36a8 Pyflakes 2015-03-31 13:03:31 +01:00
Kegan Dougal
e7887e37a8 Remove appservice REST servlets 2015-03-31 09:32:40 +01:00
Erik Johnston
0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
Erik Johnston
f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston
455579ca90 Make database selection configurable 2015-03-20 10:55:55 +00:00
David Baker
7564dac8cb Wire up the webclient option 
It existed but was hardcoded to True.
Give it an underscore for consistency.
Also don't pull in syweb unless we're actually using the web client.
 2015-03-17 12:45:37 +01:00
Matthew Hodgson
c8032aec17 actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0' 2015-03-14 00:12:20 +00:00
Paul "LeoNerd" Evans
b98b4c135d Option to serve metrics from their own localhost-only TCP port instead of muxed on the main listener 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
a2cdd11d4a Fold the slightly-odd bind_port/secure_port/etc.. logic into SynapseHomeServer.start_listening() 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
e0214a263b Build MetricsResource as a specific HomeServer dependency 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
e07fc62833 A trivial 'hello world'-style resource on /_synapse/metrics, with optional commandline flag 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
211c14c391 No need to explicitly pass 'web_client' in to create_resource_tree as it can be found via config 2015-03-12 16:24:50 +00:00
Erik Johnston
045afd6b61 in_thread takes no arguments 2015-03-10 10:19:03 +00:00
Erik Johnston
c37eceeb9e Split out the 'run' from 'setup' 2015-03-10 09:58:33 +00:00
Erik Johnston
b8a6692657 Add documentation. When starting via twistd respect soft_file_limit config option. 2015-03-10 09:39:42 +00:00
Erik Johnston
f31e65ca8b Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf 2015-03-09 13:29:41 +00:00
Erik Johnston
c54773473f Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-03-06 14:23:41 +00:00
Erik Johnston
82b34e813d SYN-67: Finish up implementing new database schema management 2015-03-04 12:04:19 +00:00
Erik Johnston
3fce185c77 Merge pull request #83 from matrix-org/nofile_limit_config 
Add config option to set the soft fd limit on start
 2015-03-02 13:52:16 +00:00
Kegan Dougal
28d8614f48 Trailing comma 2015-02-23 17:36:37 +00:00
Kegan Dougal
0696dfd94b Actually treat this as static content, not random Resources. 2015-02-23 15:35:09 +00:00
Kegan Dougal
22399d3d8f Add RegisterFallbackResource to /_matrix/static/client/register 
Try to keep both forms of registration logic (native/fallback) close
together for sanity.
 2015-02-23 15:14:56 +00:00
Erik Johnston
1f29fafc95 Don't exit if we can't work out if we're running in a git repo 2015-02-21 13:44:46 +00:00
Erik Johnston
7c56210f20 By default set soft limit to hard limit 2015-02-20 16:09:44 +00:00
Erik Johnston
939273c4b0 Rename resource variable so as to not shadow module import 2015-02-19 11:53:13 +00:00
Erik Johnston
c3eb7dd9c5 Add config option to set the soft fd limit on start 2015-02-19 11:50:49 +00:00
Erik Johnston
6311ae8968 Conform to header spec take two 2015-02-18 17:34:26 +00:00
Erik Johnston
b6771037a6 Make version_string conform to User-Agent and Server spec 2015-02-18 17:31:50 +00:00
Erik Johnston
1df8bad63e pyflakes 2015-02-18 16:54:25 +00:00
Erik Johnston
aa577df064 When computing git version run git commands in same dir as source files 2015-02-18 16:52:04 +00:00
Erik Johnston
d122e215ff Generate a version string that includes git details if run from git checkout 2015-02-18 16:52:04 +00:00
Erik Johnston
2635d4e634 Merge branch 'develop' of github.com:matrix-org/synapse into develop 2015-02-18 10:29:54 +00:00
Erik Johnston
ec847059f3 Rename _fail_fetch_pdu_cache to _get_pdu_cache 2015-02-18 10:14:10 +00:00
Erik Johnston
1177245e86 Merge branch 'hotfixes-v0.7.0g' of github.com:matrix-org/synapse into develop 2015-02-17 17:30:11 +00:00
Erik Johnston
8b919c00f3 Start the get_pdu cache 2015-02-17 15:44:01 +00:00
Erik Johnston
ea1d6c16cd Don't write bytecode 2015-02-17 10:54:06 +00:00
Mark Haines
0d872f5aa6 Merge pull request #50 from matrix-org/application-services 
Application Services
 2015-02-13 15:06:14 +00:00
Mark Haines
183b3d4e47 Prepare the database whenever a connection is opened from the db_pool so that in-memory databases will work 2015-02-13 14:38:24 +00:00
Erik Johnston
5758dafb4e Merge branch 'develop' of github.com:matrix-org/synapse into state-chache 2015-02-11 10:35:13 +00:00
Kegan Dougal
9978c5c103 Merge branch 'develop' into application-services 2015-02-11 10:03:24 +00:00
Mark Haines
84a769cdb7 Fix code-style 2015-02-10 17:58:36 +00:00
Erik Johnston
c86ebe7673 Merge branch 'develop' of github.com:matrix-org/synapse into state-chache 
Conflicts:
	synapse/app/homeserver.py
	synapse/state.py
 2015-02-10 11:04:37 +00:00
Kegan Dougal
53557fc532 Merge branch 'develop' into application-services 2015-02-09 15:20:56 +00:00
Erik Johnston
75656712e3 Time how long we're spending on the database thread 2015-02-09 14:22:52 +00:00
Erik Johnston
24cc6979fb Log when we receive a request, when we send a response and how long it took to process it. 2015-02-09 13:46:22 +00:00
Erik Johnston
5bf318e9a6 Bug fixes. 2015-02-06 16:52:22 +00:00
Kegan Dougal
c059c9fea5 Merge branch 'develop' into application-services 
Conflicts:
	synapse/handlers/__init__.py
	synapse/storage/__init__.py
 2015-02-02 15:57:59 +00:00
David Baker
c291a4d522 Merge branch 'develop' into pushers 
Conflicts:
	synapse/handlers/events.py
	synapse/server.py
 2015-01-28 12:51:05 +00:00
Kegan Dougal
fa8e6ff900 Add stub application services REST API. 2015-01-27 14:01:51 +00:00
Mark Haines
7b814d3f7f Add client v2_alpha resource to synapse server resource tree 2015-01-23 18:55:19 +00:00
Mark Haines
7256def8e4 Merge rest servlets into the client json resource object 2015-01-23 10:37:38 +00:00
David Baker
5c6189ea3e Merge branch 'develop' into pushers 
Conflicts:
	synapse/rest/__init__.py
 2015-01-22 17:46:16 +00:00
Mark Haines
97c68c508d Move rest APIs back under the rest directory 2015-01-22 16:10:07 +00:00
Erik Johnston
7f058c5ff7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf 
Conflicts:
	synapse/app/homeserver.py
 2015-01-22 13:35:34 +00:00
Mark Haines
42529cbced Fix pyflakes errors 2015-01-19 15:33:04 +00:00
Mark Haines
00e9c08609 Fix syntax 2015-01-19 15:30:48 +00:00
Mark Haines
3e85e52b3f Allow ':memory:' as the database path for sqlite3 2015-01-19 15:26:19 +00:00
David Baker
c06a9063e1 Merge branch 'develop' into pushers 2015-01-13 13:15:51 +00:00
Mark Haines
63403aa7a5 Check the existance and versions of necessary modules when starting synapse, log which modules are used 2015-01-08 17:08:57 +00:00
Erik Johnston
42507b0011 Log server version on startup 2015-01-07 17:25:28 +00:00
Erik Johnston
82be4457de Add twisted Service interface 2015-01-07 13:46:37 +00:00
Mark Haines
5cf45c4319 Merge branch 'master' into develop 2015-01-06 19:48:53 +00:00
Mark Haines
adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Matthew Hodgson
407c299828 improve error msg 2014-12-24 17:50:42 +00:00
David Baker
b56730bb6e Merge branch 'develop' into pushers 
Conflicts:
	synapse/api/errors.py
	synapse/server.py
	synapse/storage/__init__.py
 2014-12-18 15:15:22 +00:00
Erik Johnston
2e44714214 Make failure to run appropraite upgrade scripts more helpful. 2014-12-16 14:20:32 +00:00
Mark Haines
5da65085d1 Get uploads working with new media repo 2014-12-02 19:51:47 +00:00
David Baker
58f82e2e54 Merge branch 'develop' into pushers 2014-11-20 18:25:31 +00:00
David Baker
23465a30b6 Merge branch 'develop' into pushers 2014-11-20 18:17:46 +00:00
David Baker
f1c7f8e813 Merge branch 'develop' into http_client_refactor 2014-11-20 17:49:48 +00:00
David Baker
e377d33652 Separate out the matrix http client completely because just about all of its code it now separate from the simple case we need for standard HTTP(S) 2014-11-20 17:41:56 +00:00
Mark Haines
db9ce032a4 Fix pep8 codestyle warnings 2014-11-20 17:26:36 +00:00
Mark Haines
dfdda2c871 Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str' 2014-11-20 17:10:37 +00:00
David Baker
ebf6c08a47 Merge branch 'http_client_refactor' into pushers 2014-11-20 14:01:41 +00:00
David Baker
dc60eee50e Refactor the HTTP clients a little. 2014-11-20 13:53:34 +00:00
David Baker
74c3879760 Start creating a module to do generic notifications (just prints them to stdout currently!) 2014-11-19 18:20:59 +00:00
Erik Johnston
c6ea29d916 Revert accidental commit of bad file 2014-11-18 15:57:00 +00:00
Erik Johnston
95614e5220 Fix auth to correctly handle initial creation of rooms 2014-11-18 15:36:41 +00:00
Mark Haines
26fc878944 Stop before starting when restarting 2014-11-17 19:16:15 +00:00
Mark Haines
e903c941cb Merge branch 'develop' into request_logging 
Conflicts:
	setup.py
	synapse/storage/_base.py
	synapse/util/async.py
 2014-11-14 11:16:50 +00:00
Mark Haines
e7c6d2c9d9 SYN-138: Rewrite synctl in python and include it in the python distribution 2014-11-13 14:39:30 +00:00
Erik Johnston
3cb678f84c Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization 2014-11-07 10:55:28 +00:00
Mark Haines
dca3ba2f77 Determine webclient path by the python module it is contained in 2014-11-04 16:19:16 +00:00
Erik Johnston
af7ae048f8 Add option to not bind to HTTPS port. This is useful if running behind an ssl load balancer 2014-11-03 15:06:40 +00:00
Mark Haines
7a756e5d9d Remove unused 'context' variables to appease pyflakes 2014-10-30 11:15:39 +00:00
Mark Haines
7c06399512 Merge branch 'develop' into request_logging 
Conflicts:
	synapse/config/logger.py
 2014-10-30 11:13:58 +00:00
Mark Haines
7d709542ca Fix pep8 warnings 2014-10-30 11:10:17 +00:00
Mark Haines
b29517bd01 Add a request-id to each log line 2014-10-30 01:21:33 +00:00
Mark Haines
984e207b59 Merge branch develop into server2server_signing 
Conflicts:
	synapse/app/homeserver.py
 2014-10-13 10:58:50 +01:00
Mark Haines
4f11518934 Split PlainHttpClient into separate clients for talking to Identity servers and talking to Capatcha servers 2014-10-02 14:03:26 +01:00
Mark Haines
e3117a2a23 Add a _matrix/key/v1 resource with the verification keys of the local server 2014-09-23 18:43:34 +01:00
Paul "LeoNerd" Evans
55397f6347 prepare_database() on db_conn, not plain name, so we can pass in the connection from outside 2014-09-10 16:23:58 +01:00
Paul "LeoNerd" Evans
2faffc52ee Make sure not to open our TCP ports until /after/ the DB is nicely prepared ready for use 2014-09-10 16:16:24 +01:00
Paul "LeoNerd" Evans
6c1f0055dc No need for a tiny run() function any more, just use reactor.run() directly 2014-09-10 16:07:44 +01:00
Paul "LeoNerd" Evans
ce55a8cc4b Move database preparing code out of homserver.py into storage where it belongs 2014-09-10 15:42:15 +01:00
Erik Johnston
9dd4570b68 Generate m.room.aliases event when the HS creates a room alias 2014-09-05 21:35:56 +01:00
Matthew Hodgson
8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans
756e171ad0 Store SQL DDL deltas as well; attempt to upgrade the database on startup if it's too old 2014-09-03 14:14:39 +01:00
Erik Johnston
b50ea730b1 Merge branch 'develop' of github.com:matrix-org/synapse into develop 
Conflicts:
	synapse/http/server.py
        synapse/http/content_repository.py
 2014-09-03 12:01:14 +01:00
Erik Johnston
bc21350298 Add option to change content repo location 2014-09-03 11:57:23 +01:00
Mark Haines
bf10cf5f1a move contentrepo class to it's own file 2014-09-03 11:10:44 +01:00
Mark Haines
c7a7cdf734 Add ratelimiting function to basehandler 2014-09-02 17:57:04 +01:00
Mark Haines
399e004884 Add unsecure listener port to homeserver 2014-09-01 22:38:52 +01:00
Erik Johnston
bcfaaf7da6 That was a breaking db change. You need to recreate the databases. (In reality, it's enough to just run the im.sql through your db and change the schema version) 2014-09-01 20:34:07 +01:00
Mark Haines
a53946a8a1 Enable SSL for s2s http client 2014-09-01 18:30:00 +01:00
Mark Haines
ef6a8e4f32 Listen using SSL 2014-09-01 16:30:43 +01:00
Mark Haines
9ea1de432d Fix homeserver config parsing 2014-09-01 15:51:15 +01:00
Mark Haines
d9ebe531ed Add config tree to synapse. Add support for reading config from a file 2014-08-31 16:06:39 +01:00
Mark Haines
1d95e78759 Merge branch 'develop' into storage_transactions 2014-08-27 16:54:12 +01:00
Mark Haines
46a2f6a816 Remove call to get_federation from homeserver 2014-08-27 14:36:20 +01:00
Paul "LeoNerd" Evans
05fa81fee4 A reliable logger.info() message /after/ the TCP port has been opened and is listening; this is essential for avoiding races in wrapper scripts e.g. integration testing 2014-08-27 13:08:55 +01:00
Paul "LeoNerd" Evans
e677a3114e Use SQLite's PRAGMA user_version to check if the database file really matches the schema we have in mind 2014-08-27 11:45:16 +01:00
Paul "LeoNerd" Evans
648796ef1d Neater database setup at application startup time; only .connect() it once, not once per schema file; don't build the db_pool twice 2014-08-27 11:45:16 +01:00
Erik Johnston
1291ac93f3 Add the ability to turn on the twisted manhole telnet service. 2014-08-26 13:43:55 +01:00
Erik Johnston
0c3b4a1f63 For the content repo, don't just use homeserver.hostname as that might not include the port due to SRV. 2014-08-24 11:56:55 +01:00
Erik Johnston
acf5127604 Make the content repo work with in daemon mode. Return the full url on upload. Update the webclient to use new content repo api. 2014-08-22 10:25:32 +01:00
Matthew Hodgson
1b0d427285 host a webclient by default 2014-08-21 23:35:45 +01:00
Kegan Dougal
58548ab557 Implemented GETs for the ContentRepoResource. It all actually appears to be working. 2014-08-18 17:18:54 +01:00
Kegan Dougal
35da1bf4a3 Auth content uploads. Added a mapping function from request > filename. Added exception handling for content uploads. webclient: Only prefix the client API path on doRequest, not doBaseRequest (this would've broken the identity server auth too). Added matrixService.uploadContent. May not require mFileUpload anymore. 2014-08-18 17:18:54 +01:00
Kegan Dougal
a18b1a649c Added /matrix/content path, HS resource_for_content_repo attribute and FileUploadResource. Added stub methods. 2014-08-18 17:18:54 +01:00
Erik Johnston
e4061383b8 Change relative db paths to absolute paths in case we daemonize. 2014-08-14 14:07:22 +01:00
Paul "LeoNerd" Evans
e37de2aef3 chmod +x homeserver.py 2014-08-14 14:05:05 +01:00
Kegan Dougal
d253a35539 Added web client prefix 2014-08-14 11:54:37 +01:00
Kegan Dougal
c75add6ec8 Added a urls module for keeping client and federation prefixes. 2014-08-14 11:52:56 +01:00
Kegan Dougal
9fd445eb92 If the web client is enabled, automatically redirect root '/' to the web client path. 2014-08-14 11:37:13 +01:00
Kegan Dougal
e543d6a91d Fixed dynamic resource mapping to clobber dummy Resources with the actual desired Resource in the event of a collision (as is the case for '/matrix/client' and '/matrix/client/api/v1') 2014-08-14 11:18:18 +01:00
Kegan Dougal
de65c34fcf Honour the -w flag to enable the web client at /matrix/client 2014-08-14 10:24:17 +01:00
Kegan Dougal
9a1638ed21 Removed http_server from HomeServer. Updated unit tests to use either resource_for_federation or resource_for_client depending on what is being tested. 2014-08-14 10:18:54 +01:00
Kegan Dougal
29aa13f0d4 Make federation use resource_for_federation as well. 2014-08-14 10:05:06 +01:00
Kegan Dougal
9f863d3466 Start phasing out HttpServer: we should be using Resources instead. Added resource_for_client/federation/web_client to the HomeServer and hooked the C-S servlets to operate on resource_for_client. Dynamically construct the Resource tree. 2014-08-14 09:55:16 +01:00
Kegan Dougal
2da1e1526e Don't pass host_web_client flag to register_servlets, it needs to be a Resource not a RestServlet. 2014-08-13 15:57:58 +01:00
Kegan Dougal
38c7e92331 Merge branch 'master' of github.com:matrix-org/synapse 2014-08-13 15:53:44 +01:00
Paul "LeoNerd" Evans
b370f9903c Fix logging verbosity parsing code 2014-08-13 15:14:22 +01:00
Kegan Dougal
5a59da8a94 Merge branch 'master' of github.com:matrix-org/synapse 2014-08-13 13:50:05 +01:00
Kegan Dougal
7f40fa1d46 Added a -w flag which will host the web client if specified. Currently this just delegates to the webclient RestServlet. 2014-08-13 13:50:01 +01:00
Matthew Hodgson
2a0f7541c7 verbose by default please 2014-08-13 12:06:58 +01:00
Matthew Hodgson
f98e6380f1 add in whitespace after copyright statements to improve legibility 2014-08-13 03:14:34 +01:00
matrix.org
4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00