Mark Haines
774f3a692c
Merge pull request #889 from matrix-org/markjh/synctl_workers
...
Optionally start or stop workers in synctl.
2016-06-21 17:58:19 +01:00
Mark Haines
5cc7564c5c
Optionally start or stop workers in synctl.
...
Optionally start or stop an individual worker by passing -w with
the path to the worker config.
Optionally start or stop every worker and the main synapse by
passing -a with a path to a directory containing worker configs.
The "-w" is intended to be used to bounce individual workers proceses.
THe "-a" is intended for when you want to restart all the workers
simultaneuously, for example when performing database upgrades.
2016-06-21 16:38:05 +01:00
Mark Haines
0fe0b0eeb6
Merge pull request #888 from matrix-org/markjh/content_repo
...
Remove the legacy v0 content upload API.
2016-06-21 14:01:01 +01:00
Mark Haines
13e334506c
Remove the legacy v0 content upload API.
...
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
2016-06-21 11:47:39 +01:00
Mark Haines
d5fb561709
Optionally make committing to postgres asynchronous.
...
Useful when running tests when you don't care whether the server
will lose data that it claims that it has committed.
2016-06-20 17:53:38 +01:00
Erik Johnston
d8ec81cc31
Merge pull request #879 from matrix-org/erikj/linearize_fed_server
...
Linearize some federation endpoints based on (origin, room_id)
2016-06-20 17:34:29 +01:00
Erik Johnston
bc72d381b2
Merge branch 'release-v0.16.1' of github.com:matrix-org/synapse
2016-06-20 14:18:04 +01:00
Erik Johnston
4d362a61ea
Bump version and changelog
2016-06-20 14:17:42 +01:00
Mark Haines
41e4b2efea
Add the create_profile method back since the tests use it
2016-06-17 19:20:47 +01:00
Mark Haines
0c13d45522
Add a comment on why we don't create a profile for upgrading users
2016-06-17 19:18:53 +01:00
Mark Haines
9f1800fba8
Remove registered_users from the distributor.
...
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.
This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
2016-06-17 19:14:16 +01:00
Erik Johnston
8f4a9bbc16
Linearize some federation endpoints based on (origin, room_id)
2016-06-17 16:43:45 +01:00
Erik Johnston
9ba2bf1570
Merge pull request #878 from matrix-org/erikj/ujson
...
Disable responding with canonical json for federation
2016-06-17 16:22:12 +01:00
Erik Johnston
120c238705
Disable responding with canonical json for federation
2016-06-17 16:10:37 +01:00
Erik Johnston
1c1f633b13
Merge pull request #877 from matrix-org/erikj/frozen_default
...
Turn use_frozen_events off by default
2016-06-17 15:33:55 +01:00
Erik Johnston
6660f37558
Merge pull request #876 from matrix-org/erikj/sign_own
...
Only re-sign our own events
2016-06-17 15:23:20 +01:00
Erik Johnston
3e41de05cc
Turn use_frozen_events off by default
2016-06-17 15:11:22 +01:00
Erik Johnston
2884712ca7
Only re-sign our own events
2016-06-17 14:47:33 +01:00
Mark Haines
ded01c3bf6
Fix `KeyError: 'msgtype'
. Use
.get
`
...
Fixes a key error where the mailer tried to get the ``msgtype`` of an
event that was missing a ``msgtype``.
```
File "synapse/push/mailer.py", line 264, in get_notif_vars
File "synapse/push/mailer.py", line 285, in get_message_vars
File ".../frozendict/__init__.py", line 10, in __getitem__
return self.__dict[key]
KeyError: 'msgtype'
```
2016-06-17 13:49:16 +01:00
Mark Haines
8c75040c25
Fix setting gc thresholds in the workers
2016-06-17 11:48:12 +01:00
Mark Haines
a352b68acf
Use worker_ prefixes for worker config, use existing support for multiple config files
2016-06-16 17:29:50 +01:00
Mark Haines
364d616792
Access the event_cache_size directly from the server object.
...
This means that the workers can override the event_cache_size
directly without clobbering the value in the main synapse config.
2016-06-16 12:53:15 +01:00
Mark Haines
bde13833cb
Access replication_url from the worker config directly
2016-06-16 12:44:40 +01:00
Mark Haines
80a1bc7db5
Comment on what's going on in clobber_with_worker_config
2016-06-16 11:29:45 +01:00
Mark Haines
f1f70bf4b5
Merge remote-tracking branch 'origin/develop' into markjh/worker_config
2016-06-16 11:20:17 +01:00
Mark Haines
dbb5a39b64
Add worker config module
2016-06-16 11:09:15 +01:00
Mark Haines
885ee861f7
Inline the synchrotron and pusher configs into the main config
2016-06-16 11:06:12 +01:00
Erik Johnston
486b9a6a2d
Merge pull request #873 from vt0r/bugfix/bcrypt-utf8-encode
...
Fix TypeError in call to bcrypt.hashpw
2016-06-16 10:12:17 +01:00
Salvatore LaMendola
ed5f43a55a
Fix TypeError in call to bcrypt.hashpw
...
- At the very least, this TypeError caused logins to fail on my own
running instance of Synapse, and the simple (explicit) UTF-8
conversion resolved login errors for me.
Signed-off-by: Salvatore LaMendola <salvatore.lamendola@gmail.com>
2016-06-16 00:43:42 -04:00
Erik Johnston
09a17f965c
Line lengths
2016-06-15 16:58:12 +01:00
Erik Johnston
1e9026e484
Handle floats as img widths
2016-06-15 16:58:05 +01:00
Erik Johnston
a60169ea09
Handle og props with not content
2016-06-15 16:57:48 +01:00
Erik Johnston
0ef0655b83
Bump version and changelog
2016-06-15 15:50:48 +01:00
Erik Johnston
d41a1a91d3
Linearize fetching of gaps on incoming events
...
This potentially stops the server from doing multiple requests for the
same data.
2016-06-15 15:16:14 +01:00
Richard van der Hoff
15bf3e3376
Merge pull request #870 from matrix-org/rav/work_around_tls_bug
...
Work around TLS bug in twisted
2016-06-15 13:28:13 +01:00
Erik Johnston
b31c49d676
Correctly mark backfilled events as backfilled
2016-06-15 10:59:08 +01:00
Richard van der Hoff
255c229f23
Work around TLS bug in twisted
...
Wrap up twisted's FileBodyProducer to work around
https://twistedmatrix.com/trac/ticket/8473 . Hopefully this fixes
https://matrix.org/jira/browse/SYN-700 .
2016-06-15 10:39:08 +01:00
Erik Johnston
d12134ce37
Merge pull request #868 from matrix-org/erikj/invalid_id
...
Make get_domain_from_id throw SynapseError on invalid ID
2016-06-14 15:09:07 +01:00
Erik Johnston
36e2aade87
Make get_domain_from_id throw SynapseError on invalid ID
2016-06-14 13:25:29 +01:00
Matthew Hodgson
33546b58aa
point to the CAPTCHA docs
2016-06-12 23:11:29 +01:00
Mark Haines
41b35412bf
Merge pull request #863 from matrix-org/markjh/load_config
...
Add function to load config without generating it
2016-06-10 11:12:32 +01:00
Mark Haines
7dbb473339
Add function to load config without generating it
...
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.
The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.
As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
2016-06-09 18:50:38 +01:00
Erik Johnston
16a8884233
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-06-09 16:57:33 +01:00
Erik Johnston
a31befbcd0
Bump version and changelog
2016-06-09 13:23:41 +01:00
Erik Johnston
5ee5b655b2
Merge pull request #862 from matrix-org/erikj/media_remote_error
...
502 on /thumbnail when can't contact remote server
2016-06-09 13:11:02 +01:00
Erik Johnston
0a43219a27
Merge pull request #860 from negzi/bug_fix_get_or_create_user
...
Fix a bug caused by a change in auth_handler function
2016-06-09 11:40:12 +01:00
Erik Johnston
eba4ff1bcb
502 on /thumbnail when can't contact remote server
2016-06-09 11:29:43 +01:00
Erik Johnston
95f305c35a
Remove redundant exception log in /events
2016-06-09 11:15:04 +01:00
Negar Fazeli
6e7dc7c7dd
Fix a bug caused by a change in auth_handler function
...
Fix the relevant unit test cases
2016-06-08 23:22:39 +02:00
Erik Johnston
b7fbc9bd95
Merge pull request #859 from matrix-org/erikj/public_room_performance
...
Pull full state for each room all at once
2016-06-08 16:36:06 +01:00
Erik Johnston
919a2c74f6
Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop
2016-06-08 16:26:26 +01:00
Erik Johnston
81c07a32fd
Pull full state for each room all at once
2016-06-08 15:51:49 +01:00
Erik Johnston
690029d1a3
Don't make rooms visibile by default
2016-06-08 14:47:42 +01:00
Erik Johnston
d88faf92d1
Fix up federation PublicRoomList
2016-06-08 14:39:31 +01:00
Erik Johnston
958c968d02
Merge pull request #856 from matrix-org/erikj/fed_pub_rooms
...
Enable auth on /publicRoom endpoints
2016-06-08 14:36:09 +01:00
Erik Johnston
746b2f5657
Merge pull request #854 from matrix-org/erikj/federation_logging
...
Add some logging for when servers ask for missing events
2016-06-08 14:31:41 +01:00
Erik Johnston
efeabd3180
Log user that is making /publicRooms calls
2016-06-08 14:23:15 +01:00
Erik Johnston
1fd6eb695d
Enable auth on federation PublicRoomList
2016-06-08 14:15:18 +01:00
Erik Johnston
17aab5827a
Add some logging for when servers ask for missing events
2016-06-08 11:55:31 +01:00
Erik Johnston
1a815fb04f
Don't hit DB for noop replications queries
2016-06-08 11:33:30 +01:00
Erik Johnston
bab916bccc
Bump version and changelog to v0.16.0-rc2
2016-06-08 11:05:45 +01:00
Erik Johnston
e0fda29f94
Merge pull request #850 from matrix-org/erikj/gc_threshold
...
Add gc_threshold to pusher and synchrotron
2016-06-08 10:05:56 +01:00
Erik Johnston
0f2165ccf4
Don't track total objects as its too expensive to calculate
2016-06-07 17:00:45 +01:00
Erik Johnston
18f0cc7d99
Record some more GC metrics
2016-06-07 16:55:49 +01:00
Erik Johnston
2d1d1025fa
Add gc_threshold to pusher and synchrotron
2016-06-07 16:26:25 +01:00
Erik Johnston
b01e71e719
Merge pull request #849 from matrix-org/erikj/gc_threshold
...
Allow setting of gc.set_thresholds
2016-06-07 16:07:54 +01:00
Erik Johnston
dded389ac1
Allow setting of gc.set_thresholds
2016-06-07 15:45:56 +01:00
Mark Haines
0b2158719c
Remove dead code.
...
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.
The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.
The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston
48e65099b5
Also record number of unreachable objects
2016-06-07 13:40:22 +01:00
Erik Johnston
75331c5fca
Change the way we do stats
2016-06-07 13:33:13 +01:00
Erik Johnston
8c966fbd51
Merge pull request #771 from matrix-org/erikj/gc_tick
...
Manually run GC on reactor tick.
2016-06-07 13:18:36 +01:00
Mark Haines
88625db05f
Notify users for events in rooms they join.
...
Change how the notifier updates the map from room_id to user streams on
receiving a join event. Make it update the map when it notifies for the
join event, rather than using the "user_joined_room" distributor signal
2016-06-07 11:33:36 +01:00
Erik Johnston
84379062f9
Fix AS retries, but with correct ordering
2016-06-07 10:24:50 +01:00
Erik Johnston
310197bab5
Fix AS retries
2016-06-07 09:34:50 +01:00
Mark Haines
b0932b34cb
Merge pull request #845 from matrix-org/markjh/synchrotron_presence
...
Fix a KeyError in the synchrotron presence
2016-06-06 16:52:27 +01:00
Mark Haines
4a5bbb1941
Fix a KeyError in the synchrotron presence
2016-06-06 16:37:12 +01:00
Mark Haines
87f60e7053
Merge pull request #844 from matrix-org/markjh/yield_on_sleep
...
Yield on the sleeps intended to backoff replication
2016-06-06 16:20:54 +01:00
Mark Haines
5ef84da4f1
Yield on the sleeps intended to backoff replication
2016-06-06 16:05:28 +01:00
Erik Johnston
216a05b3e3
.values() returns list of sets
2016-06-06 16:00:09 +01:00
Erik Johnston
96dc600579
Fix typos
2016-06-06 15:44:41 +01:00
Erik Johnston
377eb480ca
Fire after 30s not 8h
2016-06-06 15:14:21 +01:00
Erik Johnston
e4134c5e13
Merge pull request #841 from matrix-org/erikj/event_counter
...
Add metric counter for number of persisted events
2016-06-06 14:17:40 +01:00
Erik Johnston
7aa778fba9
Add metric counter for number of persisted events
2016-06-06 11:58:09 +01:00
Erik Johnston
70aee0717c
Add events to cache when we persist them
2016-06-06 11:34:53 +01:00
Erik Johnston
3210f4c385
Merge pull request #836 from matrix-org/erikj/change_event_cache
...
Change the way we cache events
2016-06-03 18:47:46 +01:00
Erik Johnston
cffe46408f
Don't rely on options when inserting event into cache
2016-06-03 18:25:21 +01:00
Mark Haines
ac9716f154
Fix spelling
2016-06-03 18:10:00 +01:00
Mark Haines
8f79084bd4
Add get_presence_list_accepted to the broken caches in synchrotron
2016-06-03 18:03:40 +01:00
Erik Johnston
10ea3f46ba
Change the way we cache events
2016-06-03 17:57:50 +01:00
Erik Johnston
f6be734be9
Merge pull request #835 from matrix-org/erikj/get_event_txn
...
Remove event fetching from DB threads
2016-06-03 17:30:00 +01:00
Erik Johnston
05e01f21d7
Remove event fetching from DB threads
2016-06-03 17:22:13 +01:00
David Baker
72c4d482e9
3rd time lucky: we'd already calculated it above
2016-06-03 16:39:50 +01:00
David Baker
fbf608decb
Oops, we're using the dict form
2016-06-03 16:38:39 +01:00
David Baker
06d40c8b98
Add substitutions to email notif From
2016-06-03 16:31:23 +01:00
Erik Johnston
5f88549f4a
Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop
2016-06-03 16:27:24 +01:00
Mark Haines
ca457f594e
Merge pull request #831 from matrix-org/markjh/synchrotronII
...
Add a separate process that can handle /sync requests
2016-06-03 16:06:25 +01:00
Erik Johnston
21961c93c7
Bump changelog and version
2016-06-03 15:33:14 +01:00
Mark Haines
48340e4f13
Clear the list of ongoing syncs on shutdown
2016-06-03 15:02:27 +01:00
Erik Johnston
fcbc282f56
Merge branch 'release-v0.15.0' of github.com:matrix-org/synapse into release-v0.16.0
2016-06-03 15:02:04 +01:00
Erik Johnston
51773bcbaf
Merge pull request #832 from matrix-org/erikj/presence_coount
...
Change def of small delta in presence stream. Add metrics.
2016-06-03 14:57:00 +01:00
Mark Haines
da491e75b2
Appease flake8
2016-06-03 14:56:36 +01:00
Mark Haines
0b3c80a234
Use ClientIpStore to record client ips
2016-06-03 14:55:01 +01:00
Mark Haines
dd6f62ed99
Merge branch 'develop' into markjh/synchrotronII
2016-06-03 14:51:33 +01:00
Mark Haines
eef541a291
Move insert_client_ip to a separate class
2016-06-03 14:42:35 +01:00
Mark Haines
80aade3805
Send updates to the syncing users every ten seconds or immediately if they've just come online
2016-06-03 14:24:19 +01:00
Erik Johnston
ab116bdb0c
Fix typo
2016-06-03 14:03:42 +01:00
Erik Johnston
4ce84a1acd
Change metric style
2016-06-03 13:49:16 +01:00
Erik Johnston
a7ff5a1770
Presence metrics. Change def of small delta
2016-06-03 13:40:55 +01:00
Matthew Hodgson
8d740132f4
Merge branch 'develop' into matthew/brand-from-header
2016-06-03 12:14:18 +01:00
Erik Johnston
3b096c5f5c
Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop
2016-06-03 12:00:33 +01:00
Erik Johnston
43b7f371f5
Merge pull request #830 from matrix-org/erikj/metrics_perf
...
Change CacheMetrics to be quicker
2016-06-03 11:57:29 +01:00
Mark Haines
abb151f3c9
Add a separate process that can handle /sync requests
2016-06-03 11:57:26 +01:00
Erik Johnston
4982b28868
Merge pull request #829 from matrix-org/erikj/poke_notifier
...
Poke notifier on next reactor tick
2016-06-03 11:52:10 +01:00
Erik Johnston
d06f2a229e
Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room
...
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:50:30 +01:00
Erik Johnston
58a224a651
Pull out update_results_dict
2016-06-03 11:47:07 +01:00
Mark Haines
20eccd84d4
Merge pull request #827 from matrix-org/markjh/more_slaved_methods
...
Add methods to events, account data and receipt slaves
2016-06-03 11:46:21 +01:00
Erik Johnston
722472b48c
Merge pull request #825 from matrix-org/erikj/cache_push_rules
...
Load push rules in storage layer so that they get cached
2016-06-03 11:44:32 +01:00
Erik Johnston
73c7112433
Change CacheMetrics to be quicker
...
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
Mark Haines
b09f348530
Merge pull request #824 from matrix-org/markjh/slaved_presence_store
...
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
Erik Johnston
4c04222fa5
Poke notifier on next reactor tick
2016-06-03 11:24:16 +01:00
Erik Johnston
ccb56fc24b
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:20:23 +01:00
Mark Haines
81cf449daa
Add methods to events, account data and receipt slaves
...
Adds the methods needed by /sync to the slaved events,
account data and receipt stores.
2016-06-03 11:19:27 +01:00
Erik Johnston
e043ede4a2
Small optimisation to CacheListDescriptor
2016-06-03 11:19:22 +01:00
Mark Haines
b821c839dc
Merge pull request #823 from matrix-org/markjh/more_slaved_stores
...
Add slaved stores for filters, tokens, and push rules
2016-06-03 11:17:43 +01:00
Erik Johnston
597013caa5
Make cachedList go a bit faster
2016-06-03 11:13:29 +01:00
Erik Johnston
6a0afa582a
Load push rules in storage layer, so that they get cached
2016-06-03 11:10:00 +01:00
Mark Haines
3ae915b27e
Add a slaved store for presence
2016-06-03 11:05:53 +01:00
Erik Johnston
59f2d73522
Remove unnecessary sets
2016-06-03 11:05:45 +01:00
Erik Johnston
9c26b390a2
Only get local users
2016-06-03 11:04:31 +01:00
Mark Haines
f88d747f79
Add a comment explaining why the filter cache doesn't need exipiring
2016-06-03 11:03:10 +01:00
Erik Johnston
065e739d6e
Merge pull request #811 from matrix-org/erikj/state_users_in_room
...
Use state to calculate get_users_in_room
2016-06-03 10:58:27 +01:00
Erik Johnston
696d7c5937
Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room
...
Add get_users_with_read_receipts_in_room cache
2016-06-03 10:58:24 +01:00
Mark Haines
0eae075723
Add slaved stores for filters, tokens, and push rules
2016-06-03 10:58:03 +01:00
Matthew Hodgson
79d1f072f4
brand the email from header
2016-06-02 21:34:40 +01:00
David Baker
6bb9aacf9d
Merge pull request #821 from matrix-org/dbkr/email_unsubscribe
...
Email unsubscribe links that don't require logging in
2016-06-02 17:44:55 +01:00
David Baker
745ddb4dd0
peppate
2016-06-02 17:38:41 +01:00
David Baker
7a5a5f2df2
Merge pull request #820 from matrix-org/dbkr/email_notif_string_fmt_error
...
Fix error in email notification string formatting
2016-06-02 17:26:06 +01:00
David Baker
1f31cc37f8
Working unsubscribe links going straight to the HS
...
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
Matthew Hodgson
2675c1e40e
add some branding debugging
2016-06-02 17:21:12 +01:00
David Baker
c71177f285
Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe
2016-06-02 17:20:56 +01:00
David Baker
07a5559916
Fix error in email notification string formatting
2016-06-02 17:17:16 +01:00
Mark Haines
56d15a0530
Store the typing users as user_id strings. ( #819 )
...
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
David Baker
812b5de0fe
Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe
2016-06-02 15:33:28 +01:00
Mark Haines
80f34d7b57
Fix setting the _clock in SQLBaseStore
2016-06-02 15:23:56 +01:00
Mark Haines
661a540dd1
Deduplicate presence entries in sync ( #818 )
2016-06-02 15:20:28 +01:00
Mark Haines
70599ce925
Allow external processes to mark a user as syncing. ( #812 )
...
* Add infrastructure to the presence handler to track sync requests in external processes
* Expire stale entries for dead external processes
* Add an http endpoint for making users as syncing
Add some docstrings and comments.
* Fixes
2016-06-02 15:20:15 +01:00
David Baker
fb2193cc63
Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler
...
Split out the auth handler
2016-06-02 14:31:35 +01:00
Erik Johnston
356f13c069
Disable INCLUDE_ALL_UNREAD_NOTIFS
2016-06-02 14:07:38 +01:00
Erik Johnston
02ac463dbf
Merge pull request #800 from matrix-org/erikj/sync_refactor
...
Refactor SyncHandler
2016-06-02 14:02:13 +01:00
Matthew Hodgson
c5af1b6b00
Merge pull request #814 from matrix-org/matthew/3pid_invite_auth
...
special case m.room.third_party_invite event auth to match invites,
2016-06-02 13:47:40 +01:00