Commit Graph

320 Commits

Author SHA1 Message Date
Richard van der Hoff
41d4b07a53
Merge pull request #3142 from matrix-org/rav/reraise
reraise exceptions more carefully
2018-04-27 12:16:19 +01:00
Richard van der Hoff
6493b22b42 reraise exceptions more carefully
We need to be careful (under python 2, at least) that when we reraise an
exception after doing some error handling, we actually reraise the original
exception rather than anything that might have been raised (and handled) during
the error handling.
2018-04-27 11:40:06 +01:00
Richard van der Hoff
9255a6cb17 Improve exception handling for background processes
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Richard van der Hoff
639480e14a Avoid creating events with huge numbers of prev_events
In most cases, we limit the number of prev_events for a given event to 10
events. This fixes a particular code path which created events with huge
numbers of prev_events.
2018-04-16 18:41:37 +01:00
Erik Johnston
f8e8ec013b Note why we're limiting concurrent event sends 2018-04-10 14:00:46 +01:00
Erik Johnston
e5082494eb Limit concurrent event sends for a room 2018-04-09 12:07:39 +01:00
Richard van der Hoff
05630758f2 Use static JSONEncoders
using json.dumps with custom options requires us to create a new JSONEncoder on
each call. It's more efficient to create one upfront and reuse it.
2018-03-29 23:13:33 +01:00
Erik Johnston
fa72803490 Merge branch 'master' of github.com:matrix-org/synapse into develop 2018-03-19 11:41:01 +00:00
Erik Johnston
926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
Erik Johnston
d0fcc48f9d extra_users is actually a list of UserIDs 2018-03-13 11:20:06 +00:00
Richard van der Hoff
e48c7aac4d Add transactional API to history purge
Make the purge request return quickly, and allow scripts to poll for updates.
2018-03-12 16:22:55 +00:00
Richard van der Hoff
1708412f56 Return an error when doing two purges on a room
Queuing up purges doesn't sound like a good thing.
2018-03-12 16:22:54 +00:00
Richard van der Hoff
f8bfcd7e0d Provide a means to pass a timestamp to purge_history 2018-03-05 14:37:23 +00:00
Erik Johnston
f381d63813 Check event auth on the worker 2018-03-01 10:18:37 +00:00
Erik Johnston
6b8604239f Correctly send ratelimit and extra_users params 2018-03-01 10:08:39 +00:00
Erik Johnston
f756f961ea Fixup comments 2018-03-01 10:05:27 +00:00
Erik Johnston
28e973ac11 Calculate push actions on worker 2018-02-28 18:02:30 +00:00
Erik Johnston
b96278d6fe Ensure that we delete staging push actions on errors 2018-02-15 15:47:06 +00:00
Erik Johnston
c0c9327fe0
Merge pull request #2854 from matrix-org/erikj/event_create_worker
Create a worker for event creation
2018-02-13 18:07:10 +00:00
Richard van der Hoff
10b34dbb9a
Merge pull request #2858 from matrix-org/rav/purge_updates
delete_local_events for purge_room_history
2018-02-09 14:11:00 +00:00
Richard van der Hoff
74fcbf741b delete_local_events for purge_history
Add a flag which makes the purger delete local events
2018-02-09 13:07:41 +00:00
Richard van der Hoff
671540dccf rename delete_old_state -> purge_history
(beacause it deletes more than state)
2018-02-09 13:07:41 +00:00
Erik Johnston
50fe92cd26 Move presence handling into handle_new_client_event
As we want to have it run on the main synapse instance
2018-02-07 10:34:09 +00:00
Erik Johnston
24dd73028a Add replication http endpoint for event sending 2018-02-07 10:32:32 +00:00
Erik Johnston
e3624fad5f Remove pointless ratelimit check
The intention was for the check to be called as early as possible in the
request, but actually was called just before the main ratelimit check,
so was fairly pointless.
2018-02-07 10:30:25 +00:00
Erik Johnston
3e1e69ccaf Update copyright 2018-02-06 16:40:38 +00:00
Erik Johnston
770b2252ca s/_create_new_client_event/create_new_client_event/ 2018-02-06 16:40:30 +00:00
Erik Johnston
5ff3d23564 Split event creation into a separate handler 2018-02-05 16:01:48 +00:00
Richard van der Hoff
eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
David Baker
b78bae2d51 fix isinstance 2017-10-11 14:49:09 +01:00
David Baker
c3b7a45e84 Allow error strings from spam checker 2017-10-11 14:39:22 +01:00
Erik Johnston
535cc49f27 Merge pull request #2466 from matrix-org/erikj/groups_merged
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Erik Johnston
bc7f3eb32f Merge pull request #2483 from jeremycline/unfreeze-ujson-dump
Unfreeze event before serializing with ujson
2017-10-11 13:18:52 +01:00
David Baker
27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
Jeremy Cline
cafb8de132
Unfreeze event before serializing with ujson
In newer versions of https://github.com/esnme/ultrajson, ujson does not
serialize frozendicts (introduced in esnme/ultrajson@53f85b1). Although
the PyPI version is still 1.35, Fedora ships with a build from commit
esnme/ultrajson@2f1d487. This causes the serialization to fail if the
distribution-provided package is used.

This runs the event through the unfreeze utility before serializing it.

Thanks to @ignatenkobrain for tracking down the root cause.

fixes #2351

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
2017-09-30 11:22:37 -04:00
Erik Johnston
8090fd4664 Fix /joined_members to work with AS users 2017-09-28 10:09:32 +01:00
Erik Johnston
adec03395d Fix bug where /joined_members didn't check user was in room 2017-09-27 15:14:39 +01:00
David Baker
6cd5fcd536 Make the spam checker a module 2017-09-26 19:20:23 +01:00
Richard van der Hoff
5ed109d59f PoC for filtering spammy events (#2456)
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00
Erik Johnston
bf81f3cf2c Split out profile handler to fix tests 2017-08-25 14:34:56 +01:00
Erik Johnston
5a7f561a9b Fix bug where pusherpool didn't start and broke some rooms
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.

This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
2017-07-06 17:55:51 +01:00
Erik Johnston
1bce3e6b35 Remove unused variables 2017-06-26 14:03:27 +01:00
Erik Johnston
fcf01dd88e Reject local events that don't round trip the DB 2017-06-19 11:33:40 +01:00
Erik Johnston
cafe659f72 Store ActionGenerator in HomeServer 2017-05-19 10:09:56 +01:00
Erik Johnston
66d8ffabbd Faster push rule calculation via push specific cache
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
2017-05-17 16:55:40 +01:00
Erik Johnston
b990b2fce5 Add per user ratelimiting overrides 2017-05-10 11:05:43 +01:00
Erik Johnston
3e5a62ecd8 Add more granular event send metrics 2017-05-02 14:23:26 +01:00
Erik Johnston
34e682d385 Fix invite state to always include all events 2017-04-26 16:18:08 +01:00
Richard van der Hoff
7eb9f34cc3 Remove spurious yield
In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`:
it doesn't return anything anyway.
2017-04-03 15:44:19 +01:00
Paul "LeoNerd" Evans
10e48d8310 Don't clobber a displayname or avatar_url if provided by an m.room.member event 2017-01-24 18:06:07 +00:00
Erik Johnston
3b4de17d2b Comment 2017-01-10 14:05:53 +00:00
Erik Johnston
586f474a44 Don't block messages sending on bumping presence 2017-01-10 12:46:00 +00:00
Erik Johnston
d53a80af25 Merge pull request #1620 from matrix-org/erikj/concurrent_room_access
Limit the number of events that can be created on a given room concurrently
2016-12-12 10:30:23 +00:00
Erik Johnston
f8ee66250a Handle sending events and device messages over federation 2016-11-17 15:48:04 +00:00
Erik Johnston
64038b806c Comments 2016-11-11 10:42:08 +00:00
Erik Johnston
2bd4513a4d Limit the number of events that can be created on a given room concurretnly 2016-11-10 16:44:35 +00:00
Erik Johnston
a4632783fb Sample correctly 2016-11-08 11:20:26 +00:00
Erik Johnston
eeda4e618c Limit the number of prev_events of new events 2016-11-08 11:02:29 +00:00
Erik Johnston
d04e2ff3a4 Fix incredubly slow back pagination query
If a client didn't specify a from token when paginating backwards
synapse would attempt to query the (global) maximum topological token.
This a) doesn't make much sense since they're room specific and b) there
are no indices that lets postgres do this efficiently.
2016-10-24 13:35:51 +01:00
Erik Johnston
f2f74ffce6 Comment 2016-10-19 14:21:28 +01:00
Erik Johnston
550308c7a1 Check whether to ratelimit sooner to avoid work 2016-10-19 10:45:24 +01:00
Erik Johnston
87528f0756 Support /initialSync in synchrotron worker 2016-09-21 11:46:28 +01:00
Erik Johnston
69a2d4e38c Use get_joined_users_from_context instead of manually looking up hosts 2016-09-05 13:44:40 +01:00
Erik Johnston
c10cb581c6 Correctly handle the difference between prev and current state 2016-08-31 14:26:22 +01:00
Erik Johnston
a3dc1e9cbe Replace context.current_state with context.current_state_ids 2016-08-25 17:32:22 +01:00
Erik Johnston
9219139351 Preserve some logcontexts 2016-08-24 11:58:40 +01:00
Erik Johnston
a98d215204 Add filter param to /messages API 2016-07-14 16:30:56 +01:00
Erik Johnston
8f8798bc0d Add ReadWriteLock for pagination and history prune 2016-07-05 15:30:25 +01:00
Erik Johnston
2d21d43c34 Add purge_history API 2016-07-05 10:28:51 +01:00
Erik Johnston
4c04222fa5 Poke notifier on next reactor tick 2016-06-03 11:24:16 +01:00
Mark Haines
b153f5b150 Merge pull request #787 from matrix-org/markjh/liberate_presence_handler
Move the presence handler out of the Handlers object
2016-05-17 10:09:43 +01:00
Mark Haines
1a3a2002ff Spell "domain" correctly
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Mark Haines
3b86ecfa79 Move the presence handler out of the Handlers object 2016-05-16 18:56:37 +01:00
Mark Haines
7e23476814 move filter_events_for_client out of base handler 2016-05-11 13:42:37 +01:00
Mark Haines
1620578b13 Shuffle when we get the signing_key attribute.
Wait until we sign a message to get the signing key from the homeserver
config. This means that the message handler can be created without
having a signing key in the config which means that separate processes
like the pusher that don't send messages and don't need to sign them can
still access the handlers.
2016-05-11 12:20:57 +01:00
Mark Haines
30057b1e15 Move _create_new_client_event and handle_new_client_event out of base handler 2016-05-11 09:09:20 +01:00
David Baker
424a7f48f8 Run filter_events_for_client
so we don't accidentally mail out events people shouldn't see
2016-04-27 17:50:49 +01:00
Mark Haines
7e2f971c08 Remove some unused functions (#711)
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
Mark Haines
86be915cce Call profile handler get_displayname directly rather than using collect_presencelike_data 2016-04-07 18:11:49 +01:00
Erik Johnston
d76d89323c Use computed prev event ids 2016-04-01 17:39:32 +01:00
Erik Johnston
8d73cd502b Add concurrently_execute function 2016-04-01 14:06:00 +01:00
Mark Haines
b7a3be693b Merge pull request #618 from matrix-org/markjh/pushrule_stream
Add a stream for push rule updates
2016-03-04 16:35:08 +00:00
Erik Johnston
13cbd31040 Spelling 2016-03-04 15:22:39 +00:00
Mark Haines
ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Erik Johnston
ed61a49169 Add profile information to invites 2016-03-04 14:35:02 +00:00
Daniel Wagner-Hall
b4022cc487 Pass whole requester to ratelimiting
This will enable more detailed decisions
2016-03-03 16:43:42 +00:00
Mark Haines
ddf9e7b302 Hook up the push rules to the notifier 2016-03-03 14:57:45 +00:00
Erik Johnston
e5ad2e5267 Merge pull request #582 from matrix-org/erikj/presence
Rewrite presence for performance.
2016-02-19 09:37:50 +00:00
Daniel Wagner-Hall
f8d21e1431 Review comments 2016-02-18 11:02:14 +00:00
Erik Johnston
e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall
d1fb790818 Some cleanup 2016-02-16 14:25:23 +00:00
Daniel Wagner-Hall
4bfb32f685 Branch off member and non member sends
Unclean, needs tidy-up, but works
2016-02-15 18:21:30 +00:00
Daniel Wagner-Hall
e71095801f Merge implementation of /join by alias or ID
This code is kind of rough (passing the remote servers down a long
chain), but is a step towards improvement.
2016-02-15 15:39:16 +00:00
Erik Johnston
4e7948b47a Allow paginating backwards from stream token 2016-01-28 11:52:34 +00:00
Richard van der Hoff
5cba88ea7c Make it possible to paginate forwards from stream tokens
In order that we can fill the gap after a /sync, make it possible to paginate
forwards from a stream token.
2016-01-27 17:42:45 +00:00
Daniel Wagner-Hall
da417aa56d Allow non-guests to peek on rooms using /events 2016-01-20 15:34:07 +00:00
Daniel Wagner-Hall
ac5a4477ad Require unbanning before other membership changes 2016-01-15 16:27:26 +00:00
Daniel Wagner-Hall
2ef6de928d Skip, rather than erroring, invalid guest requests
Erroring causes problems when people make illegal requests, because they
don't know what limit parameter they should pass.

This is definitely buggy. It leaks message counts for rooms people don't
have permission to see, via tokens. But apparently we already
consciously decided to allow that as a team, so this preserves that
behaviour.
2016-01-05 18:12:37 +00:00
Mark Haines
517fb9a023 Move the doc string to the public facing method 2015-12-22 18:53:47 +00:00
Mark Haines
9ac417fa88 Add a cache for initialSync responses that expires after 5 minutes 2015-12-22 18:27:56 +00:00
Mark Haines
491f3d16dc Make state updates in the C+S API idempotent 2015-12-02 15:50:50 +00:00
Mark Haines
5eb4d13aaa Fix typo in collect_presencelike_data 2015-12-02 10:50:58 +00:00
Mark Haines
a9526831a4 Wrap calls to distributor.fire in appropriately named functions so that static analysis can work out want is calling what 2015-12-01 20:53:04 +00:00
Mark Haines
95f30ecd1f Add API for setting account_data globaly or on a per room basis 2015-12-01 18:41:32 +00:00
Daniel Wagner-Hall
924d85a75e Merge pull request #375 from matrix-org/daniel/guestroominitialsync
Clean up room initialSync for guest users
2015-11-23 16:10:49 +00:00
Mark Haines
1c960fbb80 s/private_user_data/account_data/ 2015-11-18 15:31:04 +00:00
Daniel Wagner-Hall
641420c5e0 Clean up room initialSync for guest users 2015-11-13 15:44:57 +00:00
Daniel Wagner-Hall
06bfd0a3c0 Merge pull request #367 from matrix-org/daniel/readafterleave
Merge pull request #367 from matrix-org/daniel/readafterleave

Tweak guest access permissions
2015-11-12 15:22:02 +00:00
Daniel Wagner-Hall
0d08670f61 Merge pull request #360 from matrix-org/daniel/guestroominitialsync
Merge pull request #360 from matrix-org/daniel/guestroominitialsync

Allow guest access to room initialSync
2015-11-12 15:19:55 +00:00
Daniel Wagner-Hall
fb7e260a20 Tweak guest access permissions
* Allow world_readable rooms to be read by guests who have joined and
   left
 * Allow regular users to access world_readable rooms
2015-11-12 15:02:00 +00:00
Daniel Wagner-Hall
50f1afbd5b Consider joined guest users as joined users
Otherwise they're inconveniently allowed to write events to the room
but not to read them from the room.
2015-11-12 13:37:07 +00:00
Daniel Wagner-Hall
f15ba926cc Allow guest access to room initialSync 2015-11-11 17:13:24 +00:00
Daniel Wagner-Hall
38d82edf0e Allow guest users to join and message rooms 2015-11-10 16:57:13 +00:00
Daniel Wagner-Hall
ca2f90742d Open up /events to anonymous users for room events only
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
Daniel Wagner-Hall
f522f50a08 Allow guests to register and call /events?room_id=
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
2015-11-04 17:29:07 +00:00
Mark Haines
ddd8566f41 Store room tag content and return the content in the m.tag event 2015-11-02 15:11:31 +00:00
Mark Haines
79b65f3875 Include tags in v1 room initial sync 2015-10-30 16:28:19 +00:00
Mark Haines
fdf73c6855 Include room tags v1 /initialSync 2015-10-30 16:22:32 +00:00
Erik Johnston
366af6b73a Amalgamate _filter_events_for_client 2015-10-16 14:52:48 +01:00
Erik Johnston
858634e1d0 Remove unused room_id arg 2015-10-14 09:31:20 +01:00
Mark Haines
dc72021748 Add a flag to initial sync to indicate we want rooms that the user has left 2015-10-08 17:26:23 +01:00
Erik Johnston
a14665bde7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/invite_state 2015-09-25 11:38:28 +01:00
Mark Haines
49c0a0b5c4 Clarify that room_initial_sync returns a python dict 2015-09-21 14:21:03 +01:00
Mark Haines
95c304e3f9 Fix doc string to point at the right class 2015-09-21 14:18:47 +01:00
Mark Haines
0c16285989 Add explicit "elif event.membership == Membership.LEAVE" for clarity 2015-09-21 14:17:16 +01:00
Mark Haines
1e101ed4a4 Clamp the "to" token for /rooms/{roomId}/messages to when the user left
the room.

There isn't a way for the client to learn a valid "to" token for a room
that they have left in the C-S API but that doesn't stop a client making
one up.
2015-09-21 14:13:10 +01:00
Mark Haines
e2054ce21a Allow users to GET individual state events for rooms that they have left 2015-09-10 15:06:47 +01:00
Erik Johnston
49ae42bbe1 Bundle in some room state in the unsigned bit of the invite when sending to invited servers 2015-09-10 14:25:54 +01:00
Mark Haines
09cb5c7d33 Allow users that have left a room to get the messages that happend in the room before they left 2015-09-09 17:31:09 +01:00
Mark Haines
3c166a24c5 Remove undocumented and unimplemented 'feedback' parameter from the Client-Server API 2015-09-09 16:05:09 +01:00
Mark Haines
1d579df664 Allow rooms/{roomId}/state for a room that has been left 2015-09-09 14:12:24 +01:00
Mark Haines
89ae0166de Allow room initialSync for users that have left the room, returning a snapshot of how the room was when they left it 2015-09-09 13:25:22 +01:00
Mark Haines
dd42bb78d0 Include rooms that a user has left in an initialSync. Include the state and messages at the point they left the room 2015-09-08 18:16:09 +01:00
Daniel Wagner-Hall
a0b181bd17 Remove completely unused concepts from codebase
Removes device_id and ClientInfo

device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.

This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
Erik Johnston
1a9510bb84 Implement a batched presence_handler.get_state and use it 2015-08-17 10:40:23 +01:00
Erik Johnston
7b0e797080 Fix _filter_events_for_client 2015-08-12 17:05:24 +01:00
Erik Johnston
a7eeb34c64 Simplify staggered deferred lists 2015-08-12 16:02:05 +01:00
Erik Johnston
f7e2f981ea Use list comprehension instead of filter 2015-08-12 16:01:10 +01:00
Erik Johnston
dc8399ee00 Remove debug loggers 2015-08-11 11:30:59 +01:00
Erik Johnston
1b994a97dd Fix application of ACLs 2015-08-11 10:41:40 +01:00
Erik Johnston
ffdb8c3828 Don't be too enthusiatic with defer.gatherResults 2015-08-07 18:13:48 +01:00
Erik Johnston
4d6cb8814e Speed up event filtering (for ACL) logic 2015-08-04 09:32:23 +01:00
Erik Johnston
5989637f37 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-13 13:50:57 +01:00
Erik Johnston
87311d1b8c Hook up receipts to v1 initialSync 2015-07-08 11:02:04 +01:00
Erik Johnston
fb47c3cfbe Rename key and values for m.room.history_visibility. Support 'invited' value 2015-07-06 13:05:52 +01:00
Erik Johnston
41938afed8 Make v1 initial syncs respect room history ACL 2015-07-02 17:12:35 +01:00
Erik Johnston
1a60545626 Add basic impl for room history ACL on GET /messages client API 2015-07-02 16:20:10 +01:00
Erik Johnston
eceb554a2f Use another deferred list 2015-06-16 17:12:27 +01:00
Erik Johnston
b849a64f8d Use DeferredList 2015-06-16 17:03:24 +01:00
Erik Johnston
0460406298 Don't do unecessary db ops in presence.get_state 2015-06-16 16:59:38 +01:00