Commit Graph

1246 Commits

Author SHA1 Message Date
Erik Johnston
8f03aa9f61 Add StorageProvider concept 2018-01-09 16:16:12 +00:00
Erik Johnston
2442e9876c Make PreviewUrlResource use MediaStorage 2018-01-09 16:15:07 +00:00
Erik Johnston
9d30a7691c Make ThumbnailResource use MediaStorage 2018-01-09 16:15:07 +00:00
Erik Johnston
9e20840e02 Use MediaStorage for remote media 2018-01-09 16:15:07 +00:00
Erik Johnston
dd3092c3a3 Use MediaStorage for local files 2018-01-09 16:15:07 +00:00
Erik Johnston
ada470bccb Add MediaStorage class 2018-01-09 16:15:07 +00:00
Erik Johnston
1ee787912b Add some helper classes 2018-01-09 16:15:07 +00:00
Erik Johnston
47ca5eb882 Split out add_file_headers 2018-01-09 16:15:07 +00:00
Erik Johnston
b6c9deffda Remove dead TODO 2018-01-09 15:53:23 +00:00
Richard van der Hoff
51c9d9ed65 Add /room/{id}/event/{id} to synapse
Turns out that there is a valid usecase for retrieving event by id (notably
having received a push), but event ids should be scoped to room, so /event/{id}
is wrong.
2018-01-09 14:39:12 +00:00
Erik Johnston
b30cd5b107 Remove dead code related to default thumbnails 2018-01-09 14:38:33 +00:00
Richard van der Hoff
75c1b8df01 Better logging when login can't find a 3pid 2017-12-20 19:31:00 +00:00
Richard van der Hoff
16ec3805e5 Fix error when deleting devices
This was introduced in d7ea8c4 / PR 
2017-12-05 09:49:22 +00:00
Richard van der Hoff
d7ea8c4800 Factor out a validate_user_via_ui_auth method
Collect together all the places that validate a logged-in user via UI auth.
2017-12-05 09:42:30 +00:00
Richard van der Hoff
d5f9fb06b0 Refactor UI auth implementation
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
2017-12-05 09:40:05 +00:00
Richard van der Hoff
ad7e570d07 Delete devices in various logout situations
Make sure that we delete devices whenever a user is logged out due to any of
the following situations:

 * /logout
 * /logout_all
 * change password
 * deactivate account (by the user or by an admin)
 * invalidate access token from a dynamic module

Fixes .
2017-11-29 16:44:35 +00:00
Richard van der Hoff
ae31f8ce45 Move set_password into its own handler
Non-functional refactoring to move set_password. This means that we'll be able
to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
Richard van der Hoff
7ca5c68233 Move deactivate_account into its own handler
Non-functional refactoring to move deactivate_account. This means that we'll be
able to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
Luke Barnard
ab1b2d0ff2 Allow guest access to group APIs for reading 2017-11-28 11:23:00 +00:00
Richard van der Hoff
5a4da5bf78
Merge pull request from matrix-org/rav/fix_urlcache_index_error
Fix error on sqlite 3.7
2017-11-27 12:25:48 +00:00
Richard van der Hoff
8132a6b7ac Fix OPTIONS on preview_url
Fixes 
2017-11-23 17:52:31 +00:00
Richard van der Hoff
2908f955d1 Check database in has_completed_background_updates
so that the right thing happens on workers.
2017-11-22 18:02:15 +00:00
Richard van der Hoff
7098b65cb8 Fix error on sqlite 3.7
Create the url_cache index on local_media_repository as a background update, so
that we can detect whether we are on sqlite or not and create a partial or
complete index accordingly.

To avoid running the cleanup job before we have built the index, add a bailout
which will defer the cleanup if the bg updates are still running.

Fixes https://github.com/matrix-org/synapse/issues/2572.
2017-11-21 11:14:17 +00:00
Richard van der Hoff
30d2730ee2 Declare support for r0.3.0 2017-11-15 16:24:22 +00:00
Richard van der Hoff
7bd6c87eca
Merge pull request from turt2live/travis/whoami
Add a route for determining who you are
2017-11-14 09:54:21 +00:00
Travis Ralston
812c191939 Remove redundent call
Signed-off-by: Travis Ralston <travpc@gmail.com>
2017-11-13 12:44:21 -07:00
Travis Ralston
2d314b771f Add a route for determining who you are
Useful for applications which may have an access token, but no idea as to who owns it.

Signed-off-by: Travis Ralston <travpc@gmail.com>
2017-11-12 23:39:38 -07:00
Richard van der Hoff
5d15abb120 Bit more logging 2017-11-10 16:58:04 +00:00
Richard van der Hoff
46790f50cf Cache failures in url_preview handler
Reshuffle the caching logic in the url_preview handler so that failures are
cached (and to generally simplify things and fix the logcontext leaks).
2017-11-10 16:50:50 +00:00
Richard van der Hoff
e508145c9b Add some more comments appservice user registration
Explain why we don't validate userids registered via app services
2017-11-10 12:39:45 +00:00
Richard van der Hoff
e0ebd1e4bd Downcase userids for shared-secret registration 2017-11-10 12:39:05 +00:00
Richard van der Hoff
9b599bc18d Downcase userid on registration
Force username to lowercase before attempting to register

https://github.com/matrix-org/synapse/issues/2660
2017-11-09 22:20:01 +00:00
Erik Johnston
4e2b2508af Register group servlet 2017-11-09 15:49:42 +00:00
Erik Johnston
e8814410ef Have an explicit API to update room config 2017-11-08 16:13:27 +00:00
Erik Johnston
94ff2cda73
Revert "Modify group room association API to allow modification of is_public" 2017-11-08 15:43:34 +00:00
Matthew Hodgson
3db5ff69b2
Merge pull request from maximevaillancourt/exclude-noscript-url-preview
Ignore <noscript> tags when generating URL preview descriptions
2017-11-07 11:09:22 +00:00
Richard van der Hoff
6c3a02072b support inhibit_login in /register
Allow things to pass inhibit_login when registering to ... inhibit logins.
2017-11-02 16:31:07 +00:00
Richard van der Hoff
4c8f94ac94 Allow password_auth_providers to return a callback
... so that they have a way to record access tokens.
2017-11-01 16:51:03 +00:00
David Baker
4f0488b307 Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete 2017-11-01 16:20:19 +00:00
David Baker
e5e930aec3
Merge pull request from matrix-org/rav/break_auth_device_dep
Break dependency of auth_handler on device_handler
2017-11-01 16:06:31 +00:00
Richard van der Hoff
dd13310fb8 Move access token deletion into auth handler
Also move duplicated deactivation code into the auth handler.

I want to add some hooks when we deactivate an access token, so let's bring it
all in here so that there's somewhere to put it.
2017-11-01 15:46:22 +00:00
David Baker
0bb253f37b Apparently this is python 2017-11-01 14:02:52 +00:00
David Baker
59e7e62c4b Log login requests
Carefully though, to avoid logging passwords
2017-11-01 13:58:01 +00:00
Richard van der Hoff
74c56f794c Break dependency of auth_handler on device_handler
I'm going to need to make the device_handler depend on the auth_handler, so I
need to break this dependency to avoid a cycle.

It turns out that the auth_handler was only using the device_handler in one
place which was an edge case which we can more elegantly handle by throwing an
error rather than fixing it up.
2017-11-01 10:27:06 +00:00
Luke Barnard
20fe347906 Modify group room association API to allow modification of is_public
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
Richard van der Hoff
1b65ae00ac Refactor some logic from LoginRestServlet into AuthHandler
I'm going to need some more flexibility in handling login types in password
auth providers, so as a first step, move some stuff from LoginRestServlet into
AuthHandler.

In particular, we pass everything other than SAML, JWT and token logins down to
the AuthHandler, which now has responsibility for checking the login type and
fishing the password out of the login dictionary, as well as qualifying the
user_id if need be. Ideally SAML, JWT and token would go that way too, but
there's no real need for it right now and I'm trying to minimise impact.

This commit *should* be non-functional.
2017-10-31 10:48:41 +00:00
Richard van der Hoff
4d83632009 Merge pull request from matrix-org/rav/device_delete_auth
Device deletion: check UI auth matches access token
2017-10-27 12:30:10 +01:00
Richard van der Hoff
110b373e9c Merge pull request from matrix-org/rav/as_deactivate_account
Allow ASes to deactivate their own users
2017-10-27 12:29:32 +01:00
Luke Barnard
d8c26162a1 Merge pull request from matrix-org/luke/group-is-public
Add is_public to groups table to allow for private groups
2017-10-27 11:41:13 +01:00
Richard van der Hoff
7a6546228b Device deletion: check UI auth matches access token
(otherwise there's no point in the UI auth)
2017-10-27 00:04:31 +01:00
Richard van der Hoff
785bd7fd75 Allow ASes to deactivate their own users 2017-10-27 00:01:00 +01:00
Luke Barnard
595fe67f01 delint 2017-10-26 17:20:24 +01:00
Luke Barnard
9b2feef9eb Add is_public to groups table to allow for private groups
Prevent group API access to non-members for private groups

Also make all the group code paths consistent with `requester_user_id` always being the User ID of the requesting user.
2017-10-26 16:51:32 +01:00
Krombel
8299b323ee add release endpoints for /thirdparty 2017-10-26 16:58:20 +02:00
Krombel
9b436c8b4c register some /unstable endpoints in /r0 as well 2017-10-26 15:22:50 +02:00
Maxime Vaillancourt
5287e57c86 Ignore noscript tags when generating URL previews 2017-10-25 20:44:34 -04:00
Richard van der Hoff
eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Richard van der Hoff
d03cfc4258 Fix a logcontext leak in the media repo 2017-10-23 14:34:27 +01:00
Richard van der Hoff
631d7b87b5 Remove pointless create() method
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
2017-10-20 22:14:55 +01:00
Erik Johnston
bd5718d0ad Fix typo in thumbnail generation 2017-10-19 10:27:18 +01:00
Erik Johnston
fe6e9f580b Merge pull request from krombel/fix_thumbnail_2548
fix thumbnailing ()
2017-10-17 15:35:18 +01:00
David Baker
4e242b3e20 Merge pull request from matrix-org/dbkr/auto_join_rooms
Add config option to auto-join new users to rooms
2017-10-17 11:45:49 +01:00
Krombel
a6245478c8 fix thumbnailing ()
in commit 0e28281a the code for thumbnailing got refactored and the
renaming of this variables was not done correctly.

Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2017-10-17 12:45:33 +02:00
David Baker
c05e6015cc Add config option to auto-join new users to rooms
New users who register on the server will be dumped into all rooms in
auto_join_rooms in the config.
2017-10-16 17:57:27 +01:00
Luke Barnard
85f5674e44 Delint 2017-10-16 15:52:17 +01:00
Luke Barnard
2c5972f87f Implement GET /groups/$groupId/invited_users 2017-10-16 15:31:11 +01:00
Erik Johnston
1b6b0b1e66 Add try/finally block to close t_byte_source 2017-10-13 15:34:08 +01:00
Erik Johnston
6b725cf56a Remove old comment 2017-10-13 15:23:41 +01:00
Erik Johnston
2b24416e90 Don't reuse source but instead copy from primary media store to backup 2017-10-13 14:11:34 +01:00
Erik Johnston
b92a8e6e4a PEP8 2017-10-13 13:58:57 +01:00
Erik Johnston
31aa7bd8d1 Move type into key 2017-10-13 13:47:38 +01:00
Erik Johnston
ad1911bbf4 Comment 2017-10-13 13:47:05 +01:00
Erik Johnston
c021c39cbd Remove spurious addition 2017-10-13 13:46:53 +01:00
Erik Johnston
1f43d22397 Don't needlessly rename variable 2017-10-13 11:42:07 +01:00
Erik Johnston
a675bd08bd Add paths back in... 2017-10-13 11:41:06 +01:00
Erik Johnston
4d7e1dde70 Remove unnecessary diff 2017-10-13 11:36:32 +01:00
Erik Johnston
ae5d18617a Make things be absolute paths again 2017-10-13 11:35:44 +01:00
Erik Johnston
9732ec6797 s/write_to_file/write_to_file_and_backup/ 2017-10-13 11:34:41 +01:00
Erik Johnston
0e28281a02 Fix up 2017-10-13 11:33:49 +01:00
Erik Johnston
505371414f Fix up thumbnailing function 2017-10-13 11:23:53 +01:00
Erik Johnston
e3428d26ca Fix typo 2017-10-13 10:39:59 +01:00
Erik Johnston
35332298ef Fix up comments 2017-10-13 10:39:32 +01:00
Erik Johnston
64db043a71 Move makedirs to thread 2017-10-13 10:25:01 +01:00
Erik Johnston
b60859d6cc Use make_deferred_yieldable 2017-10-13 10:24:19 +01:00
Erik Johnston
d76621a47b Fix comments 2017-10-12 18:16:25 +01:00
Erik Johnston
4ae85ae121 Don't close prematurely.. 2017-10-12 17:57:31 +01:00
Erik Johnston
cc505b4b5e getvalue closes buffer 2017-10-12 17:52:30 +01:00
Erik Johnston
1259a76047 Get len before close 2017-10-12 17:39:23 +01:00
Erik Johnston
802ca12d05 Don't close file prematurely 2017-10-12 17:37:21 +01:00
Erik Johnston
e283b555b1 Copy everything to backup 2017-10-12 17:31:24 +01:00
Erik Johnston
b77a13812c Typo 2017-10-12 15:32:32 +01:00
Erik Johnston
6dfde6d485 Remove dead code 2017-10-12 15:30:26 +01:00
Erik Johnston
c8eeef6947 Fix typos 2017-10-12 15:28:24 +01:00
Erik Johnston
67cb89fbdf Fix typo 2017-10-12 15:23:41 +01:00
Erik Johnston
bf4fb1fb40 Basic implementation of backup media store 2017-10-12 15:20:59 +01:00
David Baker
27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
Erik Johnston
d5694ac5fa Only log if we've removed media 2017-09-28 16:08:08 +01:00
Erik Johnston
7cc483aa0e Clear up expired url cache every 10s 2017-09-28 13:56:53 +01:00
Erik Johnston
e1e7d76cf1 Actually assign result to variable 2017-09-28 13:55:29 +01:00
Erik Johnston
5f501ec7e2 Fix typo in url cache expiry timer 2017-09-28 12:59:01 +01:00
Erik Johnston
ace8079086 Support new and old style media id formats 2017-09-28 12:52:51 +01:00
Erik Johnston
ae79764fe5 Change expires column to expires_ts 2017-09-28 12:37:53 +01:00
Erik Johnston
9ccb4226ba Delete expired url cache data 2017-09-28 12:18:06 +01:00
Erik Johnston
8090fd4664 Fix /joined_members to work with AS users 2017-09-28 10:09:32 +01:00
Erik Johnston
adec03395d Fix bug where /joined_members didn't check user was in room 2017-09-27 15:14:39 +01:00
Erik Johnston
17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Erik Johnston
b5feaa5a49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged 2017-09-19 11:07:45 +01:00
Erik Johnston
69ef4987a6 Add left section to /keys/changes 2017-09-08 14:44:36 +01:00
Erik Johnston
53cc8ad35a Send down device list change notif when member leaves/rejoins room 2017-09-07 15:08:39 +01:00
Erik Johnston
bf81f3cf2c Split out profile handler to fix tests 2017-08-25 14:34:56 +01:00
Erik Johnston
ba3ff7918b Fixup 2017-08-11 13:42:42 +01:00
Erik Johnston
ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston
b880ff190a Allow update group publicity 2017-08-08 14:19:41 +01:00
hera
eae04f1952 fix english 2017-08-04 23:56:42 +01:00
hera
5699b05072 typo 2017-08-04 23:44:37 +01:00
Erik Johnston
09552f9d9c Reduce spammy log line in synchrotrons 2017-08-02 17:29:51 +01:00
Erik Johnston
96917d5552 Merge pull request from matrix-org/erikj/group_sync_support
Add groups to sync stream
2017-07-21 11:05:39 +01:00
Erik Johnston
b238cf7f6b Remove spurious content param 2017-07-20 17:49:55 +01:00
Erik Johnston
c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston
6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston
14a34f12d7 Comments 2017-07-18 17:28:42 +01:00
Erik Johnston
3431ec55dc Comments 2017-07-18 17:23:50 +01:00
Erik Johnston
2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Krombel
85b9f76f1d split out reducing stuff; just make encode_* static 2017-07-11 13:14:35 +02:00
Krombel
2f82de18ee fix test 2017-07-10 17:34:58 +02:00
Krombel
6e16aca8b0 encode sync-response statically; omit empty objects from sync-response 2017-07-10 16:42:17 +02:00
Erik Johnston
5a7f561a9b Fix bug where pusherpool didn't start and broke some rooms
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.

This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
2017-07-06 17:55:51 +01:00
Erik Johnston
7fe8ed1787 Store URL cache preview downloads seperately
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
Erik Johnston
b8b936a6ea Add API to quarantine media 2017-06-19 17:39:21 +01:00
Erik Johnston
36e51aad3c Remove unused import 2017-06-19 14:42:21 +01:00
Erik Johnston
b490299a3b Change to create new room and join other users 2017-06-19 14:10:13 +01:00
Erik Johnston
5db7070dd1 Forget room 2017-06-19 12:40:29 +01:00
Erik Johnston
d7fe6b356c Add shutdown room API 2017-06-19 12:37:27 +01:00
Erik Johnston
a9d6fa8b2b Include users who share room with requester in user directory 2017-06-15 10:17:21 +01:00
Erik Johnston
48d2949416 Throw exception when not retrying when downloading media 2017-06-13 10:23:14 +01:00
Erik Johnston
63fda37e20 Add comments 2017-05-31 15:00:29 +01:00
Erik Johnston
535c99f157 Use POST 2017-05-31 14:15:45 +01:00
Erik Johnston
45a5df5914 Add REST API 2017-05-31 14:11:55 +01:00
Erik Johnston
9a804b2812 Merge pull request from matrix-org/matthew/fix-url-preview-length-again
actually trim oversize og:description meta
2017-05-23 13:26:28 +01:00
Matthew Hodgson
836d5c44b6 actually trim oversize og:description meta 2017-05-22 21:14:20 +01:00
Erik Johnston
b4f59c7e27 Add count of one time keys to sync stream 2017-05-19 15:47:55 +01:00
Erik Johnston
2c9475b58e Merge pull request from psaavedra/sync_timeline_limit_filter_by_name
Configurable maximum number of events requested by /sync and /messages
2017-05-15 16:08:46 +01:00
Erik Johnston
d12ae7fd1c Don't log exceptions for NotRetryingDestination 2017-05-15 15:42:18 +01:00
Pablo Saavedra
224137fcf9 Fixed syntax nits 2017-05-15 16:21:02 +02:00
Pablo Saavedra
627e6ea2b0 Fixed implementation errors
* Added HS as property in SyncRestServlet
* Fixed set_timeline_upper_limit function implementat¡ion
2017-05-15 14:51:43 +02:00
Pablo Saavedra
9da4316ca5 Configurable maximum number of events requested by /sync and /messages ()
Set the limit on the returned events in the timeline in the get and sync
operations. The default value is -1, means no upper limit.

For example, using `filter_timeline_limit: 5000`:

POST /_matrix/client/r0/user/user:id/filter
{
room: {
    timeline: {
      limit: 1000000000000000000
    }
}
}

GET /_matrix/client/r0/user/user:id/filter/filter:id

{
room: {
    timeline: {
      limit: 5000
    }
}
}

The server cuts down the room.timeline.limit.
2017-05-13 18:17:54 +02:00
Luke Barnard
369195caa5 Modify register/available to be GET with query param
- GET is now the method for register/available
- a query parameter "username" is now used

Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`
2017-05-10 17:23:55 +01:00
Luke Barnard
f7278e612e Change register/available to POST (from GET) 2017-05-10 11:40:18 +01:00
Luke Barnard
6aa423a1a8 Merge pull request from matrix-org/luke/username-availability
Implement username availability checker
2017-05-04 09:58:40 +01:00
Luke Barnard
3669065466 Appease the flake8 gods 2017-05-03 18:05:49 +01:00