diff --git a/CHANGES.rst b/CHANGES.rst new file mode 100644 index 000000000..fc6385cb1 --- /dev/null +++ b/CHANGES.rst @@ -0,0 +1,28 @@ +Changes in synapse 0.0.1 (2014-08-22) +===================================== +Presence has been disabled in this release due to a bug that caused the +homeserver to spam other remote homeservers. + +Homeserver: + * Completely change the database schema to support generic event types. + * Improve presence reliability. + * Improve reliability of joining remote rooms. + * Fix bug where room join events were duplicated. + * Improve initial sync API to return more information to the client. + * Stop generating fake messages for room membership events. + +Webclient: + * Add tab completion of names. + * Add ability to upload and send images. + * Add profile pages. + * Improve CSS layout of room. + * Disambiguate identical display names. + * Don't get remote users display names and avatars individually. + * Use the new initial sync API to reduce number of round trips to the homeserver. + * Change url scheme to use room aliases instead of room ids where known. + * Increase longpoll timeout. + +Changes in synapse 0.0.0 (2014-08-13) +===================================== + + * Initial alpha release diff --git a/README.rst b/README.rst index 378b460d0..cfdc2a1c7 100644 --- a/README.rst +++ b/README.rst @@ -24,11 +24,8 @@ To get up and running: - To run your own **private** homeserver on localhost:8080, install synapse with ``python setup.py develop --user`` and then run one with - ``python synapse/app/homeserver.py`` - - - To run your own webclient, add ``-w``: - ``python synapse/app/homeserver.py -w`` and hit http://localhost:8080/matrix/client - in your web browser (a recent Chrome, Safari or Firefox for now, + ``python synapse/app/homeserver.py`` - you will find a webclient running + at http://localhost:8080 (use a recent Chrome, Safari or Firefox for now, please...) - To make the homeserver **public** and let it exchange messages with @@ -36,7 +33,12 @@ To get up and running: up port 8080 and run ``python synapse/app/homeserver.py --host machine.my.domain.name``. Then come join ``#matrix:matrix.org`` and say hi! :) - + +For more detailed setup instructions, please see further down this document. + +[1] VoIP currently in development + + About Matrix ============ @@ -87,8 +89,6 @@ https://github.com/matrix-org/synapse/issues or at matrix@matrix.org. Thanks for trying Matrix! -[1] VoIP currently in development - [2] Cryptographic signing of messages isn't turned on yet [3] End-to-end encryption is currently in development @@ -146,6 +146,13 @@ This should end with a 'PASSED' result:: PASSED (successes=143) +Upgrading an existing homeserver +================================ + +Before upgrading an existing homeserver to a new version, please refer to +UPGRADE.rst for any additional instructions. + + Setting up Federation ===================== @@ -201,9 +208,7 @@ http://localhost:8080. Simply run:: Running The Demo Web Client =========================== -You can run the web client when you run the homeserver by adding ``-w`` to the -command to run ``homeserver.py``. The web client can be accessed via -http://localhost:8080/matrix/client +The homeserver runs a web client by default at http://localhost:8080. If this is the first time you have used the client from that browser (it uses HTML5 local storage to remember its config), you will need to log in to your diff --git a/UPGRADE.rst b/UPGRADE.rst new file mode 100644 index 000000000..2e75d77bc --- /dev/null +++ b/UPGRADE.rst @@ -0,0 +1,24 @@ +Upgrading to v0.0.1 +=================== + +This release completely changes the database schema and so requires upgrading +it before starting the new version of the homeserver. + +The script "database-prepare-for-0.0.1.sh" should be used to upgrade the +database. This will save all user information, such as logins and profiles, +but will otherwise purge the database. This includes messages, which +rooms the home server was a member of and room alias mappings. + +Before running the command the homeserver should be first completely +shutdown. To run it, simply specify the location of the database, e.g.: + + ./database-prepare-for-0.0.1.sh "homeserver.db" + +Once this has successfully completed it will be safe to restart the +homeserver. You may notice that the homeserver takes a few seconds longer to +restart than usual as it reinitializes the database. + +On startup of the new version, users can either rejoin remote rooms using room +aliases or by being reinvited. Alternatively, if any other homeserver sends a +message to a room that the homeserver was previously in the local HS will +automatically rejoin the room. diff --git a/VERSION b/VERSION new file mode 100644 index 000000000..8acdd82b7 --- /dev/null +++ b/VERSION @@ -0,0 +1 @@ +0.0.1 diff --git a/database-prepare-for-0.0.1.sh b/database-prepare-for-0.0.1.sh new file mode 100755 index 000000000..43d759a5c --- /dev/null +++ b/database-prepare-for-0.0.1.sh @@ -0,0 +1,21 @@ +#!/bin/bash + +# This is will prepare a synapse database for running with v0.0.1 of synapse. +# It will store all the user information, but will *delete* all messages and +# room data. + +set -e + +cp "$1" "$1.bak" + +DUMP=$(sqlite3 "$1" << 'EOF' +.dump users +.dump access_tokens +.dump presence +.dump profiles +EOF +) + +rm "$1" + +sqlite3 "$1" <<< "$DUMP" diff --git a/database-save.sh b/database-save.sh index c80f676f7..040c8a494 100755 --- a/database-save.sh +++ b/database-save.sh @@ -8,7 +8,7 @@ # # $ sqlite3 homeserver.db < table-save.sql -sqlite3 homeserver.db <<'EOF' >table-save.sql +sqlite3 "$1" <<'EOF' >table-save.sql .dump users .dump access_tokens .dump presence diff --git a/docs/client-server/swagger_matrix/api-docs b/docs/client-server/swagger_matrix/api-docs new file mode 100644 index 000000000..d974dbb37 --- /dev/null +++ b/docs/client-server/swagger_matrix/api-docs @@ -0,0 +1,38 @@ +{ + "apiVersion": "1.0.0", + "swaggerVersion": "1.2", + "apis": [ + { + "path": "/login", + "description": "Login operations" + }, + { + "path": "/registration", + "description": "Registration operations" + }, + { + "path": "/rooms", + "description": "Room operations" + }, + { + "path": "/profile", + "description": "Profile operations" + }, + { + "path": "/presence", + "description": "Presence operations" + } + ], + "authorizations": { + "token": { + "scopes": [] + } + }, + "info": { + "title": "Matrix Client-Server API Reference", + "description": "This contains the client-server API for the reference implementation of the home server", + "termsOfServiceUrl": "http://matrix.org", + "license": "Apache 2.0", + "licenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.html" + } +} diff --git a/docs/client-server/swagger_matrix/events b/docs/client-server/swagger_matrix/events new file mode 100644 index 000000000..c9eb3f6ff --- /dev/null +++ b/docs/client-server/swagger_matrix/events @@ -0,0 +1,299 @@ +{ + "apiVersion": "1.0.0", + "swaggerVersion": "1.2", + "basePath": "http://petstore.swagger.wordnik.com/api", + "resourcePath": "/user", + "produces": [ + "application/json" + ], + "apis": [ + { + "path": "/user", + "operations": [ + { + "method": "POST", + "summary": "Create user", + "notes": "This can only be done by the logged in user.", + "type": "void", + "nickname": "createUser", + "authorizations": { + "oauth2": [ + { + "scope": "test:anything", + "description": "anything" + } + ] + }, + "parameters": [ + { + "name": "body", + "description": "Created user object", + "required": true, + "type": "User", + "paramType": "body" + } + ] + } + ] + }, + { + "path": "/user/logout", + "operations": [ + { + "method": "GET", + "summary": "Logs out current logged in user session", + "notes": "", + "type": "void", + "nickname": "logoutUser", + "authorizations": {}, + "parameters": [] + } + ] + }, + { + "path": "/user/createWithArray", + "operations": [ + { + "method": "POST", + "summary": "Creates list of users with given input array", + "notes": "", + "type": "void", + "nickname": "createUsersWithArrayInput", + "authorizations": { + "oauth2": [ + { + "scope": "test:anything", + "description": "anything" + } + ] + }, + "parameters": [ + { + "name": "body", + "description": "List of user object", + "required": true, + "type": "array", + "items": { + "$ref": "User" + }, + "paramType": "body" + } + ] + } + ] + }, + { + "path": "/user/createWithList", + "operations": [ + { + "method": "POST", + "summary": "Creates list of users with given list input", + "notes": "", + "type": "void", + "nickname": "createUsersWithListInput", + "authorizations": { + "oauth2": [ + { + "scope": "test:anything", + "description": "anything" + } + ] + }, + "parameters": [ + { + "name": "body", + "description": "List of user object", + "required": true, + "type": "array", + "items": { + "$ref": "User" + }, + "paramType": "body" + } + ] + } + ] + }, + { + "path": "/user/{username}", + "operations": [ + { + "method": "PUT", + "summary": "Updated user", + "notes": "This can only be done by the logged in user.", + "type": "void", + "nickname": "updateUser", + "authorizations": { + "oauth2": [ + { + "scope": "test:anything", + "description": "anything" + } + ] + }, + "parameters": [ + { + "name": "username", + "description": "name that need to be deleted", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "body", + "description": "Updated user object", + "required": true, + "type": "User", + "paramType": "body" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "Invalid username supplied" + }, + { + "code": 404, + "message": "User not found" + } + ] + }, + { + "method": "DELETE", + "summary": "Delete user", + "notes": "This can only be done by the logged in user.", + "type": "void", + "nickname": "deleteUser", + "authorizations": { + "oauth2": [ + { + "scope": "test:anything", + "description": "anything" + } + ] + }, + "parameters": [ + { + "name": "username", + "description": "The name that needs to be deleted", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "Invalid username supplied" + }, + { + "code": 404, + "message": "User not found" + } + ] + }, + { + "method": "GET", + "summary": "Get user by user name", + "notes": "", + "type": "User", + "nickname": "getUserByName", + "authorizations": {}, + "parameters": [ + { + "name": "username", + "description": "The name that needs to be fetched. Use user1 for testing.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "Invalid username supplied" + }, + { + "code": 404, + "message": "User not found" + } + ] + } + ] + }, + { + "path": "/user/login", + "operations": [ + { + "method": "GET", + "summary": "Logs user into the system", + "notes": "", + "type": "string", + "nickname": "loginUser", + "authorizations": {}, + "parameters": [ + { + "name": "username", + "description": "The user name for login", + "required": true, + "type": "string", + "paramType": "query" + }, + { + "name": "password", + "description": "The password for login in clear text", + "required": true, + "type": "string", + "paramType": "query" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "Invalid username and password combination" + } + ] + } + ] + } + ], + "models": { + "User": { + "id": "User", + "properties": { + "id": { + "type": "integer", + "format": "int64" + }, + "firstName": { + "type": "string" + }, + "username": { + "type": "string" + }, + "lastName": { + "type": "string" + }, + "email": { + "type": "string" + }, + "password": { + "type": "string" + }, + "phone": { + "type": "string" + }, + "userStatus": { + "type": "integer", + "format": "int32", + "description": "User Status", + "enum": [ + "1-registered", + "2-active", + "3-closed" + ] + } + } + } + } +} \ No newline at end of file diff --git a/docs/client-server/swagger_matrix/login b/docs/client-server/swagger_matrix/login new file mode 100644 index 000000000..4410d3c88 --- /dev/null +++ b/docs/client-server/swagger_matrix/login @@ -0,0 +1,102 @@ +{ + "apiVersion": "1.0.0", + "apis": [ + { + "operations": [ + { + "method": "GET", + "nickname": "get_login_info", + "notes": "All login stages MUST be mentioned if there is >1 login type.", + "summary": "Get the login mechanism to use when logging in.", + "type": "LoginInfo" + }, + { + "method": "POST", + "nickname": "submit_login", + "notes": "If this is part of a multi-stage login, there MUST be a 'session' key.", + "parameters": [ + { + "description": "A login submission", + "name": "body", + "paramType": "body", + "required": true, + "type": "LoginSubmission" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "Bad login type" + }, + { + "code": 400, + "message": "Missing JSON keys" + } + ], + "summary": "Submit a login action.", + "type": "LoginResult" + } + ], + "path": "/login" + } + ], + "basePath": "http://localhost:8080/matrix/client/api/v1", + "consumes": [ + "application/json" + ], + "models": { + "LoginInfo": { + "id": "LoginInfo", + "properties": { + "stages": { + "description": "Multi-stage login only: An array of all the login types required to login.", + "format": "string", + "type": "array" + }, + "type": { + "description": "The login type that must be used when logging in.", + "type": "string" + } + } + }, + "LoginResult": { + "id": "LoginResult", + "properties": { + "access_token": { + "description": "The access token for this user's login if this is the final stage of the login process.", + "type": "string" + }, + "next": { + "description": "Multi-stage login only: The next login type to submit.", + "type": "string" + }, + "session": { + "description": "Multi-stage login only: The session token to send when submitting the next login type.", + "type": "string" + } + } + }, + "LoginSubmission": { + "id": "LoginSubmission", + "properties": { + "type": { + "description": "The type of login being submitted.", + "type": "string" + }, + "session": { + "description": "Multi-stage login only: The session token from an earlier login stage.", + "type": "string" + }, + "_login_type_defined_keys_": { + "description": "Keys as defined by the specified login type, e.g. \"user\", \"password\"" + } + } + } + }, + "produces": [ + "application/json" + ], + "resourcePath": "/login", + "swaggerVersion": "1.2" +} + diff --git a/docs/client-server/swagger_matrix/presence b/docs/client-server/swagger_matrix/presence new file mode 100644 index 000000000..ee9deb12f --- /dev/null +++ b/docs/client-server/swagger_matrix/presence @@ -0,0 +1,164 @@ +{ + "apiVersion": "1.0.0", + "swaggerVersion": "1.2", + "basePath": "http://localhost:8080/matrix/client/api/v1", + "resourcePath": "/presence", + "produces": [ + "application/json" + ], + "consumes": [ + "application/json" + ], + "apis": [ + { + "path": "/presence/{userId}/status", + "operations": [ + { + "method": "PUT", + "summary": "Update this user's presence state.", + "notes": "This can only be done by the logged in user.", + "type": "void", + "nickname": "update_presence", + "parameters": [ + { + "name": "body", + "description": "The new presence state", + "required": true, + "type": "PresenceUpdate", + "paramType": "body" + }, + { + "name": "userId", + "description": "The user whose presence to set.", + "required": true, + "type": "string", + "paramType": "path" + } + ] + }, + { + "method": "GET", + "summary": "Get this user's presence state.", + "notes": "Get this user's presence state.", + "type": "PresenceUpdate", + "nickname": "get_presence", + "parameters": [ + { + "name": "userId", + "description": "The user whose presence to get.", + "required": true, + "type": "string", + "paramType": "path" + } + ] + } + ] + }, + { + "path": "/presence_list/{userId}", + "operations": [ + { + "method": "GET", + "summary": "Retrieve a list of presences for all of this user's friends.", + "notes": "", + "type": "array", + "items": { + "$ref": "Presence" + }, + "nickname": "get_presence_list", + "parameters": [ + { + "name": "userId", + "description": "The user whose presence list to get.", + "required": true, + "type": "string", + "paramType": "path" + } + ] + }, + { + "method": "POST", + "summary": "Add or remove users from this presence list.", + "notes": "Add or remove users from this presence list.", + "type": "void", + "nickname": "modify_presence_list", + "parameters": [ + { + "name": "userId", + "description": "The user whose presence list is being modified.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "body", + "description": "The modifications to make to this presence list.", + "required": true, + "type": "PresenceListModifications", + "paramType": "body" + } + ] + } + ] + } + ], + "models": { + "PresenceUpdate": { + "id": "PresenceUpdate", + "properties": { + "state": { + "type": "string", + "description": "Enum: The presence state.", + "enum": [ + "offline", + "unavailable", + "online", + "free_for_chat" + ] + }, + "status_msg": { + "type": "string", + "description": "The user-defined message associated with this presence state." + } + }, + "subTypes": [ + "Presence" + ] + }, + "Presence": { + "id": "Presence", + "properties": { + "mtime_age": { + "type": "integer", + "format": "int64", + "description": "The last time this user's presence state changed, in milliseconds." + }, + "user_id": { + "type": "string", + "description": "The fully qualified user ID" + } + } + }, + "PresenceListModifications": { + "id": "PresenceListModifications", + "properties": { + "invite": { + "type": "array", + "description": "A list of user IDs to add to the list.", + "items": { + "type": "string", + "description": "A fully qualified user ID." + } + }, + "drop": { + "type": "array", + "description": "A list of user IDs to remove from the list.", + "items": { + "type": "string", + "description": "A fully qualified user ID." + } + } + } + } + } +} diff --git a/docs/client-server/swagger_matrix/profile b/docs/client-server/swagger_matrix/profile new file mode 100644 index 000000000..1ebde62e2 --- /dev/null +++ b/docs/client-server/swagger_matrix/profile @@ -0,0 +1,122 @@ +{ + "apiVersion": "1.0.0", + "swaggerVersion": "1.2", + "basePath": "http://localhost:8080/matrix/client/api/v1", + "resourcePath": "/profile", + "produces": [ + "application/json" + ], + "consumes": [ + "application/json" + ], + "apis": [ + { + "path": "/profile/{userId}/displayname", + "operations": [ + { + "method": "PUT", + "summary": "Set a display name.", + "notes": "This can only be done by the logged in user.", + "type": "void", + "nickname": "set_display_name", + "parameters": [ + { + "name": "body", + "description": "The new display name for this user.", + "required": true, + "type": "DisplayName", + "paramType": "body" + }, + { + "name": "userId", + "description": "The user whose display name to set.", + "required": true, + "type": "string", + "paramType": "path" + } + ] + }, + { + "method": "GET", + "summary": "Get a display name.", + "notes": "This can be done by anyone.", + "type": "DisplayName", + "nickname": "get_display_name", + "parameters": [ + { + "name": "userId", + "description": "The user whose display name to get.", + "required": true, + "type": "string", + "paramType": "path" + } + ] + } + ] + }, + { + "path": "/profile/{userId}/avatar_url", + "operations": [ + { + "method": "PUT", + "summary": "Set an avatar URL.", + "notes": "This can only be done by the logged in user.", + "type": "void", + "nickname": "set_avatar_url", + "parameters": [ + { + "name": "body", + "description": "The new avatar url for this user.", + "required": true, + "type": "AvatarUrl", + "paramType": "body" + }, + { + "name": "userId", + "description": "The user whose avatar url to set.", + "required": true, + "type": "string", + "paramType": "path" + } + ] + }, + { + "method": "GET", + "summary": "Get an avatar url.", + "notes": "This can be done by anyone.", + "type": "AvatarUrl", + "nickname": "get_avatar_url", + "parameters": [ + { + "name": "userId", + "description": "The user whose avatar url to get.", + "required": true, + "type": "string", + "paramType": "path" + } + ] + } + ] + } + ], + "models": { + "DisplayName": { + "id": "DisplayName", + "properties": { + "displayname": { + "type": "string", + "description": "The textual display name" + } + } + }, + "AvatarUrl": { + "id": "AvatarUrl", + "properties": { + "avatar_url": { + "type": "string", + "description": "A url to an image representing an avatar." + } + } + } + } +} diff --git a/docs/client-server/swagger_matrix/registration b/docs/client-server/swagger_matrix/registration new file mode 100644 index 000000000..ccd542d11 --- /dev/null +++ b/docs/client-server/swagger_matrix/registration @@ -0,0 +1,75 @@ +{ + "apiVersion": "1.0.0", + "apis": [ + { + "operations": [ + { + "method": "POST", + "nickname": "register", + "notes": "Volatile: This API is likely to change.", + "parameters": [ + { + "description": "A registration request", + "name": "body", + "paramType": "body", + "required": true, + "type": "RegistrationRequest" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "No JSON object." + }, + { + "code": 400, + "message": "User ID must only contain characters which do not require url encoding." + }, + { + "code": 400, + "message": "User ID already taken." + } + ], + "summary": "Register with the home server.", + "type": "RegistrationResponse" + } + ], + "path": "/register" + } + ], + "basePath": "http://localhost:8080/matrix/client/api/v1", + "consumes": [ + "application/json" + ], + "models": { + "RegistrationResponse": { + "id": "RegistrationResponse", + "properties": { + "access_token": { + "description": "The access token for this user.", + "type": "string" + }, + "user_id": { + "description": "The fully-qualified user ID.", + "type": "string" + } + } + }, + "RegistrationRequest": { + "id": "RegistrationRequest", + "properties": { + "user_id": { + "description": "The desired user ID. If not specified, a random user ID will be allocated.", + "type": "string", + "required": false + } + } + } + }, + "produces": [ + "application/json" + ], + "resourcePath": "/register", + "swaggerVersion": "1.2" +} + diff --git a/docs/client-server/swagger_matrix/rooms b/docs/client-server/swagger_matrix/rooms new file mode 100644 index 000000000..47a888724 --- /dev/null +++ b/docs/client-server/swagger_matrix/rooms @@ -0,0 +1,807 @@ +{ + "apiVersion": "1.0.0", + "swaggerVersion": "1.2", + "basePath": "http://localhost:8080/matrix/client/api/v1", + "resourcePath": "/rooms", + "produces": [ + "application/json" + ], + "consumes": [ + "application/json" + ], + "authorizations": { + "token": [] + }, + "apis": [ + { + "path": "/rooms/{roomId}/messages/{userId}/{messageId}", + "operations": [ + { + "method": "PUT", + "summary": "Send a message in this room.", + "notes": "Send a message in this room.", + "type": "void", + "nickname": "send_message", + "consumes": [ + "application/json" + ], + "parameters": [ + { + "name": "body", + "description": "The message contents", + "required": true, + "type": "Message", + "paramType": "body" + }, + { + "name": "roomId", + "description": "The room to send the message in.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "userId", + "description": "The fully qualified message sender's user ID.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "messageId", + "description": "A message ID which is unique for each room and user.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 403, + "message": "Must send messages as yourself." + } + ] + }, + { + "method": "GET", + "summary": "Get a message from this room.", + "notes": "Get a message from this room.", + "type": "Message", + "nickname": "get_message", + "parameters": [ + { + "name": "roomId", + "description": "The room to send the message in.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "userId", + "description": "The fully qualified message sender's user ID.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "messageId", + "description": "A message ID which is unique for each room and user.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 404, + "message": "Message not found." + } + ] + } + ] + }, + { + "path": "/rooms/{roomId}/topic", + "operations": [ + { + "method": "PUT", + "summary": "Set the topic for this room.", + "notes": "Set the topic for this room.", + "type": "void", + "nickname": "set_topic", + "consumes": [ + "application/json" + ], + "parameters": [ + { + "name": "body", + "description": "The topic contents", + "required": true, + "type": "Topic", + "paramType": "body" + }, + { + "name": "roomId", + "description": "The room to set the topic in.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 403, + "message": "Must send messages as yourself." + } + ] + }, + { + "method": "GET", + "summary": "Get the topic for this room.", + "notes": "Get the topic for this room.", + "type": "Topic", + "nickname": "get_topic", + "parameters": [ + { + "name": "roomId", + "description": "The room to get topic in.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 404, + "message": "Topic not found." + } + ] + } + ] + }, + { + "path": "/rooms/{roomId}/messages/{msgSenderId}/{messageId}/feedback/{senderId}/{feedbackType}", + "operations": [ + { + "method": "PUT", + "summary": "Send feedback to a message.", + "notes": "Send feedback to a message.", + "type": "void", + "nickname": "send_feedback", + "consumes": [ + "application/json" + ], + "parameters": [ + { + "name": "body", + "description": "The feedback contents", + "required": true, + "type": "Feedback", + "paramType": "body" + }, + { + "name": "roomId", + "description": "The room to send the feedback in.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "msgSenderId", + "description": "The fully qualified message sender's user ID.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "messageId", + "description": "A message ID which is unique for each room and user.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "senderId", + "description": "The fully qualified feedback sender's user ID.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "feedbackType", + "description": "The type of feedback being sent.", + "required": true, + "type": "string", + "paramType": "path", + "enum": [ + "d", + "r" + ] + } + ], + "responseMessages": [ + { + "code": 403, + "message": "Must send feedback as yourself." + }, + { + "code": 400, + "message": "Bad feedback type." + } + ] + }, + { + "method": "GET", + "summary": "Get feedback for a message.", + "notes": "Get feedback for a message.", + "type": "Feedback", + "nickname": "get_feedback", + "parameters": [ + { + "name": "roomId", + "description": "The room to send the message in.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "msgSenderId", + "description": "The fully qualified message sender's user ID.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "messageId", + "description": "A message ID which is unique for each room and user.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "senderId", + "description": "The fully qualified feedback sender's user ID.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "feedbackType", + "description": "Enum: The type of feedback being sent.", + "required": true, + "type": "string", + "paramType": "path", + "enum": [ + "d", + "r" + ] + } + ], + "responseMessages": [ + { + "code": 404, + "message": "Feedback not found." + } + ] + } + ] + }, + { + "path": "/rooms/{roomId}/members/{userId}/state", + "operations": [ + { + "method": "PUT", + "summary": "Change the membership state for a user in a room.", + "notes": "Change the membership state for a user in a room.", + "type": "void", + "nickname": "set_membership", + "consumes": [ + "application/json" + ], + "parameters": [ + { + "name": "body", + "description": "The new membership state", + "required": true, + "type": "Member", + "paramType": "body" + }, + { + "name": "userId", + "description": "The user whose membership is being changed.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "roomId", + "description": "The room which has this user.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "No membership key." + }, + { + "code": 400, + "message": "Bad membership value." + }, + { + "code": 403, + "message": "When inviting: You are not in the room." + }, + { + "code": 403, + "message": "When inviting: is already in the room." + }, + { + "code": 403, + "message": "When joining: Cannot force another user to join." + }, + { + "code": 403, + "message": "When joining: You are not invited to this room." + } + ] + }, + { + "method": "GET", + "summary": "Get the membership state of a user in a room.", + "notes": "Get the membership state of a user in a room.", + "type": "Member", + "nickname": "get_membership", + "parameters": [ + { + "name": "userId", + "description": "The user whose membership state you want to get.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "roomId", + "description": "The room which has this user.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 404, + "message": "Member not found." + } + ] + }, + { + "method": "DELETE", + "summary": "Leave a room.", + "notes": "Leave a room.", + "type": "void", + "nickname": "remove_membership", + "parameters": [ + { + "name": "userId", + "description": "The user who is leaving.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "roomId", + "description": "The room which has this user.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 403, + "message": "You are not in the room." + }, + { + "code": 403, + "message": "Cannot force another user to leave." + } + ] + } + ] + }, + { + "path": "/join/{roomAlias}", + "operations": [ + { + "method": "PUT", + "summary": "Join a room via a room alias.", + "notes": "Join a room via a room alias.", + "type": "RoomInfo", + "nickname": "join_room_via_alias", + "consumes": [ + "application/json" + ], + "parameters": [ + { + "name": "roomAlias", + "description": "The room alias to join.", + "required": true, + "type": "string", + "paramType": "path" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "Bad room alias." + } + ] + } + ] + }, + { + "path": "/rooms", + "operations": [ + { + "method": "POST", + "summary": "Create a room.", + "notes": "Create a room.", + "type": "RoomInfo", + "nickname": "create_room", + "consumes": [ + "application/json" + ], + "parameters": [ + { + "name": "body", + "description": "The desired configuration for the room.", + "required": true, + "type": "RoomConfig", + "paramType": "body" + } + ], + "responseMessages": [ + { + "code": 400, + "message": "Body must be JSON." + }, + { + "code": 400, + "message": "Room alias already taken." + } + ] + } + ] + }, + { + "path": "/rooms/{roomId}/messages/list", + "operations": [ + { + "method": "GET", + "summary": "Get a list of messages for this room.", + "notes": "Get a list of messages for this room.", + "type": "MessagePaginationChunk", + "nickname": "get_messages", + "parameters": [ + { + "name": "roomId", + "description": "The room to get messages in.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "from", + "description": "The token to start getting results from.", + "required": false, + "type": "string", + "paramType": "query" + }, + { + "name": "to", + "description": "The token to stop getting results at.", + "required": false, + "type": "string", + "paramType": "query" + }, + { + "name": "limit", + "description": "The maximum number of messages to return.", + "required": false, + "type": "integer", + "paramType": "query" + } + ] + } + ] + }, + { + "path": "/rooms/{roomId}/members/list", + "operations": [ + { + "method": "GET", + "summary": "Get a list of members for this room.", + "notes": "Get a list of members for this room.", + "type": "MemberPaginationChunk", + "nickname": "get_members", + "parameters": [ + { + "name": "roomId", + "description": "The room to get a list of members from.", + "required": true, + "type": "string", + "paramType": "path" + }, + { + "name": "from", + "description": "The token to start getting results from.", + "required": false, + "type": "string", + "paramType": "query" + }, + { + "name": "to", + "description": "The token to stop getting results at.", + "required": false, + "type": "string", + "paramType": "query" + }, + { + "name": "limit", + "description": "The maximum number of members to return.", + "required": false, + "type": "integer", + "paramType": "query" + } + ] + } + ] + } + ], + "models": { + "Topic": { + "id": "Topic", + "properties": { + "topic": { + "type": "string", + "description": "The topic text" + } + } + }, + "Message": { + "id": "Message", + "properties": { + "msgtype": { + "type": "string", + "description": "The type of message being sent, e.g. \"m.text\"", + "required": true + }, + "_msgtype_defined_keys_": { + "description": "Additional keys as defined by the msgtype, e.g. \"body\"" + } + } + }, + "Feedback": { + "id": "Feedback", + "properties": { + } + }, + "Member": { + "id": "Member", + "properties": { + "membership": { + "type": "string", + "description": "Enum: The membership state of this member.", + "enum": [ + "invite", + "join", + "leave", + "knock" + ] + } + } + }, + "RoomInfo": { + "id": "RoomInfo", + "properties": { + "room_id": { + "type": "string", + "description": "The allocated room ID.", + "required": true + }, + "room_alias": { + "type": "string", + "description": "The alias for the room.", + "required": false + } + } + }, + "RoomConfig": { + "id": "RoomConfig", + "properties": { + "visibility": { + "type": "string", + "description": "Enum: The room visibility.", + "required": false, + "enum": [ + "public", + "private" + ] + }, + "room_alias_name": { + "type": "string", + "description": "The alias to give the new room.", + "required": false + } + } + }, + "PaginationRequest": { + "id": "PaginationRequest", + "properties": { + "from": { + "type": "string", + "description": "The token to start getting results from." + }, + "to": { + "type": "string", + "description": "The token to stop getting results at." + }, + "limit": { + "type": "integer", + "description": "The maximum number of entries to return." + } + } + }, + "PaginationChunk": { + "id": "PaginationChunk", + "properties": { + "start": { + "type": "string", + "description": "A token which correlates to the first value in \"chunk\" for paginating.", + "required": true + }, + "end": { + "type": "string", + "description": "A token which correlates to the last value in \"chunk\" for paginating.", + "required": true + } + }, + "subTypes": [ + "MessagePaginationChunk" + ] + }, + "MessagePaginationChunk": { + "id": "MessagePaginationChunk", + "properties": { + "chunk": { + "type": "array", + "description": "A list of message events.", + "items": { + "$ref": "MessageEvent" + }, + "required": true + } + } + }, + "MemberPaginationChunk": { + "id": "MemberPaginationChunk", + "properties": { + "chunk": { + "type": "array", + "description": "A list of member events.", + "items": { + "$ref": "MemberEvent" + }, + "required": true + } + } + }, + "Event": { + "id": "Event", + "properties": { + "event_id": { + "type": "string", + "description": "An ID which uniquely identifies this event.", + "required": true + }, + "room_id": { + "type": "string", + "description": "The room in which this event occurred.", + "required": true + } + }, + "subTypes": [ + "MessageEvent" + ] + }, + "MessageEvent": { + "id": "MessageEvent", + "properties": { + "content": { + "type": "Message" + } + } + }, + "MemberEvent": { + "id": "MemberEvent", + "properties": { + "content": { + "type": "Member" + } + } + }, + "Tag": { + "id": "Tag", + "properties": { + "id": { + "type": "integer", + "format": "int64" + }, + "name": { + "type": "string" + } + } + }, + "Pet": { + "id": "Pet", + "required": [ + "id", + "name" + ], + "properties": { + "id": { + "type": "integer", + "format": "int64", + "description": "unique identifier for the pet", + "minimum": "0.0", + "maximum": "100.0" + }, + "category": { + "$ref": "Category" + }, + "name": { + "type": "string" + }, + "photoUrls": { + "type": "array", + "items": { + "type": "string" + } + }, + "tags": { + "type": "array", + "items": { + "$ref": "Tag" + } + }, + "status": { + "type": "string", + "description": "pet status in the store", + "enum": [ + "available", + "pending", + "sold" + ] + } + } + }, + "Category": { + "id": "Category", + "properties": { + "id": { + "type": "integer", + "format": "int64" + }, + "name": { + "type": "string" + }, + "pet": { + "$ref": "Pet" + } + } + } + } +} diff --git a/setup.py b/setup.py index fca3c7770..f01eec436 100644 --- a/setup.py +++ b/setup.py @@ -25,7 +25,7 @@ def read(fname): setup( name="SynapseHomeServer", - version="0.1", + version="0.0.1", packages=find_packages(exclude=["tests"]), description="Reference Synapse Home Server", install_requires=[ diff --git a/synapse/__init__.py b/synapse/__init__.py index 1e7b2ab27..47fc1b2ea 100644 --- a/synapse/__init__.py +++ b/synapse/__init__.py @@ -15,3 +15,5 @@ """ This is a reference implementation of a synapse home server. """ + +__version__ = "0.0.1" diff --git a/synapse/api/events/__init__.py b/synapse/api/events/__init__.py index 921fd0883..aa04dbece 100644 --- a/synapse/api/events/__init__.py +++ b/synapse/api/events/__init__.py @@ -51,6 +51,7 @@ class SynapseEvent(JsonEncodedObject): "depth", "destinations", "origin", + "outlier", ] required_keys = [ diff --git a/synapse/api/events/factory.py b/synapse/api/events/factory.py index b61dac7ac..c2cdcddf4 100644 --- a/synapse/api/events/factory.py +++ b/synapse/api/events/factory.py @@ -33,16 +33,21 @@ class EventFactory(object): RoomConfigEvent ] - def __init__(self): + def __init__(self, hs): self._event_list = {} # dict of TYPE to event class for event_class in EventFactory._event_classes: self._event_list[event_class.TYPE] = event_class + self.clock = hs.get_clock() + def create_event(self, etype=None, **kwargs): kwargs["type"] = etype if "event_id" not in kwargs: kwargs["event_id"] = random_string(10) + if "ts" not in kwargs: + kwargs["ts"] = int(self.clock.time_msec()) + if etype in self._event_list: handler = self._event_list[etype] else: diff --git a/synapse/app/homeserver.py b/synapse/app/homeserver.py index ca102236c..40e3561ee 100755 --- a/synapse/app/homeserver.py +++ b/synapse/app/homeserver.py @@ -37,6 +37,7 @@ import logging import logging.config import sqlite3 import os +import re logger = logging.getLogger(__name__) @@ -56,7 +57,7 @@ class SynapseHomeServer(HomeServer): return File("webclient") # TODO configurable? def build_resource_for_content_repo(self): - return ContentRepoResource("uploads", self.auth) + return ContentRepoResource(self, self.upload_dir, self.auth) def build_db_pool(self): """ Set up all the dbs. Since all the *.sql have IF NOT EXISTS, so we @@ -235,8 +236,8 @@ def setup(): parser.add_argument('--pid-file', dest="pid", help="When running as a " "daemon, the file to store the pid in", default="hs.pid") - parser.add_argument("-w", "--webclient", dest="webclient", - action="store_true", help="Host the web client.") + parser.add_argument("-W", "--webclient", dest="webclient", default=True, + action="store_false", help="Don't host a web client.") args = parser.parse_args() verbosity = int(args.verbose) if args.verbose else None @@ -255,9 +256,16 @@ def setup(): logger.info("Server hostname: %s", args.host) + if re.search(":[0-9]+$", args.host): + domain_with_port = args.host + else: + domain_with_port = "%s:%s" % (args.host, args.port) + hs = SynapseHomeServer( args.host, - db_name=db_name + domain_with_port=domain_with_port, + upload_dir=os.path.abspath("uploads"), + db_name=db_name, ) # This object doesn't need to be saved because it's set as the handler for diff --git a/synapse/federation/replication.py b/synapse/federation/replication.py index 8030d0963..cf634a64b 100644 --- a/synapse/federation/replication.py +++ b/synapse/federation/replication.py @@ -509,10 +509,10 @@ class _TransactionQueue(object): # a transaction in progress. If we do, stick it in the pending_pdus # table and we'll get back to it later. - destinations = [ + destinations = set([ d for d in pdu.destinations if d != self.server_name - ] + ]) logger.debug("Sending to: %s", str(destinations)) diff --git a/synapse/handlers/_base.py b/synapse/handlers/_base.py index c2f4685c9..3f07b5aa4 100644 --- a/synapse/handlers/_base.py +++ b/synapse/handlers/_base.py @@ -24,4 +24,5 @@ class BaseHandler(object): self.notifier = hs.get_notifier() self.room_lock = hs.get_room_lock_manager() self.state_handler = hs.get_state_handler() + self.distributor = hs.get_distributor() self.hs = hs diff --git a/synapse/handlers/federation.py b/synapse/handlers/federation.py index 351bb3c08..16bac9533 100644 --- a/synapse/handlers/federation.py +++ b/synapse/handlers/federation.py @@ -32,6 +32,15 @@ logger = logging.getLogger(__name__) class FederationHandler(BaseHandler): """Handles events that originated from federation.""" + def __init__(self, hs): + super(FederationHandler, self).__init__(hs) + + self.distributor.observe( + "user_joined_room", + self._on_user_joined + ) + + self.waiting_for_join_list = {} @log_function @defer.inlineCallbacks @@ -103,6 +112,13 @@ class FederationHandler(BaseHandler): if not backfilled: yield self.notifier.on_new_room_event(event, store_id) + if event.type == RoomMemberEvent.TYPE: + if event.membership == Membership.JOIN: + user = self.hs.parse_userid(event.target_user_id) + self.distributor.fire( + "user_joined_room", user=user, room_id=event.room_id + ) + @log_function @defer.inlineCallbacks @@ -152,8 +168,10 @@ class FederationHandler(BaseHandler): yield federation.handle_new_event(new_event) - store_id = yield self.store.persist_event(new_event) - self.notifier.on_new_room_event(new_event, store_id) + # TODO (erikj): Time out here. + d = defer.Deferred() + self.waiting_for_join_list.setdefault((joinee, room_id), []).append(d) + yield d try: yield self.store.store_room( @@ -166,3 +184,10 @@ class FederationHandler(BaseHandler): defer.returnValue(True) + + + @log_function + def _on_user_joined(self, user, room_id): + waiters = self.waiting_for_join_list.get((user.to_string(), room_id), []) + while waiters: + waiters.pop().callback(None) diff --git a/synapse/handlers/presence.py b/synapse/handlers/presence.py index 540e114b8..c88cc1878 100644 --- a/synapse/handlers/presence.py +++ b/synapse/handlers/presence.py @@ -142,6 +142,10 @@ class PresenceHandler(BaseHandler): @defer.inlineCallbacks def is_presence_visible(self, observer_user, observed_user): + defer.returnValue(True) + return + # FIXME (erikj): This code path absolutely kills the database. + assert(observed_user.is_mine) if observer_user == observed_user: @@ -187,6 +191,10 @@ class PresenceHandler(BaseHandler): @defer.inlineCallbacks def set_state(self, target_user, auth_user, state): + return + # TODO (erikj): Turn this back on. Why did we end up sending EDUs + # everywhere? + if not target_user.is_mine: raise SynapseError(400, "User is not hosted on this Home Server") diff --git a/synapse/handlers/room.py b/synapse/handlers/room.py index 049b4884a..5489de841 100644 --- a/synapse/handlers/room.py +++ b/synapse/handlers/room.py @@ -24,6 +24,7 @@ from synapse.api.events.room import ( RoomConfigEvent ) from synapse.api.streams.event import EventStream, EventsStreamData +from synapse.handlers.presence import PresenceStreamData from synapse.util import stringutils from ._base import BaseHandler @@ -257,21 +258,38 @@ class MessageHandler(BaseHandler): membership_list=[Membership.INVITE, Membership.JOIN] ) - ret = [] + rooms_ret = [] + + now_rooms_token = yield self.store.get_room_events_max_id() + + # FIXME (erikj): Fix this. + presence_stream = PresenceStreamData(self.hs) + now_presence_token = yield presence_stream.max_token() + presence = yield presence_stream.get_rows( + user_id, 0, now_presence_token, None, None + ) + + # FIXME (erikj): We need to not generate this token, + now_token = "%s_%s" % (now_rooms_token, now_presence_token) for event in room_list: d = { "room_id": event.room_id, "membership": event.membership, } - ret.append(d) + + if event.membership == Membership.INVITE: + d["inviter"] = event.user_id + + rooms_ret.append(d) if event.membership != Membership.JOIN: continue try: messages, token = yield self.store.get_recent_events_for_room( event.room_id, - limit=50, + limit=10, + end_token=now_rooms_token, ) d["messages"] = { @@ -279,10 +297,17 @@ class MessageHandler(BaseHandler): "start": token[0], "end": token[1], } + + current_state = yield self.store.get_current_state(event.room_id) + d["state"] = [c.get_dict() for c in current_state] except: logger.exception("Failed to get snapshot") - logger.debug("snapshot_all_rooms returning: %s", ret) + user = self.hs.parse_userid(user_id) + + ret = {"rooms": rooms_ret, "presence": presence[0], "end": now_token} + + # logger.debug("snapshot_all_rooms returning: %s", ret) defer.returnValue(ret) diff --git a/synapse/http/server.py b/synapse/http/server.py index c28d9a33f..66f966fca 100644 --- a/synapse/http/server.py +++ b/synapse/http/server.py @@ -212,8 +212,9 @@ class ContentRepoResource(resource.Resource): """ isLeaf = True - def __init__(self, directory, auth): + def __init__(self, hs, directory, auth): resource.Resource.__init__(self) + self.hs = hs self.directory = directory self.auth = auth @@ -250,7 +251,8 @@ class ContentRepoResource(resource.Resource): file_ext = re.sub("[^a-z]", "", file_ext) suffix += "." + file_ext - file_path = os.path.join(self.directory, prefix + main_part + suffix) + file_name = prefix + main_part + suffix + file_path = os.path.join(self.directory, file_name) logger.info("User %s is uploading a file to path %s", auth_user.to_string(), file_path) @@ -259,8 +261,8 @@ class ContentRepoResource(resource.Resource): attempts = 0 while os.path.exists(file_path): main_part = random_string(24) - file_path = os.path.join(self.directory, - prefix + main_part + suffix) + file_name = prefix + main_part + suffix + file_path = os.path.join(self.directory, file_name) attempts += 1 if attempts > 25: # really? Really? raise SynapseError(500, "Unable to create file.") @@ -272,11 +274,14 @@ class ContentRepoResource(resource.Resource): # servers. # TODO: A little crude here, we could do this better. - filename = request.path.split(self.directory + "/")[1] + filename = request.path.split('/')[-1] # be paranoid filename = re.sub("[^0-9A-z.-_]", "", filename) file_path = self.directory + "/" + filename + + logger.debug("Searching for %s", file_path) + if os.path.isfile(file_path): # filename has the content type base64_contentype = filename.split(".")[1] @@ -304,6 +309,10 @@ class ContentRepoResource(resource.Resource): self._async_render(request) return server.NOT_DONE_YET + def render_OPTIONS(self, request): + respond_with_json_bytes(request, 200, {}, send_cors=True) + return server.NOT_DONE_YET + @defer.inlineCallbacks def _async_render(self, request): try: @@ -313,8 +322,15 @@ class ContentRepoResource(resource.Resource): with open(fname, "wb") as f: f.write(request.content.read()) + + # FIXME (erikj): These should use constants. + file_name = os.path.basename(fname) + url = "http://%s/matrix/content/%s" % ( + self.hs.domain_with_port, file_name + ) + respond_with_json_bytes(request, 200, - json.dumps({"content_token": fname}), + json.dumps({"content_token": url}), send_cors=True) except CodeMessageException as e: diff --git a/synapse/rest/register.py b/synapse/rest/register.py index eb457562b..f17ec11cf 100644 --- a/synapse/rest/register.py +++ b/synapse/rest/register.py @@ -33,10 +33,10 @@ class RegisterRestServlet(RestServlet): try: register_json = json.loads(request.content.read()) if "password" in register_json: - password = register_json["password"] + password = register_json["password"].encode("utf-8") if type(register_json["user_id"]) == unicode: - desired_user_id = register_json["user_id"] + desired_user_id = register_json["user_id"].encode("utf-8") if urllib.quote(desired_user_id) != desired_user_id: raise SynapseError( 400, diff --git a/synapse/server.py b/synapse/server.py index d4c248148..c5b0a3275 100644 --- a/synapse/server.py +++ b/synapse/server.py @@ -159,7 +159,7 @@ class HomeServer(BaseHomeServer): return DataStore(self) def build_event_factory(self): - return EventFactory() + return EventFactory(self) def build_handlers(self): return Handlers(self) diff --git a/synapse/storage/__init__.py b/synapse/storage/__init__.py index 773290692..d06033b98 100644 --- a/synapse/storage/__init__.py +++ b/synapse/storage/__init__.py @@ -105,6 +105,11 @@ class DataStore(RoomMemberStore, RoomStore, "processed": True, } + if hasattr(event, "outlier"): + vals["outlier"] = event.outlier + else: + vals["outlier"] = False + if backfilled: if not self.min_token_deferred.called: yield self.min_token_deferred @@ -123,7 +128,7 @@ class DataStore(RoomMemberStore, RoomStore, except: logger.exception( "Failed to persist, probably duplicate: %s", - event_id + event.event_id ) return diff --git a/synapse/storage/_base.py b/synapse/storage/_base.py index 36cc57c1b..75aab2d3b 100644 --- a/synapse/storage/_base.py +++ b/synapse/storage/_base.py @@ -294,6 +294,11 @@ class SQLBaseStore(object): def _parse_event_from_row(self, row_dict): d = copy.deepcopy({k: v for k, v in row_dict.items() if v}) + + d.pop("stream_ordering", None) + d.pop("topological_ordering", None) + d.pop("processed", None) + d.update(json.loads(row_dict["unrecognized_keys"])) d["content"] = json.loads(d["content"]) del d["unrecognized_keys"] diff --git a/synapse/storage/roommember.py b/synapse/storage/roommember.py index 1f8984b6e..a9a09e142 100644 --- a/synapse/storage/roommember.py +++ b/synapse/storage/roommember.py @@ -146,7 +146,7 @@ class RoomMemberStore(SQLBaseStore): rows = yield self._execute_and_decode(sql, *where_values) - logger.debug("_get_members_query Got rows %s", rows) + # logger.debug("_get_members_query Got rows %s", rows) results = [self._parse_event_from_row(r) for r in rows] defer.returnValue(results) diff --git a/synapse/storage/schema/im.sql b/synapse/storage/schema/im.sql index ea04261ff..e92f21ef3 100644 --- a/synapse/storage/schema/im.sql +++ b/synapse/storage/schema/im.sql @@ -22,9 +22,15 @@ CREATE TABLE IF NOT EXISTS events( content TEXT NOT NULL, unrecognized_keys TEXT, processed BOOL NOT NULL, + outlier BOOL NOT NULL, CONSTRAINT ev_uniq UNIQUE (event_id) ); +CREATE INDEX IF NOT EXISTS events_event_id ON events (event_id); +CREATE INDEX IF NOT EXISTS events_stream_ordering ON events (stream_ordering); +CREATE INDEX IF NOT EXISTS events_topological_ordering ON events (topological_ordering); +CREATE INDEX IF NOT EXISTS events_room_id ON events (room_id); + CREATE TABLE IF NOT EXISTS state_events( event_id TEXT NOT NULL, room_id TEXT NOT NULL, @@ -33,6 +39,12 @@ CREATE TABLE IF NOT EXISTS state_events( prev_state TEXT ); +CREATE UNIQUE INDEX IF NOT EXISTS state_events_event_id ON state_events (event_id); +CREATE INDEX IF NOT EXISTS state_events_room_id ON state_events (room_id); +CREATE INDEX IF NOT EXISTS state_events_type ON state_events (type); +CREATE INDEX IF NOT EXISTS state_events_state_key ON state_events (state_key); + + CREATE TABLE IF NOT EXISTS current_state_events( event_id TEXT NOT NULL, room_id TEXT NOT NULL, @@ -41,6 +53,11 @@ CREATE TABLE IF NOT EXISTS current_state_events( CONSTRAINT curr_uniq UNIQUE (room_id, type, state_key) ON CONFLICT REPLACE ); +CREATE INDEX IF NOT EXISTS curr_events_event_id ON current_state_events (event_id); +CREATE INDEX IF NOT EXISTS current_state_events_room_id ON current_state_events (room_id); +CREATE INDEX IF NOT EXISTS current_state_events_type ON current_state_events (type); +CREATE INDEX IF NOT EXISTS current_state_events_state_key ON current_state_events (state_key); + CREATE TABLE IF NOT EXISTS room_memberships( event_id TEXT NOT NULL, user_id TEXT NOT NULL, @@ -49,6 +66,10 @@ CREATE TABLE IF NOT EXISTS room_memberships( membership TEXT NOT NULL ); +CREATE INDEX IF NOT EXISTS room_memberships_event_id ON room_memberships (event_id); +CREATE INDEX IF NOT EXISTS room_memberships_room_id ON room_memberships (room_id); +CREATE INDEX IF NOT EXISTS room_memberships_user_id ON room_memberships (user_id); + CREATE TABLE IF NOT EXISTS feedback( event_id TEXT NOT NULL, feedback_type TEXT, @@ -77,5 +98,6 @@ CREATE TABLE IF NOT EXISTS rooms( CREATE TABLE IF NOT EXISTS room_hosts( room_id TEXT NOT NULL, - host TEXT NOT NULL + host TEXT NOT NULL, + CONSTRAINT room_hosts_uniq UNIQUE (room_id, host) ON CONFLICT IGNORE ); diff --git a/synapse/storage/stream.py b/synapse/storage/stream.py index e994017bf..3a17a723f 100644 --- a/synapse/storage/stream.py +++ b/synapse/storage/stream.py @@ -177,6 +177,7 @@ class StreamStore(SQLBaseStore): "((room_id IN (%(current)s)) OR " "(event_id IN (%(invites)s))) " "AND e.stream_ordering > ? AND e.stream_ordering < ? " + "AND e.outlier = 0 " "ORDER BY stream_ordering ASC LIMIT %(limit)d " ) % { "current": current_room_membership_sql, @@ -224,7 +225,7 @@ class StreamStore(SQLBaseStore): sql = ( "SELECT * FROM events " - "WHERE room_id = ? AND %(bounds)s " + "WHERE outlier = 0 AND room_id = ? AND %(bounds)s " "ORDER BY topological_ordering %(order)s, stream_ordering %(order)s %(limit)s " ) % {"bounds": bounds, "order": order, "limit": limit_str} @@ -249,15 +250,14 @@ class StreamStore(SQLBaseStore): ) @defer.inlineCallbacks - def get_recent_events_for_room(self, room_id, limit, with_feedback=False): + def get_recent_events_for_room(self, room_id, limit, end_token, + with_feedback=False): # TODO (erikj): Handle compressed feedback - end_token = yield self.get_room_events_max_id() - sql = ( "SELECT * FROM events " "WHERE room_id = ? AND stream_ordering <= ? " - "ORDER BY topological_ordering, stream_ordering DESC LIMIT ? " + "ORDER BY topological_ordering DESC, stream_ordering DESC LIMIT ? " ) rows = yield self._execute_and_decode( diff --git a/tests/handlers/test_presence.py b/tests/handlers/test_presence.py index 8b88c49a0..6d3cd76db 100644 --- a/tests/handlers/test_presence.py +++ b/tests/handlers/test_presence.py @@ -190,6 +190,7 @@ class PresenceStateTestCase(unittest.TestCase): ), SynapseError ) + test_get_disallowed_state.skip = "Presence polling is disabled" @defer.inlineCallbacks def test_set_my_state(self): @@ -214,6 +215,7 @@ class PresenceStateTestCase(unittest.TestCase): state={"state": OFFLINE}) self.mock_stop.assert_called_with(self.u_apple) + test_set_my_state.skip = "Presence polling is disabled" class PresenceInvitesTestCase(unittest.TestCase): @@ -653,6 +655,7 @@ class PresencePushTestCase(unittest.TestCase): observed_user=self.u_banana, statuscache=ANY), # self-reflection ]) # and no others... + test_push_local.skip = "Presence polling is disabled" @defer.inlineCallbacks def test_push_remote(self): @@ -704,6 +707,7 @@ class PresencePushTestCase(unittest.TestCase): ) yield put_json.await_calls() + test_push_remote.skip = "Presence polling is disabled" @defer.inlineCallbacks def test_recv_remote(self): @@ -996,6 +1000,8 @@ class PresencePollingTestCase(unittest.TestCase): self.assertFalse("banana" in self.handler._local_pushmap) self.assertFalse("clementine" in self.handler._local_pushmap) + test_push_local.skip = "Presence polling is disabled" + @defer.inlineCallbacks def test_remote_poll_send(self): @@ -1044,6 +1050,7 @@ class PresencePollingTestCase(unittest.TestCase): put_json.await_calls() self.assertFalse(self.u_potato in self.handler._remote_recvmap) + test_remote_poll_send.skip = "Presence polling is disabled" @defer.inlineCallbacks def test_remote_poll_receive(self): diff --git a/tests/handlers/test_presencelike.py b/tests/handlers/test_presencelike.py index bba5dd4e5..c25c6889b 100644 --- a/tests/handlers/test_presencelike.py +++ b/tests/handlers/test_presencelike.py @@ -135,6 +135,7 @@ class PresenceProfilelikeDataTestCase(unittest.TestCase): mocked_set.assert_called_with("apple", {"state": UNAVAILABLE, "status_msg": "Away"}) + test_set_my_state.skip = "Presence polling is disabled" @defer.inlineCallbacks def test_push_local(self): @@ -209,6 +210,8 @@ class PresenceProfilelikeDataTestCase(unittest.TestCase): "displayname": "I am an Apple", "avatar_url": "http://foo", }, statuscache.state) + test_push_local.skip = "Presence polling is disabled" + @defer.inlineCallbacks def test_push_remote(self): @@ -239,6 +242,7 @@ class PresenceProfilelikeDataTestCase(unittest.TestCase): ], }, ) + test_push_remote.skip = "Presence polling is disabled" @defer.inlineCallbacks def test_recv_remote(self): diff --git a/tests/rest/test_presence.py b/tests/rest/test_presence.py index 8ac246b4d..970405d27 100644 --- a/tests/rest/test_presence.py +++ b/tests/rest/test_presence.py @@ -114,6 +114,7 @@ class PresenceStateTestCase(unittest.TestCase): self.assertEquals(200, code) mocked_set.assert_called_with("apple", {"state": UNAVAILABLE, "status_msg": "Away"}) + test_set_my_status.skip = "Presence polling is disabled" class PresenceListTestCase(unittest.TestCase): @@ -309,3 +310,4 @@ class PresenceEventStreamTestCase(unittest.TestCase): "mtime_age": 0, }}, ]}, response) + test_shortpoll.skip = "Presence polling is disabled" diff --git a/webclient/app-controller.js b/webclient/app-controller.js index 96656e12c..84cb94dc7 100644 --- a/webclient/app-controller.js +++ b/webclient/app-controller.js @@ -31,31 +31,15 @@ angular.module('MatrixWebClientController', ['matrixService']) $rootScope.$on('$routeChangeSuccess', function (event, current, previous) { $scope.location = $location.path(); }); - - - // Manage the display of the current config - $scope.config; - - // Toggles the config display - $scope.showConfig = function() { - if ($scope.config) { - $scope.config = undefined; - } - else { - $scope.config = matrixService.config(); - } - }; - - $scope.closeConfig = function() { - if ($scope.config) { - $scope.config = undefined; - } - }; if (matrixService.isUserLoggedIn()) { - eventStreamService.resume(); + // eventStreamService.resume(); } + $scope.go = function(url) { + $location.url(url); + }; + // Logs the user out $scope.logout = function() { // kill the event stream @@ -66,7 +50,7 @@ angular.module('MatrixWebClientController', ['matrixService']) matrixService.saveConfig(); // And go to the login page - $location.path("login"); + $location.url("login"); }; // Listen to the event indicating that the access token is no longer valid. diff --git a/webclient/app-filter.js b/webclient/app-filter.js index 64c3bb04d..b8f4ed25b 100644 --- a/webclient/app-filter.js +++ b/webclient/app-filter.js @@ -54,12 +54,15 @@ angular.module('matrixWebClient') }); // FIXME: we shouldn't disambiguate displayNames on every orderMembersList - // invocation but keep track of duplicates incrementally somewhere + // invocation but keep track of duplicates incrementally somewhere angular.forEach(displayNames, function(value, key) { if (value.length > 1) { // console.log(key + ": " + value); - for (i=0; i < value.length; i++) { + for (var i=0; i < value.length; i++) { var v = value[i]; + // FIXME: this permenantly rewrites the displayname for a given + // room member. which means we can't reset their name if it is + // no longer ambiguous! members[v].displayname += " (" + v + ")"; // console.log(v + " " + members[v]); }; diff --git a/webclient/app.css b/webclient/app.css index d2b951d3b..dfa17fae6 100644 --- a/webclient/app.css +++ b/webclient/app.css @@ -1,3 +1,71 @@ +/*** Mobile voodoo ***/ +@media all and (max-device-width: 640px) { + + #messageTableWrapper { + margin-right: 0px ! important; + } + + .leftBlock { + width: 8em ! important; + } + + #header, + #messageTable, + #wrapper, + #roomName, + #controls { + max-width: 640px ! important; + } + + #userIdCell, + #usersTableWrapper, + #extraControls { + display: none; + } + + #buttonsCell { + width: 60px ! important; + padding-left: 20px ! important; + } + + #roomLogo { + display: none; + } + + #roomName { + text-align: left ! important; + top: -35px ! important; + } + + .bubble { + font-size: 12px ! important; + min-height: 20px ! important; + } + + #page { + top: 35px ! important; + bottom: 70px ! important; + } + + #header, + #page { + margin: 5px ! important; + } + + #header { + padding: 5px ! important; + } + + /* stop zoom on select */ + select:focus, + textarea, + input + { + font-size: 16px ! important; + } + +} + body { font-family: "Myriad Pro", "Myriad", Helvetica, Arial, sans-serif; font-size: 12pt; @@ -17,7 +85,6 @@ h1 { left: 0px; right: 0px; margin: 20px; - margin: 20px; } #wrapper { @@ -32,8 +99,7 @@ h1 { text-align: right; top: -40px; position: absolute; - font-size: 16pt; - margin-bottom: 10px; + font-size: 16px; } #controlPanel { @@ -50,6 +116,10 @@ h1 { margin: auto; } +#buttonsCell { + width: 150px; +} + #inputBarTable { width: 100%; } @@ -111,13 +181,13 @@ h1 { color: #fff; margin: 2px; bottom: 0px; - font-size: 8pt; + font-size: 12px; word-break: break-all; } .userPresence { text-align: center; - font-size: 8pt; + font-size: 12px; color: #fff; background-color: #aaa; border-bottom: 1px #ddd solid; @@ -145,6 +215,7 @@ h1 { max-width: 1280px; width: 100%; border-collapse: collapse; + table-layout: fixed; } #messageTable td { @@ -152,12 +223,13 @@ h1 { } .leftBlock { - width: 10em; + width: 14em; + word-wrap: break-word; vertical-align: top; background-color: #fff; color: #888; font-weight: medium; - font-size: 8pt; + font-size: 12px; text-align: right; border-top: 1px #ddd solid; } @@ -190,24 +262,13 @@ h1 { object-fit: cover; } -.text { - background-color: #eee; - border: 1px solid #d8d8d8; - height: 31px; - display: inline-table; - max-width: 90%; - font-size: 16px; - /* word-wrap: break-word; */ - word-break: break-all; -} - .emote { - background-color: #fff ! important; + background-color: transparent ! important; border: 0px ! important; } .membership { - background-color: #fff ! important; + background-color: transparent ! important; border: 0px ! important; } @@ -219,32 +280,45 @@ h1 { height: auto; } +.text { + vertical-align: top; +} + .bubble { + background-color: #eee; + border: 1px solid #d8d8d8; + display: inline-block; + margin-bottom: -1px; + max-width: 90%; + font-size: 16px; + word-wrap: break-word; padding-top: 7px; padding-bottom: 5px; padding-left: 1em; padding-right: 1em; vertical-align: middle; + -webkit-text-size-adjust:100% } .differentUser td { - padding-top: 5px ! important; - margin-top: 5px ! important; + padding-bottom: 5px ! important; } .mine { text-align: right; } -.mine .text { - background-color: #f8f8ff ! important; -} - -.mine .emote { - background-color: #fff ! important; +.text.emote .bubble, +.text.membership .bubble, +.mine .text.emote .bubble, +.mine .text.membership .bubble + { + background-color: transparent ! important; + border: 0px ! important; } .mine .text .bubble { + background-color: #f8f8ff ! important; text-align: left ! important; } @@ -273,7 +347,7 @@ h1 { .profile-avatar { width: 160px; height: 160px; - display:table-cell; + display: table-cell; vertical-align: middle; text-align: center; } @@ -289,13 +363,19 @@ h1 { } #user-displayname { - font-size: 16pt; + font-size: 24px; } /******************************/ -#header { - padding-left: 20px; - padding-right: 20px; +#header +{ + padding: 20px; + max-width: 1280px; + margin: auto; +} + +#logo, +#roomLogo { max-width: 1280px; margin: auto; } @@ -304,18 +384,6 @@ h1 { float: right; } -#config { - position: absolute; - z-index: 100; - top: 100px; - left: 50%; - width: 500px; - margin-left: -250px; - text-align: center; - padding: 20px; - background-color: #aaa; -} - .text_entry_section { position: fixed; bottom: 0; diff --git a/webclient/app.js b/webclient/app.js index f27ebedc6..6cd50c5e5 100644 --- a/webclient/app.js +++ b/webclient/app.js @@ -19,7 +19,8 @@ var matrixWebClient = angular.module('matrixWebClient', [ 'MatrixWebClientController', 'LoginController', 'RoomController', - 'RoomsController', + 'HomeController', + 'SettingsController', 'UserController', 'matrixService', 'eventStreamService', @@ -44,16 +45,20 @@ matrixWebClient.config(['$routeProvider', '$provide', '$httpProvider', templateUrl: 'room/room.html', controller: 'RoomController' }). - when('/rooms', { - templateUrl: 'rooms/rooms.html', - controller: 'RoomsController' + when('/', { + templateUrl: 'home/home.html', + controller: 'HomeController' + }). + when('/settings', { + templateUrl: 'settings/settings.html', + controller: 'SettingsController' }). when('/user/:user_matrix_id', { templateUrl: 'user/user.html', controller: 'UserController' }). otherwise({ - redirectTo: '/rooms' + redirectTo: '/' }); $provide.factory('AccessTokenInterceptor', ['$q', '$rootScope', @@ -80,6 +85,6 @@ matrixWebClient.run(['$location', 'matrixService', 'eventStreamService', functio $location.path("login"); } else { - eventStreamService.resume(); + // eventStreamService.resume(); } }]); diff --git a/webclient/components/fileUpload/file-upload-service.js b/webclient/components/fileUpload/file-upload-service.js index 6606f31e2..5f01478fd 100644 --- a/webclient/components/fileUpload/file-upload-service.js +++ b/webclient/components/fileUpload/file-upload-service.js @@ -33,7 +33,7 @@ angular.module('mFileUpload', ['matrixService', 'mUtilities']) console.log("Uploading " + file.name + "... to /matrix/content"); matrixService.uploadContent(file).then( function(response) { - var content_url = location.origin + "/matrix/content/" + response.data.content_token; + var content_url = response.data.content_token; console.log(" -> Successfully uploaded! Available at " + content_url); deferred.resolve(content_url); }, @@ -82,6 +82,7 @@ angular.module('mFileUpload', ['matrixService', 'mUtilities']) // First, get the image size mUtilities.getImageSize(imageFile).then( function(size) { + console.log("image size: " + JSON.stringify(size)); // The final operation: send imageFile var uploadImage = function() { diff --git a/webclient/components/matrix/event-handler-service.js b/webclient/components/matrix/event-handler-service.js index b8529895f..b5eb73d92 100644 --- a/webclient/components/matrix/event-handler-service.js +++ b/webclient/components/matrix/event-handler-service.js @@ -35,6 +35,8 @@ angular.module('eventHandlerService', []) $rootScope.events = { rooms: {}, // will contain roomId: { messages:[], members:{userid1: event} } }; + + $rootScope.presence = {}; var initRoom = function(room_id) { if (!(room_id in $rootScope.events.rooms)) { @@ -44,6 +46,12 @@ angular.module('eventHandlerService', []) $rootScope.events.rooms[room_id].members = {}; } } + + var reInitRoom = function(room_id) { + $rootScope.events.rooms[room_id] = {}; + $rootScope.events.rooms[room_id].messages = []; + $rootScope.events.rooms[room_id].members = {}; + } var handleMessage = function(event, isLiveEvent) { if ("membership_target" in event.content) { @@ -69,11 +77,23 @@ angular.module('eventHandlerService', []) var handleRoomMember = function(event, isLiveEvent) { initRoom(event.room_id); + + // add membership changes as if they were a room message if something interesting changed + if (event.content.prev !== event.content.membership) { + if (isLiveEvent) { + $rootScope.events.rooms[event.room_id].messages.push(event); + } + else { + $rootScope.events.rooms[event.room_id].messages.unshift(event); + } + } + $rootScope.events.rooms[event.room_id].members[event.user_id] = event; $rootScope.$broadcast(MEMBER_EVENT, event, isLiveEvent); }; var handlePresence = function(event, isLiveEvent) { + $rootScope.presence[event.content.user_id] = event; $rootScope.$broadcast(PRESENCE_EVENT, event, isLiveEvent); }; @@ -107,6 +127,10 @@ angular.module('eventHandlerService', []) for (var i=0; i height) { - if (width > MAX_WIDTH) { - height *= MAX_WIDTH / width; - width = MAX_WIDTH; + if (width > height) { + if (width > MAX_WIDTH) { + height *= MAX_WIDTH / width; + width = MAX_WIDTH; + } + } else { + if (height > MAX_HEIGHT) { + width *= MAX_HEIGHT / height; + height = MAX_HEIGHT; + } } - } else { - if (height > MAX_HEIGHT) { - width *= MAX_HEIGHT / height; - height = MAX_HEIGHT; - } - } - canvas.width = width; - canvas.height = height; - var ctx = canvas.getContext("2d"); - ctx.drawImage(img, 0, 0, width, height); + canvas.width = width; + canvas.height = height; + var ctx = canvas.getContext("2d"); + ctx.drawImage(img, 0, 0, width, height); - var dataUrl = canvas.toDataURL("image/jpeg", 0.7); - deferred.resolve(self.dataURItoBlob(dataUrl)); + // Extract image data in the same format as the original one. + // The 0.7 compression value will work with formats that supports it like JPEG. + var dataUrl = canvas.toDataURL(imageFile.type, 0.7); + deferred.resolve(self.dataURItoBlob(dataUrl)); + }; + img.onerror = function(e) { + deferred.reject(e); + }; }; reader.onerror = function(e) { deferred.reject(e); diff --git a/webclient/home/home-controller.js b/webclient/home/home-controller.js new file mode 100644 index 000000000..35d0ef165 --- /dev/null +++ b/webclient/home/home-controller.js @@ -0,0 +1,162 @@ +/* +Copyright 2014 matrix.org + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +'use strict'; + +angular.module('HomeController', ['matrixService', 'mFileInput', 'mFileUpload', 'eventHandlerService']) +.controller('HomeController', ['$scope', '$location', 'matrixService', 'mFileUpload', 'eventHandlerService', 'eventStreamService', + function($scope, $location, matrixService, mFileUpload, eventHandlerService, eventStreamService) { + + $scope.config = matrixService.config(); + $scope.rooms = {}; + $scope.public_rooms = []; + $scope.newRoomId = ""; + $scope.feedback = ""; + + $scope.newRoom = { + room_id: "", + private: false + }; + + $scope.goToRoom = { + room_id: "", + }; + + $scope.joinAlias = { + room_alias: "", + }; + + $scope.$on(eventHandlerService.MEMBER_EVENT, function(ngEvent, event, isLive) { + var config = matrixService.config(); + if (event.target_user_id === config.user_id && event.content.membership === "invite") { + console.log("Invited to room " + event.room_id); + // FIXME push membership to top level key to match /im/sync + event.membership = event.content.membership; + // FIXME bodge a nicer name than the room ID for this invite. + event.room_display_name = event.user_id + "'s room"; + $scope.rooms[event.room_id] = event; + } + }); + + var assignRoomAliases = function(data) { + for (var i=0; i + +
+
+ +
+
+ + + + + +
+
+ +
+
+
+
{{ config.displayName }}
+
{{ config.user_id }}
+
+
+
+
+ +

My rooms

+ +
+
+ {{ room.room_display_name }} {{room.membership === 'invite' ? ' (invited)' : ''}} +
+
+
+ +

Public rooms

+ + +
+ +
+
+ + private + +
+
+
+
+ + +
+
+
+ + {{ feedback }} + +
+
+ diff --git a/webclient/index.html b/webclient/index.html index 27d920819..938d70c86 100644 --- a/webclient/index.html +++ b/webclient/index.html @@ -2,10 +2,12 @@ [matrix] - + + + @@ -15,10 +17,11 @@ + - + @@ -33,22 +36,11 @@ -
-
Home server: {{ config.homeserver }}
-
User ID: {{ config.user_id }}
-
Access token: {{ config.access_token }}
-
-
-
- -
diff --git a/webclient/login/login-controller.js b/webclient/login/login-controller.js index 35886c558..51f9a3bdf 100644 --- a/webclient/login/login-controller.js +++ b/webclient/login/login-controller.js @@ -53,7 +53,7 @@ angular.module('LoginController', ['matrixService']) matrixService.saveConfig(); eventStreamService.resume(); // Go to the user's rooms list page - $location.path("rooms"); + $location.url("home"); }, function(error) { if (error.data) { @@ -86,7 +86,7 @@ angular.module('LoginController', ['matrixService']) }); matrixService.saveConfig(); eventStreamService.resume(); - $location.path("rooms"); + $location.url("home"); } else { $scope.feedback = "Failed to login: " + JSON.stringify(response.data); diff --git a/webclient/login/login.html b/webclient/login/login.html index b1488b37f..4b2ea6092 100644 --- a/webclient/login/login.html +++ b/webclient/login/login.html @@ -1,4 +1,6 @@ -