Commit Graph

6989 Commits

Author SHA1 Message Date
Matthew Hodgson
b31ed22738 Merge branch 'release-v0.18.6' into develop 2017-01-05 13:03:02 +00:00
Mark Haines
dd3df11c55 More logging for the linearizer and for get_events 2017-01-05 12:32:47 +00:00
Matthew Hodgson
468749c9fc fix comment 2017-01-05 12:00:11 +00:00
Matthew Hodgson
eedf400d05 limit total timeout for get_missing_events to 10s 2017-01-05 11:58:15 +00:00
Matthew Hodgson
8e82611f37 fix comment 2017-01-05 11:44:44 +00:00
Mark Haines
f784980d2b Only send events that originate on this server.
Or events that are sent via the federation "send_join" API.

This should match the behaviour from before v0.18.5 and #1635 landed.
2017-01-05 11:26:30 +00:00
Mark Haines
e02bdaf08b Get the destinations from the state from before the event
Rather than the state after then event.
2017-01-04 15:17:15 +00:00
Mark Haines
b6b67715ed Send ALL membership events to the server that was affected.
Send all membership changes to the server that was affected.
This ensures that if the last member of a room on a server
was kicked or banned they get told about it.
2017-01-04 13:56:20 +00:00
Matthew Hodgson
555d702e34 limit total timeout for get_missing_events to 10s 2016-12-31 15:21:37 +00:00
Matthew Hodgson
899a3a1268 Merge branch 'release-v0.18.6' into develop 2016-12-31 02:38:26 +00:00
Mark Haines
f3de4f8cb7 Bump version and changelog 2016-12-30 20:21:04 +00:00
Mark Haines
62ce3034f3 s/aquire/acquire/g 2016-12-30 20:04:44 +00:00
Mark Haines
0aff09f6c9 Add more useful logging when we block fetching events 2016-12-30 20:00:44 +00:00
Mark Haines
cc50b1ae53 Remove fallback from get_missing_events.
get_missing_events used to fallback to fetching the missing events
individually requesting from every server in the room, one by one.e

This could be unacceptably slow, possibly causing #1732
2016-12-30 18:13:15 +00:00
Mark Haines
f576c34594 Merge remote-tracking branch 'origin/release-v0.18.6' into develop 2016-12-30 15:13:49 +00:00
Mark Haines
822cb39dfa Use the new twisted logging framework.
Hopefully adding an observer to the new framework will avoid a memory
leak https://twistedmatrix.com/trac/ticket/8164
2016-12-30 11:09:24 +00:00
Mark Haines
342fb8dae9 Merge branch 'release-v0.18.6' into develop 2016-12-29 17:33:46 +00:00
Mark Haines
f023be9293 Bump changelog and version 2016-12-29 16:18:04 +00:00
Mark Haines
97ffc5690b Manually abort the underlying TLS connection.
The abort() method calls loseConnection() which tries to shutdown the
TLS connection cleanly. We now call abortConnection() directly which
should promptly close both the TLS connection and the underlying TCP
connection.

I also added some TODO markers to consider cancelling the old previous
timeout rather than checking time.time(). But given how urgently we want
to get this code released I'd rather leave the existing code with the
duplicate timeouts and the time.time() check.
2016-12-29 15:51:04 +00:00
Erik Johnston
b4bc6fef5b Respect long_retries param and default to off 2016-12-29 00:58:34 +00:00
Erik Johnston
68030fd37b Spelling and comments 2016-12-29 00:10:49 +00:00
Erik Johnston
b7336ff32d Clean up 2016-12-29 00:09:33 +00:00
Erik Johnston
5b6672c66d Wrap connections in an N minute timeout to ensure they get reaped correctly 2016-12-29 00:06:53 +00:00
David Baker
84cf00c645 Fix another comment typo 2016-12-21 09:51:43 +00:00
David Baker
0c88ab1844 Add /account/3pid/delete endpoint
Also fix a typo in a comment
2016-12-20 18:27:30 +00:00
Johannes Löthberg
702c020e58 Fix check for bind_address
The empty string is a valid setting for the bind_address option, so
explicitly check for None here instead.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-20 01:37:50 +01:00
Matthew Hodgson
09f15918be Merge pull request #1711 from matrix-org/matthew/utf8-password-change
fix ability to change password to a non-ascii one
2016-12-20 00:02:13 +00:00
Matthew Hodgson
da2c8f3c94 Merge pull request #1709 from kyrias/bind_addresses
Add support for specifying multiple bind addresses
2016-12-19 23:49:34 +00:00
Matthew Hodgson
a58e4e0d48 Merge pull request #1696 from kyrias/ipv6
IPv6 support
2016-12-19 23:49:07 +00:00
Matthew Hodgson
f2a5aebf98 fix ability to change password to a non-ascii one
https://github.com/vector-im/riot-web/issues/2658
2016-12-18 22:25:21 +00:00
Johannes Löthberg
a9c1b419a9 Bump twisted dependency
At least 16.0.0 is needed for wrapClientTLS support.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 23:16:43 +01:00
Johannes Löthberg
f5cd5ebd7b Add IPv6 comment to default config
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 23:14:32 +01:00
Johannes Löthberg
c95e9fff99 Make default homeserver config use bind_addresses
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 21:51:56 +01:00
Johannes Löthberg
7dfd70fc83 Add support for specifying multiple bind addresses
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 21:51:56 +01:00
Erik Johnston
b2f8642d3d Cache network room list queries. 2016-12-16 16:11:43 +00:00
Erik Johnston
b9b6d17ab1 Bump version and changelog 2016-12-16 10:18:02 +00:00
Richard van der Hoff
edc6a1e4f9 Add some logging for syncing to_device events
Attempt to track down the loss of to_device events
(https://github.com/vector-im/riot-web/issues/2711 etc).
2016-12-15 18:16:10 +00:00
Erik Johnston
35129ac998 Merge pull request #1698 from matrix-org/erikj/room_list
Fix caching on public room list
2016-12-15 15:40:28 +00:00
Erik Johnston
aa1336c00a Merge pull request #1700 from matrix-org/erikj/backfill_filter
Fix /backfill returning events it shouldn't
2016-12-15 14:21:30 +00:00
Marcin Bachry
24c16fc349 Fix crash in url preview when html tag has no text
Signed-off-by: Marcin Bachry <hegel666@gmail.com>
2016-12-14 22:38:18 +01:00
Erik Johnston
b8255eba26 Comment 2016-12-14 13:49:54 +00:00
Erik Johnston
b2999a7055 Fix /backfill returning events it shouldn't 2016-12-14 13:41:45 +00:00
Erik Johnston
c3208e45c9 Fixup membership query 2016-12-14 10:46:58 +00:00
Erik Johnston
9d95351cad Merge branch 'develop' of github.com:matrix-org/synapse into erikj/room_list 2016-12-13 17:57:08 +00:00
Erik Johnston
1de53a7a1a Fix caching on public room list 2016-12-13 17:33:24 +00:00
Erik Johnston
b3d398343e Bump changelog and version 2016-12-13 11:07:27 +00:00
Johannes Löthberg
0648e76979 Remove spurious newline
Apparently I just removed the spaces instead...

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-12 18:41:30 +01:00
Erik Johnston
8588d0eb3d Merge pull request #1697 from matrix-org/erikj/fix_bg_member
Fix background update that prematurely stopped
2016-12-12 17:20:17 +00:00
Erik Johnston
1574b839e0 Merge pull request #1676 from matrix-org/erikj/room_list
Add new API appservice specific public room list
2016-12-12 17:00:10 +00:00
Erik Johnston
7ec2bf9b77 Fix background update that prematurely stopped 2016-12-12 16:54:58 +00:00
Erik Johnston
2bf5a47b3e Rename network_id to instance_id on client side 2016-12-12 16:05:45 +00:00
Johannes Löthberg
d3bd94805f Fixup for #1689 and #1690
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-12 16:32:47 +01:00
Erik Johnston
09cbcb78d3 Add cache to get_public_room_ids_at_stream_id 2016-12-12 14:41:51 +00:00
Erik Johnston
631376e2ac Notify replication. Use correct network_id 2016-12-12 14:28:15 +00:00
Richard van der Hoff
abed247182 Remove unspecced GET endpoints for e2e keys
GET /keys/claim is a terrible idea, since it isn't idempotent; also it throws
500 errors if you call it without all the right params.

GET /keys/query is arguable, but it's unspecced, so let's get rid of it too to
stop people relying on unspecced APIs.
2016-12-12 12:31:40 +00:00
Erik Johnston
d45c984653 Docstring 2016-12-12 11:00:27 +00:00
Erik Johnston
d53a80af25 Merge pull request #1620 from matrix-org/erikj/concurrent_room_access
Limit the number of events that can be created on a given room concurrently
2016-12-12 10:30:23 +00:00
Richard van der Hoff
85cd30b1fd Merge pull request #1686 from matrix-org/rav/fix_federation_key_fails
E2E key query: handle federation fails
2016-12-12 09:33:39 +00:00
Richard van der Hoff
deca951241 Remove unused import 2016-12-12 09:24:35 +00:00
Glyph
9f07f4c559 IPv6 support for endpoint.py
Similar to https://github.com/matrix-org/synapse/pull/1689, but for endpoint.py
2016-12-11 11:10:32 +01:00
Glyph
6e18805ac2 IPv6 support for client.py
This is an (untested) general sketch of how to use wrapClientTLS to implement TLS over IPv6, as well as faster connections over IPv4.
2016-12-11 11:10:32 +01:00
Richard van der Hoff
efa4ccfaee E2E key query: handle federation fails
Don't fail the whole request if we can't connect to a particular server.
2016-12-09 18:31:01 +00:00
Erik Johnston
fbaf868f62 Correctly handle timeout errors 2016-12-09 16:30:29 +00:00
Erik Johnston
be14c24cea Fix rare notifier bug where listeners dont timeout
There was a race condition that caused the notifier to 'miss' the
timeout notification, since there were no other checks for the timeout
this caused listeners to get stuck in a loop until something happened.
2016-12-09 15:43:18 +00:00
Erik Johnston
1697f6a323 Merge pull request #1680 from matrix-org/erikj/joined_rooms
Add new room membership APIs
2016-12-09 11:35:48 +00:00
Erik Johnston
52d12ca782 Add /room/<room_id>/joined_members API
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
Erik Johnston
c45d8e9ba2 Add profile data to the room_membership table for joins 2016-12-08 13:08:41 +00:00
Richard van der Hoff
b08f76bd23 Fix ignored read-receipts
Don't ignore read-receipts which arrive in the same EDU as a read-receipt for
an old event.
2016-12-08 12:13:01 +00:00
Richard van der Hoff
bd07a35c29 Fix result of insert_receipt
This should fix the absence of notifications when new receipts arrive.
2016-12-08 12:11:34 +00:00
Erik Johnston
de796f27e6 Add joined_rooms servlet 2016-12-08 11:39:03 +00:00
Erik Johnston
2687af82d4 Comments 2016-12-07 09:58:33 +00:00
Erik Johnston
3727d66a0e Don't include appservice id 2016-12-06 17:04:26 +00:00
Erik Johnston
f32fb65552 Add new API appservice specific public room list 2016-12-06 16:12:27 +00:00
Richard van der Hoff
1529c19675 Prevent user tokens being used as guest tokens (#1675)
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
2016-12-06 15:31:37 +00:00
Erik Johnston
9609c91e7d Merge pull request #653 from matrix-org/erikj/preset_guest_join
Enable guest access for private rooms by default
2016-12-05 17:47:14 +00:00
Erik Johnston
338df4f409 Merge pull request #1649 from matrix-org/dbkr/log_ui_auth_args
Log the args that we have on UI auth completion
2016-12-05 16:40:58 +00:00
pik
c46e7a9c9b Bugfix: Console logging handler missing default filter 2016-12-03 20:14:58 -03:00
Johannes Löthberg
32c8b5507c preview_url_resource: Ellipsis must be in unicode string
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-01 13:12:13 +01:00
Richard van der Hoff
6841d8ff55 Fix doc-string
Remove refresh_token reference
2016-12-01 11:42:17 +00:00
Richard van der Hoff
aa09d6b8f0 Rip out more refresh_token code
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.

Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
Richard van der Hoff
dc4b23e1a1 Merge branch 'develop' into rav/no_more_refresh_tokens 2016-11-30 17:10:04 +00:00
Richard van der Hoff
8379a741cc Merge pull request #1660 from matrix-org/rav/better_content_type_validation
More intelligent Content-Type parsing
2016-11-30 16:54:03 +00:00
Richard van der Hoff
321fe5c44c Merge pull request #1656 from matrix-org/rav/remove_time_caveat
Stop putting a time caveat on access tokens
2016-11-30 16:53:20 +00:00
Richard van der Hoff
b5b3a7e867 More intelligent Content-Type parsing
Content-Type is allowed to contain options (`; charset=utf-8`, for
instance). We should allow that.
2016-11-30 15:07:32 +00:00
Richard van der Hoff
4febfe47f0 Comments
Update comments in verify_macaroon
2016-11-30 07:36:32 +00:00
Richard van der Hoff
77eca2487c Merge pull request #1653 from matrix-org/rav/guest_e2e
Implement E2E for guests
2016-11-29 17:41:35 +00:00
Richard van der Hoff
1c4f05db41 Stop putting a time caveat on access tokens
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.

Let's move in the right direction by not lying in our caveats.
2016-11-29 16:49:41 +00:00
Richard van der Hoff
5c4edc83b5 Stop generating refresh tokens
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
2016-11-28 10:13:01 +00:00
Richard van der Hoff
b6146537d2 Merge pull request #1655 from matrix-org/rav/remove_redundant_macaroon_checks
Remove redundant list of known caveat prefixes
2016-11-25 16:57:19 +00:00
Richard van der Hoff
f62b69e32a Allow guest access to endpoints for E2E
Expose /devices, /keys, and /sendToDevice to guest users, so that they can use
E2E.
2016-11-25 15:26:34 +00:00
Richard van der Hoff
7f02e4d008 Give guest users a device_id
We need to create devices for guests so that they can use e2e, but we don't
have anywhere to store it, so just use a fixed one.
2016-11-25 15:25:30 +00:00
Erik Johnston
11bfe438a2 Use correct var 2016-11-24 15:26:53 +00:00
Erik Johnston
aaecffba3a Correctly handle 500's and 429 on federation 2016-11-24 15:04:49 +00:00
Richard van der Hoff
e1d7c96814 Remove redundant list of known caveat prefixes
Also add some comments.
2016-11-24 12:38:17 +00:00
Erik Johnston
7e03f9a484 Bump version and changelog 2016-11-24 12:29:58 +00:00
Erik Johnston
46ca345b06 Don't send old events as federation 2016-11-24 12:29:02 +00:00
Erik Johnston
f36ea03741 Bump changelog and version 2016-11-24 11:08:01 +00:00
David Baker
c9d4e7b716 Clarify that creds doesn not contain passwords. 2016-11-24 10:54:59 +00:00
David Baker
f681aab895 Log the args that we have on UI auth completion
This will be super helpful for debugging if we have more
registration woes.
2016-11-24 10:11:45 +00:00
Erik Johnston
1985860c6e Comment 2016-11-23 15:59:59 +00:00
Erik Johnston
2ac516850b More efficient notif count queries 2016-11-23 15:57:04 +00:00
Erik Johnston
302fbd218d Merge pull request #1635 from matrix-org/erikj/split_out_fed_txn
Split out federation transaction sending to a worker
2016-11-23 15:39:12 +00:00
Erik Johnston
b2d6e63b79 Merge pull request #1641 from matrix-org/erikj/as_pushers
Ignore AS users when fetching push rules
2016-11-23 15:21:52 +00:00
Erik Johnston
feec718265 Shuffle receipt handler around so that worker apps don't need to load it 2016-11-23 15:14:24 +00:00
Erik Johnston
26072df6af Ensure only main or federation_sender process can send federation traffic 2016-11-23 14:09:47 +00:00
Erik Johnston
b69f76c106 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn 2016-11-23 11:31:53 +00:00
Erik Johnston
4d9b5c60f9 Comment 2016-11-23 11:11:41 +00:00
Erik Johnston
0163466d72 Ignore AS users when fetching push rules
By ignoring AS users early on when fetching push rules for a room we can
avoid needlessly hitting the DB and filling up the caches.
2016-11-23 11:01:01 +00:00
Erik Johnston
4c79a63fd7 Explicit federation ack 2016-11-23 10:40:44 +00:00
Erik Johnston
54fed21c04 Fix tests and flake8 2016-11-22 18:18:31 +00:00
Erik Johnston
90565d015e Invalidate retry cache in both directions 2016-11-22 17:45:44 +00:00
Kegan Dougal
83bcdcee61 Return early on /sync code paths if a '*' filter is used
This is currently very conservative in that it only does this if there is no
`since` token. This limits the risk to clients likely to be doing one-off
syncs (like bridges), but does mean that normal human clients won't benefit
from the time savings here. If the savings are large enough, I would consider
generalising this to just check the filter.
2016-11-22 16:38:35 +00:00
Kegsay
d4a459f7cb Merge pull request #1638 from matrix-org/kegan/sync-event-fields
Implement "event_fields" in filters
2016-11-22 14:02:38 +00:00
Kegan Dougal
c3d963ac24 Review comments 2016-11-22 13:42:11 +00:00
Kegan Dougal
6d4e6d4cba Also check for dict since sometimes they aren't frozen 2016-11-22 10:39:41 +00:00
Erik Johnston
baf9e74a73 Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-11-22 10:31:48 +00:00
Kegan Dougal
cea4e4e7b2 Glue only_event_fields into the sync rest servlet 2016-11-22 10:14:05 +00:00
Kegan Dougal
0a8b0eeca1 More tests 2016-11-22 09:59:27 +00:00
Erik Johnston
51e89709aa Comments 2016-11-21 17:59:39 +00:00
Kegan Dougal
70a2157b64 Start adding some tests 2016-11-21 17:52:45 +00:00
Kegan Dougal
f97511a1f3 Move event_fields filtering to serialize_event
Also make it an inclusive not exclusive filter, as the spec demands.
2016-11-21 17:42:16 +00:00
Erik Johnston
88d85ebae1 Add some metrics 2016-11-21 17:36:05 +00:00
Erik Johnston
50934ce460 Comments 2016-11-21 16:55:23 +00:00
Kegan Dougal
e90fcd9edd Add filter_event_fields and filter_field to FilterCollection 2016-11-21 15:18:18 +00:00
Erik Johnston
9687e039e7 Remove explicit calls to send_pdu 2016-11-21 14:48:51 +00:00
Kegan Dougal
a2a6c1c22f Fail with a coherent error message if /sync?filter= is invalid 2016-11-21 13:15:25 +00:00
Erik Johnston
524d61bf7e Fix tests 2016-11-21 11:53:02 +00:00
Erik Johnston
7c9cdb2245 Store federation stream positions in the database 2016-11-21 11:33:08 +00:00
Mark Haines
a289150943 Fix flake8 2016-11-18 17:15:02 +00:00
David Baker
544722bad2 Work around client replacing reg params
Works around https://github.com/vector-im/vector-android/issues/715
and equivalent for iOS
2016-11-18 17:07:35 +00:00
Erik Johnston
f8ee66250a Handle sending events and device messages over federation 2016-11-17 15:48:04 +00:00
Erik Johnston
ed787cf09e Hook up the send queue and create a federation sender worker 2016-11-16 17:34:44 +00:00
Erik Johnston
1587b5a033 Add initial cut of federation send queue 2016-11-16 14:47:52 +00:00
Erik Johnston
59ef517e6b Use new federation_sender DI 2016-11-16 14:47:52 +00:00
Erik Johnston
847d5db1d1 Add transaction queue and transport layer to DI 2016-11-16 14:47:52 +00:00
Erik Johnston
daec6fc355 Move logic into transaction_queue 2016-11-16 14:47:52 +00:00
Erik Johnston
0e830d3770 Rename transaction queue functions to send_* 2016-11-16 14:47:52 +00:00
Erik Johnston
dc6cede78e Merge pull request #1628 from matrix-org/erikj/ldap_split_out
Use external ldap auth pacakge
2016-11-15 16:53:34 +00:00
Erik Johnston
c7546b3cdb Merge pull request #1617 from matrix-org/erikj/intern_state_dict
Correctly intern keys in state cache
2016-11-15 16:45:55 +00:00
Erik Johnston
d56c39cf24 Use external ldap auth pacakge 2016-11-15 13:03:19 +00:00
Erik Johnston
f9d156d270 New Flake8 fixes 2016-11-15 11:22:29 +00:00
Erik Johnston
9d58ccc547 Bump changelog and version 2016-11-14 15:05:04 +00:00
Kegsay
9355a5c42b Merge pull request #1624 from matrix-org/kegan/idempotent-requests
Store Promise<Response> instead of Response for HTTP API transactions
2016-11-14 12:45:30 +00:00
Kegan Dougal
3991b4cbdb Clean transactions based on time. Add HttpTransactionCache tests. 2016-11-14 11:19:24 +00:00
Kegan Dougal
af4a1bac50 Move .observe() up to the cache to make things neater 2016-11-14 09:52:41 +00:00
Erik Johnston
0964005d84 Merge pull request #1625 from DanielDent/patch-1
Add support for durations in minutes
2016-11-12 11:20:46 +00:00
Daniel Dent
1c93cd9f9f Add support for durations in minutes 2016-11-12 00:10:23 -08:00
Kegan Dougal
8ecaff51a1 Review comments 2016-11-11 17:47:03 +00:00
Kegan Dougal
f6c48802f5 More flake8 2016-11-11 15:08:24 +00:00
Kegan Dougal
a88bc67f88 Flake8 and fix whoopsie 2016-11-11 15:02:29 +00:00
Kegan Dougal
42c43cfafd Use ObservableDeferreds instead of Deferreds as they behave as intended 2016-11-11 14:54:10 +00:00
Kegan Dougal
c7daf3136c Use observable deferreds because they are sane 2016-11-11 14:13:32 +00:00
Erik Johnston
64038b806c Comments 2016-11-11 10:42:08 +00:00
Erik Johnston
2bd4513a4d Limit the number of events that can be created on a given room concurretnly 2016-11-10 16:44:35 +00:00
Erik Johnston
d073cb7ead Add Limiter: limit concurrent access to resource 2016-11-10 16:29:51 +00:00
Kegan Dougal
8a8ad46f48 Flake8 2016-11-10 15:22:11 +00:00
Kegan Dougal
2771447c29 Store Promise<Response> instead of Response for HTTP API transactions
This fixes a race whereby:
 - User hits an endpoint.
 - No cached transaction so executes main code.
 - User hits same endpoint.
 - No cache transaction so executes main code.
 - Main code finishes executing and caches response and returns.
 - Main code finishes executing and caches response and returns.

 This race is common in the wild when Synapse is struggling under load.
 This commit fixes the race by:
  - User hits an endpoint.
  - Caches the promise to execute the main code and executes main code.
  - User hits same endpoint.
  - Yields on the same promise as the first request.
  - Main code finishes executing and returns, unblocking both requests.
2016-11-10 14:49:26 +00:00
Erik Johnston
ac507e7ab8 Don't assume providers raise ConfigError's 2016-11-08 17:23:28 +00:00
Erik Johnston
e6651e8046 Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-11-08 14:43:49 +00:00
Erik Johnston
3c09818d91 Bump version and changelog 2016-11-08 14:39:55 +00:00
Erik Johnston
27d3f2e7ab Explicitly set authentication mode in ldap3
This only makes a difference for versions of ldap3 before 1.0, but a)
its best to be explicit and b) there are distributions that package
ancient versions for ldap3 (e.g. debian).
2016-11-08 14:35:25 +00:00
Erik Johnston
17e0a58020 Merge pull request #1615 from matrix-org/erikj/limit_prev_events
Limit the number of prev_events of new events
2016-11-08 12:06:15 +00:00
Erik Johnston
587d8ac60f Correctly intern keys in state cache 2016-11-08 11:53:25 +00:00
Erik Johnston
a4632783fb Sample correctly 2016-11-08 11:20:26 +00:00
Erik Johnston
24772ba56e Respect use_frozen_dicts option in workers 2016-11-08 11:07:18 +00:00
Erik Johnston
eeda4e618c Limit the number of prev_events of new events 2016-11-08 11:02:29 +00:00
Erik Johnston
d24197bead Merge pull request #1198 from euank/more-ip-blacklist
default config: blacklist more internal ips
2016-11-07 09:41:34 +00:00
Euan Kemp
c6bbad109b default config: blacklist more internal ips 2016-11-06 17:02:25 -08:00
Erik Johnston
63772443e6 Comment 2016-11-04 10:53:42 +00:00
Erik Johnston
a3f6576084 Remove unused but buggy function 2016-11-04 10:48:20 +00:00
Paul "LeoNerd" Evans
2938a00825 Rename the python-specific metrics now the docs claim that we have done 2016-11-03 17:03:52 +00:00
Paul "LeoNerd" Evans
5219f7e060 Since we don't export per-filetype fd counts any more, delete all the code related to that too 2016-11-03 16:41:32 +00:00
Paul "LeoNerd" Evans
93ebeb2aa8 Remove now-unused 'resource' import 2016-11-03 16:37:09 +00:00
Paul "LeoNerd" Evans
c1b077cd19 Now we have new-style metrics don't bother exporting legacy-named process ones 2016-11-03 16:34:16 +00:00
Erik Johnston
64c6566980 Remove spurious comment 2016-11-03 15:04:32 +00:00
Erik Johnston
8fd4d9129f Replace postgres GIN with GIST
This is because GIN can be slow to write too, especially when the table
gets large.
2016-11-03 15:00:03 +00:00
David Baker
9084720993 Don't error on non-ascii passwords 2016-11-03 10:42:14 +00:00
Mark Haines
b1c27975d0 Set CORs headers on responses from the media repo 2016-11-02 11:29:25 +00:00
Erik Johnston
2746e805fe Merge pull request #1188 from matrix-org/erikj/sent_transactions
Remove sent_transactions table.
2016-11-01 14:52:55 +00:00
Erik Johnston
3c91c5b216 Bump version and changelog 2016-11-01 13:06:36 +00:00
Erik Johnston
760469c812 Continue to clean up received_transactions 2016-11-01 11:42:08 +00:00
Erik Johnston
f09d2b692f Removed unused stuff 2016-10-31 17:10:56 +00:00
Erik Johnston
4c3eb14d68 Increase batching of sent transaction inserts
This should further reduce the number of individual inserts,
transactions and updates that are required for keeping sent_transactions
up to date.
2016-10-31 16:07:45 +00:00
Mark Haines
82e71a259c Bump changelog and version 2016-10-28 11:16:05 +01:00
Mark Haines
490b97d3e7 Merge branch 'develop' into release-v0.18.2 2016-10-28 11:12:31 +01:00
Paul "LeoNerd" Evans
1cc22da600 Set up the process collector during metrics __init__; that way all split-process workers have it 2016-10-27 18:09:34 +01:00
Paul "LeoNerd" Evans
aac13b1f9a Pass the Metrics group into the process collector instead of having it find its own one; this avoids it needing to import from synapse.metrics 2016-10-27 18:08:15 +01:00
Paul "LeoNerd" Evans
ccc1a3d54d Allow creation of a 'subspace' within a Metrics object, returning another one 2016-10-27 18:07:34 +01:00
Erik Johnston
665e53524e Bump changelog and version 2016-10-27 14:52:47 +01:00
Erik Johnston
a9111786f9 Use most recently added binding, not most recently seen user. 2016-10-27 14:32:45 +01:00
Erik Johnston
1fc1bc2a51 Fix user_threepids schema delta
The delta `37/user_threepids.sql` aimed to update all the email
addresses to be lower case, however duplicate emails may exist in the
table already.

This commit adds a step where the delta moves the duplicate emails to a
new `medium` `email_old`. Only the most recently used account keeps the
binding intact. We move rather than delete so that we retain some record
of which emails were associated with which account.
2016-10-27 14:14:44 +01:00
Erik Johnston
ab731d8f8e Bump changelog and version 2016-10-27 11:07:02 +01:00
Mark Haines
177f104432 Merge pull request #1098 from matrix-org/markjh/bearer_token
Allow clients to supply access_tokens as headers
2016-10-25 17:33:15 +01:00
Erik Johnston
22fbf86e4f Merge branch 'release-v0.18.2' of github.com:matrix-org/synapse into develop 2016-10-25 15:09:46 +01:00
Erik Johnston
855645c719 Bump version and changelog 2016-10-25 11:16:16 +01:00
Erik Johnston
2ef617bc06 Fix infinite typing bug
There's a bug somewhere that causes typing notifications to not be timed
out properly. By adding a paranoia timer and using correct inequalities
notifications should stop being stuck, even if it the root cause hasn't
been fixed.
2016-10-24 15:51:22 +01:00
Erik Johnston
b6800a8ecd Actually use the new function 2016-10-24 13:39:49 +01:00
Erik Johnston
d04e2ff3a4 Fix incredubly slow back pagination query
If a client didn't specify a from token when paginating backwards
synapse would attempt to query the (global) maximum topological token.
This a) doesn't make much sense since they're room specific and b) there
are no indices that lets postgres do this efficiently.
2016-10-24 13:35:51 +01:00
Paul Evans
a842fed418 Merge pull request #1177 from matrix-org/paul/standard-metric-names
Standardise prometheus metrics
2016-10-21 13:06:19 +01:00
Luke Barnard
e01a1bc92d Merge pull request #1175 from matrix-org/luke/feature-configurable-as-rate-limiting
Allow Configurable Rate Limiting Per AS
2016-10-20 16:21:10 +01:00
Luke Barnard
6fdd31915b Style 2016-10-20 13:53:15 +01:00
Luke Barnard
07caa749bf Closing brace on following line 2016-10-20 12:07:16 +01:00
Luke Barnard
f09db236b1 as_user->app_service, less redundant comments, better positioned comments 2016-10-20 12:04:54 +01:00
Luke Barnard
8bfd01f619 flake8 2016-10-20 11:52:46 +01:00
Luke Barnard
1b17d1a106 Use real AS object by passing it through the requester
This means synapse does not have to check if the AS is interested, but instead it effectively re-uses what it already knew about the requesting user
2016-10-20 11:43:05 +01:00
Paul "LeoNerd" Evans
b01aaadd48 Split callback metric lambda functions down onto their own lines to keep line lengths under 90 2016-10-19 18:26:13 +01:00
Paul "LeoNerd" Evans
1071c7d963 Adjust code for <100 char line limit 2016-10-19 18:23:25 +01:00
Paul "LeoNerd" Evans
6453d03edd Cut the raw /proc/self/stat line up into named fields at collection time 2016-10-19 18:21:40 +01:00
Paul "LeoNerd" Evans
3ae48a1f99 Move the process metrics collector code into its own file 2016-10-19 18:10:24 +01:00
Paul "LeoNerd" Evans
4cedd53224 A slightly neater way to manage metric collector functions 2016-10-19 17:54:09 +01:00
Paul "LeoNerd" Evans
5663137e03 appease pep8 2016-10-19 16:09:42 +01:00
Paul "LeoNerd" Evans
b202531be6 Also guard /proc/self/fds-related code with a suitable psuedoconstant 2016-10-19 15:37:41 +01:00
Paul "LeoNerd" Evans
1b179455fc Guard registration of process-wide metrics by existence of the requisite /proc entries 2016-10-19 15:34:38 +01:00
Paul "LeoNerd" Evans
981f852d54 Add standard process_start_time_seconds metric 2016-10-19 15:05:22 +01:00
Paul "LeoNerd" Evans
def63649df Add standard process_max_fds metric 2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
06f1ad1625 Add standard process_open_fds metric 2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
95fc70216d Add standard process_*_memory_bytes metrics 2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
9b0316c75a Use /proc/self/stat to generate the new process_cpu_*_seconds_total metrics 2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
03c2720940 Export CPU usage metrics also under prometheus-standard metric name 2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans
b21b9dbc37 Callback metric values might not just be integers - allow floats 2016-10-19 15:05:15 +01:00
Erik Johnston
78c083f159 Merge pull request #1164 from pik/error-codes
Clarify Error codes for GET /filter/
2016-10-19 14:26:17 +01:00
Erik Johnston
3aa8925091 Merge pull request #1176 from matrix-org/erikj/eager_ratelimit_check
Check whether to ratelimit sooner to avoid work
2016-10-19 14:25:52 +01:00
Erik Johnston
f2f74ffce6 Comment 2016-10-19 14:21:28 +01:00
David Baker
0108ed8ae6 Latest delta is now 37 2016-10-19 11:40:35 +01:00
David Baker
a7f48320b1 Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive 2016-10-19 11:28:56 +01:00
David Baker
df2a616c7b Convert emails to lowercase when storing
And db migration sql to convert existing addresses.
2016-10-19 11:13:55 +01:00
Erik Johnston
550308c7a1 Check whether to ratelimit sooner to avoid work 2016-10-19 10:45:24 +01:00
pik
e8b1d2a452 Refactor test_filter to use real DataStore
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
Luke Barnard
5b54d51d1e Allow Configurable Rate Limiting Per AS
This adds a flag loaded from the registration file of an AS that will determine whether or not its users are rate limited (by ratelimit in _base.py). Needed for IRC bridge reasons - see https://github.com/matrix-org/matrix-appservice-irc/issues/240.
2016-10-18 17:04:09 +01:00
Erik Johnston
8ca05b5755 Fix push notifications for a single unread message 2016-10-18 10:57:33 +01:00
Erik Johnston
f0ca088280 Reduce redundant database work in email pusher
Update the last stream ordering if the
`get_unread_push_actions_for_user_in_range_for_email` returns no new
push actions. This reduces the range that it needs to check next
iteration.
2016-10-18 10:52:47 +01:00
Erik Johnston
b95dbdcba4 Bump version and changelog 2016-10-17 15:59:12 +01:00
Erik Johnston
6942d68247 Bump schema version 2016-10-17 11:17:45 +01:00
Erik Johnston
b59994b454 Remove TODO 2016-10-17 11:17:02 +01:00
Erik Johnston
816988baaa Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_auth 2016-10-17 11:10:37 +01:00
Erik Johnston
2869a29fd7 Drop some unused indices 2016-10-17 11:08:19 +01:00
Erik Johnston
a68ade6ed3 Merge pull request #1162 from larroy/master
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-14 21:42:55 +01:00
David Baker
29c5922021 Revert part of 6207399
older sqlite doesn't support indexes on expressions, lets just
store things lowercase in the db
2016-10-14 16:20:24 +01:00
Alexander Maznev
d9350b0db8 Error codes for filters
* add tests

Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
2016-10-14 10:18:28 -05:00
David Baker
bcb1245a2d Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive 2016-10-14 15:10:38 +01:00
David Baker
62073992c5 Make password reset email field case insensitive 2016-10-14 13:56:53 +01:00
Erik Johnston
6f7540ada4 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fix_email_notifs 2016-10-14 10:22:43 +01:00
Erik Johnston
1d107d8484 Fix email push notifs being dropped
A lot of email push notifications were failing to be sent due to an
exception being thrown along one of the (many) paths. This was due to a
change where we moved from pulling out the full state for each room, but
rather pulled out the event ids for the state and separately loaded the
full events when needed.
2016-10-13 13:40:38 +01:00
Richard van der Hoff
f7aed3d7a2 Merge pull request #1168 from matrix-org/rav/ui_auth_on_device_delete
User-interactive auth on delete device
2016-10-13 09:38:41 +01:00
Richard van der Hoff
9009143fb9 Handle delete device requests with no body
We should probably return a 401 rather than a 400 for existing clients that
don't know they have to do the UIA dance to delete a device.
2016-10-12 18:47:28 +01:00
Richard van der Hoff
fbd3866bc6 User-interactive auth on delete device 2016-10-12 16:16:31 +01:00
Mark Haines
9e18e0b1cb Merge pull request #1167 from matrix-org/markjh/fingerprints
Add config option for adding additional TLS fingerprints
2016-10-12 15:27:44 +01:00
Mark Haines
c61ddeedac Explain how long the servers can cache the TLS fingerprints for 2016-10-12 14:48:24 +01:00
Mark Haines
0af6213019 Improve comment formatting 2016-10-12 14:45:13 +01:00
Erik Johnston
35e2cc8b52 Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth
Implement pluggable password auth
2016-10-12 11:41:20 +01:00
Mark Haines
6e9f3ab415 Add config option for adding additional TLS fingerprints 2016-10-11 19:14:46 +01:00
Erik Johnston
3061dac53e Merge branch 'develop' of github.com:matrix-org/synapse into erikj/replication_noop 2016-10-11 14:08:29 +01:00
Erik Johnston
668f91d707 Fix check of wrong variable 2016-10-11 13:57:22 +01:00
Richard van der Hoff
0061e8744f Merge pull request #1166 from matrix-org/rav/grandfather_broken_riot_signup
Work around email-spamming Riot bug
2016-10-11 11:58:58 +01:00
Richard van der Hoff
fa74fcf512 Work around email-spamming Riot bug
5d9546f9 introduced a change to synapse behaviour, in that failures in the
interactive-auth process would return the flows and params data as well as an
error code (as specced in https://github.com/matrix-org/matrix-doc/pull/397).

That change exposed a bug in Riot which would make it request a new validation
token (and send a new email) each time it got a 401 with a `flows` parameter
(see https://github.com/vector-im/vector-web/issues/2447 and the fix at
https://github.com/matrix-org/matrix-react-sdk/pull/510).

To preserve compatibility with broken versions of Riot, grandfather in the old
behaviour for the email validation stage.
2016-10-11 11:34:40 +01:00
Erik Johnston
a2f2516199 Merge pull request #1157 from Rugvip/nolimit
Remove rate limiting from app service senders and fix get_or_create_user requester
2016-10-11 11:20:54 +01:00
Erik Johnston
a940618c94 Merge pull request #1150 from Rugvip/state_key
api/auth: fix for not being allowed to set your own state_key
2016-10-11 11:19:55 +01:00
Pedro Larroy
c57f871184 Use sys.executable instead of hardcoded python. fixes #1161 2016-10-08 23:55:20 +02:00
Richard van der Hoff
8681aff4f1 Merge pull request #1160 from matrix-org/rav/401_on_password_fail
Interactive Auth: Return 401 from for incorrect password
2016-10-07 10:57:43 +01:00
Richard van der Hoff
5d9546f9f4 Interactive Auth: Return 401 from for incorrect password
This requires a bit of fettling, because I want to return a helpful error
message too but we don't want to distinguish between unknown user and invalid
password. To avoid hardcoding the error message into 15 places in the code,
I've had to refactor a few methods to return None instead of throwing.

Fixes https://matrix.org/jira/browse/SYN-744
2016-10-07 00:00:00 +01:00
Patrik Oldsberg
7b5546d077 rest/client/v1/register: use the correct requester in createUser
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 22:12:32 +02:00
Richard van der Hoff
5d34e32d42 Merge pull request #1159 from matrix-org/rav/uia_fallback_postmessage
window.postmessage for Interactive Auth fallback
2016-10-06 19:56:43 +01:00
Richard van der Hoff
f382117852 window.postmessage for Interactive Auth fallback
If you're a webapp running the fallback in an iframe, you can't set set a
window.onAuthDone function. Let's post a message back to window.opener instead.
2016-10-06 18:16:59 +01:00
Patrik Oldsberg
3de7c8a4d0 handlers/profile: added admin override for set_displayname and set_avatar_url
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 15:24:59 +02:00
Patrik Oldsberg
2ff2d36b80 handers: do not ratelimit app service senders
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 15:24:59 +02:00
Patrik Oldsberg
9bfc617791 storage/appservice: make appservice methods only relying on the cache synchronous 2016-10-06 15:24:59 +02:00
Erik Johnston
4285be791d Bump changelog and version 2016-10-05 14:40:38 +01:00
Erik Johnston
850b103b36 Implement pluggable password auth
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
2016-10-03 10:36:40 +01:00
Patrik Oldsberg
24a70e19c7 api/auth: fix for not being allowed to set your own state_key
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-09-30 13:08:25 +02:00
Erik Johnston
04aa2f2863 Bump version and changelog 2016-09-30 10:34:57 +01:00
Martin Weinelt
3027ea22b0 Restructure ldap authentication
- properly parse return values of ldap bind() calls
- externalize authentication methods
- change control flow to be more error-resilient
- unbind ldap connections in many places
- improve log messages and loglevels
2016-09-29 15:30:08 +01:00
Erik Johnston
5875a65253 Merge pull request #1145 from matrix-org/erikj/fix_reindex
Fix background reindex of origin_server_ts
2016-09-29 13:53:48 +01:00
Erik Johnston
9040c9ffa1 Fix background reindex of origin_server_ts
The storage function `_get_events_txn` was removed everywhere except
from this background reindex. The function was removed due to it being
(almost) completely unused while also being large and complex.
Therefore, instead of resurrecting `_get_events_txn` we manually
reimplement the bits that are needed directly.
2016-09-27 11:23:49 +01:00
Erik Johnston
4974147aa3 Remove duplication 2016-09-27 09:27:54 +01:00
Erik Johnston
13122e5e24 Remove unused variable 2016-09-27 09:21:51 +01:00
Erik Johnston
cf3e1cc200 Fix perf of fetching state in SQLite 2016-09-26 17:16:24 +01:00
Erik Johnston
a38d46249e Merge pull request #1140 from matrix-org/erikj/typing_fed_timeout
Time out typing over federation
2016-09-26 11:24:14 +01:00
Erik Johnston
748d8fdc7b Reduce DB hits for replication
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.

This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
2016-09-23 16:49:21 +01:00
Erik Johnston
655891d179 Move FEDERATION_PING_INTERVAL timer. Update log line 2016-09-23 15:43:34 +01:00
Erik Johnston
4225a97f4e Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-09-23 15:36:59 +01:00
Erik Johnston
22578545a0 Time out typing over federation 2016-09-23 14:00:52 +01:00
Erik Johnston
667fcd54e8 Merge pull request #1136 from matrix-org/erikj/fix_signed_3pid
Allow invites via 3pid to bypass sender sig check
2016-09-22 13:41:49 +01:00
Erik Johnston
f96020550f Update comments 2016-09-22 12:54:22 +01:00
Erik Johnston
81964aeb90 Merge pull request #1132 from matrix-org/erikj/initial_sync_split
Support /initialSync in synchrotron worker
2016-09-22 12:45:02 +01:00
Erik Johnston
2e9ee30969 Add comments 2016-09-22 11:59:46 +01:00
Erik Johnston
a61e4522b5 Shuffle things around to make unit tests work 2016-09-22 11:08:12 +01:00
Erik Johnston
1168cbd54d Allow invites via 3pid to bypass sender sig check
When a server sends a third party invite another server may be the one
that the inviting user registers with. In this case it is that remote
server that will issue an actual invitation, and wants to do it "in the
name of" the original invitee. However, the new proper invite will not
be signed by the original server, and thus other servers would reject
the invite if it was seen as coming from the original user.

To fix this, a special case has been added to the auth rules whereby
another server can send an invite "in the name of" another server's
user, so long as that user had previously issued a third party invite
that is now being accepted.
2016-09-22 10:56:53 +01:00
Erik Johnston
bbc0d9617f Merge pull request #1134 from matrix-org/erikj/fix_stream_public_deletion
Fix _delete_old_forward_extrem_cache query
2016-09-21 17:04:04 +01:00
Erik Johnston
8009d84364 Match against event_id, rather than room_id 2016-09-21 16:46:59 +01:00
Erik Johnston
dc692556d6 Remove spurious AS clause 2016-09-21 16:28:47 +01:00
Erik Johnston
dc78db8c56 Update correct table 2016-09-21 15:52:44 +01:00
Erik Johnston
4f78108d8c Readd entries to public_room_list_stream that were deleted 2016-09-21 15:24:22 +01:00
Erik Johnston
0b78d8adf2 Fix _delete_old_forward_extrem_cache query 2016-09-21 15:20:56 +01:00
Erik Johnston
90c070c850 Add total_room_count_estimate to /publicRooms 2016-09-21 13:30:05 +01:00
Erik Johnston
87528f0756 Support /initialSync in synchrotron worker 2016-09-21 11:46:28 +01:00
Erik Johnston
2b8ff4659f Bump version and changelog 2016-09-19 17:16:56 +01:00
Erik Johnston
ddfcdd4778 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.18.0 2016-09-19 17:15:24 +01:00
Erik Johnston
6f0c5e5d9b Merge pull request #1131 from matrix-org/matthew/e2e-notifs
Notify on e2e events
2016-09-19 10:50:26 +01:00
Erik Johnston
49cf205dc7 _id field must uniquely identify different conditions 2016-09-19 10:34:01 +01:00
Erik Johnston
39af634dd2 Merge pull request #1130 from matrix-org/erikj/fix_pubroom_pag
Handle fact that _generate_room_entry may not return a room entry
2016-09-19 10:13:59 +01:00
Matthew Hodgson
3f6ec271ba proposal for notifying on e2e events 2016-09-17 22:05:06 +01:00
Erik Johnston
4d49e0bdfd PEP8 2016-09-17 18:09:22 +01:00
Erik Johnston
81570abfb2 Handle fact that _generate_room_entry may not return a room entry 2016-09-17 18:01:54 +01:00
Erik Johnston
ddc89df89d Enable guest access to POST /publicRooms 2016-09-17 15:55:24 +01:00
Erik Johnston
eb24aecf8c Merge pull request #1129 from matrix-org/erikj/fix_pubroom_pag
Fix and clean up publicRooms pagination
2016-09-17 15:30:34 +01:00
Erik Johnston
e1ba98d724 Merge pull request #1127 from matrix-org/dbkr/publicroom_search_case_insensitive
Make public room search case insensitive
2016-09-17 15:01:17 +01:00
Erik Johnston
a298331de4 Spelling 2016-09-17 14:59:40 +01:00
Erik Johnston
71edaae981 Fix and clean up publicRooms pagination 2016-09-17 14:46:19 +01:00
Matthew Hodgson
883df2e983 fix logger for client_reader worker 2016-09-17 14:12:04 +01:00
David Baker
5336acd46f Make public room search case insensitive 2016-09-16 19:02:42 +01:00
Erik Johnston
19fe990476 Update changelog and bump version 2016-09-16 17:30:59 +01:00
Erik Johnston
995f2f032f Fix public room pagination for client_reader app 2016-09-16 14:48:21 +01:00
Erik Johnston
a68807d426 Comment 2016-09-16 11:36:20 +01:00
Erik Johnston
2e67cabd7f Make POST /publicRooms require auth 2016-09-16 11:32:51 +01:00
Erik Johnston
b7b62bf9ea Comment 2016-09-16 11:00:29 +01:00
Erik Johnston
d84319ae10 Add remote reoom cache 2016-09-16 10:31:59 +01:00
Erik Johnston
23b6701a28 Support filtering remote room lists 2016-09-16 10:24:15 +01:00
Erik Johnston
e58a9d781c Filter remote rooms lists locally 2016-09-16 10:19:32 +01:00
Erik Johnston
74d4cdee25 Don't cache searched in /publicRooms 2016-09-16 09:05:11 +01:00
Erik Johnston
418bcd4309 Add new storage function to slave store 2016-09-16 08:37:39 +01:00
Erik Johnston
098db4aa52 Add very basic filter API to /publicRooms 2016-09-15 17:50:16 +01:00
Erik Johnston
c33b25fd8d Change the way we calculate new_limit in /publicRooms and add POST API 2016-09-15 17:35:20 +01:00
Erik Johnston
de4f798f01 Handling expiring stream extrems correctly. 2016-09-15 17:34:59 +01:00
Erik Johnston
ea6dc356b0 Merge pull request #1125 from matrix-org/erikj/public_room_cache
Change get_pos_of_last_change to return upper bound
2016-09-15 15:48:53 +01:00
Erik Johnston
955f34d23e Change get_pos_of_last_change to return upper bound 2016-09-15 15:12:07 +01:00
Erik Johnston
241d7d2d62 Merge pull request #1124 from matrix-org/erikj/enable_state_caching_workers
Enable state caches on workers
2016-09-15 15:01:47 +01:00
Erik Johnston
4be85281f9 Enable state caches on workers 2016-09-15 14:31:22 +01:00
Erik Johnston
cb3edec6af Use stream_change cache to make get_forward_extremeties_for_room cache more effective 2016-09-15 14:28:13 +01:00
Erik Johnston
55e6fc917c Add cache to get_forward_extremeties_for_room 2016-09-15 14:04:28 +01:00
Erik Johnston
68c1ed4d1a Remove default public rooms limit 2016-09-15 13:56:20 +01:00
Erik Johnston
b82fa849c8 Merge pull request #1120 from matrix-org/erikj/push_invite_cache
Ensure we don't mutate the cache of push rules
2016-09-15 13:27:18 +01:00
Erik Johnston
e457034e99 Merge pull request #1121 from matrix-org/erikj/public_room_paginate
Add pagination support to publicRooms
2016-09-15 13:27:09 +01:00
Erik Johnston
1d98cf26be By default limit /publicRooms to 100 entries 2016-09-15 13:18:35 +01:00
Erik Johnston
211786ecd6 Stream public room changes down replication 2016-09-15 11:47:23 +01:00
Erik Johnston
4fb65a1091 Base public room list off of public_rooms stream 2016-09-15 11:27:04 +01:00
Erik Johnston
5810cffd33 Pass since/from parameters over federation 2016-09-15 10:36:19 +01:00
Erik Johnston
f3eead0660 Allow paginating both forwards and backwards 2016-09-15 10:15:37 +01:00
Erik Johnston
4131381123 Remove support for aggregate room lists 2016-09-15 09:28:15 +01:00
Erik Johnston
6a5ded5988 Ensure we don't mutate the cache of push rules 2016-09-15 09:16:13 +01:00
Erik Johnston
4f181f361d Accept optional token to public room list 2016-09-15 09:08:57 +01:00
Erik Johnston
c566f0ee17 Calculate the public room list from a stream_ordering 2016-09-14 17:42:47 +01:00
Erik Johnston
772c6067a3 Refactor public rooms to not pull out the full state for each room 2016-09-14 17:29:25 +01:00
Erik Johnston
baffe96d95 Add a room visibility stream 2016-09-14 17:29:19 +01:00
Erik Johnston
264a48aedf Merge pull request #1117 from matrix-org/erikj/fix_state
Ensure we don't mutate state cache entries
2016-09-14 16:50:37 +01:00
Erik Johnston
ed992ae6ba Add a DB index to figure out past state at a stream ordering in a room 2016-09-14 16:20:27 +01:00
Erik Johnston
e0b6db29ed Split out public room list into a worker process 2016-09-14 14:42:51 +01:00
Erik Johnston
a70a43bc51 Move RoomListHandler into a separate file 2016-09-14 14:07:37 +01:00
Erik Johnston
f2b2cd8eb4 Amalgamate two identical consecutive if statements 2016-09-14 11:16:22 +01:00
Erik Johnston
00f51493f5 Fix reindex 2016-09-14 10:18:30 +01:00
Erik Johnston
d5ae1f1291 Ensure we don't mutate state cache entries 2016-09-14 10:03:48 +01:00
Matthew Hodgson
1b01488d27 Merge pull request #1111 from matrix-org/matthew/device-ids
make device IDs more useful for human disambiguation
2016-09-14 01:18:02 +01:00
Paul Evans
0f73f0e70e Merge pull request #1116 from matrix-org/paul/tiny-fixes
Fix typo "persiting"
2016-09-13 13:27:42 +01:00
Paul "LeoNerd" Evans
ca35e54d6b Fix typo "persiting" 2016-09-13 13:26:33 +01:00
Mark Haines
ad816b0add Limit how often we ask for keys from dead servers 2016-09-13 11:53:50 +01:00
Erik Johnston
43253c10b8 Remove redundant event_auth index 2016-09-13 11:47:48 +01:00
Mark Haines
18ab019a4a Move the E2E key handling into the e2e handler 2016-09-13 11:35:35 +01:00
Mark Haines
76b09c29b0 Merge pull request #1110 from matrix-org/markjh/e2e_timeout
Add a timeout parameter for end2end key queries.
2016-09-13 10:50:45 +01:00
Erik Johnston
ba6bc2faa0 Merge pull request #1109 from matrix-org/erikj/partial_indices
Add WHERE clause support to index creation
2016-09-13 09:06:16 +01:00
Matthew Hodgson
edbcb4152b make device IDs more useful for human disambiguation 2016-09-13 00:02:39 +01:00
Mark Haines
949c2c5435 Add a timeout parameter for end2end key queries.
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.

Fetch keys from different servers in parallel rather than in series.

Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
Erik Johnston
b17af156c7 Remove where clause 2016-09-12 17:05:54 +01:00
Erik Johnston
1c9da43a95 Merge pull request #1108 from matrix-org/erikj/create_dm
Add is_direct param to /createRoom
2016-09-12 16:57:16 +01:00
Erik Johnston
0b32bb20bb Index contains_url for file search queries 2016-09-12 16:57:05 +01:00
Erik Johnston
c94de0ab60 Add WHERE clause support to index creation 2016-09-12 16:55:01 +01:00
Erik Johnston
502c901e11 Merge pull request #1107 from matrix-org/erikj/backfill_none
Fix backfill when cannot find an event.
2016-09-12 16:48:01 +01:00
Erik Johnston
48a5a7552d Add is_direct param to /createRoom 2016-09-12 16:34:20 +01:00
Erik Johnston
706b5d76ed Fix backfill when cannot find an event.
`get_pdu` can succeed but return None.
2016-09-12 14:59:51 +01:00
Erik Johnston
7c679b1118 Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrent
Create new index concurrently
2016-09-12 14:38:40 +01:00
Erik Johnston
d080b3425c Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fast
Optimise /notifications query
2016-09-12 14:34:12 +01:00
Erik Johnston
03a98aff3c Create new index concurrently 2016-09-12 14:27:01 +01:00
Erik Johnston
fa20c9ce94 Change the index to be stream_ordering, highlight 2016-09-12 14:04:08 +01:00
Erik Johnston
5ef5435529 Remove unused import 2016-09-12 13:32:58 +01:00
Mark Haines
aa7b890cfe Merge pull request #1104 from matrix-org/markjh/direct_to_device_federation_sync
Fix direct to device messages recieved over federation to notify sync
2016-09-12 13:25:23 +01:00
Erik Johnston
7cd6edb947 Use register_background_index_update 2016-09-12 12:54:48 +01:00
Erik Johnston
0294c14ec4 Add back in query change 2016-09-12 12:43:56 +01:00
Erik Johnston
7fe42cf949 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast 2016-09-12 12:37:09 +01:00
Erik Johnston
15ca0c6a4d Make reindex happen in bg 2016-09-12 12:36:36 +01:00
Mark Haines
a232e06100 Fix direct to device messages recieved over federation to notify sync 2016-09-12 12:30:46 +01:00
Mark Haines
4a32d25d4c Merge branch 'develop' into markjh/bearer_token 2016-09-12 11:14:56 +01:00
Mark Haines
31f85f9db9 Add comments to existing schema deltas that used "CREATE INDEX" directly 2016-09-12 11:00:26 +01:00
Mark Haines
ec609f8094 Fix unit tests 2016-09-12 10:46:02 +01:00
Erik Johnston
54417999b6 Revert "Add index to event_push_actions" 2016-09-12 10:39:55 +01:00
Erik Johnston
45dc260060 Merge pull request #1101 from matrix-org/erikj/state_types_idx
Change state fetch query for postgres to be faster
2016-09-12 10:20:38 +01:00
Erik Johnston
d1c217c823 Merge pull request #1097 from matrix-org/erikj/replication_typing_rest
Correctly handle typing stream id resetting
2016-09-12 10:10:15 +01:00
Erik Johnston
897d57bc58 Change state fetch query for postgres to be faster
It turns out that postgres doesn't like doing a list of OR's and is
about 1000x slower, so we just issue a query for each specific type
seperately.
2016-09-12 10:05:07 +01:00
Erik Johnston
555460ae1b Merge pull request #1095 from matrix-org/erikj/batch_edus
Clobber EDUs in send queue
2016-09-12 08:04:15 +01:00
Richard van der Hoff
4162f820ff Merge pull request #1100 from VShell/fix-cas
Conform better to the CAS protocol specification
2016-09-09 21:57:42 +01:00
Shell Turner
29205e9596 Conform better to the CAS protocol specification
Redirect to CAS's /login endpoint properly, and
don't require an <attributes> element.

Signed-off-by: Shell Turner <cam.turn@gmail.com>
2016-09-09 21:20:14 +01:00
David Baker
b91e2833b3 Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast 2016-09-09 19:11:34 +01:00
David Baker
f2acc3dcf9 Add index to event_push_actions
and remove room_id caluse so it uses it

Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
Mark Haines
3ddec016ff Merge branch 'develop' into markjh/bearer_token 2016-09-09 18:51:22 +01:00
Mark Haines
8e01263587 Allow clients to supply access_tokens as headers
Clients can continue to supply access tokens as query parameters
or can supply the token as a header:

   Authorization: Bearer <access_token_goes_here>

This matches the ouath2 format of
https://tools.ietf.org/html/rfc6750#section-2.1
2016-09-09 18:17:42 +01:00
Erik Johnston
3265def8c7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus 2016-09-09 18:06:01 +01:00
Erik Johnston
af4701b311 Fix incorrect attribute name 2016-09-09 17:36:56 +01:00
Erik Johnston
44330a21e9 Comment 2016-09-09 17:22:07 +01:00
Erik Johnston
464ffd1b5e Comment 2016-09-09 17:17:23 +01:00
Erik Johnston
327425764e Add edu.type as part of key. Remove debug logging 2016-09-09 17:13:30 +01:00
Mark Haines
dbff7e9436 Merge pull request #1096 from matrix-org/markjh/get_access_token
Add helper function for getting access_tokens from requests
2016-09-09 17:09:27 +01:00
Erik Johnston
a4339de9de Correctly handle typing stream id resetting 2016-09-09 16:44:26 +01:00
Mark Haines
8aee5aa068 Add helper function for getting access_tokens from requests
Rather than reimplementing the token parsing in the various places.
This will make it easier to change the token parsing to allow access_tokens
in HTTP headers.
2016-09-09 16:33:15 +01:00
Erik Johnston
52b2318777 Clobber EDUs in send queue 2016-09-09 15:59:08 +01:00
Paul Evans
56f38d1776 Merge pull request #1091 from matrix-org/paul/third-party-lookup
Improvements to 3PE lookup API
2016-09-09 15:43:11 +01:00
Paul Evans
8cb252d00c Merge pull request #1094 from matrix-org/paul/get-state-whole-event
Allow clients to ask for the whole of a single state event
2016-09-09 15:26:17 +01:00
Paul "LeoNerd" Evans
776594f99d Log if rejecting 3PE query metadata result due to type check 2016-09-09 15:11:31 +01:00
Paul "LeoNerd" Evans
ed44c475d8 Reject malformed 3PE query metadata results earlier in AS API handling code 2016-09-09 15:07:04 +01:00
Erik Johnston
ab80d5e0a9 Drop replication log levels 2016-09-09 14:56:50 +01:00
Paul "LeoNerd" Evans
f25d74f69c Minor fixes from PR comments 2016-09-09 14:54:16 +01:00
Paul "LeoNerd" Evans
d271383e63 Filter returned events for client-facing format 2016-09-09 14:40:15 +01:00
Paul "LeoNerd" Evans
0fc0a3bdff Allow clients to specify the format a room state event is returned in 2016-09-09 14:34:29 +01:00
Erik Johnston
6c4d582144 Deduplicate presence in _update_states 2016-09-09 14:28:22 +01:00
Erik Johnston
685da5a3b0 Merge pull request #1092 from matrix-org/erikj/transaction_queue_check
Check if destination is ready for retry earlier
2016-09-09 13:49:56 +01:00
Erik Johnston
a6c6750166 Check if destination is ready for retry earlier 2016-09-09 13:46:05 +01:00
Paul "LeoNerd" Evans
bdbcfc2a80 appease pep8 2016-09-09 13:31:39 +01:00
Paul "LeoNerd" Evans
6eb0c8a2e4 Python isn't JavaScript; have to quote dict keys 2016-09-09 13:31:17 +01:00
Mark Haines
6b54fa81de Merge pull request #1089 from matrix-org/markjh/direct_to_device_stream
Track the max device stream_id in a separate table,
2016-09-09 13:31:07 +01:00
Paul "LeoNerd" Evans
25eb769b26 Efficiency fix for lookups of a single protocol 2016-09-09 13:25:02 +01:00
Paul "LeoNerd" Evans
3328428d05 Allow lookup of a single 3PE protocol query metadata 2016-09-09 13:19:04 +01:00
Erik Johnston
4598682b43 Fix tightloop on sending transaction 2016-09-09 13:12:53 +01:00
Paul "LeoNerd" Evans
033d43e419 Don't corrupt shared cache on subsequent protocol requests 2016-09-09 13:10:36 +01:00
Mark Haines
647c724573 Use the previous MAX value if any to set the stream_id 2016-09-09 11:52:44 +01:00
Erik Johnston
a15ba15e64 Merge pull request #1088 from matrix-org/erikj/transaction_queue_check
Correctly guard against multiple concurrent transactions
2016-09-09 11:49:06 +01:00
Mark Haines
6a6cbfcf1e Track the max_stream_device_id in a separate table, since we delete from the inbox table 2016-09-09 11:48:23 +01:00
Erik Johnston
d2688d7f03 Correctly guard against multiple concurrent transactions 2016-09-09 11:44:36 +01:00
Mark Haines
303b6f29f0 Merge pull request #1087 from matrix-org/markjh/reapply_delta
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
Erik Johnston
1fe7ca1362 Merge branch 'release-v0.17.3' of github.com:matrix-org/synapse into develop 2016-09-09 11:15:40 +01:00
Erik Johnston
66efcbbff1 Bump changelog and version 2016-09-09 11:07:39 +01:00
Mark Haines
0877157353 Just move the schema and add some DROPs 2016-09-09 11:04:47 +01:00
Erik Johnston
2ffec928e2 Reduce batch size to be under SQL limit 2016-09-09 11:03:31 +01:00
Erik Johnston
b390756150 Update last_device_stream_id_by_dest if there is nothing to send 2016-09-09 11:00:15 +01:00
Matthew Hodgson
b8f84f99ff Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
Mark Haines
43b77c5d97 Only catch databas errors 2016-09-08 17:44:21 +01:00
Paul "LeoNerd" Evans
2f267ee160 Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client 2016-09-08 17:43:53 +01:00
Mark Haines
7d5b142547 Add a stub run_upgrade 2016-09-08 17:39:11 +01:00
David Baker
c3276aef25 Merge pull request #1080 from matrix-org/dbkr/fix_notifications_api_with_from
Fix /notifications API when used with `from` param
2016-09-08 17:35:35 +01:00
Mark Haines
fa722a699c Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop 2016-09-08 17:35:16 +01:00
Erik Johnston
5c688739d6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/reindex_state_groups 2016-09-08 16:52:09 +01:00
Erik Johnston
ebb46497ba Add delta file 2016-09-08 16:38:54 +01:00
Mark Haines
91ec972277 Merge pull request #1084 from matrix-org/markjh/direct_to_device_wildcard
Support wildcard device_ids for direct to device messages
2016-09-08 16:28:08 +01:00
Erik Johnston
5beda10bbd Reindex state_groups_state after pruning 2016-09-08 16:18:01 +01:00
Erik Johnston
257025ac89 Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
Erik Johnston
3f9889bfd6 Use parse_string 2016-09-08 15:51:10 +01:00
Erik Johnston
caa22334b3 Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-09-08 15:33:00 +01:00
Erik Johnston
b152ee71fe Bump version and changelog 2016-09-08 15:23:44 +01:00
Mark Haines
a1c8f268e5 Support wildcard device_ids for direct to device messages 2016-09-08 15:13:05 +01:00
Erik Johnston
8b93af662d Check the user_id for presence/typing matches origin 2016-09-08 15:07:38 +01:00
Mark Haines
fa9d36e050 Merge branch 'develop' into markjh/direct_to_device_federation 2016-09-08 13:43:43 +01:00
David Baker
4ef222ab61 Implement only=highlight on /notifications 2016-09-08 13:43:35 +01:00
Erik Johnston
61cd9af09b Log delta files we're applying 2016-09-08 13:40:46 +01:00
Erik Johnston
791658b576 Add server param to /publicRooms 2016-09-08 11:53:05 +01:00
David Baker
c5b49eb7ca Fix /notifications API when used with from param 2016-09-08 09:40:10 +01:00
Erik Johnston
b568ca309c Temporarily disable sequential scans for state fetching 2016-09-08 09:38:54 +01:00
Mark Haines
3c320c006c Merge pull request #1077 from matrix-org/markjh/device_logging
Log the types and values when failing to store devices
2016-09-07 18:24:24 +01:00
Mark Haines
85b51fdd6b Log the types and values when failing to store devices 2016-09-07 17:19:18 +01:00
Mark Haines
43954d000e Add a new method to enqueue the device messages rather than sending a dummy EDU 2016-09-07 16:10:51 +01:00
Mark Haines
2a0159b8ae Fix the stream change cache to work over replication 2016-09-07 15:58:00 +01:00
Mark Haines
cb98ac261b Move the check for federated device_messages.
Move the check into _attempt_new_transaction.
Only delete messages if there were messages to delete.
2016-09-07 15:39:13 +01:00
Mark Haines
31a07d2335 Add stream change caches for device messages 2016-09-07 15:27:07 +01:00
Erik Johnston
513188aa56 Comment 2016-09-07 14:53:23 +01:00
Erik Johnston
fadb01551a Add appopriate framing clause 2016-09-07 14:39:01 +01:00
Erik Johnston
d25c20ccbe Use windowing function to make use of index 2016-09-07 14:22:22 +01:00
Mark Haines
7d893beebe Comment the add_messages storage functions 2016-09-07 12:03:37 +01:00
Erik Johnston
94a83b534f Merge pull request #1065 from matrix-org/erikj/state_storage
Move to storing state_groups_state as deltas
2016-09-07 09:39:58 +01:00
Mark Haines
d4a35ada28 Send device messages over federation 2016-09-06 18:16:20 +01:00
Mark Haines
e020834e4f Add storage methods for federated device messages 2016-09-06 15:12:13 +01:00
Mark Haines
2ad72da931 Add tables for federated device messages
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
2016-09-06 15:10:40 +01:00
Erik Johnston
3c4208a057 Record counts of state changes 2016-09-06 11:31:01 +01:00
Mark Haines
f4164edb70 Move _add_messages_to_device_inbox_txn into a separate method 2016-09-06 11:26:37 +01:00
Erik Johnston
2eed4d7af4 Merge pull request #1072 from matrix-org/erikj/presence_fiddle
Fiddle should_notify to better report stats
2016-09-06 10:57:48 +01:00
Erik Johnston
438ef47637 Short circuit if presence is the same 2016-09-06 10:28:35 +01:00
Erik Johnston
74a3b4a650 Fiddle should_notify to better report stats 2016-09-06 10:23:38 +01:00
Erik Johnston
d51b8a1674 Add quotes and be explicity about script-src 2016-09-05 17:35:01 +01:00
Erik Johnston
662b031a30 Allow PDF to be rendered from media repo 2016-09-05 17:25:26 +01:00
Erik Johnston
0595413c0f Scale the batch size so that we're not bitten by the minimum 2016-09-05 15:49:57 +01:00
Erik Johnston
a7032abb2e Correctly handle reindexing state groups that already have an edge 2016-09-05 15:07:23 +01:00
Erik Johnston
9e6d88f4e2 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-05 15:01:33 +01:00
Erik Johnston
8c93e0bae7 Merge pull request #1070 from matrix-org/erikj/presence_stats
Record why we have chosen to notify
2016-09-05 15:01:30 +01:00
Erik Johnston
70332a12dd Take value in a better way 2016-09-05 14:57:14 +01:00
Erik Johnston
373654c635 Comment about sqlite and WITH RECURSIVE 2016-09-05 14:50:36 +01:00
Erik Johnston
485d999c8a Correctly delete old state groups in purge history API 2016-09-05 14:49:08 +01:00
Erik Johnston
69054e3d4c Record why we have chosen to notify 2016-09-05 14:12:11 +01:00
Erik Johnston
69a2d4e38c Use get_joined_users_from_context instead of manually looking up hosts 2016-09-05 13:44:40 +01:00
Erik Johnston
f12993ec16 Bump changelog and version 2016-09-05 13:01:14 +01:00
Erik Johnston
bb36b93f71 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-05 11:53:11 +01:00
Erik Johnston
d87b87adf7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/bulk_push 2016-09-05 11:52:49 +01:00
Erik Johnston
caed150363 Remove unused imports 2016-09-05 10:52:01 +01:00
Erik Johnston
80a6a445fa Only fetch local pushers 2016-09-05 10:43:32 +01:00
Erik Johnston
628e65721b Add comments 2016-09-05 10:41:27 +01:00
Mark Haines
274c2f50a5 Merge pull request #1067 from matrix-org/markjh/idempotent
Fix membership changes to be idempotent
2016-09-05 10:21:25 +01:00
Erik Johnston
a99e933550 Add upgrade script that will slowly prune state_groups_state entries 2016-09-05 10:05:36 +01:00
Erik Johnston
3847fa38c4 Make bulk_get_push_rules_for_room use get_joined_users_from_context cache 2016-09-05 10:02:38 +01:00
Mark Haines
f2690c6423 Fix membership changes to be idempotent 2016-09-02 19:23:22 +01:00
Mark Haines
81b94c5750 Merge pull request #1066 from matrix-org/markjh/direct_to_device_lowerbound
Only return new device messages in /sync
2016-09-02 16:18:34 +01:00
Mark Haines
65fa37ac5e Only return new device messages in /sync 2016-09-02 15:50:37 +01:00
Erik Johnston
3baf641a48 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-02 14:54:07 +01:00
Erik Johnston
c0238ecbed Explicitly specify state_key for history_visibility fetching 2016-09-02 14:53:46 +01:00
Erik Johnston
273b6bcf22 Merge pull request #1064 from matrix-org/erikj/on_receive_check
Only check if host is in room if we have state and auth_chain
2016-09-02 14:43:35 +01:00
Erik Johnston
f7f1027d3d Comment on when auth chain and state are None 2016-09-02 14:42:38 +01:00
Erik Johnston
34e5e17f91 Comment 2016-09-02 14:26:07 +01:00
Erik Johnston
b96c6c3185 Docstrings 2016-09-02 14:19:22 +01:00
Erik Johnston
1ffe9578d1 Merge pull request #1063 from matrix-org/erikj/pull_out_ids_only
Only pull out IDs from DB for /state_ids/ request
2016-09-02 14:16:28 +01:00
Erik Johnston
cce957e254 Bump max_entries on get_destination_retry_timings 2016-09-02 14:08:33 +01:00
Erik Johnston
bd9b8d87ae Only check if host is in room if we have state and auth_chain 2016-09-02 13:40:28 +01:00
Erik Johnston
657847e4c6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-02 11:04:48 +01:00
Mark Haines
965168a842 Merge branch 'develop' into markjh/direct_to_device_synchrotron 2016-09-02 10:59:24 +01:00
Erik Johnston
2854ee2a52 Only pull out IDs from DB for /state_ids/ request 2016-09-02 10:53:36 +01:00
Erik Johnston
598317927c Limit the length of state chains 2016-09-02 10:41:38 +01:00
Mark Haines
7ed5acacf4 Fix up the calls to the notifier for device messages 2016-09-01 18:08:40 +01:00
Erik Johnston
051a9ea921 Linearize state resolution to help caches 2016-09-01 14:55:03 +01:00
Erik Johnston
265d847ffd Fix typo in log line 2016-09-01 14:50:06 +01:00
Erik Johnston
f4778d4cd9 Merge branch 'erikj/pdu_check' of github.com:matrix-org/synapse into develop 2016-09-01 14:40:36 +01:00
Erik Johnston
9e25443db8 Move to storing state_groups_state as deltas 2016-09-01 14:31:26 +01:00
Erik Johnston
44982606ee Merge pull request #1060 from matrix-org/erikj/state_ids
Assign state groups in state handler.
2016-09-01 14:20:42 +01:00
Erik Johnston
516a272aca Ensure we only return a validated pdu in get_pdu 2016-09-01 10:55:02 +01:00
Erik Johnston
0cfd6c3161 Use state_groups table to test existence 2016-08-31 16:25:57 +01:00
Erik Johnston
5405351b14 Lower get_linearized_receipts_for_room cache size 2016-08-31 16:19:44 +01:00
Erik Johnston
ed7a703d4c Handle the fact that workers can't generate state groups 2016-08-31 15:53:19 +01:00
Erik Johnston
1671913287 Merge pull request #1059 from matrix-org/erikj/sent_transaction_delete
Clean up old sent transactions
2016-08-31 15:07:33 +01:00
Erik Johnston
f51888530d Always specify state_group so that its in the cache 2016-08-31 14:58:11 +01:00
Erik Johnston
826ca61745 Add storage function to SlaveStore 2016-08-31 14:45:04 +01:00
Erik Johnston
fbd2615de4 Merge pull request #1057 from matrix-org/erikj/fix_email_name
Fix email notifs by adding missing param
2016-08-31 14:33:45 +01:00
Erik Johnston
c10cb581c6 Correctly handle the difference between prev and current state 2016-08-31 14:26:22 +01:00
Erik Johnston
ef0cc648cf Clean up old sent transactions 2016-08-31 11:12:02 +01:00
Mark Haines
761f9fccff Merge pull request #1058 from matrix-org/markjh/direct_to_device_synchrotron
Add a replication stream for direct to device messages
2016-08-31 11:08:18 +01:00
Mark Haines
a662252758 Return the current stream position from add_messages_to_device_inbox 2016-08-31 10:42:52 +01:00
Mark Haines
1aa3e1d287 Add a replication stream for direct to device messages 2016-08-31 10:38:58 +01:00
Erik Johnston
1bb8ec296d Generate state group ids in state layer 2016-08-31 10:09:46 +01:00
Erik Johnston
d80f64d370 Fix email notifs by adding missing param 2016-08-30 21:46:39 +01:00
Kegan Dougal
c882783535 flake8 2016-08-30 17:20:31 +01:00
Kegan Dougal
572acde483 Use None instead of the empty string
Change how we validate the 'url' field as a result.
2016-08-30 17:16:00 +01:00
Erik Johnston
5dc2a702cf Make _state_groups_id_gen a normal IdGenerator 2016-08-30 16:55:11 +01:00
Erik Johnston
3e784eff74 Remove state replication stream 2016-08-30 16:51:36 +01:00
Kegan Dougal
16b652f0a3 Flake8 2016-08-30 16:30:12 +01:00
Kegan Dougal
e82247f990 Allow application services to have an optional 'url'
If 'url' is not specified, they will not be pushed for events or queries. This
is useful for bots who simply wish to reserve large chunks of user/alias
namespace, and don't care about being pushed for events.
2016-08-30 16:21:16 +01:00
Erik Johnston
c7f665d700 Merge pull request #1055 from matrix-org/erikj/occaisonally_persist
Occaisonally persist unpersisted presence updates
2016-08-30 15:59:00 +01:00
Erik Johnston
097330bae8 Check correct variable 2016-08-30 15:50:20 +01:00
Erik Johnston
21b977ccfe Occaisonally persist unpersisted presence updates 2016-08-30 15:39:50 +01:00
Erik Johnston
bc1a8b1f7a Don't notify for online -> online transitions.
Specifically, if currently_active remains true then we should not notify
if only the last active time changes.
2016-08-30 15:05:32 +01:00
Erik Johnston
67f0c990f8 Merge pull request #1051 from matrix-org/erikj/fix_push_names
Fix push room names for rooms with only an alias
2016-08-30 11:50:34 +01:00
Erik Johnston
c8cd87b21b Comment about message deletion 2016-08-30 11:23:26 +01:00
Erik Johnston
55e17d3697 Fix push room names for rooms with only an alias 2016-08-30 11:19:59 +01:00
Erik Johnston
1ee6285905 Fix check 2016-08-30 11:17:46 +01:00
Erik Johnston
68e1a872fd Noop get_new_messages_for_device if token hasn't changed 2016-08-30 10:58:46 +01:00
Erik Johnston
55fc17cf4b Merge pull request #1049 from matrix-org/erikj/presence_users_in_room
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-30 10:50:37 +01:00
Erik Johnston
41788bba50 Add to slave store 2016-08-30 09:55:17 +01:00
Erik Johnston
873f870e5a Add new direct message storage functions to slave store 2016-08-30 09:40:32 +01:00
Erik Johnston
bed10f9880 Use state handler instead of get_users_in_room/get_joined_hosts 2016-08-26 14:54:30 +01:00
Mark Haines
4bbef62124 Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device 2016-08-26 14:35:31 +01:00
Erik Johnston
a234e895cf Fix room name in email notifs 2016-08-26 14:10:21 +01:00
Erik Johnston
4daa397a00 Add is_host_joined to slave storage 2016-08-26 13:10:56 +01:00
Erik Johnston
c7cd35d682 Typo 2016-08-26 11:23:58 +01:00
Erik Johnston
54cc69154e Make None optional 2016-08-26 11:20:59 +01:00
Erik Johnston
11faa4296d Measure _filter_events_for_server 2016-08-26 11:15:40 +01:00
Erik Johnston
f6338d6a3e Don't pull out full state for _filter_events_for_server 2016-08-26 11:13:16 +01:00
Erik Johnston
1ccdc1e93a Cache check_host_in_room 2016-08-26 10:59:40 +01:00
Erik Johnston
25414b44a2 Add measure on check_host_in_room 2016-08-26 10:47:00 +01:00
Erik Johnston
50943ab942 Add new state storage funcs to replication 2016-08-26 09:57:32 +01:00
Erik Johnston
30961182f2 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids 2016-08-26 09:48:13 +01:00
Erik Johnston
c1a133a6b6 Merge pull request #1043 from matrix-org/erikj/backfill_fix
Fix None check in backfill
2016-08-26 09:07:05 +01:00
Erik Johnston
778fa85f47 Make sync not pull out full state 2016-08-25 18:59:44 +01:00
Erik Johnston
7356d52e73 Fix up push to use get_current_state_ids 2016-08-25 18:35:49 +01:00
Paul "LeoNerd" Evans
9459137f1e Just sprintf the 'kind' argument into uri directly 2016-08-25 18:35:38 +01:00
Paul "LeoNerd" Evans
1294d4a329 Move ThirdPartyEntityKind into api.constants so the expectation becomes that the value is significant 2016-08-25 18:34:47 +01:00
Mark Haines
ab34fdecb7 Merge branch 'develop' into markjh/direct_to_device 2016-08-25 18:34:46 +01:00
Mark Haines
b162cb2e41 Add some TODOs 2016-08-25 18:18:53 +01:00
Erik Johnston
0e1900d819 Pull out full state less 2016-08-25 18:15:51 +01:00
Mark Haines
641efb6a39 Fix the deduplication of incoming direct-to-device messages 2016-08-25 18:14:02 +01:00
Paul "LeoNerd" Evans
e7af8be5ae Root the 3PE lookup API within /_matrix/app/unstable instead of at toplevel 2016-08-25 18:06:29 +01:00
Paul "LeoNerd" Evans
142983b4ea APP_SERVICE_PREFIX is never used; don't bother 2016-08-25 18:06:05 +01:00
Erik Johnston
721414d98a Add desc 2016-08-25 17:49:05 +01:00
Mark Haines
e993925279 Add store-and-forward direct-to-device messaging 2016-08-25 17:35:37 +01:00
Erik Johnston
a3dc1e9cbe Replace context.current_state with context.current_state_ids 2016-08-25 17:32:22 +01:00
Paul Evans
d9dcb2ba3a Merge pull request #1041 from matrix-org/paul/third-party-lookup
Extend 3PE lookup APIs for metadata query
2016-08-25 17:06:53 +01:00
Paul "LeoNerd" Evans
adf53f04ce appease pep8 2016-08-25 16:00:31 +01:00
Paul "LeoNerd" Evans
c435bfee9c Don't need toplevel cache on 3PE lookup metadata any more 2016-08-25 15:57:07 +01:00
Paul "LeoNerd" Evans
db7283cc6b Implement a ResponseCache around 3PE lookup metadata lookups 2016-08-25 15:56:27 +01:00
Paul "LeoNerd" Evans
d0b8d49f71 Kill PROTOCOL_META since I'm not using it any more 2016-08-25 15:45:28 +01:00
Paul "LeoNerd" Evans
5474824975 Actually query over AS API for 3PE lookup metadata 2016-08-25 15:29:36 +01:00
Erik Johnston
17f4f14df7 Pull out event ids rather than full events for state 2016-08-25 13:42:44 +01:00
Erik Johnston
cd5b264b03 Fix None check in backfill 2016-08-25 10:39:19 +01:00
Erik Johnston
60a015550a Bump changelog and version 2016-08-24 14:31:13 +01:00
Erik Johnston
90d5983d7a Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.1 2016-08-24 14:28:57 +01:00
Erik Johnston
c20cb5160d Remove tracer 2016-08-24 13:22:47 +01:00
Erik Johnston
fda97dd58a Merge branch 'develop' of github.com:matrix-org/synapse into erikj/preserve_log_contexts 2016-08-24 13:22:02 +01:00
Paul "LeoNerd" Evans
8e1ed09dff Move static knowledge of protocol metadata into AS handler; cache the result 2016-08-24 13:01:53 +01:00
Paul "LeoNerd" Evans
965f33c901 Declare 'gitter' known protocol, with user lookup 2016-08-24 12:34:03 +01:00
Paul "LeoNerd" Evans
9899824b85 Initial hack at the 3PN protocols metadata lookup API 2016-08-24 12:33:01 +01:00
Erik Johnston
9219139351 Preserve some logcontexts 2016-08-24 11:58:40 +01:00
Paul "LeoNerd" Evans
63c19e1df9 Move 3PU/3PL lookup APIs into /thirdparty containing entity 2016-08-24 11:55:57 +01:00
Erik Johnston
86bcf4d6a7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/pagination 2016-08-24 10:38:21 +01:00
Erik Johnston
ba07d4a70e Add None checks to backfill 2016-08-24 10:31:05 +01:00
Kegan Dougal
4b31426a02 Pass through user-supplied content in /join/$room_id
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.

If the user specifies keys like "avatar_url" then they will be clobbered.
2016-08-23 16:32:04 +01:00
Erik Johnston
122c7a43c9 Merge pull request #1038 from matrix-org/erikj/receved_txn_purge
Delete old received_transactions rows
2016-08-23 11:02:40 +01:00
Erik Johnston
14047126d8 Bump changelog and version 2016-08-22 18:26:07 +01:00
Erik Johnston
d143f211c8 Merge pull request #1028 from matrix-org/dbkr/notifications_api
Add the Notifications API
2016-08-22 18:23:24 +01:00
Erik Johnston
aad8a1a825 Delete old received_transactions 2016-08-22 16:29:46 +01:00
Mark Haines
689f4cb914 Update comment 2016-08-22 16:17:31 +01:00
Mark Haines
c8f9b45bc2 Add usage stats to prometheus monitoring 2016-08-22 15:34:38 +01:00
Erik Johnston
e65bc7d315 Merge pull request #1031 from matrix-org/erikj/measure_notifier
Add more Measure blocks
2016-08-22 12:13:07 +01:00
Erik Johnston
33f3624ff7 Add exception logging. Fix typo 2016-08-22 10:49:31 +01:00
Erik Johnston
8c52160b07 Allow request handlers to override metric name 2016-08-22 10:44:45 +01:00
Erik Johnston
a093fab253 Use top level measure 2016-08-22 10:18:12 +01:00
Matthew Hodgson
6e80c03d45 Merge branch 'develop' into dbkr/notifications_api 2016-08-20 00:16:18 +01:00
Erik Johnston
27e0178da9 Add a top level measure 2016-08-19 18:49:37 +01:00
Erik Johnston
6d1a94d218 Remove redundant measure 2016-08-19 18:40:31 +01:00
Erik Johnston
8731197e54 Only abort Measure on Exceptions 2016-08-19 18:23:45 +01:00
Erik Johnston
afbf6b33fc defer.returnValue must not be called within Measure 2016-08-19 18:23:44 +01:00
Erik Johnston
37adde32dc Move defer.returnValue out of Measure 2016-08-19 18:23:44 +01:00
Erik Johnston
04fc8bbcb0 Update keyring Measure 2016-08-19 18:23:44 +01:00
Erik Johnston
39b900b316 Measure http.server render 2016-08-19 18:23:44 +01:00
Erik Johnston
47dd8f02a1 Measure _get_event_from_row 2016-08-19 18:23:44 +01:00
Erik Johnston
2426c2f21a Measure keyrings 2016-08-19 18:23:44 +01:00
Erik Johnston
39242090e3 Add measure blocks to notifier 2016-08-19 18:23:44 +01:00
Erik Johnston
e6784daf07 Merge pull request #1030 from matrix-org/erikj/cache_contexts
Add concept of cache contexts
2016-08-19 16:29:58 +01:00
Erik Johnston
45fd2c8942 Ensure invalidation list does not grow unboundedly 2016-08-19 16:09:16 +01:00
Erik Johnston
c0d7d9d642 Rename to on_invalidate 2016-08-19 15:13:58 +01:00
Erik Johnston
dc76a3e909 Make cache_context an explicit option 2016-08-19 15:02:38 +01:00
Erik Johnston
f164fd9220 Move _bulk_get_push_rules_for_room to storage layer 2016-08-19 14:29:20 +01:00
Erik Johnston
ba214a5e32 Remove lru option 2016-08-19 14:17:11 +01:00
Erik Johnston
4161ff2fc4 Add concept of cache contexts 2016-08-19 14:17:07 +01:00
Erik Johnston
b770435389 Make get_new_events_for_appservice use indices 2016-08-19 10:28:42 +01:00
Paul Evans
5674ea3e6c Merge pull request #1026 from matrix-org/paul/thirdpartylookup
3rd party entity lookup
2016-08-18 20:52:50 +01:00
David Baker
1e4217c90c Explicit join 2016-08-18 17:53:44 +01:00
David Baker
0acdd0f1ea Use tuple comparison
Hopefully easier to read
2016-08-18 17:51:08 +01:00
Paul "LeoNerd" Evans
65201631a4 Move validation logic for AS 3PE query response into ApplicationServiceApi class, to keep the handler logic neater 2016-08-18 17:33:56 +01:00
Paul "LeoNerd" Evans
697872cf08 More warnings about invalid results from AS 3PE query 2016-08-18 17:24:39 +01:00
Paul "LeoNerd" Evans
b515f844ee Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration 2016-08-18 17:19:55 +01:00
David Baker
602c84cd9c Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-18 17:15:26 +01:00
Paul "LeoNerd" Evans
2a91799fcc Minor syntax neatenings 2016-08-18 16:58:25 +01:00
Paul "LeoNerd" Evans
fcf1dec809 Appease pep8 2016-08-18 16:26:19 +01:00
Paul "LeoNerd" Evans
105ff162d4 Authenticate 3PE lookup requests 2016-08-18 16:19:23 +01:00
Paul "LeoNerd" Evans
06964c4a0a Copypasta the 3PU support code to also do 3PL 2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans
f3afd6ef1a Remove TODO note about request fields being strings - they're always strings 2016-08-18 15:53:01 +01:00
Erik Johnston
bcbd74dc5b Remove log lines 2016-08-18 15:52:10 +01:00
Paul "LeoNerd" Evans
d7b42afc74 Log a warning if an AS yields an invalid 3PU lookup result 2016-08-18 15:49:55 +01:00
Paul "LeoNerd" Evans
80f4740c8f Scattergather the call out to ASes; validate received results 2016-08-18 15:40:41 +01:00
Erik Johnston
07229bbdae Add appservice worker 2016-08-18 14:59:55 +01:00
Paul "LeoNerd" Evans
434bbf2cb5 Filter 3PU lookups by only ASes that declare knowledge of that protocol 2016-08-18 14:56:02 +01:00
Paul "LeoNerd" Evans
d5bf7a4a99 Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup 2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans
718ffcf8bb Since empty lookups now return 200/empty list not 404, we can safely log failures as exceptions 2016-08-18 14:18:37 +01:00
Paul "LeoNerd" Evans
3856582741 Ensure that 3PU lookup request fields actually get passed in 2016-08-18 14:06:02 +01:00
Paul "LeoNerd" Evans
f0c73a1e7a Extend individual list results into the main return list, don't append 2016-08-18 13:53:54 +01:00
Paul "LeoNerd" Evans
b3511adb92 Don't catch the return-value-as-exception that @defer.inlineCallbacks will use 2016-08-18 13:45:18 +01:00
Erik Johnston
9da84a9a1e Make AppserviceHandler stream events from database
This is for two reasons:

1. Suppresses duplicates correctly, as the notifier doesn't do any
   duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
   stream.
2016-08-18 11:54:41 +01:00
Mark Haines
403ecd8a2c Missed a s/federation reader/media repository/ in a log message 2016-08-18 10:26:15 +01:00
Mark Haines
47fbff7a05 Merge pull request #1024 from matrix-org/markjh/media_repository
Add a media repository worker
2016-08-18 10:12:17 +01:00
Mark Haines
396624864a Add a media repository worker 2016-08-18 09:38:42 +01:00
Erik Johnston
e73dcb66da Merge pull request #1022 from matrix-org/erikj/as_notify_perf
Make notify_interested_services faster
2016-08-18 09:35:19 +01:00
Paul "LeoNerd" Evans
3ec10dffd6 Actually make 3PU lookup calls out to ASes 2016-08-18 00:39:09 +01:00
Erik Johnston
732cf72b86 Fix push_display_name_rename schema update 2016-08-17 18:12:21 +01:00
Erik Johnston
abcb9aee5b Make push Measure finer grained 2016-08-17 18:00:18 +01:00
Erik Johnston
2a0d8f8219 Merge pull request #1021 from matrix-org/erikj/mediasecurity_policy
Set `Content-Security-Policy` on media repo
2016-08-17 17:22:43 +01:00
Erik Johnston
320dfe523c Make notify_interested_services faster 2016-08-17 17:20:50 +01:00
Erik Johnston
0af9e1a637 Set Content-Security-Policy on media repo
This is to inform browsers that they should sandbox the returned
media. This is particularly cruical for javascript/HTML files.
2016-08-17 16:27:39 +01:00
Paul "LeoNerd" Evans
fa87c981e1 Thread 3PU lookup through as far as the AS API object; which currently noöps it 2016-08-17 16:17:28 +01:00
David Baker
0d7cef0943 Merge pull request #1011 from matrix-org/dbkr/contains_display_name_override
Move display name rule
2016-08-17 16:15:45 +01:00
Erik Johnston
f90b3d83a3 Add None check to _iterate_over_text 2016-08-17 15:17:17 +01:00
Matrix
f743471380 Change name of metric 2016-08-17 15:05:50 +01:00
Erik Johnston
b9e888858c Move Measure block inside loop 2016-08-17 14:52:26 +01:00
Erik Johnston
973d67a033 Merge pull request #1019 from matrix-org/erikj/appservice_clean
Clean up _ServiceQueuer
2016-08-17 14:37:21 +01:00
Paul "LeoNerd" Evans
e3e3fbc23a Initial empty implementation that just registers an API endpoint handler 2016-08-17 12:46:49 +01:00
Erik Johnston
7321f45457 Clean up _ServiceQueuer 2016-08-17 12:03:04 +01:00
Erik Johnston
b9abf3e4e3 Remove dead appservice code 2016-08-17 11:48:23 +01:00
Erik Johnston
62c5245c87 Measure notify_interested_services 2016-08-17 11:12:29 +01:00
Erik Johnston
949629291c Do it in storage function 2016-08-16 17:05:34 +01:00
David Baker
ad42322257 Add migration script
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
David Baker
0bba2799b6 Merge remote-tracking branch 'origin/develop' into dbkr/contains_display_name_override 2016-08-16 16:46:37 +01:00
Erik Johnston
64a2acb161 Don't update caches replication stream if tokens haven't advanced 2016-08-16 16:44:19 +01:00
David Baker
1594eba29e s/underride/override/ in the rule_id too 2016-08-16 16:44:07 +01:00
Erik Johnston
109a560905 Flake8 2016-08-16 14:57:21 +01:00
Erik Johnston
48b5829aea Fix up preview URL API. Add tests.
This includes:

- Splitting out methods of a class into stand alone functions, to make
  them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
Erik Johnston
7c6f4f9427 Merge pull request #1012 from matrix-org/erikj/limit_backfill_uri
Limit number of extremeties in backfill request
2016-08-16 12:55:42 +01:00
Erik Johnston
25c2332071 Merge pull request #1010 from matrix-org/erikj/refactor_deletions
Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.
2016-08-16 11:37:53 +01:00
Erik Johnston
2ee1bd124c Limit number of extremeties in backfill request
This works around a bug where if we make a backfill request with too
many extremeties it causes the request URI to be too long.
2016-08-16 11:34:36 +01:00
Erik Johnston
a2427981b7 Use cached get_user_by_access_token in slaves 2016-08-16 11:24:32 +01:00
Erik Johnston
6cbd1b495e Merge branch 'fix_integrity_retry' of https://github.com/Ralith/synapse into Ralith-fix_integrity_retry 2016-08-16 10:50:24 +01:00
David Baker
1c7c317df1 Move display name rule
As per https://github.com/matrix-org/matrix-doc/pull/373 and comment
2016-08-15 18:34:53 +01:00
Erik Johnston
dc3a00f24f Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves. 2016-08-15 17:04:39 +01:00
Erik Johnston
75299af4fc Merge pull request #1009 from matrix-org/erikj/event_split
Split out /events to synchrotron
2016-08-15 15:39:05 +01:00
Erik Johnston
89e786bd85 Doc get_next() context manager usage 2016-08-15 13:45:26 +01:00
Erik Johnston
d9664344ec Rename table. Add docs. 2016-08-15 11:45:57 +01:00
Erik Johnston
784a2d4f2c Remove broken cache stuff 2016-08-15 11:25:48 +01:00
Erik Johnston
0be963472b Use cached version of get_aliases_for_room 2016-08-15 11:24:12 +01:00
Erik Johnston
64e7e11853 Implement cache replication stream 2016-08-15 11:16:45 +01:00
Erik Johnston
4d70d1f80e Add some invalidations to a cache_stream 2016-08-15 11:15:17 +01:00
Erik Johnston
99bbd90b0d Always run txn.after_callbacks 2016-08-15 09:45:44 +01:00
Benjamin Saunders
8a57cc3123 Add missing database corruption recovery case
Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
2016-08-14 11:50:22 -07:00
Daniel Ehlers
dfaf0fee31
Log the value which is observed in the first place.
The name 'result' is of bool type and has no len property,
resulting in a TypeError. Futhermore in the flow control
conn.response is observed and hence should be reported.

Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:49:05 +02:00
Daniel Ehlers
e380538b59
Fix AttributeError when bind_dn is not defined.
In case one does not define bind_dn in ldap configuration, filter
attribute is not declared. Since auth code only uses ldap_filter attribute
when according LDAP mode is selected, it is safe to only declare the
attribute in that case.

Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
2016-08-14 16:48:33 +02:00
Erik Johnston
4e1cebd56f Make synchrotron accept /events 2016-08-12 15:31:44 +01:00
Erik Johnston
866a5320de Dont invoke get_handlers fromClientV1RestServlet
hs.get_handlers() can not be invoked from split out processes. Moving
the invocations down a level means that we can slowly split out
individual servlets.
2016-08-12 10:03:19 +01:00
Erik Johnston
448ac6cf0d Only process one local membership event per room at a time 2016-08-12 09:32:19 +01:00
Erik Johnston
832799dbff Merge pull request #997 from Half-Shot/develop
Don't change status_msg on /sync
2016-08-11 14:10:55 +01:00
David Baker
b4ecf0b886 Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-11 14:09:13 +01:00
Will Hunt
5b5148b7ec Synced up synchrotron set_state with PresenceHandler set_state 2016-08-11 11:48:30 +01:00
Erik Johnston
1d40373c9d Include prev_content in redacted state events 2016-08-11 10:24:41 +01:00
Erik Johnston
c315922b5f PEP8 2016-08-10 16:34:10 +01:00
Erik Johnston
ca8abfbf30 Clean up TransactionQueue 2016-08-10 16:24:16 +01:00
Erik Johnston
5aeadb7414 Merge pull request #999 from matrix-org/erikj/measure_more
Measure federation send transaction resources
2016-08-10 14:16:14 +01:00
Erik Johnston
487bc49bf8 Don't stop on 4xx series errors 2016-08-10 13:39:12 +01:00
Erik Johnston
739ea29d1e Also check if server is in the room 2016-08-10 13:32:23 +01:00
Erik Johnston
ea8c4094db Also pull out rejected events 2016-08-10 13:26:13 +01:00
Erik Johnston
7f41bcbeec Correctly auth /event/ requests 2016-08-10 13:22:20 +01:00
Erik Johnston
11fdfaf03b Only resign our own events 2016-08-10 13:16:58 +01:00
Will Hunt
2510db3e76 Don't change status_msg on /sync 2016-08-10 12:59:59 +01:00
Erik Johnston
f91df1f761 Store if we fail to fetch an event from a destination 2016-08-10 11:31:46 +01:00
Erik Johnston
3bc9629be5 Measure federation send transaction resources 2016-08-10 10:56:38 +01:00
Erik Johnston
d45489474d Merge pull request #996 from matrix-org/erikj/tls_error
Don't print stack traces when failing to get remote keys
2016-08-10 10:56:08 +01:00
Erik Johnston
fa1ce4d8ad Don't print stack traces when failing to get remote keys 2016-08-10 10:44:37 +01:00
Richard van der Hoff
79ebfbe7c6 /login: Respond with a 403 when we get an invalid m.login.token 2016-08-09 16:29:28 +01:00
David Baker
cd41c6ece2 Merge pull request #995 from matrix-org/rav/clean_up_cas_login
Clean up CAS login code
2016-08-09 10:21:56 +01:00
David Baker
27771b2495 Merge pull request #994 from matrix-org/rav/fix_cas_login
Fix CAS login
2016-08-08 17:47:45 +01:00
Richard van der Hoff
d3250499c1 Merge pull request #993 from matrix-org/rav/fix_token_login
Fix token login
2016-08-08 17:43:02 +01:00
Richard van der Hoff
65666fedd5 Clean up CAS login code
Remove some apparently unused code.

Clean up parse_cas_response, mostly to catch the exception if the CAS response
isn't valid XML.
2016-08-08 17:17:25 +01:00
Richard van der Hoff
0682ca04b3 Fix CAS login
Attempting to log in with CAS was giving a 500 error.
2016-08-08 17:01:30 +01:00
Richard van der Hoff
6fe6a6f029 Fix login with m.login.token
login with token (as used by CAS auth) was broken by 067596d, such that it
always returned a 401.
2016-08-08 16:40:39 +01:00
Erik Johnston
27fe3e2d4f Bump changelog and version 2016-08-08 11:31:12 +01:00
Erik Johnston
3410142741 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0 2016-08-08 11:24:53 +01:00
Erik Johnston
7c1a92274c Make psutil optional 2016-08-08 11:12:21 +01:00
Erik Johnston
f5deaff424 Merge pull request #991 from matrix-org/erikj/retry_make
Retry joining via other servers if first one failed. Fix some other bugs.
2016-08-05 18:21:27 +01:00
Erik Johnston
5f360182c6 Fix a couple of python bugs 2016-08-05 18:08:32 +01:00
Erik Johnston
46453bfc2f Retry joining via other servers if first one failed 2016-08-05 18:02:03 +01:00
Erik Johnston
6bf6bc1d1d Merge pull request #990 from matrix-org/erikj/fed_vers
Add federation /version API
2016-08-05 16:59:16 +01:00
Erik Johnston
24f36469bc Add federation /version API 2016-08-05 16:36:07 +01:00
Erik Johnston
597c79be10 Change the way we specify if we require auth or not 2016-08-05 16:17:04 +01:00
Erik Johnston
e6021c370e Merge pull request #989 from matrix-org/erikj/raise_404
Raise 404 when couldn't find event
2016-08-05 15:54:16 +01:00
Erik Johnston
a8b946decb Raise 404 when couldn't find event 2016-08-05 15:31:02 +01:00
Erik Johnston
5bcccfde6c Don't include html comments in description 2016-08-05 14:45:11 +01:00
Erik Johnston
4d87d3659a Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0 2016-08-05 13:19:22 +01:00
Erik Johnston
93acf49e9b Fix backfill auth events 2016-08-05 12:59:04 +01:00
Erik Johnston
b2c290a6e5 Bump version and changelog 2016-08-05 11:16:54 +01:00
Erik Johnston
499dc1b349 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0 2016-08-05 11:15:48 +01:00
Erik Johnston
2d4de61fb7 Fix typo 2016-08-05 10:48:56 +01:00
Erik Johnston
fccadb7719 Check if we already have the events returned by /state/ 2016-08-05 10:43:47 +01:00
Erik Johnston
f0fa66f495 Delete more tables 2016-08-05 10:40:08 +01:00
Erik Johnston
1515d1b581 Fallback to /state/ on both 400 and 404 2016-08-05 10:24:23 +01:00
Benjamin Saunders
a2b7102eea Tweak integrity error recovery to work as intended 2016-08-04 20:38:08 -07:00
Erik Johnston
a5d7968b3e Merge pull request #973 from matrix-org/erikj/xpath_fix
Change the way we summarize URLs
2016-08-04 16:37:25 +01:00
Erik Johnston
b5525c76d1 Typo 2016-08-04 16:10:08 +01:00
Erik Johnston
e97648c4e2 Test summarization 2016-08-04 16:09:09 +01:00
Erik Johnston
835ceeee76 Merge pull request #983 from matrix-org/erikj/retry_on_integrity_error
Retry event persistence on IntegrityError
2016-08-04 15:40:03 +01:00
Erik Johnston
b3682df2ca Merge branch 'develop' of github.com:matrix-org/synapse into erikj/xpath_fix 2016-08-04 15:29:45 +01:00
Erik Johnston
8ad8490cff Fix typo 2016-08-04 15:21:29 +01:00
Erik Johnston
59fa91fe88 Retry event persistence on IntegrityError
Due to a bug in the porting script some backfilled events were not
correctly persisted, causing irrecoverable IntegrityErrors on future
attempts to persist those events.

This commit adds a retry mechanism invoked upon IntegrityError,
where when retried the tables are purged for all references to the
events being persisted.
2016-08-04 15:02:15 +01:00
Erik Johnston
257c41cc2e Fix typos. 2016-08-04 14:05:45 +01:00
Erik Johnston
b4e2290d89 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids_api 2016-08-04 14:04:35 +01:00
Erik Johnston
e3ee63578f Tidy up get_events 2016-08-04 14:01:18 +01:00
Richard van der Hoff
f131cd9e53 keys/query: Omit device displayname if null
... which makes it more consistent with user displaynames.
2016-08-04 10:59:51 +01:00
Erik Johnston
edb33eb163 Rename fields to _ids 2016-08-03 17:19:15 +01:00
Erik Johnston
bcc9cda8ca Fix copy + paste fails 2016-08-03 17:17:26 +01:00
Richard van der Hoff
98385888b8 PEP8 2016-08-03 15:42:08 +01:00
Richard van der Hoff
68264d7404 Include device name in /keys/query response
Add an 'unsigned' section which includes the device display name.
2016-08-03 15:42:08 +01:00
Richard van der Hoff
91fa69e029 keys/query: return all users which were asked for
In the situation where all of a user's devices get deleted, we want to
indicate this to a client, so we want to return an empty dictionary, rather
than nothing at all.
2016-08-03 15:41:44 +01:00
Erik Johnston
4c56bedee3 Actually call get_room_state 2016-08-03 15:04:29 +01:00
Erik Johnston
520ee9bd2c Fix syntax error 2016-08-03 15:03:15 +01:00
Erik Johnston
a60a2eaa02 Comment 2016-08-03 14:52:43 +01:00
Erik Johnston
e3a720217a Add /state_ids federation API
The new API only returns the event_ids for the state, as most
requesters will already have the vast majority of the events already.
2016-08-03 14:47:37 +01:00
Richard van der Hoff
530bc862dc Merge branch 'rav/null_default_device_displayname' into develop 2016-08-03 14:30:32 +01:00
Richard van der Hoff
a6f5cc65d9 PEP8 2016-08-03 14:30:06 +01:00
Richard van der Hoff
e555bc6551 Merge branch 'rav/refactor_device_query' into develop 2016-08-03 14:26:01 +01:00
Richard van der Hoff
a843868fe9 E2eKeysHandler: minor tweaks
PR feedback
2016-08-03 14:24:33 +01:00
Erik Johnston
80ad710217 Remove other bit of deduplication 2016-08-03 13:25:59 +01:00
Richard van der Hoff
4fec5e57be Default device_display_name to null
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
Erik Johnston
a8a32d2714 Ensure we only persist an event once at a time 2016-08-03 11:23:39 +01:00
Mark Haines
921f17f938 Merge branch 'develop' into rav/refactor_device_query 2016-08-03 11:12:47 +01:00
Erik Johnston
58c9653c6b Don't infer paragrahs from newlines 2016-08-02 18:50:24 +01:00
Erik Johnston
6b58ade2f0 Comment on why we clone 2016-08-02 18:41:22 +01:00
Erik Johnston
9e66c58ceb Spelling. 2016-08-02 18:37:31 +01:00
Erik Johnston
f83f5fbce8 Make it actually compile 2016-08-02 18:32:42 +01:00
Erik Johnston
aecaec3e10 Change the way we summarize URLs
Using XPath is slow on some machines (for unknown reasons), so use a
different approach to get a list of text nodes.

Try to generate a summary that respect paragraph and then word
boundaries, adding ellipses when appropriate.
2016-08-02 18:25:53 +01:00
Richard van der Hoff
1efee2f52b E2E keys: Make federation query share code with client query
Refactor the e2e query handler to separate out the local query, and then make
the federation handler use it.
2016-08-02 18:12:00 +01:00
Erik Johnston
49e047c55e Bump version and changelog 2016-08-02 17:10:26 +01:00
Erik Johnston
59a2c6d60e Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0 2016-08-02 17:07:54 +01:00
Erik Johnston
c9154b970c Don't double wrap 200 2016-08-02 16:45:53 +01:00
Erik Johnston
b3d5c4ad9d Fix response cache 2016-08-02 16:42:21 +01:00
Erik Johnston
8f650bd338 Bump changeog and version 2016-08-02 15:43:52 +01:00
Erik Johnston
7b0f6293f2 Merge pull request #940 from matrix-org/erikj/fed_state_cache
Cache federation state responses
2016-08-02 15:21:37 +01:00
Erik Johnston
342e072024 Merge pull request #967 from matrix-org/erikj/fed_reader
Split out the federation reading portions into a separate.
2016-08-02 13:59:47 +01:00
Mark Haines
54de6a812a Merge branch 'develop' into dbkr/fix_add_email_on_register 2016-08-02 10:18:15 +01:00
Erik Johnston
733bf44290 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader 2016-08-02 09:56:31 +01:00
Richard van der Hoff
986615b0b2 Move e2e query logic into a handler 2016-08-01 18:02:07 +01:00
Erik Johnston
b260f92936 Ignore AlreadyCalled errors on timer cancel 2016-07-31 16:00:12 +01:00
David Baker
271d3e7865 Fix adding emails on registration
Synapse was not adding email addresses to accounts registered with an email address, due to too many different variables called 'result'. Rename both of them. Also remove the defer.returnValue() with no params because that's not a thing.
2016-07-29 15:25:24 +01:00
Paul Evans
18b7eb830b Merge pull request #958 from matrix-org/paul/SYN-738
Forbid non-ASes from registering users whose names begin with '_'
2016-07-29 14:10:45 +01:00
Erik Johnston
2a9ce8c422 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader 2016-07-29 11:33:46 +01:00
Erik Johnston
cbea0c7044 Merge pull request #964 from matrix-org/erikj/fed_join_fix
Handle the case of missing auth events when joining a room
2016-07-29 11:33:16 +01:00
Erik Johnston
5aa024e501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader 2016-07-29 11:24:56 +01:00
Erik Johnston
c51a52f300 Mention that func will fetch auth events 2016-07-29 11:17:04 +01:00
Erik Johnston
3d13c3a295 Update docstring 2016-07-29 10:45:05 +01:00
Mark Haines
8dad08a950 Fix SQL to supply arguments in the same order 2016-07-29 09:57:13 +01:00
Mark Haines
0a7d3cd00f Create separate methods for getting messages to push
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.

The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.

The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
2016-07-28 20:24:24 +01:00
Erik Johnston
ec8b217722 Add destination retry to slave store 2016-07-28 17:35:53 +01:00
Erik Johnston
76b89d0edb Add slace storage functions for public room list 2016-07-28 17:03:40 +01:00
Kegan Dougal
370135ad0b Comment get_unread_push_actions_for_user_in_range function 2016-07-28 16:47:37 +01:00
Erik Johnston
0fcbca531f Add get_auth_chain to slave store 2016-07-28 16:36:28 +01:00
Erik Johnston
1e2740caab Handle the case of missing auth events when joining a room 2016-07-28 16:08:33 +01:00
Erik Johnston
6ede23ff1b Add more key storage funcs into slave store 2016-07-28 15:41:26 +01:00
Erik Johnston
5fb41a955c Merge branch 'release-v0.17.0' of github.com:matrix-org/synapse into develop 2016-07-28 14:56:32 +01:00
Mark Haines
4329f20e3f Merge pull request #962 from matrix-org/markjh/retry
Fix retry utils to check if the exception is a subclass of CME
2016-07-28 11:12:19 +01:00
Erik Johnston
a285194021 Merge branch 'erikj/key_client_fix' of github.com:matrix-org/synapse into release-v0.17.0 2016-07-28 10:47:06 +01:00
Mark Haines
bf81e38d36 Fix retry utils to check if the exception is a subclass of CME 2016-07-28 10:47:02 +01:00
Erik Johnston
78cac3e594 Merge pull request #941 from matrix-org/erikj/key_client_fix
Send the correct host header when fetching keys
2016-07-28 10:46:36 +01:00
Erik Johnston
7871790db1 Bump version and changelog 2016-07-28 10:38:56 +01:00
Erik Johnston
18e044628e Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0 2016-07-28 10:37:43 +01:00
Erik Johnston
b557b682d9 Merge pull request #961 from matrix-org/dbkr/fix_push_invite_name
Don't include name of room for invites in push
2016-07-28 10:36:41 +01:00
David Baker
389c890f14 Don't include name of room for invites in push
Avoids insane pushes like, "Bob invited you to invite from Bob"
2016-07-28 10:20:47 +01:00
Richard van der Hoff
cd8738ab63 Merge pull request #960 from matrix-org/rav/support_r0.2
Add r0.2.0 to the "supported versions" list
2016-07-28 10:14:33 +01:00
Richard van der Hoff
f6f8f81a48 Add r0.1.0 to the "supported versions" list 2016-07-28 10:14:07 +01:00
David Baker
ecd5e6bfa4 Typo 2016-07-28 10:04:52 +01:00
Richard van der Hoff
fda078f995 Add r0.2.0 to the "supported versions" list 2016-07-28 09:14:21 +01:00
Richard van der Hoff
40e539683c Merge pull request #956 from matrix-org/rav/check_device_id_on_key_upload
Make the device id on e2e key upload optional
2016-07-27 18:16:57 +01:00
Paul "LeoNerd" Evans
05f6447301 Forbid non-ASes from registering users whose names begin with '_' (SYN-738) 2016-07-27 17:54:26 +01:00
Erik Johnston
5238960850 Bump CHANGES and version 2016-07-27 17:33:09 +01:00
Richard van der Hoff
ccec25e2c6 key upload tweaks
1. Add v2_alpha URL back in, since things seem to be using it.

2. Don't reject the request if the device_id in the upload request fails to
   match that in the access_token.
2016-07-27 16:41:06 +01:00
Mark Haines
29b25d59c6 Merge branch 'develop' into markjh/verify
Conflicts:
	synapse/crypto/keyring.py
2016-07-27 15:11:02 +01:00
Mark Haines
884b800899 Merge pull request #955 from matrix-org/markjh/only_from2
Add a couple more checks to the keyring
2016-07-27 15:08:22 +01:00
Mark Haines
09d31815b4 Merge pull request #954 from matrix-org/markjh/even_more_fixes
Fix a couple of bugs in the transaction and keyring code
2016-07-27 15:08:14 +01:00
Mark Haines
fe1b369946 Clean up verify_json_objects_for_server 2016-07-27 14:10:43 +01:00
Richard van der Hoff
26cb0efa88 SQL syntax fix 2016-07-27 12:30:22 +01:00
Richard van der Hoff
d47115ff8b Delete e2e keys on device delete 2016-07-27 12:24:52 +01:00
Richard van der Hoff
2e3d90d67c Make the device id on e2e key upload optional
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.

For active access_tokens without an associated device_id, we ought to register
the device in the devices table.

Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
Mark Haines
a4b06b619c Add a couple more checks to the keyring 2016-07-26 19:50:11 +01:00
Mark Haines
c63b1697f4 Merge pull request #952 from matrix-org/markjh/more_fixes
Check if the user is banned when handling 3pid invites
2016-07-26 19:20:56 +01:00
Mark Haines
87ffd21b29 Fix a couple of bugs in the transaction and keyring code 2016-07-26 19:19:08 +01:00
Richard van der Hoff
eb359eced4 Add create_requester function
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
Mark Haines
c824b29e77 Check if the user is banned when handling 3pid invites 2016-07-26 16:39:14 +01:00
Richard van der Hoff
33d7776473 Fix typo 2016-07-26 13:32:15 +01:00
Richard van der Hoff
9ad8d9b17c Merge branch 'develop' into rav/delete_refreshtoken_on_delete_device 2016-07-26 13:29:46 +01:00
Richard van der Hoff
5b1825ba5b Merge pull request #951 from matrix-org/rav/flake8
Fix flake8 noise
2016-07-26 13:27:54 +01:00
Mark Haines
9c4cf83259 Merge pull request #948 from matrix-org/markjh/auth_fixes
Don't add rejections to the state_group, persist all rejections
2016-07-26 13:22:57 +01:00
Richard van der Hoff
05e7e5e972 Fix flake8 violation
Apparently flake8 v3 puts the error on a different line to v2. Easiest way to
make sure that happens is by putting the whole statement on one line :)
2016-07-26 11:59:08 +01:00
Richard van der Hoff
8e02494166 Delete refresh tokens when deleting devices 2016-07-26 11:10:37 +01:00
Mark Haines
a6f06ce3e2 Fix how push_actions are redacted. 2016-07-26 11:05:39 +01:00
David Baker
d34e9f93b7 Merge pull request #949 from matrix-org/rav/update_devices
Implement updates and deletes for devices
2016-07-26 10:49:55 +01:00
Mark Haines
efeb6176c1 Don't add rejected events if we've seen them befrore. Add some comments to explain what the code is doing mechanically 2016-07-26 10:49:52 +01:00
Matthew Hodgson
242c52d607 typo 2016-07-26 10:09:25 +02:00
Richard van der Hoff
012b4c1913 Implement updating devices
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
Richard van der Hoff
436bffd15f Implement deleting devices 2016-07-26 07:35:48 +01:00
Mark Haines
1b3c3e6d68 Only update the events and event_json tables for rejected events 2016-07-25 18:44:30 +01:00
Richard van der Hoff
33d08e8433 Log when adding listeners 2016-07-25 17:22:15 +01:00
Mark Haines
8f7f4cb92b Don't add the events to forward extremities if the event is rejected 2016-07-25 17:13:37 +01:00
Mark Haines
2623cec874 Don't add rejections to the state_group, persist all rejections 2016-07-25 16:12:16 +01:00
David Baker
4fcdf7b4b2 Merge pull request #946 from matrix-org/dbkr/log_recaptcha_hostname
Log the hostname the reCAPTCHA was completed on
2016-07-25 16:10:39 +01:00
Mark Haines
955ef1f06c fix: defer.returnValue takes one argument 2016-07-25 16:04:45 +01:00
Richard van der Hoff
2ee4c9ee02 background updates: fix assert again 2016-07-25 16:01:46 +01:00
Richard van der Hoff
9dbd903f41 background updates: Fix assertion to do something 2016-07-25 14:05:23 +01:00
Richard van der Hoff
465117d7ca Fix background_update tests
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
David Baker
7ed58bb347 Use get to avoid KeyErrors 2016-07-22 17:18:50 +01:00
David Baker
dad2da7e54 Log the hostname the reCAPTCHA was completed on
This could be useful information to have in the logs. Also comment about how & why we don't verify the hostname.
2016-07-22 17:00:56 +01:00
Richard van der Hoff
363786845b PEP8 2016-07-22 13:21:07 +01:00
Richard van der Hoff
ec5717caf5 Create index on user_ips in the background
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.

I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
Erik Johnston
d26b660aa6 Cache getPeer 2016-07-21 17:38:51 +01:00
Erik Johnston
aede7248ab Split out a FederationReader process 2016-07-21 17:37:44 +01:00
Richard van der Hoff
55abbe1850 make /devices return a list
Turns out I specced this to return a list of devices rather than a dict of them
2016-07-21 15:57:28 +01:00
David Baker
2c28e25bda Merge pull request #943 from matrix-org/rav/get_device_api
Implement GET /device/{deviceId}
2016-07-21 13:41:42 +01:00
David Baker
1e6e370b76 Merge pull request #942 from matrix-org/rav/fix_register_deviceid
Preserve device_id from first call to /register
2016-07-21 13:16:31 +01:00
Richard van der Hoff
1c3c202b96 Fix PEP8 errors 2016-07-21 13:15:15 +01:00
Richard van der Hoff
406f7aa0f6 Implement GET /device/{deviceId} 2016-07-21 12:00:29 +01:00
Richard van der Hoff
c445f5fec7 storage/client_ips: remove some dead code 2016-07-21 11:58:47 +01:00
Erik Johnston
cf94a78872 Set host not path 2016-07-21 11:45:53 +01:00
Richard van der Hoff
1a64dffb00 Preserve device_id from first call to /register
device_id may only be passed in the first call to /register, so make sure we
fish it out of the register `params` rather than the body of the final call.
2016-07-21 11:34:16 +01:00
Erik Johnston
081e5d55e6 Send the correct host header when fetching keys 2016-07-21 11:14:54 +01:00
Erik Johnston
248e6770ca Cache federation state responses 2016-07-21 10:30:12 +01:00
Richard van der Hoff
7314bf4682 Merge branch 'develop' into rav/get_devices_api
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff
e9e3eaa67d Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips
Record device_id in client_ips
2016-07-20 17:38:45 +01:00
Erik Johnston
d36b1d849d Don't explode if we have no snapshots yet 2016-07-20 16:59:52 +01:00
David Baker
742056be0d Merge pull request #937 from matrix-org/rav/register_device_on_register
Register a device_id in the /v2/register flow.
2016-07-20 16:51:27 +01:00
Richard van der Hoff
bc8f265f0a GET /devices endpoint
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff
ec041b335e Record device_id in client_ips
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
Richard van der Hoff
053e83dafb More doc-comments
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
Richard van der Hoff
b97a1356b1 Register a device_id in the /v2/register flow.
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.

Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
Erik Johnston
b73dc0ef4d Merge pull request #936 from matrix-org/erikj/log_rss
Add metrics for psutil derived memory usage
2016-07-20 16:32:38 +01:00
Erik Johnston
66868119dc Add metrics for psutil derived memory usage 2016-07-20 16:00:21 +01:00
Erik Johnston
aba0b2a39b Merge pull request #935 from matrix-org/erikj/backfill_notifs
Don't notify pusher pool for backfilled events
2016-07-20 13:39:16 +01:00
Erik Johnston
57dca35692 Don't notify pusher pool for backfilled events 2016-07-20 13:25:06 +01:00
Richard van der Hoff
c68518dfbb Merge pull request #933 from matrix-org/rav/type_annotations
Type annotations
2016-07-20 12:26:32 +01:00
David Baker
e967bc86e7 Merge pull request #932 from matrix-org/rav/register_refactor
Further registration refactoring
2016-07-20 11:03:33 +01:00
Erik Johnston
1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Erik Johnston
f91faf09b3 Comment 2016-07-20 10:18:09 +01:00
Richard van der Hoff
3413f1e284 Type annotations
Add some type annotations to help PyCharm (in particular) to figure out the
types of a bunch of things.
2016-07-19 18:56:16 +01:00
Richard van der Hoff
40cbffb2d2 Further registration refactoring
* `RegistrationHandler.appservice_register` no longer issues an access token:
  instead it is left for the caller to do it. (There are two of these, one in
  `synapse/rest/client/v1/register.py`, which now simply calls
  `AuthHandler.issue_access_token`, and the other in
  `synapse/rest/client/v2_alpha/register.py`, which is covered below).

* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
  access_tokens into `_create_registration_details`. This means that the normal
  flow no longer needs to call `AuthHandler.issue_access_token`; the
  shared-secret flow can tell `RegistrationHandler.register` not to generate a
  token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
David Baker
b9e997f561 Merge pull request #931 from matrix-org/rav/refactor_register
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
Richard van der Hoff
9a7a77a22a Merge pull request #929 from matrix-org/rav/support_deviceid_in_login
Add device_id support to /login
2016-07-19 15:53:04 +01:00
Richard van der Hoff
8f6281ab0c Don't bind email unless threepid contains expected fields 2016-07-19 15:50:01 +01:00
Richard van der Hoff
0da0d0a29d rest/client/v2_alpha/register.py: Refactor flow somewhat.
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.

The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
Richard van der Hoff
022b9176fe schema fix
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
Mark Haines
c41d52a042 Summary line 2016-07-19 10:28:27 +01:00
Mark Haines
7e554aac86 Update docstring on Handlers.
To indicate it is deprecated.
2016-07-19 10:20:58 +01:00
Richard van der Hoff
f863a52cea Add device_id support to /login
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Richard van der Hoff
93efcb8526 Merge pull request #928 from matrix-org/rav/refactor_login
Refactor login flow
2016-07-18 16:12:35 +01:00
Richard van der Hoff
dcfd71aa4c Refactor login flow
Make sure that we have the canonical user_id *before* calling
get_login_tuple_for_user_id.

Replace login_with_password with a method which just validates the password,
and have the caller call get_login_tuple_for_user_id. This brings the password
flow into line with the other flows, and will give us a place to register the
device_id if necessary.
2016-07-18 15:23:54 +01:00
Erik Johnston
fca90b3445 Merge pull request #924 from matrix-org/erikj/purge_history
Fix /purge_history bug
2016-07-18 15:11:11 +01:00
Mark Haines
a292454aa1 Merge pull request #925 from matrix-org/markjh/auth_fix
Fix 500 ISE when sending alias event without a state_key
2016-07-18 15:04:47 +01:00
Erik Johnston
4f81edbd4f Merge pull request #927 from Half-Shot/develop
Fall back to 'username' if 'user' is not given for appservice registration.
2016-07-18 10:44:56 +01:00
Richard van der Hoff
6344db659f Fix a doc-comment
The `store` in a handler is a generic DataStore, not just an events.StateStore.
2016-07-18 09:48:10 +01:00
Will Hunt
511a52afc8 Use body.get to check for 'user' 2016-07-16 18:44:08 +01:00
Will Hunt
e885e2a623 Fall back to 'username' if 'user' is not given for appservice reg. 2016-07-16 18:33:48 +01:00
Mark Haines
d137e03231 Fix 500 ISE when sending alias event without a state_key 2016-07-15 18:58:25 +01:00
Erik Johnston
f52565de50 Fix /purge_history bug
This was caused by trying to insert duplicate backward extremeties
2016-07-15 14:23:15 +01:00
Erik Johnston
a2d288c6a9 Merge pull request #923 from matrix-org/erikj/purge_history
Various purge_history fixes
2016-07-15 13:23:29 +01:00
Erik Johnston
bd7c51921d Merge pull request #919 from matrix-org/erikj/auth_fix
Various auth.py fixes.
2016-07-15 11:38:33 +01:00
Erik Johnston
978fa53cc2 Pull out min stream_ordering from ex_outlier_stream 2016-07-15 10:22:30 +01:00
Erik Johnston
eec9609e96 event_backwards_extremeties may not be empty 2016-07-15 10:22:09 +01:00
Erik Johnston
9e1b43bcbf Comment 2016-07-15 09:29:54 +01:00
Erik Johnston
a3036ac37e Merge pull request #921 from matrix-org/erikj/account_deactivate
Feature: Add an /account/deactivate endpoint
2016-07-14 17:25:15 +01:00
Erik Johnston
ebdafd8114 Check sender signed event 2016-07-14 17:03:24 +01:00
Erik Johnston
a98d215204 Add filter param to /messages API 2016-07-14 16:30:56 +01:00
Erik Johnston
d554ca5e1d Add support for filters in paginate_room_events 2016-07-14 15:59:04 +01:00
Erik Johnston
209e04fa11 Merge pull request #918 from negzi/bugfix_for_token_expiry
Bug fix: expire invalid access tokens
2016-07-14 15:51:52 +01:00
Erik Johnston
e5142f65a6 Add 'contains_url' to filter 2016-07-14 15:35:48 +01:00
Erik Johnston
b64aa6d687 Add sender and contains_url field to events table 2016-07-14 15:35:43 +01:00
Erik Johnston
848d3bf2e1 Add hs object 2016-07-14 10:25:52 +01:00
Erik Johnston
b55c770271 Only accept password auth 2016-07-14 10:00:38 +01:00
Erik Johnston
d543b72562 Add an /account/deactivate endpoint 2016-07-14 09:56:53 +01:00
Negar Fazeli
0136a522b1 Bug fix: expire invalid access tokens 2016-07-13 15:00:37 +02:00
Erik Johnston
2cb758ac75 Check if alias event's state_key matches sender's domain 2016-07-13 13:12:25 +01:00
Erik Johnston
560c71c735 Check creation event's room_id domain matches sender's 2016-07-13 13:07:19 +01:00
David Baker
a37ee2293c Merge pull request #915 from matrix-org/dbkr/more_requesttokens
Add requestToken endpoints
2016-07-13 11:51:46 +01:00
David Baker
c55ad2e375 be more pythonic 2016-07-12 14:15:10 +01:00
David Baker
aaa9d9f0e1 on_OPTIONS isn't neccessary 2016-07-12 14:13:14 +01:00
David Baker
75fa7f6b3c Remove other debug logging 2016-07-12 14:08:57 +01:00
David Baker
a5db0026ed Separate out requestTokens to separate handlers 2016-07-11 09:57:07 +01:00
David Baker
9c491366c5 Oops, remove debug logging 2016-07-11 09:07:40 +01:00
David Baker
385aec4010 Implement https://github.com/matrix-org/matrix-doc/pull/346/files 2016-07-08 17:42:48 +01:00
Mark Haines
dfde67a6fe Add a comment explaining allow_none 2016-07-08 15:57:06 +01:00
Mark Haines
10c843fcfb Ensure that the guest user is in the database when upgrading accounts 2016-07-08 15:15:55 +01:00
Erik Johnston
58930da52b Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-07-08 14:11:37 +01:00
Erik Johnston
0870588c20 Merge branch 'hotfixes-v0.16.1' 2016-07-08 13:22:32 +01:00
Erik Johnston
f90cf150e2 Bump version and changelog 2016-07-07 16:33:00 +01:00
Erik Johnston
067596d341 Fix bug where we did not correctly explode when multiple user_ids were set in macaroon 2016-07-07 16:22:24 +01:00
Erik Johnston
b92e7955be Comment 2016-07-07 11:42:15 +01:00
Erik Johnston
c98e1479bd Return 400 rather than 500 2016-07-07 11:41:07 +01:00
Erik Johnston
67f2c901ea Add rest servlet. Fix SQL. 2016-07-06 15:56:59 +01:00
Erik Johnston
eef7778af9 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/test2 2016-07-06 14:50:22 +01:00
Erik Johnston
a17e7caeb7 Merge branch 'erikj/shared_secret' into erikj/test2 2016-07-06 14:46:31 +01:00
Erik Johnston
f0c06ac65c Merge pull request #909 from matrix-org/erikj/shared_secret
Add an admin option to shared secret registration (breaks backwards compat)
2016-07-06 14:08:51 +01:00
Erik Johnston
76b18df3d9 Check that there are no null bytes in user and passsword 2016-07-06 11:17:53 +01:00
Erik Johnston
0da24cac8b Add null separator to hmac 2016-07-06 11:05:16 +01:00
Kent Shikama
8d9a884cee
Update password config comment
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-06 12:18:19 +09:00
Erik Johnston
be3548f7e1 Remove spurious txn 2016-07-05 17:46:51 +01:00
Erik Johnston
4adf93e0f7 Fix for postgres 2016-07-05 17:34:25 +01:00
Erik Johnston
651faee698 Add an admin option to shared secret registration 2016-07-05 17:30:22 +01:00
Erik Johnston
caf33b2d9b Protect password when registering using shared secret 2016-07-05 17:18:19 +01:00
Erik Johnston
8f8798bc0d Add ReadWriteLock for pagination and history prune 2016-07-05 15:30:25 +01:00
Erik Johnston
7335f0adda Add ReadWriteLock 2016-07-05 15:23:17 +01:00
David Baker
ef535178ff Merge pull request #904 from matrix-org/dbkr/register_email_no_untrusted_id_server
requestToken update
2016-07-05 15:13:34 +01:00
Mark Haines
04dee11e97 Merge pull request #906 from matrix-org/markjh/faster_events_around
Use a query that postgresql optimises better for get_events_around
2016-07-05 14:48:34 +01:00
Mark Haines
dd2ccee27d Fix typo 2016-07-05 14:06:07 +01:00
Mark Haines
b6b0132ac7 Make get_events_around more efficient on sqlite3 2016-07-05 13:55:18 +01:00
Kent Shikama
252ee2d979
Remove default password pepper string 2016-07-05 19:15:51 +09:00
Kent Shikama
14362bf359
Fix password config 2016-07-05 19:12:53 +09:00
Kent Shikama
1ee2584307
Fix pep8 2016-07-05 19:01:00 +09:00
Kent Shikama
507b8bb091
Add comment to prompt changing of pepper 2016-07-05 18:42:35 +09:00
Mark Haines
d44d11d864 Use true/false for boolean parameter inclusive to avoid potential for sqli, and possibly make the code clearer 2016-07-05 10:39:13 +01:00
Erik Johnston
2d21d43c34 Add purge_history API 2016-07-05 10:28:51 +01:00
Mark Haines
0fb76c71ac Use different SQL for postgres and sqlite3 for when using multicolumn indexes 2016-07-04 19:44:55 +01:00
Kent Shikama
8bdaf5f7af
Add pepper to password hashing
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-05 02:13:52 +09:00
Erik Johnston
a67bf0b074 Add storage function to purge history for a room 2016-07-04 16:02:50 +01:00
Mark Haines
f18d7546c6 Use a query that postgresql optimises better for get_events_around 2016-07-04 15:48:25 +01:00
Kent Shikama
bb069079bb
Fix style violations
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-04 22:07:11 +09:00
Kent Shikama
2e5a31f197 Use .get() instead of [] to access password_hash 2016-07-04 22:00:13 +09:00
Kent Shikama
fc8007dbec
Optionally include password hash in createUser endpoint
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-03 15:08:15 +09:00
David Baker
be8be535f7 requestToken update
Don't send requestToken request to untrusted ID servers

Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.
2016-06-30 17:51:28 +01:00
Erik Johnston
f328d95cef Feature: Add deactivate account admin API
Allows server admins to "deactivate" accounts, which:

- Revokes all access tokens
- Removes all threepids
- Removes password

The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Erik Johnston
f52cb4cd78 Remove race 2016-06-29 15:24:50 +01:00
Mark Haines
6783534a0f Merge pull request #886 from matrix-org/markjh/async_commit
Optionally make committing to postgres asynchronous.
2016-06-29 15:21:58 +01:00
Erik Johnston
a70688445d Implement purge_media_cache admin API 2016-06-29 14:57:59 +01:00
Erik Johnston
314b146b2e Track approximate last access time for remote media 2016-06-29 11:41:20 +01:00
David Baker
db74dcda5b Merge pull request #894 from matrix-org/dbkr/push_room_naming
Use similar naming we use in email notifs for push
2016-06-28 10:12:24 +01:00
Matthew Hodgson
63bb8f0df9 remove vector.im from default secondary DS list 2016-06-27 13:13:33 +04:00
David Baker
0b640aa56b even more pep8 2016-06-24 11:47:11 +01:00
David Baker
aa3a4944d5 more pep8 2016-06-24 11:45:23 +01:00
David Baker
46b7362304 pep8 2016-06-24 11:44:57 +01:00
David Baker
870c45913e Use similar naming we use in email notifs for push
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
Mark Haines
05f1a4596a Merge branch 'master' into develop 2016-06-23 11:17:48 +01:00
David Baker
b5fb7458d5 Actually we need to order these properly
otherwise we'll end up returning the wrong 20
2016-06-22 18:07:14 +01:00
David Baker
f73fdb04a6 Style 2016-06-22 17:51:40 +01:00
David Baker
3a4120e49a Put most recent 20 messages in notif
Fixes https://github.com/vector-im/vector-web/issues/1648
2016-06-22 17:47:18 +01:00
Martin Weinelt
0a32208e5d Rework ldap integration with ldap3
Use the pure-python ldap3 library, which eliminates the need for a
system dependency.

Offer both a `search` and `simple_bind` mode, for more sophisticated
ldap scenarios.
- `search` tries to find a matching DN within the `user_base` while
  employing the `user_filter`, then tries the bind when a single
  matching DN was found.
- `simple_bind` tries the bind against a specific DN by combining the
  localpart and `user_base`

Offer support for STARTTLS on a plain connection.

The configuration was changed to reflect these new possibilities.

Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
2016-06-22 17:51:59 +02:00
Mark Haines
774f3a692c Merge pull request #889 from matrix-org/markjh/synctl_workers
Optionally start or stop workers in synctl.
2016-06-21 17:58:19 +01:00
Mark Haines
5cc7564c5c Optionally start or stop workers in synctl.
Optionally start or stop an individual worker by passing -w with
the path to the worker config.
Optionally start or stop every worker and the main synapse by
passing -a with a path to a directory containing worker configs.

The "-w" is intended to be used to bounce individual workers proceses.
THe "-a" is intended for when you want to restart all the workers
simultaneuously, for example when performing database upgrades.
2016-06-21 16:38:05 +01:00
Mark Haines
0fe0b0eeb6 Merge pull request #888 from matrix-org/markjh/content_repo
Remove the legacy v0 content upload API.
2016-06-21 14:01:01 +01:00
Mark Haines
13e334506c Remove the legacy v0 content upload API.
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
2016-06-21 11:47:39 +01:00
Mark Haines
d5fb561709 Optionally make committing to postgres asynchronous.
Useful when running tests when you don't care whether the server
will lose data that it claims that it has committed.
2016-06-20 17:53:38 +01:00
Erik Johnston
d8ec81cc31 Merge pull request #879 from matrix-org/erikj/linearize_fed_server
Linearize some federation endpoints based on (origin, room_id)
2016-06-20 17:34:29 +01:00
Erik Johnston
bc72d381b2 Merge branch 'release-v0.16.1' of github.com:matrix-org/synapse 2016-06-20 14:18:04 +01:00
Erik Johnston
4d362a61ea Bump version and changelog 2016-06-20 14:17:42 +01:00
Mark Haines
41e4b2efea Add the create_profile method back since the tests use it 2016-06-17 19:20:47 +01:00
Mark Haines
0c13d45522 Add a comment on why we don't create a profile for upgrading users 2016-06-17 19:18:53 +01:00
Mark Haines
9f1800fba8 Remove registered_users from the distributor.
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.

This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
2016-06-17 19:14:16 +01:00
Erik Johnston
8f4a9bbc16 Linearize some federation endpoints based on (origin, room_id) 2016-06-17 16:43:45 +01:00
Erik Johnston
9ba2bf1570 Merge pull request #878 from matrix-org/erikj/ujson
Disable responding with canonical json for federation
2016-06-17 16:22:12 +01:00
Erik Johnston
120c238705 Disable responding with canonical json for federation 2016-06-17 16:10:37 +01:00
Erik Johnston
1c1f633b13 Merge pull request #877 from matrix-org/erikj/frozen_default
Turn use_frozen_events off by default
2016-06-17 15:33:55 +01:00
Erik Johnston
6660f37558 Merge pull request #876 from matrix-org/erikj/sign_own
Only re-sign our own events
2016-06-17 15:23:20 +01:00
Erik Johnston
3e41de05cc Turn use_frozen_events off by default 2016-06-17 15:11:22 +01:00
Erik Johnston
2884712ca7 Only re-sign our own events 2016-06-17 14:47:33 +01:00
Mark Haines
ded01c3bf6 Fix `KeyError: 'msgtype'. Use .get`
Fixes a key error where the mailer tried to get the ``msgtype`` of an
event that was missing a ``msgtype``.

```
 File "synapse/push/mailer.py", line 264, in get_notif_vars
 File "synapse/push/mailer.py", line 285, in get_message_vars
 File ".../frozendict/__init__.py", line 10, in __getitem__
    return self.__dict[key]
    KeyError: 'msgtype'
```
2016-06-17 13:49:16 +01:00
Mark Haines
8c75040c25 Fix setting gc thresholds in the workers 2016-06-17 11:48:12 +01:00
Mark Haines
a352b68acf Use worker_ prefixes for worker config, use existing support for multiple config files 2016-06-16 17:29:50 +01:00
Mark Haines
364d616792 Access the event_cache_size directly from the server object.
This means that the workers can override the event_cache_size
directly without clobbering the value in the main synapse config.
2016-06-16 12:53:15 +01:00
Mark Haines
bde13833cb Access replication_url from the worker config directly 2016-06-16 12:44:40 +01:00
Mark Haines
80a1bc7db5 Comment on what's going on in clobber_with_worker_config 2016-06-16 11:29:45 +01:00
Mark Haines
f1f70bf4b5 Merge remote-tracking branch 'origin/develop' into markjh/worker_config 2016-06-16 11:20:17 +01:00
Mark Haines
dbb5a39b64 Add worker config module 2016-06-16 11:09:15 +01:00
Mark Haines
885ee861f7 Inline the synchrotron and pusher configs into the main config 2016-06-16 11:06:12 +01:00
Erik Johnston
486b9a6a2d Merge pull request #873 from vt0r/bugfix/bcrypt-utf8-encode
Fix TypeError in call to bcrypt.hashpw
2016-06-16 10:12:17 +01:00
Salvatore LaMendola
ed5f43a55a
Fix TypeError in call to bcrypt.hashpw
- At the very least, this TypeError caused logins to fail on my own
  running instance of Synapse, and the simple (explicit) UTF-8
  conversion resolved login errors for me.

Signed-off-by: Salvatore LaMendola <salvatore.lamendola@gmail.com>
2016-06-16 00:43:42 -04:00
Erik Johnston
09a17f965c Line lengths 2016-06-15 16:58:12 +01:00
Erik Johnston
1e9026e484 Handle floats as img widths 2016-06-15 16:58:05 +01:00
Erik Johnston
a60169ea09 Handle og props with not content 2016-06-15 16:57:48 +01:00
Erik Johnston
0ef0655b83 Bump version and changelog 2016-06-15 15:50:48 +01:00
Erik Johnston
d41a1a91d3 Linearize fetching of gaps on incoming events
This potentially stops the server from doing multiple requests for the
same data.
2016-06-15 15:16:14 +01:00
Richard van der Hoff
15bf3e3376 Merge pull request #870 from matrix-org/rav/work_around_tls_bug
Work around TLS bug in twisted
2016-06-15 13:28:13 +01:00
Erik Johnston
b31c49d676 Correctly mark backfilled events as backfilled 2016-06-15 10:59:08 +01:00
Richard van der Hoff
255c229f23 Work around TLS bug in twisted
Wrap up twisted's FileBodyProducer to work around
https://twistedmatrix.com/trac/ticket/8473. Hopefully this fixes
https://matrix.org/jira/browse/SYN-700.
2016-06-15 10:39:08 +01:00
Erik Johnston
d12134ce37 Merge pull request #868 from matrix-org/erikj/invalid_id
Make get_domain_from_id throw SynapseError on invalid ID
2016-06-14 15:09:07 +01:00
Erik Johnston
36e2aade87 Make get_domain_from_id throw SynapseError on invalid ID 2016-06-14 13:25:29 +01:00
Matthew Hodgson
33546b58aa point to the CAPTCHA docs 2016-06-12 23:11:29 +01:00
Mark Haines
41b35412bf Merge pull request #863 from matrix-org/markjh/load_config
Add function to load config without generating it
2016-06-10 11:12:32 +01:00
Mark Haines
7dbb473339 Add function to load config without generating it
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.

The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.

As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
2016-06-09 18:50:38 +01:00
Erik Johnston
16a8884233 Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-06-09 16:57:33 +01:00
Erik Johnston
a31befbcd0 Bump version and changelog 2016-06-09 13:23:41 +01:00
Erik Johnston
5ee5b655b2 Merge pull request #862 from matrix-org/erikj/media_remote_error
502 on /thumbnail when can't contact remote server
2016-06-09 13:11:02 +01:00
Erik Johnston
0a43219a27 Merge pull request #860 from negzi/bug_fix_get_or_create_user
Fix a bug caused by a change in auth_handler function
2016-06-09 11:40:12 +01:00
Erik Johnston
eba4ff1bcb 502 on /thumbnail when can't contact remote server 2016-06-09 11:29:43 +01:00
Erik Johnston
95f305c35a Remove redundant exception log in /events 2016-06-09 11:15:04 +01:00
Negar Fazeli
6e7dc7c7dd Fix a bug caused by a change in auth_handler function
Fix the relevant unit test cases
2016-06-08 23:22:39 +02:00
Erik Johnston
b7fbc9bd95 Merge pull request #859 from matrix-org/erikj/public_room_performance
Pull full state for each room all at once
2016-06-08 16:36:06 +01:00
Erik Johnston
919a2c74f6 Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop 2016-06-08 16:26:26 +01:00
Erik Johnston
81c07a32fd Pull full state for each room all at once 2016-06-08 15:51:49 +01:00
Erik Johnston
690029d1a3 Don't make rooms visibile by default 2016-06-08 14:47:42 +01:00
Erik Johnston
d88faf92d1 Fix up federation PublicRoomList 2016-06-08 14:39:31 +01:00
Erik Johnston
958c968d02 Merge pull request #856 from matrix-org/erikj/fed_pub_rooms
Enable auth on /publicRoom endpoints
2016-06-08 14:36:09 +01:00
Erik Johnston
746b2f5657 Merge pull request #854 from matrix-org/erikj/federation_logging
Add some logging for when servers ask for missing events
2016-06-08 14:31:41 +01:00
Erik Johnston
efeabd3180 Log user that is making /publicRooms calls 2016-06-08 14:23:15 +01:00
Erik Johnston
1fd6eb695d Enable auth on federation PublicRoomList 2016-06-08 14:15:18 +01:00
Erik Johnston
17aab5827a Add some logging for when servers ask for missing events 2016-06-08 11:55:31 +01:00
Erik Johnston
1a815fb04f Don't hit DB for noop replications queries 2016-06-08 11:33:30 +01:00
Erik Johnston
bab916bccc Bump version and changelog to v0.16.0-rc2 2016-06-08 11:05:45 +01:00
Erik Johnston
e0fda29f94 Merge pull request #850 from matrix-org/erikj/gc_threshold
Add gc_threshold to pusher and synchrotron
2016-06-08 10:05:56 +01:00
Erik Johnston
0f2165ccf4 Don't track total objects as its too expensive to calculate 2016-06-07 17:00:45 +01:00
Erik Johnston
18f0cc7d99 Record some more GC metrics 2016-06-07 16:55:49 +01:00
Erik Johnston
2d1d1025fa Add gc_threshold to pusher and synchrotron 2016-06-07 16:26:25 +01:00
Erik Johnston
b01e71e719 Merge pull request #849 from matrix-org/erikj/gc_threshold
Allow setting of gc.set_thresholds
2016-06-07 16:07:54 +01:00
Erik Johnston
dded389ac1 Allow setting of gc.set_thresholds 2016-06-07 15:45:56 +01:00
Mark Haines
0b2158719c Remove dead code.
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.

The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.

The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston
48e65099b5 Also record number of unreachable objects 2016-06-07 13:40:22 +01:00
Erik Johnston
75331c5fca Change the way we do stats 2016-06-07 13:33:13 +01:00
Erik Johnston
8c966fbd51 Merge pull request #771 from matrix-org/erikj/gc_tick
Manually run GC on reactor tick.
2016-06-07 13:18:36 +01:00
Mark Haines
88625db05f Notify users for events in rooms they join.
Change how the notifier updates the map from room_id to user streams on
receiving a join event. Make it update the map when it notifies for the
join event, rather than using the "user_joined_room" distributor signal
2016-06-07 11:33:36 +01:00
Erik Johnston
84379062f9 Fix AS retries, but with correct ordering 2016-06-07 10:24:50 +01:00
Erik Johnston
310197bab5 Fix AS retries 2016-06-07 09:34:50 +01:00
Mark Haines
b0932b34cb Merge pull request #845 from matrix-org/markjh/synchrotron_presence
Fix a KeyError in the synchrotron presence
2016-06-06 16:52:27 +01:00
Mark Haines
4a5bbb1941 Fix a KeyError in the synchrotron presence 2016-06-06 16:37:12 +01:00
Mark Haines
87f60e7053 Merge pull request #844 from matrix-org/markjh/yield_on_sleep
Yield on the sleeps intended to backoff replication
2016-06-06 16:20:54 +01:00
Mark Haines
5ef84da4f1 Yield on the sleeps intended to backoff replication 2016-06-06 16:05:28 +01:00
Erik Johnston
216a05b3e3 .values() returns list of sets 2016-06-06 16:00:09 +01:00
Erik Johnston
96dc600579 Fix typos 2016-06-06 15:44:41 +01:00
Erik Johnston
377eb480ca Fire after 30s not 8h 2016-06-06 15:14:21 +01:00
Erik Johnston
e4134c5e13 Merge pull request #841 from matrix-org/erikj/event_counter
Add metric counter for number of persisted events
2016-06-06 14:17:40 +01:00
Erik Johnston
7aa778fba9 Add metric counter for number of persisted events 2016-06-06 11:58:09 +01:00
Erik Johnston
70aee0717c Add events to cache when we persist them 2016-06-06 11:34:53 +01:00
Erik Johnston
3210f4c385 Merge pull request #836 from matrix-org/erikj/change_event_cache
Change the way we cache events
2016-06-03 18:47:46 +01:00
Erik Johnston
cffe46408f Don't rely on options when inserting event into cache 2016-06-03 18:25:21 +01:00
Mark Haines
ac9716f154 Fix spelling 2016-06-03 18:10:00 +01:00
Mark Haines
8f79084bd4 Add get_presence_list_accepted to the broken caches in synchrotron 2016-06-03 18:03:40 +01:00
Erik Johnston
10ea3f46ba Change the way we cache events 2016-06-03 17:57:50 +01:00
Erik Johnston
f6be734be9 Merge pull request #835 from matrix-org/erikj/get_event_txn
Remove event fetching from DB threads
2016-06-03 17:30:00 +01:00
Erik Johnston
05e01f21d7 Remove event fetching from DB threads 2016-06-03 17:22:13 +01:00
David Baker
72c4d482e9 3rd time lucky: we'd already calculated it above 2016-06-03 16:39:50 +01:00
David Baker
fbf608decb Oops, we're using the dict form 2016-06-03 16:38:39 +01:00
David Baker
06d40c8b98 Add substitutions to email notif From 2016-06-03 16:31:23 +01:00
Erik Johnston
5f88549f4a Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop 2016-06-03 16:27:24 +01:00
Mark Haines
ca457f594e Merge pull request #831 from matrix-org/markjh/synchrotronII
Add a separate process that can handle /sync requests
2016-06-03 16:06:25 +01:00
Erik Johnston
21961c93c7 Bump changelog and version 2016-06-03 15:33:14 +01:00
Mark Haines
48340e4f13 Clear the list of ongoing syncs on shutdown 2016-06-03 15:02:27 +01:00
Erik Johnston
fcbc282f56 Merge branch 'release-v0.15.0' of github.com:matrix-org/synapse into release-v0.16.0 2016-06-03 15:02:04 +01:00
Erik Johnston
51773bcbaf Merge pull request #832 from matrix-org/erikj/presence_coount
Change def of small delta in presence stream. Add metrics.
2016-06-03 14:57:00 +01:00
Mark Haines
da491e75b2 Appease flake8 2016-06-03 14:56:36 +01:00
Mark Haines
0b3c80a234 Use ClientIpStore to record client ips 2016-06-03 14:55:01 +01:00
Mark Haines
dd6f62ed99 Merge branch 'develop' into markjh/synchrotronII 2016-06-03 14:51:33 +01:00
Mark Haines
eef541a291 Move insert_client_ip to a separate class 2016-06-03 14:42:35 +01:00
Mark Haines
80aade3805 Send updates to the syncing users every ten seconds or immediately if they've just come online 2016-06-03 14:24:19 +01:00
Erik Johnston
ab116bdb0c Fix typo 2016-06-03 14:03:42 +01:00
Erik Johnston
4ce84a1acd Change metric style 2016-06-03 13:49:16 +01:00
Erik Johnston
a7ff5a1770 Presence metrics. Change def of small delta 2016-06-03 13:40:55 +01:00
Matthew Hodgson
8d740132f4 Merge branch 'develop' into matthew/brand-from-header 2016-06-03 12:14:18 +01:00
Erik Johnston
3b096c5f5c Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop 2016-06-03 12:00:33 +01:00
Erik Johnston
43b7f371f5 Merge pull request #830 from matrix-org/erikj/metrics_perf
Change CacheMetrics to be quicker
2016-06-03 11:57:29 +01:00
Mark Haines
abb151f3c9 Add a separate process that can handle /sync requests 2016-06-03 11:57:26 +01:00
Erik Johnston
4982b28868 Merge pull request #829 from matrix-org/erikj/poke_notifier
Poke notifier on next reactor tick
2016-06-03 11:52:10 +01:00
Erik Johnston
d06f2a229e Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:50:30 +01:00
Erik Johnston
58a224a651 Pull out update_results_dict 2016-06-03 11:47:07 +01:00
Mark Haines
20eccd84d4 Merge pull request #827 from matrix-org/markjh/more_slaved_methods
Add methods to events, account data and receipt slaves
2016-06-03 11:46:21 +01:00
Erik Johnston
722472b48c Merge pull request #825 from matrix-org/erikj/cache_push_rules
Load push rules in storage layer so that they get cached
2016-06-03 11:44:32 +01:00
Erik Johnston
73c7112433 Change CacheMetrics to be quicker
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
Mark Haines
b09f348530 Merge pull request #824 from matrix-org/markjh/slaved_presence_store
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
Erik Johnston
4c04222fa5 Poke notifier on next reactor tick 2016-06-03 11:24:16 +01:00
Erik Johnston
ccb56fc24b Make get_joined_hosts_for_room use get_users_in_room 2016-06-03 11:20:23 +01:00
Mark Haines
81cf449daa Add methods to events, account data and receipt slaves
Adds the methods needed by /sync to the slaved events,
account data and receipt stores.
2016-06-03 11:19:27 +01:00
Erik Johnston
e043ede4a2 Small optimisation to CacheListDescriptor 2016-06-03 11:19:22 +01:00
Mark Haines
b821c839dc Merge pull request #823 from matrix-org/markjh/more_slaved_stores
Add slaved stores for filters, tokens, and push rules
2016-06-03 11:17:43 +01:00
Erik Johnston
597013caa5 Make cachedList go a bit faster 2016-06-03 11:13:29 +01:00
Erik Johnston
6a0afa582a Load push rules in storage layer, so that they get cached 2016-06-03 11:10:00 +01:00
Mark Haines
3ae915b27e Add a slaved store for presence 2016-06-03 11:05:53 +01:00
Erik Johnston
59f2d73522 Remove unnecessary sets 2016-06-03 11:05:45 +01:00
Erik Johnston
9c26b390a2 Only get local users 2016-06-03 11:04:31 +01:00
Mark Haines
f88d747f79 Add a comment explaining why the filter cache doesn't need exipiring 2016-06-03 11:03:10 +01:00
Erik Johnston
065e739d6e Merge pull request #811 from matrix-org/erikj/state_users_in_room
Use state to calculate get_users_in_room
2016-06-03 10:58:27 +01:00
Erik Johnston
696d7c5937 Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room
Add get_users_with_read_receipts_in_room cache
2016-06-03 10:58:24 +01:00
Mark Haines
0eae075723 Add slaved stores for filters, tokens, and push rules 2016-06-03 10:58:03 +01:00
Matthew Hodgson
79d1f072f4 brand the email from header 2016-06-02 21:34:40 +01:00
David Baker
6bb9aacf9d Merge pull request #821 from matrix-org/dbkr/email_unsubscribe
Email unsubscribe links that don't require logging in
2016-06-02 17:44:55 +01:00
David Baker
745ddb4dd0 peppate 2016-06-02 17:38:41 +01:00
David Baker
7a5a5f2df2 Merge pull request #820 from matrix-org/dbkr/email_notif_string_fmt_error
Fix error in email notification string formatting
2016-06-02 17:26:06 +01:00
David Baker
1f31cc37f8 Working unsubscribe links going straight to the HS
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
Matthew Hodgson
2675c1e40e add some branding debugging 2016-06-02 17:21:12 +01:00
David Baker
c71177f285 Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe 2016-06-02 17:20:56 +01:00
David Baker
07a5559916 Fix error in email notification string formatting 2016-06-02 17:17:16 +01:00
Mark Haines
56d15a0530 Store the typing users as user_id strings. (#819)
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
David Baker
812b5de0fe Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe 2016-06-02 15:33:28 +01:00
Mark Haines
80f34d7b57 Fix setting the _clock in SQLBaseStore 2016-06-02 15:23:56 +01:00
Mark Haines
661a540dd1 Deduplicate presence entries in sync (#818) 2016-06-02 15:20:28 +01:00
Mark Haines
70599ce925 Allow external processes to mark a user as syncing. (#812)
* Add infrastructure to the presence handler to track sync requests in external processes

* Expire stale entries for dead external processes

* Add an http endpoint for making users as syncing

Add some docstrings and comments.

* Fixes
2016-06-02 15:20:15 +01:00
David Baker
fb2193cc63 Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler
Split out the auth handler
2016-06-02 14:31:35 +01:00
Erik Johnston
356f13c069 Disable INCLUDE_ALL_UNREAD_NOTIFS 2016-06-02 14:07:38 +01:00
Erik Johnston
02ac463dbf Merge pull request #800 from matrix-org/erikj/sync_refactor
Refactor SyncHandler
2016-06-02 14:02:13 +01:00
Matthew Hodgson
c5af1b6b00 Merge pull request #814 from matrix-org/matthew/3pid_invite_auth
special case m.room.third_party_invite event auth to match invites,
2016-06-02 13:47:40 +01:00
David Baker
3a3fb2f6f9 Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe 2016-06-02 13:35:25 +01:00
David Baker
4a10510cd5 Split out the auth handler 2016-06-02 13:31:45 +01:00
Matthew Hodgson
f84b89f0c6 if an email pusher specifies a brand param, use it 2016-06-02 13:29:48 +01:00
David Baker
a15ad60849 Email unsubscribing that may in theory, work
Were it not for that fact that you can't use the base handler in the pusher because it pulls in the world. Comitting while I fix that on a different branch.
2016-06-02 11:44:15 +01:00
David Baker
e793866398 Use user_id in email greeting if display name is null 2016-06-02 09:41:13 +01:00
Matthew Hodgson
aaa70e26a2 special case m.room.third_party_invite event auth to match invites, otherwise they get out of sync and you get https://github.com/vector-im/vector-web/issues/1208 2016-06-01 22:13:47 +01:00
Erik Johnston
a04a2d043c Merge pull request #807 from matrix-org/erikj/push_rules_cache
Ensure we always return boolean in push rules
2016-06-01 18:07:48 +01:00
Erik Johnston
0f06b496d1 Merge pull request #806 from matrix-org/erikj/hash_cache
Cache get_event_reference_hashes
2016-06-01 18:07:42 +01:00
David Baker
e0deeff23e Fix room list spidering 2016-06-01 17:58:58 +01:00
David Baker
991af8b0d6 WIP on unsubscribing email notifs without logging in 2016-06-01 17:40:52 +01:00
David Baker
00c487a8db Merge pull request #808 from matrix-org/dbkr/room_list_spider
Add secondary_directory_servers option to fetch room list from other servers
2016-06-01 15:32:52 +01:00
Erik Johnston
c8285564a3 Use state to calculate get_users_in_room 2016-06-01 15:25:25 +01:00
David Baker
d60eed0710 Limit number of notifications in an email notification 2016-06-01 11:45:43 +01:00
Erik Johnston
43db0d9f6a Add get_users_with_read_receipts_in_room cache 2016-06-01 10:54:32 +01:00
David Baker
8e539f13c0 Merge remote-tracking branch 'origin/develop' into dbkr/room_list_spider 2016-06-01 09:54:36 +01:00
David Baker
6ecb2ca4ec pep8 2016-06-01 09:48:55 +01:00
David Baker
2a449fec4d Add cache to remote room lists
Poll for updates from remote servers, waiting for the poll if there's no cache entry.
2016-05-31 18:27:23 +01:00
David Baker
6ca4d3ae9a Add vector.im to default secondary_directory_servers and add comment explaining it's not a permanent solution 2016-05-31 17:24:50 +01:00
Erik Johnston
dea9f20f8c Force boolean 2016-05-31 17:24:30 +01:00
David Baker
963e3ed282 Apparently I am not permitted to have two blank lines here 2016-05-31 17:22:53 +01:00
David Baker
d240796ded Basic, un-cached support for secondary_directory_servers 2016-05-31 17:20:07 +01:00
Mark Haines
c8c5bf950a Fix synapse/storage/schema/delta/30/as_users.py 2016-05-31 17:10:40 +01:00
Erik Johnston
c9ca285d33 Merge pull request #805 from matrix-org/erikj/push_rules_cache
Fix GET /push_rules
2016-05-31 16:42:21 +01:00
Erik Johnston
1d4ee854e2 Fix typo 2016-05-31 15:45:53 +01:00
Erik Johnston
cca0093fa9 Change fix 2016-05-31 15:44:08 +01:00
Erik Johnston
4efa389299 Fix GET /push_rules 2016-05-31 15:37:53 +01:00
Erik Johnston
aefd2d1cbc Cache get_event_reference_hashes 2016-05-31 15:32:32 +01:00
Erik Johnston
10de8c2631 Merge pull request #804 from matrix-org/erikj/push_rules_cache
Add caches to bulk_get_push_rules*
2016-05-31 15:04:40 +01:00
Mark Haines
c626fc576a Move the AS handler out of the Handlers object.
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
Erik Johnston
e5b0bbcd33 Add caches to bulk_get_push_rules* 2016-05-31 13:46:58 +01:00
David Baker
70ecb415f5 Fix c+p fail 2016-05-31 12:00:54 +01:00
David Baker
e1625d62a8 Add federation room list servlet 2016-05-31 11:55:57 +01:00
David Baker
887c6e6f05 Split out the room list handler
So I can use it from federation bits without pulling in all the handlers.
2016-05-31 11:05:16 +01:00
Erik Johnston
85b992f621 Fix to allow start with postgres 2016-05-27 10:44:44 +01:00
Erik Johnston
cc84f7cb8e Send down correct error response if user not found 2016-05-27 10:35:15 +01:00
David Baker
209ba4d024 Merge pull request #795 from matrix-org/dbkr/delete_push_actions_after_a_month
Only delete push actions after 30 days
2016-05-24 16:22:13 +01:00
David Baker
b007ee4606 Check for presence of 'avatar_url' key 2016-05-24 15:12:05 +01:00
David Baker
cbf8d146ac Merge pull request #799 from matrix-org/matthew/quieter-email-notifs
Tune email notifs to make them quieter:
2016-05-24 14:30:51 +01:00
Erik Johnston
faad233ea6 Change short circuit path 2016-05-24 14:27:19 +01:00
Erik Johnston
6900303997 Don't send down all ephemeral events 2016-05-24 11:44:55 +01:00
David Baker
37b7e84620 Include the ts the notif was received at 2016-05-24 11:33:32 +01:00
Erik Johnston
1c5ed2a19b Only work out newly_joined_users for incremental sync 2016-05-24 11:21:34 +01:00
Erik Johnston
b08ad0389e Only include non-offline presence in initial sync 2016-05-24 11:15:05 +01:00
Erik Johnston
be2c677386 Spell builder correctly 2016-05-24 10:53:03 +01:00
Erik Johnston
79bea8ab9a Inline function. Make load_filtered_recents private 2016-05-24 10:22:24 +01:00
Erik Johnston
84f94e4cbb Add comments 2016-05-24 10:14:53 +01:00
Erik Johnston
137e6a4557 Shuffle things room 2016-05-24 09:50:55 +01:00
Matthew Hodgson
680f1d9387 catch thinko in presentable names 2016-05-23 22:55:11 +01:00
Matthew Hodgson
cb8a321bdd fix NPE in room ordering 2016-05-23 22:54:56 +01:00
Matthew Hodgson
88ea5ab2c3 consistency is the better part of valour 2016-05-23 19:33:45 +01:00
Matthew Hodgson
989bdc9e56 Tune email notifs to make them quieter:
* After initial 10 minute window, only alert every 24h for room notifs
 * Reset room state after 6h of idleness
 * Synchronise throttles for messages sent in the same notif, so the 24 hourly notifs 'line up'
 * Fix the email subjects to say what triggered the notification
 * Order the rooms in reverse activity order in the email, so the 'reason' room should always come first
2016-05-23 19:24:11 +01:00
Negi Fazeli
6fe04ffef2 Fix set profile error with Requester.
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
2016-05-23 19:50:28 +02:00
David Baker
b791a530da Actually make the 'read' flag correct 2016-05-23 18:48:02 +01:00
David Baker
a24bc5b2dc Add GET /notifications API 2016-05-23 18:33:51 +01:00
Erik Johnston
c0c79ef444 Add back concurrently_execute 2016-05-23 18:21:27 +01:00
Erik Johnston
b5605dfecc Refactor SyncHandler 2016-05-23 18:08:18 +01:00
David Baker
31b5395ab6 Remove debug logging 2016-05-23 16:32:01 +01:00
David Baker
c2da3406fc Oops, missing comma 2016-05-20 18:03:31 +01:00
David Baker
ccffb0965d Remove stale line 2016-05-20 17:59:10 +01:00
David Baker
18d68bfee4 Handle empty events table 2016-05-20 17:58:09 +01:00
David Baker
d4503e25ed Make deleting push actions more efficient
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
David Baker
149fa411e2 Only delete push actions after 30 days 2016-05-20 15:25:12 +01:00
Kegan Dougal
332d7e9b97 Allow clients to specify a server_name to avoid 'No known servers'
Multiple server_names are supported via ?server_name=foo&server_name=bar
2016-05-19 13:50:52 +01:00
Matthew Hodgson
6fb51eaf7b Merge pull request #793 from matrix-org/matthew/one-push-badge-per-convo
increment badge count per missed convo, not per msg
2016-05-18 13:56:38 +01:00
Matthew Hodgson
e837df6adb increment badge count per missed convo, not per msg 2016-05-18 11:53:25 +01:00
Erik Johnston
42368ea8db Add desc to get_presence_for_users 2016-05-18 11:38:10 +01:00
Mark Haines
0cb441fedd Move typing handler out of the Handlers object 2016-05-17 15:58:46 +01:00
Mark Haines
6a30a0bfd3 Move the functions for parsing app service config 2016-05-17 11:28:58 +01:00
Mark Haines
523d5bcd0b Merge remote-tracking branch 'origin/develop' into markjh/liberate_sync_handler 2016-05-17 10:43:58 +01:00
Matthew Hodgson
43e1e0489c Merge pull request #786 from matrix-org/matthew/email_notifs_tuning
tune email notifs, fix CSS a bit, and add debugging details
2016-05-17 10:43:24 +01:00
Mark Haines
526bf8126f Remove unused get_joined_rooms_for_user 2016-05-17 10:20:51 +01:00
Mark Haines
425e6b4983 Merge branch 'develop' into markjh/member_cleanup 2016-05-17 10:13:16 +01:00
Mark Haines
b153f5b150 Merge pull request #787 from matrix-org/markjh/liberate_presence_handler
Move the presence handler out of the Handlers object
2016-05-17 10:09:43 +01:00
Mark Haines
3fd8a07fca Merge pull request #788 from matrix-org/markjh/domian
Spell "domain" correctly
2016-05-17 10:09:34 +01:00
Mark Haines
f68eea808a Move SyncHandler out of the Handlers object 2016-05-16 20:19:26 +01:00
Mark Haines
53e171f345 Merge branch 'markjh/liberate_presence_handler' into markjh/liberate_sync_handler 2016-05-16 20:08:32 +01:00
Mark Haines
80cb9becd8 Remove get_joined_rooms_for_user from RoomMemberHandler 2016-05-16 20:06:55 +01:00
Mark Haines
816df9f267 get_room_members is unused now 2016-05-16 19:51:43 +01:00
Mark Haines
821306120a Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room 2016-05-16 19:48:07 +01:00
Mark Haines
1a3a2002ff Spell "domain" correctly
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Mark Haines
e168abbcff Don't inherit PresenceHandler from BaseHandler, remove references to self.hs from presence handler 2016-05-16 19:08:40 +01:00
Matthew Hodgson
cbd2adc95e tune email notifs, fix CSS a bit, and add debugging details 2016-05-16 18:58:38 +01:00
Mark Haines
3b86ecfa79 Move the presence handler out of the Handlers object 2016-05-16 18:56:37 +01:00
David Baker
647781ca56 Fix emailpusher import
Try importing at the root level rather than conditionally importing, as per comment
2016-05-16 18:41:32 +01:00
Erik Johnston
c39f305067 os.environ requires a string 2016-05-16 17:21:30 +01:00
Erik Johnston
678c8a7f1e Merge pull request #785 from matrix-org/erikj/cache_factor_synctly
Make synctl read a cache factor from config file
2016-05-16 17:07:35 +01:00
Erik Johnston
c5c5a7403b Make synctl read a cache factor from config file 2016-05-16 17:01:57 +01:00
Mark Haines
eb79110beb Clean up the blacklist/whitelist handling.
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
2016-05-16 13:03:59 +01:00
Mark Haines
dd95eb4cb5 Merge branch 'develop' into matthew/preview_url_ip_whitelist 2016-05-16 12:59:41 +01:00
Erik Johnston
60d53f9e95 Count number of GC collects 2016-05-16 09:34:42 +01:00
Matthew Hodgson
782471b7e1 fix matrix.to URLs 2016-05-13 17:50:16 +01:00
Mark Haines
0466454b00 Assert that stream replicated stream positions are ints 2016-05-13 17:33:44 +01:00
Mark Haines
b3f29dc1e5 Manually expire broken caches like the who_forgot_in_room 2016-05-13 17:16:27 +01:00
Mark Haines
f03ddc98ec Use the SlavedAccountDataStore 2016-05-13 17:01:28 +01:00
Mark Haines
1f71f386f6 Merge branch 'develop' into dbkr/email_notifs_on_pusher 2016-05-13 16:59:56 +01:00
Mark Haines
206eb9fd94 Shift some of the state_group methods into the SlavedEventStore 2016-05-13 16:58:14 +01:00
Erik Johnston
7d6e89ed22 Add a comment 2016-05-13 16:31:08 +01:00
Mark Haines
21018c2c13 Merge pull request #783 from matrix-org/markjh/slave_account_data
Add a slaved datastore for account data
2016-05-13 15:56:04 +01:00
Mark Haines
b7381d5338 Allow receipts for events we haven't seen in the db 2016-05-13 15:46:41 +01:00
Mark Haines
3abab26458 Add a slaved datastore for account data 2016-05-13 15:34:06 +01:00
Erik Johnston
99b5a2e560 Merge pull request #741 from negzi/create_user_with_expiry
Create user with expiry
2016-05-13 14:46:53 +01:00
Erik Johnston
ba5c616ff4 Merge pull request #778 from matrix-org/erikj/add_pusher
Fixup add_pusher
2016-05-13 14:43:23 +01:00
Erik Johnston
0c11c1be88 Spelling 2016-05-13 14:42:25 +01:00
Erik Johnston
e00e8f2166 Merge pull request #769 from matrix-org/erikj/push_actions_delete
Delete old pushers
2016-05-13 14:41:36 +01:00
Erik Johnston
fd8e921b6e Merge pull request #779 from matrix-org/erikj/receipts
Use tree cache for get_linearized_receipts_for_room
2016-05-13 14:41:21 +01:00
Erik Johnston
c9aff0736c Remove topics table 2016-05-13 14:40:38 +01:00
Negi Fazeli
40aa6e8349 Create user with expiry
- Add unittests for client, api and handler

Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
Mark Haines
9295fa30a8 Annotate the removed indicies with why they were removed. 2016-05-13 14:16:57 +01:00
Erik Johnston
5e50058473 Remove unused indices
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
Mark Haines
cdda850ce1 Merge pull request #781 from matrix-org/markjh/replication_problems
Fix a bug in replication that was causing the pusher to tight loop
2016-05-13 12:05:19 +01:00
Mark Haines
0e792e7903 Log the stream IDs in an order that makes sense 2016-05-13 11:54:44 +01:00
Mark Haines
3547e66bc6 Make sure we advance our stream position 2016-05-13 11:53:00 +01:00
Erik Johnston
6da7f39d95 Use tree cache for get_linearized_receipts_for_room 2016-05-13 11:41:23 +01:00
David Baker
b5e646a18c Make email notifs work on the pusher synapse
Plus general bugfix to email notif code
2016-05-13 11:36:50 +01:00
Erik Johnston
13d37c3c56 Fixup add_pusher 2016-05-13 11:25:02 +01:00
Mark Haines
a458a40337 missed a spot 2016-05-12 18:19:58 +01:00
Mark Haines
7e23476814 move filter_events_for_client out of base handler 2016-05-11 13:42:37 +01:00
Mark Haines
1620578b13 Shuffle when we get the signing_key attribute.
Wait until we sign a message to get the signing key from the homeserver
config. This means that the message handler can be created without
having a signing key in the config which means that separate processes
like the pusher that don't send messages and don't need to sign them can
still access the handlers.
2016-05-11 12:20:57 +01:00
Erik Johnston
108434e53d Merge pull request #775 from matrix-org/erikj/password_hash
Correctly handle NULL password hashes from the database
2016-05-11 12:18:13 +01:00
Erik Johnston
1400bb1663 Correctly handle NULL password hashes from the database 2016-05-11 12:06:02 +01:00
Mark Haines
458a435114 Fix typo 2016-05-11 10:35:33 +01:00
Mark Haines
30057b1e15 Move _create_new_client_event and handle_new_client_event out of base handler 2016-05-11 09:09:20 +01:00
David Baker
ae1af262f6 Pass through _get_state_group_for_events 2016-05-10 19:18:03 +02:00
David Baker
90afc07f39 StateStore, not EventsStore 2016-05-10 19:10:46 +02:00
David Baker
89b5ef7c4b Cached functions must be accessed through the dict 2016-05-10 19:05:22 +02:00
David Baker
35b6e6d2a8 Pass though _get_state_group_for_events 2016-05-10 18:56:40 +02:00
David Baker
3367e65476 Pass through get_state_groups 2016-05-10 18:53:15 +02:00
David Baker
0c4ccdcb83 Also pass through get_profile_displayname 2016-05-10 18:51:14 +02:00
David Baker
f28643cea9 Uncommit accidentally commited edit to cipher list 2016-05-10 18:44:32 +02:00
David Baker
5f46be19a7 Pass through get_events to pusher too 2016-05-10 18:43:40 +02:00
David Baker
d46b18a00f Pass through _get_event_txn 2016-05-10 18:27:06 +02:00
Matrix
3b1930e8ec unbreak schema 2016-05-10 16:42:37 +01:00
Matthew Hodgson
fe97b81c09 Merge pull request #759 from matrix-org/dbkr/email_notifs
Send email notifications for missed messages
2016-05-10 16:30:05 +02:00
David Baker
997db04648 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-05-10 14:40:19 +02:00
David Baker
c00b484eff More consistent config naming 2016-05-10 14:39:16 +02:00
David Baker
94040b0798 Add config option to not send email notifs for new users 2016-05-10 14:34:53 +02:00
David Baker
e04b1d6b0a Make pep8 happy 2016-05-10 14:23:16 +02:00
Matthew Hodgson
5599608887 Switch from CSS to Table layout for HTML mails so they work in Outlook aka Word
Remove templates-vector and theme templates with variables instead
    Switch to matrix.to URLs by default for links
2016-05-10 00:14:48 +02:00
Erik Johnston
c9eb6dfc1b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-09 13:21:06 +01:00
Erik Johnston
3f84da139c Merge pull request #773 from matrix-org/erikj/get_domian_from_id
Add and use get_domain_from_id
2016-05-09 13:21:00 +01:00
Erik Johnston
def64d6ef3 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-09 13:05:09 +01:00
Erik Johnston
100e2c42f6 Merge pull request #764 from matrix-org/erikj/replication_logging
Add some log information at returned replication streams
2016-05-09 11:18:00 +01:00
Erik Johnston
8715731559 Merge pull request #772 from matrix-org/erikj/get_user_cache
Add cache to get_user_by_id
2016-05-09 11:12:11 +01:00
Erik Johnston
34b3af3363 Merge pull request #770 from matrix-org/erikj/transaction_reactor
Run transaction queue on reactor
2016-05-09 11:12:06 +01:00
Erik Johnston
08dfa8eee2 Add and use get_domian_from_id 2016-05-09 10:36:03 +01:00
Erik Johnston
1f1dee94f6 Manually run GC on reactor tick.
This also adds a metric for amount of time spent in GC.
2016-05-09 10:13:25 +01:00
Erik Johnston
f6ebaf4a32 Run transaction queue on reactor
This ensures that any CPU work that happens doesn't block message
sending.
2016-05-09 10:10:06 +01:00
Erik Johnston
4ea762c1a2 Add cache to get_user_by_id 2016-05-09 10:08:21 +01:00
Erik Johnston
012cb5416c Merge branch 'develop' of github.com:matrix-org/synapse into erikj/push_actions_delete 2016-05-06 15:59:20 +01:00
Erik Johnston
fcb2c3f0db Remove unused import 2016-05-06 15:47:40 +01:00
Erik Johnston
fd85b167ec Pull loop one level up 2016-05-06 15:38:42 +01:00
Erik Johnston
b6e0be701e Queue events for persistence 2016-05-06 14:31:44 +01:00
Erik Johnston
d13459636f Pull prev txn from in memory 2016-05-06 11:30:55 +01:00
Erik Johnston
1d275dba69 Don't needlessly enter transaction 2016-05-06 11:25:58 +01:00
Erik Johnston
56b5e83e36 Reduce database inserts when sending transactions 2016-05-06 11:20:18 +01:00
Matthew Hodgson
53ca739f1f better mail subject lines 2016-05-05 15:55:44 +01:00
Matthew Hodgson
81c2176cba fix layout; handle app naming in synapse, not jinja 2016-05-05 15:54:29 +01:00
Mark Haines
573ef3f1c9 Rename openid/token to openid/request_token 2016-05-05 15:15:00 +01:00
Erik Johnston
8940281d1b Don't warn 2016-05-05 15:10:03 +01:00
Mark Haines
9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston
5d8a93a10e Add some log information at returned replication streams 2016-05-05 10:29:21 +01:00
Erik Johnston
1f0f5ffa1e Add bulk fetch storage API 2016-05-05 10:03:15 +01:00
Matthew Hodgson
634efb65f1 pep8 2016-05-05 02:10:57 +01:00
Matthew Hodgson
ce81ccb063 handle fragments correctly on mxc URLs.
switch to vector.im permalinks as matrix.to isn't ready yet.
merge overlapping notifications together.
give one message of context after a notification (in the unlikely event it exists, but it's possible thanks to throttling).
include name of app in mail templates
2016-05-05 02:00:33 +01:00
Matthew Hodgson
1cf5c379cb spell out emailpusher full path 2016-05-05 01:59:39 +01:00
Erik Johnston
fee1118a20 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-04 19:08:27 +01:00
Erik Johnston
97b9141245 Merge pull request #762 from matrix-org/erikj/report_event
Add /report endpoint
2016-05-04 19:05:49 +01:00
Erik Johnston
fcd1eb642d Add primary key 2016-05-04 16:51:51 +01:00
Erik Johnston
8e6a163f27 Add timestamp and auto incrementing ID 2016-05-04 15:19:12 +01:00
David Baker
39d0a99972 Include no context
until we can de-dup between the context and other notifs
2016-05-04 14:52:49 +01:00
David Baker
9ef05a12c3 Add date header & message id 2016-05-04 14:52:10 +01:00
David Baker
80be396464 Correct SQL statement for postgres
In standard sql, join binds tighter than comma, so we were joining on the wrong table. Postgres follows the standard (apparently).
2016-05-04 13:19:59 +01:00
Erik Johnston
5650e38e7d Move event_id to path 2016-05-04 13:19:39 +01:00
David Baker
8cc82aad87 Add db functions used for email to the pusher app 2016-05-04 11:47:59 +01:00
David Baker
de22001ab5 pep8 2016-05-04 11:41:35 +01:00
Matthew Hodgson
f1026418ea copyright 2016-05-04 11:38:01 +01:00
Matthew Hodgson
17cbf773b9 fix assorted typos in default config 2016-05-04 11:38:01 +01:00
Erik Johnston
984d4a2c0f Add /report endpoint 2016-05-04 11:28:10 +01:00
David Baker
e6bffa4475 Unused import 2016-05-04 11:26:58 +01:00
David Baker
92f0f3d21d Catch all exceptions when creating a pusher 2016-05-04 11:24:07 +01:00
Erik Johnston
a438a6d2bc Implement basic ignore user 2016-05-04 10:16:46 +01:00
Erik Johnston
7ea3b4118d Merge pull request #757 from matrix-org/erikj/event_auth_typo
Fix typo in event_auth servlet path
2016-05-03 14:23:15 +01:00
Erik Johnston
183f23f10d Delete old pushers 2016-05-03 14:22:33 +01:00
Matthew Hodgson
792def4928 add a url_preview_ip_range_whitelist config param so we can whitelist the matrix.org IP space 2016-05-01 12:44:24 +01:00
David Baker
2df75de505 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 20:28:47 +01:00
David Baker
b084e4d963 Add constant for throttle multiplier 2016-04-29 20:14:55 +01:00
David Baker
35b7b8e4bc Remove unused function 2016-04-29 20:10:34 +01:00
David Baker
6b9b6a9169 Remove unused arg 2016-04-29 20:02:52 +01:00
David Baker
c7c75e87dc Docstring 2016-04-29 19:47:35 +01:00
David Baker
b0a1036d93 Use explicit join 2016-04-29 19:28:56 +01:00
David Baker
8f99cd5996 Oops, actually specify the user id 2016-04-29 19:27:03 +01:00
David Baker
60f44c098d Remove unnecessary if 2016-04-29 19:17:10 +01:00
David Baker
50ad8005e4 Put spaces at start of line 2016-04-29 19:16:15 +01:00
David Baker
83618d719a Try imports in config 2016-04-29 19:13:52 +01:00
David Baker
e7a76b5123 Use the constant 2016-04-29 19:10:45 +01:00
David Baker
29c8cf8db8 Avoid vars builtin 2016-04-29 19:09:28 +01:00
David Baker
d3da5294e8 Use named parameter format 2016-04-29 19:04:40 +01:00
David Baker
765f2b8446 Default enable email notifs to False 2016-04-29 14:46:18 +01:00
David Baker
311b5ce051 pep8 2016-04-29 14:37:30 +01:00
David Baker
3facde2536 Remove rather pointless get function 2016-04-29 14:36:45 +01:00
David Baker
4364ea1272 Stop processing notifs once we've sent a mail 2016-04-29 14:31:27 +01:00
David Baker
4b0c3a3270 Correct public_baseurl default 2016-04-29 14:30:15 +01:00
David Baker
5048455965 Nicer get() shorthand 2016-04-29 14:27:40 +01:00
David Baker
6c8957be7f Remove redundant docstring 2016-04-29 14:25:28 +01:00
David Baker
18ce88bd2d Correct default template and add text template 2016-04-29 14:24:25 +01:00
David Baker
40d40e470d Send mail notifs with a plaintext part too 2016-04-29 13:56:21 +01:00
David Baker
dc2c527ce9 Fix password reset
Default requester to None, otherwise it isn't defined when resetting using email auth
2016-04-29 12:07:54 +01:00
Erik Johnston
62b51b8452 Fix typo in event_auth servlet path 2016-04-29 12:00:51 +01:00
David Baker
b2c04da8dc Add an email pusher for new users
If they registered with an email address and email notifs are enabled on the HS
2016-04-29 11:43:57 +01:00
David Baker
ec9cbe847d pep8 newline 2016-04-29 10:07:30 +01:00
David Baker
acded821c4 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 10:05:20 +01:00
David Baker
e800ee2f63 May as well always include room link 2016-04-28 17:28:27 +01:00
David Baker
cc0874cf71 Put back real delay before mailing 2016-04-28 17:00:40 +01:00
David Baker
68f8fc2f14 Support file messages & fix plain text 2016-04-28 16:59:57 +01:00
David Baker
4845c7359d Support image notifs 2016-04-28 15:55:53 +01:00
Mark Haines
351b50a887 Fix more typos in per-request metrics 2016-04-28 15:29:46 +01:00
David Baker
60f86fc876 pep8 2016-04-28 15:16:30 +01:00
David Baker
937c407eef Only import email pusher if email notifs are on 2016-04-28 15:12:14 +01:00
Mark Haines
dcfc10b129 Fix typo in request metrics 2016-04-28 15:11:06 +01:00
Matthew Hodgson
aebd0c9717 fix typo 2016-04-28 15:09:25 +01:00
Mark Haines
1a12766e3b Add a comment explaining why automatic metric reporting is disabled for JsonResource 2016-04-28 12:31:26 +01:00
Mark Haines
6037349512 Check if report_metrics is True 2016-04-28 12:26:07 +01:00
David Baker
ebbabc4986 Handle room invites in email notifs 2016-04-28 11:49:36 +01:00
Mark Haines
8d7ad44331 Report per request metrics for all of the things using request_handler 2016-04-28 10:57:49 +01:00
David Baker
9dba1b668c Linkify plain text messages too 2016-04-28 10:55:08 +01:00
David Baker
424a7f48f8 Run filter_events_for_client
so we don't accidentally mail out events people shouldn't see
2016-04-27 17:50:49 +01:00
David Baker
4ed1e45869 Make html messages work 2016-04-27 17:18:51 +01:00
Mark Haines
21d188bf95 Merge pull request #755 from matrix-org/markjh/right_direction
Fix backfill replication to advance the stream correctly
2016-04-27 15:54:48 +01:00
Mark Haines
8a65666454 Fix backfill replication to advance the stream correctly 2016-04-27 15:38:43 +01:00
David Baker
8781083960 Better grammar for multiple messages in a room
Say who the messages are from if there's no room name, otherwise it's a bit nonsensical
2016-04-27 15:30:41 +01:00
David Baker
fa12209c1b Hopefully all remaining bits for email notifs
Add public facing base url to the server so synapse knows what URL to use when converting mxc to http urls for use in emails
2016-04-27 15:09:55 +01:00
Mark Haines
871357d539 Check that somethign has happend before running the selects 2016-04-27 11:54:13 +01:00
Mark Haines
71df327190 Actually start the pusher daemon 2016-04-26 17:07:09 +01:00
Mark Haines
c9eab73f2a Fix typo in default pusher config 2016-04-26 17:06:18 +01:00
Mark Haines
b80b93ea0f Add a log context to the daemonized pusher 2016-04-26 15:57:28 +01:00
Mark Haines
6df5a6a833 Optionally daemonize the pusher 2016-04-26 15:37:41 +01:00
Erik Johnston
5164ccc3e5 Bump changelog and version 2016-04-26 11:26:32 +01:00
Erik Johnston
3306cf45ca Merge pull request #750 from matrix-org/erikj/jwt_optional
Make pyjwt dependency optional
2016-04-26 11:07:22 +01:00
Mark Haines
9c417c54d4 Add a couple of update methods to the PusherSlaveStore 2016-04-26 10:45:02 +01:00
David Baker
7b4715bad7 More variable calculation for email notifs
Include name of the person we're sending to and add summary text at the top giving an overview of what's happened.
2016-04-25 18:27:04 +01:00
Mark Haines
f15e9e8de4 Remove the uncomments from the comments 2016-04-25 17:56:24 +01:00
Mark Haines
72e2fafa20 Add a metrics listener and a ssh listener to the pusher 2016-04-25 17:34:25 +01:00
Mark Haines
f22f46f4f9 Move the listenTCP call outside the manhole function 2016-04-25 14:59:21 +01:00
David Baker
290f125a13 Typo 2016-04-25 14:42:59 +01:00
Erik Johnston
52ecbc2843 Make pyjwt dependency optional 2016-04-25 14:30:15 +01:00
David Baker
05e49ffbdf No we don't: it's just the display name 2016-04-22 18:44:17 +01:00
David Baker
bd0f9c2065 Actually do UTF8 correctly 2016-04-22 18:42:00 +01:00
David Baker
c5b3c6e101 Sort member events
So names of people in a room are given in order
2016-04-22 18:33:36 +01:00
David Baker
83bf65297a Mime part is binary so encode it first.
Doesn't get character enocind right yet but makes it not error.
2016-04-22 18:31:47 +01:00
David Baker
e8701e64b9 Implement group-of-people names 2016-04-22 17:28:42 +01:00
David Baker
c553797c4f No inlineCallbacks necessary on this 2016-04-22 17:27:54 +01:00
Mark Haines
5905f36f05 Split out setting up the manhole to a separate file 2016-04-22 17:09:15 +01:00
Mark Haines
62607d5452 Merge branch 'develop' into markjh/split_out_site.py
Conflicts:
	synapse/app/homeserver.py
2016-04-22 16:26:57 +01:00
Mark Haines
e856036f4c Move SynapseSite to its own file 2016-04-22 16:09:55 +01:00
Mark Haines
9e7aa98c22 Split out create_resource_tree to a separate file 2016-04-22 15:40:51 +01:00
Mark Haines
2022ae0fb9 Merge pull request #746 from matrix-org/markjh/split_out_pusher
Optionally split out the pushers into a separate process
2016-04-22 11:34:08 +01:00
Erik Johnston
64ec3493c1 Merge pull request #745 from matrix-org/erikj/search-index
Optimise event_search in postgres
2016-04-22 11:23:49 +01:00
Erik Johnston
183cacac90 Simplify query and handle finishing correctly 2016-04-22 10:01:57 +01:00
David Baker
c10ed26c30 Flesh out email templating
Mostly WIP porting the room name calculation logic from the web client so our room names in the email mirror the clients.
2016-04-21 19:19:07 +01:00
Erik Johnston
ae571810f2 Order NULLs first 2016-04-21 18:14:18 +01:00
Erik Johnston
3ddbb1687c Fix query 2016-04-21 18:02:36 +01:00
Erik Johnston
8fae3d7b1e Use special UPDATE syntax 2016-04-21 18:01:49 +01:00
Erik Johnston
b57dcb4b51 Typo 2016-04-21 17:49:00 +01:00
Erik Johnston
26db18bc90 Need to do _background_update_progress_txn in actual transaction 2016-04-21 17:45:56 +01:00
Erik Johnston
b9675ef6e6 Merge pull request #687 from nikriek/jwt-fix
Fix issues with JWT login
2016-04-21 17:42:25 +01:00
Erik Johnston
e395eb1108 Update progress when creating index 2016-04-21 17:39:24 +01:00
Erik Johnston
3b0fa77f50 Fix SQL statement 2016-04-21 17:37:42 +01:00
Erik Johnston
129e403487 Create index must be on a conn 2016-04-21 17:35:51 +01:00
Mark Haines
a3ac837599 Optionally split out the pushers into a separate process 2016-04-21 17:22:37 +01:00
Mark Haines
78741cf025 Merge pull request #743 from matrix-org/markjh/slave_pushers
Replicate the pushers
2016-04-21 17:21:29 +01:00
Erik Johnston
51bb339ab2 Create index concurrently 2016-04-21 17:16:11 +01:00
Erik Johnston
b743c1237e Add missing run_upgrade 2016-04-21 17:12:04 +01:00