Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-15 13:34:18 -05:00
Code Issues Packages Projects Releases Wiki Activity
7,303 Commits 4 Branches 609 Tags 60 MiB
e82247f990
Commit Graph

5106 Commits

Author SHA1 Message Date
Erik Johnston
fd8e921b6e Merge pull request #779 from matrix-org/erikj/receipts 
Use tree cache for get_linearized_receipts_for_room
 2016-05-13 14:41:21 +01:00
Erik Johnston
c9aff0736c Remove topics table 2016-05-13 14:40:38 +01:00
Negi Fazeli
40aa6e8349 Create user with expiry 
- Add unittests for client, api and handler

Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
 2016-05-13 15:34:15 +02:00
Mark Haines
9295fa30a8 Annotate the removed indicies with why they were removed. 2016-05-13 14:16:57 +01:00
Erik Johnston
5e50058473 Remove unused indices 
This includes removing both unused indices and indices that are subsets
of other indices.
 2016-05-13 13:28:07 +01:00
Mark Haines
cdda850ce1 Merge pull request #781 from matrix-org/markjh/replication_problems 
Fix a bug in replication that was causing the pusher to tight loop
 2016-05-13 12:05:19 +01:00
Mark Haines
0e792e7903 Log the stream IDs in an order that makes sense 2016-05-13 11:54:44 +01:00
Mark Haines
3547e66bc6 Make sure we advance our stream position 2016-05-13 11:53:00 +01:00
Erik Johnston
6da7f39d95 Use tree cache for get_linearized_receipts_for_room 2016-05-13 11:41:23 +01:00
David Baker
b5e646a18c Make email notifs work on the pusher synapse 
Plus general bugfix to email notif code
 2016-05-13 11:36:50 +01:00
Erik Johnston
13d37c3c56 Fixup add_pusher 2016-05-13 11:25:02 +01:00
Mark Haines
a458a40337 missed a spot 2016-05-12 18:19:58 +01:00
Mark Haines
7e23476814 move filter_events_for_client out of base handler 2016-05-11 13:42:37 +01:00
Mark Haines
1620578b13 Shuffle when we get the signing_key attribute. 
Wait until we sign a message to get the signing key from the homeserver
config. This means that the message handler can be created without
having a signing key in the config which means that separate processes
like the pusher that don't send messages and don't need to sign them can
still access the handlers.
 2016-05-11 12:20:57 +01:00
Erik Johnston
108434e53d Merge pull request #775 from matrix-org/erikj/password_hash 
Correctly handle NULL password hashes from the database
 2016-05-11 12:18:13 +01:00
Erik Johnston
1400bb1663 Correctly handle NULL password hashes from the database 2016-05-11 12:06:02 +01:00
Mark Haines
458a435114 Fix typo 2016-05-11 10:35:33 +01:00
Mark Haines
30057b1e15 Move _create_new_client_event and handle_new_client_event out of base handler 2016-05-11 09:09:20 +01:00
David Baker
ae1af262f6 Pass through _get_state_group_for_events 2016-05-10 19:18:03 +02:00
David Baker
90afc07f39 StateStore, not EventsStore 2016-05-10 19:10:46 +02:00
David Baker
89b5ef7c4b Cached functions must be accessed through the dict 2016-05-10 19:05:22 +02:00
David Baker
35b6e6d2a8 Pass though _get_state_group_for_events 2016-05-10 18:56:40 +02:00
David Baker
3367e65476 Pass through get_state_groups 2016-05-10 18:53:15 +02:00
David Baker
0c4ccdcb83 Also pass through get_profile_displayname 2016-05-10 18:51:14 +02:00
David Baker
f28643cea9 Uncommit accidentally commited edit to cipher list 2016-05-10 18:44:32 +02:00
David Baker
5f46be19a7 Pass through get_events to pusher too 2016-05-10 18:43:40 +02:00
David Baker
d46b18a00f Pass through _get_event_txn 2016-05-10 18:27:06 +02:00
Matrix
3b1930e8ec unbreak schema 2016-05-10 16:42:37 +01:00
Matthew Hodgson
fe97b81c09 Merge pull request #759 from matrix-org/dbkr/email_notifs 
Send email notifications for missed messages
 2016-05-10 16:30:05 +02:00
David Baker
997db04648 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-05-10 14:40:19 +02:00
David Baker
c00b484eff More consistent config naming 2016-05-10 14:39:16 +02:00
David Baker
94040b0798 Add config option to not send email notifs for new users 2016-05-10 14:34:53 +02:00
David Baker
e04b1d6b0a Make pep8 happy 2016-05-10 14:23:16 +02:00
Matthew Hodgson
5599608887 Switch from CSS to Table layout for HTML mails so they work in Outlook aka Word 
Remove templates-vector and theme templates with variables instead
    Switch to matrix.to URLs by default for links
 2016-05-10 00:14:48 +02:00
Erik Johnston
c9eb6dfc1b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-09 13:21:06 +01:00
Erik Johnston
3f84da139c Merge pull request #773 from matrix-org/erikj/get_domian_from_id 
Add and use get_domain_from_id
 2016-05-09 13:21:00 +01:00
Erik Johnston
def64d6ef3 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-09 13:05:09 +01:00
Erik Johnston
100e2c42f6 Merge pull request #764 from matrix-org/erikj/replication_logging 
Add some log information at returned replication streams
 2016-05-09 11:18:00 +01:00
Erik Johnston
8715731559 Merge pull request #772 from matrix-org/erikj/get_user_cache 
Add cache to get_user_by_id
 2016-05-09 11:12:11 +01:00
Erik Johnston
34b3af3363 Merge pull request #770 from matrix-org/erikj/transaction_reactor 
Run transaction queue on reactor
 2016-05-09 11:12:06 +01:00
Erik Johnston
08dfa8eee2 Add and use get_domian_from_id 2016-05-09 10:36:03 +01:00
Erik Johnston
1f1dee94f6 Manually run GC on reactor tick. 
This also adds a metric for amount of time spent in GC.
 2016-05-09 10:13:25 +01:00
Erik Johnston
f6ebaf4a32 Run transaction queue on reactor 
This ensures that any CPU work that happens doesn't block message
sending.
 2016-05-09 10:10:06 +01:00
Erik Johnston
4ea762c1a2 Add cache to get_user_by_id 2016-05-09 10:08:21 +01:00
Erik Johnston
012cb5416c Merge branch 'develop' of github.com:matrix-org/synapse into erikj/push_actions_delete 2016-05-06 15:59:20 +01:00
Erik Johnston
fcb2c3f0db Remove unused import 2016-05-06 15:47:40 +01:00
Erik Johnston
fd85b167ec Pull loop one level up 2016-05-06 15:38:42 +01:00
Erik Johnston
b6e0be701e Queue events for persistence 2016-05-06 14:31:44 +01:00
Erik Johnston
d13459636f Pull prev txn from in memory 2016-05-06 11:30:55 +01:00
Erik Johnston
1d275dba69 Don't needlessly enter transaction 2016-05-06 11:25:58 +01:00
Erik Johnston
56b5e83e36 Reduce database inserts when sending transactions 2016-05-06 11:20:18 +01:00
Matthew Hodgson
53ca739f1f better mail subject lines 2016-05-05 15:55:44 +01:00
Matthew Hodgson
81c2176cba fix layout; handle app naming in synapse, not jinja 2016-05-05 15:54:29 +01:00
Mark Haines
573ef3f1c9 Rename openid/token to openid/request_token 2016-05-05 15:15:00 +01:00
Erik Johnston
8940281d1b Don't warn 2016-05-05 15:10:03 +01:00
Mark Haines
9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston
5d8a93a10e Add some log information at returned replication streams 2016-05-05 10:29:21 +01:00
Erik Johnston
1f0f5ffa1e Add bulk fetch storage API 2016-05-05 10:03:15 +01:00
Matthew Hodgson
634efb65f1 pep8 2016-05-05 02:10:57 +01:00
Matthew Hodgson
ce81ccb063 handle fragments correctly on mxc URLs. 
switch to vector.im permalinks as matrix.to isn't ready yet.
merge overlapping notifications together.
give one message of context after a notification (in the unlikely event it exists, but it's possible thanks to throttling).
include name of app in mail templates
 2016-05-05 02:00:33 +01:00
Matthew Hodgson
1cf5c379cb spell out emailpusher full path 2016-05-05 01:59:39 +01:00
Erik Johnston
fee1118a20 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-04 19:08:27 +01:00
Erik Johnston
97b9141245 Merge pull request #762 from matrix-org/erikj/report_event 
Add /report endpoint
 2016-05-04 19:05:49 +01:00
Erik Johnston
fcd1eb642d Add primary key 2016-05-04 16:51:51 +01:00
Erik Johnston
8e6a163f27 Add timestamp and auto incrementing ID 2016-05-04 15:19:12 +01:00
David Baker
39d0a99972 Include no context 
until we can de-dup between the context and other notifs
 2016-05-04 14:52:49 +01:00
David Baker
9ef05a12c3 Add date header & message id 2016-05-04 14:52:10 +01:00
David Baker
80be396464 Correct SQL statement for postgres 
In standard sql, join binds tighter than comma, so we were joining on the wrong table. Postgres follows the standard (apparently).
 2016-05-04 13:19:59 +01:00
Erik Johnston
5650e38e7d Move event_id to path 2016-05-04 13:19:39 +01:00
David Baker
8cc82aad87 Add db functions used for email to the pusher app 2016-05-04 11:47:59 +01:00
David Baker
de22001ab5 pep8 2016-05-04 11:41:35 +01:00
Matthew Hodgson
f1026418ea copyright 2016-05-04 11:38:01 +01:00
Matthew Hodgson
17cbf773b9 fix assorted typos in default config 2016-05-04 11:38:01 +01:00
Erik Johnston
984d4a2c0f Add /report endpoint 2016-05-04 11:28:10 +01:00
David Baker
e6bffa4475 Unused import 2016-05-04 11:26:58 +01:00
David Baker
92f0f3d21d Catch all exceptions when creating a pusher 2016-05-04 11:24:07 +01:00
Erik Johnston
a438a6d2bc Implement basic ignore user 2016-05-04 10:16:46 +01:00
Erik Johnston
7ea3b4118d Merge pull request #757 from matrix-org/erikj/event_auth_typo 
Fix typo in event_auth servlet path
 2016-05-03 14:23:15 +01:00
Erik Johnston
183f23f10d Delete old pushers 2016-05-03 14:22:33 +01:00
Matthew Hodgson
792def4928 add a url_preview_ip_range_whitelist config param so we can whitelist the matrix.org IP space 2016-05-01 12:44:24 +01:00
David Baker
2df75de505 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 20:28:47 +01:00
David Baker
b084e4d963 Add constant for throttle multiplier 2016-04-29 20:14:55 +01:00
David Baker
35b7b8e4bc Remove unused function 2016-04-29 20:10:34 +01:00
David Baker
6b9b6a9169 Remove unused arg 2016-04-29 20:02:52 +01:00
David Baker
c7c75e87dc Docstring 2016-04-29 19:47:35 +01:00
David Baker
b0a1036d93 Use explicit join 2016-04-29 19:28:56 +01:00
David Baker
8f99cd5996 Oops, actually specify the user id 2016-04-29 19:27:03 +01:00
David Baker
60f44c098d Remove unnecessary if 2016-04-29 19:17:10 +01:00
David Baker
50ad8005e4 Put spaces at start of line 2016-04-29 19:16:15 +01:00
David Baker
83618d719a Try imports in config 2016-04-29 19:13:52 +01:00
David Baker
e7a76b5123 Use the constant 2016-04-29 19:10:45 +01:00
David Baker
29c8cf8db8 Avoid vars builtin 2016-04-29 19:09:28 +01:00
David Baker
d3da5294e8 Use named parameter format 2016-04-29 19:04:40 +01:00
David Baker
765f2b8446 Default enable email notifs to False 2016-04-29 14:46:18 +01:00
David Baker
311b5ce051 pep8 2016-04-29 14:37:30 +01:00
David Baker
3facde2536 Remove rather pointless get function 2016-04-29 14:36:45 +01:00
David Baker
4364ea1272 Stop processing notifs once we've sent a mail 2016-04-29 14:31:27 +01:00
David Baker
4b0c3a3270 Correct public_baseurl default 2016-04-29 14:30:15 +01:00
David Baker
5048455965 Nicer get() shorthand 2016-04-29 14:27:40 +01:00
David Baker
6c8957be7f Remove redundant docstring 2016-04-29 14:25:28 +01:00
David Baker
18ce88bd2d Correct default template and add text template 2016-04-29 14:24:25 +01:00
David Baker
40d40e470d Send mail notifs with a plaintext part too 2016-04-29 13:56:21 +01:00
David Baker
dc2c527ce9 Fix password reset 
Default requester to None, otherwise it isn't defined when resetting using email auth
 2016-04-29 12:07:54 +01:00
Erik Johnston
62b51b8452 Fix typo in event_auth servlet path 2016-04-29 12:00:51 +01:00
David Baker
b2c04da8dc Add an email pusher for new users 
If they registered with an email address and email notifs are enabled on the HS
 2016-04-29 11:43:57 +01:00
David Baker
ec9cbe847d pep8 newline 2016-04-29 10:07:30 +01:00
David Baker
acded821c4 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 10:05:20 +01:00
David Baker
e800ee2f63 May as well always include room link 2016-04-28 17:28:27 +01:00
David Baker
cc0874cf71 Put back real delay before mailing 2016-04-28 17:00:40 +01:00
David Baker
68f8fc2f14 Support file messages & fix plain text 2016-04-28 16:59:57 +01:00
David Baker
4845c7359d Support image notifs 2016-04-28 15:55:53 +01:00
Mark Haines
351b50a887 Fix more typos in per-request metrics 2016-04-28 15:29:46 +01:00
David Baker
60f86fc876 pep8 2016-04-28 15:16:30 +01:00
David Baker
937c407eef Only import email pusher if email notifs are on 2016-04-28 15:12:14 +01:00
Mark Haines
dcfc10b129 Fix typo in request metrics 2016-04-28 15:11:06 +01:00
Matthew Hodgson
aebd0c9717 fix typo 2016-04-28 15:09:25 +01:00
Mark Haines
1a12766e3b Add a comment explaining why automatic metric reporting is disabled for JsonResource 2016-04-28 12:31:26 +01:00
Mark Haines
6037349512 Check if report_metrics is True 2016-04-28 12:26:07 +01:00
David Baker
ebbabc4986 Handle room invites in email notifs 2016-04-28 11:49:36 +01:00
Mark Haines
8d7ad44331 Report per request metrics for all of the things using request_handler 2016-04-28 10:57:49 +01:00
David Baker
9dba1b668c Linkify plain text messages too 2016-04-28 10:55:08 +01:00
David Baker
424a7f48f8 Run filter_events_for_client 
so we don't accidentally mail out events people shouldn't see
 2016-04-27 17:50:49 +01:00
David Baker
4ed1e45869 Make html messages work 2016-04-27 17:18:51 +01:00
Mark Haines
21d188bf95 Merge pull request #755 from matrix-org/markjh/right_direction 
Fix backfill replication to advance the stream correctly
 2016-04-27 15:54:48 +01:00
Mark Haines
8a65666454 Fix backfill replication to advance the stream correctly 2016-04-27 15:38:43 +01:00
David Baker
8781083960 Better grammar for multiple messages in a room 
Say who the messages are from if there's no room name, otherwise it's a bit nonsensical
 2016-04-27 15:30:41 +01:00
David Baker
fa12209c1b Hopefully all remaining bits for email notifs 
Add public facing base url to the server so synapse knows what URL to use when converting mxc to http urls for use in emails
 2016-04-27 15:09:55 +01:00
Mark Haines
871357d539 Check that somethign has happend before running the selects 2016-04-27 11:54:13 +01:00
Mark Haines
71df327190 Actually start the pusher daemon 2016-04-26 17:07:09 +01:00
Mark Haines
c9eab73f2a Fix typo in default pusher config 2016-04-26 17:06:18 +01:00
Mark Haines
b80b93ea0f Add a log context to the daemonized pusher 2016-04-26 15:57:28 +01:00
Mark Haines
6df5a6a833 Optionally daemonize the pusher 2016-04-26 15:37:41 +01:00
Erik Johnston
5164ccc3e5 Bump changelog and version 2016-04-26 11:26:32 +01:00
Erik Johnston
3306cf45ca Merge pull request #750 from matrix-org/erikj/jwt_optional 
Make pyjwt dependency optional
 2016-04-26 11:07:22 +01:00
Mark Haines
9c417c54d4 Add a couple of update methods to the PusherSlaveStore 2016-04-26 10:45:02 +01:00
David Baker
7b4715bad7 More variable calculation for email notifs 
Include name of the person we're sending to and add summary text at the top giving an overview of what's happened.
 2016-04-25 18:27:04 +01:00
Mark Haines
f15e9e8de4 Remove the uncomments from the comments 2016-04-25 17:56:24 +01:00
Mark Haines
72e2fafa20 Add a metrics listener and a ssh listener to the pusher 2016-04-25 17:34:25 +01:00
Mark Haines
f22f46f4f9 Move the listenTCP call outside the manhole function 2016-04-25 14:59:21 +01:00
David Baker
290f125a13 Typo 2016-04-25 14:42:59 +01:00
Erik Johnston
52ecbc2843 Make pyjwt dependency optional 2016-04-25 14:30:15 +01:00
David Baker
05e49ffbdf No we don't: it's just the display name 2016-04-22 18:44:17 +01:00
David Baker
bd0f9c2065 Actually do UTF8 correctly 2016-04-22 18:42:00 +01:00
David Baker
c5b3c6e101 Sort member events 
So names of people in a room are given in order
 2016-04-22 18:33:36 +01:00
David Baker
83bf65297a Mime part is binary so encode it first. 
Doesn't get character enocind right yet but makes it not error.
 2016-04-22 18:31:47 +01:00
David Baker
e8701e64b9 Implement group-of-people names 2016-04-22 17:28:42 +01:00
David Baker
c553797c4f No inlineCallbacks necessary on this 2016-04-22 17:27:54 +01:00
Mark Haines
5905f36f05 Split out setting up the manhole to a separate file 2016-04-22 17:09:15 +01:00
Mark Haines
62607d5452 Merge branch 'develop' into markjh/split_out_site.py 
Conflicts:
	synapse/app/homeserver.py
 2016-04-22 16:26:57 +01:00
Mark Haines
e856036f4c Move SynapseSite to its own file 2016-04-22 16:09:55 +01:00
Mark Haines
9e7aa98c22 Split out create_resource_tree to a separate file 2016-04-22 15:40:51 +01:00
Mark Haines
2022ae0fb9 Merge pull request #746 from matrix-org/markjh/split_out_pusher 
Optionally split out the pushers into a separate process
 2016-04-22 11:34:08 +01:00
Erik Johnston
64ec3493c1 Merge pull request #745 from matrix-org/erikj/search-index 
Optimise event_search in postgres
 2016-04-22 11:23:49 +01:00
Erik Johnston
183cacac90 Simplify query and handle finishing correctly 2016-04-22 10:01:57 +01:00
David Baker
c10ed26c30 Flesh out email templating 
Mostly WIP porting the room name calculation logic from the web client so our room names in the email mirror the clients.
 2016-04-21 19:19:07 +01:00
Erik Johnston
ae571810f2 Order NULLs first 2016-04-21 18:14:18 +01:00
Erik Johnston
3ddbb1687c Fix query 2016-04-21 18:02:36 +01:00
Erik Johnston
8fae3d7b1e Use special UPDATE syntax 2016-04-21 18:01:49 +01:00
Erik Johnston
b57dcb4b51 Typo 2016-04-21 17:49:00 +01:00
Erik Johnston
26db18bc90 Need to do _background_update_progress_txn in actual transaction 2016-04-21 17:45:56 +01:00
Erik Johnston
b9675ef6e6 Merge pull request #687 from nikriek/jwt-fix 
Fix issues with JWT login
 2016-04-21 17:42:25 +01:00
Erik Johnston
e395eb1108 Update progress when creating index 2016-04-21 17:39:24 +01:00
Erik Johnston
3b0fa77f50 Fix SQL statement 2016-04-21 17:37:42 +01:00
Erik Johnston
129e403487 Create index must be on a conn 2016-04-21 17:35:51 +01:00
Mark Haines
a3ac837599 Optionally split out the pushers into a separate process 2016-04-21 17:22:37 +01:00
Mark Haines
78741cf025 Merge pull request #743 from matrix-org/markjh/slave_pushers 
Replicate the pushers
 2016-04-21 17:21:29 +01:00
Erik Johnston
51bb339ab2 Create index concurrently 2016-04-21 17:16:11 +01:00
Erik Johnston
b743c1237e Add missing run_upgrade 2016-04-21 17:12:04 +01:00
Mark Haines
31719ad124 Merge pull request #744 from matrix-org/markjh/replication_remove_pusher 
Add a replication endpoint for deleting pushers
 2016-04-21 17:10:49 +01:00
Niklas Riekenbrauck
565c2edb0a Fix issues with JWT login 2016-04-21 18:10:48 +02:00
Erik Johnston
c877f0f034 Optimise event_search in postgres 2016-04-21 16:56:14 +01:00
Mark Haines
cfe1ff4bdb Add a replication endpoint for deleting pushers 2016-04-21 16:33:05 +01:00
Mark Haines
d4823efad9 Replicate the pushers 2016-04-21 16:18:00 +01:00
Mark Haines
c0d8e0eb63 Replicate push actions 2016-04-21 15:25:58 +01:00
David Baker
2ed0adb075 Generate mails from a template 2016-04-20 18:35:29 +01:00
David Baker
05adc6c2de more pep8 2016-04-20 13:02:45 +01:00
David Baker
f63bd4ff47 Send a rather basic email notif 
Also pep8 fixes
 2016-04-20 13:02:01 +01:00
Erik Johnston
5bbc321588 Always use state cache entry if it exists 
Also check if the resolved state matches an existing state group.
 2016-04-20 11:49:10 +01:00
Erik Johnston
4cf4320593 Add some logging to state resolve_events 2016-04-20 11:06:02 +01:00
Erik Johnston
eab47ea1e5 Merge pull request #739 from matrix-org/erikj/cache_get_state_groups_for_groups 
Add cache to _get_state_groups_from_groups
 2016-04-19 17:37:19 +01:00
Mark Haines
f52dd35ac3 Merge pull request #738 from matrix-org/markjh/slaved_receipts 
Add a slaved receipts store
 2016-04-19 17:31:59 +01:00
Erik Johnston
61c7edfd34 Add cache to _get_state_groups_from_groups 2016-04-19 17:22:03 +01:00
Mark Haines
5bbd424ee0 Add a slaved receipts store 2016-04-19 17:14:08 +01:00
Erik Johnston
6ac40f7b65 Merge pull request #737 from matrix-org/erikj/spider_ssl_factory 
Use tls_server_context_factory for SpiderEndpoint
 2016-04-19 16:22:05 +01:00
Erik Johnston
f505575f69 Make InsecureInterceptableContextFactory work with SpiderEndpoint 2016-04-19 16:08:14 +01:00
Mark Haines
e99365f601 Replicate get_invited_rooms_for_user 2016-04-19 15:22:14 +01:00
David Baker
e2a01455af Add single instance & logging stuff 
Copy the stuff over from http pusher that prevents multiple instances of process running at once and sets up logging and measure blocks.
 2016-04-19 14:52:58 +01:00
Erik Johnston
e8884e5e9c Add self.media_repo to PreviewUrlResource 2016-04-19 14:51:34 +01:00
Erik Johnston
a7001c311b _make_dirs was moved to MediaRepository 2016-04-19 14:49:31 +01:00
Erik Johnston
9181e2f4c7 Add store to PreviewUrlResource 2016-04-19 14:48:24 +01:00
Erik Johnston
fb76a81ff7 Reorder imports 2016-04-19 14:45:05 +01:00
David Baker
07d765209d First bits of emailpusher 
Mostly logic of when to send an email
 2016-04-19 14:24:36 +01:00
Erik Johnston
0c93df89b6 Move MediaRepository to media_repository module 2016-04-19 11:31:43 +01:00
Erik Johnston
43f0941e8f Split out BaseMediaResource into MediaRepository 
This is so that a single MediaRepository can be shared across all
resources, rather than having a "copy" per resource.

In particular this allows us to guard against both the thumbnail and
download resource triggering a download of remote content at the same
time.
 2016-04-19 11:24:59 +01:00
Erik Johnston
eb8619e256 Create log context in Measure if one doesn't exist 2016-04-18 16:08:32 +01:00
Erik Johnston
4ef7a25c10 Merge pull request #733 from matrix-org/erikj/make_member_timeout 
Lower timeout for make_membership_event
 2016-04-18 15:08:05 +01:00
Erik Johnston
3727a15764 Merge pull request #732 from matrix-org/erikj/login 
Simplify _check_password
 2016-04-18 15:07:57 +01:00
Matthew Hodgson
aaabbd3e9e explicitly pass in the charset from Content-Type to lxml to fix cyrillic woes better 2016-04-15 14:32:25 +01:00
Matthew Hodgson
84f9cac4d0 fix cyrillic URL previews by hardcoding all page decoding to UTF-8 for now, rather than relying on lxml's heuristics which seem to get it wrong 2016-04-15 13:20:08 +01:00
Erik Johnston
914f1eafac Lower timeout for make_membership_event 
Calls to make_membership_event are done in response to client requests,
and so should not be retried over long timeframes.
 2016-04-15 11:22:23 +01:00
Erik Johnston
6fd2f685fe Simplify _check_password 2016-04-15 11:17:18 +01:00
Erik Johnston
cb9c465707 Use SynapseError 504 for Timeout errors 2016-04-15 10:21:32 +01:00
Mark Haines
3c79bdd7a0 Fix check_password rather than inverting the meaning of _check_local_password (#730) 2016-04-14 19:00:21 +01:00
David Baker
a4c56bf67b Merge pull request #729 from matrix-org/dbkr/fix_login_nonexistent_user 
Fix login to error for nonexistent users
 2016-04-14 18:46:45 +01:00
David Baker
4c1b32d7e2 Fix login to error for nonexistent users 
Fixes SYN-680
 2016-04-14 18:28:42 +01:00
Matthew Hodgson
f78b479118 fix urlparse import thinko breaking tiny URLs 2016-04-14 15:23:55 +01:00
Kegan Dougal
83776d6219 Make v2_alpha reg follow the AS API specification 
The spec is clear the key should be 'user' not 'username' and this is indeed
the case for v1. This is not true for v2_alpha though, which is what this
commit is fixing.
 2016-04-14 14:52:26 +01:00
Matthew Hodgson
bd77216d06 comment out 2c838f6459 due to risk of https://en.wikipedia.org/wiki/Billion_laughs attacks - thanks @torhve 2016-04-14 14:39:24 +01:00
Erik Johnston
9ae64c9910 Measure push action generator 2016-04-14 13:42:22 +01:00
Erik Johnston
b42ad359e9 Merge pull request #725 from matrix-org/dbkr/push_only_joined 
Don't push for everyone who ever sent an RR to the room
 2016-04-14 12:05:13 +01:00
David Baker
757e2c79b4 Don't push for everyone who ever sent an RR to the room 2016-04-14 12:02:50 +01:00
Erik Johnston
86e9bbc74e Add missing yield 2016-04-14 11:56:52 +01:00
Erik Johnston
e40f25ebe1 Rename log context 2016-04-14 11:54:14 +01:00
Erik Johnston
2ae91a9e2f Make send_badge private 2016-04-14 11:37:50 +01:00
Erik Johnston
d213d69fe3 Add desc arg 2016-04-14 11:36:23 +01:00
Erik Johnston
56da835eaf Add necessary logging contexts 2016-04-14 11:33:50 +01:00
Erik Johnston
96bcfb29c7 Add index 2016-04-14 11:26:33 +01:00
Erik Johnston
7be1065b8f Add extra Measure 2016-04-14 11:26:15 +01:00
Erik Johnston
a2546b9082 Fix query for get_unread_push_actions_for_user_in_range 2016-04-14 11:08:31 +01:00
Erik Johnston
ceeb5b909f Merge pull request #721 from matrix-org/erikj/spider 
Sanitize the optional dependencies for spider API
 2016-04-14 09:59:29 +01:00
David Baker
43a89cca8e Merge pull request #722 from matrix-org/dbkr/only_unread_event_actions 
Only return unread notifications
 2016-04-13 14:54:26 +01:00
Erik Johnston
f338bf9257 Give install requirements 2016-04-13 14:33:48 +01:00
David Baker
767fc0b739 pep8 2016-04-13 14:23:27 +01:00
David Baker
54d08c8868 Only return unread notifications 
Make get_unread_push_actions_for_user_in_range only return unread event actions, being more true to its name. Done in two separate sql queries to get actions after a read receipt and those in a room wiht no receipt at all. SQL queries by Erik.
 2016-04-13 14:16:45 +01:00
Erik Johnston
5880bc5417 Merge pull request #718 from matrix-org/erikj/public_room_list 
Don't return empty public rooms
 2016-04-13 14:07:26 +01:00
Erik Johnston
f613a3e332 Merge pull request #720 from matrix-org/erikj/auth_chec 
Don't auto log failed auth checks
 2016-04-13 14:07:23 +01:00
Erik Johnston
bfe586843f Add back in helpful description for missing url_preview_ip_range_blacklist 2016-04-13 13:52:57 +01:00
Erik Johnston
d0633e6dbe Sanitize the optional dependencies for spider API 2016-04-13 13:38:09 +01:00
Erik Johnston
0f2ca8cde1 Measure Auth.check 2016-04-13 11:15:59 +01:00
Erik Johnston
c53f9d561e Don't auto log failed auth checks 2016-04-13 11:11:46 +01:00
David Baker
65141161f6 Unused member variable 2016-04-12 16:25:26 +01:00
Erik Johnston
72f454b752 Don't return empty public rooms 2016-04-12 16:06:18 +01:00
David Baker
d33d623f0d Merge pull request #716 from matrix-org/dbkr/get_pushers 
Add get endpoint for pushers
 2016-04-12 14:40:37 +01:00
David Baker
7984ffdc6a Unneccessarywhitespaceisunnecessary 2016-04-12 13:55:57 +01:00
David Baker
c1267d04c5 Oops, forgot the desc. 2016-04-12 13:55:32 +01:00
David Baker
a04c076b7f Make the /set part mandatory 2016-04-12 13:54:41 +01:00
David Baker
44891b4a0a Tidy up get_pusher functions 
Decodes pushers rows on the main thread rather than the db thread and uses _simple_select_list. Also do the same to the function I copied and factor out the duplication into a helper function.
 2016-04-12 13:47:17 +01:00
David Baker
7b39bcdaae Mis-named function 2016-04-12 13:35:08 +01:00
David Baker
d937f342bb Split into separate servlet classes 2016-04-12 13:33:30 +01:00
Erik Johnston
c48465dbaa More comments 2016-04-12 12:48:30 +01:00
Erik Johnston
8be1a37909 More comments 2016-04-12 12:04:19 +01:00
Erik Johnston
d3d0be4167 Don't append to unused list 2016-04-12 11:59:00 +01:00
Erik Johnston
762ada1e07 Add back backfilled parameter that was removed 2016-04-12 11:58:04 +01:00
Erik Johnston
0d3da210f0 Add comment 2016-04-12 11:54:41 +01:00
Erik Johnston
cccf86dd05 Check if we've already backfilled events 2016-04-12 11:19:32 +01:00
David Baker
8a76094965 Add get endpoint for pushers 
As per https://github.com/matrix-org/matrix-doc/pull/308
 2016-04-11 18:00:03 +01:00
Mark Haines
790f5848b2 Fix the rule_id for .m.rule.invite_for_me (#715) 2016-04-11 16:10:39 +01:00
Mark Haines
82d7eea7e3 Move the versionstring code out of app.homeserver into util 2016-04-11 14:57:09 +01:00
David Baker
2547dffccc Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions 
Change pushers to use the event_actions table
 2016-04-11 12:58:55 +01:00
David Baker
9bb041791c Run unsafe proces in a loop until we've caught up 
and wrap unsafe process in a try block
 2016-04-11 12:48:30 +01:00
Erik Johnston
17515bae14 PEP8 2016-04-11 11:02:50 +01:00
Matthew Hodgson
4bd3d25218 Merge pull request #688 from matrix-org/matthew/preview_urls 
URL previewing support
 2016-04-11 10:40:29 +01:00
Matthew Hodgson
5ffacc5e84 fix typos and needless try/except from PR review 2016-04-11 10:39:16 +01:00
Matthew Hodgson
83b2f83da0 actually throw meaningful errors 2016-04-08 21:36:59 +01:00
Mark Haines
b36270b5e1 Fix pep8 warning 2016-04-08 19:52:23 +01:00
Matthew Hodgson
6ff7a79308 move local_media_repository_url_cache.sql to schema v31 2016-04-08 19:09:02 +01:00
Matthew Hodgson
af582b66bb fix typo 2016-04-08 19:08:47 +01:00
Matthew Hodgson
2460d904bd fix error checking for new SQL 2016-04-08 19:04:29 +01:00
Matthew Hodgson
1ccabe2965 more PR feedback 2016-04-08 18:58:08 +01:00
Matthew Hodgson
fb83f6a1fc fix SQL based on PR feedback 2016-04-08 18:55:38 +01:00
Matthew Hodgson
dafef5a688 Add url_preview_enabled config option to turn on/off preview_url endpoint. defaults to off. 
Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered.
Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered.
Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist
Add commentary and generally address PR feedback
 2016-04-08 18:37:15 +01:00
David Baker
d96a070a3a Actually check if we;re processing 2016-04-08 16:49:39 +01:00
David Baker
ed3979df5f Fix invite pushes 
* If the event is an invite event, add the invitee to list of user we run push rules for (if they have a pusher etc)
 * Move invite_for_me to be higher prio than member events otherwise member events matches them
 * Spell override right
 2016-04-08 15:29:59 +01:00
Erik Johnston
79fc4ff6f9 Merge pull request #677 from matrix-org/erikj/dns_cache 
Read from DNS cache if within TTL
 2016-04-08 14:09:56 +01:00
David Baker
7b6d519482 Make sure max stream ordering only increases 2016-04-08 14:08:16 +01:00
David Baker
52d1008661 Unsafe process should call itself if the max has changed 2016-04-08 14:06:54 +01:00
Erik Johnston
96bd8ff57c Merge pull request #707 from matrix-org/markjh/remove_changed_presencelike_data 
changed_presencelike_data isn't observed anywhere so can be removed
 2016-04-08 14:04:54 +01:00
David Baker
ce3fe52498 Comment why unsafe process is unsafe 2016-04-08 14:02:38 +01:00
Mark Haines
7e2f971c08 Remove some unused functions (#711) 
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
 2016-04-08 14:01:56 +01:00
Mark Haines
b9ee5650b0 Move all the wrapper functions for distributor.fire 
Move the functions inside the distributor and import them
where needed. This reduces duplication and makes it possible
for flake8 to detect when the functions aren't used in a
given file.
 2016-04-08 11:01:38 +01:00
Mark Haines
caef337587 changed_presencelike_data isn't observed anywhere in synapse so can be removed 2016-04-08 10:37:19 +01:00
Mark Haines
b4a5002a6e Merge pull request #708 from matrix-org/markjh/remove_collect_presencelike_data 
Call profile handler get_displayname directly
 2016-04-08 09:51:36 +01:00
Mark Haines
86be915cce Call profile handler get_displayname directly rather than using collect_presencelike_data 2016-04-07 18:11:49 +01:00
David Baker
d9f38561c8 Literally a dictionary 2016-04-07 17:45:01 +01:00
David Baker
4836864f56 generate id in the main thread 2016-04-07 17:38:48 +01:00
David Baker
a4a31fa8dc Only pass in what we need 2016-04-07 17:37:19 +01:00
Erik Johnston
f942980c0b Merge pull request #701 from DoubleMalt/ldap-auth 
Add LDAP authentication
 2016-04-07 17:35:28 +01:00
David Baker
3fb35cbd6f Oops, inequality fail 2016-04-07 17:33:37 +01:00
David Baker
15e0f1696f Wrap process in a flag so we don't process whist already processing. 2016-04-07 17:31:08 +01:00
Matthew Hodgson
d6e7333ae4 Merge branch 'develop' into matthew/preview_urls 2016-04-07 17:26:44 +01:00
David Baker
6ec02e9ecf indenting 2016-04-07 17:24:05 +01:00
David Baker
25cd5bb697 defer.gatherResults rather than doing all the pokes in series 2016-04-07 17:22:14 +01:00
David Baker
fa129ce5b5 Add measure blocks 2016-04-07 17:12:29 +01:00
David Baker
e1e042f2a1 Add comments on min_stream_id 
saying that the min stream id won't be completely accurate all the time
 2016-04-07 17:09:36 +01:00
Mark Haines
ceb599e789 Add tests for redactions 2016-04-07 16:52:07 +01:00
Mark Haines
8c82b06904 Merge pull request #704 from matrix-org/markh/slaveIII 
Add tests for get_latest_event_ids_in_room and get_current_state
 2016-04-07 16:49:34 +01:00
David Baker
05d044aac3 pep8 2016-04-07 16:45:38 +01:00
David Baker
2d5c693fd3 Fix port script for changes merged from develop 2016-04-07 16:43:54 +01:00
Mark Haines
57fa1801c3 Add sensible __eq__ operators inside the tests. 
Rather than adding them globally. This limits the changes to only
affect the tests.
 2016-04-07 16:41:37 +01:00
Erik Johnston
a294b04bf0 Merge pull request #700 from matrix-org/erikj/deduplicate_joins 
Deduplicate membership changes
 2016-04-07 16:35:40 +01:00
David Baker
9c99ab4572 Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions 2016-04-07 16:35:22 +01:00
David Baker
d549fdfa22 Remove code that's now been obsoleted or moved elsewhere 2016-04-07 16:31:38 +01:00
Erik Johnston
95ac3078da Rename things 2016-04-07 16:07:16 +01:00
David Baker
92e3071623 Send badge count pushes. 
Also fix bugs with retrying.
 2016-04-07 15:39:53 +01:00
Erik Johnston
ee5aef6c72 Log contexts and squash things together 2016-04-07 15:34:21 +01:00
Erik Johnston
639cd07d6d Add comment 2016-04-07 14:24:12 +01:00
Erik Johnston
af03ecf352 Deduplicate joins 2016-04-07 14:19:02 +01:00
Mark Haines
60ec9793fb Add tests for get_latest_event_ids_in_room and get_current_state 2016-04-07 13:17:56 +01:00
Erik Johnston
a28d066732 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache 2016-04-07 11:11:17 +01:00
Erik Johnston
1ef0365670 Set profile information when joining rooms remotely 2016-04-07 09:42:52 +01:00
Richard van der Hoff
87a30890a3 Merge pull request #699 from matrix-org/rav/show_own_leave_event 
Let users see their own leave events
 2016-04-06 17:57:06 +01:00
Christoph Witzany
ed4d18f516 fix check for failed authentication 2016-04-06 18:30:11 +02:00
Christoph Witzany
9c62fcdb68 remove line 2016-04-06 18:23:46 +02:00
Christoph Witzany
27a0c21c38 make tests for ldap more specific to not be fooled by Mocks 2016-04-06 18:23:46 +02:00
Christoph Witzany
3555a659ec output ldap version for info and to pacify pep8 2016-04-06 18:23:46 +02:00
Christoph Witzany
4c5e8adf8b conditionally import ldap 2016-04-06 18:23:46 +02:00
Christoph Witzany
875ed05bdc fix pep8 2016-04-06 18:23:46 +02:00
Christoph Witzany
67f3a50e9a fix exception handling 2016-04-06 18:23:46 +02:00
Christoph Witzany
afff321e9a code style 2016-04-06 18:23:46 +02:00
Christoph Witzany
8f0e47fae8 cleanup 2016-04-06 18:23:45 +02:00
Christoph Witzany
823b8be4b7 add tls property and twist my head around twisted 2016-04-06 18:23:45 +02:00
Christoph Witzany
92767dd703 add tls property 2016-04-06 18:23:45 +02:00
Christoph Witzany
7b9319b1c8 move LDAP authentication to AuthenticationHandler 2016-04-06 18:23:45 +02:00
Christoph Witzany
3d95405e5f Introduce LDAP authentication 2016-04-06 18:23:45 +02:00
David Baker
0fd1cd2400 pep8 2016-04-06 16:50:47 +01:00
Mark Haines
6bfec56796 Test that room membership is replicated 2016-04-06 16:20:13 +01:00
Mark Haines
e815763b7f Merge pull request #697 from matrix-org/markjh/slaveI 
Add a slaved events store class
 2016-04-06 16:19:25 +01:00
David Baker
7e2c89a37f Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do: 
* Make badges work again
 * Remove old, unused code
 2016-04-06 15:42:15 +01:00
Richard van der Hoff
1e05637e37 Let users see their own leave events 
... otherwise clients get confused.

Fixes https://matrix.org/jira/browse/SYN-662,
https://github.com/vector-im/vector-web/issues/368
 2016-04-06 15:36:19 +01:00
Erik Johnston
b713934b2e Merge pull request #698 from matrix-org/erikj/port_script_fix 
Don't require config to create database
 2016-04-06 14:32:45 +01:00
Mark Haines
75fb9ac1be Add a slaved events store class 
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
 2016-04-06 14:18:35 +01:00
Erik Johnston
8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Mark Haines
7d11f825aa Merge pull request #694 from matrix-org/markjh/caches 
Move _get_cache_dict into the SQLBaseStore
 2016-04-06 13:21:25 +01:00
Mark Haines
87f2dec8d4 Make the cache objects be per instance rather than being global 2016-04-06 13:08:05 +01:00
Mark Haines
a1e0d316ea Move _get_cache_dict into the SQLBaseStore 2016-04-06 13:05:19 +01:00
Mark Haines
2e308a3a38 Merge pull request #692 from matrix-org/markjh/replicate_reshuffle 
Separate generating the replication response...
 2016-04-05 13:23:36 +01:00
Erik Johnston
c2b429ab24 Merge pull request #693 from matrix-org/erikj/backfill_self 
Don't backfill from self
 2016-04-05 13:04:36 +01:00
Erik Johnston
6222ae51ce Don't backfill from self 2016-04-05 12:56:29 +01:00
Erik Johnston
b29f98377d Merge pull request #691 from matrix-org/erikj/member 
Fix stuck invites
 2016-04-05 12:44:39 +01:00
Mark Haines
1d4deff25a Separate generating the replication response... 
from doing the http request parsing to make it easier
to write unit tests for replication.
 2016-04-05 11:23:57 +01:00
Erik Johnston
df727f2126 Fix stuck invites 
If rejecting a remote invite fails with an error response don't fail
the entire request; instead mark the invite as locally rejected.

This fixes the bug where users can get stuck invites which they can
neither accept nor reject.
 2016-04-05 11:13:24 +01:00
Erik Johnston
7a77f8b6d5 Merge pull request #690 from matrix-org/erikj/member 
Store invites in a separate table.
 2016-04-05 09:12:27 +01:00
Erik Johnston
0c53d750e7 Docs and indents 2016-04-04 18:02:48 +01:00
Erik Johnston
92ab45a330 Add upgrade path, rename table 2016-04-04 17:07:43 +01:00
Erik Johnston
3d76b7cb2b Store invites in a separate table. 2016-04-04 16:30:15 +01:00
Erik Johnston
bf14883a04 Merge pull request #689 from matrix-org/erikj/member 
Do checks for memberships before creating events
 2016-04-04 11:56:40 +01:00
Matthew Hodgson
9f7dc2bef7 Merge branch 'develop' into matthew/preview_urls 2016-04-04 00:38:21 +01:00
Matthew Hodgson
cf51c4120e report image size (bytewise) in OG meta 2016-04-03 23:57:05 +01:00
Matthew Hodgson
0834b152fb char encoding 2016-04-03 12:59:27 +01:00
Matthew Hodgson
8b98a7e8c3 pep8 2016-04-03 12:56:29 +01:00
Matthew Hodgson
eab4d462f8 fix etag typing error. fix timestamp typing error 2016-04-03 02:02:46 +01:00
Matthew Hodgson
c3916462f6 rebase all image URLs 2016-04-03 01:33:12 +01:00
Matthew Hodgson
110780b18b remove stale todo 2016-04-03 00:48:31 +01:00
Matthew Hodgson
b09e29a03c Ensure only one download for a given URL is active at a time 2016-04-03 00:47:40 +01:00
Matthew Hodgson
7426c86eb8 add a persistent cache of URL lookups, and fix up the in-memory one to work 2016-04-03 00:31:57 +01:00
Matthew Hodgson
d1b154a10f support gzip compression, and don't pass through error msgs 2016-04-02 03:06:39 +01:00
Matthew Hodgson
9377157961 how was _respond_default_thumbnail ever meant to work? 2016-04-02 02:31:45 +01:00
Matthew Hodgson
2c838f6459 pass back SVGs as their own thumbnails 2016-04-02 02:30:07 +01:00
Matthew Hodgson
5037ee0d37 handle missing dimensions without crashing 2016-04-02 02:29:57 +01:00
Matthew Hodgson
b26e8604f1 make meta comparisons case insensitive 2016-04-02 01:35:44 +01:00
Matthew Hodgson
5fd07da764 refactor calc_og; spider image URLs; fix xpath; add a (broken) expiringcache; loads of other fixes 2016-04-02 00:35:49 +01:00
Erik Johnston
d76d89323c Use computed prev event ids 2016-04-01 17:39:32 +01:00
Erik Johnston
aa82cb38e9 Remove state hack from _create_new_client_event 2016-04-01 16:36:54 +01:00
Mark Haines
89e6839a48 Merge pull request #686 from matrix-org/markjh/doc_strings 
Use google style doc strings.
 2016-04-01 16:20:09 +01:00
Erik Johnston
c906f30661 Do checks for memberships before creating events 2016-04-01 16:17:32 +01:00
Mark Haines
2a37467fa1 Use google style doc strings. 
pycharm supports them so there is no need to use the other format.

Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
 2016-04-01 16:12:07 +01:00
Mark Haines
f2b916534b Merge pull request #684 from matrix-org/markjh/backfill_id_gen 
Use a stream id generator for backfilled ids
 2016-04-01 15:13:14 +01:00
Mark Haines
9bc5b4c663 Assert that the step != 0 2016-04-01 15:08:20 +01:00
Mark Haines
35b5c4ba1b use google style doc strings 2016-04-01 15:07:01 +01:00
Erik Johnston
3f4eb4c924 Comment 2016-04-01 14:15:27 +01:00
Erik Johnston
8d73cd502b Add concurrently_execute function 2016-04-01 14:06:00 +01:00
Mark Haines
a2866e2e6a Rename direction to step, apply checks consistently 2016-04-01 13:50:54 +01:00
Mark Haines
e36bfbab38 Use a stream id generator for backfilled ids 2016-04-01 13:29:05 +01:00
Erik Johnston
35bb465b86 Filter rooms list before chunking 2016-04-01 13:14:53 +01:00
Mark Haines
c42f46ab7d Merge pull request #682 from matrix-org/markjh/fix_invalidate 
Fix the invalidation of the names and aliases cache
 2016-04-01 10:52:29 +01:00
Mark Haines
7753fc6570 Fix the invalidation of the names and aliases cache 2016-04-01 10:34:51 +01:00
Matthew Hodgson
c60b751694 fix assorted redirect, unicode and screenscraping bugs 2016-04-01 02:17:48 +01:00
Matthew Hodgson
683e564815 handle spidered relative images correctly 2016-03-31 23:52:58 +01:00
Mark Haines
dc4c1579d4 Remove outlier parameter from compute_event_context 
Use event.internal_metadata.is_outlier instead.
 2016-03-31 15:32:24 +01:00
Mark Haines
03e406eefc Merge pull request #680 from matrix-org/markjh/remove_is_new_state 
Remove the is_new_state argument to persist event.
 2016-03-31 15:14:48 +01:00
Matthew Hodgson
72550c3803 prevent choking on invalid utf-8, and handle image thumbnailing smarter 2016-03-31 15:14:14 +01:00
Mark Haines
5d06929169 Move the check for backfilled outside the for loop 2016-03-31 15:09:09 +01:00
Mark Haines
76503f95ed Remove the is_new_state argument to persist event. 
Move the checks for whether an event is new state inside persist
event itself.

This was harder than expected because there wasn't enough information
passed to persist event to correctly handle invites from remote servers
for new rooms.
 2016-03-31 15:00:42 +01:00
Erik Johnston
fe95943305 Merge pull request #679 from matrix-org/erikj/member 
Split out RoomMemberHandler
 2016-03-31 14:45:57 +01:00
Matthew Hodgson
bb9a2ca87c synthesise basig OG metadata from pages lacking it 2016-03-31 14:15:09 +01:00
Erik Johnston
d35780eda0 Split out RoomMemberHandler 2016-03-31 13:08:45 +01:00
Matthew Hodgson
0d3d7de6fc sync in changes from matrixfederationclient 2016-03-31 12:42:27 +01:00
Mark Haines
62e395f0e3 Merge pull request #676 from matrix-org/markjh/replicate_stateIII 
Add replication streams for ex outliers and current state resets
 2016-03-31 11:20:57 +01:00
Erik Johnston
5260db7663 Line length 2016-03-31 10:49:27 +01:00
Mark Haines
2ec5426035 Use a namedtuple rather than tuple unpacking 2016-03-31 10:33:02 +01:00
Erik Johnston
f9d3665c88 Allow clock to be passed in to func 2016-03-31 10:23:48 +01:00
David Baker
c27c51484a Don't ignore the obey overlay if the rule has an enabled attribute of False 
Fixes https://github.com/vector-im/vector-web/issues/1244
 2016-03-31 10:12:31 +01:00
Erik Johnston
f699b8f997 Read from DNS cache if within TTL 2016-03-31 10:04:28 +01:00
Matthew Hodgson
a8a5dd3b44 handle requests with missing content-length headers (e.g. YouTube) 2016-03-31 01:55:21 +01:00
Matthew Hodgson
9113316b0e typo 2016-03-30 17:29:42 +01:00
Mark Haines
1fbb094c6f Add replication streams for ex outliers and current state resets 2016-03-30 17:19:56 +01:00
Mark Haines
8b8052909f return the state_group for backfill 2016-03-30 16:20:07 +01:00
Mark Haines
61407986b4 Add a entry to current_state_resets table when the current state is reset 2016-03-30 16:18:46 +01:00
Mark Haines
31a9eceda5 Add a replication stream for state groups 2016-03-30 16:01:58 +01:00
Mark Haines
fc66df1e60 Merge pull request #674 from matrix-org/markjh/replicate_state 
Use a stream id generator to assign state group ids
 2016-03-30 15:58:49 +01:00
Erik Johnston
178c9fb200 Merge pull request #673 from matrix-org/erikj/forget 
Require user to have left room to forget room
 2016-03-30 15:55:24 +01:00
Erik Johnston
73b6bf4629 Only forget room if you were in the room 2016-03-30 15:09:18 +01:00
Erik Johnston
08a8514b7a Remove spurious comment 2016-03-30 15:05:33 +01:00
Erik Johnston
d24662b88a Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-03-30 14:41:31 +01:00
Mark Haines
1e25f62ee6 Use a stream id generator to assign state group ids 2016-03-30 12:55:02 +01:00
Erik Johnston
e7aaa7c61e Bump version and changelog 2016-03-30 12:35:25 +01:00
Erik Johnston
fddb6fddc1 Require user to have left room to forget room 
This dramatically simplifies the forget API code - in particular it no
longer generates a leave event.
 2016-03-30 11:03:00 +01:00
Niklas Riekenbrauck
3f9948a069 Add JWT support 2016-03-29 14:36:36 +02:00
Matthew Hodgson
ae5831d303 fix bugs 2016-03-29 03:32:55 +01:00
Matthew Hodgson
721b2bfa85 implement redirects 2016-03-29 03:32:52 +01:00
Matthew Hodgson
19038582d3 debug 2016-03-29 03:14:16 +01:00
Matthew Hodgson
64b4aead15 make it work 2016-03-29 03:13:25 +01:00
Matthew Hodgson
dd4287ca5d make it build 2016-03-29 02:07:57 +01:00
Matthew Hodgson
e0c2490a14 Merge branch 'develop' into matthew/preview_urls 2016-03-29 01:20:25 +01:00
Matthew Hodgson
ec0cf996c9 typo 2016-03-29 01:20:14 +01:00
Matthew Hodgson
d9d48aad2d Merge branch 'develop' into matthew/preview_urls 2016-03-27 22:54:42 +01:00
Matthew Hodgson
adafa24b0a typo 2016-03-25 23:38:19 +00:00
Mark Haines
77cba688ed Fix typo 2016-03-24 18:02:37 +00:00
Mark Haines
54a546091a Add a response cache for getting the public room list 2016-03-24 18:02:10 +00:00
Mark Haines
191c7bef6b Deduplicate identical /sync requests 2016-03-24 17:47:31 +00:00
David Baker
3b554bda26 Never notify for member events. This fixes https://github.com/vector-im/vector-web/issues/828 2016-03-24 13:19:39 +00:00
Erik Johnston
15844040c2 Bump version and changelog 2016-03-23 16:57:41 +00:00
Erik Johnston
7a3815b372 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.14.0 2016-03-23 16:55:29 +00:00
Erik Johnston
8122ad7bab Simplify intern_dict 2016-03-23 16:41:54 +00:00
Erik Johnston
2f0180b09e Don't bother interning keys that are already interned 2016-03-23 16:29:46 +00:00
Erik Johnston
acdfef7b14 Intern all the things 2016-03-23 16:25:54 +00:00
Erik Johnston
f96526ffc2 Intern sender, event_id and room_id in events 2016-03-23 15:04:11 +00:00
Erik Johnston
fe9794706a Intern type and state_key on events 2016-03-23 14:58:08 +00:00
Erik Johnston
75daede92f String intern 2016-03-23 14:53:53 +00:00
Erik Johnston
fbdeb1778d Merge pull request #664 from matrix-org/erikj/public_room_list 
Don't require alias in public room list.
 2016-03-23 14:42:01 +00:00
Erik Johnston
b275765545 Comment about weird SQL 2016-03-23 14:15:32 +00:00
Erik Johnston
0c1a27b787 SQLite and postgres doesn't share a true literal 2016-03-23 14:10:49 +00:00
Erik Johnston
84afeb41f3 Ensure all old public rooms have aliases 2016-03-23 13:59:34 +00:00
Erik Johnston
b2802a1351 Ensure published rooms have public join rules 2016-03-23 13:59:31 +00:00
Erik Johnston
0677fc1c4e Comment 2016-03-23 13:25:22 +00:00
Erik Johnston
2749da542c Merge pull request #663 from matrix-org/erikj/invite_for_user 
Make get_invites return RoomsForUser
 2016-03-23 13:19:26 +00:00
Erik Johnston
e14baa7a3b Merge pull request #665 from matrix-org/erikj/dont_cache_events 
Only cache events in the event cache
 2016-03-23 13:19:12 +00:00
Erik Johnston
d87a846ebc Don't cache events in get_recent_events_for_room 2016-03-23 11:42:50 +00:00
Erik Johnston
8b0dfc9fc4 Don't cache events in get_current_state_for_key 2016-03-23 11:42:17 +00:00
Erik Johnston
34473a9c7f Don't require alias in public room list. 
Rooms now no longer require an alias to be published.

Also, changes the way we pull out state of each room to not require
fetching all state events.
 2016-03-23 10:42:19 +00:00
Erik Johnston
b6507869cd Make get_invites return RoomsForUser 2016-03-23 10:32:10 +00:00
Erik Johnston
9e2e994395 Reduce cache size 2016-03-23 09:28:07 +00:00
Erik Johnston
d531ebcb57 Key StateHandler._state_cache off of state groups 2016-03-22 18:02:36 +00:00
Erik Johnston
c4a8cbd15a Make LruCache use a dedicated _Node class 2016-03-22 16:06:21 +00:00
Erik Johnston
99f929f36b Make StateHandler._state_cache only store event_ids. 2016-03-22 16:06:04 +00:00
Erik Johnston
d787e41b20 Measure StateHandler._resolve_events 2016-03-22 14:44:48 +00:00
Erik Johnston
6cf0ba1466 Bump get_unread_event_push_actions_by_room_for_user cache 2016-03-22 14:18:21 +00:00
Erik Johnston
76d18a5776 Bump get_aliases_for_room cache 2016-03-22 14:08:13 +00:00
Mark Haines
cd9ba1ed89 Merge pull request #661 from matrix-org/markjh/member_count 
Fix membership count for BulkPushRuleEvaluator
 2016-03-22 13:59:56 +00:00
Mark Haines
5defb25ac6 Use get_users_in_room to count the number of room members rather than using read_receipts 2016-03-22 13:52:45 +00:00
Erik Johnston
fa2f96c2e3 Merge pull request #660 from matrix-org/erikj/state_cache 
Don't cache events in _state_group_cache
 2016-03-22 13:12:06 +00:00
Erik Johnston
f93304e77f Merge pull request #659 from matrix-org/erikj/state_cache_factor 
Make stateGroupCache honour CACHE_SIZE_FACTOR
 2016-03-22 13:12:01 +00:00
Erik Johnston
2c86187a1b Don't cache events in _state_group_cache 
Instead, simply cache the event ids, relying on the event cache to cache
the actual events.

The problem was that while the state groups cache was limited in the
number of groups it could hold, each individual group could consist of
thousands of events.
 2016-03-22 12:00:09 +00:00
Erik Johnston
d6ac752538 Merge pull request #657 from matrix-org/erikj/roomlist 
Add published room list edit API
 2016-03-22 11:57:39 +00:00
Erik Johnston
97785bfc0f Doc string 2016-03-22 10:41:44 +00:00
Erik Johnston
b591277620 Make stateGroupCache honour CACHE_SIZE_FACTOR 2016-03-22 10:32:50 +00:00
Matthew Hodgson
d3654694d0 an invalide is something else... 2016-03-22 00:52:31 +00:00
Mark Haines
5244c0b48e Remove unused backfilled parameter from persist_event 2016-03-21 18:06:08 +00:00
Erik Johnston
3e7fac0d56 Add published room list edit API 2016-03-21 15:06:07 +00:00
Mark Haines
58f8226c7f remove unused current_state variable from on_receive_pdu 2016-03-21 14:20:34 +00:00
Erik Johnston
9adf0e92bc Catch exceptions from EDU handling 2016-03-18 15:12:50 +00:00
Erik Johnston
1660145a08 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/edu_yield 2016-03-18 15:02:43 +00:00
Erik Johnston
8595ff7842 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/get_event 2016-03-18 14:47:19 +00:00
Erik Johnston
58e207cd77 Don't assume existence of event_id in __str__ 2016-03-18 14:31:44 +00:00
Erik Johnston
67ed8065db Dedupe requested event list in _get_events 2016-03-18 14:31:31 +00:00
Erik Johnston
916227b4df Merge pull request #652 from matrix-org/erikj/delete_alias 
Update aliases event after deletion
 2016-03-18 14:02:46 +00:00
Erik Johnston
3c5f25507b Yield on EDU handling 2016-03-18 13:55:16 +00:00
Erik Johnston
56aa4e7a9a Check canonical alias event exists 2016-03-17 15:24:19 +00:00
David Baker
384ee6eafb Merge pull request #650 from matrix-org/dbkr/register_idempotent_with_username 
Make registration idempotent, part 2
 2016-03-17 14:34:08 +00:00
Mark Haines
7a38612620 Remove another unused function from presence 2016-03-17 11:54:19 +00:00
Erik Johnston
2cd9260500 Update aliases event after deletion 
Attempt to update the appropriate `m.room.aliases` event after deleting
an alias. This may fail due to the deleter not being in the room.

Will also check if the canonical alias of the event is set to the
deleted alias, and if so will attempt to delete it.
 2016-03-17 11:42:00 +00:00
Mark Haines
673c96ce97 Remove dead code left over from presence changes 2016-03-17 11:03:47 +00:00
David Baker
5670205e2a remove debug logging 2016-03-16 19:49:42 +00:00
David Baker
f984decd66 Unused import 2016-03-16 19:40:48 +00:00
David Baker
a7daa5ae13 Make registration idempotent, part 2: be idempotent if the client specifies a username. 2016-03-16 19:36:57 +00:00
David Baker
b58d10a875 pep8 2016-03-16 16:22:20 +00:00
David Baker
3ee7d7dc7f time_msec() 2016-03-16 16:18:52 +00:00
David Baker
3176aebf9d string with symbols is a bit too symboly. 2016-03-16 15:55:49 +00:00
David Baker
9671e6750c Replace other time.time(). 2016-03-16 15:51:28 +00:00
David Baker
742b6c6d15 Use hs get_clock instead of time.time() 2016-03-16 15:42:35 +00:00
David Baker
f5e90422f5 take extra return val from check_auth in account too 2016-03-16 14:33:19 +00:00
David Baker
99797947aa pep8 & remove debug logging 2016-03-16 12:51:34 +00:00
David Baker
c12b9d719a Make registration idempotent: if you specify the same session, make it give you an access token for the user that was registered on previous uses of that session. Tweak the UI auth layer to not delete sessions when their auth has completed and hence expire themn so they don't hang around until server restart. Allow server-side data to be associated with UI auth sessions. 2016-03-16 11:56:24 +00:00
Mark Haines
ba660ecde2 Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id) 2016-03-16 10:35:00 +00:00
Mark Haines
ee32d622ce Fix a couple of errors when deleting pushers 2016-03-15 17:47:36 +00:00
Mark Haines
6df1c79c22 Merge branch 'develop' into markjh/pushers_stream 2016-03-15 17:44:39 +00:00
Mark Haines
12904932c4 Hook up adding a pusher to the notifier for replication. 2016-03-15 17:42:03 +00:00
Mark Haines
b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Erik Johnston
e5f0e58931 Remove needless PreserveLoggingContext 2016-03-15 13:48:40 +00:00
Erik Johnston
9e982750ee Persist rejection of invites over federation 2016-03-15 13:24:31 +00:00
Erik Johnston
13e29a697c Bump version and changelog 2016-03-15 09:43:32 +00:00
Erik Johnston
590fbbef03 Add config to create guest account on 3pid invite 
Currently, when a 3pid invite request is sent to an identity server, it
includes a provisioned guest access token. This allows the link in the,
say, invite email to include the guest access token ensuring that the
same account is used each time the link is clicked.

This flow has a number of flaws, including when using different servers
or servers that have guest access disabled.

For now, we keep this implementation but hide it behind a config option
until a better flow is implemented.
 2016-03-14 15:50:40 +00:00
Erik Johnston
a547e2df85 Return list, not generator. 2016-03-14 15:30:19 +00:00
Mark Haines
398cd1edfb Fix regression where synapse checked whether push rules were valid JSON before the compatibility hack that handled clients sending invalid JSON 2016-03-14 14:16:41 +00:00
Erik Johnston
494d0c8e02 Merge pull request #642 from matrix-org/erikj/logout 
Implement logout
 2016-03-11 20:16:25 +00:00
Erik Johnston
15122da0e2 Thats not how transactions work. 2016-03-11 16:45:27 +00:00
Mark Haines
e9c1cabac2 Use parse_json_object_from_request to parse JSON out of request bodies 2016-03-11 16:41:03 +00:00
Erik Johnston
b13035cc91 Implement logout 2016-03-11 16:27:50 +00:00
Erik Johnston
c081228439 Fix SQL statement 2016-03-11 15:09:17 +00:00
Erik Johnston
b5afe6bc38 Merge pull request #641 from matrix-org/dbkr/fix_change_password 
Fix logout on password change
 2016-03-11 14:48:20 +00:00
David Baker
2dee03aee5 more pep8 2016-03-11 14:38:23 +00:00
David Baker
af59826a2f Make select more sensible when dseleting access tokens, rename pusher deletion to match access token deletion and make exception arg optional. 2016-03-11 14:34:09 +00:00
David Baker
f523177850 Delete old, unused methods and rename new one to just be user_delete_access_tokens with an except_token_ids argument doing what it says on the tin. 2016-03-11 14:29:01 +00:00
David Baker
57c444b3ad Dear PyCharm, please indent sensibly for me. Thx. 2016-03-11 14:25:05 +00:00
Erik Johnston
d5fda6e3b0 Merge pull request #640 from matrix-org/erikj/keyclient_host 
Make key client send a Host header
 2016-03-11 13:32:59 +00:00
Erik Johnston
58443a022d Merge pull request #635 from matrix-org/erikj/sync_order 
Use topological orders for initial sync timeline
 2016-03-11 13:17:49 +00:00
David Baker
aa11db5f11 Fix cache invalidation so deleting access tokens (which we did when changing password) actually takes effect without HS restart. Reinstate the code to avoid logging out the session that changed the password, removed in 415c2f0549 2016-03-11 13:14:18 +00:00
Erik Johnston
2e2be463f8 Make key client send a Host header 2016-03-11 10:29:05 +00:00
Daniel Wagner-Hall
465605d616 Store appservice ID on register 2016-03-10 15:58:22 +00:00
Erik Johnston
9669a99d1a Update users table in a batched manner 2016-03-10 15:12:19 +00:00
blide
40160e24ab Register endpoint returns refresh_token 
Guest registration still doesn't return refresh_token
 2016-03-10 10:29:19 +03:00
Erik Johnston
8a88684736 Add comment 2016-03-09 16:51:22 +00:00
Erik Johnston
af2fe6110c Return the correct token form 2016-03-09 16:11:53 +00:00
Erik Johnston
3ecaabc7fd Use topological orders for initial sync timeline 2016-03-09 15:45:34 +00:00
Erik Johnston
1309b8ca97 Merge pull request #634 from matrix-org/erikj/pin_saml2_version 
Pin pysaml2 version to 3.x
 2016-03-09 11:57:33 +00:00
Erik Johnston
07cf96ebf7 Pin pysaml2 version to 3.x 
This is due to the fact that `from saml2 import config` fails in version
4.x
 2016-03-09 11:54:56 +00:00
Mark Haines
b7dbe5147a Add a parse_json_object function 
to deduplicate all the copy+pasted _parse_json functions. Also document
the parse_.* functions.
 2016-03-09 11:26:26 +00:00
Erik Johnston
158a322e82 Ensure integer is an integer 2016-03-09 10:20:48 +00:00
Mark Haines
5f5817ab05 Merge pull request #632 from matrix-org/markjh/py3v2 
Fix relative imports so they work in both py3 and py27
 2016-03-08 18:11:10 +00:00
Daniel Wagner-Hall
3b97797c8d Merge branch 'develop' into daniel/ick 2016-03-08 17:35:09 +00:00
Daniel Wagner-Hall
edca2d9891 Idempotent-ise schema update script 
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
 2016-03-08 17:32:29 +00:00
David Baker
c00f4e48ba Merge pull request #630 from matrix-org/dbkr/post_urlencoded_encode_params 
Encode unicode objects given to post_urlencode*
 2016-03-08 13:51:49 +00:00
Mark Haines
7076082ae6 Fix relative imports so they work in both py3 and py27 2016-03-08 11:45:50 +00:00
Mark Haines
ea72bd9600 Merge pull request #631 from matrix-org/markjh/py3v1 
Use syntax that's valid on both py2.7 and py3
 2016-03-08 11:08:20 +00:00
David Baker
9a3c80a348 pep8 2016-03-08 10:09:07 +00:00
David Baker
7bcee4733a Encode unicode objects given to post_urlencode* otherwise urllib.urlencode chokes. 2016-03-08 10:04:38 +00:00
Mark Haines
239badea9b Use syntax that works on both py2.7 and py3 2016-03-07 20:13:10 +00:00
David Baker
874fd43257 Send the user ID matching the guest access token, since there is no Matrix API to discover what user ID an access token is for. 2016-03-07 17:13:56 +00:00
Erik Johnston
80916e6884 Merge pull request #626 from matrix-org/erikj/visibility 
Send history visibility on boundary changes
 2016-03-07 11:56:07 +00:00
Erik Johnston
2ab0bf4b97 Send history visibility on boundary changes 2016-03-04 16:54:32 +00:00
Mark Haines
b7a3be693b Merge pull request #618 from matrix-org/markjh/pushrule_stream 
Add a stream for push rule updates
 2016-03-04 16:35:08 +00:00
Erik Johnston
beebc0a40f Merge pull request #625 from matrix-org/erikj/kick_ban_sync 
Always include kicks and bans in full /sync
 2016-03-04 16:33:52 +00:00
Mark Haines
9848b54cac Prefill from the correct stream 2016-03-04 16:20:22 +00:00
Mark Haines
deda48068c prefill the push rules stream change cache 2016-03-04 16:19:42 +00:00
Mark Haines
ebcbb23226 s/stream_ordering/event_stream_ordering/ in push 2016-03-04 16:15:23 +00:00
Mark Haines
7e9fc9b6af /FNARG/d 2016-03-04 15:54:09 +00:00
Erik Johnston
125f674eae Always include kicks and bans in full /sync 2016-03-04 15:27:55 +00:00
Erik Johnston
13cbd31040 Spelling 2016-03-04 15:22:39 +00:00
Mark Haines
ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Mark Haines
1b4f4a936f Hook up the push rules stream to account_data in /sync 2016-03-04 14:44:01 +00:00
Erik Johnston
ed61a49169 Add profile information to invites 2016-03-04 14:35:02 +00:00
Erik Johnston
389d558a3b Merge pull request #598 from Rugvip/invite-state 
config,handlers/_base: added homeserver config for what state is included in a room invite
 2016-03-04 09:58:52 +00:00
Erik Johnston
44b084a75e Merge pull request #596 from Rugvip/create 
handlers/_base: don't allow room create event to be changed
 2016-03-04 09:43:20 +00:00
Patrik Oldsberg
5fc59f009c config,handlers/_base: added homeserver config for what state is included in a room invite 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-03-04 10:43:17 +01:00
Erik Johnston
ce82b9e48f Merge pull request #610 from Rugvip/unique-user 
handlers/register: make sure another user id is generated when a collision occurs
 2016-03-04 09:40:20 +00:00
Richard van der Hoff
09b1d98070 Merge pull request #623 from matrix-org/rav/fix_createroom_race 
Make sure we add all invited members before returning from createRoom
 2016-03-04 09:29:15 +00:00
Erik Johnston
dd463e246d Merge pull request #614 from matrix-org/erikj/alias_delete 
Allow alias creators to delete aliases
 2016-03-04 09:02:33 +00:00
Richard van der Hoff
a92b4ea76f Make sure we add all invited members before returning from createRoom 
add a missing yield.
 2016-03-04 00:06:03 +00:00
Richard van der Hoff
a85179aff3 Merge remote-tracking branch 'origin/develop' into rav/SYN-642 2016-03-03 19:05:54 +00:00
Daniel Wagner-Hall
0b3083c75b Merge pull request #621 from matrix-org/daniel/ratelimiting 
Pass whole requester to ratelimiting
 2016-03-03 17:02:48 +00:00
Daniel Wagner-Hall
b4022cc487 Pass whole requester to ratelimiting 
This will enable more detailed decisions
 2016-03-03 16:43:42 +00:00
Erik Johnston
5c90451ea0 Merge pull request #619 from matrix-org/dbkr/dont_use_checkpw 
Stop using checkpw as it seems to have vanished from bcrypt.
 2016-03-03 16:25:41 +00:00
Mark Haines
3406eba4ef Move the code for formatting push rules into a separate function 2016-03-03 16:11:59 +00:00
Mark Haines
ddf9e7b302 Hook up the push rules to the notifier 2016-03-03 14:57:45 +00:00
Daniel Wagner-Hall
95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids 
Mark AS users with their AS's ID
 2016-03-03 13:56:28 +00:00
Richard van der Hoff
79f34bdbc2 Merge branch 'develop' into rav/SYN-642 2016-03-03 11:39:25 +00:00
Richard van der Hoff
74cd80e530 Fix typo 2016-03-03 10:28:58 +00:00
David Baker
ff8b87118d Stop using checkpw as it seems to have vanished from bcrypt. Use bcrypt.hashpw(password, hashed) == hashed as per the bcrypt README. 2016-03-02 18:06:45 +00:00
Mark Haines
2223204eba Hook push rules up to the replication API 2016-03-02 17:26:20 +00:00
Richard van der Hoff
fc1f932cc0 Move arg default to the start of the function 
Also don't overwrite the list that gets passed in.
 2016-03-02 16:44:14 +00:00
Matthew Hodgson
c0147f86a1 Merge pull request #616 from matrix-org/matthew/800x600 
add 800x600 thumbnails to make vector look prettier (and anyone else …
 2016-03-02 16:02:26 +00:00
Matthew Hodgson
47c361d2f8 add 800x600 thumbnails to make vector look prettier (and anyone else who likes big thumbnails) 2016-03-02 15:57:54 +00:00
Richard van der Hoff
863d3f26b3 fix pyflakes quibble 2016-03-02 15:52:50 +00:00
Richard van der Hoff
9ff940a0ef Address review comments 2016-03-02 15:40:30 +00:00
Erik Johnston
2a78dac60d Merge pull request #612 from matrix-org/erikj/cache_size 
Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
 2016-03-02 14:36:10 +00:00
Mark Haines
a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Richard van der Hoff
05ea111c47 Fix pyflakes warning 2016-03-01 17:45:24 +00:00
Richard van der Hoff
8a1d3b86af Handle rejections of invites from local users locally 
Slightly hacky fix to SYN-642, which avoids the federation codepath when trying
to reject invites from local users.
 2016-03-01 17:27:22 +00:00
Mark Haines
a612ce6659 Merge pull request #489 from matrix-org/markjh/replication 
Add a /replication API for extracting the updates that happened on synapse.
 2016-03-01 15:08:24 +00:00
Mark Haines
d50ca1b1ed Merge pull request #613 from matrix-org/markjh/yield 
Load the current id in the IdGenerator constructor
 2016-03-01 14:54:29 +00:00
Mark Haines
60a0f81c7a Add a /replication API for extracting the updates that happened on 
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
 2016-03-01 14:49:41 +00:00
Erik Johnston
f9af8962f8 Allow alias creators to delete aliases 2016-03-01 14:46:31 +00:00
Mark Haines
54172924c8 Load the current id in the IdGenerator constructor 
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.

It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
 2016-03-01 14:32:56 +00:00
Erik Johnston
374f9b2f07 Limit stream change cache size too 2016-03-01 13:30:15 +00:00
Erik Johnston
ce2cdced61 Move cache size fiddling to descriptors only. Fix tests 2016-03-01 13:21:46 +00:00
Erik Johnston
910fc0f28f Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1 2016-03-01 12:56:39 +00:00
Erik Johnston
72165e5b77 Reraise exception 2016-03-01 11:00:10 +00:00
Erik Johnston
ff2d7551c7 Correct cache miss detection 2016-03-01 10:59:17 +00:00
Erik Johnston
903fb34b39 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/expiring_cache_size 2016-03-01 09:43:27 +00:00
Patrik Oldsberg
9c48f1ed22 handlers/register: make sure another user id is generated when a collision occurs 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-02-29 23:12:37 +01:00
David Baker
bfdcc7b9b6 Merge pull request #607 from matrix-org/dbkr/send_inviter_member_event 
Send the invier's member event in room invite state
 2016-02-26 14:32:08 +00:00
Mark Haines
de27f7fc79 Add support for changing the actions for default rules 
See matrix-org/matrix-doc#283

Works by adding dummy rules to the push rules table with a negative priority class and then using those rules to clobber the default rule actions when adding the default rules in ``list_with_base_rules``
 2016-02-26 14:28:19 +00:00
David Baker
413e36b17a Merge remote-tracking branch 'origin/develop' into dbkr/send_inviter_member_event 2016-02-26 13:40:25 +00:00
Gergely Polonkai
87acd8fb07 Fix to appease the PEP8 dragon 2016-02-26 12:05:38 +01:00
Gergely Polonkai
a53774721a Add error codes for malformed/bad JSON in /login 
Signed-off-by: Gergely Polonkai <gergely@polonkaieu>
 2016-02-26 10:22:35 +01:00
David Baker
0f0b011440 Send the invier's member event in room invite state so the invitee has their display name and avatar. 2016-02-25 18:12:09 +00:00
Mark Haines
15c2ac2cac Make sure we return a JSON object when returning the values of specif… 
…ic keys from a push rule
 2016-02-25 15:13:07 +00:00
Mark Haines
4ecfbac85f Merge branch 'develop' into markjh/dead_code 2016-02-24 16:37:46 +00:00
Mark Haines
9892d017b2 Remove unused get_rule_attr method 2016-02-24 16:31:07 +00:00
Daniel Wagner-Hall
33300673b7 Generate guest access token on 3pid invites 
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
 2016-02-24 14:41:25 +00:00
Daniel Wagner-Hall
869580206d Ignore invalid POST bodies when joining rooms 2016-02-24 08:50:28 +00:00
Erik Johnston
278d6c0527 Report size of ExpiringCache 2016-02-23 16:46:21 +00:00
Erik Johnston
e7ab0e0f9f Merge pull request #603 from matrix-org/erikj/presence 
Create a new stream_id per presence update
 2016-02-23 16:14:01 +00:00
Erik Johnston
6451fcd085 Create a new stream_id per presence update 2016-02-23 15:51:39 +00:00
Erik Johnston
b5f77eb12a Check presence token interval is less than 100, rather than the token itself 2016-02-23 15:47:37 +00:00
Erik Johnston
e3e0ac6ec7 Merge pull request #602 from matrix-org/erikj/presence 
Change the way we figure out presence updates for small deltas
 2016-02-23 15:18:41 +00:00
Daniel Wagner-Hall
577951b032 Allow third_party_signed to be specified on /join 2016-02-23 15:11:25 +00:00
Erik Johnston
13f86c3489 Handle get_all_entities_changed returning None 2016-02-23 15:05:37 +00:00
Erik Johnston
6e0209112b Add comments 2016-02-23 14:57:45 +00:00
Erik Johnston
c77dae7a1a Change the way we figure out presence updates for small deltas 2016-02-23 14:54:40 +00:00
Erik Johnston
a7b2ce32f7 Merge pull request #600 from matrix-org/erikj/presence 
Measure PresenceEventSource.get_new_events
 2016-02-23 14:54:26 +00:00
Erik Johnston
02e928cf9b Don't include defer.returnValue in Measure block 2016-02-23 14:52:16 +00:00
Erik Johnston
56a94ccd9e Measure PresenceEventSource.get_new_events 2016-02-23 14:52:16 +00:00
Patrik Oldsberg
baf056bae8 handlers/sync: style fix, line too long 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-02-23 15:49:36 +01:00
Erik Johnston
138c405974 Pick up currently_active across federation 2016-02-23 10:40:11 +00:00
Patrik Oldsberg
210b7d8e00 handlers/_base: don't allow room create event to be changed 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-02-23 00:22:41 +01:00
Patrik Oldsberg
1dcfb201c4 handlers/sync: fix SyncResult not counting account_data change when converting to bool 
This fixes account_data events not triggering an immediate /sync response

Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-02-23 00:21:59 +01:00
Richard van der Hoff
f7e3de02ef Merge pull request #587 from matrix-org/rav/guest_access_after_room_join 
Allow guest users access to messages in rooms they have joined
 2016-02-22 16:36:06 +00:00
Richard van der Hoff
5be3944730 address review comments 
drop commented-out special casing for historyvisibility event
s/he/they/ for users
 2016-02-22 15:27:44 +00:00
Erik Johnston
9e696bd6a3 Remove superfluous call to get_state_at when we already have an event for that stream position 2016-02-22 13:54:46 +00:00
Richard van der Hoff
6c5b147a39 Interpret unknown visibilities the same as shared 2016-02-19 17:11:11 +00:00
Erik Johnston
3dbaeef58c Correctly filter states 2016-02-19 12:27:35 +00:00
Erik Johnston
42ac5f0c1a Only send presence updates to remote hosts if user is ours 2016-02-19 12:19:56 +00:00
Richard van der Hoff
05aee12652 Merge branch 'develop' into rav/guest_access_after_room_join 2016-02-19 12:00:16 +00:00
Erik Johnston
24d9f2c140 Add Measures to presence 2016-02-19 11:50:48 +00:00
Richard van der Hoff
b71ca2b014 Allow guest users access to messages in rooms they have joined 
There should be no difference between guest users and non-guest users in terms
of access to messages. Define the semantics of the is_peeking argument to
filter_events_for_clients (slightly) better; interpret it appropriately, and
set it correctly from /sync.
 2016-02-19 11:41:02 +00:00
Erik Johnston
4a95eb0a12 Add presence metric 2016-02-19 11:32:04 +00:00
Erik Johnston
be799453aa Remove spurious extra metrics 2016-02-19 11:29:33 +00:00
Erik Johnston
ea7786e8ca Merge pull request #586 from matrix-org/erikj/presence 
Fix presence `currently_active`. Add presence metrics.
 2016-02-19 11:26:32 +00:00
Erik Johnston
929cb0ed7d Don't set currently_active for remote presence 2016-02-19 10:58:27 +00:00
Erik Johnston
5f4eca3816 More metrics 2016-02-19 10:21:41 +00:00
Erik Johnston
5614b4dafb Add presence metrics 2016-02-19 09:50:54 +00:00
Erik Johnston
e5ad2e5267 Merge pull request #582 from matrix-org/erikj/presence 
Rewrite presence for performance.
 2016-02-19 09:37:50 +00:00
Erik Johnston
e12ec335a5 "You are not..." 2016-02-18 17:01:53 +00:00
Erik Johnston
220231d8e3 Merge pull request #573 from matrix-org/erikj/sync_fix 
Mitigate against incorrect old state in /sync.
 2016-02-18 16:40:58 +00:00
Erik Johnston
e6c5e3f28a Close cursor 2016-02-18 16:39:28 +00:00
Erik Johnston
42109a62a4 Remove unused param from get_max_token 2016-02-18 16:37:28 +00:00
Erik Johnston
b8cdec92c7 WheelTimer: Don't scan list, use index. 2016-02-18 16:33:07 +00:00
Mark Haines
b9977ea667 Remove dead code for setting device specific rules. 
It wasn't possible to hit the code from the API because of a typo
in parsing the request path. Since no-one was using the feature
we might as well remove the dead code.
 2016-02-18 16:05:13 +00:00
Erik Johnston
48b652bcbe Remove invalid arg. 2016-02-18 14:57:09 +00:00
Erik Johnston
b4796a62ee Add unit test 2016-02-18 11:52:33 +00:00
Daniel Wagner-Hall
f8d21e1431 Review comments 2016-02-18 11:02:14 +00:00
Erik Johnston
fe95f2217c Add stuff pulled from the DB to the cache 2016-02-18 10:26:24 +00:00
Erik Johnston
8351538873 PEP8 2016-02-18 10:12:12 +00:00
Erik Johnston
112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Erik Johnston
b31ec214a5 Remove status_msg when going offline. Don't offline -> online if you send a message 2016-02-18 09:54:08 +00:00
Erik Johnston
114b929f8b Check presence state is a valid one 2016-02-18 09:16:32 +00:00
Erik Johnston
ddca9c56fc Move if statement 2016-02-18 09:11:53 +00:00
Erik Johnston
58371fa263 Comment 2016-02-18 09:09:50 +00:00
Daniel Wagner-Hall
7e90fb6a57 Merge branch 'develop' into daniel/roomcleanupincremental 
Conflicts:
	synapse/rest/client/v1/room.py
 2016-02-17 15:53:59 +00:00
Daniel Wagner-Hall
591af2d074 Some cleanup 
I'm not particularly happy with the "action" switching, but there's no
convenient way to defer the work that needs to happen after it, so... :(
 2016-02-17 15:50:13 +00:00
Erik Johnston
c229c87398 Remove spurious comment 2016-02-17 15:48:29 +00:00
Erik Johnston
e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall
a4e278bfe7 Respond to federated invite with non-empty context 
Currently, we magically perform an extra database hit to find the
inviter, and use this to guess where we should send the event. Instead,
fill in a valid context, so that other callers relying on the context
actually have one.
 2016-02-17 15:25:12 +00:00
Erik Johnston
9e7900da1e Add wheeltimer impl 2016-02-17 14:29:28 +00:00
Erik Johnston
200de16440 Merge pull request #580 from Rugvip/develop 
api/filtering: don't assume that event content will always be a dict
 2016-02-17 12:51:02 +00:00
Patrik Oldsberg
536f949a1a api/filtering: don't assume that event content will always be a dict 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-02-17 12:59:41 +01:00
Patrik Oldsberg
71d5d2c669 client/v1/room: include event_id in response to state event PUT, in accordance with the spec 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-02-17 11:53:43 +01:00
Daniel Wagner-Hall
6605adf669 Some cleanup, some TODOs, more to do 2016-02-16 19:05:02 +00:00
Mark Haines
458782bf67 Fix typo in request validation for adding push rules. 2016-02-16 18:00:30 +00:00
Mark Haines
a9c9868957 Make adding push rules idempotent 
Also remove the **kwargs from the add_push_rule method.

Fixes https://matrix.org/jira/browse/SYN-391
 2016-02-16 15:53:38 +00:00
Daniel Wagner-Hall
d1fb790818 Some cleanup 2016-02-16 14:25:23 +00:00
Daniel Wagner-Hall
1f403325ac Tidy? up room creation event sending 2016-02-16 12:00:50 +00:00
Daniel Wagner-Hall
04686df17a Add comment 2016-02-16 11:52:46 +00:00
Daniel Wagner-Hall
feedaa37fa Merge branch 'develop' into daniel/roomcleanupincremental 
Conflicts:
	synapse/handlers/room.py
 2016-02-16 11:34:48 +00:00
Daniel Wagner-Hall
4bfb32f685 Branch off member and non member sends 
Unclean, needs tidy-up, but works
 2016-02-15 18:21:30 +00:00
Daniel Wagner-Hall
1a2197d7bf Simplify room creation code 2016-02-15 18:19:01 +00:00
Daniel Wagner-Hall
e560045cfd Simplify room creation code 2016-02-15 18:18:39 +00:00
Daniel Wagner-Hall
8168341e9b Use update_membership for profile updates 2016-02-15 17:14:34 +00:00
Daniel Wagner-Hall
1bbb67c452 Use update_membership to kick guests 2016-02-15 16:40:22 +00:00
Daniel Wagner-Hall
150fcde0dc Reuse update_membership from /join 2016-02-15 16:16:03 +00:00
Daniel Wagner-Hall
73e616df2a Inline _do_local_membership_update 2016-02-15 16:02:22 +00:00
Daniel Wagner-Hall
f318d4f2a4 Inline _do_join as it now only has one caller 
Also, consistently apply rate limiting.

Again, ugly, but a step in the right direction.
 2016-02-15 15:57:10 +00:00
Daniel Wagner-Hall
e71095801f Merge implementation of /join by alias or ID 
This code is kind of rough (passing the remote servers down a long
chain), but is a step towards improvement.
 2016-02-15 15:39:16 +00:00
Daniel Wagner-Hall
dbeed36dec Merge some room joining codepaths 
Force joining by alias to go through the send_membership_event checks,
rather than bypassing them straight into _do_join. This is the first of
many stages of cleanup.
 2016-02-15 14:38:27 +00:00
Daniel Wagner-Hall
4de08a4672 Revert "Merge two of the room join codepaths" 
This reverts commit cf81375b94.

It subtly violates a guest joining auth check
 2016-02-12 16:17:24 +00:00
Daniel Wagner-Hall
cf81375b94 Merge two of the room join codepaths 
There's at least one more to merge in.

Side-effects:
 * Stop reporting None as displayname and avatar_url in some cases
 * Joining a room by alias populates guest-ness in join event
 * Remove unspec'd PUT version of /join/<room_id_or_alias> which has not
   been called on matrix.org according to logs
 * Stop recording access_token_id on /join/room_id - currently we don't
   record it on /join/room_alias; I can try to thread it through at some
   point.
 2016-02-12 15:11:49 +00:00
Mark Haines
58c9f20692 Catch the exceptions thrown by twisted when you write to a closed connection 2016-02-12 13:46:59 +00:00
Erik Johnston
ec0f3836ff Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-02-12 11:19:37 +00:00
Erik Johnston
4d54d87c3e Mitigate against incorrect old state in /sync. 
There have been reports from the field that servers occasionally have
incorrect notions of what the old state of a room is. This proves
problematic as /sync relies on a correct old state.
This patch makes /sync specifically include in the 'state' key any
current state events that haven't been correctly included.
 2016-02-12 11:13:06 +00:00
Daniel Wagner-Hall
dc6da63e30 Enforce user_id exclusivity for AS registrations 
This whole set of checks is kind of an ugly mess, which I may clean up
at some point, but for now let's be correct.
 2016-02-11 17:37:38 +00:00
Daniel Wagner-Hall
763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston
2a24f906a9 Bump version and changelog 2016-02-11 16:04:15 +00:00
Erik Johnston
a79af259e9 Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-02-11 16:02:17 +00:00
Erik Johnston
ce14c7a995 Fix SYN-627, events are in incorrect room in /sync 2016-02-11 15:02:56 +00:00
Erik Johnston
1a830b751d Bump version and changelog 2016-02-11 10:53:42 +00:00
Erik Johnston
0eff740523 Return events in correct order for /events 2016-02-11 10:07:27 +00:00
Erik Johnston
a1b7902944 Add some paranoia logging 2016-02-11 09:22:37 +00:00
Matthew Hodgson
103b432c84 0.13.1 2016-02-10 16:35:17 +00:00
Matthew Hodgson
7634687057 Merge branch 'master' of git+ssh://github.com/matrix-org/synapse 2016-02-10 16:27:15 +00:00
Matthew Hodgson
b3ecb96e36 try to bump syweb to 0.6.8 2016-02-10 16:27:12 +00:00
Erik Johnston
6e89e69d08 Bump version and changelog 2016-02-10 14:36:06 +00:00
Erik Johnston
e66d0bd03a Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.13.0 2016-02-10 14:12:48 +00:00
Erik Johnston
5189bfdef4 Batch fetch _get_state_groups_from_groups 2016-02-10 13:24:42 +00:00
Erik Johnston
24f00a6c33 Use _simple_select_many for _get_state_group_for_events 2016-02-10 12:57:50 +00:00
Erik Johnston
8e49892b21 Only calculate initial sync for 10 rooms at a time 
This helps to ensure we don't completely starve other requests.
 2016-02-10 11:42:07 +00:00
Erik Johnston
e557dc80b8 Merge pull request #566 from matrix-org/erikj/logcontext 
Don't bother copying records on parent context
 2016-02-10 11:41:45 +00:00
Erik Johnston
4eb8f9ca8a Remove comment 2016-02-10 11:29:21 +00:00
Erik Johnston
f7ef5c1d57 Merge pull request #568 from matrix-org/erikj/unread_notif 
Atomically persit push actions when we persist the event
 2016-02-10 11:25:32 +00:00
Erik Johnston
00c9ad49df s/parent_context/previous_context/ 2016-02-10 11:25:19 +00:00
Erik Johnston
9777c5f49a Set parent context on instansiation 2016-02-10 11:23:32 +00:00
Erik Johnston
0214745239 Rename functions 2016-02-10 11:09:56 +00:00
Erik Johnston
6ad9586c84 Merge pull request #565 from matrix-org/erikj/macaroon_config 
Derive macaroon_secret_key from signing key.
 2016-02-09 16:34:15 +00:00
Erik Johnston
7b0d846407 Atomically persit push actions when we persist the event 2016-02-09 16:19:15 +00:00
Erik Johnston
f28cc45183 Pass in current state to push action handler 2016-02-09 16:01:40 +00:00
Erik Johnston
78d6c1b5be Change a log from debug to info 2016-02-09 14:44:12 +00:00
Erik Johnston
feb294d552 Remove dead code 2016-02-09 14:32:17 +00:00
Erik Johnston
70a8608749 Invalidate get_last_receipt_event_id_for_user cache 2016-02-09 14:27:29 +00:00
Erik Johnston
eff12e838c Don't load all ephemeral state for a room on every sync 2016-02-09 13:55:59 +00:00
Erik Johnston
82631c5f94 Fix unit tests 2016-02-09 13:50:37 +00:00
Erik Johnston
9ac9b75bc4 Merge branch 'develop' of github.com:matrix-org/synapse into develop 2016-02-09 12:58:10 +00:00
Daniel Wagner-Hall
ebaa999f92 Revert "Reject additional path segments" 
This reverts commit 1d19a5ec0f.

iOS Console is apparently relying on these paths.
 2016-02-09 12:46:52 +00:00
Erik Johnston
6c558ee8bc Measure some /sync related things 2016-02-09 11:31:42 +00:00
Erik Johnston
31a2b892d8 Revert to putting it around the entire block 2016-02-09 11:25:09 +00:00
Erik Johnston
9daa4e2a85 Don't create new logging context 2016-02-09 11:06:19 +00:00
Erik Johnston
3e2fcd67b2 Don't bother copying records on parent context 2016-02-09 10:50:31 +00:00
Erik Johnston
241b71852e Fix bug in util.metrics.Measure 2016-02-09 10:28:13 +00:00
Erik Johnston
97294ef2fd Create new context when measuring 2016-02-09 10:12:00 +00:00
Erik Johnston
549698b1e0 Don't measure across event stream call, as it lasts for a long time. 2016-02-09 09:37:09 +00:00
Erik Johnston
c486b7b41c Change logcontext warns to debug 2016-02-09 09:20:38 +00:00
Erik Johnston
f078ecbc8f Derive macaroon_secret_key from signing key. 
Unfortunately, there are people that are running synapse without a
`macaroon_sercret_key` set. Mandating they set one is a good solution,
except that breaking auto upgrades is annoying.
 2016-02-08 16:35:44 +00:00
Erik Johnston
2bb5f035af Merge pull request #564 from matrix-org/erikj/logcontext 
Fix up logcontexts
 2016-02-08 15:16:16 +00:00
Erik Johnston
cca5c06679 Merge pull request #562 from matrix-org/erikj/push_metric 
Add metrics to pushers
 2016-02-08 14:57:40 +00:00
Erik Johnston
2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Erik Johnston
13e6262659 Add metrics to pushers 2016-02-08 14:26:45 +00:00
Daniel Wagner-Hall
1d19a5ec0f Reject additional path segments 2016-02-08 10:50:55 +00:00
Daniel Wagner-Hall
79a1c0574b Allocate guest user IDs numericcally 
The current random IDs are ugly and confusing when presented in UIs.
This makes them prettier and easier to read.

Also, disable non-automated registration of numeric IDs so that we don't
need to worry so much about people carving out our automated address
space and us needing to keep retrying ID registration.
 2016-02-05 11:22:30 +00:00
Daniel Wagner-Hall
737c4223ef Host /media/r0 as well as /media/v1 2016-02-05 10:47:46 +00:00
Daniel Wagner-Hall
6a9f1209df Error if macaroon key is missing from config 
Currently we store all access tokens in the DB, and fall back to that
check if we can't validate the macaroon, so our fallback works here, but
for guests, their macaroons don't get persisted, so we don't get to
find them in the database. Each restart, we generate a new ephemeral
key, so guests lose access after each server restart.

I tried to fix up the config stuff to be less insane, but gave up, so
instead I bolt on yet another piece of custom one-off insanity.

Also, add some basic tests for config generation and loading.
 2016-02-05 01:58:23 +00:00
David Baker
34dda7cc7f Merge pull request #557 from matrix-org/dbkr/profile_dont_return_null 
Omit keys rather then returning null in profile API
 2016-02-04 15:39:12 +00:00
Erik Johnston
4d36e73230 Actually return something sensible 2016-02-03 16:35:00 +00:00
Erik Johnston
709e09e1c3 Remove old log line 2016-02-03 16:32:20 +00:00
Erik Johnston
aa4af94c69 We return dicts now. 2016-02-03 16:29:32 +00:00
Erik Johnston
b84d59c5f0 Add descriptions 2016-02-03 16:22:35 +00:00
Erik Johnston
33c71c3a4b Preserve log context over when deferring to thread pool in media repo 2016-02-03 16:17:18 +00:00
Erik Johnston
c8e4d5de7f Merge pull request #555 from matrix-org/erikj/logcontext 
Allowing tagging log contexts
 2016-02-03 15:20:00 +00:00
David Baker
156cea5b45 No braces here 2016-02-03 15:04:51 +00:00
Erik Johnston
8450114098 Merge pull request #554 from matrix-org/erikj/event_push 
Change event_push_actions_rm_tokens schema
 2016-02-03 15:02:47 +00:00
David Baker
24277fbb97 Don't return null if profile display name / avatar url isn't set: omit them instead 2016-02-03 14:59:19 +00:00
Daniel Wagner-Hall
66bb255fcd Merge pull request #556 from matrix-org/daniel/config 
Rename config field to reflect yaml name
 2016-02-03 14:55:54 +00:00
Daniel Wagner-Hall
5054806ec1 Rename config field to reflect yaml name 2016-02-03 14:42:01 +00:00
Erik Johnston
d4f72a5bfb Allowing tagging log contexts 2016-02-03 13:52:27 +00:00
Erik Johnston
f8aae79a72 Simplify get_rooms 2016-02-03 13:24:35 +00:00
Erik Johnston
9cd80a7b5c PEP8 2016-02-03 11:52:57 +00:00
Erik Johnston
772b45c745 Remove unused method 2016-02-03 11:43:26 +00:00
Erik Johnston
6f52e90065 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_room_fix 2016-02-03 11:06:29 +00:00
Erik Johnston
771528ab13 Change event_push_actions_rm_tokens schema 2016-02-03 10:50:49 +00:00
Erik Johnston
b32121a5d1 Unused import 2016-02-03 10:30:56 +00:00
Daniel Wagner-Hall
2df6114bc4 Log more diagnostics for unrecognised access tokens 2016-02-02 19:21:49 +00:00
Daniel Wagner-Hall
d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston
8a391e33ae s/get_room_changes_for_user/get_membership_changes_for_user/ 2016-02-02 16:12:10 +00:00
Erik Johnston
477b1ed6cf Fetch events in a separate transaction. 
This has a couple of benefits:

- It reduces the time of transactions, allowing other database requests
  to run.
- Fetching events is given a dedicated database thread, and so can't
  starve other database requests.
 2016-02-02 15:58:14 +00:00
Erik Johnston
04ad93e6fd Merge pull request #545 from matrix-org/erikj/sync 
Move /sync state calculations from rest to handler
 2016-02-02 15:28:43 +00:00
Erik Johnston
65e92eca49 Change the way we do public room list fetching 2016-02-02 15:21:10 +00:00
David Baker
69214ea671 Pass make_guest whne we autogen a user ID 2016-02-02 14:42:31 +00:00
Erik Johnston
b023995538 WARN if we get a topo token instead of stream. 2016-02-02 14:11:14 +00:00
David Baker
793369791a Merge pull request #547 from matrix-org/dbkr/fix_guest_upgrade 
Pull guest access token out of the auth session params
 2016-02-02 09:47:30 +00:00
Erik Johnston
854ca32f10 Comments 2016-02-01 16:52:27 +00:00
David Baker
d7ac861d3b Pull guest access token out of the auth session params, otherwise it will break if you open the email on a different device. 2016-02-01 16:33:19 +00:00
Erik Johnston
89b40b225c Order things correctly 2016-02-01 16:32:46 +00:00
Erik Johnston
4bf448be25 Switch over /events to use per room caches 2016-02-01 16:26:51 +00:00
Erik Johnston
fa48020a52 Move state calculations from rest to handler 2016-02-01 15:59:40 +00:00
Erik Johnston
2d3837bec7 Merge pull request #543 from matrix-org/erikj/sync 
Cache get_room_changes_for_user
 2016-02-01 15:05:06 +00:00
Erik Johnston
ceb6b8680a Only use room_ids if in get_room_events_stream if is_guest 2016-02-01 10:33:52 +00:00
Erik Johnston
d98a9f2583 Don't use before_token. Its wrong. Use actual limit. 2016-01-31 13:31:15 +00:00
Erik Johnston
25c311eaf6 Cache get_room_changes_for_user 2016-01-29 16:52:48 +00:00
Erik Johnston
cc9c97e0dc Invalidate _account_data_stream_cache correctly 2016-01-29 16:41:51 +00:00
Erik Johnston
e70165039c If stream pos is greater then earliest known key and entity hasn't changed, then entity hasn't changed 2016-01-29 16:41:32 +00:00
Erik Johnston
c1de91aca4 Merge pull request #540 from matrix-org/erikj/sync 
Prefill stream change caches
 2016-01-29 15:52:42 +00:00
Erik Johnston
b55b90bfb4 Merge pull request #541 from matrix-org/erikj/fixsomeofpush 
Make /events always return a newer token, if one exists
 2016-01-29 15:45:35 +00:00