Commit Graph

636 Commits

Author SHA1 Message Date
Erik Johnston
05e01f21d7 Remove event fetching from DB threads 2016-06-03 17:22:13 +01:00
Erik Johnston
73c7112433 Change CacheMetrics to be quicker
We change it so that each cache has an individual CacheMetric, instead
of having one global CacheMetric. This means that when a cache tries to
increment a counter it does not need to go through so many indirections.
2016-06-03 11:26:52 +01:00
Mark Haines
56d15a0530 Store the typing users as user_id strings. (#819)
Rather than storing them as UserID objects.
2016-06-02 16:28:54 +01:00
Mark Haines
70599ce925 Allow external processes to mark a user as syncing. (#812)
* Add infrastructure to the presence handler to track sync requests in external processes

* Expire stale entries for dead external processes

* Add an http endpoint for making users as syncing

Add some docstrings and comments.

* Fixes
2016-06-02 15:20:15 +01:00
David Baker
4a10510cd5 Split out the auth handler 2016-06-02 13:31:45 +01:00
David Baker
195254cae8 Inject fake room list handler in tests
Otherwise it tries to start the remote public room list updating looping call which breaks.
2016-06-01 11:14:16 +01:00
Mark Haines
c626fc576a Move the AS handler out of the Handlers object.
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
Negi Fazeli
6fe04ffef2 Fix set profile error with Requester.
Replace flush_user with delete access token due to function removal
Add a new test case for if the user is already registered
2016-05-23 19:50:28 +02:00
Mark Haines
0cb441fedd Move typing handler out of the Handlers object 2016-05-17 15:58:46 +01:00
Mark Haines
425e6b4983 Merge branch 'develop' into markjh/member_cleanup 2016-05-17 10:13:16 +01:00
Mark Haines
816df9f267 get_room_members is unused now 2016-05-16 19:51:43 +01:00
Mark Haines
821306120a Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room 2016-05-16 19:48:07 +01:00
Mark Haines
3b86ecfa79 Move the presence handler out of the Handlers object 2016-05-16 18:56:37 +01:00
Mark Haines
21018c2c13 Merge pull request #783 from matrix-org/markjh/slave_account_data
Add a slaved datastore for account data
2016-05-13 15:56:04 +01:00
Mark Haines
3abab26458 Add a slaved datastore for account data 2016-05-13 15:34:06 +01:00
Negi Fazeli
40aa6e8349 Create user with expiry
- Add unittests for client, api and handler

Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
Mark Haines
c0d8e0eb63 Replicate push actions 2016-04-21 15:25:58 +01:00
Mark Haines
f52dd35ac3 Merge pull request #738 from matrix-org/markjh/slaved_receipts
Add a slaved receipts store
2016-04-19 17:31:59 +01:00
Mark Haines
5bbd424ee0 Add a slaved receipts store 2016-04-19 17:14:08 +01:00
Mark Haines
e99365f601 Replicate get_invited_rooms_for_user 2016-04-19 15:22:14 +01:00
Erik Johnston
eb8619e256 Create log context in Measure if one doesn't exist 2016-04-18 16:08:32 +01:00
Erik Johnston
79fc4ff6f9 Merge pull request #677 from matrix-org/erikj/dns_cache
Read from DNS cache if within TTL
2016-04-08 14:09:56 +01:00
Mark Haines
7e2f971c08 Remove some unused functions (#711)
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
Mark Haines
ceb599e789 Add tests for redactions 2016-04-07 16:52:07 +01:00
Mark Haines
8c82b06904 Merge pull request #704 from matrix-org/markh/slaveIII
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 16:49:34 +01:00
Mark Haines
57fa1801c3 Add sensible __eq__ operators inside the tests.
Rather than adding them globally. This limits the changes to only
affect the tests.
2016-04-07 16:41:37 +01:00
Erik Johnston
af03ecf352 Deduplicate joins 2016-04-07 14:19:02 +01:00
Mark Haines
60ec9793fb Add tests for get_latest_event_ids_in_room and get_current_state 2016-04-07 13:17:56 +01:00
Erik Johnston
a28d066732 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache 2016-04-07 11:11:17 +01:00
Mark Haines
6bfec56796 Test that room membership is replicated 2016-04-06 16:20:13 +01:00
Mark Haines
e815763b7f Merge pull request #697 from matrix-org/markjh/slaveI
Add a slaved events store class
2016-04-06 16:19:25 +01:00
Mark Haines
75fb9ac1be Add a slaved events store class
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
2016-04-06 14:18:35 +01:00
Erik Johnston
8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Erik Johnston
11860637e1 Tests 2016-04-06 10:12:30 +01:00
Erik Johnston
c906f30661 Do checks for memberships before creating events 2016-04-01 16:17:32 +01:00
Erik Johnston
f699b8f997 Read from DNS cache if within TTL 2016-03-31 10:04:28 +01:00
Mark Haines
31a9eceda5 Add a replication stream for state groups 2016-03-30 16:01:58 +01:00
Mark Haines
673c96ce97 Remove dead code left over from presence changes 2016-03-17 11:03:47 +00:00
David Baker
ff7d3dc3a0 Fix tests 2016-03-16 14:25:14 +00:00
Mark Haines
b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Mark Haines
e9c1cabac2 Use parse_json_object_from_request to parse JSON out of request bodies 2016-03-11 16:41:03 +00:00
blide
1be438f2a6 Flake8 fix 2016-03-10 12:13:35 +03:00
blide
40160e24ab Register endpoint returns refresh_token
Guest registration still doesn't return refresh_token
2016-03-10 10:29:19 +03:00
David Baker
316c00936f Fix tests 2016-03-07 17:32:36 +00:00
Mark Haines
3110c37d02 Fix unit tests 2016-03-04 14:48:17 +00:00
Mark Haines
ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Patrik Oldsberg
bb0e82fff1 tests/utils: added room_invite_state_types to test config
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-03-04 10:43:17 +01:00
Daniel Wagner-Hall
b4022cc487 Pass whole requester to ratelimiting
This will enable more detailed decisions
2016-03-03 16:43:42 +00:00
Daniel Wagner-Hall
95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
Mark Haines
2223204eba Hook push rules up to the replication API 2016-03-02 17:26:20 +00:00
Erik Johnston
2a78dac60d Merge pull request #612 from matrix-org/erikj/cache_size
Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
2016-03-02 14:36:10 +00:00
Mark Haines
60a0f81c7a Add a /replication API for extracting the updates that happened on
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
Erik Johnston
ce2cdced61 Move cache size fiddling to descriptors only. Fix tests 2016-03-01 13:21:46 +00:00
Mark Haines
5e2890bd49 Check that the disable_registration config key is handled correctly 2016-02-22 16:01:29 +00:00
Mark Haines
7641a90c34 Add a test for TreeCache.__contains__ 2016-02-22 15:22:38 +00:00
Mark Haines
5c79ef9396 Test Filter.filter_rooms
Also check that the __repr__ method for FilterCollection does something
sensible.
2016-02-19 17:55:28 +00:00
Mark Haines
700487a7c7 Fix flake8 warnings for tests 2016-02-19 15:34:38 +00:00
Erik Johnston
929cb0ed7d Don't set currently_active for remote presence 2016-02-19 10:58:27 +00:00
Erik Johnston
b4796a62ee Add unit test 2016-02-18 11:52:33 +00:00
Erik Johnston
9da9826b85 Remove old tests. 2016-02-18 10:46:16 +00:00
Erik Johnston
e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Erik Johnston
9e7900da1e Add wheeltimer impl 2016-02-17 14:29:28 +00:00
Daniel Wagner-Hall
763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston
e557dc80b8 Merge pull request #566 from matrix-org/erikj/logcontext
Don't bother copying records on parent context
2016-02-10 11:41:45 +00:00
Erik Johnston
78a5482267 Typo 2016-02-09 16:23:11 +00:00
Erik Johnston
e664e9737c Fix test 2016-02-09 14:57:43 +00:00
Erik Johnston
13ba8d878c Fix test 2016-02-09 14:55:21 +00:00
Daniel Wagner-Hall
6a9f1209df Error if macaroon key is missing from config
Currently we store all access tokens in the DB, and fall back to that
check if we can't validate the macaroon, so our fallback works here, but
for guests, their macaroons don't get persisted, so we don't get to
find them in the database. Each restart, we generate a new ephemeral
key, so guests lose access after each server restart.

I tried to fix up the config stuff to be less insane, but gave up, so
instead I bolt on yet another piece of custom one-off insanity.

Also, add some basic tests for config generation and loading.
2016-02-05 01:58:23 +00:00
Daniel Wagner-Hall
66bb255fcd Merge pull request #556 from matrix-org/daniel/config
Rename config field to reflect yaml name
2016-02-03 14:55:54 +00:00
Daniel Wagner-Hall
5054806ec1 Rename config field to reflect yaml name 2016-02-03 14:42:01 +00:00
Erik Johnston
f8aae79a72 Simplify get_rooms 2016-02-03 13:24:35 +00:00
Mark Haines
f2d5ff5bf2 Fix the mock homserver used in the tests 2016-01-29 14:53:14 +00:00
Erik Johnston
b18114e19e Merge pull request #536 from matrix-org/erikj/sync
Make /sync "better".
2016-01-29 13:04:51 +00:00
Erik Johnston
02a9c3be6c Merge pull request #538 from matrix-org/erikj/fix_lru_cache
Fix LruCache. Make TreeCache track its own size.
2016-01-29 11:53:55 +00:00
Erik Johnston
4fce59f274 Add tests 2016-01-29 11:33:11 +00:00
Erik Johnston
35981c8b71 Fix test 2016-01-28 17:20:05 +00:00
Erik Johnston
167d1df699 Merge pull request #534 from matrix-org/erikj/setup
Add a Homeserver.setup method
2016-01-28 14:45:36 +00:00
Erik Johnston
4e7948b47a Allow paginating backwards from stream token 2016-01-28 11:52:34 +00:00
Erik Johnston
61eaa6ec64 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync 2016-01-28 10:50:57 +00:00
Richard van der Hoff
e26390ca46 Merge pull request #535 from matrix-org/rav/paginate_from_stream_token
Make it possible to paginate forwards from stream tokens
2016-01-28 09:49:46 +00:00
Richard van der Hoff
5cba88ea7c Make it possible to paginate forwards from stream tokens
In order that we can fill the gap after a /sync, make it possible to paginate
forwards from a stream token.
2016-01-27 17:42:45 +00:00
Erik Johnston
0487c9441f Fix tests 2016-01-27 17:33:13 +00:00
Erik Johnston
9959d9ece8 Remove redundated BaseHomeServer 2016-01-26 13:52:29 +00:00
Erik Johnston
8c6012a4af Fix tests 2016-01-25 13:12:35 +00:00
Erik Johnston
88baa3865e Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync 2016-01-22 15:57:12 +00:00
David Baker
d552861346 Revert all the bits changing keys of eeverything that used LRUCaches to tuples 2016-01-22 12:18:14 +00:00
David Baker
10f76dc5da Make LRU cache not default to treecache & add options to use it 2016-01-22 12:10:33 +00:00
David Baker
31a051b677 Test treecache directly 2016-01-22 11:22:00 +00:00
Erik Johnston
8f9c74e9f1 Fix tests 2016-01-22 10:48:27 +00:00
David Baker
4efcaa43c8 Add tests for treecache directly and test del_multi at the LruCache level too. 2016-01-22 10:37:37 +00:00
David Baker
f1f8122120 Change LRUCache to be tree-based so we can delete subtrees. 2016-01-21 19:16:25 +00:00
Erik Johnston
191070123d Cache dns lookups, and use the cache if we fail to lookup servers later 2016-01-20 11:34:09 +00:00
Erik Johnston
2c176e02ae Make unit tests work 2016-01-18 14:48:50 +00:00
Daniel Wagner-Hall
ac5a4477ad Require unbanning before other membership changes 2016-01-15 16:27:26 +00:00
Daniel Wagner-Hall
2680043bc6 Require ID and as_token be unique for ASs
Defaults ID to as_token if not specified. This will change
when IDs are fully supported.
2016-01-14 14:34:01 +00:00
Mark Haines
c0a279e808 Delete the table objects from TransactionStore 2016-01-13 11:15:20 +00:00
Daniel Wagner-Hall
42aa1f3f33 Merge pull request #478 from matrix-org/daniel/userobject
Introduce a User object

I'm sick of passing around more and more things as tuple items around
the whole world, and needing to edit every call site every time there is
more information about a user. So pass them around together as an
object.

This object has incredibly poorly named fields because we have a
convention that `user` indicates a UserID object, and `user_id`
indicates a string. I tried to clean up the whole repo to fix this, but
gave up. So instead, I introduce a second convention. A user_object is a
User, and a user_id_object is a UserId. I may have cried a little bit.
2016-01-11 17:50:22 +00:00
Daniel Wagner-Hall
2110e35fd6 Introduce a Requester object
This tracks data about the entity which made the request. This is
instead of passing around a tuple, which requires call-site
modifications every time a new piece of optional context is passed
around.

I tried to introduce a User object. I gave up.
2016-01-11 17:48:45 +00:00
David Baker
c232780081 Merge pull request #456 from matrix-org/store_event_actions
Send unread notification counts
2016-01-08 14:47:15 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00