Erik Johnston
c4a8cbd15a
Make LruCache use a dedicated _Node class
2016-03-22 16:06:21 +00:00
Erik Johnston
a547e2df85
Return list, not generator.
2016-03-14 15:30:19 +00:00
Mark Haines
239badea9b
Use syntax that works on both py2.7 and py3
2016-03-07 20:13:10 +00:00
Erik Johnston
374f9b2f07
Limit stream change cache size too
2016-03-01 13:30:15 +00:00
Erik Johnston
ce2cdced61
Move cache size fiddling to descriptors only. Fix tests
2016-03-01 13:21:46 +00:00
Erik Johnston
910fc0f28f
Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
2016-03-01 12:56:39 +00:00
Erik Johnston
72165e5b77
Reraise exception
2016-03-01 11:00:10 +00:00
Erik Johnston
ff2d7551c7
Correct cache miss detection
2016-03-01 10:59:17 +00:00
Erik Johnston
278d6c0527
Report size of ExpiringCache
2016-02-23 16:46:21 +00:00
Erik Johnston
c77dae7a1a
Change the way we figure out presence updates for small deltas
2016-02-23 14:54:40 +00:00
Erik Johnston
5614b4dafb
Add presence metrics
2016-02-19 09:50:54 +00:00
Erik Johnston
b8cdec92c7
WheelTimer: Don't scan list, use index.
2016-02-18 16:33:07 +00:00
Erik Johnston
e5999bfb1a
Initial cut
2016-02-17 15:40:50 +00:00
Erik Johnston
9e7900da1e
Add wheeltimer impl
2016-02-17 14:29:28 +00:00
Erik Johnston
e557dc80b8
Merge pull request #566 from matrix-org/erikj/logcontext
...
Don't bother copying records on parent context
2016-02-10 11:41:45 +00:00
Erik Johnston
4eb8f9ca8a
Remove comment
2016-02-10 11:29:21 +00:00
Erik Johnston
00c9ad49df
s/parent_context/previous_context/
2016-02-10 11:25:19 +00:00
Erik Johnston
9777c5f49a
Set parent context on instansiation
2016-02-10 11:23:32 +00:00
Erik Johnston
82631c5f94
Fix unit tests
2016-02-09 13:50:37 +00:00
Erik Johnston
9daa4e2a85
Don't create new logging context
2016-02-09 11:06:19 +00:00
Erik Johnston
3e2fcd67b2
Don't bother copying records on parent context
2016-02-09 10:50:31 +00:00
Erik Johnston
241b71852e
Fix bug in util.metrics.Measure
2016-02-09 10:28:13 +00:00
Erik Johnston
97294ef2fd
Create new context when measuring
2016-02-09 10:12:00 +00:00
Erik Johnston
c486b7b41c
Change logcontext warns to debug
2016-02-09 09:20:38 +00:00
Erik Johnston
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
Erik Johnston
13e6262659
Add metrics to pushers
2016-02-08 14:26:45 +00:00
Erik Johnston
d4f72a5bfb
Allowing tagging log contexts
2016-02-03 13:52:27 +00:00
Daniel Wagner-Hall
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
Erik Johnston
e70165039c
If stream pos is greater then earliest known key and entity hasn't changed, then entity hasn't changed
2016-01-29 16:41:32 +00:00
Erik Johnston
18579534ea
Prefill stream change caches
2016-01-29 14:37:59 +00:00
Erik Johnston
b18114e19e
Merge pull request #536 from matrix-org/erikj/sync
...
Make /sync "better".
2016-01-29 13:04:51 +00:00
Erik Johnston
fb7299800f
Directly set self.value
2016-01-29 11:29:14 +00:00
Erik Johnston
c046630c33
Remove spurious self.size
2016-01-29 11:17:54 +00:00
Erik Johnston
a30364c1f9
Correctly bookkeep the size of TreeCache
2016-01-29 10:44:46 +00:00
Erik Johnston
766526e114
Make TreeCache keep track of its own size.
2016-01-29 10:11:21 +00:00
Erik Johnston
50e18938a9
Reset size on clear
2016-01-29 10:00:45 +00:00
Erik Johnston
3f5dd18bd4
If the same as the earliest key, assume nothing has changed.
2016-01-28 18:11:41 +00:00
Erik Johnston
40431251cb
Correctly update _entity_to_key
2016-01-28 18:05:43 +00:00
Erik Johnston
82cf3a8043
Fix inequalities
2016-01-28 17:44:04 +00:00
Erik Johnston
0663c5bd52
Include cache hits with has_entity_changed
2016-01-28 17:27:28 +00:00
Erik Johnston
45cf827c8f
Change name and doc has_entity_changed
2016-01-28 16:39:18 +00:00
Erik Johnston
00cb3eb24b
Cache tags and account data
2016-01-28 16:37:41 +00:00
Erik Johnston
c23a8c7833
Ensure keys to RoomStreamChangeCache are ints
2016-01-28 15:55:26 +00:00
Erik Johnston
e1941442d4
Invalidate caches properly. Remove unused arg
2016-01-28 15:02:41 +00:00
Erik Johnston
b97f6626b6
Add cache to room stream
2016-01-27 17:33:26 +00:00
David Baker
7cd418d38e
Don't add the member functiopn if we're not using treecache
2016-01-22 13:40:37 +00:00
David Baker
cd80019eec
docs
2016-01-22 12:21:13 +00:00
David Baker
d552861346
Revert all the bits changing keys of eeverything that used LRUCaches to tuples
2016-01-22 12:18:14 +00:00
David Baker
10f76dc5da
Make LRU cache not default to treecache & add options to use it
2016-01-22 12:10:33 +00:00
David Baker
5b142788d2
Add __contains__
2016-01-22 11:49:59 +00:00
David Baker
eaa836e8ca
Docs for treecache
2016-01-22 11:47:22 +00:00
David Baker
8acc5cb60f
Add invalidate_many here too
2016-01-22 11:22:32 +00:00
David Baker
330be18ec5
peppate
2016-01-21 19:17:32 +00:00
David Baker
f1f8122120
Change LRUCache to be tree-based so we can delete subtrees.
2016-01-21 19:16:25 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
Mark Haines
d12c00bdc3
Add some docstring explaining the snapshot cache does
2015-12-23 15:18:11 +00:00
Mark Haines
7fa71e3267
Add a unit test for the snapshot cache
2015-12-23 11:48:03 +00:00
Mark Haines
9ac417fa88
Add a cache for initialSync responses that expires after 5 minutes
2015-12-22 18:27:56 +00:00
Mark Haines
6a5ff5f223
Track the time spent in the database per request.
...
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
Mark Haines
3dd1630848
Add a setter for the current log context.
...
Move the resource tracking inside that setter so that it is easier
to make sure that the resource tracking isn't double counting the
resource usage.
2015-12-07 10:51:18 +00:00
Mark Haines
5231737369
Add comments to explain why we are hardcoding RUSAGE_THREAD
2015-12-04 11:53:38 +00:00
Mark Haines
d6059bdd2a
Fix warnings
2015-12-04 11:34:23 +00:00
Mark Haines
48a2526d62
Track the cpu used in the main thread by each logging context
2015-12-03 21:03:01 +00:00
Mark Haines
a412b9a465
Run the background updates when starting synapse.
2015-11-10 15:50:58 +00:00
Daniel Wagner-Hall
2cebe53545
Exchange 3pid invites for m.room.member invites
2015-11-05 16:43:19 +00:00
Erik Johnston
5bc690408d
Merge pull request #340 from matrix-org/erikj/server_retries
...
Retry dead servers a lot less often
2015-11-05 16:15:50 +00:00
Mark Haines
c452dabc3d
Remove the LockManager class because it wasn't being used
2015-11-04 14:08:15 +00:00
Erik Johnston
eacb068ac2
Retry dead servers a lot less often
2015-11-02 16:56:30 +00:00
Daniel Wagner-Hall
9261ef3a15
Merge pull request #312 from matrix-org/daniel/3pidinvites
...
Stuff signed data in a standalone object
2015-10-19 15:52:34 +01:00
Mark Haines
9371a35e89
Merge pull request #306 from matrix-org/markjh/unused_methods
...
Remove some login classes from synapse.
2015-10-16 18:18:41 +01:00
Daniel Wagner-Hall
0e5239ffc3
Stuff signed data in a standalone object
...
Makes both generating it in sydent, and verifying it here, simpler at
the cost of some repetition
2015-10-16 17:45:48 +01:00
Daniel Wagner-Hall
c225d63e9e
Add signing host and keyname to signatures
2015-10-16 15:07:56 +01:00
Daniel Wagner-Hall
b8dd5b1a2d
Verify third party ID server certificates
2015-10-16 14:54:54 +01:00
Daniel Wagner-Hall
643b5fcdc8
Look for keys on the right objects
2015-10-15 13:10:30 +01:00
Daniel Wagner-Hall
f38df51e8d
Merge branch 'develop' into daniel/3pidinvites
2015-10-15 11:51:55 +01:00
Mark Haines
5338220d3a
synapse.util.emailutils was unused
2015-10-15 10:39:33 +01:00
Daniel Wagner-Hall
d82c5f7b5c
Use more descriptive error code
2015-10-13 18:02:00 +01:00
Daniel Wagner-Hall
0c38e8637f
Remove unnecessary class-wrapping
2015-10-13 18:00:38 +01:00
Mark Haines
32d66738b0
Fix pep8 warnings.
2015-10-13 17:18:29 +01:00
Mark Haines
7639c3d9e5
Bounce all deferreds through the reactor to make debugging easier.
...
If all deferreds wait a reactor tick before resolving then there is
always a chance to add an errback to the deferred so that stacktraces
get reported, rather than being discarded.
2015-10-13 17:13:04 +01:00
Daniel Wagner-Hall
17dffef5ec
Move event contents into third_party_layout field
2015-10-13 15:48:12 +01:00
Daniel Wagner-Hall
5b3e9713dd
Implement third party identifier invites
2015-10-01 17:49:52 +01:00
Mark Haines
372ac60375
synapse/util/__init__.py:unwrap_deferred was unused
2015-09-22 18:16:07 +01:00
Mark Haines
e85c7873dc
Allow non-ascii filenames for attachments
2015-08-26 16:26:37 +01:00
Erik Johnston
8e254862f4
Don't assume @cachedList function returns keys for everything
2015-08-18 11:11:33 +01:00
Erik Johnston
cfa62007a3
Docstring
2015-08-12 16:42:46 +01:00
Erik Johnston
d7451e0f22
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-12 10:30:30 +01:00
Erik Johnston
4807616e16
Wire up the dictionarycache to the metrics
2015-08-12 10:13:35 +01:00
Erik Johnston
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
Erik Johnston
dcefac3b06
Comments
2015-08-10 14:16:24 +01:00
Erik Johnston
f596ff402e
Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache
2015-08-06 13:37:56 +01:00
Erik Johnston
7eea3e356f
Make @cached cache deferreds rather than the deferreds' values
2015-08-06 13:33:34 +01:00
Erik Johnston
07507643cb
Use dictionary cache to do group -> state fetching
2015-08-05 15:11:42 +01:00
Erik Johnston
c67ba143fa
Move DictionaryCache
2015-08-04 15:58:28 +01:00
Erik Johnston
9beaedd164
Enforce ascii filenames for uploads
2015-06-30 10:31:59 +01:00
Erik Johnston
7fa1363fb0
Merge pull request #192 from matrix-org/erikj/fix_log_context
...
Fix log context when sending requests
2015-06-19 16:21:40 +01:00
Erik Johnston
a68abc79fd
Add comment on cancellation of observers
2015-06-19 11:48:55 +01:00
Erik Johnston
653533a3da
Fix log context when sending requests
2015-06-19 11:46:49 +01:00
Erik Johnston
22049ea700
Refactor the notifier.wait_for_events code to be clearer. Add _NotifierUserStream.new_listener that accpets a token to avoid races.
2015-06-18 15:49:24 +01:00
Erik Johnston
dd74436ffd
Unused import
2015-05-28 15:47:20 +01:00
Erik Johnston
31ade3b3e9
Remove a deep copy
2015-05-28 13:45:23 +01:00
Erik Johnston
4429e720ae
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-05-22 10:33:00 +01:00
Mark Haines
d61ce3f670
Add a cache for get_current_state with state_key
2015-05-21 16:41:39 +01:00
Erik Johnston
7cd6a6f6cf
Awful idea for speeding up fetching of events
2015-05-14 15:34:02 +01:00
Erik Johnston
cdb3757942
Refactor _get_events
2015-05-14 13:31:55 +01:00
Mark Haines
ec07dba29e
Merge pull request #143 from matrix-org/erikj/SYN-375
...
SYN-375 - Lots of unhandled deferred exceptions.
2015-05-12 15:25:54 +01:00
Erik Johnston
8022b27fc2
Make distributer.fire work as it did
2015-05-12 13:14:48 +01:00
Erik Johnston
95dedb866f
Unwrap defer.gatherResults failures
2015-05-12 13:14:29 +01:00
Erik Johnston
2236ef6c92
Fix up leak. Add warnings.
2015-05-08 19:53:34 +01:00
Erik Johnston
da1aa07db5
Add some docs
2015-05-08 16:52:49 +01:00
Erik Johnston
476899295f
Change the way we do logging contexts so that they survive divergences
2015-05-08 16:32:18 +01:00
Erik Johnston
fca28d243e
Change the way we create observers to deferreds so that we don't get spammed by 'unhandled errors'
2015-05-08 16:28:08 +01:00
Erik Johnston
6f8e2d517e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-27 14:41:40 +01:00
Erik Johnston
1c82fbd2eb
Implement create_observer.
...
`create_observer` takes a deferred and create a new deferred that
*observers* the original deferred. Any callbacks added to the observing
deferred will *not* affect the origin deferred.
2015-04-27 13:59:37 +01:00
Erik Johnston
806f380a8b
Make LruCache thread safe, as its used for event cache
2015-04-15 16:07:59 +01:00
Erik Johnston
a43b40449b
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-08 16:46:56 +01:00
Erik Johnston
9cef051ce2
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-04-08 16:43:17 +01:00
Erik Johnston
ccda401dbf
SYN-338: Fix typo that caused the cache to throw an exception in some instances
2015-04-08 16:34:23 +01:00
Erik Johnston
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
Erik Johnston
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
Paul "LeoNerd" Evans
d6b3ea75d4
Implement the 'key in dict' test for LruCache()
2015-03-25 19:04:59 +00:00
Erik Johnston
ed4d44d833
Merge pull request #109 from matrix-org/default_registration
...
Disable registration by default. Add script to register new users.
2015-03-18 11:38:52 +00:00
Erik Johnston
69135f59aa
Implement registering with shared secret.
2015-03-13 15:23:37 +00:00
Paul "LeoNerd" Evans
e1a7e3564f
Delete a couple of TODO markers of monitoring stats now done
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
027fd1242c
Give LruCache a __len__, so that len(cache) works
2015-03-04 17:32:28 +00:00
Erik Johnston
3077cb2915
Use contextlib.contextmanager instead of a custom class
2015-03-02 13:32:44 +00:00
Erik Johnston
0554d07082
Move federation rate limiting out of transport layer
2015-02-27 15:41:52 +00:00
Erik Johnston
cc3d3babb0
Remove unused import
2015-02-18 12:01:41 +00:00
Erik Johnston
36e144091b
Remove spurious comma. Remove temp run_on_reactor
2015-02-18 11:25:20 +00:00
Erik Johnston
b17bd31da0
Temporarily add a run_on_reactor() call
2015-02-18 11:17:26 +00:00
Erik Johnston
859fbd4423
s/self._clock/self.clock/
2015-02-18 10:39:14 +00:00
Erik Johnston
1be67eca8a
Merge branch 'keyclient_retry_scheme' of github.com:matrix-org/synapse into develop
2015-02-18 10:34:40 +00:00
Erik Johnston
fe672a04f7
Merge pull request #77 from matrix-org/failures
...
Failures
2015-02-18 10:29:29 +00:00
Erik Johnston
4fd176a41d
More docs
2015-02-18 10:11:24 +00:00
Erik Johnston
d77912ff44
Docs.
2015-02-18 10:09:54 +00:00
Erik Johnston
9371019133
Try to only back off if we think we failed to connect to the remote
2015-02-17 18:13:34 +00:00
Erik Johnston
c8436b38a0
Only update destination_retry_timings if we have succeeded when retrying
2015-02-17 17:38:38 +00:00
Erik Johnston
f91263b1e0
Remove spurious self
2015-02-17 17:37:51 +00:00
Erik Johnston
2b8f1a956c
Add per server retry limiting.
...
Factor out the pre destination retry logic from TransactionQueue so it
can be reused in both get_pdu and crypto.keyring
2015-02-17 17:20:56 +00:00
Erik Johnston
964bb43fbe
Fix typo in function name
2015-02-17 15:44:41 +00:00
Erik Johnston
e7e20417ca
ExpiringCache: purge every 1/2 interval
2015-02-17 15:44:26 +00:00
Erik Johnston
472734a8cc
Consume errors in time_bound_deferred
2015-02-17 15:13:50 +00:00
Erik Johnston
2674aeb96a
Factor out ExpiringCache from StateHandler
2015-02-16 16:16:47 +00:00
Erik Johnston
c52e8d395b
Merge pull request #61 from matrix-org/timeout-federation-requests
...
Timeout federation requests
2015-02-11 17:10:33 +00:00
Erik Johnston
42bc56dad3
Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson
2015-02-11 17:01:38 +00:00
Erik Johnston
ef276e8770
Fix so timing out connections to actually work.
2015-02-11 16:48:05 +00:00
Erik Johnston
41a9a76a99
Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests
2015-02-11 16:12:59 +00:00
Erik Johnston
ddb816cf60
Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict
2015-02-11 15:44:28 +00:00
Mark Haines
d8324d5a2b
Add a lru cache class
2015-02-11 14:52:23 +00:00
Erik Johnston
fb233dc40b
Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests
2015-02-11 10:33:19 +00:00
Erik Johnston
dcf52469e8
Move time_bound_deferred into Clock
2015-02-11 10:25:06 +00:00
Erik Johnston
0e6b3e4e40
Time out HTTP federation requests
2015-02-10 18:17:27 +00:00
Erik Johnston
ef995e6946
Add looping_call to Clock
2015-02-09 14:47:59 +00:00
Erik Johnston
dfa05f0cd6
Optimize FrozenEvent creation
2015-01-06 18:51:03 +00:00
Erik Johnston
d5ae67e67d
Fix typo where we used wrong var.
2015-01-06 16:05:01 +00:00
Erik Johnston
03a501456c
Time how long calls to _get_destination_retry_timings take
2015-01-06 15:22:28 +00:00
Mark Haines
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
Mark Haines
041ac476a5
Supply auth_chain along with current state in '/state/', fetch auth events from a remote server if we are missing some of them
2014-12-18 18:47:13 +00:00
Mark Haines
dbe77ec79a
Replace distributor deferred list, with a simple for loop until I understand why the former breaks and the latter doesn't
2014-12-18 17:47:00 +00:00
Erik Johnston
5a465b67ba
Fix pyflakes
2014-12-16 13:41:43 +00:00
Erik Johnston
58168498b0
Remove FrozenEncoder
2014-12-16 13:38:38 +00:00
Erik Johnston
bc6564bac0
Add PEP8 newlines
2014-12-09 11:01:44 +00:00
Erik Johnston
721482c83e
Add forgotten file
2014-12-08 09:10:12 +00:00
Mark Haines
15099fade5
Drop log level for incorrect logging contexts to WARN if the context is wrong and DEBUG if the context is missing
2014-11-25 10:57:31 +00:00
Mark Haines
db9ce032a4
Fix pep8 codestyle warnings
2014-11-20 17:26:36 +00:00
Mark Haines
dfdda2c871
Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str'
2014-11-20 17:10:37 +00:00
Mark Haines
32090aee16
Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts
2014-11-20 16:24:00 +00:00
Mark Haines
ca91bb2f7f
Sometimes there isn't a current logging context
2014-11-19 17:18:55 +00:00
Mark Haines
97c7c34f6f
Preserve logging context in a few more places, drop the logging context after it has been stashed to reduce potential for confusion
2014-11-19 16:37:43 +00:00
Mark Haines
e903c941cb
Merge branch 'develop' into request_logging
...
Conflicts:
setup.py
synapse/storage/_base.py
synapse/util/async.py
2014-11-14 11:16:50 +00:00
Erik Johnston
4317c8e583
Implement new replace_state and changed prev_state
...
`prev_state` is now a list of previous state ids, similiar to
prev_events. `replace_state` now points to what we think was replaced.
2014-11-06 15:10:55 +00:00
Erik Johnston
ef9c4476a0
Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
2014-10-30 11:18:28 +00:00
Mark Haines
7c06399512
Merge branch 'develop' into request_logging
...
Conflicts:
synapse/config/logger.py
2014-10-30 11:13:58 +00:00
Mark Haines
7d709542ca
Fix pep8 warnings
2014-10-30 11:10:17 +00:00
Mark Haines
fa955cc2a4
Pep8 and a few doc strings
2014-10-30 10:13:46 +00:00
Mark Haines
b29517bd01
Add a request-id to each log line
2014-10-30 01:21:33 +00:00
Erik Johnston
53216a500d
Add a run_on_reactor function
2014-10-29 17:02:22 +00:00
Kegan Dougal
cc83b06cd1
Added support for the HS to send emails. Use it to send password resets. Added email_smtp_server and email_from_address config args. Added emailutils.
2014-09-16 12:36:39 +01:00
Matthew Hodgson
8a7c1d6a00
fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch.
2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans
967b45bc1a
Allow optional non-suppression of exceptions through the Distributor
2014-09-03 10:40:21 +01:00
Mark Haines
c6eafdfbaf
Add copyright notices and fix pyflakes errors
2014-09-03 09:43:11 +01:00
Erik Johnston
54d0a75573
Merge branch 'develop' of github.com:matrix-org/synapse into presence_logging
...
Conflicts:
synapse/handlers/presence.py
2014-08-28 16:52:46 +01:00
Mark Haines
bddc1d9fff
use @wraps to set the __name__ __module__ and __doc__ correctly for logged functions
2014-08-28 15:32:38 +01:00
Erik Johnston
b8b52ca09d
Add logging to try and figure out what is going on with the presence stuff
2014-08-28 14:58:51 +01:00
Paul "LeoNerd" Evans
657ab9ba9d
Put some DEBUG logging in lockutils.py so we can debug roomlocks
2014-08-14 16:06:05 +01:00
Matthew Hodgson
f98e6380f1
add in whitespace after copyright statements to improve legibility
2014-08-13 03:14:34 +01:00
matrix.org
4f475c7697
Reference Matrix Home Server
2014-08-12 15:10:52 +01:00
