Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-22 03:54:59 -05:00
Code Issues Packages Projects Releases Wiki Activity
10,689 Commits 4 Branches 609 Tags 60 MiB
49e5a613f1
Commit Graph

347 Commits

Author SHA1 Message Date
Richard van der Hoff
b5b3a7e867 More intelligent Content-Type parsing 
Content-Type is allowed to contain options (`; charset=utf-8`, for
instance). We should allow that.
 2016-11-30 15:07:32 +00:00
Mark Haines
b1c27975d0 Set CORs headers on responses from the media repo 2016-11-02 11:29:25 +00:00
Erik Johnston
5810cffd33 Pass since/from parameters over federation 2016-09-15 10:36:19 +01:00
Mark Haines
949c2c5435 Add a timeout parameter for end2end key queries. 
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.

Fetch keys from different servers in parallel rather than in series.

Set the default timeout to 10s.
 2016-09-12 18:17:09 +01:00
Erik Johnston
33f3624ff7 Add exception logging. Fix typo 2016-08-22 10:49:31 +01:00
Erik Johnston
8c52160b07 Allow request handlers to override metric name 2016-08-22 10:44:45 +01:00
Erik Johnston
a093fab253 Use top level measure 2016-08-22 10:18:12 +01:00
Erik Johnston
27e0178da9 Add a top level measure 2016-08-19 18:49:37 +01:00
Erik Johnston
6d1a94d218 Remove redundant measure 2016-08-19 18:40:31 +01:00
Erik Johnston
39b900b316 Measure http.server render 2016-08-19 18:23:44 +01:00
Erik Johnston
ca8abfbf30 Clean up TransactionQueue 2016-08-10 16:24:16 +01:00
Richard van der Hoff
33d08e8433 Log when adding listeners 2016-07-25 17:22:15 +01:00
Richard van der Hoff
255c229f23 Work around TLS bug in twisted 
Wrap up twisted's FileBodyProducer to work around
https://twistedmatrix.com/trac/ticket/8473. Hopefully this fixes
https://matrix.org/jira/browse/SYN-700.
 2016-06-15 10:39:08 +01:00
Mark Haines
eb79110beb Clean up the blacklist/whitelist handling. 
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
 2016-05-16 13:03:59 +01:00
Matthew Hodgson
792def4928 add a url_preview_ip_range_whitelist config param so we can whitelist the matrix.org IP space 2016-05-01 12:44:24 +01:00
Mark Haines
351b50a887 Fix more typos in per-request metrics 2016-04-28 15:29:46 +01:00
Mark Haines
dcfc10b129 Fix typo in request metrics 2016-04-28 15:11:06 +01:00
Mark Haines
1a12766e3b Add a comment explaining why automatic metric reporting is disabled for JsonResource 2016-04-28 12:31:26 +01:00
Mark Haines
6037349512 Check if report_metrics is True 2016-04-28 12:26:07 +01:00
Mark Haines
8d7ad44331 Report per request metrics for all of the things using request_handler 2016-04-28 10:57:49 +01:00
Mark Haines
e856036f4c Move SynapseSite to its own file 2016-04-22 16:09:55 +01:00
Erik Johnston
f505575f69 Make InsecureInterceptableContextFactory work with SpiderEndpoint 2016-04-19 16:08:14 +01:00
Matthew Hodgson
4bd3d25218 Merge pull request #688 from matrix-org/matthew/preview_urls 
URL previewing support
 2016-04-11 10:40:29 +01:00
Matthew Hodgson
83b2f83da0 actually throw meaningful errors 2016-04-08 21:36:59 +01:00
Matthew Hodgson
dafef5a688 Add url_preview_enabled config option to turn on/off preview_url endpoint. defaults to off. 
Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered.
Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered.
Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist
Add commentary and generally address PR feedback
 2016-04-08 18:37:15 +01:00
Erik Johnston
a28d066732 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache 2016-04-07 11:11:17 +01:00
Matthew Hodgson
9f7dc2bef7 Merge branch 'develop' into matthew/preview_urls 2016-04-04 00:38:21 +01:00
Matthew Hodgson
8b98a7e8c3 pep8 2016-04-03 12:56:29 +01:00
Matthew Hodgson
7426c86eb8 add a persistent cache of URL lookups, and fix up the in-memory one to work 2016-04-03 00:31:57 +01:00
Matthew Hodgson
d1b154a10f support gzip compression, and don't pass through error msgs 2016-04-02 03:06:39 +01:00
Mark Haines
2a37467fa1 Use google style doc strings. 
pycharm supports them so there is no need to use the other format.

Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
 2016-04-01 16:12:07 +01:00
Matthew Hodgson
683e564815 handle spidered relative images correctly 2016-03-31 23:52:58 +01:00
Matthew Hodgson
0d3d7de6fc sync in changes from matrixfederationclient 2016-03-31 12:42:27 +01:00
Erik Johnston
f9d3665c88 Allow clock to be passed in to func 2016-03-31 10:23:48 +01:00
Erik Johnston
f699b8f997 Read from DNS cache if within TTL 2016-03-31 10:04:28 +01:00
Matthew Hodgson
a8a5dd3b44 handle requests with missing content-length headers (e.g. YouTube) 2016-03-31 01:55:21 +01:00
Matthew Hodgson
721b2bfa85 implement redirects 2016-03-29 03:32:52 +01:00
Matthew Hodgson
64b4aead15 make it work 2016-03-29 03:13:25 +01:00
Matthew Hodgson
dd4287ca5d make it build 2016-03-29 02:07:57 +01:00
Matthew Hodgson
e0c2490a14 Merge branch 'develop' into matthew/preview_urls 2016-03-29 01:20:25 +01:00
Matthew Hodgson
d9d48aad2d Merge branch 'develop' into matthew/preview_urls 2016-03-27 22:54:42 +01:00
Erik Johnston
acdfef7b14 Intern all the things 2016-03-23 16:25:54 +00:00
Mark Haines
398cd1edfb Fix regression where synapse checked whether push rules were valid JSON before the compatibility hack that handled clients sending invalid JSON 2016-03-14 14:16:41 +00:00
Mark Haines
e9c1cabac2 Use parse_json_object_from_request to parse JSON out of request bodies 2016-03-11 16:41:03 +00:00
Mark Haines
b7dbe5147a Add a parse_json_object function 
to deduplicate all the copy+pasted _parse_json functions. Also document
the parse_.* functions.
 2016-03-09 11:26:26 +00:00
David Baker
9a3c80a348 pep8 2016-03-08 10:09:07 +00:00
David Baker
7bcee4733a Encode unicode objects given to post_urlencode* otherwise urllib.urlencode chokes. 2016-03-08 10:04:38 +00:00
Mark Haines
58c9f20692 Catch the exceptions thrown by twisted when you write to a closed connection 2016-02-12 13:46:59 +00:00
Erik Johnston
2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Erik Johnston
13e6262659 Add metrics to pushers 2016-02-08 14:26:45 +00:00
Erik Johnston
d4f72a5bfb Allowing tagging log contexts 2016-02-03 13:52:27 +00:00
Daniel Wagner-Hall
d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Matthew Hodgson
7dd0c1730a initial WIP of a tentative preview_url endpoint - incomplete, untested, experimental, etc. just putting it here for safekeeping for now 2016-01-24 18:47:27 -05:00
Erik Johnston
c6549117a2 Fix AttributeError 2016-01-21 14:02:14 +00:00
Erik Johnston
191070123d Cache dns lookups, and use the cache if we fail to lookup servers later 2016-01-20 11:34:09 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall
2b0f8a9482 Fix typo 2015-12-16 17:59:44 +01:00
Daniel Wagner-Hall
338c0a8a69 Include errcode on Internal Server Error 2015-12-14 13:50:50 +00:00
Mark Haines
6a5ff5f223 Track the time spent in the database per request. 
and track the number of transactions that request started.
 2015-12-07 17:56:11 +00:00
Mark Haines
99e1d6777f Add metrics to track the cpu on the main thread consumed by each type of request 2015-12-04 14:42:24 +00:00
Daniel Wagner-Hall
14d7acfad4 Host /unstable and /r0 versions of r0 APIs 2015-12-01 17:34:32 +00:00
Erik Johnston
0eabfa55f6 Fix typo 2015-11-20 17:17:58 +00:00
Erik Johnston
6408541075 Don't limit connections to perspective servers 2015-11-20 17:15:44 +00:00
Erik Johnston
2f2bbb4d06 Use min and not max to set an upper bound on retry interval 2015-11-20 09:34:58 +00:00
Erik Johnston
cbf3cd6151 Add comment 2015-11-17 18:29:29 +00:00
Erik Johnston
cf4ef5f3c7 Only retry federation requests for a long time for background requests 2015-11-17 18:26:50 +00:00
Erik Johnston
afdfd12bdf Implement required method 'resumeProducing' 2015-11-17 16:57:06 +00:00
Erik Johnston
bceec65913 Slightly more aggressive retry timers at HTTP level 2015-11-17 15:10:05 +00:00
Erik Johnston
eacb068ac2 Retry dead servers a lot less often 2015-11-02 16:56:30 +00:00
Daniel Wagner-Hall
b8dd5b1a2d Verify third party ID server certificates 2015-10-16 14:54:54 +01:00
Erik Johnston
a80ef851f7 Fix previous merge to s/version_string/user_agent/ 2015-10-10 12:35:39 +01:00
Steven Hammerton
625e13bfde Add get_raw method to SimpleHttpClient, use this in CAS auth rather than requests 2015-10-10 10:54:19 +01:00
Daniel Wagner-Hall
492beb62a8 Use space not dash as delimiter 2015-10-06 09:53:33 -05:00
Daniel Wagner-Hall
e0b466bcfd Use space not dash as delimiter 2015-10-06 09:32:26 -05:00
Daniel Wagner-Hall
b28c7da0a4 Preserve version string in user agent 2015-10-05 20:49:39 -05:00
Daniel Wagner-Hall
8fc52bc56a Allow synapse's useragent to be customized 
This will allow me to write tests which verify which server made HTTP
requests in a federation context.
 2015-10-02 17:13:51 -05:00
Daniel Wagner-Hall
3bcbabc9fb Rename context factory 
Mjark is officially no fun.
 2015-09-15 15:46:22 +01:00
Daniel Wagner-Hall
6485f03d91 Fix random formatting 2015-09-09 13:05:00 +01:00
Daniel Wagner-Hall
81a93ddcc8 Allow configuration to ignore invalid SSL certs 
This will be useful for sytest, and sytest only, hence the aggressive
config key name.
 2015-09-09 12:02:07 +01:00
Mark Haines
78323ccdb3 Remove syutil dependency in favour of smaller single-purpose libraries 2015-08-24 16:17:38 +01:00
Erik Johnston
adbd720fab PEP8 2015-08-13 11:47:38 +01:00
Mark Haines
998a72d4d9 Merge branch 'develop' into markjh/twisted-15 
Conflicts:
	synapse/http/matrixfederationclient.py
 2015-08-12 17:21:14 +01:00
Mark Haines
a7dcbfe430 Don't 500 if a group is missing from the regex 2015-07-06 16:47:17 +01:00
Erik Johnston
653533a3da Fix log context when sending requests 2015-06-19 11:46:49 +01:00
Erik Johnston
18968efa0a Remove stale debug lines 2015-06-19 10:18:02 +01:00
Erik Johnston
9d112f4440 Add IDs to outbound transactions 2015-06-19 10:13:03 +01:00
Erik Johnston
aaa749d366 Disable twisted access logging. Move access logging to SynapseRequest object 2015-06-15 18:18:05 +01:00
Erik Johnston
cee69441d3 Log more when we have processed the request 2015-06-15 17:11:44 +01:00
Erik Johnston
b5209c5744 Create SynapseRequest that overrides __repr__ to not print access_token 2015-06-15 16:37:04 +01:00
Erik Johnston
f00f8346f1 Make http.server request logging more verbose, but redact access_tokens 2015-06-15 13:37:58 +01:00
Erik Johnston
83f119a84a Log requests and responses sent via http.client 2015-06-15 13:14:12 +01:00
Mark Haines
b8d49be5a1 Merge branch 'develop' into markjh/twisted-15 
Conflicts:
	synapse/python_dependencies.py
 2015-06-01 10:56:05 +01:00
Mark Haines
90abdaf3bc Use Twisted-15.2.1, Use Agent.usingEndpointFactory rather than implement our own Agent 2015-06-01 10:51:50 +01:00
Erik Johnston
d15f166093 Remove log line 2015-05-29 15:03:24 +01:00
Erik Johnston
a7b65bdedf Add config option to turn off freezing events. Use new encode_json api and ujson.loads 2015-05-29 12:17:33 +01:00
Erik Johnston
79e37a7ecb Correctly pass connection pool parameter 2015-05-28 16:48:53 +01:00
Erik Johnston
11f51e6ded Up maxPersistentPerHost count 2015-05-28 15:45:46 +01:00
Erik Johnston
086df80790 Add connection pooling to SimpleHttpClient 2015-05-28 15:43:21 +01:00
Erik Johnston
291e942332 Use connection pool for federation connections 2015-05-28 15:43:21 +01:00
Erik Johnston
284f55a7fb Add doc strings 2015-05-22 15:18:04 +01:00
Erik Johnston
5b1631a4a9 Add a timeout param to get_event 2015-05-19 14:53:32 +01:00
Erik Johnston
2236ef6c92 Fix up leak. Add warnings. 2015-05-08 19:53:34 +01:00
Erik Johnston
476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
David Baker
0c1b7f843b Unused import 2015-04-30 13:33:30 +01:00
David Baker
4b46fbec5b Doesn't look like this is used anymore 2015-04-30 12:04:08 +01:00
David Baker
6532b6e607 Merge branch 'develop' into csauth 
Conflicts:
	synapse/http/server.py
 2015-04-24 09:37:54 +01:00
Mark Haines
1967650bc4 Combine the request wrappers in rest/media/v1 and http/server into a single wrapper decorator 2015-04-21 16:35:53 +01:00
Mark Haines
1ebff9736b Split out the JsonResource request logging and error handling into a separate wrapper function 2015-04-21 16:07:20 +01:00
David Baker
cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
Mark Haines
3cbc286d06 Move server key api into rest/key/v1 2015-04-14 13:28:11 +01:00
David Baker
c5bf0343e8 Explain how I justified to myself making JsonResource not always send JSON. 2015-04-01 15:13:14 +01:00
David Baker
e9c908ebc0 Completely replace fallback auth for C/S V2: 
* Now only the auth part goes to fallback, not the whole operation
 * Auth fallback is a normal API endpoint, not a static page
 * Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
 2015-04-01 15:05:30 +01:00
David Baker
f129ee1e18 Make docs a bit more true 2015-03-31 18:25:10 +01:00
David Baker
d18e7779ca Grammar and deduplication 2015-03-31 14:40:02 +01:00
David Baker
59bf16eddc New registration for C/S API v2. Only ReCAPTCHA working currently. 2015-03-30 18:13:10 +01:00
Erik Johnston
ed4d44d833 Merge pull request #109 from matrix-org/default_registration 
Disable registration by default. Add script to register new users.
 2015-03-18 11:38:52 +00:00
Paul "LeoNerd" Evans
3f7a31d366 Add a DistributionMetric to HTTP request/response processing time in the server 2015-03-16 18:31:29 +00:00
Matthew Hodgson
c8032aec17 actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0' 2015-03-14 00:12:20 +00:00
Erik Johnston
dea236e4fa Add missing commas 2015-03-13 15:24:03 +00:00
Paul "LeoNerd" Evans
128cf2daf7 Appease pep8 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
1748605c5d Count incoming HTTP requests per servlet that responds 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b0cf867319 Use _ instead of . as a metric namespacing separator, for Prometheus 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f9478e475b Rename Metrics' "keys" to "labels" 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
7d72e44eb9 Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
c53ec53d80 Pull out all uses of the underlying HTTP user agent .request() method into a single wrapper function, to make adding metrics easier 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
9d9d39536b Slightly reduce the insane amounts of indentation in main http server response path, by 'continue'ing around a non-match or falling through 2015-03-05 16:24:13 +00:00
Kegan Dougal
2de5b14fe0 Fix bug which prevented the HS pushing events to the AS due to FrozenEvents 2015-03-02 15:36:37 +00:00
Erik Johnston
7025781df8 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.7.1 2015-02-18 17:37:43 +00:00
Erik Johnston
3f1871021e Make /keys/ return correct Server version 2015-02-18 17:32:12 +00:00
Erik Johnston
5358966a87 Use git aware version string in User-Agent and Server headers 2015-02-18 16:52:04 +00:00
Erik Johnston
7e9d59f3b4 Don't convert DNSLookupError to a 4xx SynapseError 2015-02-18 10:58:13 +00:00
Erik Johnston
65ca713ff5 Add .__name__ after type(e) 2015-02-18 10:51:32 +00:00
Erik Johnston
5e24471469 Fix up ResponseNeverReceived to str 2015-02-18 10:50:10 +00:00
Erik Johnston
0db52d43fa strings.join() expects iterable of strings 2015-02-18 10:41:46 +00:00
Erik Johnston
0647e27a41 Remove unused import 2015-02-17 15:19:54 +00:00
Erik Johnston
c02da58a9d Merge branch 'develop' of github.com:matrix-org/synapse into failures 2015-02-17 15:15:07 +00:00
Erik Johnston
4de93001bf Make matrixfederationclient log more nicely 2015-02-17 15:12:06 +00:00
Mark Haines
0d872f5aa6 Merge pull request #50 from matrix-org/application-services 
Application Services
 2015-02-13 15:06:14 +00:00
Erik Johnston
2bf0e85f3d Use encode_canonical_json for http client 2015-02-11 17:34:23 +00:00
Erik Johnston
c52e8d395b Merge pull request #61 from matrix-org/timeout-federation-requests 
Timeout federation requests
 2015-02-11 17:10:33 +00:00
Kegan Dougal
f2fdcb7c4b Merge branch 'develop' into application-services 2015-02-11 16:43:26 +00:00
Kegan Dougal
f518324426 Minor tweaks based on PR feedback. 2015-02-11 16:41:16 +00:00
Erik Johnston
41a9a76a99 Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests 2015-02-11 16:12:59 +00:00
Erik Johnston
4ebbaf0d43 Blunty replace json with simplejson 2015-02-11 14:23:10 +00:00
Erik Johnston
6370cffbbf Fix bug where variable was not always defined 2015-02-11 10:34:41 +00:00
Erik Johnston
fb233dc40b Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests 2015-02-11 10:33:19 +00:00
Erik Johnston
05b961d7e3 PEP8 2015-02-11 10:28:46 +00:00
Erik Johnston
dcf52469e8 Move time_bound_deferred into Clock 2015-02-11 10:25:06 +00:00
Erik Johnston
0e6b3e4e40 Time out HTTP federation requests 2015-02-10 18:17:27 +00:00
Kegan Dougal
53557fc532 Merge branch 'develop' into application-services 2015-02-09 15:20:56 +00:00