Matthew Hodgson
683e564815
handle spidered relative images correctly
2016-03-31 23:52:58 +01:00
Matthew Hodgson
0d3d7de6fc
sync in changes from matrixfederationclient
2016-03-31 12:42:27 +01:00
Erik Johnston
f9d3665c88
Allow clock to be passed in to func
2016-03-31 10:23:48 +01:00
Erik Johnston
f699b8f997
Read from DNS cache if within TTL
2016-03-31 10:04:28 +01:00
Matthew Hodgson
a8a5dd3b44
handle requests with missing content-length headers (e.g. YouTube)
2016-03-31 01:55:21 +01:00
Matthew Hodgson
721b2bfa85
implement redirects
2016-03-29 03:32:52 +01:00
Matthew Hodgson
64b4aead15
make it work
2016-03-29 03:13:25 +01:00
Matthew Hodgson
dd4287ca5d
make it build
2016-03-29 02:07:57 +01:00
Matthew Hodgson
e0c2490a14
Merge branch 'develop' into matthew/preview_urls
2016-03-29 01:20:25 +01:00
Matthew Hodgson
d9d48aad2d
Merge branch 'develop' into matthew/preview_urls
2016-03-27 22:54:42 +01:00
Erik Johnston
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
Mark Haines
398cd1edfb
Fix regression where synapse checked whether push rules were valid JSON before the compatibility hack that handled clients sending invalid JSON
2016-03-14 14:16:41 +00:00
Mark Haines
e9c1cabac2
Use parse_json_object_from_request to parse JSON out of request bodies
2016-03-11 16:41:03 +00:00
Mark Haines
b7dbe5147a
Add a parse_json_object function
...
to deduplicate all the copy+pasted _parse_json functions. Also document
the parse_.* functions.
2016-03-09 11:26:26 +00:00
David Baker
9a3c80a348
pep8
2016-03-08 10:09:07 +00:00
David Baker
7bcee4733a
Encode unicode objects given to post_urlencode* otherwise urllib.urlencode chokes.
2016-03-08 10:04:38 +00:00
Mark Haines
58c9f20692
Catch the exceptions thrown by twisted when you write to a closed connection
2016-02-12 13:46:59 +00:00
Erik Johnston
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
Erik Johnston
13e6262659
Add metrics to pushers
2016-02-08 14:26:45 +00:00
Erik Johnston
d4f72a5bfb
Allowing tagging log contexts
2016-02-03 13:52:27 +00:00
Daniel Wagner-Hall
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
Matthew Hodgson
7dd0c1730a
initial WIP of a tentative preview_url endpoint - incomplete, untested, experimental, etc. just putting it here for safekeeping for now
2016-01-24 18:47:27 -05:00
Erik Johnston
c6549117a2
Fix AttributeError
2016-01-21 14:02:14 +00:00
Erik Johnston
191070123d
Cache dns lookups, and use the cache if we fail to lookup servers later
2016-01-20 11:34:09 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall
2b0f8a9482
Fix typo
2015-12-16 17:59:44 +01:00
Daniel Wagner-Hall
338c0a8a69
Include errcode on Internal Server Error
2015-12-14 13:50:50 +00:00
Mark Haines
6a5ff5f223
Track the time spent in the database per request.
...
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
Mark Haines
99e1d6777f
Add metrics to track the cpu on the main thread consumed by each type of request
2015-12-04 14:42:24 +00:00
Daniel Wagner-Hall
14d7acfad4
Host /unstable and /r0 versions of r0 APIs
2015-12-01 17:34:32 +00:00
Erik Johnston
0eabfa55f6
Fix typo
2015-11-20 17:17:58 +00:00
Erik Johnston
6408541075
Don't limit connections to perspective servers
2015-11-20 17:15:44 +00:00
Erik Johnston
2f2bbb4d06
Use min and not max to set an upper bound on retry interval
2015-11-20 09:34:58 +00:00
Erik Johnston
cbf3cd6151
Add comment
2015-11-17 18:29:29 +00:00
Erik Johnston
cf4ef5f3c7
Only retry federation requests for a long time for background requests
2015-11-17 18:26:50 +00:00
Erik Johnston
afdfd12bdf
Implement required method 'resumeProducing'
2015-11-17 16:57:06 +00:00
Erik Johnston
bceec65913
Slightly more aggressive retry timers at HTTP level
2015-11-17 15:10:05 +00:00
Erik Johnston
eacb068ac2
Retry dead servers a lot less often
2015-11-02 16:56:30 +00:00
Daniel Wagner-Hall
b8dd5b1a2d
Verify third party ID server certificates
2015-10-16 14:54:54 +01:00
Erik Johnston
a80ef851f7
Fix previous merge to s/version_string/user_agent/
2015-10-10 12:35:39 +01:00
Steven Hammerton
625e13bfde
Add get_raw method to SimpleHttpClient, use this in CAS auth rather than requests
2015-10-10 10:54:19 +01:00
Daniel Wagner-Hall
492beb62a8
Use space not dash as delimiter
2015-10-06 09:53:33 -05:00
Daniel Wagner-Hall
e0b466bcfd
Use space not dash as delimiter
2015-10-06 09:32:26 -05:00
Daniel Wagner-Hall
b28c7da0a4
Preserve version string in user agent
2015-10-05 20:49:39 -05:00
Daniel Wagner-Hall
8fc52bc56a
Allow synapse's useragent to be customized
...
This will allow me to write tests which verify which server made HTTP
requests in a federation context.
2015-10-02 17:13:51 -05:00
Daniel Wagner-Hall
3bcbabc9fb
Rename context factory
...
Mjark is officially no fun.
2015-09-15 15:46:22 +01:00
Daniel Wagner-Hall
6485f03d91
Fix random formatting
2015-09-09 13:05:00 +01:00
Daniel Wagner-Hall
81a93ddcc8
Allow configuration to ignore invalid SSL certs
...
This will be useful for sytest, and sytest only, hence the aggressive
config key name.
2015-09-09 12:02:07 +01:00
Mark Haines
78323ccdb3
Remove syutil dependency in favour of smaller single-purpose libraries
2015-08-24 16:17:38 +01:00
Erik Johnston
adbd720fab
PEP8
2015-08-13 11:47:38 +01:00
Mark Haines
998a72d4d9
Merge branch 'develop' into markjh/twisted-15
...
Conflicts:
synapse/http/matrixfederationclient.py
2015-08-12 17:21:14 +01:00
Mark Haines
a7dcbfe430
Don't 500 if a group is missing from the regex
2015-07-06 16:47:17 +01:00
Erik Johnston
653533a3da
Fix log context when sending requests
2015-06-19 11:46:49 +01:00
Erik Johnston
18968efa0a
Remove stale debug lines
2015-06-19 10:18:02 +01:00
Erik Johnston
9d112f4440
Add IDs to outbound transactions
2015-06-19 10:13:03 +01:00
Erik Johnston
aaa749d366
Disable twisted access logging. Move access logging to SynapseRequest object
2015-06-15 18:18:05 +01:00
Erik Johnston
cee69441d3
Log more when we have processed the request
2015-06-15 17:11:44 +01:00
Erik Johnston
b5209c5744
Create SynapseRequest that overrides __repr__ to not print access_token
2015-06-15 16:37:04 +01:00
Erik Johnston
f00f8346f1
Make http.server request logging more verbose, but redact access_tokens
2015-06-15 13:37:58 +01:00
Erik Johnston
83f119a84a
Log requests and responses sent via http.client
2015-06-15 13:14:12 +01:00
Mark Haines
b8d49be5a1
Merge branch 'develop' into markjh/twisted-15
...
Conflicts:
synapse/python_dependencies.py
2015-06-01 10:56:05 +01:00
Mark Haines
90abdaf3bc
Use Twisted-15.2.1, Use Agent.usingEndpointFactory rather than implement our own Agent
2015-06-01 10:51:50 +01:00
Erik Johnston
d15f166093
Remove log line
2015-05-29 15:03:24 +01:00
Erik Johnston
a7b65bdedf
Add config option to turn off freezing events. Use new encode_json api and ujson.loads
2015-05-29 12:17:33 +01:00
Erik Johnston
79e37a7ecb
Correctly pass connection pool parameter
2015-05-28 16:48:53 +01:00
Erik Johnston
11f51e6ded
Up maxPersistentPerHost count
2015-05-28 15:45:46 +01:00
Erik Johnston
086df80790
Add connection pooling to SimpleHttpClient
2015-05-28 15:43:21 +01:00
Erik Johnston
291e942332
Use connection pool for federation connections
2015-05-28 15:43:21 +01:00
Erik Johnston
284f55a7fb
Add doc strings
2015-05-22 15:18:04 +01:00
Erik Johnston
5b1631a4a9
Add a timeout param to get_event
2015-05-19 14:53:32 +01:00
Erik Johnston
2236ef6c92
Fix up leak. Add warnings.
2015-05-08 19:53:34 +01:00
Erik Johnston
476899295f
Change the way we do logging contexts so that they survive divergences
2015-05-08 16:32:18 +01:00
David Baker
0c1b7f843b
Unused import
2015-04-30 13:33:30 +01:00
David Baker
4b46fbec5b
Doesn't look like this is used anymore
2015-04-30 12:04:08 +01:00
David Baker
6532b6e607
Merge branch 'develop' into csauth
...
Conflicts:
synapse/http/server.py
2015-04-24 09:37:54 +01:00
Mark Haines
1967650bc4
Combine the request wrappers in rest/media/v1 and http/server into a single wrapper decorator
2015-04-21 16:35:53 +01:00
Mark Haines
1ebff9736b
Split out the JsonResource request logging and error handling into a separate wrapper function
2015-04-21 16:07:20 +01:00
David Baker
cb03fafdf1
Merge branch 'develop' into csauth
2015-04-17 13:51:10 +01:00
Mark Haines
3cbc286d06
Move server key api into rest/key/v1
2015-04-14 13:28:11 +01:00
David Baker
c5bf0343e8
Explain how I justified to myself making JsonResource not always send JSON.
2015-04-01 15:13:14 +01:00
David Baker
e9c908ebc0
Completely replace fallback auth for C/S V2:
...
* Now only the auth part goes to fallback, not the whole operation
* Auth fallback is a normal API endpoint, not a static page
* Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
2015-04-01 15:05:30 +01:00
David Baker
f129ee1e18
Make docs a bit more true
2015-03-31 18:25:10 +01:00
David Baker
d18e7779ca
Grammar and deduplication
2015-03-31 14:40:02 +01:00
David Baker
59bf16eddc
New registration for C/S API v2. Only ReCAPTCHA working currently.
2015-03-30 18:13:10 +01:00
Erik Johnston
ed4d44d833
Merge pull request #109 from matrix-org/default_registration
...
Disable registration by default. Add script to register new users.
2015-03-18 11:38:52 +00:00
Paul "LeoNerd" Evans
3f7a31d366
Add a DistributionMetric to HTTP request/response processing time in the server
2015-03-16 18:31:29 +00:00
Matthew Hodgson
c8032aec17
actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0'
2015-03-14 00:12:20 +00:00
Erik Johnston
dea236e4fa
Add missing commas
2015-03-13 15:24:03 +00:00
Paul "LeoNerd" Evans
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
1748605c5d
Count incoming HTTP requests per servlet that responds
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b0cf867319
Use _ instead of . as a metric namespacing separator, for Prometheus
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
7d72e44eb9
Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
c53ec53d80
Pull out all uses of the underlying HTTP user agent .request() method into a single wrapper function, to make adding metrics easier
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
9d9d39536b
Slightly reduce the insane amounts of indentation in main http server response path, by 'continue'ing around a non-match or falling through
2015-03-05 16:24:13 +00:00
Kegan Dougal
2de5b14fe0
Fix bug which prevented the HS pushing events to the AS due to FrozenEvents
2015-03-02 15:36:37 +00:00
Erik Johnston
7025781df8
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.7.1
2015-02-18 17:37:43 +00:00
Erik Johnston
3f1871021e
Make /keys/ return correct Server version
2015-02-18 17:32:12 +00:00
Erik Johnston
5358966a87
Use git aware version string in User-Agent and Server headers
2015-02-18 16:52:04 +00:00
Erik Johnston
7e9d59f3b4
Don't convert DNSLookupError to a 4xx SynapseError
2015-02-18 10:58:13 +00:00
Erik Johnston
65ca713ff5
Add .__name__ after type(e)
2015-02-18 10:51:32 +00:00
Erik Johnston
5e24471469
Fix up ResponseNeverReceived to str
2015-02-18 10:50:10 +00:00
Erik Johnston
0db52d43fa
strings.join() expects iterable of strings
2015-02-18 10:41:46 +00:00
Erik Johnston
0647e27a41
Remove unused import
2015-02-17 15:19:54 +00:00
Erik Johnston
c02da58a9d
Merge branch 'develop' of github.com:matrix-org/synapse into failures
2015-02-17 15:15:07 +00:00
Erik Johnston
4de93001bf
Make matrixfederationclient log more nicely
2015-02-17 15:12:06 +00:00
Mark Haines
0d872f5aa6
Merge pull request #50 from matrix-org/application-services
...
Application Services
2015-02-13 15:06:14 +00:00
Erik Johnston
2bf0e85f3d
Use encode_canonical_json for http client
2015-02-11 17:34:23 +00:00
Erik Johnston
c52e8d395b
Merge pull request #61 from matrix-org/timeout-federation-requests
...
Timeout federation requests
2015-02-11 17:10:33 +00:00
Kegan Dougal
f2fdcb7c4b
Merge branch 'develop' into application-services
2015-02-11 16:43:26 +00:00
Kegan Dougal
f518324426
Minor tweaks based on PR feedback.
2015-02-11 16:41:16 +00:00
Erik Johnston
41a9a76a99
Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests
2015-02-11 16:12:59 +00:00
Erik Johnston
4ebbaf0d43
Blunty replace json with simplejson
2015-02-11 14:23:10 +00:00
Erik Johnston
6370cffbbf
Fix bug where variable was not always defined
2015-02-11 10:34:41 +00:00
Erik Johnston
fb233dc40b
Merge branch 'develop' of github.com:matrix-org/synapse into timeout-federation-requests
2015-02-11 10:33:19 +00:00
Erik Johnston
05b961d7e3
PEP8
2015-02-11 10:28:46 +00:00
Erik Johnston
dcf52469e8
Move time_bound_deferred into Clock
2015-02-11 10:25:06 +00:00
Erik Johnston
0e6b3e4e40
Time out HTTP federation requests
2015-02-10 18:17:27 +00:00
Kegan Dougal
53557fc532
Merge branch 'develop' into application-services
2015-02-09 15:20:56 +00:00
Erik Johnston
24cc6979fb
Log when we receive a request, when we send a response and how long it took to process it.
2015-02-09 13:46:22 +00:00
Erik Johnston
9f2573eea1
Return body of response in HttpResponseException
2015-02-06 10:55:01 +00:00
Kegan Dougal
951690e54d
Merge branch 'develop' into application-services
2015-02-05 14:28:03 +00:00
Kegan Dougal
543e84fe70
Add SimpleHttpClient.put_json with the same semantics as get_json.
2015-02-04 17:39:51 +00:00
Kegan Dougal
6d3e4f4d0a
Update user/alias query APIs to use new format of SimpleHttpClient.get_json
2015-02-04 17:32:44 +00:00
Kegan Dougal
96d4bf9012
Modify API for SimpleHttpClient.get_json and update usages.
...
Previously, this would only return the HTTP body as JSON, and discard other
response information (e.g. the HTTP response code). This has now been changed
to throw a CodeMessageException on a non-2xx response, with the response code
and body, which can then be parsed as JSON.
Affected modules include:
- Registration/Login (when using an email for IS auth)
2015-02-04 17:07:31 +00:00
Erik Johnston
ae46f10fc5
Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations
2015-02-04 16:28:12 +00:00
Erik Johnston
2ebf795c0a
Merge branch 'develop' of github.com:matrix-org/synapse into rejections
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 11:10:37 +00:00
Erik Johnston
5a3a15f5c1
Make post_json(...) actually send data.
2015-01-29 13:58:22 +00:00
Erik Johnston
c183cec8f6
Add post_json(...) method to federation client
2015-01-29 13:44:52 +00:00
David Baker
c291a4d522
Merge branch 'develop' into pushers
...
Conflicts:
synapse/handlers/events.py
synapse/server.py
2015-01-28 12:51:05 +00:00
Mark Haines
7b814d3f7f
Add client v2_alpha resource to synapse server resource tree
2015-01-23 18:55:19 +00:00
Mark Haines
e0bf18addf
Add RestServlet base class in synapse/http/servlet.py
2015-01-23 14:16:28 +00:00
David Baker
5c6189ea3e
Merge branch 'develop' into pushers
...
Conflicts:
synapse/rest/__init__.py
2015-01-22 17:46:16 +00:00
David Baker
ede491b4e0
Oops: second part of commit dc938606
2015-01-22 17:38:53 +00:00
Mark Haines
5fed042640
Finish renaming "context" to "room_id" in federation codebase
2015-01-16 19:01:03 +00:00
David Baker
c06a9063e1
Merge branch 'develop' into pushers
2015-01-13 13:15:51 +00:00
Mark Haines
c9d2cecac9
SYN-231: User agent header broken
2015-01-08 09:41:11 +00:00
Mark Haines
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
Mark Haines
773de09774
Set a content-length for JSON responses
2015-01-06 13:05:19 +00:00
Mark Haines
24b5d01853
Include version in User-Agent and Server headers
2014-12-22 10:16:02 +00:00
David Baker
b56730bb6e
Merge branch 'develop' into pushers
...
Conflicts:
synapse/api/errors.py
synapse/server.py
synapse/storage/__init__.py
2014-12-18 15:15:22 +00:00
Erik Johnston
1fc2a0e33e
Fix tests and remove debug logging
2014-12-12 15:08:29 +00:00
Erik Johnston
7b43a503f3
Consistently url decode and decode as utf 8 the URL parts
2014-12-12 15:05:37 +00:00
Mark Haines
d80d505b1f
Limit the size of images that are thumbnailed serverside. Limit the size of file that a server will download from a remote server
2014-12-11 14:19:32 +00:00
Mark Haines
4f37c0ea9d
Merge branch 'develop' into media_repository
2014-12-10 16:55:06 +00:00
Matthew Hodgson
0f4dcab238
turn back on per-request transaction retries, so that every time we try to hit a dead server we actually end up hammering 5 times :|
2014-12-10 10:28:27 +00:00
Erik Johnston
b8d30899b1
Code style.
2014-12-10 10:16:09 +00:00
Matthew Hodgson
faf12b64f8
add errbacks to enqueue_pdu deferreds; change logging for failed federation sends to warn rather than exception
2014-12-10 00:12:51 +00:00
Matthew Hodgson
2b1acb7671
squidge to 79 columns as per pep8
2014-12-10 00:03:55 +00:00
Matthew Hodgson
aed62a3583
track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover.
2014-12-07 02:26:07 +00:00
