Erik Johnston
926ba76e23
Replace ujson with simplejson
2018-03-15 23:43:31 +00:00
Richard van der Hoff
58dd148c4f
Add some docstrings to help figure this out
2018-03-09 18:05:41 +00:00
Richard van der Hoff
88541f9009
Add a metric which increments when a request is received
...
It's useful to know when there are peaks in incoming requests - which isn't
quite the same as there being peaks in outgoing responses, due to the time
taken to handle requests.
2018-03-09 16:30:26 +00:00
Richard van der Hoff
dbe80a286b
refactor JsonResource
...
rephrase the OPTIONS and unrecognised request handling so that they look
similar to the common flow.
2018-03-09 16:22:16 +00:00
Richard van der Hoff
e8f7541d3f
Merge remote-tracking branch 'origin/develop' into rav/track_db_scheduling
2018-01-17 14:01:57 +00:00
Richard van der Hoff
fb6563b4be
Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millis
...
Track db txn time in millisecs
2018-01-17 13:52:42 +00:00
Richard van der Hoff
936482d507
Fix 'NoneType' object has no attribute 'writeHeaders'
...
Avoid throwing a (harmless) exception when we try to write an error response to
an http request where the client has disconnected.
This comes up as a CRITICAL error in the logs which tends to mislead people
into thinking there's an actual problem
2018-01-16 17:58:16 +00:00
Richard van der Hoff
3d12d97415
Track DB scheduling delay per-request
...
For each request, track the amount of time spent waiting for a db
connection. This entails adding it to the LoggingContext and we may as well add
metrics for it while we are passing.
2018-01-16 17:23:32 +00:00
Richard van der Hoff
6324b65f08
Track db txn time in millisecs
...
... to reduce the amount of floating-point foo we do.
2018-01-16 15:53:18 +00:00
Richard van der Hoff
39f4e29d01
Reorganise request and block metrics
...
In order to circumvent the number of duplicate foo:count metrics increasing
without bounds, it's time for a rearrangement.
The following are all deprecated, and replaced with synapse_util_metrics_block_count:
synapse_util_metrics_block_timer:count
synapse_util_metrics_block_ru_utime:count
synapse_util_metrics_block_ru_stime:count
synapse_util_metrics_block_db_txn_count:count
synapse_util_metrics_block_db_txn_duration:count
The following are all deprecated, and replaced with synapse_http_server_response_count:
synapse_http_server_requests
synapse_http_server_response_time:count
synapse_http_server_response_ru_utime:count
synapse_http_server_response_ru_stime:count
synapse_http_server_response_db_txn_count:count
synapse_http_server_response_db_txn_duration:count
The following are renamed (the old metrics are kept for now, but deprecated):
synapse_util_metrics_block_timer:total ->
synapse_util_metrics_block_time_seconds
synapse_util_metrics_block_ru_utime:total ->
synapse_util_metrics_block_ru_utime_seconds
synapse_util_metrics_block_ru_stime:total ->
synapse_util_metrics_block_ru_stime_seconds
synapse_util_metrics_block_db_txn_count:total ->
synapse_util_metrics_block_db_txn_count
synapse_util_metrics_block_db_txn_duration:total ->
synapse_util_metrics_block_db_txn_duration_seconds
synapse_http_server_response_time:total ->
synapse_http_server_response_time_seconds
synapse_http_server_response_ru_utime:total ->
synapse_http_server_response_ru_utime_seconds
synapse_http_server_response_ru_stime:total ->
synapse_http_server_response_ru_stime_seconds
synapse_http_server_response_db_txn_count:total ->
synapse_http_server_response_db_txn_count
synapse_http_server_response_db_txn_duration:total
synapse_http_server_response_db_txn_duration_seconds
2018-01-15 17:09:44 +00:00
Richard van der Hoff
d90e8ea444
Update http request metrics before calling servlet
...
Make sure that we set the servlet name in the metrics object *before* calling
the servlet, in case the servlet throws an exception.
2018-01-09 18:27:35 +00:00
Richard van der Hoff
6be01f599b
Improve tracebacks on exceptions
...
Use failure.Failure to recover our failure, which will give us a useful
stacktrace, unlike the rethrown exception.
2017-11-27 12:05:58 +00:00
Richard van der Hoff
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Erik Johnston
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
Krombel
1ff419d343
allow Authorization header which handling got implemented in #1098
...
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-06-16 11:21:14 +02:00
Mark Haines
b1c27975d0
Set CORs headers on responses from the media repo
2016-11-02 11:29:25 +00:00
Erik Johnston
33f3624ff7
Add exception logging. Fix typo
2016-08-22 10:49:31 +01:00
Erik Johnston
8c52160b07
Allow request handlers to override metric name
2016-08-22 10:44:45 +01:00
Erik Johnston
a093fab253
Use top level measure
2016-08-22 10:18:12 +01:00
Erik Johnston
27e0178da9
Add a top level measure
2016-08-19 18:49:37 +01:00
Erik Johnston
6d1a94d218
Remove redundant measure
2016-08-19 18:40:31 +01:00
Erik Johnston
39b900b316
Measure http.server render
2016-08-19 18:23:44 +01:00
Richard van der Hoff
33d08e8433
Log when adding listeners
2016-07-25 17:22:15 +01:00
Mark Haines
351b50a887
Fix more typos in per-request metrics
2016-04-28 15:29:46 +01:00
Mark Haines
dcfc10b129
Fix typo in request metrics
2016-04-28 15:11:06 +01:00
Mark Haines
1a12766e3b
Add a comment explaining why automatic metric reporting is disabled for JsonResource
2016-04-28 12:31:26 +01:00
Mark Haines
6037349512
Check if report_metrics is True
2016-04-28 12:26:07 +01:00
Mark Haines
8d7ad44331
Report per request metrics for all of the things using request_handler
2016-04-28 10:57:49 +01:00
Erik Johnston
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
Mark Haines
58c9f20692
Catch the exceptions thrown by twisted when you write to a closed connection
2016-02-12 13:46:59 +00:00
Erik Johnston
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
Erik Johnston
13e6262659
Add metrics to pushers
2016-02-08 14:26:45 +00:00
Erik Johnston
d4f72a5bfb
Allowing tagging log contexts
2016-02-03 13:52:27 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall
2b0f8a9482
Fix typo
2015-12-16 17:59:44 +01:00
Daniel Wagner-Hall
338c0a8a69
Include errcode on Internal Server Error
2015-12-14 13:50:50 +00:00
Mark Haines
6a5ff5f223
Track the time spent in the database per request.
...
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
Mark Haines
99e1d6777f
Add metrics to track the cpu on the main thread consumed by each type of request
2015-12-04 14:42:24 +00:00
Daniel Wagner-Hall
14d7acfad4
Host /unstable and /r0 versions of r0 APIs
2015-12-01 17:34:32 +00:00
Mark Haines
78323ccdb3
Remove syutil dependency in favour of smaller single-purpose libraries
2015-08-24 16:17:38 +01:00
Mark Haines
a7dcbfe430
Don't 500 if a group is missing from the regex
2015-07-06 16:47:17 +01:00
Erik Johnston
aaa749d366
Disable twisted access logging. Move access logging to SynapseRequest object
2015-06-15 18:18:05 +01:00
Erik Johnston
cee69441d3
Log more when we have processed the request
2015-06-15 17:11:44 +01:00
Erik Johnston
b5209c5744
Create SynapseRequest that overrides __repr__ to not print access_token
2015-06-15 16:37:04 +01:00
Erik Johnston
f00f8346f1
Make http.server request logging more verbose, but redact access_tokens
2015-06-15 13:37:58 +01:00
Erik Johnston
d15f166093
Remove log line
2015-05-29 15:03:24 +01:00
Erik Johnston
a7b65bdedf
Add config option to turn off freezing events. Use new encode_json api and ujson.loads
2015-05-29 12:17:33 +01:00
Erik Johnston
2236ef6c92
Fix up leak. Add warnings.
2015-05-08 19:53:34 +01:00
David Baker
0c1b7f843b
Unused import
2015-04-30 13:33:30 +01:00
David Baker
4b46fbec5b
Doesn't look like this is used anymore
2015-04-30 12:04:08 +01:00
David Baker
6532b6e607
Merge branch 'develop' into csauth
...
Conflicts:
synapse/http/server.py
2015-04-24 09:37:54 +01:00
Mark Haines
1967650bc4
Combine the request wrappers in rest/media/v1 and http/server into a single wrapper decorator
2015-04-21 16:35:53 +01:00
Mark Haines
1ebff9736b
Split out the JsonResource request logging and error handling into a separate wrapper function
2015-04-21 16:07:20 +01:00
David Baker
c5bf0343e8
Explain how I justified to myself making JsonResource not always send JSON.
2015-04-01 15:13:14 +01:00
David Baker
e9c908ebc0
Completely replace fallback auth for C/S V2:
...
* Now only the auth part goes to fallback, not the whole operation
* Auth fallback is a normal API endpoint, not a static page
* Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
2015-04-01 15:05:30 +01:00
David Baker
f129ee1e18
Make docs a bit more true
2015-03-31 18:25:10 +01:00
David Baker
d18e7779ca
Grammar and deduplication
2015-03-31 14:40:02 +01:00
Paul "LeoNerd" Evans
3f7a31d366
Add a DistributionMetric to HTTP request/response processing time in the server
2015-03-16 18:31:29 +00:00
Matthew Hodgson
c8032aec17
actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0'
2015-03-14 00:12:20 +00:00
Paul "LeoNerd" Evans
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
1748605c5d
Count incoming HTTP requests per servlet that responds
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b0cf867319
Use _ instead of . as a metric namespacing separator, for Prometheus
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
7d72e44eb9
Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code
2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
9d9d39536b
Slightly reduce the insane amounts of indentation in main http server response path, by 'continue'ing around a non-match or falling through
2015-03-05 16:24:13 +00:00
Erik Johnston
5358966a87
Use git aware version string in User-Agent and Server headers
2015-02-18 16:52:04 +00:00
Erik Johnston
6370cffbbf
Fix bug where variable was not always defined
2015-02-11 10:34:41 +00:00
Erik Johnston
24cc6979fb
Log when we receive a request, when we send a response and how long it took to process it.
2015-02-09 13:46:22 +00:00
David Baker
ede491b4e0
Oops: second part of commit dc938606
2015-01-22 17:38:53 +00:00
Mark Haines
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
Mark Haines
773de09774
Set a content-length for JSON responses
2015-01-06 13:05:19 +00:00
Mark Haines
24b5d01853
Include version in User-Agent and Server headers
2014-12-22 10:16:02 +00:00
Erik Johnston
1fc2a0e33e
Fix tests and remove debug logging
2014-12-12 15:08:29 +00:00
Erik Johnston
7b43a503f3
Consistently url decode and decode as utf 8 the URL parts
2014-12-12 15:05:37 +00:00
Mark Haines
5da65085d1
Get uploads working with new media repo
2014-12-02 19:51:47 +00:00
Mark Haines
279c48c8b4
Write the upload portion of version 1 of the media repository
2014-12-02 17:13:14 +00:00
Mark Haines
db9ce032a4
Fix pep8 codestyle warnings
2014-11-20 17:26:36 +00:00
Mark Haines
dfdda2c871
Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str'
2014-11-20 17:10:37 +00:00
Mark Haines
b29517bd01
Add a request-id to each log line
2014-10-30 01:21:33 +00:00
Matthew Hodgson
8a7c1d6a00
fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch.
2014-09-03 17:31:57 +01:00
Mark Haines
bf10cf5f1a
move contentrepo class to it's own file
2014-09-03 11:10:44 +01:00
Mark Haines
112c7ea315
Set status message for ratelimit error responses
2014-09-03 09:37:44 +01:00
Matthew Hodgson
68a04b9282
given everything's apparently gone https:// by default for c-s, change this hardcoding...
2014-09-01 23:28:43 +01:00
Matthew Hodgson
2b7918bd6f
missed a s#/matrix#/_matrix/g
2014-08-31 14:54:58 +01:00
Erik Johnston
0c3b4a1f63
For the content repo, don't just use homeserver.hostname as that might not include the port due to SRV.
2014-08-24 11:56:55 +01:00
Erik Johnston
acf5127604
Make the content repo work with in daemon mode. Return the full url on upload. Update the webclient to use new content repo api.
2014-08-22 10:25:32 +01:00
Kegan Dougal
f48792eec4
Reduce the amount of incredibly spammy stack traces. Expected errors (e.g. SynapseErrors) shouldn't have their full trace logged every time. Don't send responses to disconnected requests.
2014-08-19 10:56:43 +01:00
Kegan Dougal
e37b040bc3
Small amounts of cleanup and bonus round comments.
2014-08-18 17:22:31 +01:00
Kegan Dougal
58548ab557
Implemented GETs for the ContentRepoResource. It all actually appears to be working.
2014-08-18 17:18:54 +01:00
Kegan Dougal
590ab24c85
hs: Make the uploads directory if it doesn't exist. Namespace uploads by the base64 encoded user id of the uploader. Make a reasonable attempt to retry clashing upload paths. Try to guess a sensible file extension depending on the content type.
2014-08-18 17:18:54 +01:00
Kegan Dougal
35da1bf4a3
Auth content uploads. Added a mapping function from request > filename. Added exception handling for content uploads. webclient: Only prefix the client API path on doRequest, not doBaseRequest (this would've broken the identity server auth too). Added matrixService.uploadContent. May not require mFileUpload anymore.
2014-08-18 17:18:54 +01:00
Kegan Dougal
a18b1a649c
Added /matrix/content path, HS resource_for_content_repo attribute and FileUploadResource. Added stub methods.
2014-08-18 17:18:54 +01:00
Kegan Dougal
9fd445eb92
If the web client is enabled, automatically redirect root '/' to the web client path.
2014-08-14 11:37:13 +01:00
Kegan Dougal
9a1638ed21
Removed http_server from HomeServer. Updated unit tests to use either resource_for_federation or resource_for_client depending on what is being tested.
2014-08-14 10:18:54 +01:00
Kegan Dougal
9f863d3466
Start phasing out HttpServer: we should be using Resources instead. Added resource_for_client/federation/web_client to the HomeServer and hooked the C-S servlets to operate on resource_for_client. Dynamically construct the Resource tree.
2014-08-14 09:55:16 +01:00
Matthew Hodgson
f98e6380f1
add in whitespace after copyright statements to improve legibility
2014-08-13 03:14:34 +01:00
Erik Johnston
4add1c70e9
Just say yes to OPTIONS requests, since we want to turn on CORS headers for all paths
2014-08-12 17:17:14 +01:00
matrix.org
4f475c7697
Reference Matrix Home Server
2014-08-12 15:10:52 +01:00
