Commit Graph

140 Commits

Author SHA1 Message Date
Richard van der Hoff
6324b65f08 Track db txn time in millisecs
... to reduce the amount of floating-point foo we do.
2018-01-16 15:53:18 +00:00
Richard van der Hoff
39f4e29d01 Reorganise request and block metrics
In order to circumvent the number of duplicate foo:count metrics increasing
without bounds, it's time for a rearrangement.

The following are all deprecated, and replaced with synapse_util_metrics_block_count:
  synapse_util_metrics_block_timer:count
  synapse_util_metrics_block_ru_utime:count
  synapse_util_metrics_block_ru_stime:count
  synapse_util_metrics_block_db_txn_count:count
  synapse_util_metrics_block_db_txn_duration:count

The following are all deprecated, and replaced with synapse_http_server_response_count:
   synapse_http_server_requests
   synapse_http_server_response_time:count
   synapse_http_server_response_ru_utime:count
   synapse_http_server_response_ru_stime:count
   synapse_http_server_response_db_txn_count:count
   synapse_http_server_response_db_txn_duration:count

The following are renamed (the old metrics are kept for now, but deprecated):

  synapse_util_metrics_block_timer:total ->
     synapse_util_metrics_block_time_seconds

  synapse_util_metrics_block_ru_utime:total ->
     synapse_util_metrics_block_ru_utime_seconds

  synapse_util_metrics_block_ru_stime:total ->
     synapse_util_metrics_block_ru_stime_seconds

  synapse_util_metrics_block_db_txn_count:total ->
     synapse_util_metrics_block_db_txn_count

  synapse_util_metrics_block_db_txn_duration:total ->
     synapse_util_metrics_block_db_txn_duration_seconds

  synapse_http_server_response_time:total ->
     synapse_http_server_response_time_seconds

  synapse_http_server_response_ru_utime:total ->
     synapse_http_server_response_ru_utime_seconds

  synapse_http_server_response_ru_stime:total ->
     synapse_http_server_response_ru_stime_seconds

   synapse_http_server_response_db_txn_count:total ->
      synapse_http_server_response_db_txn_count

   synapse_http_server_response_db_txn_duration:total
      synapse_http_server_response_db_txn_duration_seconds
2018-01-15 17:09:44 +00:00
Richard van der Hoff
d90e8ea444 Update http request metrics before calling servlet
Make sure that we set the servlet name in the metrics object *before* calling
the servlet, in case the servlet throws an exception.
2018-01-09 18:27:35 +00:00
Richard van der Hoff
6be01f599b Improve tracebacks on exceptions
Use failure.Failure to recover our failure, which will give us a useful
stacktrace, unlike the rethrown exception.
2017-11-27 12:05:58 +00:00
Richard van der Hoff
eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Erik Johnston
b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Krombel
1ff419d343 allow Authorization header which handling got implemented in #1098
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-06-16 11:21:14 +02:00
Mark Haines
b1c27975d0 Set CORs headers on responses from the media repo 2016-11-02 11:29:25 +00:00
Erik Johnston
33f3624ff7 Add exception logging. Fix typo 2016-08-22 10:49:31 +01:00
Erik Johnston
8c52160b07 Allow request handlers to override metric name 2016-08-22 10:44:45 +01:00
Erik Johnston
a093fab253 Use top level measure 2016-08-22 10:18:12 +01:00
Erik Johnston
27e0178da9 Add a top level measure 2016-08-19 18:49:37 +01:00
Erik Johnston
6d1a94d218 Remove redundant measure 2016-08-19 18:40:31 +01:00
Erik Johnston
39b900b316 Measure http.server render 2016-08-19 18:23:44 +01:00
Richard van der Hoff
33d08e8433 Log when adding listeners 2016-07-25 17:22:15 +01:00
Mark Haines
351b50a887 Fix more typos in per-request metrics 2016-04-28 15:29:46 +01:00
Mark Haines
dcfc10b129 Fix typo in request metrics 2016-04-28 15:11:06 +01:00
Mark Haines
1a12766e3b Add a comment explaining why automatic metric reporting is disabled for JsonResource 2016-04-28 12:31:26 +01:00
Mark Haines
6037349512 Check if report_metrics is True 2016-04-28 12:26:07 +01:00
Mark Haines
8d7ad44331 Report per request metrics for all of the things using request_handler 2016-04-28 10:57:49 +01:00
Erik Johnston
acdfef7b14 Intern all the things 2016-03-23 16:25:54 +00:00
Mark Haines
58c9f20692 Catch the exceptions thrown by twisted when you write to a closed connection 2016-02-12 13:46:59 +00:00
Erik Johnston
2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Erik Johnston
13e6262659 Add metrics to pushers 2016-02-08 14:26:45 +00:00
Erik Johnston
d4f72a5bfb Allowing tagging log contexts 2016-02-03 13:52:27 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall
2b0f8a9482 Fix typo 2015-12-16 17:59:44 +01:00
Daniel Wagner-Hall
338c0a8a69 Include errcode on Internal Server Error 2015-12-14 13:50:50 +00:00
Mark Haines
6a5ff5f223 Track the time spent in the database per request.
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
Mark Haines
99e1d6777f Add metrics to track the cpu on the main thread consumed by each type of request 2015-12-04 14:42:24 +00:00
Daniel Wagner-Hall
14d7acfad4 Host /unstable and /r0 versions of r0 APIs 2015-12-01 17:34:32 +00:00
Mark Haines
78323ccdb3 Remove syutil dependency in favour of smaller single-purpose libraries 2015-08-24 16:17:38 +01:00
Mark Haines
a7dcbfe430 Don't 500 if a group is missing from the regex 2015-07-06 16:47:17 +01:00
Erik Johnston
aaa749d366 Disable twisted access logging. Move access logging to SynapseRequest object 2015-06-15 18:18:05 +01:00
Erik Johnston
cee69441d3 Log more when we have processed the request 2015-06-15 17:11:44 +01:00
Erik Johnston
b5209c5744 Create SynapseRequest that overrides __repr__ to not print access_token 2015-06-15 16:37:04 +01:00
Erik Johnston
f00f8346f1 Make http.server request logging more verbose, but redact access_tokens 2015-06-15 13:37:58 +01:00
Erik Johnston
d15f166093 Remove log line 2015-05-29 15:03:24 +01:00
Erik Johnston
a7b65bdedf Add config option to turn off freezing events. Use new encode_json api and ujson.loads 2015-05-29 12:17:33 +01:00
Erik Johnston
2236ef6c92 Fix up leak. Add warnings. 2015-05-08 19:53:34 +01:00
David Baker
0c1b7f843b Unused import 2015-04-30 13:33:30 +01:00
David Baker
4b46fbec5b Doesn't look like this is used anymore 2015-04-30 12:04:08 +01:00
David Baker
6532b6e607 Merge branch 'develop' into csauth
Conflicts:
	synapse/http/server.py
2015-04-24 09:37:54 +01:00
Mark Haines
1967650bc4 Combine the request wrappers in rest/media/v1 and http/server into a single wrapper decorator 2015-04-21 16:35:53 +01:00
Mark Haines
1ebff9736b Split out the JsonResource request logging and error handling into a separate wrapper function 2015-04-21 16:07:20 +01:00
David Baker
c5bf0343e8 Explain how I justified to myself making JsonResource not always send JSON. 2015-04-01 15:13:14 +01:00
David Baker
e9c908ebc0 Completely replace fallback auth for C/S V2:
* Now only the auth part goes to fallback, not the whole operation
 * Auth fallback is a normal API endpoint, not a static page
 * Params like the recaptcha pubkey can just live in the config
Involves a little engineering on JsonResource so its servlets aren't always forced to return JSON. I should document this more, in fact I'll do that now.
2015-04-01 15:05:30 +01:00
David Baker
f129ee1e18 Make docs a bit more true 2015-03-31 18:25:10 +01:00
David Baker
d18e7779ca Grammar and deduplication 2015-03-31 14:40:02 +01:00
Paul "LeoNerd" Evans
3f7a31d366 Add a DistributionMetric to HTTP request/response processing time in the server 2015-03-16 18:31:29 +00:00
Matthew Hodgson
c8032aec17 actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0' 2015-03-14 00:12:20 +00:00
Paul "LeoNerd" Evans
128cf2daf7 Appease pep8 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
1748605c5d Count incoming HTTP requests per servlet that responds 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
b0cf867319 Use _ instead of . as a metric namespacing separator, for Prometheus 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
f9478e475b Rename Metrics' "keys" to "labels" 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans
7d72e44eb9 Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans
9d9d39536b Slightly reduce the insane amounts of indentation in main http server response path, by 'continue'ing around a non-match or falling through 2015-03-05 16:24:13 +00:00
Erik Johnston
5358966a87 Use git aware version string in User-Agent and Server headers 2015-02-18 16:52:04 +00:00
Erik Johnston
6370cffbbf Fix bug where variable was not always defined 2015-02-11 10:34:41 +00:00
Erik Johnston
24cc6979fb Log when we receive a request, when we send a response and how long it took to process it. 2015-02-09 13:46:22 +00:00
David Baker
ede491b4e0 Oops: second part of commit dc938606 2015-01-22 17:38:53 +00:00
Mark Haines
adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Mark Haines
773de09774 Set a content-length for JSON responses 2015-01-06 13:05:19 +00:00
Mark Haines
24b5d01853 Include version in User-Agent and Server headers 2014-12-22 10:16:02 +00:00
Erik Johnston
1fc2a0e33e Fix tests and remove debug logging 2014-12-12 15:08:29 +00:00
Erik Johnston
7b43a503f3 Consistently url decode and decode as utf 8 the URL parts 2014-12-12 15:05:37 +00:00
Mark Haines
5da65085d1 Get uploads working with new media repo 2014-12-02 19:51:47 +00:00
Mark Haines
279c48c8b4 Write the upload portion of version 1 of the media repository 2014-12-02 17:13:14 +00:00
Mark Haines
db9ce032a4 Fix pep8 codestyle warnings 2014-11-20 17:26:36 +00:00
Mark Haines
dfdda2c871 Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str' 2014-11-20 17:10:37 +00:00
Mark Haines
b29517bd01 Add a request-id to each log line 2014-10-30 01:21:33 +00:00
Matthew Hodgson
8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Mark Haines
bf10cf5f1a move contentrepo class to it's own file 2014-09-03 11:10:44 +01:00
Mark Haines
112c7ea315 Set status message for ratelimit error responses 2014-09-03 09:37:44 +01:00
Matthew Hodgson
68a04b9282 given everything's apparently gone https:// by default for c-s, change this hardcoding... 2014-09-01 23:28:43 +01:00
Matthew Hodgson
2b7918bd6f missed a s#/matrix#/_matrix/g 2014-08-31 14:54:58 +01:00
Erik Johnston
0c3b4a1f63 For the content repo, don't just use homeserver.hostname as that might not include the port due to SRV. 2014-08-24 11:56:55 +01:00
Erik Johnston
acf5127604 Make the content repo work with in daemon mode. Return the full url on upload. Update the webclient to use new content repo api. 2014-08-22 10:25:32 +01:00
Kegan Dougal
f48792eec4 Reduce the amount of incredibly spammy stack traces. Expected errors (e.g. SynapseErrors) shouldn't have their full trace logged every time. Don't send responses to disconnected requests. 2014-08-19 10:56:43 +01:00
Kegan Dougal
e37b040bc3 Small amounts of cleanup and bonus round comments. 2014-08-18 17:22:31 +01:00
Kegan Dougal
58548ab557 Implemented GETs for the ContentRepoResource. It all actually appears to be working. 2014-08-18 17:18:54 +01:00
Kegan Dougal
590ab24c85 hs: Make the uploads directory if it doesn't exist. Namespace uploads by the base64 encoded user id of the uploader. Make a reasonable attempt to retry clashing upload paths. Try to guess a sensible file extension depending on the content type. 2014-08-18 17:18:54 +01:00
Kegan Dougal
35da1bf4a3 Auth content uploads. Added a mapping function from request > filename. Added exception handling for content uploads. webclient: Only prefix the client API path on doRequest, not doBaseRequest (this would've broken the identity server auth too). Added matrixService.uploadContent. May not require mFileUpload anymore. 2014-08-18 17:18:54 +01:00
Kegan Dougal
a18b1a649c Added /matrix/content path, HS resource_for_content_repo attribute and FileUploadResource. Added stub methods. 2014-08-18 17:18:54 +01:00
Kegan Dougal
9fd445eb92 If the web client is enabled, automatically redirect root '/' to the web client path. 2014-08-14 11:37:13 +01:00
Kegan Dougal
9a1638ed21 Removed http_server from HomeServer. Updated unit tests to use either resource_for_federation or resource_for_client depending on what is being tested. 2014-08-14 10:18:54 +01:00
Kegan Dougal
9f863d3466 Start phasing out HttpServer: we should be using Resources instead. Added resource_for_client/federation/web_client to the HomeServer and hooked the C-S servlets to operate on resource_for_client. Dynamically construct the Resource tree. 2014-08-14 09:55:16 +01:00
Matthew Hodgson
f98e6380f1 add in whitespace after copyright statements to improve legibility 2014-08-13 03:14:34 +01:00
Erik Johnston
4add1c70e9 Just say yes to OPTIONS requests, since we want to turn on CORS headers for all paths 2014-08-12 17:17:14 +01:00
matrix.org
4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00