Commit Graph

97 Commits

Author SHA1 Message Date
Patrick Cloke
d8cc86eff4
Remove redundant types from comments. ()
Remove type hints from comments which have been added
as Python type hints. This helps avoid drift between comments
and reality, as well as removing redundant information.

Also adds some missing type hints which were simple to fill in.
2022-11-16 15:25:24 +00:00
Dirk Klimpel
1357ae869f
Add workers settings to configuration manual ()
* Add workers settings to configuration manual
* Update `pusher_instances`
* update url to python logger
* update headlines
* update links after headline change
* remove link from `daemon process`

There is no docs in Synapse for this

* extend example for `federation_sender_instances` and `pusher_instances`
* more infos about stream writers
* add link to DAG
* update `pusher_instances`
* update `worker_listeners`
* update `stream_writers`
* Update `worker_name`

Co-authored-by: David Robertson <davidr@element.io>
2022-10-27 14:39:47 +01:00
Quentin Gliech
dc37b68a25
Parse SYNAPSE_ASYNC_IO_REACTOR env variable & log the reactor on startup () 2022-10-07 15:19:59 +01:00
Shay
493c2fc44a
Remove code generating comments in configuration file () 2022-06-14 07:53:42 -07:00
David Robertson
f30bcbd84a
Fix Synapse git info missing in version strings () 2022-06-07 15:24:11 +01:00
Richard van der Hoff
8d156ec0ba
Remove special-case for twisted logger ()
This was originally added when we first added a `MemoryHandler` to the default
log config back in https://github.com/matrix-org/synapse/pull/8040, to ensure
that we didn't explode with an infinite loop if there was an error formatting
the logs.

Since then, we made additional improvements to logging which make this
workaround redundant. In particular:

 * we no longer attempt to log un-UTF8-decodable byte sequences, which were the
   most likely cause of an error in the first place.

 * https://github.com/matrix-org/synapse/pull/8268 ensures that in the unlikely
   case that there *is* an error, it won't cause an infinite loop.
2022-04-29 22:05:18 +01:00
Patrick Cloke
4586119f0b
Add missing type hints to config classes. () 2022-04-11 12:07:23 -04:00
Patrick Cloke
e69f8f0a8e
Remove support for the legacy structured logging configuration. () 2022-02-17 13:32:18 +00:00
David Robertson
4ae956c8bb
Use version string helper from matrix-common ()
* Require latest matrix-common
* Use the common function
2022-02-14 13:12:22 +00:00
Patrick Cloke
f44d729d4c
Additional type hints for config module. ()
This adds some misc. type hints to helper methods used
in the `synapse.config` module.
2021-12-01 07:28:23 -05:00
Patrick Cloke
55669bd3de
Add missing type hints to config base classes () 2021-11-23 15:21:19 +00:00
Sean Quah
2b82ec425f
Add type hints for most HomeServer parameters () 2021-10-22 18:15:41 +01:00
Patrick Cloke
bb7fdd821b
Use direct references for configuration variables (part 5). () 2021-09-24 07:25:21 -04:00
Patrick Cloke
47854c71e9
Use direct references for configuration variables (part 4). () 2021-09-23 12:03:01 -04:00
Patrick Cloke
01c88a09cd
Use direct references for some configuration variables ()
Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).
2021-09-13 13:07:12 -04:00
reivilibre
5f7b1e1f27
Make PeriodicallyFlushingMemoryHandler the default logging handler. () 2021-08-17 13:13:11 +01:00
reivilibre
f4ac934afe
Revert use of PeriodicallyFlushingMemoryHandler by default () 2021-08-03 11:30:39 +01:00
reivilibre
e16eab29d6
Add a PeriodicallyFlushingMemoryHandler to prevent logging silence ()
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2021-07-27 14:32:05 +01:00
Dirk Klimpel
56fd5fa8e1
Update links to documentation in sample config ()
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-07-07 12:35:45 +01:00
Richard van der Hoff
107c06081f
Ensure that errors during startup are written to the logs and the console. ()
* Defer stdio redirection until we are about to start the reactor

* Catch and handle exceptions during startup
2021-06-21 11:41:25 +01:00
Richard van der Hoff
3ff2251754
Improved validation for received requests ()
* Simplify `start_listening` callpath

* Correctly check the size of uploaded files
2021-04-23 19:20:44 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ()
Part of 

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke
7fdc6cefb3
Fix additional type hints. ()
Type hint fixes due to Twisted 21.2.0 adding type hints.
2021-03-09 07:41:32 -05:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase ()
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick Cloke
1619802228
Various clean-ups to the logging context code () 2020-12-14 14:19:47 -05:00
Patrick Cloke
00b24aa545
Support generating structured logs in addition to standard logs. ()
This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.

This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.
2020-10-29 07:27:37 -04:00
Patrick Cloke
34a5696f93
Fix typos and spelling errors. () 2020-10-23 12:38:40 -04:00
Richard van der Hoff
77794ebc77
Fix stack overflow when logging system encounters an error () 2020-09-07 16:54:30 +01:00
Erik Johnston
0304ad0c3d Move setting of Filter into code.
We do this to prevent foot guns. The default config uses a MemoryFilter,
but users are free to change to logging to files directly. If they do
then they have to ensure to set the `filters: [context]` on the right
handler, otherwise records get written with the wrong context.

Instead we move the logic to happen when we generate a record, which is
when we *log* rather than *handle*.

(It's possible to add filters to loggers in the config, however they
don't apply to descendant loggers and so they have to be manually set on
*every* logger used in the code base)
2020-08-11 18:10:46 +01:00
Erik Johnston
db131b6b22
Change the default log config to reduce disk I/O and storage ()
* Change default log config to buffer by default.

This batches up writes to the filesystem, which is more efficient for
disk I/O. This means that it can take some time for logs to get written
to disk. Note that ERROR logs (and above) immediately flush the buffer.

This only effects new installs, as we only write the log config if
started with `--generate-config` (in the same way we do for generating
signing keys).

* Default to keeping last 4 days of logs.

This hopefully reduces the amount of logs kept for new servers. Keeping
the last 1GB of logs is likely overkill for new servers, but equally may
not be enough for busy ones.

Instead, we keep the last four days worth of logs, enough so that admins
can investigate any problems that happened over e.g. a long weekend.
2020-08-11 18:09:46 +01:00
Erik Johnston
2c1b9d6763
Update worker docs with recent enhancements () 2020-07-29 23:22:13 +01:00
Erik Johnston
e5c67d04db
Add option to move event persistence off master () 2020-05-22 16:11:35 +01:00
Richard van der Hoff
08815566bc
Automate generation of the sample and debian log configs () 2020-01-03 17:14:00 +00:00
Richard van der Hoff
e484101306
Raise an error if someone tries to use the log_file config option ()
This has caused some confusion for people who didn't notice it going away.
2020-01-03 17:11:29 +00:00
Andrew Morgan
fe1f2b4520
Remove last usages of deprecated logging.warn method () 2019-11-01 12:03:44 +00:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase ()
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Richard van der Hoff
560c122267
Fix logging config for the docker image ()
Turns out that loggers that are instantiated before the config is loaded get
turned off.

Also bring the logging config that is generated by --generate-config into line.

Fixes .
2019-10-18 13:34:33 +02:00
Amber Brown
f743108a94
Refactor HomeserverConfig so it can be typechecked () 2019-10-10 09:39:35 +01:00
Amber Brown
b617864cd9
Fix for structured logging tests stomping on logs () 2019-09-13 02:29:55 +10:00
Jorik Schellekens
92c1550f4a Add a link to python's logging config schema () 2019-08-28 19:08:32 +01:00
Amber Brown
7dc398586c
Implement a structured logging output system. () 2019-08-28 21:18:53 +10:00
Amber Brown
6a85cb5ef7
Remove non-dedicated logging options and command line arguments () 2019-07-19 01:40:08 +10:00
Erik Johnston
e8c53b07f2 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_api_cmd 2019-07-15 14:13:22 +01:00
Erik Johnston
823e13ddf4 Change add_arguments to be a static method 2019-07-15 14:09:33 +01:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ () 2019-07-04 00:07:04 +10:00
Richard van der Hoff
16b52642e2 Don't load the generated config as the default.
It's too confusing.
2019-06-24 14:14:52 +01:00
Richard van der Hoff
7c2f8881a9 Ensure that all config options have sensible defaults
This will enable us to skip the unintuitive behaviour where the generated
config and default config are the same thing.
2019-06-24 14:14:52 +01:00
Richard van der Hoff
c3c6b00d95
Pass config_dir_path and data_dir_path into Config.read_config. ()
* Pull config_dir_path and data_dir_path calculation out of read_config_files

* Pass config_dir_path and data_dir_path into read_config
2019-06-24 11:34:45 +01:00
Richard van der Hoff
37933a3bf8
Improve logging when generating config files ()
Make it a bit clearer what's going on.
2019-06-21 17:14:56 +01:00
Amber Brown
32e7c9e7f2
Run Black. () 2019-06-20 19:32:02 +10:00