Commit Graph

10818 Commits

Author SHA1 Message Date
Erik Johnston
c435b0b441 Don't store context 2018-05-22 09:34:26 +01:00
Erik Johnston
fb2806b186 Move in_flight_requests_count to be a callback metric 2018-05-22 09:31:53 +01:00
Erik Johnston
dfa70adc33 Add in flight request metrics
This tracks CPU and DB usage while requests are in flight, rather than
when we write the response.
2018-05-21 16:23:06 +01:00
Neil Johnson
08462620bf
Merge pull request #3241 from matrix-org/fix_user_visits_insertion
fix psql compatability bug
2018-05-18 15:01:01 +00:00
Neil Johnson
ef466b3a13 fix psql compatability bug 2018-05-18 15:51:21 +01:00
Neil Johnson
2725223f08 Merge branch 'master' into develop 2018-05-18 14:07:50 +01:00
Neil Johnson
ab5e888927 Changes in synapse v0.29.1 (2018-05-17)
==========================================
 Changes:
 
 * Update docker documentation (PR #3222)
 
 Changes in synapse v0.29.0 (2018-05-16)
 ===========================================
 Not changes since v0.29.0-rc1
 
 Changes in synapse v0.29.0-rc1 (2018-05-14)
 ===========================================
 
 Notable changes, a docker file for running Synapse (Thanks to @kaiyou!) and a
 closed spec bug in the Client Server API. Additionally further prep for Python 3
 migration.
 
 Potentially breaking change:
 
 * Make Client-Server API return 401 for invalid token (PR #3161).
 
   This changes the Client-server spec to return a 401 error code instead of 403
   when the access token is unrecognised. This is the behaviour required by the
   specification, but some clients may be relying on the old, incorrect
   behaviour.
 
   Thanks to @NotAFile for fixing this.
 
 Features:
 
 * Add a Dockerfile for synapse (PR #2846) Thanks to @kaiyou!
 
 Changes - General:
 
 * nuke-room-from-db.sh: added postgresql option and help (PR #2337) Thanks to @rubo77!
 * Part user from rooms on account deactivate (PR #3201)
 * Make 'unexpected logging context' into warnings (PR #3007)
 * Set Server header in SynapseRequest (PR #3208)
 * remove duplicates from groups tables (PR #3129)
 * Improve exception handling for background processes (PR #3138)
 * Add missing consumeErrors to improve exception handling (PR #3139)
 * reraise exceptions more carefully (PR #3142)
 * Remove redundant call to preserve_fn (PR #3143)
 * Trap exceptions thrown within run_in_background (PR #3144)
 
 Changes - Refactors:
 
 * Refactor /context to reuse pagination storage functions (PR #3193)
 * Refactor recent events func to use pagination func (PR #3195)
 * Refactor pagination DB API to return concrete type (PR #3196)
 * Refactor get_recent_events_for_room return type (PR #3198)
 * Refactor sync APIs to reuse pagination API (PR #3199)
 * Remove unused code path from member change DB func (PR #3200)
 * Refactor request handling wrappers (PR #3203)
 * transaction_id, destination defined twice (PR #3209) Thanks to @damir-manapov!
 * Refactor event storage to prepare for changes in state calculations (PR #3141)
 * Set Server header in SynapseRequest (PR #3208)
 * Use deferred.addTimeout instead of time_bound_deferred (PR #3127, #3178)
 * Use run_in_background in preference to preserve_fn (PR #3140)
 
 Changes - Python 3 migration:
 
 * Construct HMAC as bytes on py3 (PR #3156) Thanks to @NotAFile!
 * run config tests on py3 (PR #3159) Thanks to @NotAFile!
 * Open certificate files as bytes (PR #3084) Thanks to @NotAFile!
 * Open config file in non-bytes mode (PR #3085) Thanks to @NotAFile!
 * Make event properties raise AttributeError instead (PR #3102) Thanks to @NotAFile!
 * Use six.moves.urlparse (PR #3108) Thanks to @NotAFile!
 * Add py3 tests to tox with folders that work (PR #3145) Thanks to @NotAFile!
 * Don't yield in list comprehensions (PR #3150) Thanks to @NotAFile!
 * Move more xrange to six (PR #3151) Thanks to @NotAFile!
 * make imports local (PR #3152) Thanks to @NotAFile!
 * move httplib import to six (PR #3153) Thanks to @NotAFile!
 * Replace stringIO imports with six (PR #3154, #3168) Thanks to @NotAFile!
 * more bytes strings (PR #3155) Thanks to @NotAFile!
 
 Bug Fixes:
 
 * synapse fails to start under Twisted >= 18.4 (PR #3157)
 * Fix a class of logcontext leaks (PR #3170)
 * Fix a couple of logcontext leaks in unit tests (PR #3172)
 * Fix logcontext leak in media repo (PR #3174)
 * Escape label values in prometheus metrics (PR #3175, #3186)
 * Fix 'Unhandled Error' logs with Twisted 18.4 (PR #3182) Thanks to @Half-Shot!
 * Fix logcontext leaks in rate limiter (PR #3183)
 * notifications: Convert next_token to string according to the spec (PR #3190) Thanks to @mujx!
 * nuke-room-from-db.sh: fix deletion from search table (PR #3194) Thanks to @rubo77!
 * add guard for None on purge_history api (PR #3160) Thanks to @krombel!
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEETQ1YthIGLQRddG54CTxDAAxPS/QFAlr+zv4ACgkQCTxDAAxP
 S/TKxQ/+OMQQJzk7c398zX+wFav0tJO7wSNEtsQ0lkBJq9Yy1Geaz+cXjxoLgPOT
 hthXhLZLVilONx13pnqsJCWrxufiBzWxbP9e7vnvWhP+v1uZca6kp1Pw4fpjGfl6
 DHgCXc3E2/UVDzk0aSIBdufs39qjp/WaUdcT7QuL16uCVwfX0K79nfZtGG+l2KeQ
 fi0SVwK58uZDcSwTrAB39Q3YITZrJzW8Pz1CRkmvG5efWH8KaRtnC/3UZvDb9kBl
 5xGsEkzj7A826PdMXp3ZXPc9H682NYpSD37545XU0ZqddR+RFNofZj0rRH464R+v
 lGVjHYqiiKMeX7y5VF7jZkMsb1j917txP5bcSwS+r2JtymG+RSxBQzUHNiu5gCaC
 MclIY6LbFGT9zJEzTQYBzOYzYA9RvjvlfqbdLV4D5R1Deb5z2VsjRJW7gqpWTj7c
 VpqkF7FP14JqjojMmciHa5b02GC5DPQ/ZSbL8Gp76DKvErbltYEC8GEgFx3eRJLT
 Dru/xcUQcSLUrx4DCk9ViHzpd9RBOGwERmdWilFNW/4HzXb/SK8CjEpWPNSENo+9
 ks3lRSa4J39U0Y/03c3wZMTKQFAQdIcqvAI3bJrlARa5OotvKbN1If5yf2nn1ZmM
 +JaVuV7KLqNMO0Z/O6R5DyuX6j70WAprkddExwGsb7Ha9Uq8Jig=
 =Fpbo
 -----END PGP SIGNATURE-----

Merge tag 'v0.29.1'

Changes in synapse v0.29.1 (2018-05-17)
==========================================
Changes:

* Update docker documentation (PR #3222)

Changes in synapse v0.29.0 (2018-05-16)
===========================================
Not changes since v0.29.0-rc1

Changes in synapse v0.29.0-rc1 (2018-05-14)
===========================================

Notable changes, a docker file for running Synapse (Thanks to @kaiyou!) and a
closed spec bug in the Client Server API. Additionally further prep for Python 3
migration.

Potentially breaking change:

* Make Client-Server API return 401 for invalid token (PR #3161).

  This changes the Client-server spec to return a 401 error code instead of 403
  when the access token is unrecognised. This is the behaviour required by the
  specification, but some clients may be relying on the old, incorrect
  behaviour.

  Thanks to @NotAFile for fixing this.

Features:

* Add a Dockerfile for synapse (PR #2846) Thanks to @kaiyou!

Changes - General:

* nuke-room-from-db.sh: added postgresql option and help (PR #2337) Thanks to @rubo77!
* Part user from rooms on account deactivate (PR #3201)
* Make 'unexpected logging context' into warnings (PR #3007)
* Set Server header in SynapseRequest (PR #3208)
* remove duplicates from groups tables (PR #3129)
* Improve exception handling for background processes (PR #3138)
* Add missing consumeErrors to improve exception handling (PR #3139)
* reraise exceptions more carefully (PR #3142)
* Remove redundant call to preserve_fn (PR #3143)
* Trap exceptions thrown within run_in_background (PR #3144)

Changes - Refactors:

* Refactor /context to reuse pagination storage functions (PR #3193)
* Refactor recent events func to use pagination func (PR #3195)
* Refactor pagination DB API to return concrete type (PR #3196)
* Refactor get_recent_events_for_room return type (PR #3198)
* Refactor sync APIs to reuse pagination API (PR #3199)
* Remove unused code path from member change DB func (PR #3200)
* Refactor request handling wrappers (PR #3203)
* transaction_id, destination defined twice (PR #3209) Thanks to @damir-manapov!
* Refactor event storage to prepare for changes in state calculations (PR #3141)
* Set Server header in SynapseRequest (PR #3208)
* Use deferred.addTimeout instead of time_bound_deferred (PR #3127, #3178)
* Use run_in_background in preference to preserve_fn (PR #3140)

Changes - Python 3 migration:

* Construct HMAC as bytes on py3 (PR #3156) Thanks to @NotAFile!
* run config tests on py3 (PR #3159) Thanks to @NotAFile!
* Open certificate files as bytes (PR #3084) Thanks to @NotAFile!
* Open config file in non-bytes mode (PR #3085) Thanks to @NotAFile!
* Make event properties raise AttributeError instead (PR #3102) Thanks to @NotAFile!
* Use six.moves.urlparse (PR #3108) Thanks to @NotAFile!
* Add py3 tests to tox with folders that work (PR #3145) Thanks to @NotAFile!
* Don't yield in list comprehensions (PR #3150) Thanks to @NotAFile!
* Move more xrange to six (PR #3151) Thanks to @NotAFile!
* make imports local (PR #3152) Thanks to @NotAFile!
* move httplib import to six (PR #3153) Thanks to @NotAFile!
* Replace stringIO imports with six (PR #3154, #3168) Thanks to @NotAFile!
* more bytes strings (PR #3155) Thanks to @NotAFile!

Bug Fixes:

* synapse fails to start under Twisted >= 18.4 (PR #3157)
* Fix a class of logcontext leaks (PR #3170)
* Fix a couple of logcontext leaks in unit tests (PR #3172)
* Fix logcontext leak in media repo (PR #3174)
* Escape label values in prometheus metrics (PR #3175, #3186)
* Fix 'Unhandled Error' logs with Twisted 18.4 (PR #3182) Thanks to @Half-Shot!
* Fix logcontext leaks in rate limiter (PR #3183)
* notifications: Convert next_token to string according to the spec (PR #3190) Thanks to @mujx!
* nuke-room-from-db.sh: fix deletion from search table (PR #3194) Thanks to @rubo77!
* add guard for None on purge_history api (PR #3160) Thanks to @krombel!
2018-05-18 14:06:34 +01:00
Neil Johnson
f3d9dca975 Merge branch 'release-v0.29.0' of https://github.com/matrix-org/synapse into release-v0.29.0 2018-05-18 13:55:19 +01:00
Richard van der Hoff
6d9dc67139
Merge pull request #3232 from matrix-org/rav/server_notices_room
Infrastructure for a server notices room
2018-05-18 11:28:04 +01:00
Richard van der Hoff
ed3125b0a1
Merge pull request #3235 from matrix-org/rav/fix_receipts_deferred
Fix error in handling receipts
2018-05-18 11:23:11 +01:00
Richard van der Hoff
67af392712
Merge pull request #3233 from matrix-org/rav/remove_dead_code
Remove unused `update_external_syncs`
2018-05-18 11:22:43 +01:00
Richard van der Hoff
011e1f4010 Better docstrings 2018-05-18 11:22:12 +01:00
Richard van der Hoff
26305788fe Make sure we reject attempts to invite the notices user 2018-05-18 11:18:39 +01:00
Richard van der Hoff
d10707c810 Replace inline docstrings with "Attributes" in class docstring 2018-05-18 11:00:55 +01:00
Erik Johnston
fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token
Make purge_history operate on tokens
2018-05-18 10:35:23 +01:00
Richard van der Hoff
8b1c856d81 Fix error in handling receipts
Fixes an error which has been happening ever since #2158 (v0.21.0-rc1):

> TypeError: argument of type 'ObservableDeferred' is not iterable

fixes #3234
2018-05-18 09:15:35 +01:00
Neil Johnson
6958459b50 bump version, change log 2018-05-17 21:35:07 +01:00
Richard van der Hoff
88d3405332 fix missing yield for server_notices_room 2018-05-17 18:33:45 +01:00
Richard van der Hoff
d43d480d86 Remove unused update_external_syncs
This method isn't used anywhere. Burninate it.
2018-05-17 18:22:19 +01:00
Neil Johnson
a2da6de40e light grammar changes 2018-05-17 18:12:00 +01:00
Michael Kaye
450f500d0c Note that secrets need to be retained. 2018-05-17 18:12:00 +01:00
Michael Kaye
82b0361f02 Document macaroon env var correctly 2018-05-17 18:12:00 +01:00
Michael Kaye
1b1b47aec6 Reference synapse docker image and docker-compose 2018-05-17 18:12:00 +01:00
Richard van der Hoff
fed62e21ad Infrastructure for a server notices room
Server Notices use a special room which the user can't dismiss. They are
created on demand when some other bit of the code calls send_notice.

(This doesn't actually do much yet becuse we don't call send_notice anywhere)
2018-05-17 17:58:25 +01:00
Richard van der Hoff
f8a1e76d64
Merge pull request #3225 from matrix-org/rav/move_creation_handler
Move RoomCreationHandler out of synapse.handlers.Handlers
2018-05-17 17:56:42 +01:00
Erik Johnston
f7906203f6
Merge pull request #3212 from matrix-org/erikj/epa_stream
Use stream rather depth ordering for push actions
2018-05-17 12:01:21 +01:00
Richard van der Hoff
ae53c71d90
Merge pull request #1756 from rubo77/patch-4
Add instructions how to setup the postgres user and clarify the final step
2018-05-17 10:52:54 +01:00
rubo77
616da9eb1d postgres.rst: Add instructions how to setup the postgres user and clarify the final step 2018-05-17 11:48:56 +02:00
Richard van der Hoff
c46367d0d7 Move RoomCreationHandler out of synapse.handlers.Handlers
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.

Also fix the docstrings on create_room.
2018-05-17 09:08:42 +01:00
Neil Johnson
85b8acdeb4 Merge branch 'master' into develop 2018-05-16 16:48:10 +01:00
Neil Johnson
3c099219e0 bump version and changelog for 0.29.0 2018-05-16 15:44:59 +01:00
Erik Johnston
680530cc7f Clarify comment 2018-05-16 11:47:29 +01:00
Erik Johnston
43e6e82c4d Comments 2018-05-16 11:13:31 +01:00
Neil Johnson
dc8930ea9e
Merge pull request #3163 from matrix-org/cohort_analytics
user visit data
2018-05-16 10:09:24 +00:00
Erik Johnston
c945af8799 Move and rename variable 2018-05-16 10:52:06 +01:00
Neil Johnson
be11a02c4f remove empty line 2018-05-16 10:45:40 +01:00
Neil Johnson
a2204cc9cc remove unused method recurring_user_daily_visit_stats 2018-05-16 09:47:20 +01:00
Neil Johnson
31c2502ca8 style and further contraining query 2018-05-16 09:46:43 +01:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler
ConsentResource to gather policy consent from users
2018-05-16 07:19:18 +01:00
Neil Johnson
c92a8aa578 pep8 2018-05-15 17:31:11 +01:00
Neil Johnson
05ac15ae82 Limit query load of generate_user_daily_visits
The aim is to keep track of when it was last called and only query from that point in time
2018-05-15 17:01:33 +01:00
Erik Johnston
5f27ed75ad Make purge_history operate on tokens
As we're soon going to change how topological_ordering works
2018-05-15 16:23:50 +01:00
Erik Johnston
37dbee6490 Use events_to_purge table rather than token 2018-05-15 16:23:47 +01:00
Richard van der Hoff
47815edcfa ConsentResource to gather policy consent from users
Hopefully there are enough comments and docs in this that it makes sense on its
own.
2018-05-15 15:11:59 +01:00
Neil Johnson
589ecc5b58 further musical chairs 2018-05-14 17:49:59 +01:00
Neil Johnson
e71fb118f4 rearrange and collect related PRs 2018-05-14 17:39:22 +01:00
Neil Johnson
aea80a0118 v0.29.0-rc1: bump version and change log 2018-05-14 15:50:57 +01:00
Neil Johnson
f077e97914 instead of inserting user daily visit data at the end of the day, instead insert incrementally through the day 2018-05-14 13:50:58 +01:00
David Baker
8cbbfd16fb
Merge pull request #3201 from matrix-org/dbkr/leave_rooms_on_deactivate
Part user from rooms on account deactivate
2018-05-14 11:31:48 +01:00
Neil Johnson
977765bde2 Merge branch 'develop' of https://github.com/matrix-org/synapse into cohort_analytics 2018-05-14 09:31:42 +01:00