Patrick Cloke
8e1e62c9e0
Update license headers
2023-11-21 15:29:58 -05:00
Erik Johnston
c485ed1c5a
Clear event caches when we purge history ( #15609 )
...
This should help a little with #13476
---------
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2023-06-08 13:14:40 +01:00
Patrick Cloke
733531ee3e
Add final type hint to synapse.server. ( #15035 )
2023-02-09 09:49:04 -05:00
Nick Mills-Barrett
db1cfe9c80
Update all stream IDs after processing replication rows ( #14723 )
...
This creates a new store method, `process_replication_position` that
is called after `process_replication_rows`. By moving stream ID advances
here this guarantees any relevant cache invalidations will have been
applied before the stream is advanced.
This avoids race conditions where Python switches between threads mid
way through processing the `process_replication_rows` method where stream
IDs may be advanced before caches are invalidated due to class resolution
ordering.
See this comment/issue for further discussion:
https://github.com/matrix-org/synapse/issues/14158#issuecomment-1344048703
2023-01-04 11:49:26 +00:00
dependabot[bot]
8bcdd712b8
Bump flake8-bugbear from 22.9.23 to 22.10.27 ( #14329 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2022-11-04 18:43:14 +00:00
Nick Mills-Barrett
a466164647
Optimise get_rooms_for_user (drop with_stream_ordering) ( #13787 )
2022-09-29 13:55:12 +00:00
Nick Mills-Barrett
6b4593a80f
Simplify cache invalidation after event persist txn ( #13796 )
...
This moves all the invalidations into a single place and de-duplicates
the code involved in invalidating caches for a given event by using
the base class method.
2022-09-26 16:26:35 +01:00
Mathieu Velten
6bd8763804
Add cache invalidation across workers to module API ( #13667 )
...
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-21 15:32:01 +02:00
Erik Johnston
43adf2521c
Refactor presence so we can prune user in room caches ( #13313 )
...
See #10826 and #10786 for context as to why we had to disable pruning on
those caches.
Now that `get_users_who_share_room_with_user` is called frequently only
for presence, we just need to make calls to it less frequent and then we
can remove the various levels of caching that is going on.
2022-07-25 09:21:06 +00:00
Nick Mills-Barrett
2ee0b6ef4b
Safe async event cache ( #13308 )
...
Fix race conditions in the async cache invalidation logic, by separating
the async & local invalidation calls and ensuring any async call i
executed first.
Signed off by Nick @ Beeper (@Fizzadar).
2022-07-19 11:25:29 +00:00
Erik Johnston
f721f1baba
Revert "Make all process_replication_rows methods async ( #13304 )" ( #13312 )
...
This reverts commit 5d4028f217
2022-07-18 14:28:14 +01:00
Nick Mills-Barrett
5d4028f217
Make all process_replication_rows methods async ( #13304 )
...
More prep work for asyncronous caching, also makes all process_replication_rows methods consistent (presence handler already is so).
Signed off by Nick @ Beeper (@Fizzadar)
2022-07-17 22:19:43 +01:00
Erik Johnston
e5716b631c
Don't pull out the full state when calculating push actions ( #13078 )
2022-07-11 20:08:39 +00:00
Erik Johnston
44de53bb79
Reduce state pulled from DB due to sending typing and receipts over federation ( #12964 )
...
Reducing the amount of state we pull from the DB is useful as fetching state is expensive in terms of DB, CPU and memory.
2022-06-06 16:46:11 +01:00
Erik Johnston
888a29f412
Wait for lazy join to complete when getting current state ( #12872 )
2022-06-01 16:02:53 +01:00
Patrick Cloke
a8da046907
Invalidate the get_users_in_room{_with_profile} caches only when necessary. ( #11878 )
...
The get_users_in_room and get_users_in_room_with_profiles
are now only invalidated when the membership of a room changes,
instead of during any state change in the room.
2022-02-02 12:24:07 -05:00
Sean Quah
5305a5e881
Type hint the constructors of the data store classes ( #11555 )
2021-12-13 17:05:00 +00:00
Sean Quah
ffd858aa68
Add type hints to synapse/storage/databases/main/events_worker.py ( #11411 )
...
Also refactor the stream ID trackers/generators a bit and try to
document them better.
2021-11-26 18:41:31 +00:00
Dan Callahan
52ed9655ed
Remove unnecessary SystemRandom from SQLBaseStore ( #9987 )
...
It's not obvious that instances of SQLBaseStore each need their own
instances of random.SystemRandom(); let's just use random directly.
Introduced by 52839886d6
2021-05-14 10:59:10 +01:00
Erik Johnston
d0aee697ac
Use get_current_users_in_room from store and not StateHandler ( #9910 )
2021-05-05 16:49:34 +01:00
Andrew Morgan
fe604a022a
Remove various bits of compatibility code for Python <3.6 ( #9879 )
...
I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+.
2021-04-27 13:13:07 +01:00
Richard van der Hoff
294c675033
Remove synapse.types.Collection ( #9856 )
...
This is no longer required, since we have dropped support for Python 3.5.
2021-04-22 16:43:50 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ( #9786 )
...
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke
b7748d3c00
Import HomeServer from the proper module. ( #9665 )
2021-03-23 07:12:48 -04:00
Patrick Cloke
637282bb50
Add additional type hints to the storage module. ( #8980 )
2020-12-30 08:09:53 -05:00
Richard van der Hoff
97647b33c2
Replace DeferredCache with LruCache where possible ( #8563 )
...
Most of these uses don't need a full-blown DeferredCache; LruCache is lighter and more appropriate.
2020-10-19 12:20:29 +01:00
Patrick Cloke
eebf52be06
Be stricter about JSON that is accepted by Synapse ( #8106 )
2020-08-19 07:26:03 -04:00
Patrick Cloke
fbe930dad2
Convert the roommember database to async/await. ( #8070 )
2020-08-12 12:14:34 -04:00
Erik Johnston
a7bdf98d01
Rename database classes to make some sense ( #8033 )
2020-08-05 21:38:57 +01:00
Patrick Cloke
f460da6031
Consistently use db_to_json to convert from database values to JSON objects. ( #7849 )
2020-07-16 11:32:19 -04:00
Richard van der Hoff
e6027562e2
remove builtins.buffer code from storage code
...
this is no longer needed on python 3
2020-05-15 19:37:41 +01:00
Erik Johnston
d7983b63a6
Support any process writing to cache invalidation stream. ( #7436 )
2020-05-07 13:51:08 +01:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ( #6957 )
...
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Erik Johnston
5d7a6ad223
Allow streaming cache invalidate all to workers. ( #6749 )
2020-01-22 10:37:00 +00:00
Richard van der Hoff
85db7f73be
Add a background update to clear tombstoned rooms from the directory ( #6648 )
...
* Add a background update to clear tombstoned rooms from the directory
* use the ABC metaclass
2020-01-07 14:18:43 +00:00
Erik Johnston
2284eb3a53
Add database config class ( #6513 )
...
This encapsulates config for a given database and is the way to get new
connections.
2019-12-18 10:45:12 +00:00
Erik Johnston
d537be1ebd
Pass Database into the data store
2019-12-06 15:49:44 +00:00
Erik Johnston
9a4fb457cf
Change DataStores to accept 'database' param.
2019-12-06 13:30:06 +00:00
Erik Johnston
8863624f78
Comments
2019-12-05 10:46:37 +00:00
Erik Johnston
756d4942f5
Move DB pool and helper functions into dedicated Database class
2019-12-05 10:46:37 +00:00
Erik Johnston
ee86abb2d6
Remove underscore from SQLBaseStore functions
2019-12-04 16:23:43 +00:00
Erik Johnston
c2f525a525
Don't call SQLBaseStore methods from outside stores
2019-12-04 16:23:43 +00:00
Erik Johnston
9186c105a0
Revert "Move get_user_count_txn out of base store"
...
This reverts commit 00f0d67566
2019-12-04 15:46:19 +00:00
Erik Johnston
00f0d67566
Move get_user_count_txn out of base store
2019-12-04 15:21:14 +00:00
Erik Johnston
1056d6885a
Move cache invalidation to main data store
2019-12-04 15:21:14 +00:00
Erik Johnston
6b2867096b
Move event fetch vars to EventWorkStore
2019-12-04 15:21:14 +00:00
Erik Johnston
ddd48b6851
Move account validity bg updates to registration store
2019-12-04 15:21:14 +00:00
Richard van der Hoff
c01d543584
Make sure that we close cursors before returning from a query ( #6408 )
...
There are lots of words in the comment as to why this is a good idea.
Fixes #6403 .
2019-11-25 21:03:17 +00:00
Andrew Morgan
3916e1b97a
Clean up newline quote marks around the codebase ( #6362 )
2019-11-21 12:00:14 +00:00
Andrew Morgan
745a48625d
Fix guest -> real account upgrade with account validity enabled ( #6359 )
2019-11-14 12:02:05 +00:00
