Commit Graph

86 Commits

Author SHA1 Message Date
Patrick Cloke
7e582a25f8
Improve /sync performance of when passing filters with empty arrays. ()
This has two related changes:

* It enables fast-path processing for an empty filter (`[]`) which was
  previously only used for wildcard not-filters (`["*"]`).
* It special cases a `/sync` filter with no-rooms to skip all room
  processing, previously we would partially skip processing, but would
  generally still calculate intermediate values for each room which were
  then unused.

Future changes might consider further optimizations:

* Skip calculating per-room account data when all rooms are filtered (currently
  this is thrown away).
* Make similar improvements to other endpoints which support filters.
2023-01-09 08:43:50 -05:00
David Robertson
b5b5f66084
Move StateFilter to synapse.types ()
* Move `StateFilter` to `synapse.types`

* Changelog
2022-12-12 16:19:30 +00:00
Erik Johnston
e3163e2e11
Reduce the amount of state we pull from the DB () 2022-06-06 09:24:12 +01:00
Erik Johnston
1e453053cb
Rename storage classes () 2022-05-31 12:17:50 +00:00
Erik Johnston
4660d9fdcf
Fix up state_store naming () 2022-05-25 12:59:04 +01:00
Andrew Morgan
83be72d76c
Add StreamKeyType class and replace string literals with constants () 2022-05-16 15:35:31 +00:00
David Robertson
6463244375
Remove unused # type: ignores ()
Over time we've begun to use newer versions of mypy, typeshed, stub
packages---and of course we've improved our own annotations. This makes
some type ignore comments no longer necessary. I have removed them.

There was one exception: a module that imports `select.epoll`. The
ignore is redundant on Linux, but I've kept it ignored for those of us
who work on the source tree using not-Linux. ()

I'm more interested in the config line which enforces this. I want
unused ignores to be reported, because I think it's useful feedback when
annotating to know when you've fixed a problem you had to previously
ignore.

* Installing extras before typechecking

Lacking an easy way to install all extras generically, let's bite the bullet and
make install the hand-maintained `all` extra before typechecking.

Now that https://github.com/matrix-org/backend-meta/pull/6 is merged to
the release/v1 branch.
2022-04-27 14:03:44 +01:00
Patrick Cloke
ab3165efb7
Remove experimental configuration flag for MSC3666. () 2022-04-11 15:37:46 -04:00
Patrick Cloke
8fe930c215
Move get_bundled_aggregations to relations handler. ()
The get_bundled_aggregations code is fairly high-level and uses
a lot of store methods, we move it into the handler as that seems
like a better fit.
2022-03-18 17:49:32 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ()
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of 
2022-02-23 11:04:02 +00:00
Sean Quah
e6acd3cf4f
Upgrade mypy to version 0.931 ()
Upgrade mypy to 0.931, mypy-zope to 0.3.5 and fix new complaints.
2022-02-18 15:57:26 +00:00
Patrick Cloke
e44f91d678
Refactor search code to reduce function size. ()
Splits the search code into a few logical functions instead of a single
unreadable function.

There are also a few additional changes for readability.

After refactoring it was clear to see there were some unused and
unnecessary variables, which were simplified.
2022-02-15 13:47:05 +00:00
Patrick Cloke
8c94b3abe9
Experimental support to include bundled aggregations in search results (MSC3666) () 2022-02-08 09:21:20 -05:00
Patrick Cloke
2897fb6b4f
Improvements to bundling aggregations. ()
This is some odds and ends found during the review of 
and while continuing to work in this code:

* Return attrs classes instead of dictionaries from some methods
  to improve type safety.
* Call `get_bundled_aggregations` fewer times.
* Adds a missing assertion in the tests.
* Do not return empty bundled aggregations for an event (preferring
  to not include the bundle at all, as the docstring states).
2022-01-26 08:27:04 -05:00
Patrick Cloke
6bf81a7a61
Bundle aggregations outside of the serialization method. ()
This makes the serialization of events synchronous (and it no
longer access the database), but we must manually calculate and
provide the bundled aggregations.

Overall this should cause no change in behavior, but is prep work
for other improvements.
2022-01-07 09:10:46 -05:00
Patrick Cloke
a19d01c3d9
Support filtering by relations per MSC3440 ()
Adds experimental support for `relation_types` and `relation_senders`
fields for filters.
2021-11-09 08:10:58 -05:00
Patrick Cloke
19d5dc6931
Refactor Filter to handle fields according to data being filtered. ()
This avoids filtering against fields which cannot exist on an
event source. E.g. presence updates don't have a room.
2021-10-27 11:26:30 -04:00
Patrick Cloke
eb9ddc8c2e
Remove the deprecated BaseHandler. ()
The shared ratelimit function was replaced with a dedicated
RequestRatelimiter class (accessible from the HomeServer
object).

Other properties were copied to each sub-class that inherited
from BaseHandler.
2021-10-08 07:44:43 -04:00
Patrick Cloke
94b620a5ed
Use direct references for configuration variables (part 6). () 2021-09-29 06:44:15 -04:00
Jonathan de Jong
98aec1cc9d
Use inline type hints in handlers/ and rest/. () 2021-07-16 18:22:36 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ()
Part of 

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke
b7748d3c00
Import HomeServer from the proper module. () 2021-03-23 07:12:48 -04:00
Patrick Cloke
1baab20352
Add type hints to various handlers. ()
With this change all handlers except the e2e_* ones have
type hints enabled.
2021-01-26 10:50:21 -05:00
Patrick Cloke
34a5696f93
Fix typos and spelling errors. () 2020-10-23 12:38:40 -04:00
Erik Johnston
7941372ec8
Make token serializing/deserializing async ()
The idea is that in future tokens will encode a mapping of instance to position. However, we don't want to include the full instance name in the string representation, so instead we'll have a mapping between instance name and an immutable integer ID in the DB that we can use instead. We'll then do the lookup when we serialize/deserialize the token (we could alternatively pass around an `Instance` type that includes both the name and ID, but that turns out to be a lot more invasive).
2020-09-30 20:29:19 +01:00
Patrick Cloke
8a4a4186de
Simplify super() calls to Python 3 syntax. ()
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke
e19de43eb5
Convert streams to async. () 2020-08-04 07:21:47 -04:00
Patrick Cloke
e739b20588
Fix up types and comments that refer to Deferreds. () 2020-07-24 10:53:25 -04:00
Patrick Cloke
be309d99cf
Convert search code to async/await. () 2020-05-11 15:12:39 -04:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ()
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Erik Johnston
28c98e51ff
Add local_current_membership table ()
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Andrew Morgan
ea0f0ad414
Prevent message search in upgraded rooms we're not in () 2019-12-11 13:07:25 +00:00
Amber Brown
020add5099
Update black to 19.10b0 ()
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
Erik Johnston
69f0054ce6 Port to use state storage 2019-10-30 14:46:54 +00:00
Amber Brown
4806651744
Replace returnValue with return () 2019-07-23 23:00:55 +10:00
Amber Brown
32e7c9e7f2
Run Black. () 2019-06-20 19:32:02 +10:00
Erik Johnston
b54b03f9e1 Allow client event serialization to be async 2019-05-14 11:58:01 +01:00
Richard van der Hoff
03c85335d1
Apply suggestions from code review
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:22:09 +00:00
Andrew Morgan
c9bfb058d8 Fix a bug with single-room search searching all rooms
* Create a new method for getting predecessor rooms
* Remove formatting change
2019-01-22 12:00:41 +00:00
Andrew Morgan
df3a661e4a Search for messages across predecessor rooms
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-18 11:19:20 +00:00
Travis Ralston
158ffb92f1 Add an option to disable search for homeservers which may not be interested in it ()
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
2018-12-04 12:01:02 +01:00
Amber Brown
3bade14ec0
Fix search 500ing () 2018-10-31 04:33:41 +11:00
Amber Brown
2608ebc04c
Port handlers/ to Python 3 () 2018-09-07 00:22:23 +10:00
Matthew Hodgson
e9b2d047f6
make /context lazyload & filter aware ()
make /context lazyload & filter aware.
2018-07-27 15:12:50 +01:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00
Erik Johnston
28ddc6cfbe Also log number of events for serach context 2018-06-22 15:42:11 +01:00
Erik Johnston
4b4cec3989 Add some logging to search queries 2018-06-22 15:42:11 +01:00
Amber Brown
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy () 2018-05-31 19:03:47 +10:00
Richard van der Hoff
eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Mark Haines
7e23476814 move filter_events_for_client out of base handler 2016-05-11 13:42:37 +01:00