anonymousland-synapse/synapse
James Salter d902181de9
Unified search query syntax using the full-text search capabilities of the underlying DB. (#11635)
Support a unified search query syntax which leverages more of the full-text
search of each database supported by Synapse.

Supports, with the same syntax across Postgresql 11+ and Sqlite:

- quoted "search terms"
- `AND`, `OR`, `-` (negation) operators
- Matching words based on their stem, e.g. searches for "dog" matches
  documents containing "dogs". 

This is achieved by 

- If on postgresql 11+, pass the user input to `websearch_to_tsquery`
- If on sqlite, manually parse the query and transform it into the sqlite-specific
  query syntax.

Note that postgresql 10, which is close to end-of-life, falls back to using
`phraseto_tsquery`, which only supports a subset of the features.

Multiple terms separated by a space are implicitly ANDed.

Note that:

1. There is no escaping of full-text syntax that might be supported by the database;
  e.g. `NOT`, `NEAR`, `*` in sqlite. This runs the risk that people might discover this
  as accidental functionality and depend on something we don't guarantee.
2. English text is assumed for stemming. To support other languages, either the target
  language needs to be known at the time of indexing the message (via room metadata,
  or otherwise), or a separate index for each language supported could be created.

Sqlite docs: https://www.sqlite.org/fts3.html#full_text_index_queries
Postgres docs: https://www.postgresql.org/docs/11/textsearch-controls.html
2022-10-25 14:05:22 -04:00
..
_scripts let update_synapse_database run on a multi-database configurations (#13422) 2022-10-19 19:08:40 +01:00
api Use servlets for /key/ endpoints. (#14229) 2022-10-20 11:32:47 -04:00
app Fix logging error on SIGHUP (#14258) 2022-10-21 16:03:44 +01:00
appservice Send the appservice access token as a header. (#13996) 2022-10-04 07:06:41 -04:00
config Implementation for MSC3664: Pushrules for relations (#11804) 2022-10-25 14:38:01 +01:00
crypto Instrument _check_sigs_and_hash_and_fetch to trace time spent in child concurrent calls (#13588) 2022-08-23 21:53:37 -05:00
events Fix docstring in EventContext (#14145) 2022-10-18 13:40:50 -07:00
federation Merge branch 'release-v1.70' into develop 2022-10-25 15:39:35 +01:00
handlers Refactor OIDC tests to better mimic an actual OIDC provider. (#13910) 2022-10-25 14:25:02 +00:00
http Implementation of HTTP 307 response for MSC3886 POST endpoint (#14018) 2022-10-18 15:52:25 +00:00
logging Update mypy and mypy-zope, attempt 3 (#13993) 2022-09-30 17:36:28 +01:00
metrics Catch BrokenPipeError from metrics server, and log as a warning (#14072) 2022-10-07 13:35:44 +01:00
module_api Deprecate the generate_short_term_login_token method in favor of an async create_login_token method in the Module API. (#13842) 2022-10-06 10:22:36 +00:00
push Implementation for MSC3664: Pushrules for relations (#11804) 2022-10-25 14:38:01 +01:00
replication Merge remote-tracking branch 'origin/release-v1.69' into develop 2022-10-14 14:11:27 -04:00
res Improve aesthetics and reusability of HTML templates. (#13652) 2022-10-21 17:44:00 +00:00
rest Implementation for MSC3664: Pushrules for relations (#11804) 2022-10-25 14:38:01 +01:00
server_notices Directly lookup local membership instead of getting all members in a room first (get_users_in_room mis-use) (#13608) 2022-08-24 14:13:12 -05:00
spam_checker_api Fix import in module_api module and docs on the new check_event_for_spam signature (#12918) 2022-05-31 12:04:53 +02:00
state Prepatory work for batching events to send (#13487) 2022-09-28 10:39:03 +01:00
static Add viewport directive to HTML templates to optimise for mobile (#13493) 2022-08-11 14:04:20 +01:00
storage Unified search query syntax using the full-text search capabilities of the underlying DB. (#11635) 2022-10-25 14:05:22 -04:00
streams Do not allow a None-limit on PaginationConfig. (#14146) 2022-10-14 12:30:05 +00:00
util Remove unused @lru_cache decorator (#13595) 2022-10-25 11:39:25 +01:00
__init__.py Parse SYNAPSE_ASYNC_IO_REACTOR env variable & log the reactor on startup (#14092) 2022-10-07 15:19:59 +01:00
event_auth.py Fix event size checks (#13710) 2022-10-21 09:49:47 +01:00
notifier.py Batch up notifications after event persistence (#14033) 2022-10-05 10:12:48 -07:00
py.typed Mark Module API error imports as re-exported and mark Synapse as containing type annotations (#11054) 2021-10-13 08:42:41 +01:00
server.py Fix error in is_mine_id when encountering a malformed ID (#13746) 2022-09-08 15:54:36 +01:00
types.py Accept & store thread IDs for receipts (implement MSC3771). (#13782) 2022-09-23 14:33:28 +00:00
visibility.py Add debug logs to figure out why an event was filtered (#14095) 2022-10-17 16:02:39 -05:00