forked-synapse/synapse/storage/schema/main/delta/73
James Salter d902181de9
Unified search query syntax using the full-text search capabilities of the underlying DB. (#11635)
Support a unified search query syntax which leverages more of the full-text
search of each database supported by Synapse.

Supports, with the same syntax across Postgresql 11+ and Sqlite:

- quoted "search terms"
- `AND`, `OR`, `-` (negation) operators
- Matching words based on their stem, e.g. searches for "dog" matches
  documents containing "dogs". 

This is achieved by 

- If on postgresql 11+, pass the user input to `websearch_to_tsquery`
- If on sqlite, manually parse the query and transform it into the sqlite-specific
  query syntax.

Note that postgresql 10, which is close to end-of-life, falls back to using
`phraseto_tsquery`, which only supports a subset of the features.

Multiple terms separated by a space are implicitly ANDed.

Note that:

1. There is no escaping of full-text syntax that might be supported by the database;
  e.g. `NOT`, `NEAR`, `*` in sqlite. This runs the risk that people might discover this
  as accidental functionality and depend on something we don't guarantee.
2. English text is assumed for stemming. To support other languages, either the target
  language needs to be known at the time of indexing the message (via room metadata,
  or otherwise), or a separate index for each language supported could be created.

Sqlite docs: https://www.sqlite.org/fts3.html#full_text_index_queries
Postgres docs: https://www.postgresql.org/docs/11/textsearch-controls.html
2022-10-25 14:05:22 -04:00
..
01event_failed_pull_attempts.sql Keep track when we try and fail to process a pulled event (#13589) 2022-09-14 13:57:50 -05:00
02add_pusher_enabled.sql Support enabling/disabling pushers (from MSC3881) (#13799) 2022-09-21 14:39:01 +00:00
02room_id_indexes_for_purging.sql Add support to purge rows from MSC2716 and other tables when purging a room (#13825) 2022-09-16 10:56:56 -05:00
03pusher_device_id.sql Track device IDs for pushers (#13831) 2022-09-21 15:31:53 +00:00
03users_approved_column.sql Allow admins to require a manual approval process before new accounts can be used (using MSC3866) (#13556) 2022-09-29 15:23:24 +02:00
04partial_join_details.sql Add new columns tracking when we partial-joined (#13892) 2022-09-27 17:26:35 +01:00
04pending_device_list_updates.sql Handle remote device list updates during partial join (#13913) 2022-09-28 13:42:43 +00:00
05old_push_actions.sql.postgres Clear out old rows from event_push_actions_staging (#14020) 2022-10-03 18:44:44 +01:00
05old_push_actions.sql.sqlite Clear out old rows from event_push_actions_staging (#14020) 2022-10-03 18:44:44 +01:00
06thread_notifications_thread_id_idx.sql Update the thread_id right before use (in case the bg update hasn't finished) (#14222) 2022-10-18 14:55:41 +00:00
08thread_receipts_non_null.sql.postgres Mark events as read using threaded read receipts from MSC3771. (#13877) 2022-10-04 10:46:42 -04:00
08thread_receipts_non_null.sql.sqlite Mark events as read using threaded read receipts from MSC3771. (#13877) 2022-10-04 10:46:42 -04:00
09partial_joined_via_destination.sql When restarting a partial join resync, prioritise the server which actioned a partial join (#14126) 2022-10-18 12:33:18 +01:00
09threads_table.sql Add an API for listing threads in a room. (#13394) 2022-10-13 08:02:11 -04:00
10_update_sqlite_fts4_tokenizer.py Unified search query syntax using the full-text search capabilities of the underlying DB. (#11635) 2022-10-25 14:05:22 -04:00