Matrix-Mirrors/pantalaimon
1
0
Fork 0
mirror of https://github.com/matrix-org/pantalaimon.git synced 2025-01-09 22:49:32 -05:00
Code Issues Packages Projects Releases Wiki Activity
413 Commits 15 Branches 26 Tags 3.2 MiB
fadcf2934d
Commit Graph

298 Commits

Author SHA1 Message Date
Damir Jelić
758ca1fd85 client: Allow the pan client sync loop to have a timeout between iterations. 2019-07-03 17:42:14 +02:00
Damir Jelić
53d1a2945d index: Make the index optional. 
This patch disables the message indexer an message store if
tantivy is not installed.

The search API endpoint forwards all search requests to the homeserver.
 2019-07-03 17:41:49 +02:00
Damir Jelić
fa1c2bb694 client: Use coroutines for the event callbacks. 2019-07-03 09:50:45 +02:00
Damir Jelić
3e32b9544b daemon: Update the devices in the UI thread after a send and ignore. 2019-07-02 14:12:27 +02:00
Damir Jelić
6510bd7eb9 client: Only send device update messages if there were changed devices. 2019-07-01 18:19:36 +02:00
Damir Jelić
f2415738f3 ui: Don't load the devices from the store. 
The device list in the UI thread is replicated so we can show UI clients
the list without the need for a lock.

The previous implementation relied on loading and reloading of the device
list from the store every time an event changed either the devices or their
trust state.

This leads to a couple of ineficiencies leading to timeouts while
waiting on the database lock if a user has a large number of devices.

The new implementation never loads devices in the UI thread from the
database, they get passed through the thread queue by the main thread
which already holds them in memory.
 2019-07-01 18:19:30 +02:00
Damir Jelić
62d0e5607e ui: Don't load pan users from the db, pass them over the queue. 2019-06-19 15:54:58 +02:00
Damir Jelić
c9ebfd71ec pantalaimon: Format the source tree using black. 2019-06-19 12:37:44 +02:00
Damir Jelić
531d686d8f pantalaimon: Run isort. 2019-06-19 12:07:56 +02:00
Damir Jelić
773b708f14 daemon: Remove unnecessary pass. 2019-06-19 11:56:32 +02:00
Damir Jelić
2c831de532 config: Set some more default values for the server config. 2019-06-19 11:21:43 +02:00
Damir Jelić
c8d81b9cf4 client: Catch keyboard exceptions for the client tasks. 2019-06-18 16:31:17 +02:00
Damir Jelić
e766b1ae4b config: Make the delay between room history fetching configurable. 2019-06-18 16:30:38 +02:00
Damir Jelić
0ad0da2151 daemon: Add missing await. 2019-06-18 15:35:50 +02:00
Damir Jelić
aafc56b44d daemon: Add CORS headers to the responses. 
This add CORS headers to all the responses, they are required for web
based clients (e.g. Riot) to work.
 2019-06-18 10:10:25 +02:00
Damir Jelić
f5ffd69a79 config: Make the batch size for room history fetching configurable. 2019-06-17 16:59:54 +02:00
Damir Jelić
0940b67eb8 pantalaimon: Add the ability to index only encrypted rooms. 
Since some public rooms can be quite large downloading the room history
for such rooms can be quite resource intensive.

Replicating the whole room history locally for such rooms might be
undesirable. This patch adds the ability to only download room history
and index messages for encrypted rooms.

Do note that the search API supports searching in multiple rooms, thus
if the indexing is restricted to encrypted rooms a search across
multiple rooms will need to make a search request to the server as well
as a local search. This mode is currently unsupported.
 2019-06-17 15:23:06 +02:00
Damir Jelić
83f62b0378 config: Make additional Homeserver requests for the search endpoint optional. 2019-06-17 12:26:38 +02:00
Damir Jelić
a6a62434e0 client: Add the context token fetching back for the search. 2019-06-17 10:29:23 +02:00
Damir Jelić
1850ead698 daemon: Sort the imports. 2019-06-14 15:23:41 +02:00
Damir Jelić
fa3e11a1ec index: Catch query parsing errors. 
Tantivy's query parser can throw errors if the syntax of the query is
invalid.

There is a more lenient query parser[1] in the works but until then
catch query parser errors and return an error response to the user.

[1] https://github.com/tantivy-search/tantivy/issues/5
 2019-06-14 15:15:58 +02:00
Damir Jelić
3a1b001244 index: Rewrite the search logic. 
This patch moves all the indexing, event storing and searching into a
separate class.

The index and message store are now represented as a single class and
messages are indexed and stored atomically now which should minimize the
chance of store/index inconsistencies.

Messages are now loaded from the store as a single SQL query and the
context for the messages is as well loaded from the store instead of
fetched from the server.

The room state and start/end tokens for the context aren't currently
loaded.
 2019-06-14 14:53:25 +02:00
Damir Jelić
a489031962 client: Fix a style issue. 2019-06-13 13:01:58 +02:00
Damir Jelić
1f5afd75c6 client: Allow per room search. 
This is not ideal. We only support a single room or all rooms, which is
good enough for now since riot does that.

The tantivy boolean query logic isn't ideal and didn't work out with
multiple rooms in the query.
 2019-06-13 12:32:52 +02:00
Damir Jelić
fe10f54ea9 client: Error out gracefully if we get an invalid limit. 2019-06-13 12:32:21 +02:00
Damir Jelić
ba09f77cb6 client: Simplify the sync logic. 2019-06-12 16:24:03 +02:00
Damir Jelić
6092e27748 client: Pass the sync token to the sync method. 2019-06-12 15:39:43 +02:00
Damir Jelić
b4e60b603a client: Store the history fetcher tasks to the db. 2019-06-12 15:39:08 +02:00
Damir Jelić
1ad2a3af28 daemon: Add access control headers to the search endpoint. 
These headers are important for riot to be able to use pan's search
endpoint instead of the default Homeserver endpoint. It fails to search
using pan with CORS errors.
 2019-06-12 15:34:08 +02:00
Damir Jelić
a2d6f3cc84 client: Fetch room members before the first encrytped send. 2019-06-12 15:33:27 +02:00
Damir Jelić
383f197e58 client: Store and load sync tokens. 2019-06-11 16:11:53 +02:00
Damir Jelić
2fc738dc20 client: Change the initial sync to contain full state. 
Fetching the full state with the initial sync is important because we
don't store any room state locally.

Since we don't store any sync token we are getting the full state
anyways. But once we do this change will be crucial for encryption to
work correctly.
 2019-06-11 15:00:35 +02:00
Damir Jelić
8c3cfbc0dd client: Fetch old room messages if there was a limited room timeline. 
This patch adds the ability to fetch the whole room history. Note that
this is still incomplete as it not fetch the whole history if it gets
interrupted.

Another deficiency is that it always tries to start fetching room
history after a restart even though we know that it doesn't need to do
so.
 2019-06-11 13:59:34 +02:00
Damir Jelić
c7ca5d9851 pantalaimon: Mypy fixes. 2019-06-11 11:24:37 +02:00
Damir Jelić
5b58294a95 pantalaimon: Isort the tree. 2019-06-11 11:24:18 +02:00
Damir Jelić
1e3d891a57 client: Add the ability to order the search by recency. 2019-06-10 16:12:45 +02:00
Damir Jelić
88dbd2000a index: Change the index format. 
Tantivy allows search results to be ordered by arbitrary index fields as
long as they are a fast field and have the FastValue trait implemented.
This trait is only for u64 and i64 fields implemented.

To order the the search results by recency we need to add a unsigned
field that stores the server timestamp of the message.
 2019-06-10 15:56:05 +02:00
Damir Jelić
818db73a48 daemon: Handle invalid order by fields for the search. 2019-06-10 15:55:37 +02:00
Damir Jelić
8927bb0c90 client: Fix the search response format. 2019-06-10 15:53:19 +02:00
Damir Jelić
71e672a431 client: Make the room messages callback more specific. 2019-06-10 15:52:22 +02:00
Damir Jelić
1db93a731f client: Add the ability to add the context and state to search results. 2019-06-07 11:59:53 +02:00
Damir Jelić
725c043e87 pantalaimon: Initial search support. 
This patch adds support for the Matrix search API endpoint.

Events are stored in the pan sqlite database while the indexing is
handled by tanvity.

An tantivy index is created for each pan user. Currently only ordering
by ranking is supported, and all the search options are ignored for now.
 2019-06-06 14:25:14 +02:00
Damir Jelić
160fbbaf98 panctl: Add a version option. 2019-05-24 13:48:39 +02:00
Damir Jelić
f8820c0e73 pantalaimon: Add a version option. 2019-05-24 13:48:21 +02:00
Damir Jelić
cc76a8159b panctl: Add a help command. 2019-05-24 11:33:02 +02:00
Damir Jelić
4fe5e7d584 pantalaimon: Isort fixes. 2019-05-23 14:46:29 +02:00
Damir Jelić
ce89b1e407 pantalaimon: Abort the daemon gracefully on keyring errors. 2019-05-23 13:50:02 +02:00
Damir Jelić
e597969a21 config: Make the OS keyring optional. 2019-05-23 13:34:59 +02:00
Damir Jelić
a25ab95f62 config: Allow notifications to be disabled in the config. 2019-05-23 12:00:39 +02:00
Damir Jelić
bfcfdd61ea ui: Add OS notifications. 2019-05-22 16:40:15 +02:00
Damir Jelić
186560dd4f daemon: Connect the queue for unverified devices messages while sending. 2019-05-22 11:51:49 +02:00
Damir Jelić
212baad49e store: Remove unused client info storing functionality. 2019-05-21 16:55:07 +02:00
Damir Jelić
a823d85cd8 pantalaimon: Isort the repo. 2019-05-21 16:48:46 +02:00
Damir Jelić
c85f82423d main: Remove spurious print. 2019-05-21 16:47:56 +02:00
Damir Jelić
d0d0e80931 daemon: Fix a typo in a variable name. 2019-05-21 16:47:31 +02:00
Damir Jelić
42a4391597 pantalaimon: Remove an unused variable and fix a whitespace issue. 2019-05-21 16:46:50 +02:00
Damir Jelić
d5b1ef27a9 pantalaimon: Add license headers. 2019-05-21 16:46:22 +02:00
Damir Jelić
3d87b468b2 daemon: Ask the homeserver about unknown access tokens. 
This patch changes the way clients register to pantalaimon. If there is
already a pan client running for a given user ID clients can now avoid
logging in.

Pantalaimon checks with the homeserver if the access token
is valid and for a user that we already have a pan client running.

This also removes the need to remember access tokens that don't belong
to a pan client.

This partially fixes #14.
 2019-05-21 16:25:23 +02:00
Damir Jelić
a6bc5de523 client: Don't verify devices automatically anymore. 2019-05-21 13:43:05 +02:00
Damir Jelić
cb38112160 tests: Fix the dbus test. 2019-05-21 13:42:48 +02:00
Damir Jelić
48da1b6b14 config: Add the ability to disable device verification. 2019-05-21 13:23:07 +02:00
Damir Jelić
1ad5cf6e66 daemon: Keep the proxy config in the daemon class around. 2019-05-21 12:35:55 +02:00
Damir Jelić
90e803900d panctl: Add send-anyways and cancel-send commands. 2019-05-21 12:33:10 +02:00
Damir Jelić
ae85ec22bb ui: Send the room display name in the UnverifiedDevices signal. 2019-05-21 12:31:58 +02:00
Damir Jelić
ff2fc7e448 daemon: Handle the case where we try to send to unverified devices. 2019-05-21 10:25:59 +02:00
Damir Jelić
c40af38b33 panctl: Print out if a device is ignored. 2019-05-20 16:50:27 +02:00
Damir Jelić
ff0f6c50f9 daemon: Return the proper content type if we are using json responses. 2019-05-20 15:30:58 +02:00
Damir Jelić
4f90e578ef store: Use the server name to store info. 
This way multiple proxy servers that point to the same homeserver are
allowed.
 2019-05-17 16:49:02 +02:00
Damir Jelić
e0b5d3e2b6 panctl: List the trust state of devices. 2019-05-17 14:57:57 +02:00
Damir Jelić
48f94851d2 ui: Allow devices to be blackliksted and unblacklisted. 2019-05-17 14:57:36 +02:00
Damir Jelić
694903e8bd pantalaimon: Allow pan users to be dynamically updated in the ui thread. 2019-05-17 14:56:40 +02:00
Damir Jelić
ed13f78383 panctl: Fix a typo in the command parser. 2019-05-17 14:35:52 +02:00
Damir Jelić
2d487c55f5 ui: Update the device list in the ui thread dynamically. 
This updates the copy of the device list in the ui thread every time the
device list or the verification of devices changes in the ui thread.
 2019-05-17 14:33:48 +02:00
Damir Jelić
2ca850bf72 client: Connection error messages need to be strings. 2019-05-17 14:27:43 +02:00
Damir Jelić
00f2b51fa9 store: Change the way device keys are loaded. 2019-05-16 10:10:27 +02:00
Damir Jelić
a114b7b1a9 ui: Add support for starting/canceling key verifications. 2019-05-15 11:51:33 +02:00
Damir Jelić
e9d0f52a5c pantalaimon: Isort the tree. 2019-05-14 21:50:30 +02:00
Damir Jelić
22fcbff44b pantalaimon: Formatting fixes. 2019-05-14 21:29:59 +02:00
Damir Jelić
69d14f3c68 daemon: Import/export key functions are now coroutines. 2019-05-14 16:11:50 +02:00
Damir Jelić
90d887e2fa client: Mypy fixes. 2019-05-14 16:11:39 +02:00
Damir Jelić
6749a0bfd2 daemon: Remove unused import. 2019-05-14 14:14:38 +02:00
Damir Jelić
2151f55337 daemon: Simplify the room send logic. 2019-05-14 13:52:55 +02:00
Damir Jelić
bfd7144b4f daemon: Fetch joined members correctly while trying to send messages. 2019-05-14 12:58:16 +02:00
Damir Jelić
42450b51c7 ui: Rework the way the dbus thread and main thread communicate. 2019-05-13 16:32:12 +02:00
Damir Jelić
1cbe5b8a6d ui: Rewrite the dbus code using pydbus. 2019-05-13 16:32:12 +02:00
Damir Jelić
c60e554fc2 store: Change the name of the device key fields in the dict. 2019-05-13 16:32:12 +02:00
Damir Jelić
671fe9dbd0 pantalaimon: Version the dbus API. 2019-05-13 16:32:12 +02:00
Damir Jelić
840e568eac client: Don't accept a SAS automatically. 2019-05-13 16:32:12 +02:00
Damir Jelić
cb04e962f1 panctl: Add a list devices command. 2019-05-13 16:32:12 +02:00
Damir Jelić
bf25a35b31 daemon: Expand the user for the import/export path. 2019-05-09 16:49:44 +02:00
Damir Jelić
ba4ed34941 client: Add a missing await. 2019-05-09 16:49:16 +02:00
Damir Jelić
a560462da7 pacntl: Switch to argparse and add more functionality. 2019-05-09 16:48:29 +02:00
Damir Jelić
6adedbaa0b panctl: Improve the completions. 2019-05-09 13:47:29 +02:00
Damir Jelić
10bb16b210 panctl: Handle an empty command string properly. 2019-05-09 10:54:42 +02:00
Damir Jelić
1d2d0d6bba daemon: Use the right queue to send messages. 2019-05-09 10:54:22 +02:00
Damir Jelić
1f548f3a7e client: Handle client errors while confirming the auth string. 2019-05-09 10:54:01 +02:00
Damir Jelić
de29df75b9 pantalaimon: Refactor out the thread messages. 2019-05-09 10:53:20 +02:00
Damir Jelić
1a8e5ca65d config: Fix a whitespace issue. 2019-05-08 17:40:16 +02:00
Damir Jelić
b687f503d4 main: Add a SIGTERM signal handler. 2019-05-08 16:11:07 +02:00
Damir Jelić
cc7b257345 main: Route ui messages to the correct daemon. 2019-05-08 15:47:16 +02:00
Damir Jelić
2946acbd6c main: Enable ssl configuration again. 2019-05-08 15:00:50 +02:00
Damir Jelić
2127a7cba5 main: Start all the configured servers. 2019-05-08 14:21:38 +02:00
Damir Jelić
0d89987ec1 config: Add the section name to the config class. 2019-05-08 14:20:47 +02:00
Damir Jelić
3d34a26dac config: Handle some invalid configurations. 2019-05-08 12:43:21 +02:00
Damir Jelić
2987589854 pantalaimon: Add a configuration file for the daemon. 2019-05-08 12:33:42 +02:00
Damir Jelić
65d1401b76 main: Remove unused function. 2019-05-08 10:27:55 +02:00
Damir Jelić
31ab200763 client: Only try to decrypt m.room.encrypted messages in the sync decryption. 2019-05-07 18:38:08 +02:00
Damir Jelić
7318690e2c daemon: Buffer the messages endpoint if there are undecryptable messages. 2019-05-07 18:37:32 +02:00
Damir Jelić
400c299187 daemon: Add Bearer to the auth header if we're overriding the token. 2019-05-07 18:36:05 +02:00
Damir Jelić
6b8758f2cb main: Remove cli commands for key export/import. 
The functionality is now available over dbus.
 2019-05-07 15:34:29 +02:00
Damir Jelić
cd5565b4b3 client: Remove unused encrypt function. 2019-05-07 15:00:51 +02:00
Damir Jelić
aa1143328f client: Remove unused asyncio events. 2019-05-07 14:57:53 +02:00
Damir Jelić
6988a1de95 daemon: The filter query parameter in a sync needs to be a str. 2019-05-07 14:52:01 +02:00
Damir Jelić
cf07cfdc16 client: Use the new AsyncClient functionality from nio for syncing. 2019-05-07 14:44:29 +02:00
Damir Jelić
c96bdea1d3 pantalaimon: Refactor out the daemon start logic into a separate file. 2019-05-07 10:42:40 +02:00
Damir Jelić
e961ff6035 ui: Create a glib class and simplify the shutdown logic. 2019-05-07 10:26:13 +02:00
Damir Jelić
57f3a00a78 client: Switch to the new confirm auth string function. 2019-05-07 09:33:12 +02:00
Damir Jelić
5ef5551977 client: Use the nio provided method to find a SAS object. 2019-05-03 16:09:06 +02:00
Damir Jelić
46ff19ea3c panctl: Remove unneeded DbusT class. 2019-05-03 14:12:33 +02:00
Damir Jelić
6dd47e7a53 pantalaimon: Rename accept_sas to confirm sas. 2019-05-03 14:01:01 +02:00
Damir Jelić
90ec4e2e96 daemon: Send some messages to the ui thread. 2019-05-03 13:48:28 +02:00
Damir Jelić
103a17998a panctl: Allow the user to confirm the short auth string. 2019-05-03 13:46:19 +02:00
Damir Jelić
7f8d18d857 daemon: Handle json errors in the sync and messages methods. 
A valid 200 response from the server doesn't necessarily mean that the
content will be valid json.

Handle decode errors and return the raw body if we're unable to decode.
 2019-05-02 14:49:24 +02:00
Damir Jelić
a996d4acd3 daemon: Don't use the text method to get the content. 
The content doesn't necessarily have to be unicode (e.g. for media
content) and the text() method tries to decode the body of the request.

Use read() instead, this way we get the raw bytes of the body, we also
set the content header manually.

This fixes #8.
 2019-05-02 14:46:53 +02:00
Damir Jelić
d90eebf1e0 daemon: Remove redundant or clause. 2019-05-02 14:44:25 +02:00
Damir Jelić
e84bcea389 pantalaimon: Catch all aiohttp client connection errors. 2019-05-02 13:55:20 +02:00
Damir Jelić
f1fefb952c daemon: Buffer the sync response if there are decryption failures. 2019-05-02 13:30:31 +02:00
Damir Jelić
46aa303654 client: Clear the synced event when we start the loop. 2019-05-02 13:28:41 +02:00
Damir Jelić
ba28580aca client: Replace undecryptable megolm events with an error message. 2019-05-02 13:27:50 +02:00
Damir Jelić
db43c3efbd client: Request room keys if we're unable to decrypt a megolm event. 2019-05-02 12:09:49 +02:00
Damir Jelić
7d58da93f9 client: No need to mark to-device messages as sent anymore. 2019-04-30 18:15:39 +02:00
Damir Jelić
0972be29c1 client: Allow other event types to be decrypted. 2019-04-30 14:54:01 +02:00
Damir Jelić
7bb47092ff ui: Disable device updates for now. 2019-04-30 13:08:22 +02:00
Damir Jelić
da552973ff daemon: Overide the filter API endpoint. 2019-04-30 13:07:11 +02:00
Damir Jelić
90a87460e8 client: Use user lazy loading. 2019-04-29 16:56:50 +02:00
Damir Jelić
94e6058ef4 daemon: Handle connection errors when doing requests. 2019-04-29 16:53:31 +02:00
Damir Jelić
253424b8c3 daemon: Make the query parameters always editable in forward_request(). 2019-04-29 13:15:21 +02:00
Damir Jelić
3671660e64 daemon: Send out non-encrypted messages as well using pan's token. 2019-04-29 13:13:00 +02:00
Damir Jelić
17a20195c0 daemon: Use pan's token to perform client syncs. 
The clients need to use pan's sync token so the transaction ids in the
sync response don't get lost since sending happens using pan's device
and token.
 2019-04-29 12:58:59 +02:00
Damir Jelić
7a2ee980ff client: Handle key verification exceptions. 2019-04-29 09:49:42 +02:00
Damir Jelić
7aa7014431 client: Handle SAS key verification. 
This adds the ability to receive and accept SAS key verification.
The verification process is accepted automatically for now.
 2019-04-28 21:17:10 +02:00
Damir Jelić
3e67010278 client: Don't check for the room id until we know it's a valid event. 2019-04-28 21:16:14 +02:00
Damir Jelić
0a1008a127 client: Send out outgoing to-device messages. 2019-04-28 21:15:14 +02:00
Damir Jelić
2c218bfd04 client: Handle server disconnect exceptions. 2019-04-28 21:14:57 +02:00
Damir Jelić
5f124806d8 pantalaimon: Add a Cli dbus based client. 2019-04-22 21:28:15 +02:00
Damir Jelić
e43b8c750c ui: Expose key imports from the dbus API. 2019-04-18 16:56:53 +02:00
Damir Jelić
2bb6ac7196 daemon: Change the working dir to our homedir. 2019-04-18 15:53:25 +02:00
Damir Jelić
3c09dbc80a ui: Expose key exports on the dbus API. 2019-04-18 15:52:21 +02:00
Damir Jelić
8f57e42c2c ui: Change the name of the Users dbus class. 2019-04-18 14:12:31 +02:00
Damir Jelić
daacd340f9 ui: Allow devices to be unverified over the dbus API. 2019-04-18 14:01:57 +02:00