Richard van der Hoff
3d12d97415
Track DB scheduling delay per-request
...
For each request, track the amount of time spent waiting for a db
connection. This entails adding it to the LoggingContext and we may as well add
metrics for it while we are passing.
2018-01-16 17:23:32 +00:00
Richard van der Hoff
8615f19d20
rework runInteraction in terms of runConnection
...
... so that we can share the code
2018-01-16 17:08:29 +00:00
Matthew Hodgson
f397153dfc
Merge branch 'develop' into matthew/search-all-local-users
2017-11-30 01:51:38 +00:00
Matthew Hodgson
3241c7aac3
untested WIP but might actually work
2017-11-29 18:27:05 +00:00
Richard van der Hoff
5a4da5bf78
Merge pull request #2697 from matrix-org/rav/fix_urlcache_index_error
...
Fix error on sqlite 3.7
2017-11-27 12:25:48 +00:00
Richard van der Hoff
63ccaa5873
Avoid retrying forever on IntegrityError
2017-11-27 12:00:07 +00:00
Richard van der Hoff
6b48b3e277
fix sql fails
2017-11-22 18:06:24 +00:00
Richard van der Hoff
2908f955d1
Check database in has_completed_background_updates
...
so that the right thing happens on workers.
2017-11-22 18:02:15 +00:00
Richard van der Hoff
77a1227870
Fix broken ref to IntegrityError
2017-11-16 16:03:38 +00:00
Richard van der Hoff
10aaa1bc15
_simple_upsert: retry on IntegrityError
...
wrap the call to _simple_upsert_txn in a loop so that we retry on an
integrityerror: this means we can avoid locking the table provided there is an
unique index.
2017-11-16 15:30:15 +00:00
Richard van der Hoff
cdc9e50a5d
Cleanup in _simple_upsert_txn
...
Bail out early to reduce indentation
2017-11-16 15:29:10 +00:00
Richard van der Hoff
4dd1bfa8c1
Revert "Revert "move _state_group_cache to statestore""
...
We're going to fix this properly on this branch, so that the _state_group_cache
can end up in StateGroupReadStore.
This reverts commit ab335edb02
2017-11-14 11:43:58 +00:00
Richard van der Hoff
6cfee09be9
Make __init__ consitstent across Store heirarchy
...
Add db_conn parameters to the `__init__` methods of the *Store classes, so that
they are all consistent, which makes the multiple inheritance work correctly
(and so that we can later extract mixins which can be used in the slavedstores)
2017-11-13 10:46:07 +00:00
Erik Johnston
ab335edb02
Revert "move _state_group_cache to statestore"
...
This reverts commit f5cf3638e9
2017-11-13 10:05:33 +00:00
Richard van der Hoff
f5cf3638e9
move _state_group_cache to statestore
...
this is internal to statestore, so let's keep it there.
2017-11-07 16:43:00 +00:00
Richard van der Hoff
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Erik Johnston
97c544f91f
Add _simple_update
2017-08-25 11:11:37 +01:00
Erik Johnston
b5e8d529e6
Define CACHE_SIZE_FACTOR once
2017-07-04 09:56:44 +01:00
Erik Johnston
c62c480dc6
Merge pull request #2259 from matrix-org/erikj/fix_state_woes
...
Fix bug where state_group tables got corrupted
2017-06-07 17:51:25 +01:00
Erik Johnston
197bd126f0
Fix bug where state_group tables got corrupted
...
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.
We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
2017-06-07 17:39:36 +01:00
Erik Johnston
21e255a8f1
Split the table in two
2017-06-01 14:50:46 +01:00
Erik Johnston
bfbc907cec
Prefill state caches
2017-05-15 15:11:13 +01:00
Erik Johnston
587f07543f
Revert "Prefill state caches"
2017-05-04 15:07:27 +01:00
Erik Johnston
a2c89a225c
Prefill state caches
2017-05-02 10:40:31 +01:00
Erik Johnston
e71940aa64
Use iter(items|values)
2017-03-24 10:57:02 +00:00
Erik Johnston
00957d1aa4
User Cursor.__iter__ instead of fetchall
...
This prevents unnecessary construction of lists
2017-03-23 17:53:49 +00:00
Luke Barnard
bbeeb97f75
Implement _simple_delete_many_txn, use it to delete devices
...
(But this doesn't implement the same for deleting access tokens or e2e keys.
Also respond to code review.
2017-03-13 17:53:23 +00:00
Richard van der Hoff
6ad71cc29d
Remove spurious SQL logging ( #1972 )
...
looks like the upsert function was accidentally sending sql logging to the
general logger. We already log the sql in `txn.execute`.
2017-03-08 18:00:44 +00:00
Erik Johnston
b84907bdbb
Intern table column names once
2017-02-28 14:38:16 +00:00
Erik Johnston
1a4f8022e6
Strip newlines from SQL queries
2017-02-23 11:15:31 +00:00
Morteza Araby
2849d3f29d
admin,storage: added more administrator functionalities
...
administrators can now:
- Set displayname of users
- Update user avatars
- Search for users by user_id
- Browse all users in a paginated API
- Reset user passwords
- Deactivate users
Helpers for doing paginated queries has also been added to storage
Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
2017-02-02 14:02:26 +01:00
Erik Johnston
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
Erik Johnston
d906206049
Increase state_group_cache_size
2017-01-17 11:31:08 +00:00
Erik Johnston
dd52d4de4c
Limit number of entries to prefill from cache
...
Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.
In future, we should add a better index to make it even faster.
2017-01-10 14:34:50 +00:00
Erik Johnston
524d61bf7e
Fix tests
2016-11-21 11:53:02 +00:00
Erik Johnston
7c9cdb2245
Store federation stream positions in the database
2016-11-21 11:33:08 +00:00
pik
e8b1d2a452
Refactor test_filter to use real DataStore
...
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
Erik Johnston
ba214a5e32
Remove lru option
2016-08-19 14:17:11 +01:00
Erik Johnston
949629291c
Do it in storage function
2016-08-16 17:05:34 +01:00
Erik Johnston
a2427981b7
Use cached get_user_by_access_token in slaves
2016-08-16 11:24:32 +01:00
Erik Johnston
89e786bd85
Doc get_next() context manager usage
2016-08-15 13:45:26 +01:00
Erik Johnston
d9664344ec
Rename table. Add docs.
2016-08-15 11:45:57 +01:00
Erik Johnston
64e7e11853
Implement cache replication stream
2016-08-15 11:16:45 +01:00
Erik Johnston
4d70d1f80e
Add some invalidations to a cache_stream
2016-08-15 11:15:17 +01:00
Erik Johnston
99bbd90b0d
Always run txn.after_callbacks
2016-08-15 09:45:44 +01:00
Richard van der Hoff
053e83dafb
More doc-comments
...
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
Erik Johnston
f328d95cef
Feature: Add deactivate account admin API
...
Allows server admins to "deactivate" accounts, which:
- Revokes all access tokens
- Removes all threepids
- Removes password
The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Mark Haines
80f34d7b57
Fix setting the _clock in SQLBaseStore
2016-06-02 15:23:56 +01:00
David Baker
d4503e25ed
Make deleting push actions more efficient
...
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
Erik Johnston
0c11c1be88
Spelling
2016-05-13 14:42:25 +01:00
