Commit Graph

296 Commits

Author SHA1 Message Date
Mark Haines
892e70ec84 Add APIs for adding and removing tags from rooms 2015-10-28 16:06:57 +00:00
Erik Johnston
f9340ea0d5 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 13:29:02 +01:00
Erik Johnston
ec398af41c Expose error more nicely 2015-10-13 11:43:43 +01:00
Erik Johnston
40b6a5aad1 Split out the schema preparation and update logic into its own module 2015-10-13 11:38:48 +01:00
Erik Johnston
c85c912562 Add basic full text search impl. 2015-10-09 15:48:31 +01:00
Daniel Wagner-Hall
6d59ffe1ce Add some docstrings 2015-09-22 13:47:40 +01:00
Daniel Wagner-Hall
7213588083 Implement configurable stats reporting
SYN-287

This requires that HS owners either opt in or out of stats reporting.

When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Daniel Wagner-Hall
a0b181bd17 Remove completely unused concepts from codebase
Removes device_id and ClientInfo

device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.

This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
Daniel Wagner-Hall
cecbd636e9 /tokenrefresh POST endpoint
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Erik Johnston
8f4165628b Add index receipts_linearized_room_stream 2015-08-17 14:43:54 +01:00
Erik Johnston
7dec0b2bee Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-07 19:28:39 +01:00
Paul "LeoNerd" Evans
0db40d3e93 Don't complain about extra .pyc files we find while hunting for database schemas 2015-08-07 17:22:11 +01:00
Erik Johnston
b8e386db59 Change Cache to not use *args in its interface 2015-08-07 11:52:21 +01:00
Erik Johnston
5989637f37 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-13 13:50:57 +01:00
Erik Johnston
716e426933 Fix various typos 2015-07-07 10:55:31 +01:00
Mark Haines
b5770f8947 Add store for client end to end keys 2015-07-06 18:46:47 +01:00
Erik Johnston
901f56fa63 Add tables for receipts 2015-06-30 15:29:47 +01:00
Erik Johnston
1e365e88bd Bump schema version 2015-06-11 15:50:39 +01:00
David Baker
63a7b3ad1e Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them. 2015-06-04 16:16:01 +01:00
Erik Johnston
4f1d984e56 Add index on events 2015-05-13 17:22:26 +01:00
Mark Haines
84e1cacea4 Bump schema version 2015-05-08 15:58:14 +01:00
Erik Johnston
657298cebd Don't lock user_ips table for upsert. 2015-05-01 10:46:48 +01:00
Erik Johnston
970a9b9d2b We can't use REPLACE when upgrading databases now we have postgres 2015-04-29 13:55:44 +01:00
Mark Haines
e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Erik Johnston
327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
David Baker
8a5be236e0 pep8 2015-04-27 11:49:18 +01:00
David Baker
b02e1006b9 Run database check before daemonizing, at the cost of database hygiene. 2015-04-27 11:46:00 +01:00
Erik Johnston
5a95cd4442 Rename user_ips.user -> user_id 2015-04-14 13:54:09 +01:00
Erik Johnston
58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Erik Johnston
ce797ad373 Bump schema version 2015-04-10 11:22:30 +01:00
Erik Johnston
4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston
0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00
Erik Johnston
6bc9edd8b2 Fix prepare_sqlite3_database's convert_param_style 2015-04-07 12:13:58 +01:00
Erik Johnston
05a35d62b6 Bump database version 2015-04-07 12:10:15 +01:00
Erik Johnston
9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal
4edcbcee3b Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Erik Johnston
0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
Erik Johnston
9115421ace Use _simple_upsert 2015-03-24 16:17:39 +00:00
Paul "LeoNerd" Evans
a63b4f7101 Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale 2015-03-20 18:25:49 +00:00
Erik Johnston
f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston
dc0c989ef4 Give sensible names for '_simple_...' transactions 2015-03-20 15:59:18 +00:00
Erik Johnston
87db64b839 Rearrange storage modules 2015-03-20 14:11:38 +00:00
Erik Johnston
cb8162d3d1 Rearrange storage modules 2015-03-20 13:52:56 +00:00
Erik Johnston
d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Kegan Dougal
835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal
f9232c7917 Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Paul "LeoNerd" Evans
59a5f012cc Also give _execute() a description 2015-03-11 17:19:17 +00:00
Paul "LeoNerd" Evans
099e4b88d8 Add a description to storage layer's _execute_and_decode() 2015-03-11 17:08:57 +00:00
David Baker
8916acbc13 These aren't defined for redacted events so don't crash 2015-03-10 11:21:37 +00:00
Kegan Dougal
21fd84dcb8 Use seconds; start gluing in the AS scheduler into the AS handler. 2015-03-09 17:01:19 +00:00
Kegan Dougal
01c099d9ef Add appservice txns sql schema 2015-03-06 17:16:47 +00:00
Erik Johnston
d56c01fff4 Note that we don't specify execution order 2015-03-04 15:10:05 +00:00
Erik Johnston
17d319a20d s/schema_deltas/applied_schema_deltas/ 2015-03-04 15:06:22 +00:00
Erik Johnston
5681264faa s/%r/%s/ 2015-03-04 14:21:53 +00:00
Erik Johnston
f701197227 Add example directory structures in doc 2015-03-04 14:20:14 +00:00
Erik Johnston
16dd87d848 Don't assume db conn is a Context Manager.
Twisted adbapi wrapped connections aren't context managers.
2015-03-04 14:03:41 +00:00
Erik Johnston
5eefd1f618 Add unique constraint on schema_version.lock schema. Use conflict clause in sql. 2015-03-04 13:52:18 +00:00
Erik Johnston
640e53935d Use context manager with db conn to correctly commit and rollback 2015-03-04 13:43:17 +00:00
Erik Johnston
c3530c3fb3 More docs. Rename 'schema/current' to 'schema/full_schemas' 2015-03-04 13:34:11 +00:00
Erik Johnston
811355ccd0 Add some docs and remove unused variables 2015-03-04 13:11:01 +00:00
Erik Johnston
82b34e813d SYN-67: Finish up implementing new database schema management 2015-03-04 12:04:19 +00:00
Erik Johnston
8d33adfbbb SYN-67: Begin changing the way we handle schema versioning 2015-03-02 18:23:55 +00:00
Erik Johnston
9f03553f48 Add missing comma 2015-03-02 16:38:40 +00:00
Erik Johnston
b41dc68773 We purposefully don't have a version 14 delta script. 2015-03-02 16:36:19 +00:00
Kegan Dougal
c3c01641d2 Run deltas and bump user_version in upgrade script 2015-03-02 13:38:57 +00:00
Erik Johnston
2c29ed3e84 Use absolute path when loading delta sql files 2015-02-17 17:22:24 +00:00
Erik Johnston
1a989c436c Bump schema version 2015-02-17 15:45:55 +00:00
Mark Haines
0d872f5aa6 Merge pull request #50 from matrix-org/application-services
Application Services
2015-02-13 15:06:14 +00:00
Mark Haines
183b3d4e47 Prepare the database whenever a connection is opened from the db_pool so that in-memory databases will work 2015-02-13 14:38:24 +00:00
Erik Johnston
42bc56dad3 Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson 2015-02-11 17:01:38 +00:00
Erik Johnston
7eef84a95b pyflakes 2015-02-11 16:52:22 +00:00
Kegan Dougal
f2fdcb7c4b Merge branch 'develop' into application-services 2015-02-11 16:43:26 +00:00
Erik Johnston
ddb816cf60 Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict 2015-02-11 15:44:28 +00:00
Mark Haines
ddc25cf4e2 Invalidate the cache for an event if it is redacted 2015-02-11 15:23:28 +00:00
Mark Haines
f5a70e0d2e Add a cache for get_event 2015-02-11 15:01:15 +00:00
Erik Johnston
4ebbaf0d43 Blunty replace json with simplejson 2015-02-11 14:23:10 +00:00
Kegan Dougal
951690e54d Merge branch 'develop' into application-services 2015-02-05 14:28:03 +00:00
Erik Johnston
03d415a6a2 Brief comment on why we do some things on every call to persist_event and not others 2015-02-04 10:40:59 +00:00
Erik Johnston
c0462dbf15 Rearrange persist_event so that do all the queries that need to be done before returning early if we have already persisted that event. 2015-02-04 10:16:51 +00:00
Erik Johnston
02be8da5e1 Add doc to get_event 2015-02-03 17:34:07 +00:00
Erik Johnston
e7ca813dd4 Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering. 2015-02-03 10:39:41 +00:00
Kegan Dougal
1a2de0c5fe Implement txns for AS (un)registration. 2015-02-02 17:39:41 +00:00
Erik Johnston
941f59101b Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator. 2015-02-02 16:56:01 +00:00
Kegan Dougal
a006d168c5 Actually merge into develop. 2015-02-02 16:05:34 +00:00
Kegan Dougal
c059c9fea5 Merge branch 'develop' into application-services
Conflicts:
	synapse/handlers/__init__.py
	synapse/storage/__init__.py
2015-02-02 15:57:59 +00:00
Mark Haines
9ec9d6f2cb Merge pull request #42 from matrix-org/replication_split
Replication split
2015-01-30 15:14:10 +00:00
Erik Johnston
91015ad008 Remove merge conflict 2015-01-30 14:58:54 +00:00
Erik Johnston
4f7fe63b6d Remember to add schema file to list 2015-01-30 14:57:53 +00:00
Erik Johnston
fdd2ac495a Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
2015-01-30 14:57:33 +00:00
Erik Johnston
471c47441d Merge pull request #37 from matrix-org/client_v2_filter
Client v2 filter
2015-01-30 14:56:08 +00:00
David Baker
322a047502 Add room member count condition and default rule to make a noise on rooms of only 2 people. 2015-01-30 14:46:45 +00:00
Erik Johnston
84b78c3b5f Merge branch 'rejections_storage' of github.com:matrix-org/synapse into replication_split 2015-01-30 14:17:47 +00:00
Erik Johnston
e0b7c521cb Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
Mark Haines
9d8f798a3f Merge changes from develop 2015-01-29 14:55:27 +00:00
David Baker
60b143a52e Move pushers delta to v12 and bump schema version 2015-01-28 15:48:28 +00:00
Kegan Dougal
42876969b9 Add basic application_services SQL, and hook up parts of the appservice store to read from it. 2015-01-28 11:59:38 +00:00
Paul "LeoNerd" Evans
8398f19bce Created schema delta 2015-01-27 19:00:09 +00:00
Paul "LeoNerd" Evans
06cc147012 Initial stab at real SQL storage implementation of user filter definitions 2015-01-27 18:46:03 +00:00
Paul "LeoNerd" Evans
54e513b4e6 Move storage of user filters into real datastore layer; now have to mock it out in the REST-level tests 2015-01-27 17:48:13 +00:00
Kegan Dougal
51449e0665 Add appservice handler and store. Glue together rest > handler > store. 2015-01-27 15:50:28 +00:00