Erik Johnston
67cd4dad81
Implement MSC 1813 - Add room version to make APIs
...
We also implement `make_membership_event` converting the returned
room version to an event format version.
2019-01-23 16:51:46 +00:00
Erik Johnston
be1065af59
isort
2019-01-23 11:48:16 +00:00
Erik Johnston
c5a296b10c
Add support for persisting event format versions
...
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.
We specify the format version rather than the room version as:
1. We don't necessarily know the room version, existing events may be
either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
events in the same room reported to be of different versions, which
sounds annoying.
2019-01-23 11:30:01 +00:00
Erik Johnston
bc80b3f454
Add helpers for getting prev and auth events ( #4139 )
...
* Add helpers for getting prev and auth events
This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
Amber Brown
7232917f12
Disable frozen dicts by default ( #3987 )
2018-10-02 22:53:47 +10:00
Erik Johnston
89a76d1889
Fix handling of redacted events from federation
...
If we receive an event that doesn't pass their content hash check (e.g.
due to already being redacted) then we hit a bug which causes an
exception to be raised, which then promplty stops the event (and
request) from being processed.
This effects all sorts of federation APIs, including joining rooms with
a redacted state event.
2018-09-13 15:44:12 +01:00
Richard van der Hoff
c1f80effbe
Handle delta_ids being None in _update_context_for_auth_events
...
it's easier to create the new state group as a delta from the existing one.
(There's an outside chance this will help with
https://github.com/matrix-org/synapse/issues/3364 )
2018-07-23 22:06:50 +01:00
Erik Johnston
8b9f164fff
Comments
2018-07-23 17:43:01 +01:00
Erik Johnston
50c60e5fad
Only get cached state from context in persist_event
...
We don't want to bother pulling out the current state from the DB since
until we know we have to. Checking the context for state is just an
optimisation.
2018-07-23 17:21:40 +01:00
Erik Johnston
0faa3223cd
Fix missing attributes on workers.
...
This was missed during the transition from attribute to getter for
getting state from context.
2018-07-23 16:28:00 +01:00
Erik Johnston
999bcf9d01
Fix EventContext when using workers
...
We were:
1. Not correctly setting all attributes
2. Using defer.inlineCallbacks in a non-generator
2018-07-23 15:24:21 +01:00
Erik Johnston
4797ed000e
Update docstrings to make sense
2018-07-23 15:05:56 +01:00
Erik Johnston
027bc01a1b
Add support for updating state
2018-07-23 13:17:25 +01:00
Erik Johnston
440b8845b5
Make EventContext lazy load state
2018-07-23 12:56:56 +01:00
Erik Johnston
842cdece42
pep8
2018-07-23 12:31:35 +01:00
Erik Johnston
acbfdc3442
Refcator EventContext to accept state during init
2018-07-23 12:20:55 +01:00
Amber Brown
3132b89f12
Make the rest of the .iterwhatever go away ( #3562 )
2018-07-21 15:47:18 +10:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Amber Brown
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
Adrian Tschira
d9fe2b2d9d
Replace some more comparisons with six
...
plus a bonus b"" string I missed last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
Adrian Tschira
0c9db26260
add comment explaining attributeerror
2018-04-30 09:49:10 +02:00
Adrian Tschira
bfc2ade9b3
Make event properties raise AttributeError instead
...
They raised KeyError before. I'm changing this because the code uses
hasattr() to check for the presence of a key. This worked accidentally
before, because hasattr() silences all exceptions in python 2. However,
in python3, this isn't the case anymore.
I had a look around to see if anything depended on this raising a
KeyError and I couldn't find anything. Of course, I could have simply
missed it.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:16:59 +02:00
Erik Johnston
4810f7effd
Remove context.push_actions
2018-02-15 15:47:06 +00:00
Erik Johnston
106906a65e
Don't serialize current state over replication
2018-02-15 13:53:18 +00:00
Erik Johnston
24dd73028a
Add replication http endpoint for event sending
2018-02-07 10:32:32 +00:00
Erik Johnston
3d33eef6fc
Store state groups separately from events ( #2784 )
...
* Split state group persist into seperate storage func
* Add per database engine code for state group id gen
* Move store_state_group to StateReadStore
This allows other workers to use it, and so resolve state.
* Hook up store_state_group
* Fix tests
* Rename _store_mult_state_groups_txn
* Rename StateGroupReadStore
* Remove redundant _have_persisted_state_group_txn
* Update comments
* Comment compute_event_context
* Set start val for state_group_id_seq
... otherwise we try to recreate old state groups
* Update comments
* Don't store state for outliers
* Update comment
* Update docstring as state groups are ints
2018-02-06 14:31:24 +00:00
Richard van der Hoff
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Richard van der Hoff
631d7b87b5
Remove pointless create() method
...
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
2017-10-20 22:14:55 +01:00
David Baker
f878e6f8af
Spam checking: add the invitee to user_may_invite
2017-10-05 14:02:28 +01:00
David Baker
d8ce68b09b
spam check room publishing
2017-10-04 14:29:33 +01:00
David Baker
197c14dbcf
Add room creation checks to spam checker
...
Lets the spam checker deny attempts to create rooms and add aliases
to them.
2017-10-04 10:47:54 +01:00
David Baker
1e2ac54351
s/roomid/room_id/
2017-10-03 17:41:38 +01:00
David Baker
1e375468de
pass room id too
2017-10-03 17:13:14 +01:00
David Baker
537088e7dc
Actually write warpper function
2017-10-03 14:28:12 +01:00
David Baker
ef3a5ae787
Don't test is spam_checker not None
...
Sometimes it's a Mock object which is not none but is still not
what we're after
2017-09-27 11:24:19 +01:00
David Baker
8c06dd6071
Remove unintentional debugging
2017-09-27 10:31:14 +01:00
David Baker
60c78666ab
pep8
2017-09-27 10:26:13 +01:00
David Baker
6cd5fcd536
Make the spam checker a module
2017-09-26 19:20:23 +01:00
Matthew Hodgson
f496399ac4
fix thinko'd docstring
2017-09-22 15:34:14 +01:00
Richard van der Hoff
5ed109d59f
PoC for filtering spammy events ( #2456 )
...
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00
Erik Johnston
3e5a62ecd8
Add more granular event send metrics
2017-05-02 14:23:26 +01:00
Erik Johnston
46826bb078
Comment and remove spurious logging
2017-04-27 17:25:44 +01:00
Erik Johnston
1e166470ab
Fix tests
2017-04-26 16:23:30 +01:00
Erik Johnston
34e682d385
Fix invite state to always include all events
2017-04-26 16:18:08 +01:00
Richard van der Hoff
5068fb16a5
Refactoring and cleanups
...
A few non-functional changes:
* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
mind-bending conditional.
2017-03-17 15:06:07 +00:00
Erik Johnston
5d6bad1b3c
Optimise state resolution
2017-01-17 13:22:19 +00:00
Matthew Hodgson
2f4b2f4783
gah, fix mangled merge of 0.18.7 into develop
2017-01-07 04:00:42 +00:00
Mark Haines
f784980d2b
Only send events that originate on this server.
...
Or events that are sent via the federation "send_join" API.
This should match the behaviour from before v0.18.5 and #1635 landed.
2017-01-05 11:26:30 +00:00
Kegan Dougal
c3d963ac24
Review comments
2016-11-22 13:42:11 +00:00
Kegan Dougal
6d4e6d4cba
Also check for dict since sometimes they aren't frozen
2016-11-22 10:39:41 +00:00
Kegan Dougal
cea4e4e7b2
Glue only_event_fields into the sync rest servlet
2016-11-22 10:14:05 +00:00
Kegan Dougal
0a8b0eeca1
More tests
2016-11-22 09:59:27 +00:00
Kegan Dougal
70a2157b64
Start adding some tests
2016-11-21 17:52:45 +00:00
Kegan Dougal
f97511a1f3
Move event_fields filtering to serialize_event
...
Also make it an inclusive not exclusive filter, as the spec demands.
2016-11-21 17:42:16 +00:00
Erik Johnston
628e65721b
Add comments
2016-09-05 10:41:27 +01:00
Erik Johnston
9e25443db8
Move to storing state_groups_state as deltas
2016-09-01 14:31:26 +01:00
Erik Johnston
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
Erik Johnston
54cc69154e
Make None optional
2016-08-26 11:20:59 +01:00
Erik Johnston
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
Erik Johnston
17f4f14df7
Pull out event ids rather than full events for state
2016-08-25 13:42:44 +01:00
Erik Johnston
1d40373c9d
Include prev_content in redacted state events
2016-08-11 10:24:41 +01:00
Mark Haines
76503f95ed
Remove the is_new_state argument to persist event.
...
Move the checks for whether an event is new state inside persist
event itself.
This was harder than expected because there wasn't enough information
passed to persist event to correctly handle invites from remote servers
for new rooms.
2016-03-31 15:00:42 +01:00
Mark Haines
1e25f62ee6
Use a stream id generator to assign state group ids
2016-03-30 12:55:02 +01:00
Erik Johnston
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
Erik Johnston
f96526ffc2
Intern sender, event_id and room_id in events
2016-03-23 15:04:11 +00:00
Erik Johnston
fe9794706a
Intern type and state_key on events
2016-03-23 14:58:08 +00:00
Erik Johnston
58e207cd77
Don't assume existence of event_id in __str__
2016-03-18 14:31:44 +00:00
Erik Johnston
7b0d846407
Atomically persit push actions when we persist the event
2016-02-09 16:19:15 +00:00
Erik Johnston
2068678b8c
Make Event objects behave more like dicts
2016-01-18 14:43:50 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
Mark Haines
306415391d
Only add the user_id if the sender is present
2015-12-01 11:14:48 +00:00
Mark Haines
bde8d78b8a
Copy rather than move the fields to shuffle between a v1 and a v2 event.
...
This should make all v1 APIs compatible with v2 clients. While still
allowing v1 clients to access the fields.
This makes the documentation easier since we can just document the v2
format and explain that some of the fields, in some of the APIs are
duplicated for backwards compatibility, rather than having to document
two separate event formats.
2015-11-30 17:46:35 +00:00
Richard van der Hoff
3c3fc6b268
Flatten the /sync response to remove the event_map
2015-11-19 10:51:11 +00:00
Mark Haines
d58edd98e9
Update the other place check_joined_room is called
2015-10-30 11:15:37 +00:00
Paul "LeoNerd" Evans
243a79d291
Surely we don't need to preserve 'events_default' twice
2015-10-16 18:25:19 +01:00
Erik Johnston
49ae42bbe1
Bundle in some room state in the unsigned bit of the invite when sending to invited servers
2015-09-10 14:25:54 +01:00
Erik Johnston
1f7642efa9
Fix bug where we didn't correctly serialize the redacted_because key over federation
2015-08-21 09:36:07 +01:00
Erik Johnston
fb47c3cfbe
Rename key and values for m.room.history_visibility. Support 'invited' value
2015-07-06 13:05:52 +01:00
Erik Johnston
00ab882ed6
Add m.room.history_visibility to list of auth events
2015-07-03 10:31:24 +01:00
Erik Johnston
ca580ef862
Don't copy twice
2015-05-29 15:02:55 +01:00
Erik Johnston
a7b65bdedf
Add config option to turn off freezing events. Use new encode_json api and ujson.loads
2015-05-29 12:17:33 +01:00
Paul "LeoNerd" Evans
1b988b051b
Store the rejected reason in (Frozen)Event structs
2015-03-25 19:06:05 +00:00
Erik Johnston
b2e6ee5b43
Remove concept of context.auth_events, instead use context.current_state
2015-03-16 13:06:23 +00:00
Erik Johnston
7eef84a95b
pyflakes
2015-02-11 16:52:22 +00:00
Erik Johnston
ddb816cf60
Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict
2015-02-11 15:44:28 +00:00
Erik Johnston
c0462dbf15
Rearrange persist_event so that do all the queries that need to be done before returning early if we have already persisted that event.
2015-02-04 10:16:51 +00:00
Erik Johnston
9bace3a367
Actually, the old prune_event function was non-deterministic, so no point keeping it around :(
2015-02-03 15:32:17 +00:00
Erik Johnston
0dd3aea319
Keep around the old (buggy) version of the prune_event function so that we can use it to check signatures for events on old servers
2015-02-03 14:58:30 +00:00
Erik Johnston
7d897f5bfc
Merge pull request #43 from matrix-org/rejections
...
Rejections
2015-01-30 16:11:14 +00:00
Mark Haines
6dc92d3427
Merge pull request #41 from matrix-org/client_v2_sync
...
Client v2 sync
2015-01-30 15:54:25 +00:00
Erik Johnston
e0b7c521cb
Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
Erik Johnston
2ebf795c0a
Merge branch 'develop' of github.com:matrix-org/synapse into rejections
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 11:10:37 +00:00
Erik Johnston
c1c7b39827
Fix bug where we changes in outlier in metadata dict propogated to other events
2015-01-30 10:30:54 +00:00
Erik Johnston
78015948a7
Initial implementation of auth conflict resolution
2015-01-29 16:52:33 +00:00
Mark Haines
b9c442c85c
Include transaction ids in unsigned section of events in the sync results for the clients that made those requests
2015-01-29 02:46:00 +00:00
Mark Haines
1b4a164c02
Add support for formatting events in the way a v2 client expects
2015-01-29 02:34:35 +00:00
Mark Haines
a56008842b
Start implementing incremental initial sync
2015-01-27 16:24:22 +00:00
Mark Haines
436513068d
Start implementing the non-incremental sync portion of the v2 /sync API
2015-01-26 18:53:31 +00:00
Mark Haines
7f6f3f9d62
Pass the current time to serialize event, rather than passing an
...
HS and getting a clock from it and calling time_msec on the clock.
Remove the serialize_event method from the HS since it is no longer
needed.
2015-01-26 16:11:28 +00:00
Erik Johnston
b1b85753d7
Add support for storing rejected events in EventContext and data stores
2015-01-22 15:50:17 +00:00
Mark Haines
3891597eb3
Remove unused functions
2015-01-13 15:57:26 +00:00
Erik Johnston
38e3241eb7
Merge branch 'hotfixes-v0.6.1b' of github.com:matrix-org/synapse into develop
2015-01-13 10:01:22 +00:00
Erik Johnston
39585bf556
Insert 'age' into top level when returning events to clients
2015-01-13 09:57:32 +00:00
Kegan Dougal
edb557b2ad
Return the raw federation event rather than adding extra keys for federation data.
2015-01-08 14:28:08 +00:00
Kegan Dougal
5720ab59e0
Add 'raw' query parameter to expose the event graph and signatures to savvy clients.
2015-01-08 13:57:40 +00:00
Erik Johnston
a039e2544c
Remove unused import
2015-01-07 09:48:03 +00:00
Erik Johnston
dfa05f0cd6
Optimize FrozenEvent creation
2015-01-06 18:51:03 +00:00
Mark Haines
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
Mark Haines
5dbe820e9a
Remove unneeded federation keys from events
2014-12-19 12:16:26 +00:00
Mark Haines
3c7857e49b
clean up coding style a bit
2014-12-16 15:24:03 +00:00
Erik Johnston
882dc8dcab
Persist internal_metadata
2014-12-16 13:17:09 +00:00
Erik Johnston
65b2e49429
Fix pyflakes
2014-12-15 17:35:37 +00:00
Erik Johnston
f280929a12
Use frozenutils
2014-12-15 17:31:36 +00:00
Erik Johnston
670dcdfc14
Remove unused functions
2014-12-15 16:16:58 +00:00
Erik Johnston
b75adaedca
Finish up upgrade script
2014-12-15 16:14:34 +00:00
Erik Johnston
c8dd3314d6
Fix bug where we ignored event_edge_hashes table
2014-12-15 13:55:22 +00:00
Erik Johnston
63810c777d
Validate message, topic and name event contents
2014-12-12 11:01:09 +00:00
Erik Johnston
9191292b0f
Fix prev_content
2014-12-11 15:16:55 +00:00
Erik Johnston
8cdebce470
Fix redactions. Fix 'age' key
2014-12-11 13:25:19 +00:00
Erik Johnston
b245ee34ed
Add some basic event validation
2014-12-10 17:59:47 +00:00
Erik Johnston
cabead6194
Actually fix bug when uploading state with empty state_key
2014-12-10 14:49:52 +00:00
Erik Johnston
02db7eb209
Fix bug when uploading state with empty state_key
2014-12-10 14:02:48 +00:00
Erik Johnston
02db1fd2e7
Fix AttributeError
2014-12-10 12:00:05 +00:00
Erik Johnston
102d2373b4
Add __str__ to FrozenEvent
2014-12-10 11:38:08 +00:00
Erik Johnston
95aa903ffa
Try and figure out how and why signatures are being changed.
2014-12-10 11:37:47 +00:00
Erik Johnston
8c48450682
Add PEP8 newlines
2014-12-09 10:58:31 +00:00
Erik Johnston
609c31e8df
More bug fixes
2014-12-08 17:50:56 +00:00
Erik Johnston
d044121168
Various typos and bug fixes.
2014-12-08 09:08:26 +00:00
Erik Johnston
6630e1b579
Start making more things use EventContext rather than event.*
2014-12-05 16:20:48 +00:00
Erik Johnston
c31dba86ec
Convert rest and handlers to use new event structure
2014-12-04 15:50:01 +00:00
Erik Johnston
5d7c9ab789
Begin converting things to use the new Event structure
2014-12-04 11:27:59 +00:00
Erik Johnston
75b4329aaa
WIP for new way of managing events.
2014-12-03 16:07:21 +00:00
Erik Johnston
c1e66800a9
Begin fleshing out a new Event object
2014-12-02 11:40:22 +00:00
