Traditionally, when a user clicks "report" in a Matrix client, this goes
to the homeserver administrator, who often is the wrong person for the job.
MSC3215 introduces a mechanism to let clients cooperate with a bot to send
the report to the moderator instead. Client support has landed in Element Web
(behind a Labs flag) in in 2021. This allows Mjölnir to serve as the partner
bot.
This PR:
- creates an OpenMetrics server that enables collecting performance data from this process by e.g. a Prometheus server;
- exposes as metrics the performance of http requests with MatrixBot.
Further metrics may of course be added.
This started out as just a way to find out why mjolnir was syncing with lists several times for each update to a policy list.
The main changes are
- Verbosity was irrelevant to the sync command but for some reason was an option.
Unfortunately all this did was suppress whether to tell you when it had finished, meaning it wouldn't
when verbose logging was disabled. Historically this was probably a parameter that got passed through
to applyServerAcl/applyUserBans, which can be horribly verbose, but they access the config directly.
- Stop emitting `'PolicyList.update'` when there are no changes.
- Include a revision ID for the `'PolicyList.update'`method and event.
- Use the revision ID in the `ProtectedRoomsSet` so that we don't unnecessarily resynchronize all rooms when the `'PolicyList.update'` event is received. Though not when the `sync` command is used. Since this is supposed to `sync` in the case when there is a state reset or otherwise or the user has changed some room settings.
- insert an await lock around the `PolicyList.update` method to avoid a race condition where a call can be started and finished within the extent of an existing call (via another task, this can happen if the server is slow with handling one request). `PolicyList.udpate` now has a helper that is synchronous to be called directly after requesting the room state. The reason for this is to enforce that no one `await`s while updating the policy list's cache of rules. Which is important because it is one of the biggest methods that I tolerate and visually checking for `await` is impossible.
- The revision ID uses a ULID, but this is unnecessary and could have just been a "dumb counter".
closes https://github.com/matrix-org/mjolnir/issues/447
Also assert that we if `autojoinOnlyIfManager` is enabled that
the user has provided a space for `acceptInvitesFromSpace`.
It does sound like `autojoinOnlyIfManager` would imply that
anyone could send an invitation to the mjolnir if
`autojoinONlyIfManager` is false.
This has never been the case though, and it is not sensible
either, especially if `protectAllJoinedRooms` is also true.
Additionally the documentation in `config/default.yaml`
has always claimed that `autojoinOnlyIfManager` is "true by default".
This setting has confused users in #mjolnir:matrix.org before
Closes https://github.com/matrix-org/mjolnir/issues/436.
Also fixes an issue in the appservice where we require
`autojoinOnlyIfManager` to always be explicitly set to false
or it crashes any Mjolnir receiving an invite.
For some reason we were relying on a mjolnir listening to
`'PolicyList.batch'` to update policy lists.
This was exposing an implementation detail to Mjolnir
and including it as part of the implementation of
`PolicyList.updateForEvent()` which is supposed to cause
the `PolicyList` to update (eventually).
I am confident this was because of a need before batching was
introduced to get the changes to a policy list directly
from the method call to `PolicyList.update()`, whereas
now you can just listen to `PolicyList.update`.
The `'PolicyList.batch'` event has now been removed
and the PolicyList event batcher (`UpdateBatcher`)
now calls `PolicyList.update()` internally.
* Refactor Matrix event listener in Mjolnir and ManagedMjolnir.
closes https://github.com/matrix-org/mjolnir/issues/411.
Issue #411 says that we have to be careful about room.join,
but this was before we figured how to make matrix-appservice-bridge
echo events sent by its own intents.
* Remove MatrixClientListener since it isn't actually needed.
* Protect which config values can be used for ManagedMjolnirs.
* Introduce MatrixSendClient
so listeners aren't accidentally added to a MatrixClient instead
of MatrixEmitter.
* doc
* Move provisioned mjolnir config to src/config.
This just aids maintance so whenever someone goes to change the config
of the bot they will see this and update it.
* doc for matrix intent listener.
The Sentry package is very useful for monitoring runtime errors. With this PR,
we simply add the necessary mechanism to:
- log to sentry any uncaught error that reaches the toplevel, including startup errors.
`Mjolnir.resyncAllJoinedRooms` needs policy lists to be loaded into mjolnir
in order to filter them out of the protect rooms set (unless explicitly protected).
This is so that you don't end up having mjolnir complain about protecting
a list which you have no control over, and are just watching (e.g. #matrix-org-coc-bl:matrix.org).
The combination of `resyncJoinedRooms`, `unprotectedWatchedListRooms`,
`explicitlyProtectedRoomIds`, `protectedJoinedRoomIds` was incomprehensible.
https://github.com/matrix-org/mjolnir/issues/370
Separating out the management of `explicitlyProtectedRoomIds`, then
making sure all policy lists have to be explicitly protected
(in either setting of `config.protectAllJoinedRooms`) will make
this code much much simpler.
We will later change the `status` command to explicitly show
which lists are watched and which are watched and protected.
* Attempt to factor out protected rooms from Mjolnir.
This is useful to the appservice because it means we don't
have to wrap a Mjolnir that is designed to sync.
It's also useful if we later on want to have specific
settings per space.
It's also just a nice seperation between Mjolnir's needs while
syncing via client-server and the behaviour of syncing policy rooms.
### Things that have changed
- `ErrorCache` no longer a static class (phew), gets used by `ProtectedRooms`.
- `ManagementRoomOutput` class gets created to handle logging back to the management room.
- Responsibilities for syncing member bans and server ACL are handled by `ProtectedRooms`.
- Responsibilities for watched lists should be moved to `ProtectedRooms` if they haven't been.
- `EventRedactionQueue` is moved to `ProtectedRooms` since this needs to happen after
member bans.
- ApplyServerAcls moved to `ProtectedRooms`
- ApplyMemberBans move to `ProtectedRooms`
- `logMessage` and `replaceRoomIdsWithPills` moved to `ManagementRoomOutput`.
- `resyncJoinedRooms` has been made a little more clear, though I am concerned about how often it does run because it does seem expensive.
* ProtectedRooms is not supposed to track joined rooms.
The reason is because it is supposed to represent a specific
set of rooms to protect, not do horrible logic
for working out what rooms mjolnir is supposed to protect.
* Rework the banning and unbanning of entities in PolicyLists.
1. We keep track of the event that created a list rule so that we
can remove the rule by having a way to determine the original state key for the rule.
This is because the state key of rules can be anything and should not be
relied on by Mjolnir to unban things (which it was doing).
2. The old scheme for producing a state key was causing for some entities to escape bans
https://github.com/matrix-org/mjolnir/issues/322.
We could have used a hash or something similar, but we know that
the reason for the `rule:${entity}` scheme existed was for ease of debugging
and finding rules in devtools. So instead we have followed a scheme simalar to
bridges where the first character of an mxid is replaced with an underscore.
Everything else just gets put into the state key. Since domains can't have '@'
and room ids, aliases can't either.
3. We have stopped the need for Mjolnir to wait for the next response from sync after banning,
unbanning an entity so that we can apply ACL's sooner.
* Use PolicyList's `banEntity` method to create imported rules.
Replace acceptInvitesFromGroup with acceptInvitesFromSpace.
https://github.com/matrix-org/mjolnir/issues/125https://github.com/matrix-org/mjolnir/issues/99
acceptInvitesFromGroup was implemented with an experimental api
that was a precursor to spaces which was refereed to
as either communities or groups.
Support for communities/groups ended in Synapse 1.61.0
https://github.com/matrix-org/synapse/releases/tag/v1.61.0.
To test we just edit the config dynamically which changes how the join room listener functions
though idk, shouldn't we have just made a new mjolnir instance
for this test, or changed the config before the test started somehow?
Co-authored-by: jesopo <github@lolnerd.net>
* Stop the config being global (in almost all contexts).
* make sure unit test has a config
* Make failing word list more visible
* Only use Healthz from index.ts
Not really sure how useful it is anyways?
* banListTest would applyACL before rules appeared in `/state`.
Mjolnir will call applyServerAcls several times while a policy list is being updated, sometimes concurrently. This means a request to set a server ACL in a room which has old data can finish after a more recent recent request with the correct ACL. This means that the old ACL gets applied to the rooms (for a moment).
This is a follow up from 551065815e
* Only allow one invocation of applyServerAcls at a time as to not conflict with each other by using a promise chain.
We don't use the throttle queue because we don't want to be blocked by other background tasks.
We don't make another throttle queue because we don't want throttling and we don't want to delay the ACL application, which can happen even with throttle time of 0.
Towards opinions in PolicyLists.
This changeset is part of an ongoing effort to implement "opinions"
within policy lists, as per MSC3847.
For the time being:
- we rename BanList into PolicyList;
- we cleanup a little dead code;
- we replace a few `string`s with `enum`;
- `ListRule` becomes an abstract class with two concrete subclasses `ListRuleBan` and `ListRuleOpinion`.
* Have the callback work on the packed event Obj.
And also explicitly pack the Mjolnir object inside the event.
* Add VS ignores.
* Commit suggested review changes.
- Add bind to callback
- Add type cast for event packed object
* Fix missing parenthesis.
* Don't pass Mjolnir obj to handleReport().
Co-authored-by: David Teller <davidt@element.io>
There is no reason to call process.exit() from `index.ts` or in `Mjolnir.start()` because
https://nodejs.org/api/process.html#warning-using-uncaughtexception-correctly
>The 'uncaughtException' event is emitted when an uncaught JavaScript exception bubbles all the way back to the event loop. By default, Node.js handles such exceptions by printing the stack trace to stderr and exiting with code 1, overriding any previously set process.exitCode. Adding a handler for the 'uncaughtException' event overrides this default behaviour.
* standard protection consequences
* add integration test to make sure good users aren't banned
* the less far `event` propagates, the better
* better document consequence.ts
* improve innocent user integration test
* switch to room.event emit
* Test for batching ACL.
* Batch events from sync within BanList.
* Introduce the BanList.batch event to the BanList emitter to let Mjolnir sync after new events have been added from sync.
Fixes#203
This is an experimental ruleserver that will serve the combined rules from
the active policy lists to a Synapse module over a web api.
This makes it easier to communicate changes in policy lists to Synapse workers
that do not have an immediate view over all of the policy rooms at
the same time.
This also allows us to express moderation actions to the homeserver
beyond what is currently expressible via MSC2313 policy
lists.
Only shows changes to lists made by other accounts (than the one used by Mjolnir).
Displays when rules are added, removed and modified by either replacing the state event or redacting them.
