The combination of `resyncJoinedRooms`, `unprotectedWatchedListRooms`,
`explicitlyProtectedRoomIds`, `protectedJoinedRoomIds` was incomprehensible.
https://github.com/matrix-org/mjolnir/issues/370
Separating out the management of `explicitlyProtectedRoomIds`, then
making sure all policy lists have to be explicitly protected
(in either setting of `config.protectAllJoinedRooms`) will make
this code much much simpler.
We will later change the `status` command to explicitly show
which lists are watched and which are watched and protected.
The ACL unit allows you to combine an policy lists and conveniently test users and servers against them.
The main motivation for this work is provide access control on who can provision and continue to use mjolnir instances in the appservice component.
We include a new recommendation type org.matrix.mjolnir.allow which can be used with user and server entity types to create allow lists.
We have also replaced the destructing of policy lists in applyServerACL and applyMemberBans (in ProtectedRooms.ts) with calls to the AccessControlUnit.
Adding commands to add/remove allowed entities is not something i want to do at the moment.
In both our instructions and our tests, we use the r0 endpoint to intercept abuse reports. This endpoint is deprecated and not implemented by all clients. This PR updates the instructions and tests to the new endpoint.
* src/commands/CommandHandler.ts:
- fix small typo that's bugging me in `!status` response.
change "projection" -> "protection"
Signed-off-by: Than Harrison <infosecvoid@proton.me>
Signed-off-by: Than Harrison <infosecvoid@proton.me>
* Attempt to factor out protected rooms from Mjolnir.
This is useful to the appservice because it means we don't
have to wrap a Mjolnir that is designed to sync.
It's also useful if we later on want to have specific
settings per space.
It's also just a nice seperation between Mjolnir's needs while
syncing via client-server and the behaviour of syncing policy rooms.
### Things that have changed
- `ErrorCache` no longer a static class (phew), gets used by `ProtectedRooms`.
- `ManagementRoomOutput` class gets created to handle logging back to the management room.
- Responsibilities for syncing member bans and server ACL are handled by `ProtectedRooms`.
- Responsibilities for watched lists should be moved to `ProtectedRooms` if they haven't been.
- `EventRedactionQueue` is moved to `ProtectedRooms` since this needs to happen after
member bans.
- ApplyServerAcls moved to `ProtectedRooms`
- ApplyMemberBans move to `ProtectedRooms`
- `logMessage` and `replaceRoomIdsWithPills` moved to `ManagementRoomOutput`.
- `resyncJoinedRooms` has been made a little more clear, though I am concerned about how often it does run because it does seem expensive.
* ProtectedRooms is not supposed to track joined rooms.
The reason is because it is supposed to represent a specific
set of rooms to protect, not do horrible logic
for working out what rooms mjolnir is supposed to protect.
* Rework the banning and unbanning of entities in PolicyLists.
1. We keep track of the event that created a list rule so that we
can remove the rule by having a way to determine the original state key for the rule.
This is because the state key of rules can be anything and should not be
relied on by Mjolnir to unban things (which it was doing).
2. The old scheme for producing a state key was causing for some entities to escape bans
https://github.com/matrix-org/mjolnir/issues/322.
We could have used a hash or something similar, but we know that
the reason for the `rule:${entity}` scheme existed was for ease of debugging
and finding rules in devtools. So instead we have followed a scheme simalar to
bridges where the first character of an mxid is replaced with an underscore.
Everything else just gets put into the state key. Since domains can't have '@'
and room ids, aliases can't either.
3. We have stopped the need for Mjolnir to wait for the next response from sync after banning,
unbanning an entity so that we can apply ACL's sooner.
* Use PolicyList's `banEntity` method to create imported rules.
If you add `"no-floating-promises": true` it's very easy
to find where this is done accidentally.
Not sure we can keep that on all the time yet though..
Replace acceptInvitesFromGroup with acceptInvitesFromSpace.
https://github.com/matrix-org/mjolnir/issues/125https://github.com/matrix-org/mjolnir/issues/99
acceptInvitesFromGroup was implemented with an experimental api
that was a precursor to spaces which was refereed to
as either communities or groups.
Support for communities/groups ended in Synapse 1.61.0
https://github.com/matrix-org/synapse/releases/tag/v1.61.0.
To test we just edit the config dynamically which changes how the join room listener functions
though idk, shouldn't we have just made a new mjolnir instance
for this test, or changed the config before the test started somehow?
Co-authored-by: jesopo <github@lolnerd.net>
### Auditing the lock file
```
npm install --package-lock-only
npm audit fix
rm yarn.lock
yarn import
```
```
npm audit
json-schema <0.4.0
Severity: critical
json-schema is vulnerable to Prototype Pollution - https://github.com/advisories/GHSA-896r-f27r-55mw
fix available via `npm audit fix`
node_modules/json-schema
jsprim 0.3.0 - 1.4.1 || 2.0.0 - 2.0.1
Depends on vulnerable versions of json-schema
node_modules/jsprim
minimist <1.2.6
Severity: critical
Prototype Pollution in minimist - https://github.com/advisories/GHSA-xvch-5gv4-984h
fix available via `npm audit fix`
node_modules/minimist
nanoid 3.0.0 - 3.1.30
Severity: moderate
Exposure of Sensitive Information to an Unauthorized Actor in nanoid - https://github.com/advisories/GHSA-qrpm-p2h7-hrv2
fix available via `npm audit fix`
node_modules/nanoid
node_modules/postcss/node_modules/nanoid
mocha 8.2.0 - 9.1.4
Depends on vulnerable versions of nanoid
node_modules/mocha
5 vulnerabilities (2 moderate, 3 critical)
To address all issues, run:
npm audit fix
```
### minimist
minimist@1.2.5
used by mocha, tslint and matrix-bot-sdk@0.5.19
via
```
MatrixClient::replyHtmlText
MatrixClient::replyHtmlNotice
MatrixClient::sendHtmlNotice
MatrixClient::sendHtmlTex
```
none of which we use.
### nanoid
As for nanoid this is used by mocha.
It's also used by postcss vis the bot sdk
```
├─┬ matrix-bot-sdk@0.5.19
│ └─┬ sanitize-html@2.7.1
│ └─┬ postcss@8.4.16
│ ├── nanoid@3.3.4
```
though unless i'm missing something nanoid@3.3.4 doesn't fit into the vulnerable versions `3.0.0 - 3.1.30`
### json-schema
As for json-schema, it is used by jsprim@1.4.2 within 'validateJsonObjectJS'.
fortunately we depend on jsprim via the http-signatures@1.2.0 package which only use jsprim for rfc1123.
(which request depends upon in the matrix-bot-sdk).
```
├─┬ matrix-bot-sdk@0.5.19
│ ├─┬ request@2.88.2
│ │ ├─┬ http-signature@1.2.0
│ │ │ ├─┬ jsprim@1.4.2
│ │ │ │ ├── json-schema@0.4.0
The implementation is rubbish, as it doesn't avoid the exponential backoff
Remove default rate limit testing.
It doesn't work. No there really isn't more to say about it
you're welcome to dispute it if you're going to do the work investigating. I'm not.
We used to have a test here that tested whether Mjolnir was going to carry out a redact order the default limits in a reasonable time scale.
Now I think that's never going to happen without writing a new algorithm for respecting rate limiting.
Which is not something there is time for.
https://github.com/matrix-org/synapse/pull/13018
Synapse rate limits were broken and very permitting so that's why the current hack worked so well.
Now it is not broken, so our rate limit handling is.
b850e4554c
Honestly I don't think we can expect anyone to be able to use Mjolnir under default rate limits.
well, it's not quite simple as broken, but it is broken. With the default level in synapse (which is what matrix.org uses) it is struggling to redact 15 messages within 5 minutes. that means 5 messages over the burst count. This is ofc ontop mjolnir sending reactions / responding to replies (which isn't much but... enough to mess with the rate limiter since ofc, Synapse tells requests to wait x amount of time before trying again, but that doesn't help for concurrent requests since ofc there's only 1 slot available at that future time. This means Synapse just wacks everything with exponentially longer shit without many (or any?) events going through
it used to be fine
because rate limiting in synapse used to be a lot more liberal because it was "broken" or something, that's not me saying it's broken that's just what synapse devs say which is probably true.
if all requests went into a queue then yeah you could eliminate one problem
but that's a lot of work and i don't think we should be doing it
cos no one uses mjolnir like this anyways
