This started out as just a way to find out why mjolnir was syncing with lists several times for each update to a policy list.
The main changes are
- Verbosity was irrelevant to the sync command but for some reason was an option.
Unfortunately all this did was suppress whether to tell you when it had finished, meaning it wouldn't
when verbose logging was disabled. Historically this was probably a parameter that got passed through
to applyServerAcl/applyUserBans, which can be horribly verbose, but they access the config directly.
- Stop emitting `'PolicyList.update'` when there are no changes.
- Include a revision ID for the `'PolicyList.update'`method and event.
- Use the revision ID in the `ProtectedRoomsSet` so that we don't unnecessarily resynchronize all rooms when the `'PolicyList.update'` event is received. Though not when the `sync` command is used. Since this is supposed to `sync` in the case when there is a state reset or otherwise or the user has changed some room settings.
- insert an await lock around the `PolicyList.update` method to avoid a race condition where a call can be started and finished within the extent of an existing call (via another task, this can happen if the server is slow with handling one request). `PolicyList.udpate` now has a helper that is synchronous to be called directly after requesting the room state. The reason for this is to enforce that no one `await`s while updating the policy list's cache of rules. Which is important because it is one of the biggest methods that I tolerate and visually checking for `await` is impossible.
- The revision ID uses a ULID, but this is unnecessary and could have just been a "dumb counter".
closes https://github.com/matrix-org/mjolnir/issues/447
The Sentry package is very useful for monitoring runtime errors. With this PR,
we simply add the necessary mechanism to:
- log to sentry any uncaught error that reaches the toplevel, including startup errors.
* upgrade to matrix-appservice-bridge 8.0.0
this is so we can use their new logger
* Configure and use matrix-appservice-bridge's `Logger`
https://github.com/matrix-org/mjolnir/issues/422
Haven't changed all of the mjolnir components to use this,
just the appservice.
The fact that we've configured this properly means we get
logging from matrix-appservice-bridge components too (we didn't before).
* use try/catch instead
Mjolnir can now be run as an application service,
meaning it will host multiple independent mjolnirs that can be requested by users.
If the user is on the same homeserver as the appservice is deployed on,
then they can provision a mjolnir via a widget https://github.com/matrix-org/mjolnir-widget.
Otherwise they can invite the appservice bot to a room they want to protect.
This will create them a mjolnir, a management room and a policy list.
The appservice shares the same docker image as the bot,
but is started slightly differently by specifying "appservice"
as the first argument to docker run (this s managed by `mjolnir-entrypoint.sh`.
We could have used another Dockerfile for the appservice,
extending the existing one but we decided not to because there
would have been lots of fiddling around the entrypoint
and logistics involved around adding a tag for it via github actions.
Not to mention that this would be duplicating the image
just to run it with a different binary.
A list of followup issues can be found here https://github.com/issues?q=is%3Aopen+is%3Aissue+author%3AGnuxie+archived%3Afalse+label%3AA-Appservice.
Somewhat relevant and squashed commit messages(regrettably squashing because frankly these won't make sense in isolation):
* draft widget backend
* add `managementRoomId` to `provisionNewMjolnir`
* remove ratelimits from appservice mjolnirs
* add /join endpoint to api backend
* tighter guard around room type in PolicyList
matrix-bot-sdk imporved the types for this
* enable esModuleInterop
* launch and use postgres in a container whilst using mx-tester
* limited access control
policy list used for access control
* Redesign initialization API of many mjolnir.
It's much harder to forget to initialize the components now that you have to in order to construct them in the first place.
* Ammend config not to clash with existing CI
this means that the appsrvice bot is now called 'mjolnir-bot' by default
which was easier than going through old code base and renaming
* Change entrypoint in Dockerfile so that we can start the appservice.
We could have used another Dockerfile for the appservice,
extending the exising one but we decided not to because there
would have been lots of fiddling around the entrypoint
and logistics involved around adding a tag for it via github actions.
Not to mention that this would be duplicating the image
just to run it with a different binary.
This solution is much simpler, backwards compatible, and conscious about the future.
Co-authored-by: gnuxie <gnuxie@element.io>
The combination of `resyncJoinedRooms`, `unprotectedWatchedListRooms`,
`explicitlyProtectedRoomIds`, `protectedJoinedRoomIds` was incomprehensible.
https://github.com/matrix-org/mjolnir/issues/370
Separating out the management of `explicitlyProtectedRoomIds`, then
making sure all policy lists have to be explicitly protected
(in either setting of `config.protectAllJoinedRooms`) will make
this code much much simpler.
We will later change the `status` command to explicitly show
which lists are watched and which are watched and protected.
### Auditing the lock file
```
npm install --package-lock-only
npm audit fix
rm yarn.lock
yarn import
```
```
npm audit
json-schema <0.4.0
Severity: critical
json-schema is vulnerable to Prototype Pollution - https://github.com/advisories/GHSA-896r-f27r-55mw
fix available via `npm audit fix`
node_modules/json-schema
jsprim 0.3.0 - 1.4.1 || 2.0.0 - 2.0.1
Depends on vulnerable versions of json-schema
node_modules/jsprim
minimist <1.2.6
Severity: critical
Prototype Pollution in minimist - https://github.com/advisories/GHSA-xvch-5gv4-984h
fix available via `npm audit fix`
node_modules/minimist
nanoid 3.0.0 - 3.1.30
Severity: moderate
Exposure of Sensitive Information to an Unauthorized Actor in nanoid - https://github.com/advisories/GHSA-qrpm-p2h7-hrv2
fix available via `npm audit fix`
node_modules/nanoid
node_modules/postcss/node_modules/nanoid
mocha 8.2.0 - 9.1.4
Depends on vulnerable versions of nanoid
node_modules/mocha
5 vulnerabilities (2 moderate, 3 critical)
To address all issues, run:
npm audit fix
```
### minimist
minimist@1.2.5
used by mocha, tslint and matrix-bot-sdk@0.5.19
via
```
MatrixClient::replyHtmlText
MatrixClient::replyHtmlNotice
MatrixClient::sendHtmlNotice
MatrixClient::sendHtmlTex
```
none of which we use.
### nanoid
As for nanoid this is used by mocha.
It's also used by postcss vis the bot sdk
```
├─┬ matrix-bot-sdk@0.5.19
│ └─┬ sanitize-html@2.7.1
│ └─┬ postcss@8.4.16
│ ├── nanoid@3.3.4
```
though unless i'm missing something nanoid@3.3.4 doesn't fit into the vulnerable versions `3.0.0 - 3.1.30`
### json-schema
As for json-schema, it is used by jsprim@1.4.2 within 'validateJsonObjectJS'.
fortunately we depend on jsprim via the http-signatures@1.2.0 package which only use jsprim for rfc1123.
(which request depends upon in the matrix-bot-sdk).
```
├─┬ matrix-bot-sdk@0.5.19
│ ├─┬ request@2.88.2
│ │ ├─┬ http-signature@1.2.0
│ │ │ ├─┬ jsprim@1.4.2
│ │ │ │ ├── json-schema@0.4.0
