Commit Graph

452 Commits

Author SHA1 Message Date
gnuxie
b99227c521 doc 2022-12-08 14:27:11 +00:00
gnuxie
b267b6199f Make a Revision class for revision ids rather than using a string.
Only thing I don't like is that ulid is public for the comparison method?
2022-12-08 14:24:19 +00:00
gnuxie
0680ac71e5 Stop synchronizing protected rooms unnecessarily. 2022-12-08 09:58:05 +00:00
gnuxie
5323ebe225 Add ulidx.
Going to use this for "revisions" of a policy list.
2022-12-07 20:18:51 +00:00
Gnuxie
1d3da94f38
Make autojoinOnlyIfManager true by default. (#451)
Also assert that we if `autojoinOnlyIfManager` is enabled that
the user has provided a space for `acceptInvitesFromSpace`.
It does sound like `autojoinOnlyIfManager` would imply that
anyone could send an invitation to the mjolnir if
`autojoinONlyIfManager` is false.
This has never been the case though, and it is not sensible
either, especially if `protectAllJoinedRooms` is also true.
Additionally the documentation in `config/default.yaml`
has always claimed that `autojoinOnlyIfManager` is "true by default".

This setting has confused users in #mjolnir:matrix.org before
Closes https://github.com/matrix-org/mjolnir/issues/436.

Also fixes an issue in the appservice where we require
`autojoinOnlyIfManager` to always be explicitly set to false
or it crashes any Mjolnir receiving an invite.
2022-12-07 17:00:05 +00:00
Gnuxie
5de0dae62a
Make updateForEvent actually update PolicyLists. (#448)
For some reason we were relying on a mjolnir listening to
`'PolicyList.batch'` to update policy lists.

This was exposing an implementation detail to Mjolnir
and including it as part of the implementation of
`PolicyList.updateForEvent()` which is supposed to cause
the `PolicyList` to update (eventually).

I am confident this was because of a need before batching was
introduced to get the changes to a policy list directly
from the method call to `PolicyList.update()`, whereas
now you can just listen to `PolicyList.update`.

The `'PolicyList.batch'` event has now been removed
and the PolicyList event batcher (`UpdateBatcher`)
now calls `PolicyList.update()` internally.
2022-12-07 13:57:39 +00:00
Gnuxie
704bb660c2
Refactor how we listen for matrix events. (#446)
* Refactor Matrix event listener in Mjolnir and ManagedMjolnir.

closes https://github.com/matrix-org/mjolnir/issues/411.

Issue #411 says that we have to be careful about room.join,
but this was before we figured how to make matrix-appservice-bridge
echo events sent by its own intents.

* Remove MatrixClientListener since it isn't actually needed.

* Protect which config values can be used for ManagedMjolnirs.

* Introduce MatrixSendClient

so listeners aren't accidentally added to a MatrixClient instead
of MatrixEmitter.

* doc

* Move provisioned mjolnir config to src/config.

This just aids maintance so whenever someone goes to change the config
of the bot they will see this and update it.

* doc for matrix intent listener.
2022-12-06 17:17:40 +00:00
Gnuxie
262e80acc2
Protect the first list we craete in Mjolnir appservice. (#444) 2022-12-05 16:57:12 +00:00
Gnuxie
d5e0baaac5
Missing newline and indentation in command handler help. (#445) 2022-12-05 16:56:57 +00:00
David Teller
2915757b7d
Very basic support for Sentry. (#398)
The Sentry package is very useful for monitoring runtime errors. With this PR,
we simply add the necessary mechanism to:

- log to sentry any uncaught error that reaches the toplevel, including startup errors.
2022-11-30 16:06:02 +01:00
Gnuxie
e35b855744
Gnuxie/appservice logging (#441)
* upgrade to matrix-appservice-bridge 8.0.0

this is so we can use their new logger

* Configure and use matrix-appservice-bridge's `Logger`

https://github.com/matrix-org/mjolnir/issues/422
Haven't changed all of the mjolnir components to use this,
just the appservice.
The fact that we've configured this properly means we get
logging from matrix-appservice-bridge components too (we didn't before).

* use try/catch instead
2022-11-30 13:30:38 +00:00
David Teller
38c5714027
CI: Scheduling daily builds (#426) 2022-11-29 15:02:58 +01:00
Gnuxie
38b18cda4f
Appservice tests weren't added to tsconfig.json properly. (#440)
Some minor fixes now that they have been.
2022-11-28 12:55:43 +00:00
Børlaag
f52f17e381
docs: fix small typo in moderators.md (#439)
Signed-off-by: Børlaag <borlaag@proton.me>

Signed-off-by: Børlaag <borlaag@proton.me>
2022-11-28 09:45:37 +00:00
gnuxie
1bc329b5e8 v1.6.1 2022-11-23 10:56:46 +00:00
Gnuxie
333c55e18c
Config fixes (#432)
* Use the npm package `config` to load the config.

This is what was used prior to https://github.com/matrix-org/mjolnir/pull/347.
It was a nice idea motivated to drop a dependency that was confusing.
It was just never followed through and was underestimated how much disruption it would cause.
It was also believed that the library would mean there could only ever be one global copy of the config,
It was followed up by:
https://github.com/matrix-org/mjolnir/pull/369
https://github.com/matrix-org/mjolnir/pull/357
https://github.com/matrix-org/mjolnir/pull/429
https://github.com/matrix-org/mjolnir/pull/397/files
https://github.com/matrix-org/mjolnir/issues/365

For simplicity sake I am reinstating the library.
The practice of loading default.yaml by default is also dangerous
and has led to issues multiple times in #mjolnir:matrix.org.
It is a sample and not a default.

In a following commit I will be adding the ability to specify the
config to use from the cli.

* Allow config to be specified with an explicit cli argument.

* Update doc to transition away from old config handling
2022-11-23 10:55:22 +00:00
Gnuxie
b2c0e23e36
Update CONTRIBUTING.md to show how to debug mjolnir. (#433) 2022-11-23 10:55:00 +00:00
Gnuxie
410ceebd62
Stop Mjolnir blindly protecting all policy lists at startup. (#431)
`Mjolnir.resyncAllJoinedRooms` needs policy lists to be loaded into mjolnir
in order to filter them out of the protect rooms set (unless explicitly protected).
This is so that you don't end up having mjolnir complain about protecting
a list which you have no control over, and are just watching (e.g. #matrix-org-coc-bl:matrix.org).
2022-11-22 15:34:50 +00:00
gnuxie
0845636d39 v1.6.0 2022-11-22 10:44:19 +00:00
Gnuxie
85cb0f0215
Update selfbuld docs to no longer require on old behaviour. (#429)
The package we used to load config in the past, `node-config`, would
default to `development`.
f54b419900/lib/config.js (L561)
https://github.com/matrix-org/mjolnir/pull/347
2022-11-22 10:38:04 +00:00
Gnuxie
aa50ab8e00
WordList was matching everything. (#427)
Made several improvements to catch edge cases.
2022-11-22 10:37:37 +00:00
dependabot[bot]
4b357732eb
Bump minimatch from 3.0.4 to 3.1.2 (#423)
Bumps [minimatch](https://github.com/isaacs/minimatch) from 3.0.4 to 3.1.2.
- [Release notes](https://github.com/isaacs/minimatch/releases)
- [Commits](https://github.com/isaacs/minimatch/compare/v3.0.4...v3.1.2)

---
updated-dependencies:
- dependency-name: minimatch
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-15 18:21:21 +00:00
Jess Porter
50f80f2392
manymjolnir appservice (#364)
Mjolnir can now be run as an application service,
meaning it will host multiple independent mjolnirs that can be requested by users.
If the user is on the same homeserver as the appservice is deployed on,
then they can provision a mjolnir via a widget https://github.com/matrix-org/mjolnir-widget.
Otherwise they can invite the appservice bot to a room they want to protect.
This will create them a mjolnir, a management room and a policy list.

The appservice shares the same docker image as the bot,
but is started slightly differently by specifying "appservice"
as the first argument to docker run (this s managed by `mjolnir-entrypoint.sh`. 
We could have used another Dockerfile for the appservice,
extending the existing one but we decided not to because there
would have been lots of fiddling around the entrypoint
and logistics involved around adding a tag for it via github actions.
Not to mention that this would be duplicating the image
just to run it with a different binary.

A list of followup issues can be found here https://github.com/issues?q=is%3Aopen+is%3Aissue+author%3AGnuxie+archived%3Afalse+label%3AA-Appservice.

Somewhat relevant and squashed commit messages(regrettably squashing because frankly these won't make sense in isolation): 

* draft widget backend

* add `managementRoomId` to `provisionNewMjolnir`

* remove ratelimits from appservice mjolnirs

* add /join endpoint to api backend


* tighter guard around room type in PolicyList

matrix-bot-sdk imporved the types for this

* enable esModuleInterop

* launch and use postgres in a container whilst using mx-tester


* limited access control

policy list used for access control

* Redesign initialization API of many mjolnir.

It's much harder to forget to initialize the components now that you have to in order to construct them in the first place.


* Ammend config not to clash with existing CI

this means that the appsrvice bot is now called 'mjolnir-bot' by default
which was easier than going through old code base and renaming


* Change entrypoint in Dockerfile so that we can start the appservice.

We could have used another Dockerfile for the appservice,
extending the exising one but we decided not to because there
would have been lots of fiddling around the entrypoint
and logistics involved around adding a tag for it via github actions.
Not to mention that this would be duplicating the image
just to run it with a different binary.

This solution is much simpler, backwards compatible, and conscious about the future.


Co-authored-by: gnuxie <gnuxie@element.io>
2022-11-15 18:03:06 +00:00
David Teller
81cd91c250
Unbitrotting ruleserver tests (#418) 2022-11-15 13:06:41 +01:00
Gnuxie
fb52e3dcb2
Improve the clarity of the unban command (#402)
Also fix a years long bug where the Flooding/Media protection wouldn't stop redacting users

Closes #393
Closes #394

* Warn about room level bans when using unban

https://github.com/matrix-org/mjolnir/issues/394

* Clear internal queues within protections when unban happens.

https://github.com/matrix-org/mjolnir/issues/393
2022-11-07 11:28:59 +00:00
Gnuxie
2c00ac4372
Always echo policy list changes (#401)
closes https://github.com/matrix-org/mjolnir/issues/392.
2022-11-07 11:27:54 +00:00
gnuxie
a970733d0b Update roomMemberTest for removal of mjolnir.protectedRooms object.
https://github.com/matrix-org/mjolnir/issues/370.
2022-10-19 15:21:51 +01:00
gnuxie
cfea765493 Fix list rooms command improperly iterating protected rooms. 2022-10-19 15:21:51 +01:00
gnuxie
c7ba745081 Stop ProtectedRoomsSet verifying permissions for unprotected rooms. 2022-10-19 15:21:51 +01:00
gnuxie
c537f2067e protected rooms config test
https://github.com/matrix-org/mjolnir/issues/370
2022-10-19 15:21:51 +01:00
gnuxie
cea6944c92 Status command can distinguish between protected and watched lists.
https://github.com/matrix-org/mjolnir/issues/370
2022-10-19 15:21:51 +01:00
gnuxie
5c2e4ab0bb Change list creation command to also protect the new room
https://github.com/matrix-org/mjolnir/issues/370
2022-10-19 15:21:51 +01:00
gnuxie
97673cdccb Make Mjolnir use ProtectedRoomsConfig
https://github.com/matrix-org/mjolnir/issues/370
2022-10-19 15:21:51 +01:00
gnuxie
58e36d4e23 Factor out protected rooms config management from Mjolnir.
The combination of `resyncJoinedRooms`, `unprotectedWatchedListRooms`,
`explicitlyProtectedRoomIds`, `protectedJoinedRoomIds` was incomprehensible.
https://github.com/matrix-org/mjolnir/issues/370

Separating out the management of `explicitlyProtectedRoomIds`, then
making sure all policy lists have to be explicitly protected
(in either setting of `config.protectAllJoinedRooms`) will make
this code much much simpler.
We will later change the `status` command to explicitly show
which lists are watched and which are watched and protected.
2022-10-19 15:21:51 +01:00
Gnuxie
da084328a9
Refactor list creation with MSC3784 support. (#386)
https://github.com/matrix-org/matrix-spec-proposals/pull/3784

This was extracted from the appservice mjolnir work to reduce review burden.
2022-10-19 10:33:12 +01:00
Gnuxie
5bd23ced9b
Access Control Unit (#378)
The ACL unit allows you to combine an policy lists and conveniently test users and servers against them.
The main motivation for this work is provide access control on who can provision and continue to use mjolnir instances in the appservice component.
We include a new recommendation type org.matrix.mjolnir.allow which can be used with user and server entity types to create allow lists.
We have also replaced the destructing of policy lists in applyServerACL and applyMemberBans (in ProtectedRooms.ts) with calls to the AccessControlUnit.
Adding commands to add/remove allowed entities is not something i want to do at the moment.
2022-10-18 16:28:11 +01:00
David Teller
7b0edadd17
Update tests and instructions to intercept reports also with v3 endpoint (#388)
In both our instructions and our tests, we use the r0 endpoint to intercept abuse reports. This endpoint is deprecated and not implemented by all clients. This PR updates the instructions and tests to the new endpoint.
2022-10-18 15:48:39 +02:00
mahdi1234
938b9fea8f
Fixed typo (#389) 2022-10-18 13:45:57 +01:00
David Teller
28265cec2e
CI: Let's try and cache mx-tester installs (#383) 2022-10-10 17:17:22 +02:00
Than Harrison
0379cc45fb
Fix small typo (#382)
* src/commands/CommandHandler.ts:
 - fix small typo that's bugging me in `!status` response.
   change "projection" -> "protection"

Signed-off-by: Than Harrison <infosecvoid@proton.me>

Signed-off-by: Than Harrison <infosecvoid@proton.me>
2022-10-10 12:35:35 +02:00
Gnuxie
77ad40e27a
Refactor protected rooms. (#371)
* Attempt to factor out protected rooms from Mjolnir.

This is useful to the appservice because it means we don't
have to wrap a Mjolnir that is designed to sync.

It's also useful if we later on want to have specific
settings per space.

It's also just a nice seperation between Mjolnir's needs while
syncing via client-server and the behaviour of syncing policy rooms.

 ### Things that have changed

- `ErrorCache` no longer a static class (phew), gets used by `ProtectedRooms`.
- `ManagementRoomOutput` class gets created to handle logging back to the management room.
- Responsibilities for syncing member bans and server ACL are handled by `ProtectedRooms`.
- Responsibilities for watched lists should be moved to `ProtectedRooms` if they haven't been.
- `EventRedactionQueue` is moved to `ProtectedRooms` since this needs to happen after
  member bans.
- ApplyServerAcls moved to `ProtectedRooms`
- ApplyMemberBans move to `ProtectedRooms`
- `logMessage` and `replaceRoomIdsWithPills` moved to `ManagementRoomOutput`.
- `resyncJoinedRooms` has been made a little more clear, though I am concerned about how often it does run because it does seem expensive.


* ProtectedRooms is not supposed to track joined rooms.

The reason is because it is supposed to represent a specific
set of rooms to protect, not do horrible logic
for working out what rooms mjolnir is supposed to protect.
2022-09-29 14:49:09 +01:00
Jess Porter
f108935d07
support compound consequences, switch WordList to consequences (#351) 2022-09-26 16:57:21 +01:00
Denis Lisov
89b7ec1a18
Add list shortcodes to status command output (#312) 2022-09-14 10:32:14 +01:00
Gnuxie
4d5447cb50
Rework the banning and unbanning of entities in PolicyLists. (#345)
* Rework the banning and unbanning of entities in PolicyLists.

1. We keep track of the event that created a list rule so that we
can remove the rule by having a way to determine the original state key for the rule.
This is because the state key of rules can be anything and should not be
relied on by Mjolnir to unban things (which it was doing).

2. The old scheme for producing a state key was causing for some entities to escape bans
https://github.com/matrix-org/mjolnir/issues/322.

We could have used a hash or something similar, but we know that
the reason for the `rule:${entity}` scheme existed was for ease of debugging
and finding rules in devtools. So instead we have followed a scheme simalar to
bridges where the first character of an mxid is replaced with an underscore.
Everything else just gets put into the state key. Since domains can't have '@'
and room ids, aliases can't either.

3. We have stopped the need for Mjolnir to wait for the next response from sync after banning,
unbanning an entity so that we can apply ACL's sooner.

* Use PolicyList's `banEntity` method to create imported rules.
2022-08-19 13:09:08 +01:00
Gnuxie
8bafa16495
Missing awaits on floating promises in tests. (#359)
If you add `"no-floating-promises": true` it's very easy
to find where this is done accidentally.
Not sure we can keep that on all the time yet though..
2022-08-18 13:09:03 +01:00
Erik Johnston
a2d919b57a
Fix example to have correct incantation (#358) 2022-08-18 11:25:07 +01:00
Jess Porter
10b7233e48
matrix.to urls for aliases don't need a via param (#348) 2022-08-18 09:37:30 +01:00
Jess Porter
625b62ce6d
use requested path to configs (#357) 2022-08-17 16:17:23 +01:00
Gnuxie
9bcb0b7a59
Replace acceptInvitesFromGroup with acceptInvitesFromSpace. (#338)
Replace acceptInvitesFromGroup with acceptInvitesFromSpace.

https://github.com/matrix-org/mjolnir/issues/125
https://github.com/matrix-org/mjolnir/issues/99

acceptInvitesFromGroup was implemented with an experimental api
that was a precursor to spaces which was refereed to
as either communities or groups.
Support for communities/groups ended in Synapse 1.61.0
https://github.com/matrix-org/synapse/releases/tag/v1.61.0.

To test we just edit the config dynamically which changes how the join room listener functions
though idk, shouldn't we have just made a new mjolnir instance
for this test, or changed the config before the test started somehow?


Co-authored-by: jesopo <github@lolnerd.net>
2022-08-17 10:05:23 +01:00
Gnuxie
f5a1a39861
audit yarn.lock (#356)
### Auditing the lock file

```
npm install --package-lock-only
npm audit fix
rm yarn.lock
yarn import
```

```
npm audit

json-schema  <0.4.0
Severity: critical
json-schema is vulnerable to Prototype Pollution - https://github.com/advisories/GHSA-896r-f27r-55mw
fix available via `npm audit fix`
node_modules/json-schema
  jsprim  0.3.0 - 1.4.1 || 2.0.0 - 2.0.1
  Depends on vulnerable versions of json-schema
  node_modules/jsprim

minimist  <1.2.6
Severity: critical
Prototype Pollution in minimist - https://github.com/advisories/GHSA-xvch-5gv4-984h
fix available via `npm audit fix`
node_modules/minimist

nanoid  3.0.0 - 3.1.30
Severity: moderate
Exposure of Sensitive Information to an Unauthorized Actor in nanoid - https://github.com/advisories/GHSA-qrpm-p2h7-hrv2
fix available via `npm audit fix`
node_modules/nanoid
node_modules/postcss/node_modules/nanoid
  mocha  8.2.0 - 9.1.4
  Depends on vulnerable versions of nanoid
  node_modules/mocha

5 vulnerabilities (2 moderate, 3 critical)

To address all issues, run:
  npm audit fix
```

### minimist

minimist@1.2.5
used by mocha, tslint and matrix-bot-sdk@0.5.19

via
```
MatrixClient::replyHtmlText
MatrixClient::replyHtmlNotice
MatrixClient::sendHtmlNotice
MatrixClient::sendHtmlTex
```

none of which we use.

### nanoid

As for nanoid this is used by mocha.
It's also used by postcss vis the bot sdk

```
├─┬ matrix-bot-sdk@0.5.19
│ └─┬ sanitize-html@2.7.1
│   └─┬ postcss@8.4.16
│     ├── nanoid@3.3.4

```
though unless i'm missing something nanoid@3.3.4 doesn't fit into the vulnerable versions  `3.0.0 - 3.1.30`


### json-schema

As for json-schema, it is used by jsprim@1.4.2 within 'validateJsonObjectJS'.
fortunately we depend on jsprim via the http-signatures@1.2.0 package which only use jsprim for rfc1123.
(which request depends upon in the matrix-bot-sdk).

```
├─┬ matrix-bot-sdk@0.5.19
│ ├─┬ request@2.88.2
│ │ ├─┬ http-signature@1.2.0
│ │ │ ├─┬ jsprim@1.4.2
│ │ │ │ ├── json-schema@0.4.0
2022-08-16 18:49:33 +01:00