synapse-product/synapse
Eric Eastwood daf498e099
Fix 500 error on /messages when we accumulate more than 5 backward extremities (#11027)
Found while working on the Gitter backfill script and noticed
it only happened after we sent 7 batches, https://gitlab.com/gitterHQ/webapp/-/merge_requests/2229#note_665906390

When there are more than 5 backward extremities for a given depth,
backfill will throw an error because we sliced the extremity list
to 5 but then try to iterate over the full list. This causes
us to look for state that we never fetched and we get a `KeyError`.

Before when calling `/messages` when there are more than 5 backward extremities:
```
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/synapse/http/server.py", line 258, in _async_render_wrapper
    callback_return = await self._async_render(request)
  File "/usr/local/lib/python3.8/site-packages/synapse/http/server.py", line 446, in _async_render
    callback_return = await raw_callback_return
  File "/usr/local/lib/python3.8/site-packages/synapse/rest/client/room.py", line 580, in on_GET
    msgs = await self.pagination_handler.get_messages(
  File "/usr/local/lib/python3.8/site-packages/synapse/handlers/pagination.py", line 396, in get_messages
    await self.hs.get_federation_handler().maybe_backfill(
  File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 133, in maybe_backfill
    return await self._maybe_backfill_inner(room_id, current_depth, limit)
  File "/usr/local/lib/python3.8/site-packages/synapse/handlers/federation.py", line 386, in _maybe_backfill_inner
    likely_extremeties_domains = get_domains_from_state(states[e_id])
KeyError: '$zpFflMEBtZdgcMQWTakaVItTLMjLFdKcRWUPHbbSZJl'
```
2021-10-14 18:53:45 -05:00
..
_scripts Remove shebang line from module files (#10415) 2021-07-29 21:34:14 +01:00
api Remove the deprecated BaseHandler. (#11005) 2021-10-08 07:44:43 -04:00
app Port the Password Auth Providers module interface to the new generic interface (#10548) 2021-10-13 11:21:52 +00:00
appservice Use inline type hints in various other places (in synapse/) (#10380) 2021-07-15 11:02:43 +01:00
config Port the Password Auth Providers module interface to the new generic interface (#10548) 2021-10-13 11:21:52 +00:00
crypto Use direct references for configuration variables (part 5). (#10897) 2021-09-24 07:25:21 -04:00
events Add type hints to synapse.events.*. (#11066) 2021-10-13 07:24:07 -04:00
federation Strip "join_authorised_via_users_server" from join events which do not need it. (#10933) 2021-10-01 11:39:17 -04:00
groups Use direct references for configuration variables (part 4). (#10893) 2021-09-23 12:03:01 -04:00
handlers Fix 500 error on /messages when we accumulate more than 5 backward extremities (#11027) 2021-10-14 18:53:45 -05:00
http Relax ignore-missing-imports for modules that have stubs now and update mypy (#11006) 2021-10-08 14:49:41 +01:00
logging Stop user directory from failing if it encounters users not in the users table. (#11053) 2021-10-13 09:38:22 +00:00
metrics Relax ignore-missing-imports for modules that have stubs now and update mypy (#11006) 2021-10-08 14:49:41 +01:00
module_api Port the Password Auth Providers module interface to the new generic interface (#10548) 2021-10-13 11:21:52 +00:00
push disallow-untyped-defs for synapse.push (#11023) 2021-10-11 17:42:10 +01:00
replication Stop user directory from failing if it encounters users not in the users table. (#11053) 2021-10-13 09:38:22 +00:00
res Allow configuration of the oEmbed URLs. (#10714) 2021-08-31 18:37:07 -04:00
rest Ensure each charset is attempted only once during media preview. (#11089) 2021-10-14 18:51:44 +00:00
server_notices disallow-untyped-defs for synapse.server_notices (#11021) 2021-10-07 19:55:15 +01:00
spam_checker_api Remove redundant "coding: utf-8" lines (#9786) 2021-04-14 15:34:27 +01:00
state disallow-untyped-defs for synapse.state (#11004) 2021-10-06 17:55:25 +00:00
static Display an error page during failure of fallback UIA. (#10561) 2021-08-18 08:13:35 -04:00
storage Resolve and share state_groups for all historical events in batch (MSC2716) (#10975) 2021-10-13 17:44:00 -05:00
streams Remove unnecessary parentheses around tuples returned from methods (#10889) 2021-09-23 11:59:07 +01:00
util Relax ignore-missing-imports for modules that have stubs now and update mypy (#11006) 2021-10-08 14:49:41 +01:00
__init__.py 1.45.0rc2 2021-10-14 10:59:27 +01:00
event_auth.py Ensure that we reject events which use rejected events for auth (#10956) 2021-10-05 13:23:29 +01:00
notifier.py Add type hints for event streams. (#10856) 2021-09-21 13:34:26 -04:00
py.typed Mark Module API error imports as re-exported and mark Synapse as containing type annotations (#11054) 2021-10-13 08:42:41 +01:00
python_dependencies.py Validate new m.room.power_levels events (#10232) 2021-08-26 17:07:58 +01:00
server.py Port the Password Auth Providers module interface to the new generic interface (#10548) 2021-10-13 11:21:52 +00:00
types.py Remove unnecessary parentheses around tuples returned from methods (#10889) 2021-09-23 11:59:07 +01:00
visibility.py [pyupgrade] synapse/ (#10348) 2021-07-19 15:28:05 +01:00