Fix bug where a notary server would sometimes forget old keys.