Correctly clear caches when we delete a room.