Improve rebuild speed for the "synapse-workers" docker image.