High-fidelity quantum memory for the silicon-vacancy defect in diamond