The study of spin-glass dynamics, long considered the paradigmatic complex system, has reached important milestones. The availability of high-quality single crystals has allowed the experimental measurement of spin-glass coherence lengths of almost macroscopic dimensions, while the advent of special-purpose massive computers—by the Janus Collaboration—enables dynamical simulations that approach experimental timescales and length scales. This review provides an account of the quantitative convergence of these two avenues of research, with precise experimental measurements of the expected scaling laws and numerical reproduction of classic experimental results, such as memory and rejuvenation. The review opens with an examination of the defining spin-glass properties—randomness and frustration—and their experimental consequences. These apparently simple characteristics are shown to generate rich and complex physics. Models are introduced that enable quantitative dynamical descriptions, either analytically or through simulations. The many theoretical pictures of the low-temperature phase are reviewed. After a summary of the main numerical results in equilibrium, paying particular attention to the concept of temperature chaos, this review examines off-equilibrium dynamics in the absence of a magnetic field and shows how it can be related to the structure of the equilibrium spin-glass phase through the fluctuation-dissipation relations. The nonlinear response at a given temperature is then developed, including experiments and scaling in the vicinity of the spin-glass transition temperature Tg. The consequences of temperature change—including temperature chaos, rejuvenation, and memory—are reviewed. The interpretation of these phenomena requires several length scales relevant to dynamics to be identified, which, in turn, generates new insights. Finally, issues for future investigations are introduced, including what is to be “nailed down” theoretically, why the Ising Edwards-Anderson model is so successful at modeling spin-glass dynamics, and experiments yet to be undertaken. This review updates the field of spin glasses with broad application to a large variety of physical systems. In particular, this review tracks the progress of experiment, theory, and large-scale simulations. It highlights the importance of their synergy, from the inception of the field to the present day, and includes future opportunities for research.