The concept of plasmonic "hotspots" is central to the broad field of nanophotonics. In surface-enhanced Raman scattering (SERS), hotspots can increase Raman scattering efficiency by orders of magnitude. Hotspot dimensions may range from a few nanometers down to the atomic scale and are able to generate SERS signals from single molecules. However, these single-molecule SERS signals often show significant fluctuations, and the concept of intense, localized, yet static hotspots has come into question. Recent experiments have shown these SERS intensity fluctuations (SIFs) to occur over an extremely wide range of timescales, from seconds to microseconds, due to the various physical mechanisms causing SERS and the dynamic nature of light-matter interaction at the nanoscale. The underlying source of single-molecule SERS fluctuations is therefore likely to be a complex interplay of several different effects at different timescales. A high-speed acquisition system that captures a full SERS spectrum with microsecond time resolution can therefore provide information about these dynamic processes. Here, we show an acquisition system that collects at a rate of 100,000 SERS spectra per second, allowing high-speed characterization. We find that while each individual SIF event will enhance a different portion of the SERS spectrum, including a single peak, over 10s to 100s of microseconds, the SIF events overall do not favor one region of the spectrum over another. These high-speed SIF events can therefore occur with relatively equal probability over a broad spectral range, covering both the anti-Stokes and the Stokes sides of the spectrum, sometimes leading to anomalously large anti-Stokes peaks. This indicates that both temporally and spectrally transient hotspots drive the SERS fluctuations at high speeds.