Quantum information technology strongly relies on the coupling of optical photons with narrowband quantum systems, such as quantum dots, color centers, and atomic systems. This coupling requires matching the optical wavelength and bandwidth to the desired system, which presents a considerable problem for most available sources of quantum light. Here we demonstrate the coupling of alkali dipole transitions with a tunable source of photon pairs. Our source is based on spontaneous parametric downconversion in a triply resonant whispering gallery mode resonator. For this, we have developed novel wavelength-tuning mechanisms that allow a coarse tuning to either the cesium or rubidium wavelength, with subsequent continuous fine-tuning to the desired transition. As a demonstration of the functionality of the source, we performed a heralded single-photon measurement of the atomic decay. We present a major advance in controlling the spontaneous downconversion process, which makes our bright source of heralded single photons now compatible with a plethora of narrowband resonant systems.