In this work we demonstrate the use of adapted classical phase retrieval algorithms to perform control-free quantum phase estimation. We eliminate the costly controlled time evolution and Hadamard test commonly required to access the complex time-series needed to reconstruct the spectrum. This significant reduction of the number of coherent controlled-operations lowers the circuit depth and considerably simplifies the implementation of statistical quantum phase estimation in near-term devices. This seemingly impossible task can be achieved by extending the problem that one wishes to solve to one with a larger set of input signals while exploiting natural constraints on the signal and/or the spectrum. We leverage well-established algorithms that are widely used in the context of classical signal processing, demonstrating two complementary methods to do this, vectorial phase retrieval and two-dimensional phase retrieval. We numerically investigate the feasibility of both approaches for estimating the spectrum of the Fermi-Hubbard model and discuss their resilience to inherent statistical noise.