This paper proposes a novel regression-based approach to the estimation of Gaussian dynamic term structure models that avoids numerical optimization. This new estimator is an asymptotic least squares estimator defined by the no-arbitrage conditions upon which these models are built. We discuss some efficiency considerations of this estimator, and show that it is asymptotically equivalent to maximum likelihood estimation. Further, we note that our estimator remains easy-to-compute and asymptotically efficient in a variety of situations in which other recently proposed approaches lose their tractability. We provide an empirical application in the context of the Canadian bond market.