We analyze the uncertainty quantification for the limit of a Stochastic Approximation (SA) algorithm. In our setup, this limit Phi* is defined as the zero of an intractable function and is modeled as uncertain through a parameter Theta: we aim at deriving the probabilistic distribution of Phi*(Theta), given a probability distribution for Theta. We introduce the so-called Uncertainty Quantification for SA (UQSA) algorithm, an SA algorithm in increasing dimension for computing the basis coefficients of a chaos expansion of Phi*(·) on an orthogonal basis of a suitable Hilbert space. UQSA returns a finite set of coefficients, it provides an approximation of the expectation, of the variance-covariance matrix and of higher order moments of Phi*(Theta). The almost-sure and Lp-convergences of UQSA, in the Hilbert space, are established under mild, tractable conditions and constitute original results, not covered by the existing literature for convergence analysis of infinite dimensional SA algorithms. Finally, UQSA is illustrated and the role of its design parameters is discussed through a numerical analysis.