Moments, method of (in probability theory)

A method for determining a probability distribution by its moments (cf. Moment). Theoretically the method of moments is based on the uniqueness of the solution of the moment problem: If $ \alpha _ {0} = 1 , \alpha _ {1} , \alpha _ {2} \dots $ are constants, then under what conditions does there exist a unique distribution $ {\mathsf P} $ such that

$$ \alpha _ {n} = \int\limits x ^ {n} {\mathsf P} ( dx ) $$

are the moments of $ {\mathsf P} $ for all $ n $? There are various types of sufficient conditions for a distribution to be uniquely determined by its moments, for example, the Carleman condition

$$ \sum _ { n= } 1 ^ \infty \frac{1}{\alpha _ {2n} ^ {1 / 2n } } = \infty . $$

The use of the method of moments in the proof of limit theorems in probability theory and mathematical statistics is based on the correspondence between moments and the convergence of distributions: If $ F _ {n} $ is a sequence of distribution functions with finite moments $ \alpha _ {k} ( n) $ of any order $ k \geq 1 $, and if $ \alpha _ {k} ( n) \rightarrow \beta _ {k} $, as $ n \rightarrow \infty $, for each $ k $, then the $ \beta _ {k} $ are the moments of a distribution function $ F $; if $ F $ is uniquely determined by its moments, then as $ n \rightarrow \infty $, the $ F _ {n} $ converge weakly to $ F $. The method of moments in the case of convergence to a normal distribution was first treated by P.L. Chebyshev (1887), and a proof of the central limit theorem by the method of moments was accomplished by A.A. Markov (1898).

The method of moments in mathematical statistics is one of the general methods for finding statistical estimators of unknown parameters of a probability distribution from results of observations. The method of moments was first used to this end by K. Pearson (1894) to solve the problem of the approximation of an empirical distribution by a system of Pearson distributions (cf. Pearson distribution). The procedure in the method of moments is this: The moments of the empirical distribution are determined (the sample moments), equal in number to the number of parameters to be estimated; they are then equated to the corresponding moments of the probability distribution, which are functions of the unknown parameters; the system of equations thus obtained is solved for the parameters and the solutions are the required estimates. In practice the method of moments often leads to very simple calculations. Under fairly general conditions the method of moments allows one to find estimators that are asymptotically normal, have mathematical expectation that differs from the true value of the parameter only by a quantity of order $ 1 / n $ and standard deviation that deviates by a quantity of order $ 1/ \sqrt {n } $. However, the estimators found by the method of moments need not be best possible from the point of view of efficiency: their variance need not be minimal. For a normal distribution the method of moments leads to estimators that coincide with the estimators of the maximum-likelihood method, that is, with asymptotically-unbiased asymptotically-efficient estimators.

References

[1]	Yu.V. [Yu.V. Prokhorov] Prohorov, Yu.A. Rozanov, "Probability theory, basic concepts. Limit theorems, random processes" , Springer (1969) (Translated from Russian)
[2]	H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)
[3]	M.G. Kendall, A. Stuart, "The advanced theory of statistics" , 1 , Griffin (1987)