Factorization theorem

factorization criterion

A theorem in the theory of statistical estimation giving a necessary and sufficient condition for a statistic $ T $ to be sufficient for a family of probability distributions $ \{ P _ \theta \} $( cf. Sufficient statistic).

Let $ X $ be a random vector taking values in a sample space $ ( \mathfrak X , {\mathcal B} , P _ \theta ) $, $ \theta \in \Theta $, where the family of probability distributions $ \{ P _ \theta \} $ is dominated by some measure $ \mu $, and let

$$ p ( x; \theta ) = \ \frac{dP _ \theta ( x) }{d \mu } ,\ \ \theta \in \Theta . $$

Further, let $ T = T ( X) $ be a statistic constructed from the observation vector of $ X $ and mapping the measurable space $ ( \mathfrak X , {\mathcal B} ) $ into the measurable space $ ( \mathfrak Y , {\mathcal A} ) $. Under these conditions the following question arises: When is $ T $ sufficient for the family $ \{ P _ \theta \} $? As an answer to this question, the factorization theorem asserts: For a statistic $ T $ to be sufficient for a family $ \{ P _ \theta \} $ that admits sufficient statistics, it is necessary and sufficient that for every $ \theta \in \Theta $ the probability density $ p ( x; \theta ) $ can be factorized in the following way:

$$ \tag{* } p ( x; \theta ) = \ g ( x) h ( T ( x); \theta ), $$

where $ g ( \cdot ) $ is a $ {\mathcal B} $- measurable function on $ ( \mathfrak X , {\mathcal B} ) $, and $ h ( \cdot , \theta ) $ is an $ {\mathcal A} $- measurable function on $ ( \mathfrak Y , {\mathcal A} ) $. The factorization theorem, beyond giving a criterion for sufficiency, in many cases enables one to determine the concrete form of the sufficient statistic $ T $ for which the density $ p ( x; \theta ) $ must factorize by the formula (*). In practice it is usually preferable to deal with the likelihood function $ L ( \theta ) = p ( X; \theta ) $ rather than with the density $ p ( x; \theta ) $. In terms of the likelihood function the condition (*) has the form $ L ( \theta ) = g ( X) h ( T; \theta ) $, explicitly containing $ T $.

References

[1]	R.A. Fischer, "On the mathematical foundations of theoretical statistics" Philos. Trans. Roy. Soc. London Ser. A , 222 (1922) pp. 309–368
[2]	J. Neyman, "Su un teorema concernente le cosiddette statistiche sufficienti" Giorn. Istit. Ital. Att. , 6 (1935) pp. 320–334
[3]	E.L. Lehmann, "Testing statistical hypotheses" , Wiley (1959)
[4]	I.A. Ibragimov, R.Z. [R.Z. Khas'minskii] Has'minskii, "Statistical estimation: asymptotic theory" , Springer (1981) (Translated from Russian)
[5]	P.R. Halmos, L.J. Savage, "Application of the Radon–Nikodym theorem to the theory of sufficient statistics" Ann. of Math. Statist. , 20 (1949) pp. 225–241
[a1]	D.R. Cox, D.V. Hinkley, "Theoretical statistics" , Chapman & Hall (1974) pp. 21