Hellinger distance

A distance between probability measures, expressed in terms of the Hellinger integral. Suppose that on a measurable space $ ( \mathfrak X , {\mathcal B} ) $ a family of probability measures $ \{ {\mathsf P} _ \theta \} $, $ \theta \in \Theta $, is given that are all absolutely continuous relative to some $ \sigma $- finite measure $ \mu $ on $ {\mathcal B} $.

The Hellinger distance between two measures $ {\mathsf P} _ {\theta _ {1} } $ and $ {\mathsf P} _ {\theta _ {2} } $( $ \theta _ {1} , \theta _ {2} \in \Theta $) is defined by the formula

$$ r ( \theta _ {1} , \theta _ {2} ) = \ \{ 2 [ 1 - H ( \theta _ {1} , \theta _ {2} )] \} ^ {1/2\ } = $$

$$ = \ \left \{ \int\limits _ { \mathfrak X } \left [ \sqrt { \frac{d {\mathsf P} _ {\theta _ {1} } }{d \mu } } - \sqrt { \frac{d {\mathsf P} _ {\theta _ {2} } }{d \mu } } \right ] ^ {2} d \mu \right \} ^ {1/2} , $$

where

$$ H ( \theta _ {1} , \theta _ {2} ) = \ \int\limits _ { \mathfrak X } \sqrt { \frac{d {\mathsf P} _ {\theta _ {1} } }{d \mu } } \sqrt { \frac{d {\mathsf P} _ {\theta _ {2} } }{d \mu } } d \mu $$

is the Hellinger integral. The Hellinger distance does not depend on the choice of the measure $ \mu $ and has the following properties:

1) $ 0 \leq r ( \theta _ {1} , \theta _ {2} ) \leq \sqrt 2 $;

2) $ r ( \theta _ {1} , \theta _ {2} ) = \sqrt 2 $ if and only if the measures $ {\mathsf P} _ {\theta _ {1} } $ and $ {\mathsf P} _ {\theta _ {2} } $ are mutually singular;

3) $ r ( \theta _ {1} , \theta _ {2} ) = 0 $ if and only if $ {\mathsf P} _ {\theta _ {1} } = {\mathsf P} _ {\theta _ {2} } $.

Let

$$ \| {\mathsf P} _ {\theta _ {1} } - {\mathsf P} _ {\theta _ {2} } \| = \ \sup _ {B \in {\mathcal B} } \ | {\mathsf P} _ {\theta _ {1} } ( B) - {\mathsf P} _ {\theta _ {2} } ( B) | = $$

$$ = \ { \frac{1}{2} } \int\limits _ { \mathfrak X } \left | \frac{d {\mathsf P} _ {\theta _ {1} } }{d \mu } - \frac{d {\mathsf P} _ {\theta _ {2} } }{d \mu } \right | d \mu $$

be the distance in variation between the measures $ {\mathsf P} _ {\theta _ {1} } $ and $ {\mathsf P} _ {\theta _ {2} } $. Then

$$ { \frac{1}{2} } r ^ {2} ( \theta _ {1} , \theta _ {2} ) \leq \ \| {\mathsf P} _ {\theta _ {1} } - {\mathsf P} _ {\theta _ {2} } \| \leq \ r ( \theta _ {1} , \theta _ {2} ). $$

References

[1]	H.H. Kuo, "Gaussian measures on Banach spaces" , Springer (1975)
[2]	H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)
[3]	I.A. Ibragimov, R.Z. [R.Z. Khas'minskii] Has'minskii, "Statistical estimation: asymptotic theory" , Springer (1981) (Translated from Russian)
[4]	V.M. Zolotarev, "Properties and relations of certain types of metrics" Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Akad. Nauk. USSR , 87 (1979) pp. 18–35; 206–212 (In Russian) (English summary)