Namespaces
Variants
Actions

Difference between revisions of "Hellinger distance"

From Encyclopedia of Mathematics
Jump to: navigation, search
(Importing text file)
 
m (tex encoded by computer)
 
Line 1: Line 1:
A distance between probability measures, expressed in terms of the [[Hellinger integral|Hellinger integral]]. Suppose that on a measurable space <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468901.png" /> a family of probability measures <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468902.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468903.png" />, is given that are all absolutely continuous relative to some <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468904.png" />-finite measure <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468905.png" /> on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468906.png" />.
+
<!--
 +
h0468901.png
 +
$#A+1 = 24 n = 0
 +
$#C+1 = 24 : ~/encyclopedia/old_files/data/H046/H.0406890 Hellinger distance
 +
Automatically converted into TeX, above some diagnostics.
 +
Please remove this comment and the {{TEX|auto}} line below,
 +
if TeX found to be correct.
 +
-->
  
The Hellinger distance between two measures <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468907.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468908.png" /> (<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h0468909.png" />) is defined by the formula
+
{{TEX|auto}}
 +
{{TEX|done}}
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689010.png" /></td> </tr></table>
+
A distance between probability measures, expressed in terms of the [[Hellinger integral|Hellinger integral]]. Suppose that on a measurable space  $  ( \mathfrak X , {\mathcal B} ) $
 +
a family of probability measures  $  \{ {\mathsf P} _  \theta  \} $,
 +
$  \theta \in \Theta $,
 +
is given that are all absolutely continuous relative to some  $  \sigma $-
 +
finite measure  $  \mu $
 +
on  $  {\mathcal B} $.
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689011.png" /></td> </tr></table>
+
The Hellinger distance between two measures  $  {\mathsf P} _ {\theta _ {1}  } $
 +
and  $  {\mathsf P} _ {\theta _ {2}  } $(
 +
$  \theta _ {1} , \theta _ {2} \in \Theta $)
 +
is defined by the formula
 +
 
 +
$$
 +
r ( \theta _ {1} , \theta _ {2} )  = \
 +
\{ 2 [ 1 - H ( \theta _ {1} , \theta _ {2} )] \}  ^ {1/2\ } =
 +
$$
 +
 
 +
$$
 +
= \
 +
\left \{ \int\limits _ { \mathfrak X } \left [ \sqrt {
 +
\frac{d {\mathsf P} _ {\theta _ {1}  } }{d \mu }
 +
} - \sqrt {
 +
\frac{d {\mathsf P} _ {\theta _ {2}  } }{d \mu }
 +
} \right ]  ^ {2}  d \mu \right \}  ^ {1/2} ,
 +
$$
  
 
where
 
where
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689012.png" /></td> </tr></table>
+
$$
 +
H ( \theta _ {1} , \theta _ {2} )  = \
 +
\int\limits _ { \mathfrak X }
 +
\sqrt {
 +
\frac{d {\mathsf P} _ {\theta _ {1}  } }{d \mu }
 +
}
 +
\sqrt {
 +
\frac{d {\mathsf P} _ {\theta _ {2}  } }{d \mu }
 +
}  d \mu
 +
$$
  
is the Hellinger integral. The Hellinger distance does not depend on the choice of the measure <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689013.png" /> and has the following properties:
+
is the Hellinger integral. The Hellinger distance does not depend on the choice of the measure $  \mu $
 +
and has the following properties:
  
1) <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689014.png" />;
+
1) 0 \leq  r ( \theta _ {1} , \theta _ {2} ) \leq  \sqrt 2 $;
  
2) <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689015.png" /> if and only if the measures <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689016.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689017.png" /> are mutually singular;
+
2) $  r ( \theta _ {1} , \theta _ {2} ) = \sqrt 2 $
 +
if and only if the measures $  {\mathsf P} _ {\theta _ {1}  } $
 +
and $  {\mathsf P} _ {\theta _ {2}  } $
 +
are mutually singular;
  
3) <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689018.png" /> if and only if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689019.png" />.
+
3) $  r ( \theta _ {1} , \theta _ {2} ) = 0 $
 +
if and only if $  {\mathsf P} _ {\theta _ {1}  } = {\mathsf P} _ {\theta _ {2}  } $.
  
 
Let
 
Let
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689020.png" /></td> </tr></table>
+
$$
 +
\| {\mathsf P} _ {\theta _ {1}  } -
 +
{\mathsf P} _ {\theta _ {2}  } \|  = \
 +
\sup _ {B \in {\mathcal B} } \
 +
| {\mathsf P} _ {\theta _ {1}  } ( B) -
 +
{\mathsf P} _ {\theta _ {2}  } ( B) | =
 +
$$
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689021.png" /></td> </tr></table>
+
$$
 +
= \
 +
{
 +
\frac{1}{2}
 +
} \int\limits _ { \mathfrak X } \left |
 +
\frac{d {\mathsf P} _ {\theta _ {1}  } }{d \mu }
 +
-  
 +
\frac{d {\mathsf P} _ {\theta _ {2}  } }{d \mu }
 +
\right |  d \mu
 +
$$
  
be the distance in variation between the measures <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689022.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689023.png" />. Then
+
be the distance in variation between the measures $  {\mathsf P} _ {\theta _ {1}  } $
 +
and $  {\mathsf P} _ {\theta _ {2}  } $.  
 +
Then
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/h/h046/h046890/h04689024.png" /></td> </tr></table>
+
$$
 +
{
 +
\frac{1}{2}
 +
} r  ^ {2} ( \theta _ {1} , \theta _ {2} )  \leq  \
 +
\| {\mathsf P} _ {\theta _ {1}  } -
 +
{\mathsf P} _ {\theta _ {2}  } \|  \leq  \
 +
r ( \theta _ {1} , \theta _ {2} ).
 +
$$
  
 
====References====
 
====References====
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  H.H. Kuo,  "Gaussian measures on Banach spaces" , Springer  (1975)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  H. Cramér,  "Mathematical methods of statistics" , Princeton Univ. Press  (1946)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  I.A. Ibragimov,  R.Z. [R.Z. Khas'minskii] Has'minskii,  "Statistical estimation: asymptotic theory" , Springer  (1981)  (Translated from Russian)</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  V.M. Zolotarev,  "Properties and relations of certain types of metrics"  ''Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Akad. Nauk. USSR'' , '''87'''  (1979)  pp. 18–35; 206–212  (In Russian)  (English summary)</TD></TR></table>
 
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  H.H. Kuo,  "Gaussian measures on Banach spaces" , Springer  (1975)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  H. Cramér,  "Mathematical methods of statistics" , Princeton Univ. Press  (1946)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  I.A. Ibragimov,  R.Z. [R.Z. Khas'minskii] Has'minskii,  "Statistical estimation: asymptotic theory" , Springer  (1981)  (Translated from Russian)</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  V.M. Zolotarev,  "Properties and relations of certain types of metrics"  ''Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Akad. Nauk. USSR'' , '''87'''  (1979)  pp. 18–35; 206–212  (In Russian)  (English summary)</TD></TR></table>

Latest revision as of 22:10, 5 June 2020


A distance between probability measures, expressed in terms of the Hellinger integral. Suppose that on a measurable space $ ( \mathfrak X , {\mathcal B} ) $ a family of probability measures $ \{ {\mathsf P} _ \theta \} $, $ \theta \in \Theta $, is given that are all absolutely continuous relative to some $ \sigma $- finite measure $ \mu $ on $ {\mathcal B} $.

The Hellinger distance between two measures $ {\mathsf P} _ {\theta _ {1} } $ and $ {\mathsf P} _ {\theta _ {2} } $( $ \theta _ {1} , \theta _ {2} \in \Theta $) is defined by the formula

$$ r ( \theta _ {1} , \theta _ {2} ) = \ \{ 2 [ 1 - H ( \theta _ {1} , \theta _ {2} )] \} ^ {1/2\ } = $$

$$ = \ \left \{ \int\limits _ { \mathfrak X } \left [ \sqrt { \frac{d {\mathsf P} _ {\theta _ {1} } }{d \mu } } - \sqrt { \frac{d {\mathsf P} _ {\theta _ {2} } }{d \mu } } \right ] ^ {2} d \mu \right \} ^ {1/2} , $$

where

$$ H ( \theta _ {1} , \theta _ {2} ) = \ \int\limits _ { \mathfrak X } \sqrt { \frac{d {\mathsf P} _ {\theta _ {1} } }{d \mu } } \sqrt { \frac{d {\mathsf P} _ {\theta _ {2} } }{d \mu } } d \mu $$

is the Hellinger integral. The Hellinger distance does not depend on the choice of the measure $ \mu $ and has the following properties:

1) $ 0 \leq r ( \theta _ {1} , \theta _ {2} ) \leq \sqrt 2 $;

2) $ r ( \theta _ {1} , \theta _ {2} ) = \sqrt 2 $ if and only if the measures $ {\mathsf P} _ {\theta _ {1} } $ and $ {\mathsf P} _ {\theta _ {2} } $ are mutually singular;

3) $ r ( \theta _ {1} , \theta _ {2} ) = 0 $ if and only if $ {\mathsf P} _ {\theta _ {1} } = {\mathsf P} _ {\theta _ {2} } $.

Let

$$ \| {\mathsf P} _ {\theta _ {1} } - {\mathsf P} _ {\theta _ {2} } \| = \ \sup _ {B \in {\mathcal B} } \ | {\mathsf P} _ {\theta _ {1} } ( B) - {\mathsf P} _ {\theta _ {2} } ( B) | = $$

$$ = \ { \frac{1}{2} } \int\limits _ { \mathfrak X } \left | \frac{d {\mathsf P} _ {\theta _ {1} } }{d \mu } - \frac{d {\mathsf P} _ {\theta _ {2} } }{d \mu } \right | d \mu $$

be the distance in variation between the measures $ {\mathsf P} _ {\theta _ {1} } $ and $ {\mathsf P} _ {\theta _ {2} } $. Then

$$ { \frac{1}{2} } r ^ {2} ( \theta _ {1} , \theta _ {2} ) \leq \ \| {\mathsf P} _ {\theta _ {1} } - {\mathsf P} _ {\theta _ {2} } \| \leq \ r ( \theta _ {1} , \theta _ {2} ). $$

References

[1] H.H. Kuo, "Gaussian measures on Banach spaces" , Springer (1975)
[2] H. Cramér, "Mathematical methods of statistics" , Princeton Univ. Press (1946)
[3] I.A. Ibragimov, R.Z. [R.Z. Khas'minskii] Has'minskii, "Statistical estimation: asymptotic theory" , Springer (1981) (Translated from Russian)
[4] V.M. Zolotarev, "Properties and relations of certain types of metrics" Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Akad. Nauk. USSR , 87 (1979) pp. 18–35; 206–212 (In Russian) (English summary)
How to Cite This Entry:
Hellinger distance. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Hellinger_distance&oldid=16453
This article was adapted from an original article by M.S. Nikulin (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article