Namespaces
Variants
Actions

Difference between revisions of "Pearson product-moment correlation coefficient"

From Encyclopedia of Mathematics
Jump to: navigation, search
(Importing text file)
 
m (texified)
 
Line 1: Line 1:
While the modern theory of [[Correlation|correlation]] and [[Regression|regression]] has its roots in the work of F. Galton, the version of the product-moment correlation coefficient in current use (2000) is due to K. Pearson [[#References|[a2]]]. Pearson's product-moment correlation coefficient <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300601.png" /> is a measure of the strength of a linear relationship between two random variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300602.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300603.png" /> (cf. also [[Random variable|Random variable]]) with means <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300604.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300605.png" /> and finite variances <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300606.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300607.png" />:
+
While the modern theory of [[Correlation|correlation]] and [[Regression|regression]] has its roots in the work of F. Galton, the version of the product-moment correlation coefficient in current use (2000) is due to K. Pearson [[#References|[a2]]]. Pearson's product-moment correlation coefficient $\rho$ is a measure of the strength of a linear relationship between two random variables $X$ and $Y$ (cf. also [[Random variable|Random variable]]) with means $\mu_x=\mathsf{E}(X)$, $\mu_y=\mathsf{E}(Y)$ and finite variances $\sigma^2_x=\text{var}(X)$, $\sigma^2_y=\text{var}(Y)$:
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300608.png" /></td> </tr></table>
+
\begin{equation}\rho=\text{corr}(X,Y)=\frac{\text{cov}(X,Y)}{\sigma_x\:\sigma_y},\end{equation}
  
where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p1300609.png" /> is the [[Covariance|covariance]] of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006010.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006011.png" />,
+
where $\text{cov}(X,Y)$ is the [[Covariance|covariance]] of $X$ and $Y$,
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006012.png" /></td> </tr></table>
+
\begin{equation}\text{cov}(X,Y)=\mathsf{E}[(X-\mu_x)(Y-\mu_y)]=\mathsf{E}(XY)-\mu_x\:\mu_y.\end{equation}
  
It readily follows that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006013.png" />, and that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006014.png" /> is equal to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006015.png" /> or <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006016.png" /> if and only if each of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006017.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006018.png" /> is almost surely a linear function of the other, i.e., <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006019.png" /> (<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006020.png" />) with probability <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006021.png" /> (furthermore, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006022.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006023.png" /> have the same sign). If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006024.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006025.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006026.png" /> are said to be uncorrelated. Independent random variables are always uncorrelated, however uncorrelated random variables need not be independent (cf. also [[Independence|Independence]]).
+
It readily follows that $-1\leq\rho\leq+1$, and that $\rho$ is equal to $-1$ or $+1$ if and only if each of $X$ and $Y$ is almost surely a linear function of the other, i.e., $Y=\alpha+\beta X(\beta\neq0)$ ($1$) with probability $1$ (furthermore, $\rho$ and $\beta$ have the same sign). If $\rho=0$, $X$ and $Y$ are said to be uncorrelated. Independent random variables are always uncorrelated, however uncorrelated random variables need not be independent (cf. also [[Independence|Independence]]).
  
The term  "product-moment"  refers to the observation that <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006027.png" />, where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006028.png" /> denotes the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006029.png" />th product moment of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006030.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006031.png" /> about their means.
+
The term  "product-moment"  refers to the observation that $\rho=\mu_{11}/\sqrt{\mu_{20}\:\mu_{02}}$, where $\mu_{ij}=\mathsf{E}[(X-\mu_x)^i(Y-\mu_y)^j]$ denotes the $(i,j)$th product moment of $X$ and $Y$ about their means.
  
The coefficient <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006032.png" /> also plays a role in linear regression (cf. also [[Regression analysis|Regression analysis]]). If the regression of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006033.png" /> on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006034.png" /> is linear, then <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006035.png" />, and if the regression of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006036.png" /> on <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006037.png" /> is linear, then <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006038.png" />. Note that the product of the two slopes is <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006039.png" />.
+
The coefficient $\rho$ also plays a role in linear regression (cf. also [[Regression analysis|Regression analysis]]). If the regression of $Y$ on $X$ is linear, then $y=\mathsf{E}(Y|X=x)=\mu_y+\rho(\sigma_y/\sigma_x)(x-\mu_x)$, and if the regression of $X$ on $Y$ is linear, then $x=\mathsf{E}(X|Y=y)=\mu_x+\rho(\sigma_x/\sigma_y)(y-\mu_y)$. Note that the product of the two slopes is $\rho^2$.
  
When <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006040.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006041.png" /> have a bivariate normal distribution (cf. also [[Normal distribution|Normal distribution]]), <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006042.png" /> is a parameter of the joint density function
+
When $X$ and $Y$ have a bivariate normal distribution (cf. also [[Normal distribution|Normal distribution]]), $\rho$ is a parameter of the joint density function
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006043.png" /></td> </tr></table>
+
\begin{equation}\phi(x,y)=\frac{1}{2\pi\:\sigma_x\:\sigma_y\sqrt{1-\rho^2}}\exp\bigg[\frac{-1}{2(1-\rho^2)}Q\bigg],\\-\infty<x,y<\infty,\end{equation}
 
 
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006044.png" /></td> </tr></table>
 
  
 
with
 
with
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006045.png" /></td> </tr></table>
+
\begin{equation}=\bigg(\frac{x-\mu_x}{\sigma_x}\bigg)-2\rho\bigg(\frac{x-\mu_x}{\sigma_x}\bigg)\bigg(\frac{y-\mu_y}{\sigma_y}\bigg)+\bigg(\frac{y-\mu_y}{\sigma_y}\bigg)^2\end{equation}
 
 
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006046.png" /></td> </tr></table>
 
  
 
Unlike the general situation, uncorrelated random variables with a bivariate normal distribution are independent.
 
Unlike the general situation, uncorrelated random variables with a bivariate normal distribution are independent.
  
For a random sample <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006047.png" /> from a bivariate population, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006048.png" /> is estimated by the sample correlation coefficient (cf. also [[Correlation coefficient|Correlation coefficient]]) <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006049.png" />, given by
+
For a random sample $\{(x_i,y_i)\}^n_{i=1}$ from a bivariate population, $\rho$ is estimated by the sample correlation coefficient (cf. also [[Correlation coefficient|Correlation coefficient]]) $r$, given by
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006050.png" /></td> </tr></table>
+
\begin{equation}r=\frac{\sum^n_{i=1}(x_i-\overline{x})(y_i-\overline{y})}{\sqrt{\sum^n_{i=1}(x_i-\overline{x})^2\sum^n_{i=1}(y_i-\overline{y})^2}}.\end{equation}
  
If <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006051.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006052.png" /> denote, respectively, the vectors <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006053.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006054.png" />, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006055.png" /> denotes the angle between <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006056.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006057.png" />, then
+
If $x$ and $y$ denote, respectively, the vectors $(x_1-\overline{x},...,x_n-\overline{x})$ and $(y_1-\overline{y},...,y_n-\overline{y})$, and $\theta$ denotes the angle between $x$ and $y$, then
  
<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006058.png" /></td> </tr></table>
+
\begin{equation}r=\frac{xy}{|x||y|}=\cos\theta\end{equation}
  
Further interpretations of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006059.png" /> can be found in [[#References|[a3]]]. For details on the use of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/p/p130/p130060/p13006060.png" /> in hypothesis testing, and for large-sample theory, see [[#References|[a1]]].
+
Further interpretations of $r$ can be found in [[#References|[a3]]]. For details on the use of $r$ in hypothesis testing, and for large-sample theory, see [[#References|[a1]]].
  
 
====References====
 
====References====
 
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  O.J. Dunn,  V.A. Clark,  "Applied statistics: analysis of variance and regression" , Wiley  (1974)</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  K. Pearson,  "Mathematical contributions to the theory of evolution. III. Regression, heredity and panmixia"  ''Philos. Trans. Royal Soc. London Ser. A'' , '''187'''  (1896)  pp. 253–318</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  J.L. Rodgers,  W.A. Nicewander,  "Thirteen ways to look at the correlation coefficient"  ''The Amer. Statistician'' , '''42'''  (1988)  pp. 59–65</TD></TR></table>
 
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  O.J. Dunn,  V.A. Clark,  "Applied statistics: analysis of variance and regression" , Wiley  (1974)</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  K. Pearson,  "Mathematical contributions to the theory of evolution. III. Regression, heredity and panmixia"  ''Philos. Trans. Royal Soc. London Ser. A'' , '''187'''  (1896)  pp. 253–318</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  J.L. Rodgers,  W.A. Nicewander,  "Thirteen ways to look at the correlation coefficient"  ''The Amer. Statistician'' , '''42'''  (1988)  pp. 59–65</TD></TR></table>

Latest revision as of 14:38, 19 February 2021

While the modern theory of correlation and regression has its roots in the work of F. Galton, the version of the product-moment correlation coefficient in current use (2000) is due to K. Pearson [a2]. Pearson's product-moment correlation coefficient $\rho$ is a measure of the strength of a linear relationship between two random variables $X$ and $Y$ (cf. also Random variable) with means $\mu_x=\mathsf{E}(X)$, $\mu_y=\mathsf{E}(Y)$ and finite variances $\sigma^2_x=\text{var}(X)$, $\sigma^2_y=\text{var}(Y)$:

\begin{equation}\rho=\text{corr}(X,Y)=\frac{\text{cov}(X,Y)}{\sigma_x\:\sigma_y},\end{equation}

where $\text{cov}(X,Y)$ is the covariance of $X$ and $Y$,

\begin{equation}\text{cov}(X,Y)=\mathsf{E}[(X-\mu_x)(Y-\mu_y)]=\mathsf{E}(XY)-\mu_x\:\mu_y.\end{equation}

It readily follows that $-1\leq\rho\leq+1$, and that $\rho$ is equal to $-1$ or $+1$ if and only if each of $X$ and $Y$ is almost surely a linear function of the other, i.e., $Y=\alpha+\beta X(\beta\neq0)$ ($1$) with probability $1$ (furthermore, $\rho$ and $\beta$ have the same sign). If $\rho=0$, $X$ and $Y$ are said to be uncorrelated. Independent random variables are always uncorrelated, however uncorrelated random variables need not be independent (cf. also Independence).

The term "product-moment" refers to the observation that $\rho=\mu_{11}/\sqrt{\mu_{20}\:\mu_{02}}$, where $\mu_{ij}=\mathsf{E}[(X-\mu_x)^i(Y-\mu_y)^j]$ denotes the $(i,j)$th product moment of $X$ and $Y$ about their means.

The coefficient $\rho$ also plays a role in linear regression (cf. also Regression analysis). If the regression of $Y$ on $X$ is linear, then $y=\mathsf{E}(Y|X=x)=\mu_y+\rho(\sigma_y/\sigma_x)(x-\mu_x)$, and if the regression of $X$ on $Y$ is linear, then $x=\mathsf{E}(X|Y=y)=\mu_x+\rho(\sigma_x/\sigma_y)(y-\mu_y)$. Note that the product of the two slopes is $\rho^2$.

When $X$ and $Y$ have a bivariate normal distribution (cf. also Normal distribution), $\rho$ is a parameter of the joint density function

\begin{equation}\phi(x,y)=\frac{1}{2\pi\:\sigma_x\:\sigma_y\sqrt{1-\rho^2}}\exp\bigg[\frac{-1}{2(1-\rho^2)}Q\bigg],\\-\infty<x,y<\infty,\end{equation}

with

\begin{equation}=\bigg(\frac{x-\mu_x}{\sigma_x}\bigg)-2\rho\bigg(\frac{x-\mu_x}{\sigma_x}\bigg)\bigg(\frac{y-\mu_y}{\sigma_y}\bigg)+\bigg(\frac{y-\mu_y}{\sigma_y}\bigg)^2\end{equation}

Unlike the general situation, uncorrelated random variables with a bivariate normal distribution are independent.

For a random sample $\{(x_i,y_i)\}^n_{i=1}$ from a bivariate population, $\rho$ is estimated by the sample correlation coefficient (cf. also Correlation coefficient) $r$, given by

\begin{equation}r=\frac{\sum^n_{i=1}(x_i-\overline{x})(y_i-\overline{y})}{\sqrt{\sum^n_{i=1}(x_i-\overline{x})^2\sum^n_{i=1}(y_i-\overline{y})^2}}.\end{equation}

If $x$ and $y$ denote, respectively, the vectors $(x_1-\overline{x},...,x_n-\overline{x})$ and $(y_1-\overline{y},...,y_n-\overline{y})$, and $\theta$ denotes the angle between $x$ and $y$, then

\begin{equation}r=\frac{xy}{|x||y|}=\cos\theta\end{equation}

Further interpretations of $r$ can be found in [a3]. For details on the use of $r$ in hypothesis testing, and for large-sample theory, see [a1].

References

[a1] O.J. Dunn, V.A. Clark, "Applied statistics: analysis of variance and regression" , Wiley (1974)
[a2] K. Pearson, "Mathematical contributions to the theory of evolution. III. Regression, heredity and panmixia" Philos. Trans. Royal Soc. London Ser. A , 187 (1896) pp. 253–318
[a3] J.L. Rodgers, W.A. Nicewander, "Thirteen ways to look at the correlation coefficient" The Amer. Statistician , 42 (1988) pp. 59–65
How to Cite This Entry:
Pearson product-moment correlation coefficient. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Pearson_product-moment_correlation_coefficient&oldid=18562
This article was adapted from an original article by R.B. Nelsen (originator), which appeared in Encyclopedia of Mathematics - ISBN 1402006098. See original article