Multi-dimensional distribution

multivariate distribution

A probability distribution on the $\sigma$- algebra of Borel sets of an $s$- dimensional Euclidean space $\mathbf R ^ {s}$. One usually speaks of a multivariate distribution as the distribution of a multi-dimensional random variable, or random vector, $X = ( X _ {1} \dots X _ {s} )$, meaning by this the joint distribution of the real random variables $X _ {1} ( \omega ) \dots X _ {s} ( \omega )$ given on the same space of elementary events $\Omega$( $X _ {1} \dots X _ {s}$ may be regarded as coordinate variables in the space $\Omega = \mathbf R ^ {s}$). A multivariate distribution is uniquely determined by its distribution function — the function

$$F ( x _ {1} \dots x _ {s} ) = \ {\mathsf P} \{ X _ {1} < x _ {1} \dots X _ {s} < x _ {s} \}$$

of the real variables $x _ {1} \dots x _ {s}$.

As in the one-dimensional case, the most widespread multivariate distributions are the discrete and the absolutely-continuous distributions. In the discrete case a multivariate distribution is concentrated on a finite or countable set of points $( x _ {i _ {1} } \dots x _ {i _ {s} } )$ of $\mathbf R ^ {s}$ such that

$${\mathsf P} \{ X _ {1} = x _ {i _ {1} } \dots X _ {s} = x _ {i _ {s} } \} = \ p _ {i _ {1} \dots i _ {s} } \geq 0 ,$$

$$\sum _ {i _ {1} \dots i _ {s} } p _ {i _ {1} \dots i _ {s} } = 1$$

(see, for example, Multinomial distribution). In the absolutely-continuous case almost-everywhere (with respect to Lebesgue measure) on $\mathbf R ^ {s}$,

$$\frac{\partial ^ {s} F ( x _ {1} \dots x _ {s} ) }{\partial x _ {1} \dots \partial x _ {s} } = p ( x _ {1} \dots x _ {s} ) ,$$

where $p ( x _ {1} \dots x _ {s} ) \geq 0$ is the density of the multivariate distribution:

$${\mathsf P} \{ X \in A \} = \ \int\limits _ { A } p ( x _ {1} \dots x _ {s} ) \ d x _ {1} \dots d x _ {s} ,$$

for any $A$ from the $\sigma$- algebra of Borel subsets of $\mathbf R ^ {s}$, and

$$\int\limits _ {\mathbf R ^ {s} } p ( x _ {1} \dots x _ {s} ) \ d x _ {1} \dots d x _ {s} = 1 .$$

The distribution of any random variable $X _ {i}$( and also, for any $m < s$, the distribution of the variables $X _ {i _ {1} } \dots X _ {i _ {m} }$) relative to a multivariate distribution is called a marginal distribution. The marginal distributions are completely determined by the given multivariate distribution. When $X _ {1} \dots X _ {s}$ are independent, then

$$F ( x _ {1} \dots x _ {s} ) = \ F _ {1} ( x _ {1} ) \dots F _ {s} ( x _ {s} )$$

and

$$p ( x _ {1} \dots x _ {s} ) = \ p _ {1} ( x _ {1} ) \dots p _ {s} ( x _ {s} ) ,$$

where $F _ {i} ( x)$ and $p _ {i} ( x)$ are, respectively, the marginal distribution functions and densities of the $X _ {i}$.

The mathematical expectation of any function $f ( X _ {1} \dots X _ {s} )$ of $X _ {1} \dots X _ {s}$ is defined by the integral of this function with respect to the multivariate distribution; in particular, in the absolutely-continuous case it is defined by the integral

$${\mathsf E} f ( X _ {1} \dots X _ {s} ) =$$

$$= \ \int\limits _ {\mathbf R ^ {s} } f ( x _ {1} \dots x _ {s} ) p ( x _ {1} \dots x _ {s} ) d x _ {1} \dots d x _ {s} .$$

The characteristic function of a multivariate distribution is the function of $t = ( t _ {1} \dots t _ {s} )$ given by

$$\phi ( t) = {\mathsf E} e ^ {i t x } ^ \prime ,$$

where $t x ^ \prime = t _ {1} x _ {1} + \dots + t _ {s} x _ {s}$. The fundamental characteristics of a multivariate distribution are the moments (cf. Moment): the mixed moments ${\mathsf E} X _ {1} ^ {k _ {1} } \dots X _ {s} ^ {k _ {s} }$ and the central mixed moments ${\mathsf E} ( X _ {1} - {\mathsf E} X _ {1} ) ^ {k _ {1} } \dots ( X _ {s} - {\mathsf E} X _ {s} ) ^ {k _ {s} }$, where $k _ {1} + \dots + k _ {s}$ is the order of the corresponding moment. The roles of the expectation and the variance for a multivariate distribution are played by ${\mathsf E} X = ( {\mathsf E} X _ {1} \dots {\mathsf E} X _ {s} )$ and the set of second-order central mixed moments, which form the covariance matrix. If ${\mathsf E} ( X _ {i} - {\mathsf E} X _ {i} ) ( X _ {j} - {\mathsf E} X _ {j} ) = 0$ for all $i , j$, $i \neq j$, then $X _ {1} \dots X _ {s}$ are called pairwise uncorrelated or orthogonal (the covariance matrix is diagonal). If the rank $r$ of the covariance matrix is less than $s$, then the multivariate distribution is called a degenerate distribution; in this case the distribution is concentrated on some linear manifold in $\mathbf R ^ {s}$ of dimension $r < n$.

For methods of investigating dependencies between $X _ {1} \dots X _ {s}$ see Correlation; Regression.

References

[a1]	N.L. Johnson, S. Kotz, "Discrete distributions" , Houghton Mifflin (1969)
[a2]	N.L. Johnson, S. Kotz, "Continuous multivariate distributions" , Wiley (1942)