View source for Multi-dimensional statistical analysis

''multivariate statistical analysis''

The branch of [[Mathematical statistics|mathematical statistics]] devoted to mathematical methods for constructing optimal designs for the collection, systematization and processing of multivariate statistical data, directed towards clarifying the nature and the structure of the correlations between the components of the multivariate attribute in question, and intended for obtaining scientific and practical inferences. By a multivariate attribute is meant a <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651401.png" />-dimensional vector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651402.png" /> of components (laws, variables) <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651403.png" /> which may be quantitative, that is, measuring in some fixed scale the degree of manifestation of the studied property of an object, it may be ordering (or ordinal), that is, allowing the objects being analyzed to be ordered relative to the degree of manifestation in them of the studied property, and it may be classifying (or nominal), that is, allow the collection of objects being investigated, which does not lend itself to ordering, to be separated into homogeneous (relative to the analyzed property) classes. The results of measuring these components,

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651404.png" /></td> <td valign="top" style="width:5%;text-align:right;">(1)</td></tr></table>

for each of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651405.png" /> objects of a collection, forms a sequence of multivariate observations, or an initial ensemble of multivariate data, for conducting a multivariate statistical analysis. A significant part of multivariate statistical analysis involves the situation in which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651406.png" /> is interpreted as a multivariate random variable, and the corresponding sequence of observations (1) is a population sample. In this case the choice of a method for processing the initial statistical data and the analysis of their properties is carried out on the basis of assumptions regarding the nature of the multivariate (joint) law of the probability distribution <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651407.png" />.

The content of multivariate statistical analysis can be conventionally divided into three basic subdivisions: the multivariate statistical analysis of multivariate distributions and their basic characteristics; the multivariate statistical analysis of the nature and structure of the correlations between the components of the multivariate attribute being investigated; and the multivariate statistical analysis of the geometric structure of the set of multi-dimensional observations being investigated.

==Multivariate statistical analysis of multivariate distributions and their fundamental characteristics.==
This branch covers only situations in which the observations (1) being processed have a probabilistic nature, that is, can be interpreted as a sample from a corresponding population. The basic problems of this branch are: the statistical estimation, for the multivariate distributions in question, of their fundamental numerical characteristics and parameters; the investigation of the properties of the statistical estimators used; and the investigation of the probability distributions of a number of statistics that are used to construct statistical tests for the verification of various hypotheses on the nature of the multi-dimensional data being analyzed. The fundamental results are related to the particular case when the attribute in question, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651408.png" />, is subject to a multivariate normal law <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m0651409.png" />, with density function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514010.png" /> given by

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514011.png" /></td> <td valign="top" style="width:5%;text-align:right;">(2)</td></tr></table>

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514012.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514013.png" /> is the vector of mathematical expectations (cf. [[Mathematical expectation|Mathematical expectation]]) of the components of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514014.png" />, that is, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514015.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514016.png" />, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514017.png" /> is the [[Covariance matrix|covariance matrix]] of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514018.png" />, that is, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514019.png" /> is the covariance of these components of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514020.png" /> (the non-degenerate case <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514021.png" /> is considered; in case <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514022.png" />, all the results remain true, but in a subspace of a smaller dimension <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514023.png" /> on which the probability distribution of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514024.png" /> is concentrated).

Thus, if (1) is a sequence of independent observations, forming a random sample from <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514025.png" />, then the maximum-likelihood estimators for the parameters <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514026.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514027.png" /> in (2) are, respectively, the statistics (see [[#References|[1]]], [[#References|[2]]])

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514028.png" /></td> <td valign="top" style="width:5%;text-align:right;">(3)</td></tr></table>

and

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514029.png" /></td> <td valign="top" style="width:5%;text-align:right;">(4)</td></tr></table>

where the random vector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514030.png" /> is subject to the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514031.png" />-dimensional normal law <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514032.png" /> and is statistically independent of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514033.png" />, and the joint distribution of the elements of the matrix <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514034.png" /> is described by the so-called [[Wishart distribution|Wishart distribution]] (see [[#References|[4]]]) with density

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514035.png" /></td> </tr></table>

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514036.png" /></td> </tr></table>

if <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514037.png" /> is positive definite, and 0 otherwise.

Within this scheme, the distribution and moments of sampling characteristics of multivariate random variables such as the coefficients of paired, partial and multiple correlations, the generalized variance (i.e., the statistic <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514038.png" />) and the generalized Hotelling <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514039.png" />-statistic (cf. [[Hotelling-T^2-distribution|Hotelling <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514040.png" />-distribution]] and [[#References|[5]]]) have been investigated. In particular (see [[#References|[1]]]), if the sample covariance matrix <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514041.png" /> is defined as the estimator <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514042.png" /> made  "unbiased" , namely:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514043.png" /></td> <td valign="top" style="width:5%;text-align:right;">(5)</td></tr></table>

then the distribution of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514044.png" /> tends to <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514045.png" /> as <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514046.png" />, and the random variables

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514047.png" /></td> <td valign="top" style="width:5%;text-align:right;">(6)</td></tr></table>

and

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514048.png" /></td> <td valign="top" style="width:5%;text-align:right;">(7)</td></tr></table>

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514049.png" /></td> </tr></table>

have the [[Fisher-F-distribution|Fisher <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514050.png" />-distribution]] with degrees of freedom <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514051.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514052.png" />, respectively. In (7), <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514053.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514054.png" /> are the sizes of two independent samples of the form (1) taken from the same population <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514055.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514056.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514057.png" /> being estimators of the form (3) and (4)–(5), constructed with respect to the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514058.png" />-th sample, and

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514059.png" /></td> </tr></table>

is the common sample covariance matrix constructed with respect to the estimators <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514060.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514061.png" />.

==Multivariate statistical analysis of the nature and structure of correlations between the components of the multivariate attribute in question.==
This branch unifies the ideas and results used in such methods and models of multivariate statistical analysis as multiple [[Regression|regression]]; multivariate [[Dispersion analysis|dispersion analysis]] and [[Covariance analysis|covariance analysis]]; [[Factor analysis|factor analysis]]; the method of principal components; and the analysis of canonical correlations. The results of this branch may be conventionally divided into two basic types.

1) The construction of best (in a specified sense) statistical estimators for the parameters of these models and the analysis of their properties (more precisely, and in a probabilistic formulation, of their distribution laws, confidence regions, etc.). Thus, let the multivariate attribute <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514062.png" /> be interpreted as a vector-valued random variable subject to the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514063.png" />-dimensional normal distribution <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514064.png" />, and let it be partitioned into two subvectors <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514065.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514066.png" /> of dimensions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514067.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514068.png" />, respectively. This defines a corresponding partition of the expectation vector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514069.png" /> and of the theoretical and sample covariance matrices <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514070.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514071.png" />, namely:

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514072.png" /></td> </tr></table>

Then (see [[#References|[1]]], [[#References|[2]]]) the conditional distribution of the subvector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514073.png" /> (under the condition that the second subvector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514074.png" /> takes a fixed value) will also be normal <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514075.png" />. Here the maximum-likelihood estimators <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514076.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514077.png" /> of the matrices of regression coefficients <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514078.png" /> and covariances <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514079.png" />, in this classical multivariate model of multiple regression

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514080.png" /></td> <td valign="top" style="width:5%;text-align:right;">(8)</td></tr></table>

will be the mutually independent statistics

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514081.png" /></td> </tr></table>

respectively. Here the distribution of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514082.png" /> is the normal law <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514083.png" />, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514084.png" /> has the Wishart distribution with parameters <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514085.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514086.png" /> (the elements of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514087.png" /> are given in terms of the elements of <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514088.png" />).

The basic results on the construction of estimators of parameters and in the investigation of their properties in models of factor analysis, of principal components and of canonical correlations are related to the analysis of the probabilistic-statistical properties of the eigen values (characteristic values) and eigen vectors of the various covariance matrices.

In schemes not falling within the limits of the classical normal model or even within the limits of any probabilistic model, the basic results are concerned with the construction of algorithms (and the investigation of their properties) for calculating estimators of parameters which are best from the point of view of some exogeneously given functional of the quality (or adequacy) of the model.

2) The construction of statistical tests for the verification of various hypotheses on the structure of the correlations being investigated. Within the limits of a multivariate normal model (sequences of observations of the form (1) are interpreted as random samples from the corresponding multivariate normal population) statistical tests have been constructed for testing, for example, the following hypotheses.

I) The hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514089.png" />, i.e., that the expectation of the variables studied be equal to a specific vector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514090.png" />; this is tested via the Hotelling <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514091.png" />-statistic by substituting <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514092.png" /> in (6).

II) The hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514093.png" /> of equality of the expectation vectors in two populations (with identical but unknown covariance matrices), based on two samples; this is tested via the statistic <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514094.png" /> (see [[#References|[7]]]).

III) The hypothesis <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514095.png" /> of equality of the expectation vectors in several populations (with identical but unknown covariance matrices), based on samples from them; this is tested via the statistic

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514096.png" /></td> </tr></table>

in which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514097.png" /> is the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514098.png" />-th <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m06514099.png" />-dimensional observation in a sample of size <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140100.png" />, representing the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140101.png" />-th population, and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140102.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140103.png" /> are estimators of the form (3), constructed separately with respect to each of the samples and with respect to the joint sample of size <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140104.png" />, respectively.

IV) The hypotheses <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140105.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140106.png" /> of equivalence of several normal populations, based on samples from them <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140107.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140108.png" />; this is tested via the statistic

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140109.png" /></td> </tr></table>

in which the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140110.png" /> are estimators of the form (4) constructed separately with respect to the observations from the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140111.png" />-th sample, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140112.png" />.

V) The hypothesis of mutual independence of the subvectors <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140113.png" /> of dimensions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140114.png" />, respectively, into which the initial <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140115.png" />-dimensional vector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140116.png" /> has been partitioned, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140117.png" />; this is tested via the statistic

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140118.png" /></td> </tr></table>

in which <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140119.png" /> and <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140120.png" /> are sample covariance matrices of the form (4) for the vector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140121.png" /> and its subvectors <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140122.png" />, respectively.

==Multivariate statistical analysis of the geometric structure of the set of multi-dimensional observations being investigated.==
This branch unifies notions and results of models and schemes such as [[Discriminant analysis|discriminant analysis]], mixtures of probability distributions, cluster analysis, taxonomy, and multi-dimensional scaling. The key in all of these schemes is a notion of distance (measure of proximity, measure of similarity) between the elements being analyzed. Here the objects being analyzed may both be real objects, in each of which the values of the components <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140123.png" /> are fixed — then in the geometrical representation the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140124.png" />-th object will be a point <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140125.png" /> in the corresponding <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140126.png" />-dimensional space, as well as the variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140127.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140128.png" />, themselves — in the geometrical representation the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140129.png" />-th index will be a point <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140130.png" /> in the corresponding <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140131.png" />-dimensional space.

The methods and results of discriminant analysis (see [[#References|[1]]], [[#References|[2]]], [[#References|[7]]]) are directed to the solution of the following problem. Suppose that the existence of a specific number <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140132.png" /> of populations is known and that there is a sample from each (a  "training sample" ) known. It is required to construct, on the basis of training samples, the best, in a specified sense, classifying rule which allows one to attribute some new element (an observation <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140133.png" />) to its population, when the investigator does not know in advance to which population the element belongs. Usually, a classification rule means a sequence of actions; the calculation of a scalar function of the variables in question, based on which a decision is taken on assigning the element to one of the classes (the construction of a discriminant function); an ordering of the variables themselves according to their degree of informativeness from the point of view of a proper assignment of elements to classes; and a calculation of the corresponding probabilities of the errors in the classification.

The problem of analysis of a mixture of probability distributions (see [[#References|[7]]]) most often (but not always) also arises in connection with the investigation of the  "geometric structure"  of some population. Here the idea of the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140134.png" />-th homogeneous class is formalized with the help of a population described by some (as a rule, unimodal) distribution law <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140135.png" />, so that the distribution of the general population from which the sample (1) is extracted is described by a mixture of distributions of the form

<table class="eq" style="width:100%;"> <tr><td valign="top" style="width:94%;text-align:center;"><img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140136.png" /></td> </tr></table>

where <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140137.png" /> is the a priori probability (the specific weight of the elements) of the <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140138.png" />-th class in the general population. The problem is to give a  "good"  statistical estimation (with respect to a sample <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140139.png" />) of the unknown parameters <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140140.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140141.png" />, and sometimes even <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140142.png" />. This, in particular, allows one to reduce the problem of the classification of the elements to a scheme of discriminant analysis, although in this case training samples are absent.

The methods and results of cluster analysis (classification, taxonomy, pattern recognition  "without a teacher" , see [[#References|[2]]], [[#References|[6]]], [[#References|[7]]]) are directed to the solution of the following problem. The geometric structure of the set of elements to be analyzed is given either by the coordinates of the corresponding points (that is, by the matrix <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140143.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140144.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140145.png" />), or by geometric characteristics of their mutual disposition, for example, by the matrix of pairwise distances <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140146.png" />. It is required to partition the set of elements being investigated into a comparatively small (known in advance or not) number of classes, so that the elements of a class are at a small distance from each other, and at the same time different classes should, as far as possible, be sufficiently far from each other and could not be partitioned into other subsets equally far from each other.

The problem of multi-dimensional scaling (see [[#References|[6]]]) is related to the situation when the set of elements being investigated is given via a matrix of mutual distances <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140147.png" /> and consists of attributing to each of the elements a given number (<img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140148.png" />) of coordinates so that the structure of the mutual distances between the elements, measured using these auxiliary coordinates, would on the average differ least from that given. It should be noted that the basic results and methods of cluster analysis and multi-dimensional scaling have usually been developed without any assumptions regarding the probabilistic nature of the initial data.

==The merits of multivariate statistical analysis in practice.==
These consist mainly in processing the following three problems.

===The problem of statistical investigation of dependence between the variables being analyzed.===
Suppose that the set of recorded statistical variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140149.png" /> partitions, according to the meaning of these variables and the final aim of investigation, into a <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140150.png" />-dimensional subvector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140151.png" /> of (dependent) variables to be predicted and a <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140152.png" />-dimensional subvector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140153.png" /> of predicting (independent) variables. Then it can be said that the problem is to determine, on the basis of a sample (1), a <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140154.png" />-dimensional vector-valued function <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140155.png" /> from the class of acceptable decisions <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140156.png" />, which would give the best, in a specific sense, approximation of the behaviour of the subvector <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140157.png" />. Depending on the concrete form of the functional of the quality of the approximation and the nature of the variables being analyzed, one arrives at some scheme of multiple regression, variance, covariance, or confluence analysis [[#References|[8]]].

===The problem of classifying elements.===
This problem in a general (non-rigorous) formulation is that the whole set of elements (objects or variables) being analyzed, represented statistically as a matrix <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140158.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140159.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140160.png" />, or a matrix <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140161.png" />, <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140162.png" />, partitions into a comparatively small number of homogeneous (in a specified sense) groups [[#References|[7]]]. Depending on the behaviour of the a priori information and the concrete form of the functional giving the criteria for the quality of the classification, one arrives at some scheme of discriminant analysis, cluster analysis (taxonomy, pattern recognition  "without a teacher" ), or splitting mixtures of distributions.

===The problem of lowering the dimension of the factor space being investigated and the selection of the most informative variables.===
This consists of defining a set of a comparatively small number <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140163.png" /> of variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140164.png" /> in the class of admissible transformations <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140165.png" /> of the initial variables <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140166.png" /> for which some exogeneously given measure of informativity for an <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140167.png" />-dimensional system of tests attains its least upper bound (see [[#References|[7]]]). Concretization of the functional giving the measure of self-informativity (that is, aimed at a maximal preservation of the information contained in the statistical ensemble (1) relative to the initial attributes themselves) results, in particular, in various schemes of factor analysis and principal components, and in the method of extremal grouping of tests. Functionals giving a measure of external informativity, that is, aimed at extracting from (1) maximum information relative to certain other variables or phenomena not directly contained in <img align="absmiddle" border="0" src="https://www.encyclopediaofmath.org/legacyimages/m/m065/m065140/m065140168.png" />, lead to different methods of selecting the most informative variables in schemes of statistical research into dependences and discriminant analysis.

==Fundamental mathematical tools in multivariate statistical analysis.==
These consist of special methods of the theory of systems of linear equations and matrices (the method of solution of simple and generalized problems on eigen values and vectors; simple inversion and pseudo-inversion of matrices; a procedure for the diagonalization of matrices; etc.) and certain optimization algorithms (methods of coordinate-wise descent, conjugate gradients, branch-and-bound, various versions of random scanning and stochastic approximation, etc.).

====References====
<table><TR><TD valign="top">[1]</TD> <TD valign="top">  T.W. Anderson,   "An introduction to multivariate statistical analysis" , Wiley  (1958)</TD></TR><TR><TD valign="top">[2]</TD> <TD valign="top">  M.G. Kendall,   A. Stuart,   "The advanced theory of statistics" , '''3''' , Griffin  (1983)</TD></TR><TR><TD valign="top">[3]</TD> <TD valign="top">  L.N. Bol'shev,   ''Bull. Int. Stat. Inst.'' , '''43'''  (1969)  pp. 425–441</TD></TR><TR><TD valign="top">[4]</TD> <TD valign="top">  J. Wishart,   ''Biometrika'' , '''20A'''  (1928)  pp. 32–52</TD></TR><TR><TD valign="top">[5]</TD> <TD valign="top">  H. Hotelling,   "The generalization of student's ratio"  ''Ann. Math. Statist.'' , '''2'''  (1931)  pp. 360–378</TD></TR><TR><TD valign="top">[6]</TD> <TD valign="top">  J.B. Kruskal,   "Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis"  ''Psychometrika'' , '''29'''  (1964)  pp. 1–27</TD></TR><TR><TD valign="top">[7]</TD> <TD valign="top">  S.A. Aivazyan,   V.M. Bukhshtaber,   I.S. Yenyukov,   L.D. Meshalkin,   "Applied statistics: classification and reduction of dimensionality" , Moscow  (1989)  (In Russian)</TD></TR><TR><TD valign="top">[8]</TD> <TD valign="top">  S.A. Aivazyan,   I.S. Yenyukov,   L.D. Meshalkin,   "Applied statistics: study of relationships" , Moscow  (1985)  (In Russian)</TD></TR></table>



====Comments====


====References====
<table><TR><TD valign="top">[a1]</TD> <TD valign="top">  R. Gnanadesikan,   "Methods for statistical data analysis of multivariate observations" , Wiley  (1977)</TD></TR><TR><TD valign="top">[a2]</TD> <TD valign="top">  M.J. Schervish,   ''Stat. Science'' , '''2'''  (1987)  pp. 396–433</TD></TR><TR><TD valign="top">[a3]</TD> <TD valign="top">  R. Farrell,   "Techniques of multivariate calculation" , Springer  (1976)</TD></TR><TR><TD valign="top">[a4]</TD> <TD valign="top">  M.L. Eaton,   "Multivariate statistics: A vector space approach" , Wiley  (1983)</TD></TR><TR><TD valign="top">[a5]</TD> <TD valign="top">  R.J. Muirhead,   "Aspects of multivariate statistical theory" , Wiley  (1982)</TD></TR></table>