Modern Stochastics: Theory and Applications

The concept of distance covariance was introduced by Székely, Rizzo and Bakirov [37] as a measure of dependence between two random vectors of arbitrary dimensions. Their starting point is to consider a weighted ${L}^{2}$-integral of the difference of the (joint) characteristic functions ${f_{X}},{f_{Y}}$ and ${f_{(X,Y)}}$ of the (${\mathbb{R}}^{m}$- and ${\mathbb{R}}^{n}$-valued) random variables X, Y and $(X,Y)$, The weight w is given by $w(s,t):={c_{\alpha ,m}}|s{|}^{-m-\alpha }{c_{\alpha ,n}}|t{|}^{-n-\alpha }$ for $\alpha \in (0,2)$.

We are going to embed this into a more general framework. In order to illustrate the new features of our approach we need to recall some results on distance covariance. Among several other interesting properties, [37] shows that distance covariance characterizes independence, in the sense that ${V}^{2}(X,Y;w)=0$ if, and only if, X and Y are independent. Moreover, they show that in the case $\alpha =1$ the distance covariance ${}^{N}{V}^{2}(X,Y;w)$ of the empirical distributions of two samples $({x_{1}},{x_{2}},\dots ,{x_{N}})$ and $({y_{1}},{y_{2}},\dots ,{y_{N}})$ takes a surprisingly simple form. It can be represented as where A and B are double centrings (cf. Lemma 4.2) of the Euclidean distance matrices of the samples, i.e. of ${(|{x_{k}}-{x_{l}}|)}_{k,l=1,\dots ,N}$ and ${(|{y_{k}}-{y_{l}}|)}_{k,l=1,\dots ,N}$. If $\alpha \ne 1$ then Euclidean distance has to be replaced by its power with exponent α. The connection between the weight function w in (1) and the (centred) Euclidean distance matrices in (2) is given by the Lévy–Khintchine representation of negative definite functions, i.e. where ${c_{p}}$ is a suitable constant, cf. Section 2.1, Table 1. Finally, the representation (2) of ${}^{N}{V}^{2}(X,Y;w)$ and its asymptotic properties as $N\to \infty $ are used by Székely, Rizzo and Bakirov to develop a statistical test for independence in [37].

Yet another interesting representation of distance covariance is given in the follow-up paper [34]: Let $({X_{\text{cop}}},{Y_{\text{cop}}})$ be an independent copy of $(X,Y)$ and let W and ${W^{\prime }}$ be Brownian random fields on ${\mathbb{R}}^{m}$ and ${\mathbb{R}}^{n}$, independent from each other and from $X,Y,{X_{\text{cop}}},{Y_{\text{cop}}}$. The paper [34] defines the Brownian covariance where ${X}^{W}:=W(X)-\mathbb{E}[W(X)\mid W]$ for any random variable X and random field W with matching dimensions. Surprisingly, as shown in [34], Brownian covariance coincides with distance covariance, i.e. ${\mathcal{W}}^{2}(X,Y)={\mathcal{V}}^{2}(X,Y;w)$ when $\alpha =1$ is chosen for the kernel w.

The paper [34] was accompanied by a series of discussion papers [25, 6, 22, 11, 15, 18, 26, 17, 35] where various extensions, applications and open questions were suggested. Let us highlight the three problems which we are going to address: While insights and partial results on these questions can be found in all of the mentioned discussion papers, a definitive and unifying answer was missing for a long time. In the present paper we propose a generalization of distance covariance which resolves these closely related questions. In a follow-up paper [9] we extend our results to the detection of independence of d random variables $({X}^{1},{X}^{2},\dots ,{X}^{d})$, answering a question of [15, 1].

More precisely, we introduce in Definition 3.1 the generalized distance covariance where μ and ν are symmetric Lévy measures, as a natural extension of distance covariance of Székely et al. [37]. The Lévy measures μ and ν are linked to negative definite functions Φ and Ψ by the well-known Lévy–Khintchine representation, cf. Section 2 where examples and important properties of negative definite functions are discussed. In Section 3 we show that several different representations (related to [23]) of ${V}^{2}(X,Y)$ in terms of the functions Φ and Ψ can be given. In Section 4 we turn to the finite-sample properties of generalized distance covariance and show that the representation (2) of ${}^{N}{V}^{2}(X,Y)$ remains valid, with the Euclidean distance matrices replaced by the matrices We also show asymptotic properties of ${}^{N}{V}^{2}(X,Y)$ as N tends to infinity, paralleling those of [34, 36] for Euclidean distance covariance. After some remarks on uniqueness and normalization, we show in Section 7 that the representation (3) remains also valid, when the Brownian random fields W and ${W^{\prime }}$ are replaced by centered Gaussian random fields ${G_{\varPhi }}$ and ${G_{\varPsi }}$ with covariance kernel and analogously for ${G_{\varPsi }}$.

To use generalized distance covariance (and distance multivariance) in applications all necessary functions and tests are provided in the R package multivariance [8]. Extensive examples and simulations can be found in [7], therefore we concentrate in the current paper on the theoretical foundations.

In this section we collect some tools and concepts which will be needed in the sequel.

Székely et al. [37, 34] introduced distance covariance for two random variables X and Y with values in ${\mathbb{R}}^{m}$ and ${\mathbb{R}}^{n}$ as with the weight $w(x,y)={w_{\alpha ,m}}(x){w_{\alpha ,n}}(y)$ where ${w_{\alpha ,m}}(x)=c(p,\alpha )|x{|}^{-m-\alpha }$, $m,n\in \mathbb{N}$, $\alpha \in (0,2)$. It is well known from the study of infinitely divisible distributions (see also Székely & Rizzo [33]) that ${w_{\alpha ,m}}(x)$ is the density of an m-dimensional α-stable Lévy measure, and the corresponding cndf is just $|x{|}^{\alpha }$.

We are going to extend distance covariance to products of Lévy measures.

By definition, $V(X,Y)=\| {f_{(X,Y)}}-{f_{X}}\otimes {f_{Y}}{\| _{{L}^{2}(\rho )}}$ and, in view of the discussion in Section 2.2, we have where $U:=({X_{1}},{Y_{1}}),V:=({X_{2}},{Y_{3}})$ and $({X_{1}},{Y_{1}}),({X_{2}},{Y_{2}}),({X_{3}},{Y_{3}})$ are i.i.d. copies of $(X,Y)$.3 It is clear that the product measure ρ inherits the properties “symmetry” and “full support” from its marginals μ and ν.

From the discussion following Definition 2.3 we immediately get the next lemma.

Let ${({x_{i}},{y_{i}})}_{i=1,\dots ,N}$ be a sample of $(X,Y)$ and denote by $({\widehat{X}}^{(N)},{\widehat{Y}}^{(N)})$ the random variable which has the corresponding empirical distribution, i.e. the uniform distribution on ${({x_{i}},{y_{i}})}_{i=1,\dots ,N}$; to be precise, repeated points will have the corresponding multiple weight. By definition, $({\widehat{X}}^{(N)},{\widehat{Y}}^{(N)})$ is a bounded random variable and for i.i.d. copies with $\mathcal{L}(({\widehat{X}_{i}^{(N)}},{\widehat{Y}_{i}^{(N)}}))=\mathcal{L}(({\widehat{X}}^{(N)},{\widehat{Y}}^{(N)}))$ for $i=1,\dots ,4$, we have using (34) The formulae (27)–(33) hold, in particular, for the empirical random variables $({\widehat{X}}^{(N)},{\widehat{Y}}^{(N)})$, and so we get The sum in (46) is – for functions g which are symmetric under permutations of their variables – a V-statistic.

In abuse of notation, we also write ${}^{N}{V}^{2}(({x_{1}},\dots ,{x_{N}}),({y_{1}},\dots ,{y_{N}}))$ and ${}^{N}{V}^{2}(\boldsymbol{x},\boldsymbol{y})$ with $\boldsymbol{x}=({x_{1}},\dots ,{x_{N}})$, $\boldsymbol{y}=({y_{1}},\dots ,{y_{N}})$ instead of the precise ${}^{N}{V}^{2}(({x_{1}},{y_{1}}),\dots ,({x_{N}},{y_{N}}))$.

Since all random variables are bounded, we could use any of the representations (46)–(49) to define an estimator. A computationally feasible representation is given in the following lemma.

We will now show that ${}^{N}{V}^{2}$ is a consistent estimator for ${V}^{2}(X,Y)$.

Next we study the behaviour of the estimator under the hypothesis of independence.

We still have to prove (63).

It is a natural question whether it is possible to extend distance covariance further by taking a measure ρ in the definition (22) which does not factorize. To ensure the finiteness of $V(X,Y)$ one still has to assume see also Székely & Rizzo [36, Eq. (2.4)] and (22) at the beginning of Section 3.2. Furthermore, it is no restriction to assume that ρ is symmetric, since the integrand in (22) is symmetric, hence we can always symmetrize any non-symmetric measure ρ without changing the value of the integral. Thus, the function is well-defined and symmetric in each variable. The corresponding generalized distance covariance ${V}^{2}(X,Y)$ can be expressed by (28), if the expectations on the right-hand side are finite. Note, however, that the nice and computationally feasible representations of ${V}^{2}(X,Y)$ make essential use of the factorization of ρ which means that they are no longer available in this setting.

Let X and Y be random variables with values in ${\mathbb{R}}^{m}$ and ${\mathbb{R}}^{n}$, respectively, such that for some $x\in {\mathbb{R}}^{m}$ and $y\in {\mathbb{R}}^{n}$ A direct calculation of (28), using $\varTheta (0,y)=\varTheta (x,0)=0$, gives with where $({X_{i}},{Y_{i}}),i=1,\dots ,6$, are i.i.d. copies of the random vector $(X,Y)$.

Now suppose that ${V}^{2}(X,Y)$ is homogeneous and/or rotationally invariant, i.e. for some $\alpha ,\beta \in (0,2)$, all scalars $a,b>0$ and orthogonal matrices $A\in {\mathbb{R}}^{n\times n},B\in {\mathbb{R}}^{m\times m}$ hold. The homogeneity, (86), yields and the rotational invariance, (87), shows that $\varTheta (x/|x|,y/|y|)$ is a constant. In particular, homogeneity of degree $\alpha =\beta =1$ and rotational invariance yield that $\varTheta (x,y)=\text{const}\cdot |x|\cdot |y|$. Since the Lévy–Khintchine formula furnishes a one-to-one correspondence between the cndf and its Lévy triplet, see (the comments following) Theorem 2.1, this determines ρ uniquely: it factorizes into two Cauchy Lévy measures. This means that – even in a larger class of weights – the assumptions (86) and (87) imply a unique (up to a constant) choice of weights, and we have recovered Székely-and-Rizzo’s uniqueness result from [36].

For completeness, let us mention that the constants $c(\alpha ,m)$ and $c(\beta ,n)$ are of the form see e.g. [10, p. 34, Example 2.4.d)] or [3, p. 184, Exercise 18.23].

We continue our discussion in the setting of Section 3.2. Let $\rho =\mu \otimes \nu $ and assume that μ and ν are symmetric Lévy measures on ${\mathbb{R}}^{m}\setminus \{0\}$ and ${\mathbb{R}}^{n}\setminus \{0\}$, each with full support. For m- and n-dimensional random variables X and Y the generalized distance covariance is, cf. Definition 3.1, We set and define generalized distance correlation as Using the Cauchy–Schwarz inequality it follows from (38) that, whenever $R(X,Y)$ is well defined, one has

The sample distance correlation is given by where we use the notation introduced in Lemma 4.2.

Let us finally show that the results on Brownian covariance of Székely & Rizzo [34, Sec. 3] do have an analogue for the generalized distance covariance.

For a symmetric Lévy measure with corresponding continuous negative definite function $\varPhi :{\mathbb{R}}^{m}\to \mathbb{R}$ let ${G_{\varPhi }}$ be the Gaussian field indexed by ${\mathbb{R}}^{m}$ with Analogously we define the random field ${G_{\varPsi }}$.

For a random variable Z with values in ${\mathbb{R}}^{d}$ and for a Gaussian random field G indexed by ${\mathbb{R}}^{d}$ we set

We can now identify Gaussian covariance and generalized distance covariance.

We have shown that the concept of distance covariance introduced by Székely et al. [37] can be embedded into a more general framework based on Lévy measures, cf. Section 3. In this generalized setting the key results for statistical applications are: the convergence of the estimators and the fact that also the limit distribution of the (scaled) estimators is known, cf. Section 4. Moreover – for applications this is of major importance – the estimators have the numerically efficient representation (50).

The results allow the use of generalized distance covariance in the tests for independence developed for distance covariance, e.g. tests based on a general Gaussian quadratic form estimate or resampling tests. The test statistic is the function $T:=\frac{N\cdot {}^{N}{V}^{2}}{{a_{N}}{b_{N}}}$ discussed in Corollary 4.8. Using the quadratic form estimate (see [37] for details) its p-value can be estimated by $1-F(T)$ where F is the distribution function of the Chi-squared distribution with 1 degree of freedom. This test and resampling tests are studied in detail in [9] and [7], respectively. In addition, these papers contain illustrating examples which show that the new flexibility provided by the choice of the Lévy measures (equivalently: by the choice of the continuous negative definite function) can be used to improve the power of these tests. Moreover, new test procedures using distance covariance and its generalizations are developed in [5].

Finally, the presented results are also the foundation for a new approach to testing and measuring multivariate dependence, i.e. the mutual (in)dependence of an arbitrary number of random vectors. This approach is developed in [9] accompanied by extensive examples and further applications in [7]. All functions required for the use of generalized distance covariance in applications are implemented in the R package multivariance [8].