VMSTA Modern Stochastics: Theory and Applications 2351-60542351-60462351-6046 VTeXMokslininkų g. 2A, 08412 Vilnius, Lithuania VMSTA225 10.15559/23-VMSTA225 Research Article Consistency of LSE for the many-dimensional symmetric textured surface parameters https://orcid.org/0000-0002-3497-9726 DykyiOleksandrdykyiolexandr@gmail.com∗ https://orcid.org/0000-0001-5250-6781 IvanovAlexanderalexntuu@gmail.com Department of Mathematical Analysis and Probability Theory, Faculty for Physics and Mathematics, National Technical University of Ukraine, “Igor Sikorsky Kyiv Polytechnic Institute”, Peremohy Avenue, 37, 03057 Kyiv, Ukraine Corresponding author. 2023 163202310326728581220222722023232023 © 2023 The Author(s). Published by VTeX2023 Open access article under the CC BY license.

A multivariate trigonometric regression model is considered. In the paper strong consistency of the least squares estimator for amplitudes and angular frequencies is obtained for such a multivariate model on the assumption that the random noise is a homogeneous or homogeneous and isotropic Gaussian, specifically, strongly dependent random field on R M , M ≥ 3.

 Multivariate trigonometric model homogeneous and isotropic strongly dependent Gaussian random field least squares estimate in the Walker–Brillinger sense strong consistency 62J02 62J99 Introduction

We consider here a trigonometric regression model on R + M = 0 , ∞ M , M ≥ 3, where the arguments of cosine and sine functions are linear forms with unknown coefficients being angular frequencies of the sum of multivariate harmonic oscillations. Moreover, the noise is a homogeneous or homogeneous and isotropic [9, 3] Gaussian random field satisfying some conditions on its covariance function.

In the case M = 2 several discrete modifications of such a regression model were studied in numerous works on signal and image processing due to their applications to texture analysis. A number of useful references to publications in this area can be found in the papers [5, 4, 6]. Besides, in [5, 4] strong consistency and asymptotic normality of the least squares estimate (LSE) for amplitudes and frequencies of sinusoidal model on R + 2 with the above described Gaussian random field are proved. In turn, in [6] the asymptotic normality of the consistent LSE for the same trigonometric model on R + M , M ≥ 3, as in the article, is obtained.

It is important to mention that in the paper [1] a multivariate harmonic oscillation observed discretely against the background of a homogeneous random field having spectral densities of all orders is considered. For this model some asymptotic results on LSE and periodogram estimate of unknown amplitudes and frequencies are formulated.

Note also that from the mathematical point of view a setting of the estimation problem for M ≥ 2 in trigonometric models is a natural generalization of the problem of detection of hidden periodicities ( M = 1 ), see, for example, [2, 7] and references therein.

The further text related to the definition of LSE to some extent repeats the text of paper [6].

Let ⟨ φ , t ⟩ = ∑ l = 1 M φ l t l , ‖ t ‖ = ⟨ t , t ⟩ for vectors φ = φ 1 , … , φ M , t = t 1 , … , t M , M ≥ 3.

Consider the regression model X ( t ) = g t , θ 0 + ε t , t ∈ R + M , where g ( t , θ 0 ) = ∑ k = 1 N A k 0 cos ⟨ φ k 0 , t ⟩ + B k 0 sin ⟨ φ k 0 , t ⟩ , φ k 0 = φ 1 k 0 , … , φ M k 0 , k = 1 , N ‾ , θ 0 = A 1 0 , B 1 0 , φ 11 0 , … , φ M 1 0 , … , A N 0 , B N 0 , φ 1 N 0 , … , φ M N 0 , A k 0 2 + B k 0 2 > 00$]]>, k = 1 , N ‾; ε = ε t , t ∈ R M is a random noise defined on a probability space Ω , F , P and satisfying the next assumption.

A. ε is a sample continuous homogeneous Gaussian random field with zero mean and covariance functions B t = E ε t ε 0 , t ∈ R M , satisfying one of the conditions:

ε is isotropic field and B t = B ˜ ‖ t ‖ = L ‖ t ‖ / ‖ t ‖ α , α ∈ 0 , M − M 2 , with nondecreasing slowly varying at infinity function L;

B · ∈ L 1 R M .

To prove the consistency of LSE of the parameters (3) it is reasonable to modify the standard definition of LSE using parametric sets that allow a good enough distinction between angular frequencies [8, 1, 5].

Consider the space R M N = R N × ⋯ × R N ︸ M times , and in each space R N for some fixed numbers 0 ≤ φ _ l < φ ‾ l < ∞ define the sets Λ l = φ l = φ l 1 , … , φ l N ∈ R N : 0 ≤ φ _ l < φ l k < φ ‾ l < ∞ , k = 1 , N ‾ , l = 1 , M ‾ , containing all the corresponding true value of frequencies in (3) for fixed l.

Introduce the functional Q T θ = T − M ∫ 0 , T M X t − g t , θ 2 d t , d t = ∏ l = 1 M d t l . According to the standard definition a random vector (6) θ T = A 1 T , B 1 T , φ 11 , T , … , φ M 1 , T , … , A N T , B N T , φ 1 N , T , … , φ M N , T is called LSE of θ 0 , if it minimizes (5) on the parametric set Θ c ⊂ R M + 2 N . In Θ c amplitudes A k , B k , k = 1 , N ‾, can take any values and the frequencies φ l , l = 1 , M ‾, take values in the closed set Λ c = ∏ l = 1 M Λ l c .

To prove the strong consistency of Θ c it is necessary to provide convergence, as T → ∞, to zero almost surely (a.s.) of the fractions (see [2, 7, 5]) sin T φ l k , T − φ l j , T T φ l k , T − φ l j , T , sin T φ l k , T − φ l j 0 T φ l k , T − φ l j 0 , k ≠ j ; sin T φ l k , T T φ l k , T , l = 1 , M ‾ , k , j = 1 , N ‾ . However, the use of the previous definition of LSE does not allow to control the behavior of fractions (7), (8), as T → ∞.

A.M. Walker [8] modified the definition of LSE of the frequencies in the classical formulation of the problem of detecting of hidden periodicities so that the terms (7), (8) tend to zero, as T → ∞. In our setting the sense of similar modification is that LSE (6) is defined as an absolute minimum point of (5) on the parametric set depending on T and distinguishing frequencies properly, as T → ∞.

Following D.R. Brillinger [1], consider monotonically nondecreasing families of open sets Λ l T ⊂ Λ l , l = 1 , M ‾, T > T 0 > 0{T_{0}}>0$]]>, such that ⋃ T > T 0 Λ l T = Λ ˜ l {T_{0}}}{\Lambda _{lT}}={\widetilde{\Lambda }_{l}}$]]>, Λ ˜ l c = Λ l c , and satisfying the following conditions.

B. For l = 1 , M ‾ and k , k ′ = 1 , N ‾:

φ l 0 = φ l 1 0 , … , φ l N 0 ∈ Λ l T , T > T 0 {T_{0}}$]]> (do not confuse with φ k 0 in (2));

lim T → ∞ inf φ l ∈ Λ l T T φ l k − φ l k ′ = ∞, k ≠ k ′ ;

lim T → ∞ inf φ l ∈ Λ l T T φ l k = ∞.

 The meaning of (2) and (3) is to cover cases of close true frequencies and close to zero true frequencies.

Any random vector (6) such that it is an absolute minimum point of (5) on the parametric set Θ T c ⊂ R M + 2 N , where amplitudes A k , B k , k = 1 , N ‾, can take any values and angular frequencies take values in the set Λ T c = ∏ l = 1 M Λ l T c , T > T 0 > 0 , {T_{0}}>0,\]]]> is called LSE in the Walker–Brillinger sense.

We study in the paper just such an estimate.

Obviously, additional prior information about frequencies in the trigonometric model (1) can clarify the description of parametric sets containing true frequencies. The corresponding example is given in [6].

 LLN for finite Fourier transform of random noise

In the section a strong LLN, uniform in frequencies, is obtained for the finite Fourier transform of the random field ε from the model (1).

Under condition A ξ T = sup φ ∈ R M T − M ∫ 0 , T M e − i φ , t ε t d t → 0 a . s . , a s T → ∞ . }}\varepsilon \left(t\right)dt\right|\to 0\hspace{1em}a.s.,\hspace{0.2778em}as\hspace{0.2778em}T\to \infty .\]]]>

Denote by η T φ the expression under the supremum sign in (9). Then η T 2 φ = T − 2 M ∫ 0 , T 2 M exp { − i φ , t − s } ε t ε s d t d s = T − 2 M ∫ 0 , T 2 M − 2 exp − i ∑ l = 2 M φ l t l − s l × ∫ 0 , T 2 e − i φ 1 t 1 − s 1 ε t ε s d t 1 d s 1 d t 2 ... d t M d s 2 ... d s M . \}\varepsilon \left(t\right)\varepsilon \left(s\right)dtds\\ {} =& {T^{-2M}}\underset{{\left[0,T\right]^{2M-2}}}{\int }\exp \left\{-i{\sum \limits_{l=2}^{M}}{\varphi _{l}}\left({t_{l}}-{s_{l}}\right)\right\}\\ {} & \times \left(\underset{{\left[0,T\right]^{2}}}{\int }{e^{-i{\varphi _{1}}\left({t_{1}}-{s_{1}}\right)}}\varepsilon \left(t\right)\varepsilon \left(s\right)d{t_{1}}d{s_{1}}\right)d{t_{2}}...d{t_{M}}d{s_{2}}...d{s_{M}}.\end{aligned}\]]]> Transform the inner integral in this expression: ∫ 0 , T 2 e − i φ 1 t 1 − s 1 ε t ε s d t 1 d s 1 = ∫ 0 T ∫ 0 T − u 1 e − i φ 1 u 1 ε v 1 + u 1 , t 2 , … , t M ε v 1 , s 2 , … , s M d v 1 d u 1 + ∫ 0 T ∫ 0 T − u 1 e i φ 1 u 1 ε v 1 , t 2 , … , t M ε v 1 + u 1 , s 2 , … , s M d v 1 d u 1 . Thus η T 2 φ = T − 2 M ∫ 0 T ∫ 0 T − u 1 e − i φ 1 u 1 ( ∫ 0 , T 2 M − 4 exp − i ∑ l = 3 M φ l t l − s l × ∫ 0 , T 2 e − i φ 2 t 2 − s 2 ε v 1 + u 1 , t 2 , … , t M ε v 1 , s 2 , … , s M d t 2 d s 2 × d t 3 ... d t M d s 3 ... d s M ) d v 1 d u 1 + T − 2 M ∫ 0 T ∫ 0 T − u 1 e i φ 1 u 1 ( ∫ 0 , T 2 M − 4 exp − i ∑ l = 3 M φ l t l − s l × ∫ 0 , T 2 e − i φ 2 t 2 − s 2 ε v 1 , t 2 , … , t M ε v 1 + u 1 , s 2 , … , s M d t 2 d s 2 × d t 3 ... d t M d s 3 ... d s M ) d v 1 d u 1 = I 1 + I 0 .

In (10) and below we will write index “1” in the summands I with indices when in the entry ε t 1 , … , t M after the changes of variables, instead of t l the sum v l + u l stands, and index “0” when, instead of t l , the variable v l appears.

In I 1 and I 0 we make similar changes of variables in double integrals over the variables t 2 , s 2 . Then we get I 1 = I 11 + I 10 , I 0 = I 01 + I 00 , η T 2 φ = I 11 + I 10 + I 01 + I 00 .

In the expressions obtained, the change of variables t 3 , s 3 will lead to the sums of integrals I 11 = I 111 + I 110 , I 10 = I 101 + I 100 , I 01 = I 011 + I 010 , I 00 = I 001 + I 000 , η T 2 φ = I 111 + I 110 + I 101 + I 100 + I 011 + I 010 + I 001 + I 000 .

Continuing to apply the same changes of variables t 4 , s 4 , ..., t M , s M , we get the expression of η T 2 φ as a sum of 2 M integrals. An explicit representation of this ordered sum requires some efforts.

Denote by b j , j = 1 , 2 M ‾, binary sets of length M which are indices of the specified sum terms, i.e. η T 2 φ = ∑ j = 1 2 M I b j .

Generally speaking the terms in the sum (11) are written in the definite order using the following inductive rule. For M = 1 , 2 , 3 these sums are written above. If we have a sum ∑ j = 1 2 M ′ I b j ′ for some M ′ , then to obtain the similar sum for M ′ + 1, we have to take any term I b j ′ of the sum for M ′ and replace it with the sum of two terms I b j ′ 1 + I b j ′ 0 , j = 1 , 2 M ′ ‾. In the final analysis it turns out that terms in the sum (11) are arranged in descending lexicographic order.

We will say that the binary set b ‾ is the opposite of the binary set b = β 1 , … , β M , if b ‾ = β ‾ 1 , … , β ‾ M . The ordered collection of 2 M binary sets b j in (11) has the property that binary sets equidistant from the beginning and end of this sum are opposite: b 2 M − j − 1 = b ‾ j , j = 1 , 2 M − 1 ‾ .

Note that the frequency φ 1 is included in the 1st term of the sum (10) with the sign “−”, and this sign corresponds to the sum v 1 + u 1 in the 1st factor depending on t, of the product of the random field ε values. In the 2nd term of (10) the frequency φ 1 enters with the sign “+”, and this sign corresponds to the variable v 1 in the 1st factor of the product of the ε values. However, the sum v 1 + u 1 corresponds to the index “1” in the 1st term (10), and the variable v 1 corresponds to the index “0” in the 2nd term of (10). Further changes of variables do not change these rules, and 2 M-fold integrals I b j in the sum (11) include the exponents exp i c j φ , u \right\}$]]>, where the vector c j = γ j 1 , … , γ j M is obtained from the binary set b j by replacing the coordinates “0” with “1” and the coordinates “1” with “ − 1”, c j φ : = γ j 1 φ 1 , … , γ j M φ M . Due to (12), c 2 M − j − 1 = − c j , j = 1 , 2 M − 1 ‾ .

It follows from (13) and the previous considerations that the integrals equidistant from the beginning and end of the sum (11) are conjugate complex numbers, and it turns into the sum η T 2 φ = ∑ j = 1 2 M − 1 I c j , I c j = 2 T − 2 M ∫ 0 , T M cos c j φ , u ∫ ∏ T u ε v + b j u ε v + b ‾ j u d v d u , j = 1 , 2 M − 1 ‾ , b j u : = β j 1 u 1 , … , β j M u M , ∏ T u = 0 , T − u 1 × ⋯ × 0 , T − u M . \underset{{\textstyle\prod _{T}}\left(u\right)}{\int }\varepsilon \left(v+{b_{j}}u\right)\varepsilon \left(v+{\overline{b}_{j}}u\right)dvdu,\\ {} & \hspace{2em}j=\overline{1,{2^{M-1}}},\hspace{0.2778em}{b_{j}}u:=\left({\beta _{{j_{1}}}}{u_{1}},\dots ,{\beta _{{j_{M}}}}{u_{M}}\right),\\ {} & \hspace{2em}{\prod _{T}}\left(u\right)=\left[0,T-{u_{1}}\right]\times \cdots \times \left[0,T-{u_{M}}\right].\end{aligned}\]]]>

From (14) and (15) we get E ξ T 2 = E sup φ ∈ R M η T 2 φ ≤ 2 ∑ j = 1 2 M − 1 T − 2 M ∫ 0 , T M E ∫ ∏ T u ε v + b j u ε v + b ‾ j u d v d u ≤ 2 ∑ j = 1 2 M − 1 T − 2 M ∫ 0 , T M Ψ j 1 / 2 u d u , where Ψ j u = ∫ ∏ T 2 u E ε v + b j u ε v + b ‾ j u ε w + b j u ε w + b ‾ j u d v d w , j = 1 , 2 M − 1 ‾ .

To deal with the integrals (17) we use Isserlis’ theorem E ε v + b j u ε v + b ‾ j u ε w + b j u ε w + b ‾ j u = B 2 b j − b ‾ j u + + B 2 v − w + B v − w + b j − b ‾ j u B v − w + b ‾ j − b j u . Then Ψ j u = T − u 1 2 T − u 2 2 … T − u M 2 B 2 b j − b ‾ j u + ∫ ∏ T 2 u B 2 v − w d v d w + ∫ ∏ T 2 u B v − w + b j − b ‾ j u B v − w + b ‾ j − b j u d v d w = Ψ j 1 u + Ψ j 2 u + Ψ j 3 u , j = 1 , 2 M − 1 ‾ .

Let us first estimate the term Ψ j 3 u using notation B v − w + b j − b ‾ j u × × B v − w + b ‾ j − b j u = R u j v − w : Ψ j 3 u = ∫ ∏ T 2 u R u j v − w d v d w = ∏ l = 1 M T − u l ∫ − T − u 1 T − u 1 ⋯ ∫ − T − u M T − u M ∏ l = 1 M 1 − t l T − u l R u j t d t ≤ ∏ l = 1 M T − u l ∫ − T , T M R u j t d t = T M ∏ l = 1 M T − u l ∫ − 1 , 1 M R u j T t d t , 2 T − 2 M ∫ 0 , T M Ψ j 3 1 / 2 u d u = 2 T − M ∫ 0 , 1 M Ψ j 3 1 / 2 T u d u ≤ 2 ∫ 0 , 1 M ∏ l = 1 M 1 − u l ∫ − 1 , 1 M R T u j T t d t d u .

Consider the integral under the root sign in (19) if condition A ( i ) is met. ∫ − 1 , 1 M R T u j T t d t = ∫ − 1 , 1 M B ˜ T ‖ t + b j − b ‾ j u ‖ B ˜ T ‖ t + b ‾ j − b j u ‖ d t = ∫ − 1 , 1 M L T ‖ t + b j − b ‾ j u ‖ T α ‖ t + b j − b ‾ j u ‖ α · L T ‖ t + b ‾ j − b j u ‖ T α ‖ t + b ‾ j − b j u ‖ α d t .

If L is monotone nondecreasing function, as it is assumed in condition A ( i ), then the numerators in (20) can always be bounded as follows. L T ‖ t ± b j − b ‾ j u ‖ ≤ L 2 M T < 1 + δ L T for any δ > 00$]]>, as T > T δ T\left(\delta \right)$]]>.

The denominators in (20) need to be estimated more carefully. Let b = ( β 1 , … , β M ) be an arbitrary binary set. Using the notation ∫ − b ‾ i b i = ∫ − β ‾ i 1 β i 1 ∫ − β ‾ i 2 β i 2 ⋯ ∫ − β ‾ i M β i M , i = 1 , 2 M ‾ , we get for any fixed j ∫ − 1 , 1 M R T u j T t d t = ∑ i = 1 2 M ∫ − b ‾ i b i R T u j T t d t . In the sum (23), for any fixed i put the M-fold integral ∫ − b ‾ i b i in correspondence with the set of pluses and minuses q i of the length M according to the following rule: the sign “+” in q i corresponds to the integral over [0,1], the sign “−” corresponds to the integral over [−1,0]. Consider another set q j consisting of the signs of the b j − b ‾ j in the denominator of the 1st fraction in (20) under the integral sign of the ith term in (23). Similarly we get the 3rd set q ‾ j consisting of the signs of the opposite set b ‾ j − b j in the denominator of the 2nd fraction of the ith term in (23).

Denote by d i j the number of coincidences of signs in the sets q i , q j and by d ‾ i j the number of coincidences in q i and q ‾ j . If d i j ≥ M − M 2 , then the 2nd fraction of the right hand side in (20) we bound by B ˜ 0 in the ith term of (23), and the integral of ‖ t + b j − b ‾ j u ‖ − α related to the 1st fraction in (20) we will estimate below. On the other hand, if d i j < M − M 2 , then d ‾ i j ≥ M − M 2 . And similarly we bound the 1st fraction in our integral by B ˜ 0 and the integral of the 2nd denominator we have to bound separately. Thus m = max d i j , d ‾ i j ≥ M − M 2 .

Coordinates t r , r = 1 , M ‾, of the vector t can be positive or negative according to which integral, ∫ 0 1 or ∫ − 1 0 , corresponds to the variable t r . All the coordinates of the vector u are positive, however coordinates of the vectors b j − b ‾ j and b ‾ j − b j take values + 1 or − 1.

Let m = d i j . The case m = d ‾ i j can be considered similarly. In the norm ‖ t + b − b ‾ u ‖ the squares t l + β j l − β ‾ j l u l 2 where the signs of t l and β j l − β ‾ j l u l differs are estimated from below by zeros. Denote by l k , k = 1 , m ‾, the numbers coordinates of the vectors t and b j − b ‾ j u with coinciding signs. Then ( t l k + β j l k − β ‾ j l k u l k ) 2 ≥ t l k 2 , k = 1 , m ‾, ‖ t + b j − b ‾ j u ‖ ≥ ∑ k = 1 m t l k 2 1 2 , and, respectively, ‖ t + b j − b ‾ j u ‖ − α ≤ ∑ k = 1 m t l k 2 − α 2 . Then for any δ > 00$]]> and T > T δ > 0T\left(\delta \right)>0$]]> taking into account (21), (22) and (24) we obtain ∫ − b ‾ i b i R T u j T t d t ≤ 1 + δ B ˜ 0 B ˜ T ∫ − β ‾ i l 1 β i l 1 ⋯ ∫ − β ‾ i l m β i l m ∑ k = 1 m t l k 2 − α 2 d t l 1 … d t l m .

Denote by t m = ∑ k = 1 m t l k 2 1 / 2 the norm in R m , by c m = − β ‾ i l 1 , β i l 1 × ⋯ × − β ‾ i l m , β i l m unit cubes in R m , and by V m m the ball of radius m centered at the origin, I α ; m = ∫ c m t − α d t.

Then for α < M − M 2 I α ; m ≤ ∫ V m m t − α d t = 2 π m 2 Γ m 2 m m − α 2 m − α , and due to (19)–(26) for any j = 1 , 2 M − 1 ‾ T − 2 M ∫ 0 , T M Ψ j 3 1 / 2 u d u = O B ˜ 1 / 2 T , as T → ∞ .

It follows from (18) that Ψ j 2 u are equal for all j. Using similar but much simpler reasoning, we arrive at the inequality T − 2 M ∫ 0 , T M Ψ j 2 1 / 2 u d u = T − 2 M ∫ 0 , T M ∫ ∏ T 2 u B 2 v − w 2 d v d w d u ≤ 2 3 M B ˜ 1 / 2 0 ∫ − 1 , 1 M B T t d t 1 / 2 = O B ˜ 1 / 2 T , as T → ∞ .

And finally T − 2 M ∫ 0 , T M Ψ j 1 1 / 2 u d u = T − 2 M ∫ 0 , T M ∏ l = 1 M T − u l B ˜ u d u ≤ T − M ∫ 0 , T M B ˜ u d u = ∫ 0 , 1 M B ˜ T u d u = O B ˜ T , as T → ∞ .

From (16)–(18), (27)–(29) we conclude that under condition A ( i ) E ξ T 2 = O B ˜ 1 / 2 T , as T → ∞ .

Let condition A ( i i ) be satisfied now. Using the same notation we get for the even function R u j · that T − 2 M ∫ 0 , T M Ψ j 3 u 1 / 2 d u ≤ T − 2 M ∫ 0 , T M ∫ ∏ T 2 u R u j v − w d v d w d u , ∫ ∏ T 2 u R u j v − w d v d w ≤ ∏ l = 1 M T − u l ∫ − T − u 1 T − u 1 ⋯ ∫ − T − u M T − u M R u j t d t ≤ B 0 B 1 ∏ l = 1 M T − u l , B 1 = ∫ R M B t d t , T − 2 M ∫ 0 , T M Ψ j 3 u 1 / 2 d u ≤ 2 3 M B 1 / 2 0 B 1 1 / 2 T − M 2 . For the same integral of Ψ j 2 u we obtain the same bound. And finally T − 2 M ∫ 0 , T M Ψ j 1 1 / 2 u d u = T − 2 M ∫ 0 , T M ∏ l = 1 M T − u l B u d u ≤ B 1 T − M . Therefore under the condition A ( i i ) E ξ T 2 = O T − M 2 , as T → ∞ .

We show now that under condition A ( i ) (30) implies assertion (9) of the theorem. Obtaining of (9) from (31) under condition A ( i i ) is similar. Consider a sequence T n = n β , n ≥ 1, where the number β > 00$]]> is such that 1 2 α β > 11$]]>. Then ∑ n = 1 ∞ E ξ T n 2 < ∞ , and ξ T n → 0 a.s., as n → ∞.

Consider also the sequence of random variables ζ n = sup T n ≤ T < T n + 1 ξ T − ξ T n ≤ sup T n ≤ T < T n + 1 sup φ ∈ R M T − M ∫ 0 , T M e − i φ , t ε t d t − T n − M ∫ 0 , T n M e − i φ , t ε t d t . }}\varepsilon \left(t\right)dt-{T_{n}^{-M}}\underset{{\left[0,{T_{n}}\right]^{M}}}{\int }{e^{-i\left<\varphi ,t\right>}}\varepsilon \left(t\right)dt\right|.\end{aligned}\]]]> We get successively T − M ∫ 0 , T M e − i φ , t ε t d t − T n − M ∫ 0 , T n M e − i φ , t ε t d t = T − M ∫ 0 , T n M ⋃ 0 , T M ∖ 0 , T n M e − i φ , t ε t d t − T n − M ∫ 0 , T n M e − i φ , t ε t d t ≤ T − M − T n − M ∫ 0 , T n M e − i φ , t ε t d t + T − M ∫ 0 , T M ∖ 0 , T n M ε t d t = J 1 + J 2 , }}\varepsilon \left(t\right)dt-{T_{n}^{-M}}\underset{{\left[0,{T_{n}}\right]^{M}}}{\int }{e^{-i\left<\varphi ,t\right>}}\varepsilon \left(t\right)dt\right|\\ {} & \hspace{1em}=\left|{T^{-M}}\underset{{\left[0,{T_{n}}\right]^{M}}\textstyle\bigcup \left({\left[0,T\right]^{M}}\setminus {\left[0,{T_{n}}\right]^{M}}\right)}{\int }{e^{-i\left<\varphi ,t\right>}}\varepsilon \left(t\right)dt-{T_{n}^{-M}}\underset{{\left[0,{T_{n}}\right]^{M}}}{\int }{e^{-i\left<\varphi ,t\right>}}\varepsilon \left(t\right)dt\right|\\ {} & \hspace{1em}\le \left|{T^{-M}}-{T_{n}^{-M}}\right|\left|\underset{{\left[0,{T_{n}}\right]^{M}}}{\int }{e^{-i\left<\varphi ,t\right>}}\varepsilon \left(t\right)dt\right|+{T^{-M}}\underset{{\left[0,T\right]^{M}}\setminus {\left[0,{T_{n}}\right]^{M}}}{\int }\left|\varepsilon \left(t\right)\right|dt\\ {} & \hspace{1em}={J_{1}}+{J_{2}},\end{aligned}\]]]> and max T n ≤ T < T n + 1 J 1 = T n + 1 T n M − 1 ξ T n → 0 a . s . , as n → ∞ . On the other hand, sup T n ≤ t < T n + 1 J 2 ≤ T n − M ∫ 0 , T n + 1 M ∖ 0 , T n M ε t d t .

To write the last integral conveniently, let us first consider the obvious case M = 2: ∫ 0 T n + 1 ∫ 0 T n + 1 = ∫ 0 T n ∫ 0 T n + ∫ T n T n + 1 ∫ 0 T n + ∫ 0 T n ∫ T n T n + 1 + ∫ T n T n + 1 ∫ T n T n + 1 . To obtain the similar representation for M = 3, consider a formal “multiplication” of integrals, which is in fact an application of the distributivity law of union and direct product set operations: ∫ 0 T n + 1 ∫ 0 T n + 1 ∫ 0 T n + 1 = ∫ 0 T n ∫ 0 T n + ∫ T n T n + 1 ∫ 0 T n + ∫ 0 T n ∫ T n T n + 1 + ∫ T n T n + 1 ∫ T n T n + 1 ∫ 0 T n + ∫ T n T n + 1 = ∫ 0 T n ∫ 0 T n ∫ 0 T n + ∫ T n T n + 1 ∫ 0 T n ∫ 0 T n + ∫ 0 T n ∫ T n T n + 1 ∫ 0 T n + ∫ T n T n + 1 ∫ T n T n + 1 ∫ 0 T n + ∫ 0 T n ∫ 0 T n ∫ T n T n + 1 + ∫ T n T n + 1 ∫ 0 T n ∫ T n T n + 1 + ∫ 0 T n ∫ T n T n + 1 ∫ T n T n + 1 + ∫ T n T n + 1 ∫ T n T n + 1 ∫ T n T n + 1 . Note that the sum (32) contains 2 3 threefold integrals whose lower bounds of integration form a complete set of symbols 0 and T n of length 3.

By induction it can be proved (the proof is obvious, it actually repeats the calculation (32)) that for any M ∈ N T n − M ∫ 0 , T n + 1 M ∖ 0 , T n M ε t d t = T n − M ∑ k = 1 M ∑ i = 1 C M k ∫ e i k n ε t 1 , … , t M d t 1 … d t M , where in noncommutative binomial formula ∫ e i k n are the M-fold integrals, which contain k ≥ 1 integrals ∫ T n T n + 1 and M − k integrals ∫ 0 T n , and e i k n denote the sets of symbols 0 and T n of length M.

We will show that each term in (33), denoted by J i k n , vanishes a.s., as n → ∞. For this purpose consider E J i k 2 n = T n − 2 M ∫ e i k n ∫ e i k n E | ε t 1 1 , … , t M 1 × ε t 1 2 , … , t M 2 | d t 1 1 … d t M 1 d t 1 2 … d t M 2 ≤ B 0 T n − 2 M T n + 1 − T n 2 k T n 2 M − 2 k = B 0 T n + 1 T n − 1 2 k = O n − 2 k , as n → ∞ . Since k ≥ 1, then ∑ n = 1 ∞ E J i k 2 n < ∞ , i = 1 , C M k ‾ , k = 1 , M ‾ , that is, sup T n ≤ T < T n + 1 J 2 → 0 a.s., as n → ∞. Theorem 1 is proved.  □

 Strong consistency of LSE

In this section, we prove a theorem on the strong consistency of LSE θ T in the trigonometric model (1)–(3).

Let conditions A and B be satisfied. Then LSE θ T is a strongly consistent estimate of parameter θ 0 in the sense that A k T → A k 0 , B k T → B k 0 , T φ l k , T − φ l k 0 → 0, a.s., as T → ∞, l = 1 , M ‾, k = 1 , N ‾.

Consider a system of linear equations for A k T , B k T , k = 1 , N ‾, which is a subsystem of the system of normal equations for finding θ T : ∂ Q T θ ∂ A p | θ = θ T = ∂ Q T θ ∂ B p | θ = θ T = 0 , p = 1 , N ‾ , and write it in the form ∑ k = 1 N a k p 1 A k T + ∑ k = 1 N b k p 1 B k T = c p 1 , p = 1 , N ‾ , ∑ k = 1 N a k p 2 A k T + ∑ k = 1 N b k p 2 B k T = c p 2 , p = 1 , N ‾ .

We introduce the following notation. cos ∑ l = 1 M φ l k , T t l = cos k t , sin ∑ l = 1 M φ l k , T t l = sin k t , cos ∑ l = 1 M φ l k 0 t l = cos k 0 t , sin ∑ l = 1 M φ l k 0 t l = sin k 0 t . Then the coefficients of system (34) can be written as a k p 1 = T − M ∫ 0 , T M cos k t cos p t d t , a k p 2 = T − M ∫ 0 , T M cos k t sin p t d t , b k p 1 = T − M ∫ 0 , T M sin k t cos p t d t , b k p 2 = T − M ∫ 0 , T M sin k t sin p t d t , c p 1 = T − M ∫ 0 , T M X t cos p t d t , c p 2 = T − M ∫ 0 , T M X t sin p t d t .

We also denote by o T 1 , T > 00$]]>, generally speaking, different stochastic processes converging to zero a.s., as T → ∞. Using condition B we find a k p 1 = o T 1 , k ≠ p , a p p 1 = 1 2 + o T 1 ; a k p 2 = o T 1 , k , p = 1 , N ‾ ; b k p 1 = a p k 2 = o T 1 ; b k p 2 = o T 1 , k ≠ p , b p p 1 = 1 2 + o T 1 , k , p = 1 , N ‾ . On the other hand, c p 1 = T − M ∫ 0 , T M ε t cos p t d t + T − M ∫ 0 , T M g t , θ 0 cos p t d t . The 1st term of this sum is o T 1 according to Theorem 1. The study of the 2nd term is much more difficult.

For fixed p, T − M ∫ 0 , T M g t , θ 0 cos p t d t = ∑ k = 1 N A k 0 T − M ∫ 0 , T M cos k 0 t cos p t d t + ∑ k = 1 N B k 0 T − M ∫ 0 , T M sin k 0 t cos p t d t = ∑ 1 + ∑ 2 . Obviously, ∑ 1 = 1 2 ∑ k = 1 N A k 0 T − M ∫ 0 , T M cos ∑ l = 1 M φ l p , T + φ l k 0 t l d t + 1 2 ∑ k = 1 N A k 0 T − M ∫ 0 , T M cos ∑ l = 1 M φ l p , T − φ l k 0 t l d t = ∑ 11 + ∑ 12 . Subject to B ∑ 11 = o T 1 , ∑ 12 = 1 2 A p 0 T − M ∫ 0 , T M cos ∑ l = 1 M φ l p , T − φ l p 0 t l d t + o T 1 .

Similarly, ∑ 2 = 1 2 ∑ k = 1 N B k 0 T − M ∫ 0 , T M sin ∑ l = 1 M φ l k 0 + φ l p , T t l d t − 1 2 ∑ k = 1 N B k 0 T − M ∫ 0 , T M sin ∑ l = 1 M φ l p , T − φ l k 0 t l d t = ∑ 21 + ∑ 22 . And again, due to condition B, ∑ 21 = o T 1 , ∑ 22 = − 1 2 B p 0 T − M ∫ 0 , T M sin ∑ l = 1 M φ l p , T − φ l p 0 t l d t + o T 1 .

Set x l p = sin T φ l p , T − φ l p 0 T φ l p , T − φ l p 0 , y l p = 1 − cos T φ l p , T − φ l p 0 T φ l p , T − φ l p 0 , l = 1 , M ‾ , p = 1 , N ‾ , and note that integrals in (41) and (43) are some homogeneous polynomials of variables x l p , y l p , l = 1 , M ‾, for which we will use the notation r ≥ 3 T − r ∫ 0 , T r cos ∑ l = 1 r φ l p , T − φ l p 0 t l d t 1 … d t r = C r x l p , y l p = C r p , T − r ∫ 0 , T r sin ∑ l = 1 r φ l p , T − φ l p 0 t l d t 1 … d t r = S r x l p , y l p = S r p . Then ∑ 12 = 1 2 A p 0 C M p + o T 1 , ∑ 22 = − 1 2 B p 0 S M p + o T 1 , c p 1 = 1 2 A p 0 C M p − 1 2 B p 0 S M p + o T 1 .

Besides, similarly to c p 1 , c p 2 = T − M ∫ 0 , T M X t sin p t d t = T − M ∫ 0 , T M g t , θ 0 sin p t d t + o T 1 = ∑ k = 1 N A k 0 T − M ∫ 0 , T M cos ∑ l = 1 M φ l k 0 t l sin ∑ l = 1 M φ l p , T t l d t + ∑ k = 1 N B k 0 T − M ∫ 0 , T M sin ∑ l = 1 M φ l k 0 t l sin ∑ l = 1 M φ l p , T t l d t = 1 2 A p 0 S M p + 1 2 B p 0 C M p + o T 1 .

From (34), (39), (40), (46) and (47) we get for p = 1 , N ‾ A p T = A p 0 C M p − B p 0 S M p + o T 1 , B p T = A p 0 S M p + B p 0 C M p + o T 1 . From (45) and (48) it follows A p T , B p T ≤ A p 0 + B p 0 + o T 1 , p = 1 , N ‾ .

Using the function Φ T θ 1 , θ 2 = T − M ∫ 0 , T M g t , θ 1 − g t , θ 2 2 d t , from the definition of θ T we derive Q T θ T − Q θ 0 = Φ T θ T , θ 0 + 2 T − M ∫ 0 , T M ε t g t , θ 0 − g t , θ T d t ≤ 0 a . s . By Theorem 1 and (49) T − M ∫ 0 , T M ε t g t , θ 0 − g t , θ T d t → 0 a . s . , as T → ∞ . Using (50) and (51) we arrive at the convergence Φ T θ T , θ 0 → 0 a . s . , as T → ∞ .

Consider the value Φ T θ T , θ 0 in more detail. Φ T θ T , θ 0 = T − M ∫ 0 , T M g 2 t , θ T d t + T − M ∫ 0 , T M g 2 t , θ 0 d t − 2 T − M ∫ 0 , T M g t , θ T g t , θ 0 d t = Φ 1 + Φ 2 + Φ 3 . Taking into account condition B and (49) we obtain Φ 1 = 1 2 ∑ p = 1 N A p T 2 + B p T 2 + o T 1 , Φ 2 = 1 2 ∑ p = 1 N A p 0 2 + B p 0 2 + o T 1 , Φ 3 = − 2 ∑ p = 1 N T − M ∫ 0 , T M A p T A p 0 cos p t cos p 0 t + A p T B p 0 cos p t sin p 0 t d t − 2 ∑ p = 1 N T − M ∫ 0 , T M B p T A p 0 sin p t cos p 0 t + B p T B p 0 sin p t sin p 0 t d t + o T 1 = ∑ p = 1 N A p T A p 0 C M p + ∑ p = 1 N A p T B p 0 S M p − ∑ p = 1 N B p T A p 0 S M p − ∑ p = 1 N B p T B p 0 C M p .

Substitute now formulas (48) into (54) and (56). Then after reduction of similar terms we get Φ T θ T , θ 0 = 1 2 ∑ p = 1 N A p 0 C M p − B p 0 S M p 2 + 1 2 ∑ p = 1 N A p 0 S M p + B p 0 C M p 2 + 1 2 ∑ p = 1 N A p 0 2 + B p 0 2 − ∑ p = 1 N A p 0 C M p − B p 0 S M p A p 0 C M p + ∑ p = 1 N A p 0 C M p − B p 0 S M p B p 0 S M p − ∑ p = 1 N A p 0 S M p + B p 0 C M p A p 0 S M p − ∑ p = 1 N A p 0 S M p + B p 0 C M p B p 0 C M p + o T 1 = 1 2 ∑ p = 1 N A p 0 2 + B p 0 2 1 − C M p 2 − S M p 2 + o T 1 .

The last sum can be written in a more convenient form.

For any M ≥ 3 and p = 1 , N ‾ C M p 2 x l p , y l p + S M p 2 x l p , y l p = ∏ l = 1 M x l p 2 + y l p 2 .

For M = 3 it is easy to calculate that C 3 p = x 1 p x 2 p x 3 p − y 1 p y 2 p x 3 p − y 1 p x 2 p y 3 p − x 1 p y 2 p y 3 p , S 3 p = y 1 p x 2 p x 3 p + x 1 p y 2 p x 3 p + x 1 p x 2 p y 3 p − y 1 p y 2 p y 3 p , and C 3 p 2 + S 3 p 2 = x 1 p 2 + y 1 p 2 x 2 p 2 + y 2 p 2 x 3 p 2 + y 3 p 2 .

Assume (58) is true. Show that the similar identity is correct for M + 1 as well. Using obvious iterative formulas C M + 1 , p = C M p x M + 1 , p − S M p y M + 1 , p , S M + 1 , p = S M p x M + 1 , p + C M p y M + 1 , p , we find C M + 1 , p 2 + S M + 1 , p 2 = C M p x M + 1 , p − S M p y M + 1 , p 2 + S M p x M + 1 , p + C M p y M + 1 , p 2 = C M p 2 + S M p 2 x M + 1 , p 2 + y M + 1 , p 2 = ∏ l = 1 M + 1 x l p 2 + y l p 2 .  □

Note that according to the formulas (44) x l p 2 + y l p 2 = sin T φ l p , T − φ l p 0 / 2 T φ l p , T − φ l p 0 / 2 2 , l = 1 , M ‾ , p = 1 , N ‾ . As follows from (57), Lemma 1, and (60) Φ T θ T , θ 0 = 1 2 ∑ p = 1 N A p 0 2 + B p 0 2 × 1 − ∏ l = 1 M sin T φ l p , T − φ l p 0 / 2 T φ l p , T − φ l p 0 / 2 2 + o T 1 . Together with (52) this means that T φ l p , T − φ l p 0 → 0 a . s . as T → ∞ , l = 1 , M ‾ , p = 1 , N ‾ .

From (44) it follows also x l p → 1 , y l p → 0 a . s . , as T → ∞ , l = 1 , M ‾ , p = 1 , N ‾ . In turn, from (63) and recurrent formulas (59) it follows C M p → 1, S M p → 0 a.s., as T → ∞, l = 1 , M ‾.

Finally, from (48) we get A p T → A p 0 , B p T → B p 0 , a . s . , as T → ∞ , p = 1 , N ‾ , and Theorem 2 is proved.  □

 References Brillinger, D.R.: Regression for randomly sampled spatial series: the trigonometric case. J. Appl. Probab. 23A, 275–289, (1986). MR0803178. https://doi.org/10.1017/s0021900200117139 Ivanov, A.V.: Consistency of the least squares estimator of the amplitudes and angular frequencies of a sum of harmonic oscillations in models with long-range dependence. Theory Probab. Math. Stat. 80, 61–69, (2010). MR2541952. https://doi.org/10.1090/S0094-9000-2010-00794-0 Ivanov, A.V., Leonenko, N.N.: Statistical Analysis of Random Fields. Mathematics and its Applications (Soviet Series). Kluwer Academic Publishers Group, Dordrecht (1989). With a preface by A. V. Skorohod Translated from the Russian by A. I. Kochubinskii. MR1009786. https://doi.org/10.1007/978-94-009-1183-3 Ivanov, A.V., Lymar, O.V.: The asymptotic normality for the least squares estimator of parameters in a two dimensional sinusoidal model of observations. Theory Probab. Math. Stat. 100, 107–131 (2020). MR3992995. https://doi.org/10.1090/tpms/1100 Ivanov, A.V., Malyar, O.V.: Consistency of the least squares estimators of parameters in the texture surface sinusoidal model. Theory Probab. Math. Stat. 97, 73–84, (2018). MR3746000. https://doi.org/10.1090/tpms/1049 Ivanov, A.V., Savych, I.M.: On the least squares estimator asymptotic normality of the multivariate symmetric textured surface parameters. Theory Probab. Math. Stat. 105, 151–169 (2021). MR4421369. https://doi.org/10.1090/tpms/1161 Ivanov, A.V., Leonenko, N.N., Ruiz-Medina, M.D., Zhurakovsky, B.M.: Estimation of harmonic component in regression with cyclically dependent errors. Statistics 49, 156–186, (2015). MR3304373. https://doi.org/10.1080/02331888.2013.864656 Walker, A.M.: On the estimation of a harmonic component in a time series with stationary dependent residuals. Adv. Appl. Probab. 5, 217–241, (1973). MR0336943. https://doi.org/10.2307/1426034 Yadrenko, M.I.: Spectral Theory of Random Fields. Translations Series in Mathematics and Engineering. Springer, (1983). MR0697386