text:
# Nonlinear effects in E⊗(𝑏₁+𝑏₂) Jahn-Teller model: Variational approach with excited phonon states and mode correlations.

## I Introduction

Recently, revival of interest in two-level electron-phonon systems occurs owing to the experimental evidence that Jahn-Teller structural phase transition occurs in spatially anisotropic complex structures (fullerides, manganite perovskites, etc) ,,,,.

JT model is a protopype model for phonon removing the degeneracy of electron levels ,. Current investigations were focused mainly on E$``$e JT model with electron coupling to two degenerate intramolecular phonon modes, an antisymmetric and symmetric one with respect to the reflection.

The reflection symmetry of two-level electron-phonon models like the exciton and the dimer model with onsite electron coupling to one phonon mode implies nonlinear peculiarities of quantum nature . For these models, Shore et al.  introduced variational wave function in a form of linear combination of the harmonic oscillator wave functions related to two levels of different parity with respect to the reflection. This picture can be understood in terms of two or more asymmetric local minima of the effective polaron potential (i.e. the potential energy expression for the trial wavefunction in the space of variational parameters). Here, respective ground state wave function can be approximated by a linear combination of two oscillators with parameters corresponding to these local minima and coupled by means of further variational parameters. This approach was shown to yield the lowest ground state energy for the two-level models ,. The peculiarities due to reflection phenomena occurred in the rotation-symmetric E$``$e JT model  as well. Strongly localized non-displaced phonon (exotic) states appeared in the numerical spectra. They were considered useful for interpretation of the ”fast” component of luminiscence spectra. However, in order to respect the rotational symmetry of the $`Ee`$ model the proper variational approach should be formulated in radial coordinates .

In crystals exhibiting high spatial anisotropy with tensor properties of bulk characteristics (e.g. perovskites, fullerids, etc.) the rotation symmetry of Jahn-Teller molecules is generally broken. Therefore, it is reasonable to investigate JT model assuming different coupling strengths $`\alpha `$ and $`\beta `$ for the onsite intralevel and interlevel electron-phonon couplings, respectively (E$`(b_1+b_2)`$ model ). Such a model can be also considered as a generalization of the exciton-phonon or the dimer-phonon model assuming the electron tunneling to be phonon-assisted.

In order to understand the physical nature of the nonlinear effects we propose variational ansatz inspired by the shape of the numerical ground state wave function (Fig.1): the “principal” part of Gaussian character for both oscillators (in the absolute minimum of the nonlinear effective potential (Fig.1 of our previous paper  and Fig.2 of this paper) and a minor “reflective” part which corresponds to the another (local) minimum of the potential. For this minor part we consider the admixture of the first excited harmonic oscillator of the symmetric mode (rather than the only ground oscillator state, as it was commonly considered elsewhere). This admixture in the variation trial function leads to essential improvement of the results, as it will be shown in Sect. III.

Formulation of the variational ansatz and calculation of the ground state energy is presented in the Section II. In the Section III, analysis of the interplay of quantum effects and nonlinearity and related discussions as well as the reliability of different variational alternatives was investigated by comparison with results of exact numerical simulations.

## II Variational wave function of the generalized JT model

We investigate local spinless double degenerate electron states linearly coupled to two intramolecular phonon modes described by Hamiltonian

$$H=\mathrm{\Omega }(b_1^{}b_1+b_2^{}b_2+1)I+\alpha (b_1^{}+b_1)\sigma _z\beta (b_2^{}+b_2)\sigma _x,$$
(1)
where $`\sigma _x=\left(\begin{array}{c}0,1\\ 1,0\end{array}\right)`$, $`\sigma _z=\left(\begin{array}{c}1,0\\ 0,1\end{array}\right)`$ are Pauli matrices, $`I`$ is the unit matrix. This pseudospin notation refers to two-level electronic system, the Hamiltonian is thus $`2\times 2`$ matrix.

The antisymmetric phonon mode $`b_1`$ splits the degenerate unperturbed electron level ($`j=1,2`$) while the symmetric mode $`b_2`$ mediates the electron transitions between the levels. This latter term represents phonon-assisted tunneling, a mechanism of the nonclassical (nonadiabatic) nature as well as is the pure tunneling in related exciton and dimer models. Evidently, for $`\beta =0`$, the one-level Holstein model (1) is harmonic, while the coupling to the higher level ($`\beta 0`$) is the origin of the strong nonlinearity in the phonon space as will be seen below.

For $`\beta =\alpha `$, the interaction part of (1) yields the rotationally symmetric $`Ee`$ form  with a pair (an antisymmetric and a symmetric under reflection) of double degenerated vibrations.

The general case $`\alpha \beta `$ exploited here breaks the common invariance of $`Ee`$ Jahn-Teller model under the exchange of $`1`$ and $`2`$ phonons; in other words, the rotational symmetry of the Hamiltonian (1) is broken while the reflection symmetry is kept. This symmetry is inherent property of the Jahn-Teller model and it is crucial for the diagonalization of (1) and construction of the variational ansatz as it is evident from what follows.

Hamiltonian (1) can be diagonalized in the electronic subspace using the Fulton-Gouterman unitary operator

$$U=\frac{1}{\sqrt{2}}\left(\begin{array}{c}1,G_1\\ 1,G_1\end{array}\right),G_1=\mathrm{exp}(i\pi b_1^{}b_1),$$
(2)
as follows

$`H_{FG}=UHU^1=\mathrm{\Omega }\left(b_1^{}b_1+b_2^{}b_2+1\right)+\alpha (b_1^{}+b_1)I\beta (b_2^{}+b_2)\sigma _zG_1H_{ph}+H_\alpha +H_\beta .`$ (3)
The operator $`G_1`$ in (2) is the phonon reflection operator: $`G_1(b_1^{}+b_1)=(b_1^{}+b_1),G_1^2=1`$. We see that Fulton-Gouterman transformation reveals high nonlinearity in the system (term with $`G_1`$ in (3)), otherwise this nonlinearity was hidden in the initial Hamiltonian (1). Hamiltonian distincts from the exciton (dimer) by the phonon-2 assistance of the tunneling amplitude $`\beta G_1`$. The factor $`G_1`$ (2) represents continuous virtual emission and absorption of the phonons $`1`$ and mediates Rabi oscillations of the electron between the levels. These quantum oscillations are essentially the origin of the nonlinearity of the reflection symmetric model as will be seen below.

The full reflection operator is $`G=G^{el}G_1`$, where the electron reflection operator is defined by $`G^{el}|1=|2`$. Equivalently to the FG transformation one can exploit commutation of $`G`$ with Hamiltonian (1), $`[H,G]=0`$, so that the wave function of Hamiltonian (1) related to the representation of the inversion group $`p=\pm 1`$ is a linear combination of the base functions

$$|\mathrm{\Psi }^{(p)}=\frac{1}{\sqrt{2}}(1+pG)|1|\varphi ^{(p)},$$
(4)
where $`G_1|\varphi ^{(p)}=|\varphi ^{(p)}`$. Hamiltonian (3) though diagonalized is no more reflection symmetric, but the interaction part is antisymmetric against reflection. As a consequence, in the limit $`\alpha =\beta `$ the rotation symmetry is broken. Therefore, use of the FG transformation in the case of E$``$e JT is inappropriate: it breaks the symmetry which is necessary for the proper choice of the ground state in this case (see the discussion in the Conclusion).

Inserting the representation of the wave functions (5) into the Schrödinger equation related to (1) we are left with the Fulton-Gouterman equation

$$H_{FG}^{(p)}\varphi ^{(p)}=[\mathrm{\Omega }(b_1^{}b_1+b_2^{}b_2+1)+\alpha (b_1^{}+b_1)p\beta (b_2^{}+b_2)G_1)]\varphi ^{(p)}=E^{(p)}\varphi ^{(p)},p=\pm 1.$$
(5)
It is evident that procedure yielding the set (5) is equivalent to the FG transformation yielding Hamiltonian (3) in the Pauli $`2\times 2`$ matrix representation. From the diagonalized form (5) we can see that $`\beta `$-coupling breaks the degeneration of two electron levels which do not have the same energy.

As it was pointed out to us, Hamiltonian (1) and, equivalently also (5) (and all the related quantities), are symmetric against simultaneous exchange of $`pp`$, $`\alpha \beta `$ and, ensuingly, in terms of parameters introduced later on, $`\chi =\beta /\alpha 1/\chi `$ and $`\mu \alpha ^2/\mathrm{\Omega }^2\beta ^2/\mathrm{\Omega }^2`$.

In what follows we shall investigate variationally the ground state ($`p=1`$) with lower energy only in the representation of Eq. (5).

In two-level electron-phonon systems with linear coupling the coherent phonon subsystem does not conserve the number of phonons. Therefore, the upper level will share partly the distribution of phonons even in the ground state. From this reason, the two-center wave function in the form of asymmetric nonunitary ansatz with a variational parameter $`\eta `$ proposed by Shore et al  and Sonnek et al  for exciton or dimer models coupled to one phonon mode

$$|\mathrm{\Psi }^{(1)}=\frac{1}{\sqrt{C}_0}(1+\eta G)|1|\varphi ^{(1)},$$
(6)
($`C_0`$ is a normalization constant) was proved to yield better (lower) estimation of energy of the ground state , when compared with the eigenfunction $`|\varphi ^{(1)}`$ of $`H_{FG}`$ (5).

Our present suggestion of the variational ansatz for the ground state is motivated by the numerical solution for the wave function to the diagonalized Eq. (5). In the following numerical and analytical calculations it is convenient to use two basic parameters: the asymmetry parameter $`\chi =\beta /\alpha `$ and the effective coupling strength $`\mu =\alpha ^2/2\mathrm{\Omega }^2`$. The wavefunctions for the strong coupling $`\mu =2`$ and $`\chi =0.9`$, $`\chi =1`$, and $`\chi =1.5`$ are given in Figs. 1a,b,c (the standard numerical simulation procedure is outlined in the next Chapter) where the wavefunctions are depicted in the coordinate representation in the space of two phonon oscillators $`Q_1Q_2`$. Three distinct forms of the phonon solutions correspond to small, intermediate and large values of the parameter $`\chi `$.

Fig. 1a represents the “selftrapping” region in which contribution of the electron transitions between the levels (assisted by the oscillator 2) is small. Here the main Gaussian of the wave function $`\varphi ^+(Q_1)`$ at negative values of $`Q_1`$ ($`p=+1`$) is accompanied in the $`Q_1`$-subspace by the reflective part $`\eta \varphi ^{}(Q_1)`$ corresponding to $`p=1`$ parity which have already inspired the introduction of the nonunitary ansatz (6) representing the minor reflection respective to the axis $`Q_1=0`$ . However, in Fig. 1a, the admixture of the first excited state of the oscillator 2 (coordinate $`Q_20`$) of the parity $`p=1`$ (displaced to the right to $`Q_1>0`$) can be recognized, while for $`p=+1`$ both oscillators (region $`Q_1<0`$) remain in the Gaussian ground state. The variational treatment which is the topic of present paper aims mostly at capturing the situation of Fig. 1a as adequate as possible.

In Fig. 1b ($`\chi =1`$) which represents the case of E$``$e Jahn-Teller molecule, the rotational symmetric nature of the ground state at $`\chi =1`$ is easily recognizable. We mention by passing that an ideal rotation symmetry of the phonon wavefunction (“mexican hat”) which might be expected in the adiabatic case (that is that of big $`\mu `$) of a rotationally symmetric problem can not be reached even for very large $`\mu `$, as it was explained in detail by Eiermann et al. For the symmetric $`Ee`$ Jahn-Teller case we have merely an approximation to the “mexican hat” wavefunction profile spoiled by the non-zero angular momentum part which prevents the profile to show the complete rotational symmetry. This picture combines in it the features of self-trapping (Fig. 1a) and tunnelling (Fig. 1c) cases. The region close to E$``$e Jahn-Teller case appears also to show the most serious discrepancies of the variational treatment.

Fig. 1c represents another limit case - that of large values of $`\chi `$ corresponding to the (quantum) region of dominated phonon-assisted tunnelling. There the Gaussian form of the wave function is retained, although the second oscillator is displaced towards $`Q_2>0`$, while phonons 1 remain undisplaced (the Gaussian is centered at $`Q_10`$).

In Figs. 2a,b,c, we sketched three shapes of the effective potential (energy expression from the trial functions) controlled by the displacement parameters $`\gamma _1`$ and $`\gamma _2`$ only which refer to the variants of the wave functions in Figs. 1a,b,c, respectively.

Thus, guided by Fig. 1a, the variational wave function can be proposed in the following nonunitary form

$$\mathrm{\Psi }=\frac{1}{\sqrt{C}}\left[\varphi _0^{(+)}(\gamma _1,\gamma _2,r_1,r_2,\lambda )+\eta _1\varphi _0^{()}(\gamma _1,\gamma _2,r_1,r_2,\lambda )+\eta _2\varphi _1^{()}(\gamma _1,\gamma _2,r_1,r_2,\lambda )\right]$$
(7)
where $`7`$ variational parameters $`\gamma _i`$, $`r_i`$, $`\eta _i`$, and $`\lambda `$ are introduced and defined below.

Phonon wave functions $`\varphi _i^{(\pm )},i=0,1`$, are supposed to be squeezed coherent and correlated oscillators produced applying the set of generators on the phonon vacuum state as follows:

$`\varphi _0^{(\pm )}(\gamma _1,\gamma _2,r_1,r_2,\lambda )=D_1(\pm \gamma _1)S_1(r_1)D_2(\gamma _2)S_2(r_2)S_{12}(\pm \lambda )|0,`$ (8)
$`\varphi _1^{()}(\gamma _1,\gamma _2,r_1,r_2,\lambda )=D_1(\gamma _1)S_1(r_1)D_2(\gamma _2)S_2(r_2)S_{12}(\lambda )b_2^{}|0,`$ (9)
($`|0`$ is the phonon-$`1,2`$ vacuum state; indices $`0,1`$ at $`\varphi _0,\varphi _1`$ denote the ground and the first excited state of displaced phonons, respectively).

Here we defined the generators of variational displacements $`\gamma _i`$

$$D_i(\gamma _i)=\mathrm{exp}[\gamma _i(b_i^{}b_i)],$$
(10)
and those of squeezings parametrized by $`r_i`$

$$S_i(r_i)=\mathrm{exp}[r_i(b_i^2b_i^2)]$$
(11)
which are functions of variational parameters of displacement $`\gamma _i`$ and squeezing $`r_i`$ for $`i=1,2`$. In Eq. (5) the phonon modes $`1`$ and $`2`$ appear coupled in a highly non-linear way (through the term with $`\beta `$), therefore one also includes into the ansatz (7) the mode correlation generator

$$S_{12}(\lambda )=\mathrm{exp}[\lambda (b_1^{}b_2^{}b_1b_2)]$$
(12)
with the correlation variational parameter $`\lambda `$.

The functions $`\varphi _0^{(+)}`$ and $`\eta _1\varphi _0^{()}`$ in (7-9) represent displaced and squeezed oscillators in $`Q_1\times Q_2`$ space whose weight is shifted to the points $`(+\gamma _1,+\gamma _2)`$ and $`(\gamma _1,+\gamma _2)`$ respectively (see Fig.1a); The function $`\varphi _0^{}`$ is merely the reflection of $`\varphi _0^{(+)}`$ weighted by $`\eta _1`$: $`\varphi _0^{()}=G_1\varphi _0^{(+)}`$ (see (4-6)).

The function $`\varphi _1^{()}`$ is the excited oscillator $`b_2^+|0>_1|0>_2`$ displaced likewise $`\varphi _0^{()}`$ into the point $`(\gamma _1,\gamma _2)`$ of $`Q_1Q_2`$ space and squeezed as well (by parameters $`r_1,r_2`$) with weighting parameter $`\eta _2`$. In what follows we show that introducing this admixture of the excited state of the oscillator-2 in (7) essentially improves variational results.

The last variational parameter $`\lambda `$ enters in generators $`S_{12}(\pm \lambda )`$ which mix phonon modes together; this can be visualized as effective rotation of the two-dimensional Gaussian in the plane $`(Q_1,Q_2)`$; different signs $`\pm \lambda `$ keep trace of the reflection symmetry against the line $`Q_1=0`$ (it is best seen from Fig.1b that left and right “hills” should rotate in the opposite directions).

The complete expression for the mean value of the Hamiltonian $`H=H_{ph}+H_\alpha +H_\beta `$ (3) in the state (7)-(12) is given in Appendix B. A useful representation of that expression can be the following decomposition which separates the contributions of “ground” and “excited” parts of trial functions:

$`H={\displaystyle \frac{1}{C}}\left[(\varphi _0^{}+\eta _2\varphi _1^{()})H(\varphi _0+\eta _2\varphi _1^{()})\right]`$ (13)
$`={\displaystyle \frac{1}{C}}\left[\varphi _0^{}H\varphi _0+2\eta _2\varphi _0^{}H\varphi _1^{()}+\eta _2^2\varphi _1^{()}H\varphi _1^{()}\right],`$ (14)
where $`\varphi _0=\varphi _0^{(+)}+\eta _1\varphi _0^{()}`$, and

$`C=\left(1+\eta _1^2+2\eta _1\epsilon +\eta _2^2\right),\epsilon ={\displaystyle \frac{\mathrm{exp}\left(\frac{2\stackrel{~}{\gamma }_1^2}{\mathrm{cosh}2\lambda }\right)}{\mathrm{cosh}2\lambda }}.`$ (15)
The effective Hamiltonian (14) involves a highly nonlinear interplay of the variational parameters: the admixture of the state $`\varphi _1^{()}`$ contributes by the terms due to overlapping of the ground and first excited state of the oscillator $`\eta _2`$ and by its own excitation energy $`\eta _2^2`$.

In what follows, we investigate joint effects of quantum fluctuations and nonlinearity in the ground state of (14) by minimalization of the energy expression with respect to involved VPs $`\gamma _1,\gamma _2,r_1,r_2,\lambda ,\eta _1,\eta _2`$. Parameters of the displacement $`\gamma _1,\gamma _2`$ are defined by the displacement generators $`D_i(\gamma _i)`$ (10), parameters of squeezing $`r_i`$ by the generators of squeezing $`S_i(r_i)`$ (11), parameter of the mode correlation $`\lambda `$ by the generator of the correlation $`S_{12}(\lambda )`$ (12) and the parameters of asymmetry $`\eta _1,\eta _2`$ by the linear combination (7).

## III Interplay of quantum and nonlinear effects in selftrapping and tunneling

As it was shown in the last Section, the reflection symmetry hidden in the original Hamiltonian reveals in Eq. (5) due to the diagonalization by FG transformation in a highly nonlinear way. This nonlinearity implies new purely quantum region of the ground state with strong mixing of the nonlinearity and quantum fluctuations. There are several mechanisms supporting the quantum (nonadiabatic) fluctuations:

In the present model the relevant nonadiabaticity parameter is the ratio of the frequency and the coupling parameter $`\mathrm{\Omega }/\alpha `$. The ratio of the polaron energy $`\frac{\alpha ^2}{\mathrm{\Omega }}`$ and of the frequency $`\mathrm{\Omega }`$, $`\alpha ^2/\mathrm{\Omega }^2=2\mu `$ is a measure of the competition between the classical (polaron selftrapping) and quantum effects due to zero energy fluctuations. The quantum effects related to $`\mathrm{\Omega }`$ are thus relevant at weak couplings $`\mu `$.

The competition between the selftrapping ($`\alpha `$) and tunneling ($`\beta `$) terms results in occurrence of two regions of the ground state: In the phase plane $`\chi `$, $`\mu `$, the ground state exhibits two phases separated by the crossover line close to $`\chi =1`$ (Figs. 3, 4, 5, see also pertaining discussion in our earlier paper ). It means that the effective polaron potential exhibits two competing minima (Fig. 2a) governed by the model parameters $`\mu `$ and $`\chi `$. The minima coincide within the border of the regions lying close to the line $`\chi =1`$ (Fig. 2b). The phase $`\chi <1`$ is selftrapping dominated, with quantum fluctuations reflected in parameters $`r_1,r_2,\lambda ,\eta _i`$. The phase $`\chi >1`$ is the phonon 2-assisted tunneling dominated region with continuous virtual emission and absorption of phonons $`1`$. This phonon-1 exchange couples the levels within one minimum displaced merely by $`\gamma _2`$ due to the phonons 2. This minimum is much more sensitive to the change of model parameters $`\mu ,\chi `$ as well as to quantum fluctuations reflected in $`r_1,r_2`$, $`\lambda `$ and insensitive with respect to $`\eta _i,`$ while $`\eta _i0`$.

From the electronic point of view, the electron in the selftrapping dominated region is trapped by the phonons $`1`$, but due to the interactions mediated by the phonons 2 it can tunnel to the higher level. Then, owing to the reflection symmetry of the phonons 2, continuous oscillations of the electron at simultaneous virtual emission and absorption of phonons 1 occur. These oscillations couple the levels and thus the electrons into pairs localizing them in one minimum (Fig. 2c). This mechanism was described in a recent paper  for a one-dimensional lattice model.

An insight to the importance of different variational parameters can be gained by analytical minimalization of Hamiltonian (14) in various approximations. Numerical considerations show that contributions from the quantum parameters $`r_i,\eta _i,\lambda `$ are at least by an order smaller than those from the classical parameters. Including also $`\eta _2`$, we get approximately ($`\mathrm{\Omega }=1`$):

$$H=\gamma _1^2+\gamma _2^2+1+2\alpha \gamma _1\frac{(1\eta _2^2)}{(1+\eta _2^2)}2\beta \gamma _2\epsilon +2\eta _2(\gamma _2\epsilon \beta ).$$
(16)
Assuming all nonadiabatic parameters small and minimalizing (16) we get approximately

$$\gamma _1\left(1+4\beta ^2\epsilon ^2\right)=\alpha ,\gamma _2=\beta \epsilon ,$$
(17)
where $`\epsilon \mathrm{exp}(2\gamma _1^2)`$ (15). From these equations, approximate expressions for both regions (small and large $`\chi `$) are summarized below:

In the “selftrapping” region $`\alpha >\beta `$ ($`\chi <1`$) there is $`\epsilon 1`$ and we get:

$$\gamma _1\alpha ,\gamma _2=\beta \epsilon 1,$$
(18)
(that is small $`\gamma _2`$ and large negative $`\gamma _1`$); In the framework of this approximation (taking also $`\eta _2`$ small) respective ground state energy in the selftrapping region results in

$$E_G^\alpha 1\frac{\alpha ^2}{1+8\beta ^2\mathrm{exp}(4\alpha ^2)}\beta ^2\mathrm{exp}(4\alpha ^2).$$
(19)
For the tunnelling dominated region, vice versa:

$$\gamma _1\frac{\alpha }{1+4\beta ^2}1,\epsilon 12\gamma _1^2,\gamma _2=\beta \epsilon \beta .$$
(20)
In this case the ground state energy is approximated by

$$E_G^\beta 1\frac{\alpha ^2}{1+8\beta ^2}\beta ^2.$$
(21)
Comparing both ground state energies (19) and (21) we observe the asymmetry against the exchange of the $`\alpha `$ and $`\beta `$ of both results due to the screening of the tunneling term $`\beta ^2`$ by $`\epsilon =\mathrm{exp}(2\gamma _1^2)`$ which is either vanishingly small (18) or $`1`$ (20). Evidently, it is caused by the presence of the nonlinear $`G_1`$ factor in the Fulton-Gouterman Hamiltonian (3) or (5).

Let us examine more thoroughly the parameter $`\eta _2`$ representing the relative weight of the “excited” admixture. Qualitatively important features brought by the Ansatz (7) can be found analytically merely from the simplified version of the Hamiltonian (14) and (45)-(59) with only displacements $`\gamma _i`$ and $`\eta _2`$.

The equation for optimized value $`\eta _2`$ for both regions reads exactly as

$$\eta _2(14\alpha \gamma _1)(1\eta _2^2)\beta (1\epsilon ^2)=0;$$
(22)
Inserting there the above expressions for $`\gamma _i`$ (17-20) we get:

* for the selftrapping region

$$\eta _2^{(I)}\frac{\beta (1\epsilon ^2)}{1+4\alpha ^2},$$
(23)
* for the tunnelling region

$$\eta _2^{(II)}\frac{4\beta \gamma _1^2}{14\alpha \gamma _1}0,$$
(24)

$`\gamma _1`$ given by (20).

This analytical estimation shows that the admixture of the excited $`2`$-phonons should play the most important part in the selftrapping region only; This conclusion is in the complete accordance with the shapes of the wave functions for both regions (Figs 1a, 1c), as well as with further exposed results of minimalization of variational energies (Figs. 3-5).

Further, exploiting the influence of $`\lambda `$ and $`\eta _1`$ separately (using (14), Appendix B and (18) for $`\chi <1`$ ), we get estimations:

$`\mathrm{sinh}(2\lambda )2\beta \epsilon \gamma _14\chi \mu \mathrm{exp}(4\mu ),\eta _1{\displaystyle \frac{\beta \gamma _2}{\alpha \gamma _1}}\chi ^2\mathrm{exp}(4\mu ),`$ (25)
where $`\mu =\alpha ^2/2\mathrm{\Omega }^2`$ was defined at the beginning of the Section III. as the parameter of the effective interaction. (The first dependence of (25) is recognizable on Fig. 6 and will be discussed later in relation to the necessity of accounting for mode correlation).

We calculated then the optimized values of the variational parameters finding numerically the minimum of the energy functional in different approximations \- starting from the most complicated case of complete expression (Appendix B) with 7 parameters included and ending with the adiabatic ansatz with the displacements $`\gamma _1,\gamma _2`$ only.

In order to check the validity of the variational calculations we performed also the numerical diagonalization of the Hamiltonian in the phonon-$`1,2`$ space. We truncated the (infinite) phonon space by $`N_1`$ 1-phonon states and $`N_2`$ 2-phonon states, thus the state vector is $`N_1\times N_2`$ dimensional. As numerical diagonalization results show, about 20-50 phonon states are sufficient for convergence. In Figs. 3 to 5 we also show the results of numerical diagonalization of Hamiltonian matrix as function of $`\chi `$ for $`\mu =0.5`$, $`\mu =1`$ and $`\mu =4`$. In the first two cases we took 20$`\times `$20 state vector, while in the latter case to achieve satisfactory convergence (especially for the tunneling-dominated region when $`\chi >1`$) we had to increase the number of phonons-2 up to 50.

The two energy curves from exact numerical solution and “adiabatic” variational treatment present correspondingly the lower and upper bound for variationally calculated energy, thus any reasonable variational results should lie between these bounds, and reliability of a variation ansatz for a given parameter region can be judged according to how close the corresponding ground state energy is to these limits (Figs. 3-5).

We limited ourselves by showing merely some crossections of the plane $`\chi ,\mu `$ which seem to be typical in discussing the validity of the variational approach vs “true” solution via numerical diagonalization in the phonon subspace. From Figs. 3b to 5b showing the differences between “variational” and “exact” energy for various variational approximations it is seen that while the curves with all parameters $`\eta _i,\lambda ,r_i,\gamma _i`$ included give minimal discrepancy (which is evident, since increasing the number of variational parameters within the same trial function class leads to improving the results), the maximal discrepancy, and hence the maximal effect of additional parameters $`\eta _i`$, $`\lambda `$ is observed near the line $`\chi 1`$, this region of observed maximal discrepancy being shifted to the point $`\chi =1`$ with growing $`\mu `$.

From the results of numerical minimalization of the variational expressions for energy (see Fig. 3-5) one can see that the “excitation-reflection” ansatz (ERA) (7) ($`\eta _2`$ included) results in fascinating improvement of the variational simulation of the problem in comparison with the “simple reflection” ansatz (SRA) ($`\eta _1`$ only); especially it is evident from Figs. 3b, 4b, 5b, where the differences between variational and “exact” energies are plotted. Although this ansatz can be to a larger extent inspired by merely the shape of the wave function (Fig. 1a) one might wonder that the ansatz containing an excited oscillator as the “reflective” part of the trial function lowers the total energy of the system in comparison with the ansatz containing the “zero” state of the oscillator ($`\eta _2=0`$). An insight to better understanding this phenomenon can be gained examining the variational energy expression (14), (45)-(59). The energy can be split into three parts representing respectively the energy of the “main” Gaussian, that of the reflection part and “overlapping” exchange terms (those containing $`\eta _i^0`$, $`\eta _i^2`$, $`\eta _i^1`$ respectively). Indeed when we switch from SRA towards ERA the energy of the reflection part is increased by $`\eta _2^2`$ by virtue of one extra displaced “phonon” (59); But, if one compares the overlapping terms for both expressions (below) one can see that it is the overlapping term of ERA (53), (26) which significantly decreases the overall energy, while the respective overlapping term of SRA (45), (27) contributes only slightly.

The main contribution to the overlapping integral is contained in the term $`\mathrm{\Phi }_0(\gamma _1)|Q_2G|\mathrm{\Phi }_1(\gamma _1)=\mathrm{\Phi }_0(\gamma _1)|Q_2|\mathrm{\Phi }_1(\gamma _1)1`$ ($`Q_2`$, second phonon coordinate, is a short-hand for $`(b_2^++b_2)`$); other terms contain a small “overlapping” factor $`\epsilon \mathrm{exp}(2\gamma _1^2)`$. Using rough estimations (the same as those leading to (17)) we get following expression for the “overlapping” part of energy in the ERA:

$$E_l2\eta _2(\beta e^{2r_2}\mathrm{cosh}\lambda +\alpha e^{2r_1}\mathrm{sinh}\lambda )2\eta _2\beta ,$$
(26)
(this expression is valid for the region $`\chi <1`$). In (26), it is the $`\beta `$-term which yields the main contribution.

As for the SRA, its counterpart reads as $`\mathrm{\Phi }_0(\gamma _1)|Q_2|\mathrm{\Phi }_0(\gamma _1)`$ and vanishes due to symmetry, leaving us only smaller terms $`\epsilon `$,:

$$E_l4\beta \eta _1\epsilon $$
(27)
(the higher order terms $`\epsilon ^2`$, $`\epsilon ^2\eta `$ etc. are omitted).

Comparing these expressions we can see that ERA yields better use of the reflection symmetry property of Hamiltonian contributing greatly to the overlapping integral (to the negative exchange energy) while for the SRA this principal contribution vanishes merely because of symmetry, leaving us with minor contributions $`\epsilon `$ only, thus there the whole idea of the reflective ansatz losses much of its effectivity.

This effect of lowering energy in the excitation ansatz due to overlapping finds its origin in the presence of phonon-$`2`$ assistance. In the dimer or exciton models instead of $`\beta \widehat{Q_2}\sigma _x`$ term of the model Hamiltonian (1) there stands merely $`\mathrm{\Delta }\sigma _x`$ with the constant $`\mathrm{\Delta }`$ and this principal part of the exchange energy (overlapping integral) for the excitation ansatz vanishes.

The very general impression from Figs 3-5 (b) makes us to state that introducing nonunitary parameters ($`\eta _i`$) essentially improves the variational treatment for the self-trapping region; In the tunnelling region merely Gaussian expressions for displaced oscillators with squeezing (parameters $`r_i`$) gives us a satisfactory fit. Indeed, as it was demonstrated from analytical estimations and as it is seen from Figs. 1 a,c, the admixture of reflection part is relevant rather for selftrapping region where this form of the trial function is the best choice.

However, the closer we are to the intermediate region between selftrapping and tunnelling phases, the stronger are discrepancies for all curves. As it is seen from Fig.1 b it is the case where the wavefunctions display their radial symmetric structure. Examining on Figs. 3-5 the curves corresponding to variational ansatzes with or without mixing parameter $`\lambda `$ we see that this parameter essentially lowers the energy exactly in the region of $`\chi 1`$. It is worth comparing Fig. 3 for $`\mu =0.5`$ (weak coupling) with Figures 4 and 5 ($`\mu =1,\mu =4`$), both representing strong couplings. We see immediately that the coupling parameter $`\lambda `$ gains importance rather for small couplings where it improves the results for wider range of $`\chi `$, and not only for $`\chi 1`$.

Fig. 6 where we plotted the differences of the variational energy calculated with and without taking into account the mode correlation $`S_{12}(\lambda )`$ illustrates this statement (see also (20)) by showing the regions of importance of the correlation parameter $`\lambda `$ in the whole plane ($`\chi ,\mu `$). The mode correlation represented by $`\lambda `$ (Fig. 6) appears to be by one order larger than the contribution of the competing nonlinearity due to the reflection level mixing $`\eta _1`$. The correlation $`\lambda `$ is most effective for weak effective couplings $`\mu `$ at $`\chi (0.5,1.5)`$ where it competes the selflocalization in support of the tunneling phase. For large $`\mu `$ it contributes only very close to $`\chi =1`$, where it reveals a maximum for all $`\mu `$. This is quite understandable if we note that introducing $`S_{12}(\lambda )`$ (12) means effective rotation of trial functions (displaced Gaussians) in the plane $`Q_1Q_2`$, and $`\varphi ^+(\lambda )`$, $`\varphi ^{}(\lambda )`$ are rotated in the opposite directions symmetrically with respect to the line $`Q_1=0`$, which indeed repeats Fig.1b, fitting the rotation symmetry features. This degree of freedom allows to represent the picture of the wave function especially in the transition region where selftrapping and tunnelling regions are mixed together and are hardly distinguishable, which is the case of weak couplings. At strong couplings those regions are more pronounced, the border between them is sharper; in this case the parameter $`\lambda `$ looses its importance with exception of the vicinity of $`\chi 1`$ (Fig. 6).

In the case of omitting $`\lambda `$, from (40)-(59) one can see that the optimized value $`r_2=0`$, i.e. the contribution due to $`r_2`$ in (40) is mediated merely by the correlation $`\lambda `$. For $`\lambda 0`$, the squeezing $`r_2`$ significantly interplays with $`r_1`$ especially for small $`\mu `$ and $`\chi 1`$, as it brings almost half of the contribution of $`r_2`$. This effect was omitted in the variational treatment of E$``$e model by Lo. Some authors disregarded this circumstance setting $`r_1=r_2`$ for simplicity, but omitting the mode correlation ($`\lambda `$) which is therefore not selfconsistent.

In the lattice case, the coupling with the lattice is represented by a transfer term in the Hamiltonian of the order of magnitude of the bandwidth $`T`$. When $`T`$ is sufficiently large so that the “transfer” part of the energy is comparable to the “local” energy contribution, the effect of $`\eta _1`$ is considerably stronger, and we do not observe suppression caused by introducing extra correlation between phonon modes (that is an analogue to the parameter $`\lambda `$). The phonon-$`1,2`$ correlation in the lattice case is of smaller order of magnitude than the contributions from the transfer terms (respectively of the order $`\beta \mathrm{exp}(\gamma _1^2)`$ and $`T`$). Because of that introducing the correlation VP in the lattice case does not yield considerable improvement of the results.

## IV Conclusion

Hamiltonian (3) allows us to distinguish two competing regimes of the electron-phonon system according to the relations of the parameters specifying (a) selflocalization $`\alpha `$ vs quantum fluctuations $`\mathrm{\Omega }`$ and (b) tunneling $`\beta `$ vs selflocalization $`\alpha `$. Then, in terms of the relevant parameters $`\mu =\frac{\alpha ^2}{2\mathrm{\Omega }^2}`$ (effective interaction) and $`\chi =\frac{\beta }{\alpha }`$ (asymmetry), two quantum regions can be identified: (i) $`\mu 0.5`$ and (ii) $`\beta /\alpha 1`$. In these regions the quantum fluctuations are most pronounced and variational ansatz which pretends to be the most suitable should fit there the numerical data at best. Our choice of the wave function (7)-(12) covers both regions in a complementary way: while the choice of ERA with the admixture of the excited state of the symmetric phonon mode weighted by $`\eta _2`$ and including the mode correlation $`\lambda `$ improves greatly the variational results in the selftrapping region, $`\chi 1`$, in the whole range of $`\mu `$ (Figs. 2, 4, and 5), with increasing $`\mu `$ effectiveness of all quantum variational parameters vanishes except of $`\eta _2`$ (Figs. 4 and 5). In the tunneling region, $`\chi >1`$, the choice of ERA looses its justification in benefit of SRA, while all the remaining parameters keep their effectiveness even for large $`\mu `$ (Figs. 4 and 5).

The cooperative effect of the reflection (antisymmetric phonon mode) and of the assistance of the symmetric mode in the tunneling results in a nonlinear interplay of both modes. It consists in the competition between the negative contribution of the overlapping of the wave functions of different parity with respect to the reflection and the increase of excitation energy of the respective mode. This concept leads to the effective energy decrease of the excited symmetric reflected mode (ERA) rather than of its ground state (SRA).

The complex nonlinear interplay of the modes was elucidated by exact numerical diagonalization of Hamiltonian (3): in fact, we took inspiration for ERA from the exact ground state wave function for $`\alpha \beta `$ (Fig. 1a). It suggested presence of admixture of the first excited oscillator state of the symmetric mode in the reflected part of the wave function. In the case of $`\alpha <\beta `$, the numerical wave function exhibited only a single symmetric peak of a well defined harmonic oscillator. The peak was located close to the center of the reflection symmetry $`Q_1=0`$ but displaced by the phonons $`2`$. It corresponded to a new minimum of the effective potential which opened due to the ”bond selflocalization” on account of the joint effect of both modes (Fig. 2c). Note that these states are well localized, in the contrast to the states in the “selftrapping” region. It is interesting to mention in this context a special class of states in the excited spectra of J-T models which were called “exotic states”  and were characterized by a pronounced localization of the corresponding phonon mode. These states in the excitation spectra can be explained as the consequence of the energy resonance and hence the tunnelling between two wells of the effective potential (visualized, e.g., on Fig. 2) which suggested opening of the additional potential well in the position $`\gamma _10`$ where the exotic mode is to be localized. Although in the present paper we investigated merely the ground state of the model, we can see that our localized modes in the tunnelling dominated region ($`\chi >1`$) have essentially the same origin (the “localized” minimum at zero displacement along $`Q_1`$ of the effective potential) and bear interesting resemblance to the Wagner’s exotic states. In support of this statement the full spectrum of the phonon states, and not the ground one only should be examined, but this problem merits a special paper. We just mention now our own results on numerical calculation of the energy spectrum for the excited modes in the tunnelling region which show specific periodic (in model parameter $`\mu `$) chaotic “windows” especially for higher modes. This periodicity in the coupling strength, which we at the very beginning scaled by the phonon frequency $`\mathrm{\Omega }`$ clearly indicated the resonance behaviour, i.e. occasional coincidence of two incommensurable characteristic frequencies of the system whose nature can be identified with the origin of Wagner’s resonant exotic states. For low lying modes like those of the ground state this behaviour is not so pronounced, but merely its traces are also recognizable.

The results exposed above, especially when speaking about the validity of the variational approach chosen, are most adequate far from the rotationally symmetric Jahn-Teller case ($`\alpha =\beta `$, or $`\chi =1`$) which was to be expected on basis of the comment to the FG transformation in the Section II. In this case one gets an almost degenerated degree of freedom in the space of variational parameters; namely, if we introduce an analogue of the polar coordinates in the phonon-1,2 space (as some authors, ,  do), certain degeneracy of the energy profile over the angular coordinate would be observed and thus this angular parameter should have been excluded from being variated. (Strictly speaking, the angularly degenerated true “mexican hat” appears only in adiabatic approximation). The authors exploiting various variational treatments disregarding this circumstance must have encountered such problems for the E$``$e Jahn-Teller case. However, these inconsistencies become crucial rather at big coupling strength $`\mu `$, e.g., Fig. 4, 5). However, the rotational symmetry can be to some extent retained even within the formalism of rectangular $`Q_1Q_2`$ space by introducing the mode mixing parameter $`\lambda `$ which gains its significance especially in the region of weak coupling effectively spanning both regions (Fig. 6). The variational approach exploiting essentially the rotational symmetry of the model should present a complementary description suitable in the vicinity of $`\chi =1`$, the whole problem however being a subject of our further considerations.

The support from the Grant Agency of the Czech Republic of our project No. 202/01/1450 is highly acknowledged. We thank also the grant agency VEGA (No. 2/7174/20) for partial support.

## Appendix A.

We have used following formulas for $`D_i`$, $`S_i`$, and $`S_{12}`$ defined by (10)-(12) which can be found elsewhere ,:

$`D_i(\gamma _i)^1b_iD_i(\gamma _i)=b_i+\gamma _i,i=1,2,`$ (28)
$`D_1(\gamma _1)S_1(r_1)=S_1(r_1)D_1(\stackrel{~}{\gamma }_1),\stackrel{~}{\gamma }_1=\gamma _1e^{2r},`$ (29)
$`S_i^1(r_i)b_iS_i(r_i)=b_i\mathrm{cosh}2r_i+b_i^{}\mathrm{sinh}2r_i,`$ (30)
$`S_{12}^1(\lambda )b_1S_{12}(\lambda )=b_1\mathrm{cosh}\lambda b_2^{}\mathrm{sinh}\lambda ,`$ (31)
$`0|S_1^{}(r_1)D_1^{}(\gamma _1)\mathrm{exp}(\lambda b_1^{})={\displaystyle \frac{1}{(\mathrm{cosh}2r_1)^{1/2}}}\times `$ (32)
$`\times \mathrm{exp}\left[{\displaystyle \frac{\lambda ^2}{2}}\mathrm{tanh}(2r)\lambda \gamma _1(\mathrm{tanh}2r1)+{\displaystyle \frac{\gamma _1^2}{2}}(\mathrm{tanh}2r1)\right],`$ (33)
$`0|S_1(r)D_1(\gamma _1)b_1^m={\displaystyle \frac{d^m}{d\lambda ^m}}0|S_1(r_1)D_1(\gamma _1)\mathrm{exp}(\lambda b_1^{})|_{\lambda =0},`$ (34)
$`S_{12}(\lambda )=T^{}\left({\displaystyle \frac{\pi }{4}}\right)S_1\left({\displaystyle \frac{\lambda }{2}}\right)S_2\left({\displaystyle \frac{\lambda }{2}}\right)T\left({\displaystyle \frac{\pi }{4}}\right),`$ (35)
$`T(\delta )=\mathrm{exp}\left(\delta (b_1^{}b_2b_1b_2^{})\right)`$ (36)
$`T\left({\displaystyle \frac{\pi }{4}}\right)\left(\begin{array}{c}b_1\\ b_2\end{array}\right)T^{}\left({\displaystyle \frac{\pi }{4}}\right)={\displaystyle \frac{1}{\sqrt{2}}}\left(\begin{array}{c}b_1b_2\\ b_1+b_2\end{array}\right),`$ (37)
$`T^{}|0=T|0=|0.`$ (38)
## Appendix B.

$`H={\displaystyle \frac{1}{C}}[(\mathrm{\Psi }_0^{(+)}+\eta _1\mathrm{\Psi }_0^{()})H(\mathrm{\Psi }_0^{(+)}+\eta _1\mathrm{\Psi }_0^{()})+2\eta _2\mathrm{\Psi }_0^{(+)}H\mathrm{\Psi }_1^{()}`$ (39)
$`+2\eta _1\eta _2\mathrm{\Psi }_0^{()}H\mathrm{\Psi }_1^{()}+\eta _2^2\mathrm{\Psi }_1^{()}H\mathrm{\Psi }_1^{()}]`$ (40)
where

$`(\mathrm{\Psi }_0^{(+)}+\eta _1\mathrm{\Psi }_0^{()})H(\mathrm{\Psi }_0^{(+)}+\eta _1\mathrm{\Psi }_0^{()}=(1+\eta _1^2+2\eta _1\epsilon )[{\displaystyle \frac{1}{2}}(\mathrm{cosh}4r_1+\mathrm{cosh}4r_2)\mathrm{cosh}2\lambda +\gamma _1^2+\gamma _2^2]`$ (41)
$`+2\eta _1\epsilon \{\mathrm{tanh}2\lambda \mathrm{sinh}2\lambda \mathrm{cosh}2(r_1+r_2)\mathrm{cosh}2(r_1r_2)`$ (42)
$`+\stackrel{~}{\gamma }_1^2[(e^{2(r_1+r_2)}e^{(2(r_1+r_2))}\mathrm{cosh}4\lambda )(1+\mathrm{tanh}^22\lambda )+2e^{(2(r_1+r_2))}\mathrm{sinh}4\lambda \mathrm{tanh}2\lambda ]\mathrm{cosh}2(r_1r_2)`$ (43)
$`+{\displaystyle \frac{\stackrel{~}{\gamma }_1^2}{\mathrm{cosh}^22\lambda }}(\mathrm{sinh}4r_1\mathrm{sinh}4r_2)2\gamma _1(\mathrm{tanh}(2\lambda )e^{2(r_2r_1)}\gamma _2+\gamma _1)\}`$ (44)
$`+{\displaystyle \frac{2\alpha }{\mathrm{\Omega }}}(1\eta _1^2)\gamma _1{\displaystyle \frac{2\beta }{\mathrm{\Omega }}}\left[(1+\eta _1^2)[\gamma _2\gamma _1\mathrm{tanh}2\lambda e^{2(r_2r_1)}]\epsilon +2\eta _1\gamma _2\right]`$ (45)
$`\mathrm{\Psi }_0^{(+)}H\mathrm{\Psi }_1^{()}=\left[{\displaystyle \frac{1}{2}}(\mathrm{cosh}4r_1+\mathrm{cosh}4r_2)\mathrm{cosh}2\lambda +\gamma _1^2+\gamma _2^2\right]{\displaystyle \frac{1}{\sqrt{2}}}(b_1\mathrm{}+b_2\mathrm{})`$ (46)
$`+{\displaystyle \frac{1}{2\sqrt{2}}}((\mathrm{cosh}2(\lambda +2r_1)+\mathrm{cosh}2(\lambda +2r_2))]b_2\mathrm{}+(\mathrm{cosh}2(\lambda 2r_1)+\mathrm{cosh}2(\lambda 2r_2))b_1\mathrm{})`$ (47)
$`+{\displaystyle \frac{1}{4\sqrt{2}}}\left[(\mathrm{sinh}2(\lambda 2r_1)\mathrm{sinh}2(\lambda 2r_2))b_1^3\mathrm{}+(\mathrm{sinh}2(\lambda +2r_1)+\mathrm{sinh}2(\lambda +2r_2))b_2^3\mathrm{}\right]`$ (48)
$`+[\mathrm{sinh}4r_1+\mathrm{sinh}4r_2{\displaystyle \frac{1}{2}}(\mathrm{sinh}2(\lambda 2r_1)+\mathrm{sinh}2(\lambda 2r_2))]{\displaystyle \frac{1}{2\sqrt{2}}}b_1^2b_2\mathrm{}`$ (49)
$`+[\mathrm{sinh}4r_1+\mathrm{sinh}4r_2+{\displaystyle \frac{1}{2}}(\mathrm{sinh}2(\lambda +2r_1)\mathrm{sinh}2(\lambda +2r_2))]{\displaystyle \frac{1}{2\sqrt{2}}}b_1b_2^2\mathrm{}`$ (50)
$`+[(\gamma _1e^{2r_1}\mathrm{sinh}\lambda +\gamma _2e^{2r_2}\mathrm{cosh}\lambda )](b_1b_2\mathrm{}+\epsilon )`$ (51)
$`+{\displaystyle \frac{1}{2}}(\gamma _1e^{2r_1}+\gamma _2e^{2r_2})e^\lambda b_1^2\mathrm{}.+{\displaystyle \frac{1}{2}}(\gamma _1e^{2r_1}+\gamma _2e^{2r_2})e^\lambda b_2^2\mathrm{}.{\displaystyle \frac{\beta }{\mathrm{\Omega }}}\mathrm{cosh}\lambda e^{2r_2}`$ (52)
$`+{\displaystyle \frac{\alpha }{\mathrm{\Omega }}}\left[e^{2r_1}\mathrm{sinh}\lambda (\epsilon +b_1b_2\mathrm{})+{\displaystyle \frac{e^{2r_1}}{2}}\left(b_1^2\mathrm{}e^\lambda b_2^2\mathrm{}e^\lambda \right)\sqrt{2}\gamma _1\left(b_1\mathrm{}+b_2\mathrm{}\right)\right]`$ (53)
$`\mathrm{\Psi }_0^{()}H\mathrm{\Psi }_1^{()}=(\mathrm{sinh}\lambda \gamma _1e^{2r_1}+\mathrm{cosh}\lambda \gamma _2e^{2r_2}){\displaystyle \frac{\alpha }{\mathrm{\Omega }}}\mathrm{sinh}\lambda e^{2r_1}{\displaystyle \frac{\beta }{\mathrm{\Omega }}}[\sqrt{2}\gamma _2(<b_2\mathrm{}>+<b_1\mathrm{}>)`$ (54)
$`+{\displaystyle \frac{1}{2}}(e^{\lambda +2r_2}<b_2^2\mathrm{}>+e^{\lambda +2r_2}<b_1^2\mathrm{}>+\mathrm{cosh}\lambda e^{2r_2}<b_1b_2\mathrm{}>+\mathrm{cosh}\lambda e^{2r_2}<\mathrm{}>]`$ (55)
$`\mathrm{\Psi }_1^{()}H\mathrm{\Psi }_1^{()}=\gamma _1^2+\gamma _2^2+\mathrm{cosh}2\lambda (\mathrm{cosh}4r_1+\mathrm{cosh}4r_2)+\mathrm{sinh}2(r_1+r_2)\mathrm{sinh}2(r_2r_1)`$ (56)
$`{\displaystyle \frac{\alpha }{\mathrm{\Omega }}}2\gamma _1{\displaystyle \frac{\beta }{\mathrm{\Omega }}}\{2\gamma _2\mathrm{cosh}2\lambda \epsilon +\gamma _2(<b_2^2\mathrm{}><b_1^2\mathrm{}>)\mathrm{sinh}\lambda `$ (57)
$`+{\displaystyle \frac{e^{2r_2}}{2\sqrt{2}}}[(<b_1\mathrm{}>e^\lambda +<b_2\mathrm{}>e^\lambda )4\mathrm{cosh}\lambda `$ (58)
$`+((<b_2^3\mathrm{}><b_1^2b_2\mathrm{}>)e^\lambda +(<b_2^2b_1\mathrm{}><b_1^3\mathrm{}>)e^\lambda )\mathrm{sinh}\lambda ]\}`$ (59)
The mean values in (15)

$`b_1D_1(\sqrt{2}\stackrel{~}{\gamma }_1e^\lambda )S_1(\lambda )D_2(\sqrt{2}\stackrel{~}{\gamma }_1e^\lambda )S_2(\lambda )={\displaystyle \frac{\sqrt{2}\stackrel{~}{\gamma }_1e^\lambda }{\mathrm{cosh}^22\lambda }}\epsilon ;`$ (60)
$`b_1^2D_1(\sqrt{2}\stackrel{~}{\gamma }_1e^\lambda )S_1(\lambda )D_2(\sqrt{2}\stackrel{~}{\gamma }_1e^\lambda )S_2(\lambda )`$ (61)
$`={\displaystyle \frac{1}{\mathrm{cosh}2\lambda }}\left(\mathrm{tanh}2\lambda +2\stackrel{~}{\gamma }_1^2e^{2\lambda }(\mathrm{tanh}2\lambda 1)^2\right)\epsilon ;`$ (62)
$`b_1^3D_1(\sqrt{2}\stackrel{~}{\gamma }_1e^\lambda )S_1(\lambda )D_2(\sqrt{2}\stackrel{~}{\gamma }_1e^\lambda )S_2(\lambda )`$ (63)
$`={\displaystyle \frac{\sqrt{2}\stackrel{~}{\gamma }_1e^\lambda (\mathrm{tanh}2\lambda 1)}{\mathrm{cosh}2\lambda }}\left(3\mathrm{tanh}2\lambda +2\stackrel{~}{\gamma }_1^2e^{2\lambda }(\mathrm{tanh}2\lambda 1)^2\right)\epsilon ;`$ (64)
$$b_2^k\mathrm{}=b_1^k\mathrm{}|_{\gamma _1\gamma _1,\lambda \lambda },k=1,2,3.$$

## References

Figure Captions

Fig. 1. The numerical ground state wave functions at $`\mu =2`$ and $`\chi =0.9`$ (a) $`\chi =1`$ (b) and $`\chi =1.5`$ (c).

Fig. 2. Shapes of the effective potential corresponding to the wave functions at Figs.1a, 1b and 1c, respectively.

Fig. 3. (a) The ground state energies (40) for $`\mu =0.5`$. The seltrapping dominated GS spans over $`\chi <1`$ and the tunneling dominated GS over $`\chi >1`$. The curves plotted represent cases (from below): numerical simulation GS, $`E_{ex}`$; ERA, $`E_1`$; ERA, $`E_1(\lambda =0)`$; SRA, $`E_0`$; SRA, $`E_0(\eta _1=\lambda =0)`$; adiabatic GS $`E_{ad}`$.
(b) Differences of the ground states from (a) and the exact numerical GS. ERA considerably improves the results for $`\chi <1`$. It also shifts the maximum of the differences to the point $`\chi =1`$.

Fig. 4. The same as in Fig. 3 for $`\mu =1`$. With increasing $`\mu `$, for $`\chi <1`$, the loss of efficiency of all VP except of $`\eta _2`$ is evident.

Fig. 5. The same as in Fig. 3 and 4 for $`\mu =4`$.

Fig. 6. The difference of two GS: the GS without the reflection and correlation effects $`E_0(\eta =0,\lambda =0)`$ and the GS $`E_0`$ including these effects.


format: text

text:
# Limits in Function Spaces and Compact Groups11footnote 1 2000 Mathematics Subject Classification: Primary 54H11, 22C05; Secondary 46E25, 54C35. Key Words and Phrases: Compact group, pointwise topology.

## 1 Introduction

In this paper, we answer a question on topological groups asked by Barbieri, Dikranjan, Milan, and Weber , and we relate this question to some general facts in $`C_p`$ theory. All spaces considered here are Hausdorff. We recall the following standard definition; see Arkhangel’skii  for further details on such function spaces.

###### Definition 1.1

If $`X,Y`$ are topological spaces, then $`C(X,Y)`$ is the set of continuous functions from $`X`$ to $`Y`$, and $`C_p(X,Y)`$ denotes $`C(X,Y)`$ given the topology of *pointwise convergence* (i.e., regarding $`C_p(X,Y)`$ as a subset of $`Y^X`$ with the usual product topology).

An obvious question is:

* Suppose that each $`f_nC_p(X,Y)`$ and $`g`$ is a limit point of the sequence $`f_n:n\omega `$; is there an infinite $`B\omega `$ such that the subsequence $`f_n:nB`$ converges to $`g`$ pointwise (i.e., in the topology of $`C_p(X,Y)`$)?

The answer is well-known to be “no” in general, even when $`X`$ and $`Y`$ are both compact metric spaces. If $`Y`$ is metric, then for each $`xX`$, there is an infinite $`B_x`$ such that $`f_n(x):nB_xg(x)`$. $`B_x`$ may need to vary with $`x`$, but if $`X`$ is compact, then one can choose the $`B_x`$ from a filter; in Section 2 we show:

###### Proposition 1.2

Suppose that $`X`$ is compact, $`Y`$ is metric, and $`g`$ is a limit point of $`f_n:n\omega `$ in $`C_p(X,Y)`$. Then there is a filter $`[\omega ]^\omega `$ such that for each $`xX`$ there is $`B`$ such that $`f_n(x):nBg(x)`$. Furthermore, $``$ can be chosen to be an $`F_\sigma `$ subset of $`𝒫(\omega )`$ (identifying $`𝒫(\omega )`$ with $`2^\omega =\{0,1\}^\omega `$ in the standard way).

As usual, $`𝒫(\omega )`$ denotes the family of all subsets of $`\omega `$ and $`[\omega ]^\omega `$ denotes the family of infinite subsets of $`\omega `$.

One cannot replace “metric” by first countable here, since the proposition fails when $`X,Y`$ are both the lexicographically ordered square; see Example 4.2.

Observe that, in the notation of Proposition 1.2, $`g`$ must be a limit point of $`f_n:nB`$ in $`C_p(X,Y)`$ for each $`B`$ (see also Lemma 2.2). From this it is easily seen that the $``$ is not independent of the $`f_n`$ and $`g`$.

Now, let us specialize the above discussion to topological groups, where we consider one “fixed” sequences defined uniformly across all groups:

###### Definition 1.3

In a group $`X`$, define $`E_n(x)=x^n`$.

Note that $`E_0(x)=1`$ for all $`x`$. As is well-known, if $`X`$ is a compact topological group and $`xX`$, then $`1`$ is a limit point of the sequence $`x^n:n\omega `$. Since the same applies to every finite power $`X^k`$, it follows that $`E_0`$ is a limit point of $`E_n:n\omega `$ in $`C_p(X,X)`$. In particular, if $`X`$ is also metric (equivalently, second countable), then Proposition 1.2 applies to yield a filter. In fact, the same filter $``$ can be chosen to work for every compact metric group, since any $``$ which works for the group $`𝕋^\omega `$ works for all compact metric groups (see Lemma 3.2). As usual, $`𝕋`$ denotes the circle group.

Returning to (✾), is there one $`B`$ such that $`E_n:nBE_0`$? This is answered by Proposition 1.5 below, which is proved in Section 3.

###### Definition 1.4

If $`B[\omega ]^\omega `$ and $`X`$ is a topological group, let $`𝒞_B^X`$ be the set of all $`xX`$ such that $`x^n:nB`$ converges to $`1`$. If $`[\omega ]^\omega `$ is a filter, let $`𝒟_{}^X=\{𝒞_B^X:B\}`$.

So, by Proposition 1.2, for $`X`$ compact metric, one can choose $``$ so that $`𝒟_{}^X=X`$.

###### Proposition 1.5

Let $`X`$ be any non-trivial compact group. If $`X`$ is totally disconnected, then there is a $`B[\omega ]^\omega `$ such that $`E_n:nBE_0`$ (equivalently, $`𝒞_B^X=X`$). If $`X`$ is not totally disconnected, then $`𝒞_B^XX`$ for all $`B[\omega ]^\omega `$.

Note that if $`X`$ is abelian, then $`𝒞_B^X`$ and $`𝒟_{}^X`$ are subgroups of $`X`$, but this is not true in general for non-abelian groups. The notion of $`𝒞_B^X`$ was considered in , where they show that on the circle, $`𝒞_B^𝕋`$ is a Haar nullset. It is shown in  that assuming Martin’s Axiom, there is a Haar null subgroup $`N`$ of $`𝕋`$ which is not contained in any $`𝒞_B^𝕋`$, and they raise the question of whether Martin’s Axiom was needed. Here, we produce such an $`N`$ without the use of any set-theoretic axioms. More generally,

###### Theorem 1.6

Let $`[\omega ]^\omega `$ be the filter generated by all sets of the form $`\{k!+1:kD\}`$, where $`D\omega `$ has asymptotic density $`1`$. Then:

* $``$ is a Borel subset of $`𝒫(\omega )2^\omega `$.
* Whenever $`X`$ is an infinite compact group:

+ $`𝒟_{}^X`$ is a Haar nullset.
    + If $`X`$ is not totally disconnected, then $`𝒟_{}^X`$ is not a subset of $`𝒞_B^X`$ for any infinite $`B`$.

In particular, when $`X`$ is abelian and not totally disconnected, $`𝒟_{}^X`$ is a Haar null subgroup not contained in any $`𝒞_B^X`$. Of course, (1) of this theorem is obvious; (2) is proved in Section 3. Observe, by Proposition 1.5, that we cannot delete the “not totally disconnected” in (2b).

Section 3 contains some further information about properties of the $`𝒞_B^X`$. In many cases, these properties for an arbitrary $`X`$ can be inferred directly from the special cases $`X=𝕋`$ or $`X=𝕋^\omega `$.

We conclude this Introduction with some easy remarks which will simplify the notation in the next two sections.

###### Definition 1.7

If $`[\omega ]^\omega `$ is a filter, then $`\stackrel{~}{}`$ is the filter generated by $``$ and all cofinite sets. If $`B[\omega ]^\omega `$ then $`B`$ is the filter $`\{W\omega :BW\}`$.

So, $`\stackrel{~}{B}=\{W\omega :B^{}W\}`$. Since the convergence of a sequence $`f_n(x):nB`$ does not change if one modifies $`B`$ on a finite set, we can always deal with $`\stackrel{~}{}`$ rather than $``$. In particular,

###### Lemma 1.8

$`𝒟_{}^X=𝒟_\stackrel{~}{}^X`$.

Any general lemma about all $`𝒟_{}^X`$ will also apply to the $`𝒞_B^X`$ by:

###### Lemma 1.9

$`𝒞_B^X=𝒟_B^X`$.

## 2 Filters

Proof of Proposition 1.2. Let $`d`$ be a metric on $`Y`$. Choose (inductively) disjoint finite $`S_k\omega `$ such that $`S_0=\mathrm{}`$, and for $`k1`$,

$$x_0,\mathrm{}x_{k1}XnS_ki<k[d(f_n(x_i),g(x_i))<1/k].$$
$`(\text{})`$
To see that such a finite $`S_k`$ exists, let $`A_k=\omega _{j<k}S_j`$, and let

$$U_{n,k}=\{(x_0,\mathrm{}x_{k1})X^k:i<k[d(f_n(x_i),g(x_i))<1/k]\}.$$

Then $`\{U_{n,k}:nA_k\}=X^k`$ (since $`g`$ is a limit point of $`f_n:nA_k`$), so by compactness of $`X^k`$, there is a finite $`S_kA_k`$ such that $`\{U_{n,k}:nS_k\}=X^k`$.

For $`xX`$, let $`B_x=_{k\omega }\{nS_k:d(f_n(x),g(x))<1/k\}`$. Applying (), each $`B_{x_0}\mathrm{}B_{x_\mathrm{}1}`$ meets $`S_k`$ for all $`k\mathrm{}`$, and is hence an infinite set. Let $``$ be the filter generated by $`\{B_x:xX\}`$. This $``$ satisfies the theorem because $`f_n(x):nB_xg(x)`$.

To see that $``$ is an $`F_\sigma `$ in $`𝒫(\omega )2^\omega `$, let

$$𝒢_{\mathrm{}}=\{D𝒫(\omega ):x_0,\mathrm{}x_\mathrm{}1X[B_{x_0}\mathrm{}B_{x_\mathrm{}1}D]\}.$$

Then $`=_{\mathrm{}}𝒢_{\mathrm{}}`$. To prove that $`𝒢_{\mathrm{}}`$ is closed, let

$$\begin{array}{c}_{\mathrm{}}=\{(x_0,\mathrm{},x_\mathrm{}1,D)X^{\mathrm{}}\times 𝒫(\omega ):k\omega nS_k\hfill \\ [d(f_n(x_0),g(x_0))<1/k\mathrm{}d(f_n(x_\mathrm{}1),g(x_\mathrm{}1))<1/knD]\}.\hfill \end{array}$$

Then $`_{\mathrm{}}`$ is closed in $`X^{\mathrm{}}\times 𝒫(\omega )`$. Hence, $`𝒢_{\mathrm{}}𝒫(\omega )`$, which is the projection of $`_{\mathrm{}}`$, is closed as well.

The idea for obtaining the $`S_k`$ above is taken from the proof that $`C_p(X,Y)`$ has countable tightness (Kelley and Namioka , Lemma 8.19; see also , Ch. II§1). We do not have, in general, a simpler description of a filter $``$ satisfying Proposition 1.2, although $``$ is related to the neighborhood filter in $`C_p(X,Y)`$:

###### Definition 2.1

If $`g`$ is a limit point of $`f_n:n\omega `$ in $`C_p(X,Y)`$, then the *induced neighborhood filter* is the filter generated by all subsets of $`\omega `$ of the form $`\{n\omega :f_nU\}`$, where $`U`$ is a neighborhood of $`g`$ in $`C_p(X,Y)`$.

###### Lemma 2.2

If $``$ is as in Proposition 1.2 and $`𝒩`$ is the induced neighborhood filter, then $`𝒩\stackrel{~}{}`$ (see Definition 1.7).

Proof. $`𝒩`$ is generated by sets of the form $`A=\{n\omega :f_n(x)V\}`$, where $`xX`$ and $`V`$ is a neighborhood of $`g(x)`$ in $`Y`$. Fix $`B`$ with $`f_n(x):nB`$ converging to $`g(x)`$. Then $`B^{}A`$, so $`A\stackrel{~}{}`$.

In some cases, one can simply take $`=𝒩`$ in Proposition 1.2; for example, this will always work if $`Y`$ is finite. However, this does not work in the case $`X=Y=𝕋`$, with $`f_n=E_n`$ and $`g=E_0`$; see Proposition 4.3.

## 3 Groups

We consider now in more detail the $`𝒞_B^X`$ and $`𝒟_{}^X`$ for compact groups $`X`$. Basic facts about such groups can be found in Hofmann and Morris .

###### Lemma 3.1

If $`X`$ is a totally disconnected compact group, and $`B=\{k!:k\omega \}`$, then $`𝒞_B^X=X`$.

Proof. Such an $`X`$ is an inverse limit of finite groups, or, equivalently, a closed subgroup of a product of finite groups (see , Theorem 1.34).

For the other groups, we shall show that $`𝒞_B^X`$ is never all of $`X`$, while $`𝒟_{}^X`$ may or may not be all of $`X`$, depending on $``$ and $`X`$.

First, consider the “all of $`X`$” case:

###### Lemma 3.2

If $`𝒟_{}^{𝕋^\omega }=𝕋^\omega `$ then $`𝒟_{}^X=X`$ for all compact metric groups $`X`$.

Proof. Since each $`xX`$ generates a compact abelian subgroup, we may assume that $`X`$ is abelian, in which case $`X`$ is continuously isomorphic to a subgroup of $`𝕋^\omega `$.

For the “not all of $`X`$” case, there is a similar reduction to $`𝕋`$. We shall show:

###### Theorem 3.3

Let $`[\omega ]^\omega `$ be a filter which is analytic as a subset of $`𝒫(\omega )2^\omega `$, and assume that $`𝒟_{}^𝕋𝕋`$. Let $`X`$ be any infinite compact group. Then $`𝒟_{}^X`$ is a Haar nullset if at least one of the following holds:

* $`X`$ is abelian and not totally disconnected.
* $`X`$ is connected.
* For all $`B`$ and all $`m2`$: $`\{nB:mn\}`$ is infinite.

Furthermore, $`𝒟_{}^XX`$ whenever $`X`$ is not totally disconnected.

This in particular applies when $`=B`$ (see Definition 1.7 and Lemmas 1.8 and 1.9). Observe that

###### Lemma 3.4

$`𝒞_B^𝕋`$ is a Haar nullset for all infinite $`B\omega `$.

See  (Lemma 3.10) and  §4 for proofs; yet another proof is given below; see Remark 3.14. It follows that $`𝒞_B^X`$ is a Haar nullset if one of (1,2,3) from Theorem 3.3 hold. Note that these conditions cannot be dropped. For example, let $`X=O(2)`$. Then the component of $`1`$ in $`X`$ is $`SO(2)𝕋`$, which has index $`2`$ in $`X`$. The two cosets of $`SO(2)`$ are $`SO(2)`$ and $`R`$ (the reflections), and the elements of $`R`$ all have order $`2`$. Now, let $`B=\{2k:k\omega \}`$. Then $`𝒞_B^X`$ contains all of $`R`$ plus two elements of $`SO(2)`$, so $`𝒞_B^X`$ has Haar measure $`1/2`$.

Observe that if $``$ is analytic, then $`𝒟_{}^𝕋`$ is also analytic, and hence Haar measurable. Since a measurable subgroup is null iff it has infinite index, the following lemma, called the Steinhaus-Weil Theorem in , is relevant:

###### Lemma 3.5

If $`X`$ is a compact group and $`H`$ is a measurable subgroup of finite index, then $`H`$ is clopen in $`X`$.

Proof. $`H`$ has positive measure, so $`H^1H=H`$ has non-empty interior (see , Cor. 20.17), so $`H`$ is open (since it is a group), and hence clopen (since it has finite index).

In particular, if $`X`$ is connected, then either $`H`$ is null or $`H=X`$. Thus,

###### Corollary 3.6

If $``$ is an analytic filter and $`𝒟_{}^𝕋𝕋`$, then $`𝒟_{}^𝕋`$ is a Haar nullset.

Now, to prove Theorem 3.3, we prove a few lemmas which reduce the situation for a general $`X`$ to the case $`X=𝕋`$.

###### Lemma 3.7

If $`X,Y`$ are compact groups and $`\phi `$ is a continuous homomorphism from $`X`$ onto $`Y`$, then $`𝒟_{}^X\phi ^1(𝒟_{}^Y)`$.

Since $`\phi ^1`$ also preserves Haar measure, one may prove that $`𝒟_{}^X`$ is null by proving that $`𝒟_{}^Y`$ is null. Using this remark, the reduction for abelian groups is easy:

###### Lemma 3.8

If $`𝒟_{}^𝕋`$ is null, then $`𝒟_{}^X`$ is null for all infinite compact abelian groups $`X`$ which are not totally disconnected.

Proof. $`X`$ is continuously isomorphic to some compact subgroup of $`𝕋^\theta `$ for some cardinal $`\theta `$, so we may assume that $`X𝕋^\theta `$. If $`\pi _\alpha `$ is the projection onto the $`\alpha ^{\mathrm{th}}`$ coordinate, then $`\pi _\alpha (X)`$ is a compact subgroup of $`𝕋`$. Some of the $`\pi _\alpha (X)`$ may be finite, but they cannot all be finite unless $`X`$ is totally disconnected. However, if $`\pi _\alpha (X)`$ is infinite, then it must be all of $`𝕋`$, so the lemma follows by using Lemma 3.7 with $`\phi =\pi _\alpha `$.

To handle non-abelian $`X`$, we shall replace $`𝕋`$ in the above proof by a compact Lie group. For this paper, we can take as a definition that $`X`$ is a compact Lie group iff $`X`$ is continuously isomorphic to a compact subgroup of the unitary group $`U(n)`$ for some finite $`n`$. Many other equivalents are known; see . Observe that all finite groups are compact Lie groups by this definition.

###### Lemma 3.9

If $`X`$ is a compact group and is not totally disconnected, then there is a continuous homomorphism $`\pi `$ from $`X`$ onto some infinite compact Lie group $`Y`$.

Proof. By standard representation theory (see ), we may assume that $`X_{\alpha <\theta }U(n_\alpha )`$. Then each $`\pi _\alpha (X)`$ is a Lie group, and at least one of the $`\pi _\alpha (X)`$ is infinite.

Then, as in Lemma 3.8, if $`𝒟_{}^Y`$ is null then $`𝒟_{}^X`$ is null. However, in proving that $`𝒟_{}^Y`$ is null from the assumption that $`𝒟_{}^𝕋`$ is null, we cannot use a similar argument, since $`Y`$ need need not have a homomorphism onto $`𝕋`$. Rather, we use the fact that $`𝕋`$ is contained in $`Y`$. We shall apply Lemma 3.8 to the maximal tori in $`Y`$ (i.e., maximal connected abelian subgroups; see ) to get:

###### Lemma 3.10

If $`𝒟_{}^𝕋`$ is null, then $`𝒟_{}^Y`$ is null for all non-trivial connected compact Lie groups $`Y`$.

Proof. Let $`f`$ be the characteristic function of $`𝒟_{}^Y`$; so we wish to show that $`_Yf𝑑\lambda _Y=0`$, where $`\lambda _Y`$ is normalized Haar measure on $`Y`$. By Lemma 3.8, we know that $`_Hf𝑑\lambda _H=0`$ whenever $`H`$ is a maximal torus in $`Y`$. Now, observe that $`f`$ is a class function; that is $`f(x^1yx)=f(y)`$. It follows that we may integrate $`f`$ by the Weyl Integration Formula (see , eqn. (8.62)):

$$_Yf(x)𝑑\lambda _Y(x)=\frac{1}{|W|}_Hf(t)|D(t)|^2𝑑\lambda _H(t),$$

where $`W`$ is a finite group and $`D(t)`$ is a finite function of $`t`$. Since $`f(t)=0`$ for $`\lambda _H`$ – almost every $`t`$, we get $`_Yf𝑑\lambda _Y=0`$.

Note that $`Y`$ must be connected for this Weyl Integration Formula to be true, since all the maximal tori are contained in the identity component of $`Y`$; furthermore, as pointed out above, the lemma fails for $`Y=O(2)`$.

Proof of Theorem 3.3. $`𝒟_{}^𝕋`$ is null by Corollary 3.6. Now, assume that $`X`$ is not totally disconnected. Then $`𝒟_{}^X`$ is null if $`X`$ is abelian by Lemma 3.8, so we must handle the non-abelian case. If $`X`$ is connected, then we can map $`X`$ onto a non-trivial connected compact Lie group $`Y`$ by Lemma 3.9, so that $`𝒟_{}^X`$ is null by Lemmas 3.10 and 3.7. If $`X`$ is not connected, then $`Y`$ may fail to be connected. If $`Y_0`$ is the identity component of $`Y`$, then $`𝒟_{}^YY_0=𝒟_{}^{Y_0}`$ will still be null by Lemma 3.10, which proves that $`𝒟_{}^XX`$.

Now, we must prove that $`𝒟_{}^X`$ is null in Case (3) of Theorem 3.3. First, assume that $`X`$ is not totally disconnected. Using the same $`Y`$, it is again sufficient to show that $`𝒟_{}^Y`$ is null. But we already know that $`𝒟_{}^YY_0`$ is null, and in Case (3), $`𝒟_{}^YY_0`$. To see this, fix $`yY\backslash Y_0`$, and let $`m`$ be the order of $`[y]`$ in the quotient $`Y/Y_0`$. If $`y`$ were in $`𝒟_{}^Y`$, we could fix $`B`$ such that $`y^n:nB1`$. But $`C=\{nB:mn\}`$ is infinite, $`Y_0`$ is a neighborhood of $`1`$, and $`y^nY_0`$ for all $`nC`$, a contradiction.

Finally if $`X`$ is totally disconnected in Case (3), then $`𝒟_{}^X=\{1\}`$, as in the proof of Lemma 3.1.

Proof of Proposition 1.5. By Lemmas 3.1 and 3.4 and Theorem 3.3.

We now give some examples of $``$ for which $`𝒟_{}^𝕋`$ is null, using some well-known facts about Hadamard sets and the Bohr topology on $``$; see  for definitions and references to the earlier literature.

###### Definition 3.11

$`𝒩_𝖻𝒫(\omega )`$ denotes the neighborhood filter at $`0`$ in the topology $`\omega `$ inherits as a subset of $`^\mathrm{\#}`$ (that is, the group $``$ with its Bohr topology).

Equivalently, $`𝒩_𝖻`$ is the induced neighborhood filter in the sense of Definition 2.1, taking $`f_n=E_n`$ and $`g=E_0`$. Applying Lemma 2.2,

###### Lemma 3.12

If $`𝒟_{}^𝕋=𝕋`$ then $`𝒩_𝖻\stackrel{~}{}`$.

Note that $`𝒟_{𝒩_𝖻}^𝕋`$ is countable (see Proposition 4.3), so to get $`𝒟_{}^𝕋=𝕋`$ (as in Proposition 1.2), $``$ must properly extend $`𝒩_𝖻`$.

Applying Lemma 3.12 and Corollary 3.6,

###### Lemma 3.13

Suppose that $`𝒩_𝖻\stackrel{~}{}`$, and $``$ is analytic as a subset of $`𝒫(\omega )2^\omega `$. Then $`𝒟_{}^𝕋`$ is a Haar nullset.

###### Remark 3.14

This yields another proof of Lemma 3.4: apply Lemma 3.13 to $`=B`$, and note that $`𝒩_𝖻\stackrel{~}{}`$ because $`^\mathrm{\#}`$ has no convergent $`\omega `$-sequences.

Proof of Theorem 1.6(2a). $`H=\{k!+1:k\omega \}`$ is a Hadamard set, and hence closed and discrete in $`^\mathrm{\#}`$. Thus, $`\omega \backslash H𝒩_𝖻`$, so that $`𝒩_𝖻`$. Then, (2a) follows by Lemma 3.13 and Theorem 3.3 (Case (3)).

Now, let us turn to a proof of Theorem 1.6(2b). Here, most of the work will be done on the solenoid $`\widehat{}`$ (see ):

###### Definition 3.15

$`\widehat{}`$ denotes the dual of the discrete group of rationals. We shall realize $`\widehat{}`$ concretely as $`\{\stackrel{}{z}𝕋^\omega :\alpha <\omega [(z_{\alpha +1})^{\alpha +1}=z_\alpha ]\}`$.

This differs slightly from the notation in . We can identify $`\stackrel{}{z}`$ with the character of $``$ which takes $`1/\alpha !`$ to $`z_\alpha `$.

###### Lemma 3.16

If $`X`$ is an infinite compact group and is not totally disconnected, then $`X`$ has a non-trivial closed subgroup which is a continuous homomorphic image of $`\widehat{}`$.

Proof. First, WLOG, $`X`$ is abelian. To see this, let $`\pi :XY`$ be a continuous homomorphism onto an infinite compact Lie group $`Y`$ (see Lemma 3.9). Choose $`xX`$ such that $`y:=\pi (x)`$ has infinite order. Then $`\pi `$ maps $`\overline{x}`$ onto $`\overline{y}`$, which is an infinite compact Lie group, and hence not totally disconnected. It follows that $`\overline{x}`$ is not totally disconnected (see , Exercise E1.13), so we may replace $`X`$ by $`\overline{x}`$.

Second, WLOG, $`X`$ is connected, since we may replace $`X`$ by the component of $`1`$.

Now, let $`G=\widehat{X}`$, which is a discrete torsion-free abelian group. Let $`VG`$ be the divisible hull of $`G`$; then $`V`$ is torsion-free and divisible, so we may regard $`V`$ as a vector space over $``$. Choose a basis $`\{v_\alpha :\alpha <\kappa \}`$ for $`V`$ with $`v_0G`$, let $`W`$ be the vector subspace generated by $`v_0`$, and let $`\psi `$ be the canonical homomorphism of $`V`$ onto $`W`$. Let $`H=\psi (G)W`$. Then $`H`$ is torsion-free and non-trivial (since $`v_0G`$), and $`\psi \mathrm{}G`$ maps $`G`$ onto $`H`$, so $`Y:=\widehat{H}`$ is a non-trivial closed subgroup of $`X=\widehat{G}`$. But also, $`H`$ is isomorphic to a subgroup of $`W`$, so $`Y`$ is a quotient of $`\widehat{}`$.

###### Definition 3.17

$`A\omega `$ is *thin* iff $`A`$ is infinite and of the form $`\{a_k:k\omega \}`$, where $`0<a_0<a_1<\mathrm{}`$ and $`lim_ka_k/a_{k+1}=0`$.

###### Lemma 3.18

If $`A\omega `$ is thin, $`A`$ is partitioned into disjoint infinite subsets $`B,C`$, and $`\stackrel{}{v},\stackrel{}{w}\widehat{}`$, then there is a $`\stackrel{}{z}\widehat{}`$ such that $`(\stackrel{}{z})^n:nB`$ converges to $`\stackrel{}{v}`$ and $`(\stackrel{}{z})^n:nC`$ converges to $`\stackrel{}{w}`$.

Proof. List $`A`$ in increasing order as $`\{a_j:j\omega \}`$. Let $`\epsilon _j=sup_\mathrm{}j(a_{\mathrm{}}/a_{\mathrm{}+1})`$. Then $`\epsilon _j0`$, each $`a_j/a_{j+1}\epsilon _j`$, and $`\epsilon _j1`$.

Let $`\gamma _0=0`$, and let $`\gamma _{j+1}`$ be the largest integer $`\gamma `$ such that $`\gamma !1/\sqrt{\epsilon _j}`$. Note that $`\gamma _j\mathrm{}`$ and

$$(a_j/a_{j+1})(\gamma _{j+1}!)\epsilon _j(\gamma _{j+1}!)\sqrt{\epsilon _j}.$$

Hence, $`(a_j/a_{j+2})(\gamma _{j+2}!)(a_j/a_{j+1})(\gamma _{j+1}!)(a_{j+1}/a_{j+2})(\gamma _{j+2}!)\sqrt{\epsilon _j\epsilon _{j+1}}`$. Continuing in this way,

$$j<k(a_j/a_k)(\gamma _k!)\sqrt{\epsilon _j\epsilon _{j+1}\mathrm{}\epsilon _{k1}}(\sqrt{\epsilon _j})^{kj}.$$
$`(\text{})`$
Our $`\stackrel{}{z}`$ will be $`lim_j\stackrel{}{z}_j`$, where each $`\stackrel{}{z}_j\widehat{}`$; the $`\stackrel{}{z}_j`$ will be chosen by induction on $`j`$. Use $`z_{j,\alpha }`$ for the components of $`\stackrel{}{z}_j\widehat{}𝕋^\omega `$. Let $`d`$ be the metric on $`𝕋`$ obtained by identifying $`𝕋`$ with $`/`$, so that the circumference of $`𝕋`$ is $`1`$, and $`d(e^{i\theta },1)=|\theta |/2\pi `$ when $`|\theta |\pi `$. We shall get, for each $`k\omega `$:

* For $`\alpha \gamma _k`$: $`(z_{k,\alpha })^{a_k}`$ equals $`v_\alpha `$ when $`a_kB`$ and $`w_\alpha `$ when $`a_kC`$.
* For $`\alpha \gamma _k`$, when $`k>0`$: $`d(z_{k,\alpha },z_{k1,\alpha })(\gamma _k!)/a_k`$.

To ensure (1) for a given $`k`$, it is sufficient to choose $`z_{k,\gamma _k}`$ so that (1) holds for $`\alpha =\gamma _k`$; then (1) will hold for $`\alpha <\gamma _k`$ by our definition (3.15) of $`\widehat{}`$. For $`\alpha >\gamma _k`$, we just choose the $`z_{k,\alpha }`$ so that the point $`\stackrel{}{z}_k`$ lies in $`\widehat{}`$. To get (2) along with (1): We are given $`\stackrel{}{z}_{k1}`$ and we must define $`\stackrel{}{z}_k`$ by choosing $`z_{k,\gamma _k}`$ so that $`(z_{k,\gamma _k})^{a_k}`$ is $`v_{\gamma _k}`$ when $`a_kB`$ and $`w_{\gamma _k}`$ when $`a_kC`$. There are $`a_k`$ possible choices for $`z_{k,\gamma _k}`$, spaced evenly around the circle, at distance $`1/a_k`$ apart, so we can make the choice so that $`d(z_{k,\gamma _k},z_{k1,\gamma _k})1/a_k`$. Then (2) follows by our definition of $`\widehat{}`$.

Now, if we fix $`j`$ and $`\alpha \gamma _j`$, then whenever $`j<k`$, we may apply (2) and () to get $`a_jd(z_{k,\alpha },z_{k1,\alpha })(a_j/a_k)(\gamma _k!)(\sqrt{\epsilon _j})^{kj}`$. Thus, whenever $`\mathrm{}>j`$:

$$a_jd(z_{\mathrm{},\alpha },z_{j,\alpha })\underset{k=j+1}{\overset{\mathrm{}}{}}(\sqrt{\epsilon _j})^{kj}\sqrt{\epsilon _j}/(1\sqrt{\epsilon _j}).$$
$`(\text{})`$
In particular, the sequence $`z_{j,\alpha }:\alpha \omega `$ is Cauchy, so we can define $`\stackrel{}{z}=lim_j\stackrel{}{z}_j`$. Then, () yields $`d(z_\alpha ^{a_j},z_{j,\alpha }^{a_j})\sqrt{\epsilon _j}/(1\sqrt{\epsilon _j})`$ whenever $`\alpha \gamma _j`$. Applying this for the $`a_jB`$, when $`(z_{j,\alpha })^{a_j}=v_\alpha `$, we get $`(\stackrel{}{z})^n:nB\stackrel{}{v}`$, and applying it for the $`a_jC`$ yields $`(\stackrel{}{z})^n:nC\stackrel{}{w}`$.

###### Lemma 3.19

Let $`X`$ be any infinite compact group which is not totally disconnected. Let $`A\omega `$ be thin, with $`A`$ partitioned into disjoint infinite subsets $`B,C`$. Then there are $`x,yX`$ with $`y1`$ such that $`x^n:nB`$ converges to $`y`$ and $`x^n:nC`$ converges to $`1`$.

Proof. By Lemma 3.16, we may assume that there is a continuous homomorphism $`\phi `$ from $`\widehat{}`$ onto $`X`$. Then $`y`$ can be any element of $`X\backslash \{1\}`$. Now, in $`\widehat{}`$, let $`\stackrel{}{w}=1`$, and choose $`\stackrel{}{v}`$ with $`\phi (\stackrel{}{v})=y`$, and apply Lemma 3.18.

Consider this in particular with $`X=𝕋`$, where the argument of Lemma 3.18 can be done (considerably simplified) in $`𝕋`$ directly, and resembles the proof that Hadamard sets are $`I_0`$ sets. However, Lemma 3.19 fails if we only assume that $`A`$ is a Hadamard set. For example, let $`A=\{2^j:j\omega \}`$, $`B=\{2^{2k+1}:k\omega \}`$, and $`C=\{2^{2k}:k\omega \}`$, and consider $`x𝕋`$. If $`x^n:nC1`$, then $`x^{2^j}=1`$ for some $`j`$, but then also $`x^n:nB1`$.

Proof of Theorem 1.6(2b). Fix an infinite $`B`$; we need to show that $`𝒟_{}^X𝒞_B^X`$. Since $`𝒞_B^X`$ gets bigger as $`B`$ gets smaller, we may assume that $`B`$ is small enough so that for some $`C`$: $`CB=\mathrm{}`$ and $`A:=CB`$ is thin. Then, applying Lemma 3.19, we may fix $`xX`$ so that $`x^n:nC`$ converges to $`1`$ and $`x^n:nB`$ converges to $`y1`$. Then $`x𝒟_{}^X`$ and $`x𝒞_B^X`$.

## 4 Examples

First, we point out that Proposition 1.2 can fail if $`X`$ is not assumed to be compact.

###### Example 4.1

Let $`X`$ be the cardinal $`𝔠=2^\mathrm{}_0`$ with the discrete topology, and let $`Y=\omega +1`$ with the order topology. Then in $`C_p(X,Y)`$, there is a sequence $`f_n:n\omega `$ with limit point $`g`$ such that there is no filter $`[\omega ]^\omega `$ satisfying:

$$xXB[f_n(x):nBg(x)]$$
$`(\text{})`$
Proof. Let $`𝒰`$ be a non-principal ultrafilter on $`\omega `$ which is not a P-point. List $`𝒰`$ as $`\{B_\alpha :0<\alpha <𝔠\}`$. Also, partition $`\omega `$ into infinite sets $`A_k`$ (for $`k\omega `$) so that each $`A_k𝒰`$ and each $`B𝒰`$ meets some $`A_k`$ in an infinite set.

Let $`g(\alpha )=\omega `$ for all $`\alpha `$. For $`\alpha >0`$, let $`f_n(\alpha )`$ be $`n`$ for $`nB_\alpha `$ and $`0`$ for $`nB_\alpha `$. Let $`f_n(0)`$ be the $`k`$ such that $`nA_k`$.

Now, suppose that $``$ satisfied (✈). Fix $`C`$ with $`f_n(0):nC\omega `$. Then each $`CA_k`$ is finite, so $`C𝒰`$, so fix $`\alpha >0`$ with $`B_\alpha C=\mathrm{}`$. Now fix $`D`$ with $`f_n(\alpha ):nD\omega `$. Then $`D^{}B_\alpha `$, so $`DC`$ is finite. But $`DC`$, contradicting $`[\omega ]^\omega `$.

This is the simplest possible $`Y`$ for such an example, since Proposition 1.2 does hold for arbitrary $`X`$ whenever $`Y`$ is finite, taking $``$ to be the induced neighborhood filter $`𝒩`$ (as in Definition 2.1). Also, under Martin’s Axiom, Proposition 1.2 holds for all $`X`$ of size less than $`𝔠`$ whenever $`Y`$ is first countable. When $`|X|=𝔠`$, taking $`Y`$ to be first countable is not enough, even when $`X`$ compact, since Proposition 1.2 fails when $`X`$ and $`Y`$ are the lexicographically ordered square. This is the space $`[0,1]\times [0,1]`$, ordered lexicographically, and given the usual order topology; note that it is compact and first countable.

###### Example 4.2

Let $`X`$ be the lexicographically ordered square. Then there is a sequence $`f_n:n\omega `$ in $`C_p(X,X)`$ with limit point $`g`$ such that there is no filter $`[\omega ]^\omega `$ satisfying (✈) above.

Proof. Let $`\stackrel{~}{f_n}:n\omega `$ and $`\stackrel{~}{g}`$ be the functions in $`C_p([0,1]_d,\omega +1)`$ obtained in Example 4.1, identifying the ordinal $`𝔠`$ with the discrete $`[0,1]`$. We shall encode this example into $`C_p(X,X)`$.

To embed $`\omega +1`$ into $`[0,1]`$, let $`u_\omega =1/2`$ and fix $`u_n`$ with $`0<u_n<1/2`$ and $`u_nu_\omega `$. Next, in $`X`$, let $`I_r=\{r\}\times [0,1]`$; then $`I_r`$ is a homeomorphic copy of $`[0,1]`$ for each $`r[0,1]`$. We shall have $`f_n:XX`$ map each $`I_r`$ onto $`I_r`$, and encode the $`\stackrel{~}{f_n}`$ as follows: Let

$$f_n(r,1/2)=(r,u_{\stackrel{~}{f_n}(r)}),f_n(r,0)=(r,0),f_n(r,1)=(r,1).$$

Then, fill in the rest of the values $`f_n(r,y)`$ for $`y[0,1]`$ by linear interpolation, so that the graph of $`f_n`$ intersected with $`I_r\times I_r`$ is the union of two line segments.

Likewise, we get $`g`$ from $`\stackrel{~}{g}`$, but then $`g`$ is the identity function. We can now apply the same argument as in Example 4.1.

We remark on the relevance of P-points here. For arbitrary $`X`$ and first countable $`Y`$, let $`𝒩`$ be the induced neighborhood filter (as in Definition 2.1). If there is a P-point $`𝒰`$ such that $`𝒩`$ is contained in $`𝒰`$, then taking $`=𝒰`$ will work in Proposition 1.2. In Examples 4.1 and 4.2 we have $`𝒩=𝒰`$, but that $`𝒰`$ was chosen to be a non-P-point. For $`X`$ compact and $`Y`$ metric, if we assume Martin’s Axiom, then we can produce a P-point ultrafilter $`𝒩`$ which gives the convergence described in Proposition 1.2, but, of course, this $``$ will not be a Borel set. The proof of Proposition 1.2 amounts to adding to $`𝒩`$ some diagonal intersections of sets in $`𝒩`$, which is part of the P-point construction, although we do not actually build an ultrafilter. Note that $`𝒩`$ itself will not in general satisfy Proposition 1.2; for example, to get a filter $``$ with $`𝒟_{}^𝕋=𝕋`$, we cannot just take $`=𝒩_𝖻`$ (see Definition 3.11):

###### Proposition 4.3

$`𝒟_{𝒩_𝖻}^𝕋`$ is the set of all $`w𝕋`$ of finite order.

Proof. If $`w^k=1`$, then $`w^n=1`$ for all $`n\{jk:j\omega \}𝒩_𝖻`$.

Conversely, suppose $`w`$ has infinite order. Consider $`𝖻=\mathrm{Hom}(𝕋_d,)`$ in the standard way, and let $`\lambda `$ be Haar measure on $`𝖻`$. Observe that if $`B𝒩_𝖻`$, then $`\lambda (\overline{B})>0`$, whereas if $`w^n:nB1`$, then $`\lambda (\overline{B})=0`$.

The following shows that Lemma 3.2 can fail if $`X`$ is non-metrizable:

###### Example 4.4

If $`𝒟_{}^{𝕋^\omega }=𝕋^\omega `$, then $`𝒟_{}^{𝕋^𝔠}`$ has inner Haar measure $`0`$ and outer Haar measure $`1`$.

Proof. Let $`P`$ be the set of all $`f𝕋^𝔠`$ such that $`f`$ maps $`𝔠`$ onto $`𝕋`$, and let $`Q`$ be the set of elements of $`f𝕋^𝔠`$ such that $`\{\alpha :f(\alpha )1\}`$ is countable. Then $`Q`$ has outer measure $`1`$ and $`Q𝒟_{}^{𝕋^𝔠}`$ (since $`𝒟^{𝕋^\omega }=𝕋^\omega )`$, so $`𝒟_{}^{𝕋^𝔠}`$ has outer measure $`1`$. Also, $`P`$ has outer measure $`1`$ and $`𝒟_{}^{𝕋^𝔠}P=\mathrm{}`$ (since $`𝒞_B^𝕋𝕋`$ for all $`B`$), so $`𝒟_{}^{𝕋^𝔠}`$ has inner measure $`0`$.

## 5 Appendix

Here we collect a few proofs which don’t seem worth putting in the published part of the paper, since they just verify some remarks in Section 4, which is itself essentially an appendix.

First, as remarked in Section 4, one can get a P-point in a modified version of Proposition 1.2.

###### Proposition 5.1

Assume Martin’s Axiom (or, just $`𝔭=𝔠`$). Suppose that $`X`$ is compact, $`Y`$ is metric, and $`g`$ is a limit point of $`f_n:n\omega `$ in $`C_p(X,Y)`$. Then there is a P-point ultrafilter $`[\omega ]^\omega `$ such that for each $`xX`$ there is $`B`$ such that $`f_n(x):nBg(x)`$.

Proof. As remarked in Section 4, it is sufficient to construct a P-point extending the induced neighborhood filter, $`𝒩`$. As usual, $``$ will be generated by sets $`\{B_\alpha :\alpha <𝔠\}`$, where $`\alpha <\beta B_\beta ^{}B_\alpha `$. Make sure, inductively, that

$$g\text{ is a limit point of }f_n:nB_\alpha .$$
$`(\text{})`$
To make $``$ an ultrafilter, list $`[\omega ]^\omega `$ as $`\{A_\alpha :\alpha <𝔠\}`$, and let $`B_{\alpha +1}`$ be either $`B_\alpha A_\alpha `$ or $`B_\alpha \backslash A_\alpha `$, preserving (), which ensures in particular that if $`A_\alpha 𝒩`$ then $`A_\alpha `$.

Now, say $`\gamma `$ is a limit. We need $`B_\gamma ^{}B_\alpha `$ for all $`\alpha <\gamma `$, along with (❦) for $`B_\gamma `$. When $`1k<\omega `$, let $`\mathrm{SS}_k`$ be the set of all finite $`S\omega `$ such that $`\mathrm{min}(S)>k`$ and

$$x_0,\mathrm{}x_{k1}XnSi<k[d(f_n(x_i),g(x_i))<1/k].$$
$`(\text{})`$
Observe that (❦) will hold if for all $`k`$, there is an $`SB_\gamma `$ with $`S\mathrm{SS}_k`$. As in the proof of Proposition 1.2, there is an $`SB_{\alpha _1}\mathrm{}B_\alpha _{\mathrm{}}`$ with $`S\mathrm{SS}_k`$ whenever $`\alpha _1<\mathrm{}<\alpha _{\mathrm{}}<\gamma `$. Now, we obtain an appropriate $`B_\gamma `$ by a standard application of Martin’s Axiom.

Next, the argument in Proposition 4.3 requires:

###### Lemma 5.2

Consider $`\omega 𝖻=\mathrm{Hom}(𝕋_d,)`$, and let $`\lambda `$ be Haar measure on $`𝖻`$. Fix $`B\omega `$. Then:

* If $`B𝒩_𝖻`$, then $`\lambda (\overline{B})>0`$.
* If $`w𝕋`$ has infinite order then $`w^n:nB1`$, then $`\lambda (\overline{B})=0`$.

Proof. For (1): We can assume that $`B=U_1\mathrm{}U_k\omega `$, where $`U_j=\{n:d(z_j^n,1)<\epsilon \}`$. Then $`BB=U_1\mathrm{}U_k`$, a basic open set in $`^\mathrm{\#}`$. So, $`\lambda (\overline{BB})>0`$ because $``$ is dense in $`𝖻`$. But $`\overline{BB}=\overline{B}\overline{B}`$ and $`\lambda (\overline{B})=\lambda (\overline{B})`$, so $`\lambda (\overline{B})>0`$.

For (2): Fix $`\epsilon >0`$; we show that $`\lambda (\overline{B})2\epsilon `$. Let $`d`$ be the metric on $`𝕋`$ used in the proof of Lemma 3.18. By removing finitely many elements of $`B`$, we can assume, WLOG, that $`d(w^n,1)\epsilon `$ for all $`nB`$. Identifying $`𝖻`$ with $`\mathrm{Hom}(𝕋_d,)`$, let $`F=\{\phi 𝖻:d(\phi (w),1)\epsilon \}`$. Applying duality (viewing $`𝕋_d`$ as $`\widehat{𝖻}`$), let $`E_w(\phi )=\phi (w)`$; then $`E_w`$ is a continuous homomorphism from $`𝖻`$ into $`𝕋`$, and $`F=\{\phi 𝖻:d(E_w(\phi ),1)\epsilon \}`$, which is a closed subset of $`𝖻`$. Also, identifying $`\omega 𝖻`$, we have $`BF`$, so it is sufficient to show that $`\lambda (F)=2\epsilon `$ if $`\epsilon 1/2`$. But this follows from the fact that $`E_w`$ maps onto $`𝕋`$ (since $`w`$ has infinite order), since $`\{x𝕋:d(x,1)\epsilon \}`$ has measure $`2\epsilon `$.