format: text

text:
# Continuous Hahn functions as Clebsch-Gordan coefficients

## 1. Introduction

The results and techniques in this paper are mainly analytic in nature, but they are motivated by a Lie algebraic problem. As is well known, many polynomials in the Askey-scheme of orthogonal polynomials of hypergeometric type, see , have an interpretation in the representation theory of Lie groups and Lie algebras, see e.g. Vilenkin and Klimyk  and Koornwinder . The Askey-scheme can be extended to families of unitary integral transforms with a hypergeometric kernel. Many of these kernels also admit group theoretic interpretations. For example the Jacobi functions, which can be considered as a non-polynomial extension of the Jacobi polynomials and are given explicitly by a certain $`{}_{2}{}^{}F_{1}^{}`$-hypergeometric function, have an interpretation as matrix elements for irreducible representations of the Lie group $`SU(1,1)`$. The Jacobi function is the kernel in the Jacobi integral transform, which can be found by spectral analysis of the hypergeometric differential operator. For an overview of Jacobi functions in representation theory, we refer to the survey paper  by Koornwinder.

In this paper we give a generalization of the Jacobi functions. We consider the tensor product of a positive and a negative discrete series representation of the Lie algebra $`𝔰𝔲(1,1)`$. The Clebsch-Gordan coefficients for the hyperbolic basisvectors turn out to be a certain type of non-polynomial $`{}_{3}{}^{}F_{2}^{}`$-hypergeometric functions, which we call continuous Hahn functions. We show that the continuous Hahn functions are the kernel in an integral transform, that generalizes the Jacobi function transform. We emphasize that the main part (sections 3 and 5) of this paper is analytic in nature, and that the Lie algebraic interpretation is mainly restricted to section 4.

The Lie algebra $`𝔰𝔲(1,1)`$ is generated by the three elements $`H`$, $`B`$ and $`C`$. There are four classes of irreducible unitary representations for $`𝔰𝔲(1,1)`$: discrete series, i.e. the positive and the negative discrete series representations, and continuous series, i.e. the principal unitary series and the complementary series representations. There are three kinds of basis elements on which the various representations can act: the elliptic, the parabolic and the hyperbolic basis elements. These three elements are related to conjugacy classes of the group $`SU(1,1)`$. We consider the tensor product of a positive and a negative discrete series representation, which decomposes into a direct integral over the principal unitary series representations. Under certain condition discrete terms can appear. The Clebsch-Gordan coefficients for the standard (elliptic) basis vectors are continuous dual Hahn polynomials. We compute the Clebsch-Gordan coefficients for the hyperbolic basis vectors, which are non-polynomial extensions of the continuous (dual) Hahn polynomials, and are therefore called continuous Hahn functions. For the Clebsch-Gordan coefficients for the elliptic and parabolic basis, we refer to , respectively , .

The explicit expressions for the Clebsch-Gordan coefficients as $`{}_{3}{}^{}F_{2}^{}`$-series are not new, they are found by Mukunda and Radhakrishnan in . However not much seems to be known about the generalized orthogonality properties of the continuous Hahn functions, i.e. they form the kernel in a unitary integral tranform (the continuous Hahn transform). Using the Lie algebraic interpretation of the continuous Hahn functions, we can compute formally the inverse of the continuous Hahn integral transform. In section 5 we give an analytic proof for the integral transform pair.

The method we use to compute the Clebsch-Gordan coefficients is based on an idea by Granovskii and Zhedanov . The idea is to consider a self-adjoint Lie algebra element $`X_a=aH+BC`$, $`a`$. The action of $`X_a`$ in an irreducible representation gives a difference equation, for which the (generalized) eigenvectors can be expressed in terms of special functions and the standard basis vectors. The Clebsch-Gordan coefficients for the eigenvectors can be calculated using properties of the special functions. In  and  Van der Jeugt and the second author considered the action of $`X_a`$ in tensor products of positive discrete series representations of $`𝔰𝔲(1,1)`$ to find convolution formulas for orthogonal polynomials. In  the action of $`X_a`$ in the tensor product of a positive and a negative discrete series representation is investigated for $`|a|>1`$ (the elliptic case). This leads to a bilinear summation formula for Meixner polynomials \[10, Thm.3.6\]. In this paper we consider the case $`|a|<1`$ (the hyperbolic case).

The plan of the paper is as follows. In section 2 we introduce the special functions we need in this paper, and give some properties of these functions.

In section 3 we prove a bilinear summation formula for Meixner-Pollaczek polynomials by series manipulations. As a result we find a certain type of $`{}_{3}{}^{}F_{2}^{}`$-functions, which are the continuous Hahn functions. The summation formula is used in section 4.2 to compute the Clebsch-Gordan coefficients for the hyperbolic bases.

In section 4 we consider the tensor product of a positive and a negative discrete series representation of the Lie algebra $`𝔰𝔲(1,1)`$. First we recall the basic properties of $`𝔰𝔲(1,1)`$ and its irreducible unitary representations in section 4.1. Then in section 4.2 we diagonalize $`X_a`$, $`|a|<1`$, in the various irreducible representations. This leads to generalized eigenvectors of $`X_a`$, which can be considered as hyperbolic basis vectors.

For the discrete series representations, the overlap coefficients for the eigenvectors and the standard (elliptic) basisvectors are Meixner-Pollaczek polynomials, cf. \[16, §7\]. For the continuous series, the overlap coefficients are Meixner-Pollaczek functions. This follows from the spectral analysis of a doubly infinite Jacobi operator, which is carried out by Masson and Repka \[18, §3.3\] and Koelink \[14, §4.4.11\]. It turns out that the spectral projection of the Jacobi operator is on a $`2`$-dimensional space of generalized eigenvectors. So the eigenvectors of $`X_a`$, $`|a|<1`$, in the continuous series representations are $`2`$-dimensional, and we find two linearly independent Meixner-Pollaczek functions as overlap coefficients. To determine the Clebsch-Gordan coefficients for the hyperbolic bases, we use the bilinear summation formula from section 3. This leads to a pair of continuous Hahn functions as Clebsch-Gordan coefficients. By formal calculations we find an integral transform pair, with a pair of continuous Hahn functions as a kernel. To give a rigorous proof of the integral transform pair, we show that the continuous Hahn functions are eigenfunctions of a difference operator $`\mathrm{\Lambda }`$. To find this operator $`\mathrm{\Lambda }`$ we realize $`H`$, $`B`$ and $`C`$ as difference operators acting on polynomials, using the difference equation for the Meixner-Pollaczek polynomials. Then $`\mathrm{\Lambda }`$ is a restriction of the Casimir operator in the tensor product.

The spectral analyis of this difference operator is carried out in section 5. A main problem with spectral analysis of a difference operator is finding the right eigenfunctions. This is because an eigenfunction multiplied by a periodic function is again an eigenfunction. Our choice of the periodic function is mainly motivated by the Lie algebraic interpretation of the eigenfunctions. Using asymptotic methods, we find a spectral measure for the difference operator. This leads to an integral transform with a pair of continuous Hahn functions as a kernel. We call this the continuous Hahn integral transform.

*Notations.* We denote for a function $`f:`$

$$f^{}(x)=\overline{f(\overline{x})}.$$

If $`d\mu (x)`$ is a positive measure, we use the notation $`d\mu ^{\frac{1}{2}}(x)`$ for the positive measure with the property

$$d(\mu ^{\frac{1}{2}}\times \mu ^{\frac{1}{2}})(x,x)=d\mu (x).$$

The hypergeometric series is defined by

$${}_{p}{}^{}F_{q}^{}(.\genfrac{}{}{0.0pt}{}{a_1,\mathrm{},a_p}{b_1,\mathrm{},b_q}.;z)=\underset{n=0}{\overset{\mathrm{}}{}}\frac{(a_1)_n\mathrm{}(a_p)_n}{(b_1)_n\mathrm{}(b_q)_n}\frac{z^n}{n!},$$

where $`(a)_n`$ denotes the Pochhammer symbol, defined by

$$(a)_n=\frac{\mathrm{\Gamma }(a+n)}{\mathrm{\Gamma }(a)}=a(a+1)(a+2)\mathrm{}(a+n1),n_0.$$

*Acknowledgements.* We thank Ben de Pagter for useful discussions.

*Dedication.* We gladly dedicate this paper to Mizan Rahman who, with his unsurpassed mastery in dealing with ($`q`$-)series and his insight in the structures of formulas, has pushed the subject of ($`q`$)-special functions much further. We are also grateful to Mizan Rahman for his interest in our work, and for his willingness to help others in solving problems in this field.

## 2. Orthogonal polynomials and functions

In this section we recall some properties of the orthogonal polynomials and functions which we need in this paper.

Continuous dual Hahn polynomials. The Wilson polynomials, see Wilson  or \[1, §3.8\], are $`{}_{4}{}^{}F_{3}^{}`$-hypergeometric polynomials on top of the Askey-scheme of hypergeometric polynomials, see Koekoek and Swarttouw . The continuous dual Hahn polynomials are a three-parameter subclass of the Wilson polynomials, and are defined by

$$s_n(y;a,b,c)=(a+b)_n(a+c)_n{}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{n,a+ix,aix}{a+b,a+c}.;1),x^2=y,(n_0).$$
(2.1)
For real parameters $`a`$, $`b`$, $`c`$, with $`a+b`$, $`a+c`$, $`b+c`$ positive, the continuous dual Hahn polynomials are orthogonal with respect to a positive measure, supported on a subset of $``$. The orthonormal continuous dual Hahn polynomials are defined by

$$S_n(y;a,b,c)=\frac{(1)^ns_n(y;a,b,c)}{\sqrt{n!(a+b)_n(a+c)_n(b+c)_n}}.$$

By Kummer’s transformation, see e.g. \[1, Cor. 3.3.5\], the polynomials $`s_n`$ and $`S_n`$ are symmetric in $`a`$, $`b`$ and $`c`$. Without loss of generality we assume that $`a`$ is the smallest of the real parameters $`a`$, $`b`$ and $`c`$. Let $`d\mu (;a,b,c)`$ be the measure defined by

$`{\displaystyle _{}}f(y)d`$ $`\mu (y;a,b,c)={\displaystyle \frac{1}{2\pi }}{\displaystyle _0^{\mathrm{}}}\left|{\displaystyle \frac{\mathrm{\Gamma }(a+ix)\mathrm{\Gamma }(b+ix)\mathrm{\Gamma }(c+ix)}{\mathrm{\Gamma }(2ix)}}\right|^2{\displaystyle \frac{f(x^2)}{\mathrm{\Gamma }(a+b)\mathrm{\Gamma }(a+c)\mathrm{\Gamma }(b+c)}}𝑑x`$
$`+`$ $`{\displaystyle \frac{\mathrm{\Gamma }(ba)\mathrm{\Gamma }(ca)}{\mathrm{\Gamma }(2a)\mathrm{\Gamma }(b+c)}}{\displaystyle \underset{k=0}{\overset{K}{}}}(1)^k{\displaystyle \frac{(2a)_k(a+1)_k(a+b)_k(a+c)_k}{(a)_k(ab+1)_k(ac+1)_kk!}}f((a+k)^2),`$
where $`K`$ is the largest non-negative integer such that $`a+K<0`$. In particular, the measure $`d\mu (;a,b,c)`$ is absolutely continuous if $`a0`$. The measure is positive under the conditions $`a+b>0`$, $`a+c>0`$ and $`b+c>0`$. Then the polynomials $`S_n(y;a,b,c)`$ are orthonormal with respect to the measure $`d\mu (y;a,b,c)`$.

Meixner-Pollaczek polynomials. The Meixner-Pollaczek polynomials, see , \[1, Ex.6.37\], are a two-parameter subclass of the Wilson polynomials, and are defined by

$$p_n^{(\lambda )}(x;\varphi )=\frac{(2\lambda )_n}{n!}e_2^{in\varphi }F_1(.\genfrac{}{}{0.0pt}{}{n,\lambda +ix}{2\lambda }.;1e^{2i\varphi }),(n_0).$$
(2.2)
For $`\lambda >0`$ and $`0<\varphi <\pi `$, these are orthogonal polynomials with respect to a positive measure on $``$. The orthonormal Meixner-Pollaczek polynomials

$$P_n(x)=P_n^{(\lambda )}(x;\varphi )=\sqrt{\frac{n!}{(2\lambda )_n}}p_n^{(\lambda )}(x;\varphi )$$

satisfy the following three-term recurrence relation

$$2x\mathrm{sin}\varphi P_n(x)=\alpha _nP_{n+1}(x)2(n+\lambda )\mathrm{cos}\varphi P_n(x)+\alpha _{n1}P_{n1}(x),$$
(2.3)
where

$$\alpha _n=\sqrt{(n+1)(n+2\lambda )}.$$

The Meixner-Pollaczek polynomials also satisfy the difference equation

$$\begin{array}{cc}\hfill e^{i\varphi }(\lambda ix)y(x+i)+2i[x\mathrm{cos}\varphi (n+\lambda )\mathrm{sin}\varphi ]y(x)& e^{i\varphi }(\lambda +ix)y(xi)=0,\hfill \\ \hfill y(x)& =P_n^{(\lambda )}(x;\varphi ).\hfill \end{array}$$
(2.4)
Define

$$w^{(\lambda )}(x;\varphi )=\frac{1}{2\pi }(2\mathrm{sin}\varphi )^{2\lambda }e^{(2\varphi \pi )x}\frac{|\mathrm{\Gamma }(\lambda +ix)|^2}{\mathrm{\Gamma }(2\lambda )},$$

then the orthonormality relation reads

$$_{}P_m^{(\lambda )}(x;\varphi )P_n^{(\lambda )}(x;\varphi )w^{(\lambda )}(x;\varphi )𝑑x=\delta _{mn}.$$

Meixner-Pollaczek functions. The Meixner-Pollaczek functions $`u_n`$, $`n`$, can be considered as non-polynomial extensions of the Meixner-Pollaczek polynomials, see Masson and Repka  and Koelink \[14, §4.4\]. The Meixner-Pollaczek functions are defined by

$$\begin{array}{cc}\hfill u_n(x;\lambda ,\epsilon ,\varphi )=(2i\mathrm{sin}\varphi )^n& \frac{\sqrt{\mathrm{\Gamma }(n+1+\epsilon +\lambda )\mathrm{\Gamma }(n+\epsilon \lambda )}}{\mathrm{\Gamma }(n+1+\epsilon ix)}\hfill \\ \hfill \times & {}_{2}{}^{}F_{1}^{}(.\genfrac{}{}{0.0pt}{}{n+1+\epsilon +\lambda ,n+\epsilon \lambda }{n+1+\epsilon ix}.;\frac{1}{1e^{2i\varphi }}),(n).\hfill \end{array}$$
(2.5)
The parameters $`\varphi `$ and $`\epsilon `$ satisfy the conditions $`0<\varphi <\pi `$, $`0\epsilon <1`$, and $`\lambda `$ satisfies one of the following conditions: $`\frac{1}{2}<\lambda <\epsilon `$, $`\frac{1}{2}<\lambda <\epsilon 1`$, or $`\lambda =\frac{1}{2}+i\rho `$, $`\rho `$. In the last case $`u_n`$ is symmetric in $`\rho `$ and $`\rho `$, so without loss of generality we assume $`\rho 0`$. For $`0<\varphi \frac{1}{6}\pi `$ and $`\frac{5}{6}\pi \varphi <\pi `$ we use the unique analytic continuation of the $`{}_{2}{}^{}F_{1}^{}`$-function to $`[1,\mathrm{})`$. Note that the Meixner-Pollaczek function is well defined for all $`n`$, since $`\mathrm{\Gamma }(c)^1{}_{2}{}^{}F_{1}^{}(a,b;c;z)`$ is analytic in $`a`$, $`b`$ and $`c`$.

The functions $`u_n`$ and $`u_n^{}`$ satisfy the recurrence relation

$$2x\mathrm{sin}\varphi u_n(x)=\alpha _nu_{n+1}(x)+\beta _nu_n(x)+\alpha _{n1}u_{n1}(x),$$
(2.6)
where

$$\alpha _n=\sqrt{(n+\epsilon +\lambda +1)(n+\epsilon \lambda )},\beta _n=2(n+\epsilon )\mathrm{cos}\varphi .$$

Let $`L`$ be the corresponding doubly infinite Jacobi operator acting on $`\mathrm{}^2()`$,

$$L:e_n\alpha _ne_{n+1}+\beta _ne_n+\alpha _{n1}e_{n1}.$$

$`L`$ is initially defined on finite linear combinations of the basis vectors of $`\mathrm{}^2()`$, and then $`L`$ is essentially self adjoint. The spectral measure of $`L`$ is described by

$$\begin{array}{cc}\hfill u,v& =\frac{1}{2\pi }_{}(u,U(x)U(x),v+u,U^{}(x)U^{}(x),v\hfill \\ & w_1(x)u,U(x)U^{}(x),vw_1^{}(x)u,U^{}(x)U(x),v)w_0(x)dx,\hfill \end{array}$$

where

$`w_1(x)=`$ $`{\displaystyle \frac{\mathrm{\Gamma }(\lambda +1+ix)\mathrm{\Gamma }(\lambda +ix)}{\mathrm{\Gamma }(ix\epsilon )\mathrm{\Gamma }(1+\epsilon ix)}},`$
$`w_0(x)=`$ $`{\displaystyle \frac{|\mathrm{\Gamma }(\lambda +1ix)\mathrm{\Gamma }(\lambda ix)\mathrm{\Gamma }(ix\epsilon )\mathrm{\Gamma }(1+\epsilon ix)|^2}{|\mathrm{\Gamma }(ix\epsilon )\mathrm{\Gamma }(1+\epsilon ix)|^2|\mathrm{\Gamma }(\lambda +1ix)\mathrm{\Gamma }(\lambda ix)|^2}}`$
$`\times {\displaystyle \frac{(2\mathrm{sin}\varphi )^{2\epsilon }e^{2x(\varphi \frac{\pi }{2})}}{\mathrm{\Gamma }(\epsilon \lambda )\mathrm{\Gamma }(1+\lambda \epsilon )\mathrm{\Gamma }(1+\lambda +\epsilon )\mathrm{\Gamma }(\epsilon \lambda )}}`$
$`=`$ $`(2\mathrm{sin}\varphi )^{2\epsilon }e^{2x(\varphi \frac{\pi }{2})},`$
$`U(x)=`$ $`{\displaystyle \underset{n=\mathrm{}}{\overset{\mathrm{}}{}}}u_n(x;\lambda ,\epsilon ,\varphi )e_n,U^{}(x)={\displaystyle \underset{n=\mathrm{}}{\overset{\mathrm{}}{}}}u_n^{}(x;\lambda ,\epsilon ,\varphi )e_n.`$
The spectral measure for $`L`$ can be obtained from \[14, §4.4.11\], using the connection formulas given there. The expression for $`w_0(x)`$ is found using Euler’s reflection formula, elementary trigonometric identities and the conditions on $`\lambda `$.

Let $`=(\lambda ,\epsilon ,\varphi )`$ be the Hilbert space consisting of functions

$$𝐟:^2,x\left(\begin{array}{c}f_1(x)\\ f_2(x)\end{array}\right),$$

with inner product defined by

$$𝐟,𝐠=\frac{1}{2\pi }_{}\left(\begin{array}{c}g_1(x)\\ g_2(x)\end{array}\right)^{}\left(\begin{array}{cc}1& w_1(x)\\ w_1^{}(x)& 1\end{array}\right)\left(\begin{array}{c}f_1(x)\\ f_2(x)\end{array}\right)w_0(x)𝑑x.$$

Observe that the square matrix inside the integral is positive definite and self adjoint.

###### Proposition 2.1.

The functions

$$𝐮_𝐧=\left(\begin{array}{c}u_n\\ u_n^{}\end{array}\right)$$

form an orthonormal basis of the Hilbert space $``$.

The orthonormality follows by choosing $`u`$ and $`v`$ above as standard basis vectors $`e_n`$ and $`e_m`$. The completeness of the Meixner-Pollaczek functions follows from the uniqueness of the spectral measure. An alternative, group-theoretic approach, can be found in . It is only worked out for the smaller range of parameters corresponding to $`\mathrm{SU}(1,1)`$ (rather than its Lie algebra or universal covering group). We will briefly describe how the analytic arguments that lie behind the approach of  extend to the present situation. We only discuss the case $`\lambda =\frac{1}{2}+i\rho `$, $`\rho `$, which is all that we need later.

Consider the space $`L^2(𝕋)`$ on the unit circle with respect to normalized measure $`|dz|/2\pi `$. It has the orthonormal basis $`e_n(z)=z^n`$, $`n`$. If $`\lambda =\frac{1}{2}+i\rho `$, $`\rho `$ and $`\epsilon `$, it is easily checked that

$$(Uf)(x)=\frac{1}{\sqrt{\pi }}(x+i)^{\lambda \epsilon }(xi)^{\lambda +\epsilon }f\left(\frac{xi}{x+i}\right)$$

defines an isometry $`U:L^2(𝕋)L^2().`$ Next we recall that the Mellin transform, defined by

$$f(x)\frac{1}{\sqrt{2\pi }}_0^{\mathrm{}}f(y)y^{ix\frac{1}{2}}𝑑y,$$

gives an isometry $`L^2(_+)L^2()`$. Thus, we may define a “double” Mellin transform as the isometry

$$V:L^2()L^2()L^2()$$

given by

$$(Vf)(x)=\frac{1}{\sqrt{2\pi }}\left(\begin{array}{c}_0^{\mathrm{}}f(y)y^{ix\frac{1}{2}}𝑑y\\ _0^{\mathrm{}}f(y)y^{ix\frac{1}{2}}𝑑y\end{array}\right).$$

If we now let $`T_t`$, $`t`$, denote the translation operator $`(T_tf)(x)=f(x+t)`$, we may compose the above isometries to obtain the orthonormal basis

$$𝐟_𝐧=\left(\begin{array}{c}f_n^+\\ f_n^{}\end{array}\right)=(VT_tU)(e_n)$$

of $`L^2()L^2()`$. Explicitly, we have

$$f_n^\pm (x)=\frac{1}{\sqrt{2}\pi }_0^{\mathrm{}}(t+i\pm y)^{\lambda n\epsilon }(ti\pm y)^{\lambda +n+\epsilon }y^{ix\frac{1}{2}}𝑑y.$$

These integrals may be expressed in terms of Gauss’s hypergeometric function; cf. \[27, §7.7.3\]. Using Kummer’s identities \[6, §2.9\], one may then express $`f_n^\pm `$ in terms of $`u_n(x+\rho )`$ and $`u_n^{}(x+\rho )`$, where $`e^{2i\varphi }=(t+i)/(ti)`$. Rewriting the identity $`𝐟_𝐧,𝐟_𝐦=\delta _{nm}`$ in terms of Meixner-Pollaczek functions and making a final change of variables $`xx\rho `$, one recovers the orthonormal basis $`𝐮_n`$.

The group-theoretic interpretation of this proof is the following. The space $`L^2(𝕋)`$ is a natural representation space for the principal unitary series ($`e_n`$ is proportional to the $`e_n`$ in (4.5) below). The operator $`L`$ gives the action of a hyperbolic Lie algebra element; cf. also §4.2. It generates a one-parameter subgroup of the universal covering group of $`\mathrm{SU}(1,1)`$, which locally may be identified with the group of linear fractional transformations of the circle that have two common fix-points. Thus, $`L^2(𝕋)`$ splits into two invariant subspaces. The map $`T_tU`$ corresponds to mapping the fix-points to $`\{0,\mathrm{}\}`$, and the one-parameter subgroup to dilations of $``$. Finally, the operator $`V`$ is the Fourier transform with respect to these dilations. In particular, the appearance of double eigenvalues in Proposition 2.1 has a natural geometric explanation: it corresponds to the fact that a circle falls into two pieces when removing two points.

## 3. A bilinear summation formula

In this section we prove a bilinear summation formula for Meixner-Pollaczeck polynomials. This summation is related to the tensor product of a positive and a negative discrete series representation of the Lie algebra $`𝔰𝔲(1,1)`$, which will be explained in section 4. In the summation a certain type of non-polynomial $`{}_{3}{}^{}F_{2}^{}`$-functions appear. These functions will be investigated in section 5.

###### Theorem 3.1.

For $`\rho ^2\mathrm{supp}d\mu (;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2})`$, $`p`$, $`x_1,x_2`$, and $`k_1,k_2>0`$, the Meixner-Pollaczek polynomials and the continuous dual Hahn polymials satisfy the following summation formula

$$\begin{array}{cc}\hfill \frac{1}{C}\underset{n=0}{\overset{\mathrm{}}{}}& \frac{(1)^n}{(2k_1)_{p+n}(2k_2)_n}s_n(\rho ^2;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2})p_{n+p}^{(k_1)}(x_1;\varphi )p_n^{(k_2)}(x_2;\varphi )=\hfill \\ & (1)^p\frac{\mathrm{\Gamma }(i(x_1x_2)+\frac{1}{2}+i\rho )\mathrm{\Gamma }(i(x_1x_2)+\frac{1}{2}i\rho )}{\mathrm{\Gamma }(k_2k_1+i(x_1x_2)+1)\mathrm{\Gamma }(k_1k_2+i(x_1x_2)+1+p)}\hfill \\ \hfill \times & {}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{k_2ix_2,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,k_2k_1+i(x_1x_2)+1}.;1)\hfill \\ \hfill \times & {}_{2}{}^{}F_{1}^{}(.\genfrac{}{}{0.0pt}{}{p+k_1k_2+\frac{1}{2}+i\rho ,p+k_1k_2+\frac{1}{2}i\rho }{k_1k_2+i(x_1x_2)+p+1}.;\frac{1}{1e^{2i\varphi }})\hfill \\ \hfill +& \frac{\mathrm{\Gamma }(i(x_2x_1)+\frac{1}{2}+i\rho )\mathrm{\Gamma }(i(x_2x_1)+\frac{1}{2}i\rho )}{\mathrm{\Gamma }(k_2k_1+i(x_2x_1)+1)\mathrm{\Gamma }(k_1k_2+i(x_2x_1)+1+p)}\hfill \\ \hfill \times & {}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{k_2+ix_2,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,k_2k_1+i(x_2x_1)+1}.;1)\hfill \\ \hfill \times & {}_{2}{}^{}F_{1}^{}(.\genfrac{}{}{0.0pt}{}{p+k_1k_2+\frac{1}{2}+i\rho ,p+k_1k_2+\frac{1}{2}i\rho }{k_1k_2+i(x_2x_1)+p+1}.;\frac{1}{1e^{2i\varphi }}),\hfill \end{array}$$

where

$$C=\frac{e^{2x_1(\varphi \frac{\pi }{2})}p!\mathrm{\Gamma }(2k_1)}{i^p(2\mathrm{sin}\varphi )^{2k_1+p}\mathrm{\Gamma }(k_1+ix_1)\mathrm{\Gamma }(k_1ix_1)}.$$

Here we use the convention that if $`p_n`$ is a polynomial of degree $`n`$, then $`p_n=0`$ for $`n`$.

###### Remark 3.2.

(i) The $`{}_{2}{}^{}F_{1}^{}`$-series on the right hand side are Meixner-Pollaczek functions, as defined by (2.5). The $`{}_{3}{}^{}F_{2}^{}`$-series on the right hand side can be considered as continuous Hahn functions, see section 5.

(ii) Theorem 3.1 is a non-polynomial extension of \[10, Thm.3.6\]. Replacing $`k_1+ix_1`$, $`k_2+ix_2`$, $`e^{2i\varphi }`$, $`\rho ^2`$ in Theorem 3.1 by $`x_1`$, $`x_2`$, $`c`$, $`y`$ respectively, and restricting the new $`x_1`$ and $`x_2`$ to negative integers makes one term on the right hand side vanish, and we obtain \[10, Thm.3.6\].

(iii) There is an interesting limit case of the summation of Theorem 3.1; for $`x_1x_2>0`$

$$\begin{array}{cc}\hfill \underset{n=0}{\overset{\mathrm{}}{}}& \frac{(1)^n}{(2k_1)_{p+n}(2k_2)_n}s_n(\rho ^2;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2})L_{n+p}^{(2k_11)}(x_1)L_n^{(2k_21)}(x_2)=\hfill \\ & \mathrm{\Gamma }(2k_1)e_2^{x_2}F_1(.\genfrac{}{}{0.0pt}{}{k_1+k_2\frac{1}{2}+i\rho ,k_1+k_2\frac{1}{2}i\rho }{2k_2}.;\frac{x_2}{x_2x_1})\hfill \\ \hfill \times & (x_1x_2)^{k_1+k_2\frac{1}{2}i\rho }U(p+k_1k_2+\frac{1}{2}+i\rho ;1+2i\rho ;x_1x_2),\hfill \end{array}$$

where $`L_n^{(\alpha )}`$ is a Laguerre polynomial as defined in , and $`U(a;b;z)`$ denotes the second solution of the confluent hypergeometric differential equation in the notation of Slater :

$$U(a;b;z)=\frac{\mathrm{\Gamma }(1b)}{\mathrm{\Gamma }(1+ab)}_1F_1(.\genfrac{}{}{0.0pt}{}{a}{b}.;z)+\frac{\mathrm{\Gamma }(b1)}{\mathrm{\Gamma }(a)}z_1^{1b}F_1(.\genfrac{}{}{0.0pt}{}{1+ab}{2b}.;z).$$

This formula is obtained from Theorem 3.1 as follows. We replace $`x_i`$ by $`x_i/2\varphi `$, $`i=1,2`$, and transform the $`{}_{2}{}^{}F_{1}^{}`$-series on the right hand side by \[1, (2.3.12)\]. Then we let $`\varphi 0`$. Here Stirling’s formula is used, Euler’s transformation is used for the $`{}_{2}{}^{}F_{1}^{}`$-series which are obtained from the $`{}_{3}{}^{}F_{2}^{}`$-series, and Kummer’s transformation \[24, (1.4.1)\] is used for the $`{}_{1}{}^{}F_{1}^{}`$-series which are obtained from the $`{}_{2}{}^{}F_{1}^{}`$-series. This limit case can also be obtained by Lie algebraic methods, see \[9, Thm.3.10\].

(iv) Note that from

$$\begin{array}{cc}\hfill s_n& (\rho ^2;k_1k_2+\frac{1}{2},k_1+k_2\frac{1}{2},k_2k_1p+\frac{1}{2})=\hfill \\ & (1)^p|(k_1k_2+\frac{1}{2}+i\rho )_p|^2s_{np}(\rho ^2;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2}),\hfill \end{array}$$
(3.1)
see \[10, (3.13)\], it follows that the sum on the left hand side of Theorem 3.1 is invariant under $`k_1k_2`$, $`x_1x_2`$, $`pp`$.

(v) It is interesting to compare Theorem 3.1 with the results of , where summation formulas with a similar, but simpler, structure are obtained for various orthogonal polynomials. The method used in  is completely different from the method we use here.

###### Proof of Theorem 3.1.

We start with the sum on the left hand side, with orthonormal polynomials:

$$S=\underset{n=0}{\overset{\mathrm{}}{}}S_n(\rho ^2;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2})P_{n+p}^{(k_1)}(x_1;\varphi )P_n^{(k_2)}(x_2;\varphi ).$$

First we show that this sum converges absolutely. Writing out the summand $`R_n`$ explicitly gives

$$R_n=Ke^{2in\varphi }\frac{\mathrm{\Gamma }(2k_2+n)\mathrm{\Gamma }(2k_1+p+n)}{\mathrm{\Gamma }(n+1)\mathrm{\Gamma }(n+p+1)}{}_{3}{}^{}F_{2}^{}{}_{2}{}^{}F_{1}^{}{}_{2}{}^{}F_{1}^{}$$

where $`K`$ is a constant independent of $`n`$. To find the asymptotic behaviour for $`n\mathrm{}`$ of the $`\mathrm{\Gamma }`$-functions, we use the asymptotic formula for the ratio of two $`\mathrm{\Gamma }`$-functions \[21, §4.5\]

$$\frac{\mathrm{\Gamma }(a+z)}{\mathrm{\Gamma }(b+z)}=z^{ab}\left(1+𝒪(z^1)\right),|z|\mathrm{},|\mathrm{arg}(z)|<\pi .$$
(3.2)
The asymptotics for the $`{}_{3}{}^{}F_{2}^{}`$-function follows from transforming the function by \[2, p.15(2)\] and using (3.2). This gives, for $`n\mathrm{}`$,

$${}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{n,k_1+k_2\frac{1}{2}+i\rho ,k_1+k_2\frac{1}{2}i\rho }{2k_2,2k_1+p}.;1)=C_1n^{\frac{1}{2}k_1k_2i\rho }+C_2n^{\frac{1}{2}k_1k_2+i\rho },$$

where $`C_1`$ and $`C_2`$ are independent of $`n`$. If $`\rho ^2`$ is in the discrete part of $`\mathrm{supp}d\mu (;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2})`$, we assume without loss of generality that $`\mathrm{}(\rho )>0`$. In this case the second term in the transformation \[2, p.15(2)\] vanishes, so $`C_2=0`$.

The asymptotic behaviour of the $`{}_{2}{}^{}F_{1}^{}`$-functions follows from \[6, 2.3.2(14)\]. This gives, for $`0<\varphi <\pi `$ and $`n\mathrm{}`$,

$$\begin{array}{cc}\hfill {}_{2}{}^{}F_{1}^{}(.\genfrac{}{}{0.0pt}{}{(n+p),k_1+ix_1}{2k_1}.;1e^{2i\varphi })& =\left(C_3n^{k_1ix_1}+C_4n^{k_1+ix_1}e^{(n+p)(1e^{2i\varphi })}\right)\left(1+𝒪(n^1)\right),\hfill \\ \hfill {}_{2}{}^{}F_{1}^{}(.\genfrac{}{}{0.0pt}{}{n,k_2ix_2}{2k_2}.;1e^{2i\varphi })& =\left(C_5n^{k_2ix_2}+C_6n^{k_2+ix_2}e^{n(1e^{2i\varphi })}\right)\left(1+𝒪(n^1)\right),\hfill \end{array}$$

where $`C_i`$, $`i=3,\mathrm{},6`$, is independent of $`n`$. Since $`\mathrm{}(1e^{2i\varphi })=2\mathrm{sin}^2\varphi >0`$ and $`x_1,x_2`$, we find

$$R_n=(K_1n^{\frac{3}{2}+i\rho }+K_2n^{\frac{3}{2}i\rho })\left(1+𝒪(n^1)\right),n\mathrm{},$$

and then for $`\rho `$ the sum $`S`$ converges absolutely. In case $`\rho i`$, we have $`K_2=0`$ and $`\mathrm{}(\rho )>0`$, and then $`S`$ still converges absolutely.

Next we write out the polynomials in $`S`$ as hypergeometric series, using (2.2) and (2.1), and then we transform the $`{}_{3}{}^{}F_{2}^{}`$-series, using the first formula on page 142 of ;

$$\begin{array}{c}{}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{n,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,p+1}.;1)=\hfill \\ \hfill \frac{(p+k_1k_2+\frac{1}{2}i\rho )_n}{(p+1)_n}_3F_2(.\genfrac{}{}{0.0pt}{}{n,k_2k_1+\frac{1}{2}+i\rho ,k_1+k_2\frac{1}{2}+i\rho }{2k_2,k_2k_1+\frac{1}{2}+i\rho np}.;1).\end{array}$$

By \[6, §2.9(27)\] with $`(a,b,c,z)=(np,k_1+ix_1,2k_1,1e^{2i\varphi })`$ the $`{}_{2}{}^{}F_{1}^{}`$-series for $`P_{n+p}^{(k_1)}(x_1;\varphi )`$ is written as a sum of two $`{}_{2}{}^{}F_{1}^{}`$-series

$$\begin{array}{cc}& {}_{2}{}^{}F_{1}^{}(.\genfrac{}{}{0.0pt}{}{np,k_1+ix_1}{2k_1}.;1e^{2i\varphi })=\hfill \\ & \frac{(1e^{2i\varphi })^{ix_1k_1}\mathrm{\Gamma }(2k_1)\mathrm{\Gamma }(n+p+1)}{\mathrm{\Gamma }(k_1ix_1)\mathrm{\Gamma }(n+p+1+k_1+ix_1)}_2F_1(.\genfrac{}{}{0.0pt}{}{k_1+ix_1,1k_1+ix_1}{n+p+1+k_1+ix_1}.;\frac{1}{1e^{2i\varphi }})\hfill \\ & +(e^{2i\varphi })^{n+p}\frac{(1e^{2i\varphi })^{ix_1k_1}\mathrm{\Gamma }(2k_1)\mathrm{\Gamma }(n+p+1)}{\mathrm{\Gamma }(k_1+ix_1)\mathrm{\Gamma }(n+p+1+k_1ix_1)}_2F_1(.\genfrac{}{}{0.0pt}{}{k_1ix_1,1k_1ix_1}{n+p+1+k_1ix_1}.;\frac{1}{1e^{2i\varphi }}).\hfill \end{array}$$
(3.3)
Now the sum $`S`$ splits according to this: $`S=S_1+S_2`$.

First we focus on $`S_1`$. Reversing the order of summation in the $`{}_{2}{}^{}F_{1}^{}`$-series of the second Meixner-Pollaczek polynomial and using Euler’s transformation \[1, (2.2.7)\], gives

$$\begin{array}{c}_2F_1(.\genfrac{}{}{0.0pt}{}{n,k_2+ix_2}{2k_2}.;1e^{2i\varphi })=\hfill \\ \hfill e^{2in\varphi }(1e^{2i\varphi })^{ix_2k_2}\frac{(k_2+ix_2)_n}{(2k_2)_n}_2F_1(.\genfrac{}{}{0.0pt}{}{k_2ix_2,12k_2ix_2}{1k_2nix_2}.;\frac{1}{1e^{2i\varphi }}).\end{array}$$
(3.4)
Writing out the hypergeometric series as a sum, we get

$$\begin{array}{cc}\hfill S_1=& e^{ip\varphi }\frac{(1e^{2i\varphi })^{k_1ix_1}(1e^{2i\varphi })^{ix_2k_2}}{\mathrm{\Gamma }(k_1ix_1)\mathrm{\Gamma }(p+1+k_1+ix_1)}\mathrm{\Gamma }(2k_1)\sqrt{p!(2k_1)_p}\hfill \\ \hfill \times & \underset{n=0}{\overset{\mathrm{}}{}}\underset{j=0}{\overset{n}{}}\underset{l,m=0}{\overset{\mathrm{}}{}}\frac{(k_2+ix_2)_n(p+k_1k_2+\frac{1}{2}i\rho )_n(n)_j(k_2k_1+\frac{1}{2}+i\rho )_j(k_1+k_2\frac{1}{2}+i\rho )_j}{n!(p+1+k_1+ix_1)_nj!(2k_2)_j(k_2k_1+\frac{1}{2}+i\rho np)_j}\hfill \\ \hfill \times & \frac{(k_1+ix_1)_l(1k_1+ix_1)_l(k_2ix_2)_m(1k_2ix_2)_m}{l!(n+p+1+k_1+ix_1)_lm!(1k_2nix_2)_m}(1e^{2i\varphi })^{lm}.\hfill \end{array}$$

Next we interchange summations

$$\underset{n=0}{\overset{\mathrm{}}{}}\underset{j=0}{\overset{n}{}}=\underset{j=0}{\overset{\mathrm{}}{}}\underset{n=j}{\overset{\mathrm{}}{}},$$

then the sum over $`n`$ becomes

$$\mathrm{\Sigma }_1=\underset{n=j}{\overset{\mathrm{}}{}}\frac{(k_1k_2+\frac{1}{2}i\rho +pj)_n(k_2+ix_2m)_n}{(nj)!(p+1+l+k_1+ix_1)_n}.$$

We substitute $`knj`$, then we find by Stirling’s formula for the summand $`R`$ of $`S_1`$, for large $`j`$ and $`k`$,

$$RCj^{k_1k_2+p}k^{\frac{1}{2}}(j+k)^{1lm},$$

where $`C`$ is independent of $`k`$ and $`j`$. So we see that the sum $`S_1`$ converges absolutely for $`k_1k_2+p<0`$, and $`\frac{1}{6}\pi <\varphi <\frac{5}{6}\pi `$. Now the sum $`\mathrm{\Sigma }_1`$ is a multiple of a $`{}_{2}{}^{}F_{1}^{}`$-series with unit argument, which is summable by Gauss’ summation formula \[6, (46), p.104\];

$$\mathrm{\Sigma }_1=\frac{(k_2k_1+\frac{1}{2}p+i\rho )_j(k_2+ix_2m)_j\mathrm{\Gamma }(p+l+1+k_1+ix_1)\mathrm{\Gamma }(l+m+\frac{1}{2}+ix_1ix_2+i\rho )}{\mathrm{\Gamma }(l+j+\frac{1}{2}+k_2+ix_2+i\rho )\mathrm{\Gamma }(p+l+m+k_1k_2+ix_1ix_2)}.$$

This gives

$$\begin{array}{cc}\hfill S_1=& e^{ip\varphi }\frac{(1e^{2i\varphi })^{k_1ix_1}(1e^{2i\varphi })^{ix_2k_2}}{\mathrm{\Gamma }(k_1ix_1)}\mathrm{\Gamma }(2k_1)\sqrt{p!(2k_1)_p}\hfill \\ \hfill \times & \underset{l,m=0}{\overset{\mathrm{}}{}}\frac{\mathrm{\Gamma }(l+m+i(x_1x_2)+\frac{1}{2}+i\rho )(k_2ix_2)_m(k_1+ix_1)_l(1k_1+ix_1)_l}{\mathrm{\Gamma }(p+l+m+1+k_1k_2+i(x_1x_2))\mathrm{\Gamma }(l+k_2+\frac{1}{2}+ix_1+i\rho )m!l!}(1e^{2i\varphi })^{lm}\hfill \\ \hfill \times & \underset{j=0}{\overset{\mathrm{}}{}}\frac{(k_2k_1+\frac{1}{2}+i\rho )_j(k_1+k_2\frac{1}{2}+i\rho )_j(k_2+ix_2m)_j}{j!(2k_2)_j(l+\frac{1}{2}+k_2+ix_1+i\rho )_j}\hfill \end{array}$$

The sum over $`j`$ is a $`{}_{3}{}^{}F_{2}^{}`$-series, which by Kummer’s transformation \[1, Cor.3.3.5\] becomes

$$\begin{array}{cc}& \frac{\mathrm{\Gamma }(l+\frac{1}{2}+k_2+ix_1+i\rho )\mathrm{\Gamma }(l+m+\frac{1}{2}+i(x_1x_2)i\rho )}{\mathrm{\Gamma }(k_2k_1+1+i(x_1x_2)+l+m)\mathrm{\Gamma }(l+k_1+ix_1)}\hfill \\ & \times _3F_2(.\genfrac{}{}{0.0pt}{}{k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho ,k_2ix_2+m}{2k_2,k_2k_1+1+i(x_1x_2)+l+m}.;1).\hfill \end{array}$$

Here we need the condition $`k_1>0`$ for absolute convergence. We write out the $`{}_{3}{}^{}F_{2}^{}`$-series explicitly as a sum over $`j`$ and interchange summations

$$S_1=\underset{l,m,j=0}{\overset{\mathrm{}}{}}=\underset{n=0}{\overset{\mathrm{}}{}}\underset{j=0}{\overset{\mathrm{}}{}}\underset{l+m=n}{}.$$

For the sum over $`l+m=n`$ we find

$$\underset{m+l=n}{}\frac{(1k_1+ix_1)_l(k_2ix_2+j)_m}{l!m!}=\frac{(1+k_2k_1+i(x_1x_2)+j)_n}{n!}.$$

Now $`S_1`$ reduces to a double sum, which splits as a product of two sums, and we obtain

$$\begin{array}{cc}\hfill S_1=& e^{ip\varphi }\frac{(1e^{2i\varphi })^{k_1ix_1}(1e^{2i\varphi })^{ix_2k_2}\mathrm{\Gamma }(\frac{1}{2}+i(x_1x_2)+i\rho )\mathrm{\Gamma }(\frac{1}{2}+i(x_1x_2)i\rho )}{\mathrm{\Gamma }(k_1ix_1)\mathrm{\Gamma }(k_1+ix_1)\mathrm{\Gamma }(p+k_1k_2+i(x_1x_2)+1)\mathrm{\Gamma }(k_2k_1+i(x_1x_2)+1)}\hfill \\ & \times \mathrm{\Gamma }(2k_1)\sqrt{p!(2k_1)_p}_2F_1(.\genfrac{}{}{0.0pt}{}{\frac{1}{2}+i(x_1x_2)+i\rho ,\frac{1}{2}+i(x_1x_2)i\rho }{p+k_1k_2+i(x_1x_2)+1}.;\frac{1}{1e^{2i\varphi }})\hfill \\ & \times _3F_2(.\genfrac{}{}{0.0pt}{}{k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho ,k_2ix_2}{2k_2,k_2k_1+i(x_1x_2)+1}.;1).\hfill \end{array}$$

$`S_2`$ is calculated in the same way as $`S_1`$, only for the first step \[6, §2.10(4)\] is used instead of (3.4). Then we obtain

$$\begin{array}{cc}\hfill S_2& =e^{ip\varphi }\frac{(1e^{2i\varphi })^{k_2ix_2}(1e^{2i\varphi })^{ix_1k_1}}{\mathrm{\Gamma }(k_1+ix_1)\mathrm{\Gamma }(k_1ix_1)}\mathrm{\Gamma }(2k_1)\sqrt{p!(2k_1)_p}\hfill \\ & \times \frac{\mathrm{\Gamma }(i(x_2x_1)+\frac{1}{2}+i\rho )\mathrm{\Gamma }(i(x_2x_1)+\frac{1}{2}i\rho )}{\mathrm{\Gamma }(k_2k_1+i(x_2x_1)+1)\mathrm{\Gamma }(k_1k_2+i(x_2x_1)+1+p)}\hfill \\ & \times _3F_2(.\genfrac{}{}{0.0pt}{}{k_2+ix_2,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,k_2k_1+i(x_2x_1)+1}.;1)\hfill \\ & \times _2F_1(.\genfrac{}{}{0.0pt}{}{\frac{1}{2}+i(x_2x_1)+i\rho ,\frac{1}{2}+i(x_2x_1)i\rho }{k_1k_2+i(x_2x_1)+p+1}.;\frac{1}{1e^{2i\varphi }}).\hfill \end{array}$$

Finally using Euler’s transformation for the $`{}_{2}{}^{}F_{1}^{}`$-series, using

$`e^{ip\varphi }(1e^{2i\varphi })^p`$ $`=(1)^p(2i\mathrm{sin}\varphi )^p,`$
$`(1e^{2i\varphi })^{ix_1}(1e^{2i\varphi })^{ix_1}`$ $`=e^{2x_1(\varphi \frac{\pi }{2})},`$
$`(1e^{2i\varphi })^{k_1}(1e^{2i\varphi })^{k_1}`$ $`=(2\mathrm{sin}\varphi )^{2k_1},`$
and writing the polynomials in the normalization given by (2.2) and (2.1), the theorem is proved.

Let us remark that all the series used in the proof are absolutely convergent under the conditions $`\rho ^2\mathrm{supp}d\mu (;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2})`$, $`x_1,x_2`$, $`k_1>0`$, $`\frac{1}{6}\pi <\varphi <\frac{5}{6}\pi `$ and $`k_1k_2+p<0`$. The last condition can be removed using the symmetry $`(k_1,k_2,p)(k_2,k_1,p)`$ and continuity in $`k_1`$ and $`k_2`$. Using the analytic continuation of the hypergeometric function, we see that the result remains valid for $`0<\varphi <\pi `$. ∎

## 4. Clebsch-Gordan coefficients for hyperbolic basis vectors of $`𝔰𝔲(1,1)`$

In this section we consider the tensor product of a positive and a negative discrete series representation. We diagonalize a certain self-adjoint element of $`𝔰𝔲(1,1)`$ using (doubly infinite) Jacobi operators. We also give generalized eigenvectors, which can be considered as hyperbolic basis vectors. Using the summation formula from the previous section, we show that the Clebsch-Gordan coefficients for the eigenvectors are continuous Hahn functions. We find the corresponding integral transform pair by formal computations. In order to give a rigorous proof for the continuous Hahn integral transform, we realize the generators of $`𝔰𝔲(1,1)`$ in the discrete series as difference operators acting on polynomials. Using these realizations, the Casimir element in the tensor product is realized as a difference operator. Spectral analysis of this difference operator is carried out in section 5.

### 4.1. The Lie algebra $`𝖘𝖚\mathbf{(}\mathrm{𝟏}\mathbf{,}\mathrm{𝟏}\mathbf{)}`$

The Lie algebra $`𝔰𝔲(1,1)`$ is generated by the elements $`H`$, $`B`$ and $`C`$, satisfying the commutation relations

$$[H,B]=2B,[H,C]=2C,[B,C]=H.$$
(4.1)
There is a $``$-structure defined by $`H^{}=H`$ and $`B^{}=C`$. The center of $`𝒰\left(𝔰𝔲(1,1)\right)`$ is generated by the Casimir element $`\mathrm{\Omega }`$, which is given by

$$\mathrm{\Omega }=\frac{1}{4}(H^2+2H+4CB)=\frac{1}{4}(H^22H+4BC).$$
(4.2)
There are four classes of irreducible unitary representations of $`𝔰𝔲(1,1)`$, see \[27, §6.4\]:

The positive discrete series representations $`\pi _k^+`$ are representations labelled by $`k>0`$. The representation space is $`\mathrm{}^2(_0)`$ with orthonormal basis $`\{e_n\}_{n_0}`$. The action is given by

$$\begin{array}{cc}\hfill \pi _k^+(H)e_n=& 2(k+n)e_n,\hfill \\ \hfill \pi _k^+(B)e_n=& \sqrt{(n+1)(2k+n)}e_{n+1},\hfill \\ \hfill \pi _k^+(C)e_n=& \sqrt{n(2k+n1)}e_{n1},\hfill \\ \hfill \pi _k^+(\mathrm{\Omega })e_n=& k(1k)e_n.\hfill \end{array}$$
(4.3)
The negative discrete series representations $`\pi _k^{}`$ are labelled by $`k>0`$. The representation space is $`\mathrm{}^2(_0)`$ with orthonormal basis $`\{e_n\}_{n_0}`$. The action is given by

$$\begin{array}{cc}\hfill \pi _k^{}(H)e_n=& 2(k+n)e_n,\hfill \\ \hfill \pi _k^{}(B)e_n=& \sqrt{n(2k+n1)}e_{n1},\hfill \\ \hfill \pi _k^{}(C)e_n=& \sqrt{(n+1)(2k+n)}e_{n+1},\hfill \\ \hfill \pi _k^{}(\mathrm{\Omega })e_n=& k(1k)e_n.\hfill \end{array}$$
(4.4)
The principal series representations $`\pi ^{\rho ,\epsilon }`$ are labelled by $`\epsilon [0,1)`$ and $`\rho 0`$, where $`(\rho ,\epsilon )(0,\frac{1}{2})`$. The representation space is $`\mathrm{}^2()`$ with orthonormal basis $`\{e_n\}_n`$. The action is given by

$$\begin{array}{cc}\hfill \pi ^{\rho ,\epsilon }(H)e_n=& 2(\epsilon +n)e_n,\hfill \\ \hfill \pi ^{\rho ,\epsilon }(B)e_n=& \sqrt{(n+\epsilon +\frac{1}{2}i\rho )(n+\epsilon +\frac{1}{2}+i\rho )}e_{n+1},\hfill \\ \hfill \pi ^{\rho ,\epsilon }(C)e_n=& \sqrt{(n+\epsilon \frac{1}{2}i\rho )(n+\epsilon \frac{1}{2}+i\rho )}e_{n1},\hfill \\ \hfill \pi ^{\rho ,\epsilon }(\mathrm{\Omega })e_n=& (\rho ^2+\frac{1}{4})e_n.\hfill \end{array}$$
(4.5)
For $`(\rho ,\epsilon )=(0,\frac{1}{2})`$ the representation $`\pi ^{0,\frac{1}{2}}`$ splits into a direct sum of a positive and a negative discrete series representation: $`\pi ^{0,\frac{1}{2}}=\pi _{1/2}^+\pi _{1/2}^{}`$. The representation space splits into two invariant subspaces: $`\{e_n|n<0\}\{e_n|n0\}`$.

The complementary series representations $`\pi ^{\lambda ,\epsilon }`$ are labelled by $`\epsilon `$ and $`\lambda `$, where $`\epsilon [0,\frac{1}{2})`$ and $`\lambda (\frac{1}{2},\epsilon )`$ or $`\epsilon (\frac{1}{2},1)`$ and $`\lambda (\frac{1}{2},\epsilon 1)`$. The representation space is $`\mathrm{}^2()`$ with orthonormal basis $`\{e_n\}_n`$. The action is given by

$$\begin{array}{cc}\hfill \pi ^{\lambda ,\epsilon }(H)e_n=& 2(\epsilon +n)e_n,\hfill \\ \hfill \pi ^{\lambda ,\epsilon }(B)e_n=& \sqrt{(n+\epsilon +1+\lambda )(n+\epsilon \lambda )}e_{n+1},\hfill \\ \hfill \pi ^{\lambda ,\epsilon }(C)e_n=& \sqrt{(n+\epsilon +\lambda )(n+\epsilon \lambda 1)}e_{n1},\hfill \\ \hfill \pi ^{\lambda ,\epsilon }(\mathrm{\Omega })e_n=& \lambda (1+\lambda )e_n.\hfill \end{array}$$
(4.6)
Note that if we formally write $`\lambda =\frac{1}{2}+i\rho `$ the actions in the principal series and in the complementary series are the same.

We remark that the operators (4.3)-(4.6) are unbounded, with domain the set of finite linear combinations of the basis vectors. The representations are $``$-representations in the sense of Schmüdgen \[23, Ch.8\].

The decomposition of the tensor product of a positive and a negative discrete series representation of $`𝔰𝔲(1,1)`$ is determined in full generality in \[10, Thm.2.2\].

###### Theorem 4.1.

For $`k_1k_2`$ the decomposition of the tensor product of positive and negative discrete series representations of $`𝔰𝔲(1,1)`$ is

$`\pi _{k_1}^+\pi _{k_2}^{}`$ $`{\displaystyle _{}^{}{}_{0}{}^{\mathrm{}}}\pi ^{\rho ,\epsilon }d\rho ,`$ $`k_1k_2{\displaystyle \frac{1}{2}},k_1+k_2{\displaystyle \frac{1}{2}},`$
$`\pi _{k_1}^+\pi _{k_2}^{}`$ $`{\displaystyle _{}^{}{}_{0}{}^{\mathrm{}}}\pi ^{\rho ,\epsilon }d\rho \pi ^{\lambda ,\epsilon },`$ $`k_1+k_2<{\displaystyle \frac{1}{2}},`$
$`\pi _{k_1}^+\pi _{k_2}^{}`$ $`{\displaystyle _{}^{}{}_{0}{}^{\mathrm{}}}\pi ^{\rho ,\epsilon }d\rho {\displaystyle \underset{\begin{array}{c}j_0\\ k_2k_1\frac{1}{2}j>0\end{array}}{}}\pi _{k_2k_1j}^{},`$ $`k_1k_2<{\displaystyle \frac{1}{2}},`$
where $`\epsilon =k_1k_2+L`$, $`L`$ is the unique integer such that $`\epsilon [0,1)`$, and $`\lambda =k_1k_2`$. The intertwiner $`J`$ is given by

$$J(e_{n_1}e_{n_2})=(1)^{n_2}_{}S_n(y;n_1n_2)e_{n_1n_2L}𝑑\mu ^{\frac{1}{2}}(y;n_1n_2),$$
(4.7)
where $`n=\mathrm{min}\{n_1,n_2\}`$, $`S_n(y;p)`$ is an orthonormal continuous dual Hahn polynomial,

$$S_n(y;p)=\{\begin{array}{cc}S_n(y;k_1k_2+\frac{1}{2},k_1+k_2\frac{1}{2},k_2k_1p+\frac{1}{2}),\hfill & p0,\hfill \\ S_n(y;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2}),\hfill & p0,\hfill \end{array}$$

and $`d\mu (y;p)`$ is the corresponding orthogonality measure

$$d\mu (y;p)=\{\begin{array}{cc}d\mu (y;k_1k_2+\frac{1}{2},k_1+k_2\frac{1}{2},k_2k_1p+\frac{1}{2}),\hfill & p0,\hfill \\ d\mu (y;k_2k_1+\frac{1}{2},k_1+k_2\frac{1}{2},k_1k_2+p+\frac{1}{2}),\hfill & p0.\hfill \end{array}$$

The inversion of (4.7) can be given explicitly, e.g. for an element

$$fe_{rL}=_0^{\mathrm{}}f(x)e_{rL}𝑑xL^2(0,\mathrm{})\mathrm{}^2()_{}^{}{}_{0}{}^{\mathrm{}}\mathrm{}^2()dx$$

in the representation space of the direct integral representation, we have

$$J^{}(fe_{rL})=\{\begin{array}{cc}\underset{p=0}{\overset{\mathrm{}}{}}(1)^{pr}\left[_{}S_p(y;r)f(y)𝑑\mu ^{\frac{1}{2}}(y;r)\right]e_pe_{pr},\hfill & r0,\hfill \\ \underset{p=0}{\overset{\mathrm{}}{}}(1)^p\left[_{}S_p(y;r)f(y)𝑑\mu ^{\frac{1}{2}}(y;r)\right]e_{p+r}e_p,\hfill & r0.\hfill \end{array}$$
(4.8)
For the discrete components in Theorem 4.1 we can replace $`f`$ by a Dirac delta function at the appropriate points of the discrete mass of $`d\mu (;r)`$. We remark that for $`k_1=k_2<1/4`$, the occurrence of a complementary series representation in the tensor product was discovered by Neretin . This phenomenon was investigated from the viewpoint of operator theory in .

In the following subsection we assume that discrete terms do not occur in the tensor product decomposition. From the calculations it is clear how to extend the results to the general case. At the end of section 5 we briefly discuss the results for the discrete terms in the decomposition.

In the Lie algebra $`𝔰𝔲(1,1)`$ three types of elements can be distinguished: the elliptic, the parabolic and the hyperbolic elements. These are related to the three conjugacy classes of the group $`SU(1,1)`$. A basis on which an elliptic element acts diagonally is called an elliptic basis, and similarly for the parabolic and hyperbolic elements. The basisvectors $`e_n`$ in (4.3)-(4.6) are elliptic basisvectors.

We consider self-adjoint elements of the form

$$aH+BC𝔰𝔲(1,1),(a),$$

in the tensor product of a positive and a negative discrete series representation. For $`|a|=1`$ this is a parabolic element, for $`|a|<1`$ it is hyperbolic and for $`|a|>1`$ it is elliptic. For the elliptic and the parabolic case we refer to , respectively . We consider the case $`|a|<1`$.

### 4.2. Hyperbolic basisvectors

We consider a self-adjoint element $`X_\varphi `$ in $`𝔰𝔲(1,1)`$, given by

$$X_\varphi =\mathrm{cos}\varphi H+BC,0<\varphi <\pi .$$

The action of $`X_\varphi `$ in the discrete series can be identified with the three-term recurrence relation for the Meixner-Pollaczek polynomials (2.3), cf. \[15, Prop. 3.1\].

###### Proposition 4.2.

The operators $`\mathrm{\Theta }^\pm `$, defined by

$$\begin{array}{cc}\hfill \mathrm{\Theta }^\pm :\mathrm{}^2(_0)& L^2(,w^{(k)}(x;\varphi )dx)\hfill \\ \hfill e_n& P_n^{(k)}(;\varphi ),\hfill \end{array}$$

are unitary and intertwine $`\pi _k^\pm (X_\varphi )`$ with $`M(\pm 2x\mathrm{sin}\varphi )`$.

Here $`M`$ denotes the multiplication operator, i.e. $`M(f)g(x)=f(x)g(x)`$.

Proposition 4.2 states that

$$v^\pm (x)=\underset{n=0}{\overset{\mathrm{}}{}}P_n^{(k)}(x;\varphi )e_n,$$

are generalized eigenvectors of $`\pi _k^\pm (X_\varphi )`$ for eigenvalue $`\pm 2x\mathrm{sin}\varphi `$. These eigenvectors can be considered as hyperbolic basis vectors.

The action of $`X_\varphi `$ in the principal unitary series can be identified with the recurrence relation for the Meixner-Pollaczek functions (2.6). Then the spectral decomposition of the corresponding doubly infinite Jacobi operator gives the following.

###### Proposition 4.3.

The operator $`\mathrm{\Theta }^{\rho ,\epsilon }`$ defined by

$$\begin{array}{cc}\hfill \mathrm{\Theta }^{\rho ,\epsilon }:\mathrm{}^2()& (\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi ),\hfill \\ \hfill e_n& \left(\begin{array}{c}u_n(;\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )\\ u_n^{}(;\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )\end{array}\right),\hfill \end{array}$$

is unitary and intertwines $`\pi ^{\rho ,\epsilon }(X_\varphi )`$ with $`M(2x\mathrm{sin}\varphi )`$, and extends to a unitary equivalence.

From Propostion 4.3 we obtain that

$$\left(\begin{array}{c}v_{\rho ,\epsilon }(x)\\ v_{\rho ,\epsilon }^{}(x)\end{array}\right)=\underset{n=\mathrm{}}{\overset{\mathrm{}}{}}\left(\begin{array}{c}u_n(x;\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )\\ u_n^{}(x;\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )\end{array}\right)e_n$$

is a generalized eigenvector of $`\pi ^{\rho ,\epsilon }(X_\varphi )`$ for eigenvalue $`2x\mathrm{sin}\varphi `$.

Next we consider the action of $`X_\varphi `$ in the tensor product. Recall that in the tensor product we need the coproduct $`\mathrm{\Delta }`$, defined by $`\mathrm{\Delta }(Y)=1Y+Y1`$ for $`Y𝔰𝔲(1,1)`$. Then from Proposition 4.2 we find the following.

###### Proposition 4.4.

The operator $`\mathrm{{\rm Y}}`$ defined by

$$\begin{array}{cc}\hfill \mathrm{{\rm Y}}:\mathrm{}^2(_0)\mathrm{}^2(_0)& L^2(^2,w^{(k_1)}(x_1;\varphi )w^{(k_2)}(x_2;\varphi )dx_1dx_2),\hfill \\ \hfill e_{n_1}e_{n_2}& P_{n_1}^{(k_1)}(x_1;\varphi )P_{n_2}^{(k_2)}(x_2;\varphi ),\hfill \end{array}$$

is unitary and intertwines $`\pi _{k_1}^+\pi _{k_2}^{}\left(\mathrm{\Delta }(X_\varphi )\right)`$ with $`M\left(2(x_1x_2)\mathrm{sin}\varphi \right)`$.

In terms of the generalized eigenvectors $`v^+`$ and $`v^{}`$, we find from Proposition 4.4 that

$$v^+(x_1)v^{}(x_2)=\underset{n_1,n_2=0}{\overset{\mathrm{}}{}}P_{n_1}^{(k_1)}(x_1;\varphi )P_{n_2}^{(k_2)}(x_2;\varphi )e_{n_1}e_{n_2},$$

is a generalized eigenvector of $`\pi _{k_1}^+\pi _{k_2}^{}\left(\mathrm{\Delta }(X_\varphi )\right)`$ for eigenvalue $`2(x_1x_2)\mathrm{sin}\varphi `$.

To determine the action of $`\mathrm{{\rm Y}}`$ on the representation space of the direct integral representation $`^{}\pi ^{\rho ,\epsilon }𝑑\rho `$, we need to find the operator $`\stackrel{~}{\mathrm{{\rm Y}}}`$, such that $`\mathrm{{\rm Y}}=\stackrel{~}{\mathrm{{\rm Y}}}J`$. Here $`J`$ is the intertwiner defined in Theorem 4.1. For appropriate functions $`g_1`$ and $`g_2`$ we define an operator $`\stackrel{~}{\mathrm{{\rm Y}}}_g`$ by

$$\begin{array}{cc}\hfill \stackrel{~}{\mathrm{{\rm Y}}}_𝐠:L^2(0,\mathrm{})\mathrm{}^2()_{}^{}{}_{0}{}^{\mathrm{}}\mathrm{}^2()dx& _{}^{}{}_{0}{}^{\mathrm{}}(\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )d\rho ,\hfill \\ \hfill fe_n& _0^{\mathrm{}}f(\rho )\left(\begin{array}{c}g_1(\rho )\\ g_2(\rho )\end{array}\right)^{}\left(\begin{array}{c}u_n(t;\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )\\ u_n^{}(t;\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )\end{array}\right)𝑑\rho .\hfill \end{array}$$

From Proposition 4.3 we see that $`\stackrel{~}{\mathrm{{\rm Y}}}_𝐠`$ intertwines $`^{}\pi ^{\rho ,\epsilon }(X_\varphi )𝑑\rho `$ with $`M\left(2t\mathrm{sin}\varphi \right)`$. The functions $`g_1`$ and $`g_2`$ for which $`\mathrm{{\rm Y}}=\stackrel{~}{\mathrm{{\rm Y}}}_𝐠J`$ are the Clebsch-Gordan coefficients for the hyperbolic bases. To determine the Clebsch-Gordan coefficients we use the summation formula in Theorem 3.1. Define the continuous Hahn function by

$$\begin{array}{cc}\hfill \phi _\rho (x;t)=& \phi _\rho (x;t,k_1,k_2,\varphi )\hfill \\ \hfill =& \frac{e^{x(2\varphi \pi )}}{|\mathrm{\Gamma }(k_1+ix+it)|^2}_3F_2(.\genfrac{}{}{0.0pt}{}{k_2ix,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,k_2k_1+it+1}.;1).\hfill \end{array}$$
(4.9)
###### Theorem 4.5.

Let

$$\left(\begin{array}{c}g_1(\rho )\\ g_2(\rho )\end{array}\right)^{}=\left(\begin{array}{c}\phi _\rho (x_2;x_1x_2)\\ \phi _\rho ^{}(x_2;x_1x_2)\end{array}\right)^{}\left(\begin{array}{cc}m(\rho )& 0\\ 0& \overline{m(\rho )}\end{array}\right),$$

where $`\phi _\rho (x_2;x_1x_2)=\phi _\rho (x_2;x_1x_2,k_1,k_2,\varphi )`$ and $`m(\rho )`$ is given by

$$\begin{array}{cc}\hfill m(\rho )=& (i)^L\frac{(2\mathrm{sin}\varphi )^{2k_1L}e^{(x_2x_1)(2\varphi \pi )}}{\sqrt{2\pi }}\frac{\mathrm{\Gamma }(\frac{1}{2}+i(x_2x_1)+i\rho )\mathrm{\Gamma }(\frac{1}{2}+i(x_2x_1)i\rho )}{\mathrm{\Gamma }(k_2k_1+i(x_2x_1)+1)}\hfill \\ & \times \sqrt{\frac{\mathrm{\Gamma }(2k_1)}{\mathrm{\Gamma }(2k_2)}}\left|\frac{\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\rho )\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\rho )}{\mathrm{\Gamma }(2i\rho )}\right|,\hfill \end{array}$$

then we have $`\mathrm{{\rm Y}}=\stackrel{~}{\mathrm{{\rm Y}}}_𝐠J`$.

###### Proof.

To show that $`\mathrm{{\rm Y}}=\stackrel{~}{\mathrm{{\rm Y}}}_𝐠J`$, we use the summation formula of Theorem 3.1 with the orthogonal polynomials written in orthonormal form. We multiply by a continuous dual Hahn polynomial of degree $`n_2`$ with the same parameters as in Theorem 3.1, with $`p=n_1n_20`$. Then integrating against the corresponding orthogonality measure gives an equality with the following structure

$$P_{n_1}^{(k_1)}(x_1;\varphi )P_{n_2}^{(k_2)}(x_2;\varphi )=_0^{\mathrm{}}S_{n_2}(\rho )({}_{3}{}^{}F_{2}^{}{}_{2}{}^{}F_{1}^{}+{}_{3}{}^{}F_{2}^{}{}_{2}{}^{}F_{1}^{})𝑑\rho .$$

The $`{}_{2}{}^{}F_{1}^{}`$-functions are the Meixner-Pollaczek functions as defined by (2.5). From Proposition 4.2 we see that the left hand side is equal to $`\mathrm{{\rm Y}}(e_{n_1}e_{n_2})(x_1,x_2)`$. So from Theorem 4.1, with $`n_1n_2`$, and Proposition 4.3 it follows that the right hand side must be equal to

$$_0^{\mathrm{}}S_{n_2}(\rho )𝐠^{}(\rho )\mathrm{\Theta }^{\rho ,\epsilon }(e_{n_1n_2L})(x_1x_2)𝑑\mu ^{\frac{1}{2}}(\rho ),$$

where $`𝐠`$ is a vector containing the Clebsch-Gordan coefficients for the hyperbolic bases. This gives the desired result. For $`n_1n_2<0`$ the theorem follows after using (3.1). ∎

###### Remark 4.6.

The explicit expressions of the Clebsch-Gordan coefficients as $`{}_{3}{}^{}F_{2}^{}`$-series can also be found in Mukunda and Radhakrishnan . The method used in  is completely different from the method used here.

In terms of the generalized eigenvectors, Theorem 4.5 states

$$J\left(v^+(x_1)v^{}(x_2)\right)=_0^{\mathrm{}}\left(\begin{array}{c}\phi _\rho (x_2;x_1x_2)\\ \phi _\rho ^{}(x_2;x_1x_2)\end{array}\right)^{}\left(\begin{array}{cc}m(\rho )& 0\\ 0& \overline{m(\rho )}\end{array}\right)\left(\begin{array}{c}v_{\rho ,\epsilon }(x_1x_2)\\ v_{\rho ,\epsilon }^{}(x_1x_2)\end{array}\right)𝑑\rho .$$

### 4.3. The continuous Hahn integral transform

Since the continuous Hahn functions occur as Clebsch-Gordan coefficients for hyperbolic bases, they should satisfy (generalized) orthogonality relations. We find these relations by formal computations with the generalized eigenvectors.

For an element $`f\mathrm{}^2(_0)\mathrm{}^2(_0)`$ we have the transform pair

$$\{\begin{array}{cc}(\mathrm{{\rm Y}}f)(x_1,x_2)=f,v^+(x_1)v^{}(x_2),\hfill & \\ & \\ f=_^2(\mathrm{{\rm Y}}f)(x_1,x_2)v^+(x_1)v^{}(x_2)w^{(k_1)}(x_1;\varphi )w^{(k_2)}(x_2;\varphi )𝑑x_1𝑑x_2.\hfill & \end{array}$$
(4.10)
Similarly for $`f\mathrm{}^2()`$ we have the transform pair

$$\{\begin{array}{cc}(\mathrm{\Theta }^{\rho ,\epsilon }f)(x)=f,\left(\begin{array}{c}v_{\rho ,\epsilon }(x)\\ v_{\rho ,\epsilon }^{}(x)\end{array}\right)^2,\hfill & \\ & \\ f=\frac{1}{2\pi }_{}(\mathrm{\Theta }^{\rho ,\epsilon }f)(x)^{}\left(\begin{array}{cc}1& w_1(x)\\ w_1^{}(x)& 1\end{array}\right)\left(\begin{array}{c}v_{\rho ,\epsilon }(x)\\ v_{\rho ,\epsilon }^{}(x)\end{array}\right)w_0(x)𝑑x.\hfill & \end{array}$$
(4.11)
Denoting the intertwiner $`J`$ in Theorem 4.1 by $`J=_{}^{}{}_{0}{}^{\mathrm{}}J_\rho d\rho `$, we find from Theorem 4.5

$$(\mathrm{{\rm Y}}f)(x_1,x_2)=_0^{\mathrm{}}\left(\begin{array}{c}\phi _\rho (x_2;x_1x_2)\\ \phi _\rho ^{}(x_2;x_1x_2)\end{array}\right)^{}\left(\begin{array}{cc}m(\rho )& 0\\ 0& \overline{m(\rho )}\end{array}\right)\left(\mathrm{\Theta }^{\rho ,\epsilon }(J_\rho f)\right)(x_1x_2)𝑑\rho .$$

We want to invert this formula. From (4.10) and Theorem 4.5 we find formally

$$\begin{array}{cc}\hfill J_\rho f=_^2& (\mathrm{{\rm Y}}f)(x_1,x_2)w^{(k_1)}(x_1;\varphi )w^{(k_2)}(x_2;\varphi )\hfill \\ \hfill \times & \left(\begin{array}{c}\phi _\rho (x_2;x_1x_2)\\ \phi _\rho ^{}(x_2;x_1x_2)\end{array}\right)^{}\left(\begin{array}{cc}m(\rho )& 0\\ 0& \overline{m(\rho )}\end{array}\right)\left(\begin{array}{c}v_{\rho ,\epsilon }(x_1x_2)\\ v_{\rho ,\epsilon }^{}(x_1x_2)\end{array}\right)dx_1dx_2.\hfill \end{array}$$

We substitute $`x_1x+t`$, $`x_2x`$, then (4.11) with $`f`$ replaced by $`J_\rho f`$ gives

$$\begin{array}{cc}\hfill w_0(t)& \left(\begin{array}{cc}1& w_1^{}(t)\\ w_1(t)& 1\end{array}\right)\left(\mathrm{\Theta }^{\rho ,\epsilon }(J_\rho f)\right)(t)=\hfill \\ & 2\pi _{}(\mathrm{{\rm Y}}f)(x+t,x)\left(\begin{array}{cc}\overline{m(\rho )}& 0\\ 0& m(\rho )\end{array}\right)\left(\begin{array}{c}\phi _\rho (x;t)\\ \phi _\rho ^{}(x;t)\end{array}\right)w^{(k_1)}(x+t;\varphi )w^{(k_2)}(x;\varphi )𝑑x.\hfill \end{array}$$

We denote

$$(\mathrm{\Theta }^{\rho ,\epsilon }(J_\rho f))(t)=\frac{2\pi }{w_0(t)}\left(\begin{array}{cc}\overline{m(\rho )}& 0\\ 0& m(\rho )\end{array}\right)\left(\begin{array}{cc}1& w_1^{}(t)\\ w_1(t)& 1\end{array}\right)^1𝐠(\rho )$$

and $`(𝐠)(x)=(\mathrm{{\rm Y}}f)(x+t,x)`$, then we have the following integral transform pair with the continuous Hahn functions as a kernel

$$\{\begin{array}{cc}(𝐠)(x)=\frac{2\pi }{w_0(t)}_0^{\mathrm{}}\left(\begin{array}{c}\phi _\rho (x;t)\\ \phi _\rho ^{}(x;t)\end{array}\right)^{}\left(\begin{array}{cc}1& w_1^{}(t)\\ w_1(t)& 1\end{array}\right)𝐠(\rho )\frac{|m(\rho )|^2}{1|w_1(t)|^2}𝑑\rho \hfill & \\ & \\ 𝐠(\rho )=_{}(𝐠)(x)\left(\begin{array}{c}\phi _\rho ^{}(x;t)\\ \phi _\rho (x;t)\end{array}\right)w^{(k_1)}(x+t;\varphi )w^{(k_2)}(x;\varphi )𝑑x.\hfill & \end{array}$$

In section 5 we give a rigorous proof for this integral transform pair using spectral analysis of a difference operator for which the continuous Hahn functions are eigenfunctions. In the next subsection we obtain the difference operator from the action of the Casimir element in the tensor product.

### 4.4. A realization of the discrete series representations.

The following lemma is based on the fact that $`𝔰𝔩(2,)`$ is semi-simple, so $`[𝔰𝔩(2,),𝔰𝔩(2,)]=𝔰𝔩(2,).`$

###### Lemma 4.7.

$$B=\frac{1}{4}[H,X_\varphi ]+\frac{1}{2}X_\varphi +\frac{1}{2}\mathrm{cos}\varphi H,C=\frac{1}{4}[H,X_\varphi ]\frac{1}{2}X_\varphi \frac{1}{2}\mathrm{cos}\varphi H.$$

###### Proof.

This follows from the definition of $`X_\varphi `$ and the commutation relations (4.1). ∎

This lemma shows that to find the action of the generators $`H`$, $`B`$ and $`C`$ on the Meixner-Pollaczek polynomials, it is enough to find the action of $`H`$, since the action of $`X_\varphi `$ is known. The action of $`H`$ follows from the difference equation (2.4) for the Meixner-Pollaczek polynomials.

###### Proposition 4.8.

The operator $`\mathrm{\Theta }^+`$ intertwines the actions of the generators $`H`$, $`B`$, $`C`$ in the positive discrete series, with the following difference operators:

$$\begin{array}{cc}\hfill \mathrm{\Theta }^+\pi _k^+(H)& =\left[M\left(\frac{e^{i\varphi }}{i\mathrm{sin}\varphi }(kix)\right)T_i+M\left(2\frac{\mathrm{cos}\varphi }{\mathrm{sin}\varphi }x\right)+M\left(\frac{e^{i\varphi }}{i\mathrm{sin}\varphi }(k+ix)\right)T_i\right]\mathrm{\Theta }^+,\hfill \\ \hfill \mathrm{\Theta }^+\pi _k^+(B)& =\left[M\left(\frac{e^{2i\varphi }}{2i\mathrm{sin}\varphi }(kix)\right)T_i+M\left(\frac{x}{\mathrm{sin}\varphi }\right)+M\left(\frac{e^{2i\varphi }}{2i\mathrm{sin}\varphi }(k+ix)\right)T_i\right]\mathrm{\Theta }^+,\hfill \\ \hfill \mathrm{\Theta }^+\pi _k^+(C)& =\left[M\left(\frac{1}{2i\mathrm{sin}\varphi }(kix)\right)T_i+M\left(\frac{x}{\mathrm{sin}\varphi }\right)+M\left(\frac{1}{2i\mathrm{sin}\varphi }(k+ix)\right)T_i\right]\mathrm{\Theta }^+,\hfill \end{array}$$

where $`T`$ denotes the shift operator: $`T_af(x)=f(x+a)`$. For the negative discrete series, $`\mathrm{\Theta }^{}`$ intertwines the actions of $`H`$, $`B`$, $`C`$, with the following difference operators:

$$\begin{array}{cc}\hfill \mathrm{\Theta }^{}\pi _k^{}(H)& =\left[M\left(\frac{e^{i\varphi }}{i\mathrm{sin}\varphi }(kix)\right)T_i+M\left(2\frac{\mathrm{cos}\varphi }{\mathrm{sin}\varphi }x\right)+M\left(\frac{e^{i\varphi }}{i\mathrm{sin}\varphi }(k+ix)\right)T_i\right]\mathrm{\Theta }^{},\hfill \\ \hfill \mathrm{\Theta }^{}\pi _k^{}(B)& =\left[M\left(\frac{1}{2i\mathrm{sin}\varphi }(kix)\right)T_i+M\left(\frac{x}{\mathrm{sin}\varphi }\right)+M\left(\frac{1}{2i\mathrm{sin}\varphi }(k+ix)\right)T_i\right]\mathrm{\Theta }^{},\hfill \\ \hfill \mathrm{\Theta }^{}\pi _k^{}(C)& =\left[M\left(\frac{e^{2i\varphi }}{2i\mathrm{sin}\varphi }(kix)\right)T_i+M\left(\frac{x}{\mathrm{sin}\varphi }\right)+M\left(\frac{e^{2i\varphi }}{2i\mathrm{sin}\varphi }(k+ix)\right)T_i\right]\mathrm{\Theta }^{}.\hfill \end{array}$$

###### Proof.

We find the action of $`H`$ from the difference equation (2.4) for the Meixner-Pollaczek polynomials;

$$\begin{array}{cc}\hfill \mathrm{\Theta }^+\pi _k^+(H)e_n& =\mathrm{\Theta }^+(2n+2k)e_n=(2n+2k)P_n^{(k)}(x;\varphi )\hfill \\ & =\frac{e^{i\varphi }}{i\mathrm{sin}\varphi }(kix)P_n^{(k)}(x+i;\varphi )+2x\frac{\mathrm{cos}\varphi }{\mathrm{sin}\varphi }P_n^{(k)}(x;\varphi )\frac{e^{i\varphi }}{i\mathrm{sin}\varphi }(k+ix)P_n^{(k)}(xi;\varphi ).\hfill \end{array}$$

The action of $`X_\varphi `$ is given in Proposition 4.2: $`\mathrm{\Theta }^+\pi _k^+(X_\varphi )e_n=2x\mathrm{sin}\varphi P_n^{(k)}(x;\varphi )`$. Then Lemma 4.7 proves the proposition for the positive discrete series.

We find the action in the negative discrete series in the same way, or we use the Lie-algebra isomorphism $`\vartheta `$, given by

$$\vartheta (H)=H,\vartheta (B)=C,\vartheta (C)=B.$$

Then $`\pi _k^+\left(\vartheta (Y)\right)=\pi _k^{}(Y)`$ for $`Y𝔰𝔲(1,1)`$.

A straightforward calculation shows that these operators indeed satisfy the $`𝔰𝔲(1,1)`$ commutation relations. ∎

###### Remark 4.9.

To simplify notations we denote $`\mathrm{\Theta }^+\pi ^+(\mathrm{\Theta }^+)^{}`$ by $`\pi ^+`$, and similarly for $`\pi ^{}`$.

In the same way as in Proposition 4.8 it can be shown that $`\mathrm{\Theta }^{\rho ,\epsilon }`$ intertwines the actions of $`H`$, $`B`$ and $`C`$ in the principal unitary series with $`2\times 2`$ diagonal matrices with difference operators as elements. This is done by finding a difference equation for the Meixner-Pollaczek functions $`u_n`$ and $`u_n^{}`$, using contiguous relations for $`{}_{2}{}^{}F_{1}^{}`$-series. We do not need these realizations here, so we will not work this out.

To express $`\mathrm{\Delta }(\mathrm{\Omega })`$ in terms of $`H`$, $`B`$ and $`C`$, the coproduct $`\mathrm{\Delta }`$ is extended to $`𝒰\left(𝔰𝔲(1,1)\right)`$ as an algebra homomorphism. Then from the definition of the Casimir element (4.2) we find

$$\mathrm{\Delta }(\mathrm{\Omega })=1\mathrm{\Omega }+\mathrm{\Omega }1\frac{1}{2}HH(CB+BC).$$
(4.12)
Using this expression and Proposition 4.8, we find the following.

###### Proposition 4.10.

In the realizations of Proposition 4.8, we have

$$\begin{array}{cc}\hfill \pi _{k_1}^+& \pi _{k_2}^{}\left(\mathrm{\Delta }(\mathrm{\Omega })\right)|_{x_1=x+t,x_2=x}=\hfill \\ & M\left(e^{2i\varphi }\left(k_1+i(t+x)\right)\left(k_2+ix\right)\right)T_i+M\left(k_1(1k_1)+k_2(1k_2)2(x+t)x\right)\hfill \\ \hfill +& M\left(e^{2i\varphi }\left(k_1i(t+x)\right)\left(k_2ix\right)\right)T_i,\hfill \end{array}$$

where the shift operator $`T`$ acts with respect to $`x`$.

###### Proof.

Let $`F_1(x)`$ and $`F_2(x)`$ be polynomials in $`x`$, and let $`f(x,t)=F_1(x+t)F_2(x)`$, then a large but straightforward computation yields

$$\begin{array}{cc}\hfill \pi _{k_1}^+\pi _{k_2}^{}\left(\mathrm{\Delta }(\mathrm{\Omega })\right)f(x,t)=& [k_1(1k_1)+k_2(1k_2)2(x+t)x]f(x,t)\hfill \\ & e^{2i\varphi }\left(k_1+i(t+x)\right)\left(k_2+ix\right)f(xi,t)\hfill \\ & e^{2i\varphi }\left(k_1i(t+x)\right)\left(k_2ix\right)f(x+i,t).\hfill \end{array}$$

###### Remark 4.11.

The action of $`\mathrm{\Omega }`$ in the tensor product can also be found from

$`\pi _k^+(H)`$ $`=M(2ix),`$ $`\pi _k^{}(H)`$ $`=M(2ix),`$
$`\pi _k^+(B)`$ $`=M(kix)T_i,`$ $`\pi _k^{}(B)`$ $`=M\left(e^{2i\varphi }(k+ix)\right)T_i,`$
$`\pi _k^+(C)`$ $`=M(k+ix)T_i,`$ $`\pi _k^{}(C)`$ $`=M\left(e^{2i\varphi }(kix)\right)T_i.`$
These realizations are equivalent to the realizations given in Proposition 4.8.

In the next section we show that the continuous Hahn functions $`\phi _\rho (x;t,k_1,k_2,\varphi )`$ are eigenfunctions of the difference operator of Proposition 4.10, and we work out the corresponding integral transform.

## 5. The continuous Hahn integral transform

In this section we study a second order difference operator. This difference operator is obtained from the action of the Casimir operator on hyperbolic basis vectors in the tensor product of a positive and a negative discrete series representation of $`𝔰𝔲(1,1)`$, see section 4.2. The spectral analysis of this operator, leads to an integral transform pair with a certain type of $`{}_{3}{}^{}F_{2}^{}`$-series as a kernel. We call these $`{}_{3}{}^{}F_{2}^{}`$-series continuous Hahn functions, because of their similarity to continuous (dual) Hahn polynomials. The method we use is based on asymptotics, and is essentially the same method as used by Götze  and Braaksma and Meulenbeld  for the Jacobi function transform by approximating with the Fourier transform.

### 5.1. The difference operator $`𝚲`$ and the Wronskian

For $`k_1`$, $`k_2>0`$, $`t`$ and $`0<\varphi <\pi `$ the weight function $`w(x)`$ is defined by

$$w(x)=\frac{1}{2\pi }e^{(2x+t)(2\varphi \pi )}\mathrm{\Gamma }(k_1+it+ix)\mathrm{\Gamma }(k_1itix)\mathrm{\Gamma }(k_2+ix)\mathrm{\Gamma }(k_2ix).$$
(5.1)
The difference operator $`\mathrm{\Lambda }`$ is defined by

$$\mathrm{\Lambda }:g(x)\alpha _+(x)g(x+i)+\beta (x)g(x)+\alpha _{}(x)g(xi),$$

where

$$\begin{array}{cc}\hfill \alpha _\pm (x)& =e^{\pm 2i\varphi }(k_2ix)(k_1i(t+x)),\hfill \\ \hfill \beta (x)& =k_1(1k_1)+k_2(1k_2)2(t+x)x.\hfill \end{array}$$

Initially $`\mathrm{\Lambda }`$ is defined for those $`g(x)L^2(,w(x)dx)`$ that have an analytic continuation to the strip

$$𝒮_\epsilon =\{z:|\mathrm{}(z)|<1+\epsilon ,\epsilon >0\}.$$

The difference operator $`\mathrm{\Lambda }`$ corresponds to the action of the Casimir operator in the tensor product of a positive and a negative discrete series representation on hyperbolic basis vectors, see Proposition 4.10.

###### Remark 5.1.

Observe that the functions $`e^{x(2\varphi \pi )}p_n(x;k_1+it,k_2,k_1it,k_2)`$, where $`p_n`$ is a continuous Hahn polynomial in the notation of , form an orthogonal basis for $`L^2(,w(x)dx)`$, but they are not eigenfunctions of $`\mathrm{\Lambda }`$. Also, the functions $`e^{2\varphi x}p_n(x;k_1+it,k_2,k_1it,k_2)`$ are eigenfunctions of $`\mathrm{\Lambda }`$, but they are not elements of $`L^2(,w(x)dx)`$.

We define

$$f,g_{M,N}=_M^Nf(x)g^{}(x)w(x)𝑑x.$$

Then $`lim_{N,M\mathrm{}}f,g_{M,N}`$ is the inner product on $`L^2(,w(x)dx)`$, so $`f,g_{M,N}`$ is a truncated inner product.

###### Definition 5.2.

For functions $`f`$ and $`g`$ analytic in $`𝒮_\epsilon `$, the Wronskian $`[f,g]`$ is defined by

$$[f,g](y)=_y^{y+i}\left\{f(x)g^{}(xi)f(xi)g^{}(x)\right\}\alpha _{}(x)w(x)𝑑x.$$

###### Proposition 5.3.

Let $`f`$ and $`g`$ be analytic in $`𝒮_\epsilon `$, then

$$\mathrm{\Lambda }f,g_{N,M}f,\mathrm{\Lambda }g_{N,M}=[f,g](N)[f,g](M).$$

###### Proof.

Observe that $`\alpha _+^{}=\alpha ^{}`$, $`\alpha _\pm (xi)w(xi)=\alpha _\pm ^{}(x)w(x)`$ and $`\beta ^{}(x)=\beta (x)`$. Furthermore for $`x`$ we have $`g^{}(x\pm i)=\overline{g(xi)}`$. This gives for the Wronskian

$$\begin{array}{cc}\hfill [f,g](y)=& _y^{y+i}\left\{f(x)g^{}(xi)f(xi)g^{}(x)\right\}\alpha _{}(x)w(x)𝑑x\hfill \\ \hfill =& _y^{y+i}f(x)g^{}(xi)\alpha _+^{}(x)w(x)𝑑x_y^{y+i}f(xi)g^{}(x)\alpha _{}(x)w(x)𝑑x\hfill \\ \hfill =& _y^{y+i}f(x)g^{}(xi)\alpha _+(xi)w(xi)𝑑x_{yi}^yf(x)g^{}(x+i)\alpha _{}(x+i)w(x+i)𝑑x.\hfill \end{array}$$

For $`f`$ and $`g`$ analytic in $`𝒮_\epsilon `$, we find

$$\begin{array}{cc}\hfill _M^N& (\mathrm{\Lambda }f)(x)g^{}(x)w(x)dx=_M^N\left[\alpha _+(x)f(x+i)+\beta (x)f(x)+\alpha _{}(x)f(xi)\right]g^{}(x)w(x)𝑑x\hfill \\ \hfill =& _{M+i}^{N+i}\alpha _+(xi)f(x)g^{}(xi)w(xi)𝑑x+_M^N\beta (x)f(x)g^{}(x)w(x)𝑑x\hfill \\ & +_{Mi}^{Ni}\alpha _{}(x+i)f(x)g^{}(x+i)w(x+i)𝑑x.\hfill \end{array}$$

We choose a different path of integration

$$_{Mi}^{Ni}=_{Mi}^M+_M^N+_N^{Ni}$$

and similarly for $`_{M+i}^{N+i}`$. Then we find

$$\begin{array}{cc}\hfill _M^N& (\mathrm{\Lambda }f)(x)g^{}(x)w(x)dx=\hfill \\ & _{Mi}^M\alpha _{}(x+i)f(x)g^{}(x+i)w(x+i)𝑑x_{Ni}^N\alpha _{}(x+i)f(x)g^{}(x+i)w(x+i)𝑑x\hfill \\ \hfill +& _N^{N+i}\alpha _+(xi)f(x)g^{}(xi)w(xi)𝑑x_M^{M+i}\alpha _+(xi)f(x)g^{}(xi)w(xi)𝑑x\hfill \\ \hfill +& _M^Nf(x)\left[\alpha _{}(x+i)g^{}(x+i)w(x+i)+\beta (x)g^{}(x)w(x)+\alpha _+(xi)g^{}(xi)w(xi)\right]𝑑x.\hfill \end{array}$$

The last integral $`_M^N`$ equals

$$_M^Nf(x)\left[\alpha _{}^{}(x)g^{}(x+i)+\beta ^{}(x)g^{}(x)+\alpha _+^{}(x)g^{}(xi)\right]w(x)𝑑x=_M^Nf(x)(\mathrm{\Lambda }g)^{}(x)w(x)𝑑x,$$

and in the other four integrals we recognize $`[f,g](N)[f,g](M)`$. ∎

Our first goal is to show that $`\mathrm{\Lambda }`$ is a symmetric operator on a domain which will be specified, so we are interested in the limit of the Wronskian $`[f,g](y)`$ for $`y\pm \mathrm{}`$. The following lemma is useful in determining these limits.

###### Lemma 5.4.

Let $`k_1,k_2>1`$, $`x`$, and $`1y1`$, then the weight function $`w(x+iy)`$ has the following asymptotic behaviour

$$w(x+iy)=\{\begin{array}{cc}2\pi e^{2t(\varphi \pi )+4iy(\varphi \pi )}x^{2k_1+2k_22}e^{4(\pi \varphi )x}\left(1+𝒪\left(\frac{1}{x}\right)\right),\hfill & x\mathrm{},\hfill \\ 2\pi e^{2\varphi t+4i\varphi y}|x|^{2k_1+2k_22}e^{4\varphi x}\left(1+𝒪\left(\frac{1}{x}\right)\right),\hfill & x\mathrm{}.\hfill \end{array}$$

###### Proof.

From Stirling’s asymptotic formula \[21, (8.16)\] we find for $`u>0`$ and $`v`$

$$\mathrm{\Gamma }(u+iv)=\sqrt{2\pi }|v|^{u+iv\frac{1}{2}}e^{(u+iv)}e^{(iuv)\mathrm{arctan}(v/u)}+𝒪\left(\frac{1}{v}\right),v\pm \mathrm{}.$$

We use

$$\mathrm{arctan}\frac{v}{u}+\mathrm{arctan}\frac{u}{v}=\{\begin{array}{cc}\frac{\pi }{2},\hfill & v>0,\hfill \\ \frac{\pi }{2},\hfill & v<0,\hfill \end{array}$$

to find for $`v\pm \mathrm{}`$

$$(iuv)\mathrm{arctan}\frac{v}{u}=(iuv)\left(\pm \frac{\pi }{2}\frac{u}{v}\right)+𝒪\left(\frac{1}{v}\right).$$

So we have

$$\mathrm{\Gamma }(u+iv)=\sqrt{2\pi }|v|^{u+iv\frac{1}{2}}e^{\pi |v|/2iv\pm \pi iu/2}+𝒪\left(\frac{1}{v}\right),v\pm \mathrm{}.$$

Applying this formula to the four $`\mathrm{\Gamma }`$-functions in (5.1) gives the asymptotic behaviour of the weight function $`w(x+iy)`$. ∎

In general, if for some $`\epsilon >0`$

$$_{}e^{\epsilon |x|}𝑑\mu (x)<\mathrm{},$$

then the moment problem for the measure $`d\mu `$ is determinate, see e.g.  and references therein. Using this criterion with $`0<\epsilon <\mathrm{min}\{4\varphi ,4(\pi \varphi )\}`$, we find from Lemma 5.4 that the moment problem for the measure $`w(x)dx`$ is determinate. In particular this shows that the polynomials are dense in $`L^2(,w(x)dx)`$.

Let $`𝒟`$ be the space of polynomials on $``$, then $`𝒟`$ is a dense subspace of $`L^2(,w(x)dx)`$. Since

$$\alpha _{}(x+iy)=e^{2i\varphi }x^2\left(1+𝒪\left(\frac{1}{x}\right)\right),x\pm \mathrm{},$$
(5.2)
it follows from Definition 5.2 and Lemma 5.4 that $`lim_{N\pm \mathrm{}}[f,g](N)=0`$ for $`f`$, $`g`$ polynomials. Hence by Proposition 5.3 we find

###### Proposition 5.5.

The operator $`(\mathrm{\Lambda },𝒟)`$ is a densely defined symmetric operator on $`L^2(,w(x)dx)`$.

###### Remark 5.6.

The operator $`(\mathrm{\Lambda },𝒟)`$ is also densely defined and symmetric on the space spanned by $`e^{x(2\varphi \pi )}p_n(x)`$, where $`p_n`$ is a polynomial, cf. Remark 5.1.

### 5.2. Eigenfunctions of $`𝚲`$

We determine eigenfunctions of $`\mathrm{\Lambda }`$, using contiguous relations for $`{}_{3}{}^{}F_{2}^{}`$-functions. First note that for a monic polynomial of degree $`n`$, $`p_n(x)=x^n+\mathrm{}`$, we have

$$(\mathrm{\Lambda }p_n)(x)=\left[\alpha _+(x)+\beta (x)+\alpha _{}(x)\right]x^n+\text{lower order terms}.$$

Since $`\alpha _+(x)+\beta (x)+\alpha _{}(x)`$ is a polynomial of degree $`2`$, $`\mathrm{\Lambda }`$ raises the degree of a polynomial by $`2`$. Therefore $`\mathrm{\Lambda }`$ cannot have polynomial eigenfunctions.

Let $`p(x)`$ be the $`i`$-periodic function

$$p(x)=\frac{1}{\pi }e^{\pi x}\mathrm{sin}(\pi (k_1itix)),$$
(5.3)
and let $`\phi _\rho (x)=\phi _\rho (x;t,k_1,k_2,\varphi )`$ and $`\mathrm{\Phi }_\rho (x)=\mathrm{\Phi }_\rho (x;t,k_1,k_2,\varphi )`$ denote the functions

$`\phi _\rho (x)=`$ $`e^{2\varphi x}p(x){\displaystyle \frac{\mathrm{\Gamma }(1k_1+it+ix)}{\mathrm{\Gamma }(k_1+ix+it)}}_3F_2(.{\displaystyle \genfrac{}{}{0.0pt}{}{k_2ix,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,k_2k_1+it+1}}.;1),`$ (5.4)
$`\mathrm{\Phi }_\rho (x)=`$ $`e^{2\varphi x}p(x){\displaystyle \frac{\mathrm{\Gamma }(1k_1+it+ix)\mathrm{\Gamma }(1k_2+ix)}{\mathrm{\Gamma }(k_1+ix+it)\mathrm{\Gamma }(\frac{3}{2}k_1+ix+i\rho )}}`$
$`\times _3F_2(.{\displaystyle \genfrac{}{}{0.0pt}{}{k_2k_1+\frac{1}{2}+i\rho ,\frac{3}{2}k_1k_2+i\rho ,\frac{1}{2}it+i\rho }{1+2i\rho ,\frac{3}{2}k_1+ix+i\rho }}.;1).`$ (5.5)
Both $`{}_{3}{}^{}F_{2}^{}`$-series are absolutely convergent for $`\mathrm{}(k_1+it+ix)>0`$. Note that the expression for $`\phi _\rho (x)`$ is the same as (4.9) after applying Euler’s reflection formula.

###### Proposition 5.7.

For $`k_1>1`$, the functions $`\phi _\rho (x)`$ and $`\mathrm{\Phi }_\rho (x)`$ are eigenfunctions of $`\mathrm{\Lambda }`$ for eigenvalue $`(\rho ^2+\frac{1}{4})`$.

###### Remark 5.8.

(i) Observe that due to the $`i`$-periodic function $`p(x)`$, $`\phi _\rho (x)`$ is an entire function. Denote $`\stackrel{~}{\phi }_\rho (x)=\phi _\rho (x)/p(x)`$, then $`p(x)`$ cancels the poles of $`\stackrel{~}{\phi }_\rho (x)`$. There are more choices of $`i`$-periodic functions that cancel the poles of $`\stackrel{~}{\phi }_\rho (x)`$. One of the reasons for this particular choice, is that it appears in the Lie-algebraic interpretation of the function $`\phi _\rho (x)`$, see Theorem 4.5. We come back to the choice of the $`i`$-periodic function in Remark 5.15.

(ii) Observe that $`\phi _\rho ^{}(x)`$ is obtained from $`\phi _\rho (x)`$ by the substitutions $`(x,t,\varphi )(x,t,\pi \varphi )`$. Since the difference operator $`\mathrm{\Lambda }`$ and the weight function $`w(x)`$ are invariant under these substitutions, it follows from Proposition 5.7 that, for $`k_1>1`$, $`\phi _\rho ^{}(x)`$ is also an eigenfunction of $`\mathrm{\Lambda }`$ for eigenvalue $`(\rho ^2+\frac{1}{4})`$. A similar argument shows that $`\mathrm{\Phi }_\rho ^{}(x)`$ is an eigenfunction of $`\mathrm{\Lambda }`$ for eigenvalue $`(\rho ^2+\frac{1}{4})`$. Obviously $`\mathrm{\Phi }_\rho (x)`$ and $`\mathrm{\Phi }_\rho ^{}(x)`$ are also eigenfunctions of $`\mathrm{\Lambda }`$ for eigenvalue $`(\rho ^2+\frac{1}{4})`$.

###### Proof.

Combining the contiguous relations \[1, (3.7.9),(3.7.10),(3.7.13)\], gives

$$\begin{array}{c}\hfill (da)(ea)\left[F(a)F\right]+a(a+b+cde+1)\left[F(a+)F\right]=bcF,\\ \hfill F=_3F_2(.\genfrac{}{}{0.0pt}{}{a,b,c}{d,e}.;1),F(a\pm )=_3F_2(.\genfrac{}{}{0.0pt}{}{a\pm 1,b,c}{d,e}.;1).\end{array}$$

From this relation we find that

$$e^{2\varphi x}\frac{\mathrm{\Gamma }(1k_1+it+ix)}{\mathrm{\Gamma }(k_1+ix+it)}_3F_2(.\genfrac{}{}{0.0pt}{}{k_2ix,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,k_2k_1+it+1}.;1)$$

is an eigenfunction of $`\mathrm{\Lambda }`$ for eigenvalue $`\rho ^2+\frac{1}{4}`$. And then, since $`p(x)`$ is $`i`$-periodic, $`\phi _\rho (x)`$ is also an eigenfunction for eigenvalue $`\rho ^2+\frac{1}{4}`$. Since $`\phi _\rho (x)`$ must be analytic in $`𝒮_\epsilon `$, the condition $`k_1>1`$ is needed for absolute convergence of the $`{}_{3}{}^{}F_{2}^{}`$-series at the point $`x+i`$.

Denote

$$F_\pm (a)=_3F_2(.\genfrac{}{}{0.0pt}{}{a,b\pm 1,c\pm 1}{d\pm 1,e\pm 1}.;1).$$

From contiguous relations \[1, (3.7.9), (3.7.10), (3.7.13)\] we find

$$\begin{array}{c}\hfill \left[(d1)(e1)F_{}(a+)(b1)(c1)F\right](a1)(d+eabc)F\\ \hfill +\left[\frac{bc(da)(ea)}{de}F_+(a)(da)(ea)F\right]=0.\end{array}$$

From this we see that

$$\begin{array}{c}\mathrm{\Phi }_\rho (x)=e^{2\varphi x}p(x)\frac{\mathrm{\Gamma }(1k_2+ix)\mathrm{\Gamma }(1+2i\rho )\mathrm{\Gamma }(1k_1+ix+it)}{\mathrm{\Gamma }(\frac{1}{2}+it+i\rho )\mathrm{\Gamma }(k_1+\frac{1}{2}+i\rho +ix)\mathrm{\Gamma }(\frac{3}{2}k_1+ix+i\rho )}\hfill \\ \hfill \times _3F_2(.\genfrac{}{}{0.0pt}{}{\frac{1}{2}it+i\rho ,1k_2+ix,k_2+ix}{k_1+\frac{1}{2}+ix+i\rho ,\frac{3}{2}k_1+i\rho +ix}.;1),\end{array}$$
(5.6)
is another eigenfunction of $`\mathrm{\Lambda }`$ with eigenvalue $`\rho ^2+\frac{1}{4}`$. The $`{}_{3}{}^{}F_{2}^{}`$ series converges absolutely if $`\mathrm{}(\frac{1}{2}+i\rho +it)>0`$, so absolute convergence does not depend on $`k_1`$. Using \[1, Cor.3.3.5\] this can be written as (5.5), and for this expression the condition $`k_1>1`$ is needed. ∎

The function $`\phi _\rho (x)`$ can be expanded in terms of $`\mathrm{\Phi }_\rho (x)`$ and $`\mathrm{\Phi }_\rho (x)`$.

###### Proposition 5.9.

$$\phi _\rho (x)=c(\rho )\mathrm{\Phi }_\rho (x)+c(\rho )\mathrm{\Phi }_\rho (x),$$

where

$$c(\rho )=\frac{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(k_2k_1+it+1)\mathrm{\Gamma }(2i\rho )}{\mathrm{\Gamma }(k_1+k_2\frac{1}{2}i\rho )\mathrm{\Gamma }(\frac{1}{2}+iti\rho )\mathrm{\Gamma }(k_2k_1+\frac{1}{2}i\rho )}.$$

###### Proof.

This follows from \[2, p.15(2)\]

$$\begin{array}{cc}\hfill {}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{a,b,c}{d,e}.;1)=& \frac{\mathrm{\Gamma }(1a)\mathrm{\Gamma }(d)\mathrm{\Gamma }(e)\mathrm{\Gamma }(cb)}{\mathrm{\Gamma }(db)\mathrm{\Gamma }(eb)\mathrm{\Gamma }(1+ba)\mathrm{\Gamma }(c)}_3F_2(.\genfrac{}{}{0.0pt}{}{b,bd+1,be+1}{1+bc,1+ba}.;1)\hfill \\ & +\text{idem}(b;c).\hfill \end{array}$$

Here $`\text{idem}(b;c)`$ after an expression means that the expression is repeated with $`b`$ and $`c`$ interchanged. ∎

In the next subsection we consider the Wronskians $`[\phi _\rho ,\phi _\sigma ](y)`$ and $`[\phi _\rho ^{},\phi _\sigma ](y)`$ for $`y\pm \mathrm{}`$, so we need the asymptotic behaviour of $`\phi _\rho `$. We find this from Proposition 5.9 and the asymptotic behaviour of $`\mathrm{\Phi }_\rho `$.

###### Lemma 5.10.

Let $`k_1>1`$, $`\rho `$ and $`1y1`$. For $`x\pm \mathrm{}`$

$$\begin{array}{cc}\hfill \mathrm{\Phi }_\rho (x+iy)& =p(x+iy)e^{2\varphi (x+iy)}(ix)^{\frac{1}{2}k_1k_2i\rho }\left\{1+\frac{1}{2ix}\left(A(\rho )+yB(\rho )\right)+𝒪\left(\frac{1}{x^2}\right)\right\},\hfill \\ \hfill \mathrm{\Phi }_\rho ^{}(x+iy)& =p^{}(x+iy)e^{2\varphi (x+iy)}(ix)^{\frac{1}{2}k_1k_2+i\overline{\rho }}\left\{1+\frac{1}{2ix}\left(\overline{A(\rho )}y\overline{B(\rho )}\right)+𝒪\left(\frac{1}{x^2}\right)\right\},\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill A(\rho )=& 2it(12k_1)+(k_1k_2\frac{1}{2}i\rho )(\frac{3}{2}k_1k_2+i\rho )\hfill \\ & +\frac{(k_2k_1+\frac{1}{2}+i\rho )(\frac{3}{2}k_1k_2+i\rho )(\frac{1}{2}it+i\rho )}{\frac{1}{2}+i\rho },\hfill \\ \hfill B(\rho )=& 2k_1+2k_21+2i\rho .\hfill \end{array}$$

###### Proof.

This follows from (5.5) and the asymptotic formula for the ratio of two $`\mathrm{\Gamma }`$-functions \[21, §4.5\]

$$\frac{\mathrm{\Gamma }(a+z)}{\mathrm{\Gamma }(b+z)}=z^{ab}\left(1+\frac{1}{2z}(ab)(a+b1)+𝒪\left(\frac{1}{z^2}\right)\right),|z|\mathrm{},|\mathrm{arg}(z)|<\pi .$$
(5.7)
The first part of the expression for $`A(\rho )`$ is obtained from

$$(a_1b_1)(a_1+b_11)+(a_2b_2)(a_2+b_21),$$

the second part comes from the second term in the hypergeometric series. ∎

The asymptotic behaviour of the $`i`$-periodic function $`p(x)`$ is also needed;

$$p(x+iy)=\frac{e^{i\pi (k_1+2y)}}{2\pi i}e^{\pi (2x+t)}\frac{e^{i\pi k_1}}{2\pi i}e^{\pi t}=\{\begin{array}{cc}\frac{e^{i\pi (k_1+2y)}}{2\pi i}e^{\pi (2x+t)}+𝒪(1),\hfill & x\mathrm{},\hfill \\ & \\ \frac{e^{i\pi k_1}}{2\pi i}e^{\pi t}+𝒪(e^{2\pi x}),\hfill & x\mathrm{},\hfill \end{array}$$

$$p^{}(x+iy)=\frac{e^{i\pi k_1}}{2\pi i}e^{\pi t}\frac{e^{i\pi (k_12y)}}{2\pi i}e^{\pi (2x+t)}=\{\begin{array}{cc}\frac{e^{i\pi (k_12y)}}{2\pi i}e^{\pi (2x+t)}+𝒪(1),\hfill & x\mathrm{},\hfill \\ & \\ \frac{e^{i\pi k_1}}{2\pi i}e^{\pi t}+𝒪(e^{2\pi x}),\hfill & x\mathrm{}.\hfill \end{array}$$

### 5.3. Continuous spectrum

We determine the spectrum of the difference operator $`\mathrm{\Lambda }`$. In this subsection we consider the case where the spectrum only consists of a continuous part.

Since

$$|(ix)^{\frac{1}{2}k_1k_2i\rho }|^2=\{\begin{array}{cc}|x|^{12k_12k_2+2\mathrm{}(\rho )}e^{\pi \mathrm{}(\rho )},\hfill & x>0,\hfill \\ |x|^{12k_12k_2+2\mathrm{}(\rho )}e^{\pi \mathrm{}(\rho )},\hfill & x<0,\hfill \end{array}$$

we find from Proposition 5.10

$$|\mathrm{\Phi }_\rho (x)|^2w(x)=𝒪(|x|^{2\mathrm{}(\rho )1}),x\pm \mathrm{}.$$
(5.8)
So $`\mathrm{\Phi }_\rho (x)`$ is an element of $`L^2(,w(x)dx)`$ for $`\mathrm{}(\rho )<0`$. This shows that it is possible to give eigenfunctions of $`\mathrm{\Lambda }`$ for complex eigenvalues. We only consider eigenfunctions which are even in $`\rho `$, and in that case all eigenvalues of $`\mathrm{\Lambda }`$ are real.

First we consider the continuous spectrum of $`\mathrm{\Lambda }`$. We show that $`[\frac{1}{4},\mathrm{})`$ is contained in the continuous spectrum. Assume that $`\rho `$ is real and that the $`c`$-function in Proposition 5.9 does not have zeros, or, equivalently, assume that $`k_1+k_2\frac{1}{2}`$ and $`k_2k_1>\frac{1}{2}`$. Since we only consider even functions in $`\rho `$, we may assume $`\rho 0`$. We use Proposition 5.3 to calculate the truncated inner product of two eigenfunctions. This gives for $`k_1>1`$ and $`\rho \sigma `$

$$\phi _\rho ,\phi _\sigma _{M,N}=\frac{[\phi _\rho ,\phi _\sigma ](N)[\phi _\rho ,\phi _\sigma ](M)}{\rho ^2\sigma ^2}.$$
(5.9)
Multiplying both sides with an arbitrary function $`f(\rho )`$ and integrating over $`\rho `$ from $`0`$ to $`\mathrm{}`$, gives

$$_0^{\mathrm{}}f(\rho )\phi _\rho ,\phi _\sigma _{M,N}𝑑\rho =_0^{\mathrm{}}f(\rho )\frac{[\phi _\rho ,\phi _\sigma ](N)[\phi _\rho ,\phi _\sigma ](M)}{\rho ^2\sigma ^2}𝑑\rho .$$
(5.10)
The function $`f(\rho )`$ must satisfy some conditions, which we shall determine later on. We take limits $`N,M\mathrm{}`$ on both sides. To determine the limits of the Wronskians, the following lemma is used.

###### Lemma 5.11.

Let $`k_1,k_2>1`$ and $`\rho ,\sigma 0`$. For for $`x\pm \mathrm{}`$

$$[\mathrm{\Phi }_\rho ,\mathrm{\Phi }_\sigma ](x)=\pm D^\pm (\rho ,\sigma )(\rho +\sigma )|x|^{i(\sigma \rho )}\left(1+𝒪(\frac{1}{x})\right),$$

where

$$D^\pm (\rho ,\sigma )=\frac{i}{2\pi }e^{t(2\varphi \pi )}e^{\pm \frac{1}{2}\pi (\rho +\sigma +2t)}.$$

###### Proof.

From Lemma 5.10 we find for $`0y1`$ and $`x\mathrm{}`$

$$\begin{array}{cc}\hfill \mathrm{\Phi }_\rho (x+iy)& \mathrm{\Phi }_\sigma ^{}(x+iyi)\mathrm{\Phi }_\rho (x+iyi)\mathrm{\Phi }_\sigma ^{}(x+iy)=\hfill \\ & e^{2i\varphi 4i\varphi y+4i\pi y4\varphi x}|x|^{12k_12k_2}|p(x)|^2(ix)^{i\rho }(ix)^{i\sigma }\frac{B(\rho )\overline{B(\sigma )}}{2ix}\left(1+𝒪(\frac{1}{x})\right)\hfill \\ \hfill =& e^{2i\varphi 4i\varphi y+4i\pi y4\varphi x}|x|^{12k_12k_2}|p(x)|^2e^{\pm \frac{1}{2}\pi (\rho +\sigma )}|x|^{i(\sigma \rho )}\frac{\rho +\sigma }{x}\left(1+𝒪(\frac{1}{x})\right).\hfill \end{array}$$

Using the asymptotic behaviour of $`p(x)`$ and Lemma 5.4, we find the asymptotic behaviour of the integrand of the Wronskian. Note that this is independent of $`y`$. In a similar way we find the same asymptotic behaviour of the integrand for $`x\mathrm{}`$. Now the lemma follows from writing the Wronkian as

$$_x^{x+i}f(z)𝑑z=i_0^1f(x+is)𝑑s,$$

and applying dominated convergence. ∎

###### Proposition 5.12.

Let $`\sigma 0`$, and let $`f`$ be a continuous function satisfying

$$f(\rho )=\{\begin{array}{cc}𝒪(e^{\pi \rho }\rho ^{2k_2\frac{1}{2}\epsilon }),\hfill & \rho \mathrm{},\epsilon >0,\hfill \\ 𝒪(\rho ^\delta ),\hfill & \rho 0,\delta >0,\hfill \end{array}$$

then

$$\underset{N\mathrm{}}{lim}_0^{\mathrm{}}f(\rho )\phi _\rho ,\phi _\sigma _{N,N}𝑑\rho =W_0^1(\sigma )f(\sigma ),$$

where

$$W_0(\sigma )=\frac{1}{2\pi }e^{t(2\varphi \pi )}\left|\frac{\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\sigma )\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\sigma )\mathrm{\Gamma }(\frac{1}{2}+it+i\sigma )\mathrm{\Gamma }(\frac{1}{2}it+i\sigma )}{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(k_2k_1+it+1)\mathrm{\Gamma }(2i\sigma )}\right|^2.$$

###### Proof.

Let $`k_1,k_2>1`$. We use (5.10) to calculate $`lim_N\mathrm{}_0^{\mathrm{}}f(\rho )\phi _\rho ,\phi _\sigma _{N,N}𝑑\rho `$. From the $`c`$-function expansion, see Proposition 5.9, we obtain

$$[\phi _\rho ,\phi _\sigma ](x)=\underset{ϵ,\xi \{1,1\}}{}c(ϵ\rho )\overline{c(\xi \sigma )}[\mathrm{\Phi }_{ϵ\rho },\mathrm{\Phi }_{\xi \sigma }](x).$$

Then (5.9) and Lemma 5.11 give, for $`N\mathrm{}`$,

$$\begin{array}{c}\hfill \phi _\rho ,\phi _\sigma _{N,N}=\underset{ϵ,\xi \{1,1\}}{}\frac{N^{i(\xi \sigma ϵ\rho )}}{ϵ\rho \xi \sigma }c(ϵ\rho )\overline{c(\xi \sigma )}\left[D^+(ϵ\rho ,\xi \sigma )+D^{}(ϵ\rho ,\xi \sigma )\right]\left(1+𝒪\left(\frac{1}{N}\right)\right).\end{array}$$

From (5.10) we find

$$\begin{array}{cc}\hfill \underset{N\mathrm{}}{lim}_0^{\mathrm{}}f(\rho )& \phi _\rho ,\phi _\sigma _{N,N}d\rho =\hfill \\ \hfill \underset{N\mathrm{}}{lim}_0^{\mathrm{}}f(\rho )& \{\psi _1(\rho )\mathrm{cos}\left([\rho +\sigma ]\mathrm{ln}N\right)+\psi _2(\rho )\mathrm{sin}\left([\rho +\sigma ]\mathrm{ln}N\right)\hfill \\ & +\psi _3(\rho )\mathrm{cos}\left([\rho \sigma ]\mathrm{ln}N\right)+\psi _4(\rho )\frac{\mathrm{sin}\left([\rho \sigma ]\mathrm{ln}N\right)}{\rho \sigma }\}d\rho ,\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill \psi _1(\rho )& =\frac{1}{\rho +\sigma }\left[c(\rho )\overline{c(\sigma )}\left(D^+(\rho ,\sigma )+D^{}(\rho ,\sigma )\right)c(\rho )\overline{c(\sigma )}\left(D^+(\rho ,\sigma )+D^{}(\rho ,\sigma )\right)\right],\hfill \\ \hfill \psi _2(\rho )& =\frac{i}{\rho +\sigma }\left[c(\rho )\overline{c(\sigma )}\left(D^+(\rho ,\sigma )+D^{}(\rho ,\sigma )\right)+c(\rho )\overline{c(\sigma )}\left(D^+(\rho ,\sigma )+D^{}(\rho ,\sigma )\right)\right],\hfill \\ \hfill \psi _3(\rho )& =\frac{1}{\rho \sigma }\left[c(\rho )\overline{c(\sigma )}\left(D^+(\rho ,\sigma )+D^{}(\rho ,\sigma )\right)c(\rho )\overline{c(\sigma )}\left(D^+(\rho ,\sigma )+D^{}(\rho ,\sigma )\right)\right],\hfill \\ \hfill \psi _4(\rho )& =i\left[c(\rho )\overline{c(\sigma )}\left(D^+(\rho ,\sigma )+D^{}(\rho ,\sigma )\right)+c(\rho )\overline{c(\sigma )}\left(D^+(\rho ,\sigma )+D^{}(\rho ,\sigma )\right)\right].\hfill \end{array}$$

Writing out explicitly the terms between square brackets for $`\psi _3`$ gives, for $`\rho =\sigma `$,

$$e^{t(2\varphi \pi )}\left|\frac{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(k_2k_1+it+1)\mathrm{\Gamma }(2i\rho )}{\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\rho )\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\rho )}\right|^2\left(\frac{\mathrm{sin}\left(\pi (\frac{1}{2}+it+i\rho )\right)}{|\mathrm{\Gamma }(\frac{1}{2}it+i\rho )|^2}\frac{\mathrm{sin}\left(\pi (\frac{1}{2}it+i\rho )\right)}{|\mathrm{\Gamma }(\frac{1}{2}+it+i\rho )|^2}\right).$$

From Euler’s reflection formula it follows that this is equal to zero. So $`\psi _3`$ has a removable singularity at the point $`\rho =\sigma `$.

From the Riemann-Lebesgue lemma, see e.g. \[28, §9.41\], it follows that for $`f\psi _iL^1(0,\mathrm{})`$, $`i=1,2,3`$, the terms with $`\psi _i`$, $`i=1,2,3`$, vanish. This leaves us with a Dirichlet integral, for which we have the property (see e.g. \[28, §9.7\])

$$\underset{t\mathrm{}}{lim}\frac{1}{\pi }_0^{\mathrm{}}g(x)\frac{\mathrm{sin}[t(xy)]}{xy}𝑑x=g(y),$$
(5.11)
for a continuous function $`gL^1(0,\mathrm{})`$. This gives for a continuous function $`f`$ that satisfies $`f\psi _4^\pm L^1(0,\mathrm{})`$,

$$\begin{array}{cc}\hfill \underset{N\mathrm{}}{lim}& _0^{\mathrm{}}f(\rho )\phi _\rho ,\phi _\sigma _{N,N}𝑑\rho =\pi f(\sigma )\psi _4(\sigma )\hfill \\ \hfill =& \frac{1}{2}e^{t(2\varphi \pi )}\{(e^{\pi (\sigma +t)}+e^{\pi (\sigma +t)})c(\sigma )\overline{c(\sigma )}+(e^{\pi (\sigma t)}+e^{\pi (\sigma t)})c(\sigma )\overline{c(\sigma )})\}f(\sigma )\hfill \\ \hfill =& e^{t(2\varphi \pi )}\left|\frac{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(k_2k_1+it+1)\mathrm{\Gamma }(2i\sigma )}{\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\sigma )\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\sigma )}\right|^2\hfill \\ & \times \left(\frac{\mathrm{sin}\left(\pi (\frac{1}{2}+it+i\sigma )\right)}{|\mathrm{\Gamma }(\frac{1}{2}it+i\sigma )|^2}+\frac{\mathrm{sin}\left(\pi (\frac{1}{2}it+i\sigma )\right)}{|\mathrm{\Gamma }(\frac{1}{2}+it+i\sigma )|^2}\right)f(\sigma )\hfill \\ \hfill =& 2\pi e^{t(2\varphi \pi )}\left|\frac{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(k_2k_1+it+1)\mathrm{\Gamma }(2i\sigma )}{\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\sigma )\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\sigma )\mathrm{\Gamma }(\frac{1}{2}+it+i\sigma )\mathrm{\Gamma }(\frac{1}{2}it+i\sigma )}\right|^2f(\sigma )\hfill \\ \hfill =& W_0^1(\sigma )f(\sigma ).\hfill \end{array}$$

In the last step Euler’s reflection formula is used. The conditions $`k_1,k_2>1`$ can be removed by analytic continuation. Since

$$W_0^1(\sigma )=\{\begin{array}{cc}𝒪(\sigma ^{14k_2}e^{2\pi \sigma }),\hfill & \sigma \mathrm{},\hfill \\ 𝒪(1),\hfill & \sigma 0,\hfill \end{array}$$

we find that the proposition is valid for the conditions of $`f`$ as stated. ∎

Next we consider the truncated inner product $`\phi _\rho ^{},\phi _\sigma _{M,N}`$ and the corresponding Wronskians. We need to find the analogues of Lemma 5.11 and Proposition 5.12.

###### Lemma 5.13.

Let $`k_1>1`$ and $`\rho ,\sigma 0`$. For $`x\pm \mathrm{}`$

$$[\mathrm{\Phi }_\rho ^{},\mathrm{\Phi }_\sigma ](x)=E^\pm (\rho ,\sigma )(\rho \sigma )|x|^{i(\rho +\sigma )}\left(1+𝒪(\frac{1}{x})\right),$$

where

$$E^\pm (\rho ,\sigma )=\frac{1}{2\pi }e^{t(2\varphi \pi )}e^{\pm \frac{1}{2}\pi i(2k_12k_2+i\rho +i\sigma +2it)}.$$

###### Proof.

The proof is similar to the proof of Lemma 5.11. ∎

###### Proposition 5.14.

Let $`\sigma 0`$, and let $`f`$ be an even continuous function satisfying

$$f(\rho )=\{\begin{array}{cc}𝒪(e^{\pi \rho }\rho ^{2k_2\frac{1}{2}\epsilon }),\hfill & \rho \mathrm{},\epsilon >0,\hfill \\ 𝒪(\rho ^\delta ),\hfill & \rho 0,\delta >0,\hfill \end{array}$$

then

$$\underset{N\mathrm{}}{lim}_0^{\mathrm{}}f(\rho )\phi _\rho ^{},\phi _\sigma _{N,N}𝑑\rho =W_1^1(\sigma )f(\sigma ),$$

where

$$\begin{array}{cc}\hfill W_1(\sigma )=& \frac{1}{2\pi }e^{t(2\varphi \pi )}\frac{\mathrm{\Gamma }(k_1k_2+it)\mathrm{\Gamma }(\frac{1}{2}iti\sigma )\mathrm{\Gamma }(\frac{1}{2}it+i\sigma )}{\mathrm{\Gamma }(k_2k_1it+1)\mathrm{\Gamma }(2k_2)^2}\hfill \\ & \times \left|\frac{\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\sigma )\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\sigma )}{\mathrm{\Gamma }(2i\sigma )}\right|^2.\hfill \end{array}$$

###### Proof.

The proof runs along the same lines as the proof of Proposition 5.12, therefore we leave out the details.

As in the proof of Proposition 5.12 we find from Lemma 5.13

$$\begin{array}{cc}\hfill \underset{N\mathrm{}}{lim}_0^{\mathrm{}}f(\rho )& \phi _\rho ^{},\phi _\sigma _{N,N}d\rho =\hfill \\ \hfill \underset{N\mathrm{}}{lim}_0^{\mathrm{}}f(\rho )& \{\psi _1(\rho )\mathrm{cos}\left([\rho +\sigma ]\mathrm{ln}N\right)+\psi _2(\rho )\mathrm{sin}\left([\rho +\sigma ]\mathrm{ln}N\right)\hfill \\ & +\psi _3(\rho )\mathrm{cos}\left([\rho \sigma ]\mathrm{ln}N\right)+\psi _4(\rho )\frac{\mathrm{sin}\left([\rho \sigma ]\mathrm{ln}N\right)}{\rho \sigma }\}d\rho ,\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill \psi _1(\rho )& =\frac{1}{\rho +\sigma }\left[\overline{c(\rho )}\overline{c(\sigma )}\left(E^+(\rho ,\sigma )E^{}(\rho ,\sigma )\right)\overline{c(\rho )}\overline{c(\sigma )}\left(E^+(\rho ,\sigma )E^{}(\rho ,\sigma )\right)\right],\hfill \\ \hfill \psi _2(\rho )& =\frac{i}{\rho +\sigma }\left[\overline{c(\rho )}\overline{c(\sigma )}\left(E^+(\rho ,\sigma )E^{}(\rho ,\sigma )\right)+\overline{c(\rho )}\overline{c(\sigma )}\left(E^+(\rho ,\sigma )E^{}(\rho ,\sigma )\right)\right],\hfill \\ \hfill \psi _3(\rho )& =\frac{1}{\rho \sigma }\left[\overline{c(\rho )}\overline{c(\sigma )}\left(E^+(\rho ,\sigma )E^{}(\rho ,\sigma )\right)\overline{c(\rho )}\overline{c(\sigma )}\left(E^+(\rho ,\sigma )E^{}(\rho ,\sigma )\right)\right],\hfill \\ \hfill \psi _4(\rho )& =i\left[\overline{c(\rho )}\overline{c(\sigma )}\left(E^+(\rho ,\sigma )E^{}(\rho ,\sigma )\right)+\overline{c(\rho )}\overline{c(\sigma )}\left(E^+(\rho ,\sigma )E^{}(\rho ,\sigma )\right)\right].\hfill \end{array}$$

For $`\rho =\sigma `$ the term between the square brackets for $`\psi _3`$ is equal to zero, so $`\psi _3`$ has a removable singularity at the point $`\rho =\sigma `$. So, for $`f\psi _iL^1(0,\mathrm{})`$, $`i=1,2,3`$, the terms with $`\psi _i`$, $`i=1,2,3`$, vanish by the Riemann-Lebesgue lemma. This leaves us with a Dirichlet integral. Then, after applying (5.11), we find for $`f\psi _4L^1(0,\mathrm{})`$

$$\underset{N\mathrm{}}{lim}_0^{\mathrm{}}f(\rho )\phi _\rho ^{},\phi _\sigma _{N,N}𝑑\rho =\pi \psi _4(\sigma )f(\sigma ).$$

Writing out $`\psi _4`$ explicitly gives the result. ∎

###### Remark 5.15.

In Remark 5.8 we observed that the $`i`$-periodic function $`p(x)`$ cancels the poles of $`\stackrel{~}{\phi }_\rho (x)`$. Other obvious choices with the same property would be $`e^{2k\pi x}p(x)`$, for $`k`$. However for $`k0`$, the method we used here to find an integral transform pair would fail, since the method depends on the use of the Riemann-Lebesgue lemma and the Dirichlet kernel, which can no longer be used in case $`k0`$. This can e.g. be seen from Lemma 5.11, where the terms in front of $`|x|^{i(\sigma \rho )}`$ would contain a factor $`e^{2k\pi x}`$. So this gives a heuristic argument for the choice (5.3) of the $`i`$-periodic function.

Let $`f`$ be a continuous function satisfying,

$$f(\rho )=\{\begin{array}{cc}𝒪(\rho ^{2k_2\epsilon }e^{\pi \rho }),\hfill & \rho \mathrm{},\epsilon >0,\hfill \\ 𝒪(\rho ^\delta ),\hfill & \rho 0,\delta >0,\hfill \end{array}$$
(5.12)
and let $`𝐟`$ be the vector

$$𝐟(\rho )=\left(\begin{array}{c}\overline{f(\rho )}\\ f(\rho )\end{array}\right).$$

We define, for $`x`$, an operator $`\stackrel{~}{}`$ by

$$(\stackrel{~}{}𝐟)(x)=_0^{\mathrm{}}\left(\phi _\rho (x)\overline{f(\rho )}+\phi _\rho ^{}(x)f(\rho )\right)W_0(\rho )𝑑\rho .$$
(5.13)
To verify that this is a well-defined expression, we determine the behaviour of $`\phi _\rho (x)`$ and $`W_0(\rho )`$ for $`\rho \mathrm{}`$ and $`\rho 0`$. From Thomae’s transformation \[1, Cor.3.3.6\] we find

$$\phi _\rho (x)=\{\begin{array}{cc}𝒪(\rho ^{2k_2}e^{\pi \rho }),\hfill & \rho \mathrm{},\hfill \\ 𝒪(1),\hfill & \rho 0.\hfill \end{array}$$

And from (5.7) we obtain

$$W_0(\rho )=\{\begin{array}{cc}𝒪(\rho ^{4k_21}e^{2\pi \rho }),\hfill & \rho \mathrm{},\hfill \\ 𝒪(1),\hfill & \rho 0.\hfill \end{array}$$
(5.14)
Then we see that the integral in (5.13) converges absolutely for $`f`$ satisfying (5.12).

For a continuous function $`g`$ satisfying

$$g(x)=\{\begin{array}{cc}𝒪\left(|x|^{\frac{1}{2}k_1k_2\epsilon }e^{2(\pi \varphi )x}\right),\hfill & x\mathrm{},\epsilon >0,\hfill \\ 𝒪\left(|x|^{\frac{1}{2}k_1k_2\delta }e^{2\varphi x}\right),\hfill & x\mathrm{},\delta >0,\hfill \end{array}$$
(5.15)
we define, for $`\rho 0`$, an operator $`𝒢`$ by

$$(𝒢g)(\rho )=_{}g(x)\left(\begin{array}{c}\phi _\rho ^{}(x)\\ \phi _\rho (x)\end{array}\right)w(x)𝑑x.$$
(5.16)
From the asymptotic behaviour of $`\phi _\rho ^{}(x)`$ and $`w(x)`$ for $`x\pm \mathrm{}`$, see Lemmas 5.4 and 5.10, it follows that the integral in (5.16) converges absolutely.

###### Proposition 5.16.

If $`g=\stackrel{~}{}𝐟`$, and $`g`$ satisfies the conditions (5.15), then

$$(𝒢g)(\rho )=\left(\begin{array}{cc}1& W_0(\rho )/W_1(\rho )\\ W_0(\rho )/\overline{W_1(\rho )}& 1\end{array}\right)\left(\begin{array}{c}\overline{f(\rho )}\\ f(\rho )\end{array}\right).$$

###### Proof.

For a function $`g`$ satisfying (5.15) we define operators $`𝒢_1`$ and $`𝒢_2`$ by

$$\begin{array}{cc}\hfill (𝒢_1g)(\rho )& =_{}g(x)\phi _\rho ^{}(x)w(x)𝑑x,\hfill \\ \hfill (𝒢_2g)(\rho )& =_{}g(x)\phi _\rho (x)w(x)𝑑x,\hfill \end{array}$$
(5.17)
then we have

$$(𝒢g)(\rho )=\left(\begin{array}{c}(𝒢_1g)(\rho )\\ (𝒢_2g)(\rho )\end{array}\right).$$

If $`g(x)=(\stackrel{~}{}f)(x)`$ satisfies the conditions (5.15), then the integral

$$(𝒢_1g)(\sigma )=_{}(\stackrel{~}{}f)(x)\phi _\sigma ^{}(x)w(x)𝑑x$$

converges absolutely. So from (5.13) we obtain

$$(𝒢_1g)(\sigma )=\underset{N\mathrm{}}{lim}_N^N\phi _\sigma ^{}(x)\left\{_0^{\mathrm{}}\phi _\rho (x)\overline{f(\rho )}W_0(\rho )𝑑\rho +_0^{\mathrm{}}\phi _\rho ^{}(x)f(\rho )W_0(\rho )𝑑\rho \right\}w(x)𝑑x,$$

and interchanging integration gives

$$(𝒢_1g)(\sigma )=\underset{N\mathrm{}}{lim}_0^{\mathrm{}}\overline{f(\rho )}W_0(\rho )\phi _\rho ,\phi _\sigma _{N,N}𝑑\rho +\underset{N\mathrm{}}{lim}_0^{\mathrm{}}f(\rho )W_0(\rho )\phi _\rho ^{},\phi _\sigma _{N,N}𝑑\rho .$$

From (5.12) and (5.14) it follows that the functions $`\overline{f(\rho )}W_0(\rho )`$, $`f(\rho )W_0(\rho )`$ satisfy the conditions for Propositions 5.12, 5.14 respectively. Applying the propositions gives $`(𝒢_1g)(\sigma )=\overline{f(\sigma )}+f(\sigma )W_0(\sigma )/W_1(\sigma )`$. In the same way $`(𝒢_2g)(\sigma )`$ can be calculated. So we find $`(𝒢_2g)(\sigma )=f(\sigma )+\overline{f(\sigma )}W_0(\sigma )/\overline{W_1(\sigma )}`$. ∎

We define an operator $``$ by

$$\begin{array}{cc}\hfill (𝐟)(x)& =\left(\stackrel{~}{}\left(\begin{array}{cc}1& W_0/W_1\\ W_0/\overline{W_1}& 1\end{array}\right)^1𝐟\right)(x)\hfill \\ & =_0^{\mathrm{}}\left(\begin{array}{c}\phi _\rho ^{}(x)\\ \phi _\rho (x)\end{array}\right)^{}\left(\begin{array}{cc}1& W_0(\rho )/W_1(\rho )\\ W_0(\rho )/\overline{W_1(\rho )}& 1\end{array}\right)\left(\begin{array}{c}\overline{f(\rho )}\\ f(\rho )\end{array}\right)\frac{W_0(\rho )}{1\left|\frac{W_0(\rho )}{W_1(\rho )}\right|^2}𝑑\rho .\hfill \end{array}$$
(5.18)
From Proposition 5.16 we find $`\left(𝒢(𝐟)\right)(\rho )=𝐟(\rho )`$.

###### Remark 5.17.

From Euler’s reflection formula and the identity $`2\mathrm{sin}x\mathrm{sin}y=\mathrm{cos}(xy)\mathrm{cos}(x+y)`$, we find

$$1\left|\frac{W_0(\rho )}{W_1(\rho )}\right|^2=\left|\frac{\mathrm{\Gamma }(\frac{1}{2}+it+i\rho )\mathrm{\Gamma }(\frac{1}{2}it+i\rho )}{\mathrm{\Gamma }(k_1k_2+\frac{1}{2}+i\rho )\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\rho )}\right|^2.$$

We define

$$\begin{array}{cc}\hfill \psi _\rho (x)=& \frac{e^{x(2\varphi \pi )}\mathrm{\Gamma }(\frac{1}{2}iti\rho )\mathrm{\Gamma }(\frac{1}{2}it+i\rho )\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(k_2k_1+it+1)}{\mathrm{\Gamma }(k_1+ix+it)\mathrm{\Gamma }(k_2ix)\mathrm{\Gamma }(k_1+k_2it)\mathrm{\Gamma }(k_1k_2it+1)}\hfill \\ & \times _3F_2(.\genfrac{}{}{0.0pt}{}{\frac{1}{2}it+i\rho ,\frac{1}{2}iti\rho ,k_1ixit}{k_1k_2it+1,k_1+k_2it}.;1),\hfill \end{array}$$

then from \[2, §3.8(1)\] we obtain

$$\phi _\rho (x)\frac{W_0(\rho )}{\overline{W_1(\rho )}}\phi _\rho ^{}(x)=\frac{\psi _\rho (x)}{|\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\rho )|^2}.$$

So the definition of $``$ (5.18) is equivalent to

$$(𝐟)(x)=_0^{\mathrm{}}\left(\begin{array}{c}\psi _\rho ^{}(x)\\ \psi _\rho (x)\end{array}\right)^{}\left(\begin{array}{c}\overline{f(\rho )}\\ f(\rho )\end{array}\right)W_2(\rho )𝑑\rho ,$$

where

$$W_2(\rho )=\frac{e^{t(2\varphi \pi )}}{2\pi }\left|\frac{\mathrm{\Gamma }(k_1k_2+\frac{1}{2}+i\rho )\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\rho )\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\rho )}{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(k_2k_1+it+1)\mathrm{\Gamma }(2i\rho )}\right|^2.$$

Note that $`W_2(\rho )d\rho `$ is the orthogonality measure for the continuous dual Hahn polynomials.

The function $`𝐟`$ exists for all functions $`𝐟`$ for which the integral (5.18) converges. We want to find a domain on which $``$ is injective and isometric. We look for a set $`𝒮`$ of functions for which $`𝒮`$ is a dense subspace of $`L^2(,w(x)dx)`$. Recall that the set of polynomials is a dense subspace of $`L^2(,w(x)dx)`$.

###### Lemma 5.18.

Let $`p_n^{(\lambda )}(;\varphi )`$ denote a Meixner-Pollazcek polynomial as defined by (2.2), then

$$\left(𝒢p_n^{(k_2)}(;\varphi )\right)(\rho )=𝐪_𝐧(\rho )=\left(\begin{array}{c}\overline{q_n(\rho )}\\ q_n(\rho )\end{array}\right),$$

with $`q_n`$ given by

$$\begin{array}{cc}\hfill q_n(\rho )=& e^{t(2\varphi \pi )}\left(\frac{e^{i\varphi }}{1e^{2i\varphi }}\right)^{2k_2+n}\frac{(1)^n\mathrm{\Gamma }(2k_2)|(k_2k_1+\frac{1}{2}+i\rho )_n|^2}{n!(k_2k_1+it+1)_n}\hfill \\ & \times _2F_1(.\genfrac{}{}{0.0pt}{}{n+k_2k_1+\frac{1}{2}+i\rho ,n+k_2k_1+\frac{1}{2}i\rho }{n+k_2k_1+it+1}.;\frac{1}{1e^{2i\varphi }}).\hfill \end{array}$$

###### Proof.

Let $`𝒢_1`$ and $`𝒢_2`$ be as defined by (5.17). Since the Meixner-Pollaczek polynomial $`p_n`$ is real, we have $`𝒢_1p_n=\overline{𝒢_2p_n}`$. Writing out $`\overline{q_n(\rho )}=\left(𝒢_1p_n^{(k_2)}(;\varphi )\right)(\rho )`$ explicitly, gives

$$\begin{array}{cc}\hfill \overline{q_n(\rho )}=e^{t(2\varphi \pi )}e^{in\varphi }\frac{(2k_2)_n}{n!}\frac{1}{2\pi }_{}& e^{x(2\varphi \pi )}|\mathrm{\Gamma }(k_2+ix)|_2^2F_1(.\genfrac{}{}{0.0pt}{}{n,k_2ix}{2k_2}.;1e^{2i\varphi })\hfill \\ \hfill \times & {}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{k_2+ix,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,k_2k_1it+1}.;1)dx\hfill \\ \hfill =e^{t(2\varphi \pi )}e^{in\varphi }\frac{(2k_2)_n}{n!}\underset{m=0}{\overset{\mathrm{}}{}}\underset{l=0}{\overset{n}{}}& \frac{(k_2k_1+\frac{1}{2}+i\rho )_m(k_2k_1+\frac{1}{2}i\rho )_m}{m!(2k_2)_m(k_2k_1it+1)_m}\frac{(n)_l}{l!(2k_2)_l}(1e^{2i\varphi })^l\hfill \\ \hfill \times \frac{1}{2\pi }& _{}e^{x(2\varphi \pi )}\mathrm{\Gamma }(k_2+ix+m)\mathrm{\Gamma }(k_2ix+l)𝑑x.\hfill \end{array}$$

The inner integral can be evaluated by \[22, (3.3.9)\],

$$\frac{1}{2\pi i}_{ki\mathrm{}}^{k+i\mathrm{}}\mathrm{\Gamma }(s)\mathrm{\Gamma }(as)y^s𝑑s=\frac{\mathrm{\Gamma }(a)}{(1+y)^a},0<\mathrm{}(s)<\mathrm{}(a),|\mathrm{arg}(y)|<\pi ,$$

with $`s=k_2+m+ix`$ and $`a=2k_2+m+l`$. Now the sum over $`l`$ becomes a terminating $`{}_{2}{}^{}F_{1}^{}`$-series, which can be evaluated by the Chu-Vandermonde identity \[1, Cor.2.2.3\]

$${}_{2}{}^{}F_{1}^{}(.\genfrac{}{}{0.0pt}{}{n,2k_2+m}{2k_2}.;1)=\{\begin{array}{cc}0,\hfill & m<n,\hfill \\ \frac{(m)_n}{(2k_2)_n},\hfill & mn.\hfill \end{array}$$

Then $`\overline{q_n(\rho )}`$ reduces to a single sum, starting at $`m=n`$. Shifting the summation index gives the result. ∎

###### Proposition 5.19.

For a continuous function $`gL^2(,w(x)dx)`$, we have $`(𝒢g)=g`$.

###### Proof.

Let $`q_n`$ be as in Lemma 5.18. We show that $`(𝐪_𝐧)(x)=p_n^{(k_2)}(x;\varphi )`$. From this the proposition follows, since the polynomials are dense in $`L^2(,w(x)dx)`$.

Transforming the $`{}_{2}{}^{}F_{1}^{}`$-series of $`q_n`$ in Lemma 5.18 by \[1, (2.3.12)\] and using the asymptotic behaviour \[6, 2.3.2(16)\], we find

$$q_n(\rho )=𝒪\left(\rho ^{k_1k_2\frac{1}{2}+n}\right),\rho \mathrm{}.$$

From Stirling’s formula it follows that $`|W_0(\rho )/W_1(\rho )|=𝒪(e^{\pi \rho })`$ for $`\rho \mathrm{}`$. So by (5.14) and (5.18) we see that $`𝐪_𝐧`$ exists.

We calculate

$$I(x)=_0^{\mathrm{}}\psi _\rho (x)\overline{q_n(\rho )}W_2(\rho )𝑑\rho ,$$

then according to Remark 5.17 we have $`𝐪_𝐧=I+\overline{I}`$. Writing $`\psi _\rho (x)`$ and $`\overline{q_n(\rho )}`$ as a sum, and interchanging summation and integration, gives

$$I(x)=C\underset{l=0}{\overset{\mathrm{}}{}}\underset{m=0}{\overset{\mathrm{}}{}}\frac{(1e^{2i\varphi })^l}{l!(n+k_2k_1it+1)_l}\frac{(k_1ixit)_m}{m!(k_1k_2it+1)_m(k_1+k_2it)_m}I_{l,m},$$

where

$$\begin{array}{cc}\hfill I_{l,m}=& \frac{1}{2\pi }_0^{\mathrm{}}\mathrm{\Gamma }(\frac{1}{2}it+i\rho +m)\mathrm{\Gamma }(\frac{1}{2}iti\rho +m)\hfill \\ & \times \left|\frac{\mathrm{\Gamma }(k_1k_2+\frac{1}{2}+i\rho )\mathrm{\Gamma }(n+k_2k_1+\frac{1}{2}+i\rho +l)\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\rho )}{\mathrm{\Gamma }(2i\rho )}\right|^2d\rho ,\hfill \\ \hfill C=& e^{x(2\varphi \pi )}\left(\frac{e^{i\varphi }}{1e^{2i\varphi }}\right)^{2k_2+n}\frac{(1)^n}{n!}\hfill \\ & \times \frac{\mathrm{\Gamma }(k_2k_1+it+1)}{\mathrm{\Gamma }(k_1+ix+it)\mathrm{\Gamma }(k_2ix)\mathrm{\Gamma }(k_1+k_2+it)\mathrm{\Gamma }(k_2k_1it+n+1)\mathrm{\Gamma }(k_1k_2it+1)}.\hfill \end{array}$$

The integral $`I_{l,m}`$ can be evaluated by \[1, Thm.3.6.2\];

$$\begin{array}{cc}\hfill I_{l,m}=& \frac{\mathrm{\Gamma }(k_2k_1it+n+m+l+1)\mathrm{\Gamma }(k_1k_2it+m+1)\mathrm{\Gamma }(k_1+k_2it+m)}{\mathrm{\Gamma }(k_1+k_2it+n+m+l+1)}\hfill \\ & \times \mathrm{\Gamma }(n+l+1)\mathrm{\Gamma }(2k_2+l+n)\mathrm{\Gamma }(2k_1).\hfill \end{array}$$

Now the sum over $`m`$ is a $`{}_{2}{}^{}F_{1}^{}`$-series which can be summed by Gauss’s theorem;

$${}_{2}{}^{}F_{1}^{}(.\genfrac{}{}{0.0pt}{}{k_1ixit,k_2k_1it+n+l+1}{k_1+k_2it+n+l+1}.;1)=\frac{\mathrm{\Gamma }(k_1+ix+it)\mathrm{\Gamma }(k_1+k_2it+n+l+1)}{\mathrm{\Gamma }(k_2+ix+l+n+1)\mathrm{\Gamma }(2k_1)}.$$

Then the sum over $`l`$ becomes a $`{}_{2}{}^{}F_{1}^{}`$-series, and after applying Euler’s transformation we obtain

$$\begin{array}{cc}\hfill I(x)=& e^{x(2\varphi \pi )}\frac{(e^{i\varphi })^{n2ix}}{(1e^{2i\varphi })^{ix+k_2}}\frac{(1)^n\mathrm{\Gamma }(2k_2+n)}{\mathrm{\Gamma }(k_2+ix+n+1)\mathrm{\Gamma }(k_2ix)}\hfill \\ & \times _2F_1(.\genfrac{}{}{0.0pt}{}{k_2+ix,1k_2+ix}{k_2+ix+n+1}.;\frac{1}{1e^{2i\varphi }}).\hfill \end{array}$$

As in (3.3) we find from this

$$(𝐪_𝐧)(x)=I(x)+\overline{I(x)}=\frac{(2k_2)_n}{n!}e_2^{in\varphi }F_1(.\genfrac{}{}{0.0pt}{}{n,k_2+ix}{2k_2}.;1e^{2i\varphi })=p_n^{(k_2)}(x;\varphi ).$$

Note that the condition $`\frac{\pi }{6}<\varphi <\frac{5\pi }{6}`$ is needed for absolute convergence of the $`{}_{2}{}^{}F_{1}^{}`$-series of $`q_n`$. This condition can be removed by analytic continuation. ∎

We define the Hilbert space $``$ by

$$=\overline{\text{span}\left\{𝐪_𝐧\right|n_0\}},$$

then $``$ consists of functions of the form

$$𝐟(\rho )=\left(\begin{array}{c}f_1(\rho )\\ f_2(\rho )\end{array}\right)^2.$$

The inner product on $``$ is given by

$$𝐟,𝐠_{}=_0^{\mathrm{}}\left(\begin{array}{c}g_1(\rho )\\ g_2(\rho )\end{array}\right)^{}\left(\begin{array}{cc}1& W_0(\rho )/W_1(\rho )\\ W_0(\rho )/\overline{W_1(\rho )}& 1\end{array}\right)\left(\begin{array}{c}f_1(\rho )\\ f_2(\rho )\end{array}\right)\frac{W_0(\rho )}{1\left|\frac{W_0(\rho )}{W_1(\rho )}\right|^2}𝑑\rho .$$
(5.19)
###### Proposition 5.20.

The operator $`:L^2(,w(x)dx)`$ is an isometry.

###### Proof.

Let

$$𝐟(\rho )=\left(\begin{array}{c}f_1(\rho )\\ f_2(\rho )\end{array}\right),𝐠(\rho )=\left(\begin{array}{c}g_1(\rho )\\ g_2(\rho )\end{array}\right).$$

We write out $`𝐟,𝐠_{L^2(,w(x)dx)}`$ explicitly

$$\begin{array}{cc}\hfill \underset{N\mathrm{}}{lim}_N^N& _0^{\mathrm{}}\frac{W_0(\rho )}{1\left|\frac{W_0(\rho )}{W_1(\rho )}\right|^2}\left(\phi _\rho (x)f_1(\rho )\phi _\rho ^{}(x)f_1(\rho )\frac{W_0(\rho )}{\overline{W_1(\rho )}}\phi _\rho (x)f_2(\rho )\frac{W_0(\rho )}{W_1(\rho )}+\phi _\rho ^{}(x)f_2(\rho )\right)𝑑\rho \hfill \\ \hfill \times & \overline{_0^{\mathrm{}}\frac{W_0(\sigma )}{1\left|\frac{W_0(\sigma )}{W_1(\sigma )}\right|^2}\left(\phi _\sigma (x)g_1(\sigma )\phi _\sigma ^{}(x)g_1(\sigma )\frac{W_0(\sigma )}{\overline{W_1(\sigma )}}\phi _\sigma (x)g_2(\sigma )\frac{W_0(\sigma )}{W_1(\sigma )}+\phi _\sigma ^{}(x)g_2(\sigma )\right)𝑑\sigma }\hfill \\ \hfill \times & w(x)dx\hfill \\ \hfill =\underset{N\mathrm{}}{lim}& _0^{\mathrm{}}_0^{\mathrm{}}\frac{W_0(\rho )W_0(\sigma )}{\left(1\left|\frac{W_0(\rho )}{W_1(\rho )}\right|^2\right)\left(1\left|\frac{W_0(\sigma )}{W_1(\sigma )}\right|^2\right)}\hfill \\ \hfill \times \{& \left[f_1(\rho )\overline{g_1(\sigma )}f_1(\rho )\overline{g_2(\sigma )}\frac{W_0(\sigma )}{\overline{W_1(\sigma )}}f_2(\rho )\overline{g_1(\sigma )}\frac{W_0(\rho )}{W_1(\rho )}+f_2(\rho )\overline{g_2(\sigma )}\frac{W_0(\rho )W_0(\sigma )}{W_1(\rho )\overline{W_1(\sigma )}}\right]\phi _\rho ,\phi _\sigma _{N,N}\hfill \\ \hfill +& \left[f_2(\rho )\overline{g_1(\sigma )}f_2(\rho )\overline{g_2(\sigma )}\frac{W_0(\sigma )}{\overline{W_1(\sigma )}}f_1(\rho )\overline{g_1(\sigma )}\frac{W_0(\rho )}{\overline{W_1(\rho )}}+f_1(\rho )\overline{g_2(\sigma )}\frac{W_0(\rho )W_0(\sigma )}{\overline{W_1(\rho )W_1(\sigma )}}\right]\phi _\rho ^{},\phi _\sigma _{N,N}\hfill \\ \hfill +& \left[f_1(\rho )\overline{g_2(\sigma )}f_1(\rho )\overline{g_1(\sigma )}\frac{W_0(\sigma )}{W_1(\sigma )}f_2(\rho )\overline{g_2(\sigma )}\frac{W_0(\rho )}{W_1(\rho )}+f_2(\rho )\overline{g_1(\sigma )}\frac{W_0(\rho )W_0(\sigma )}{W_1(\rho )W_1(\sigma )}\right]\phi _\rho ,\phi _\sigma ^{}_{N,N}\hfill \\ \hfill +& [f_2(\rho )\overline{g_2(\sigma )}f_2(\rho )\overline{g_1(\sigma )}\frac{W_0(\sigma )}{W_1(\sigma )}f_1(\rho )\overline{g_2(\sigma )}\frac{W_0(\rho )}{\overline{W_1(\rho )}}+f_1(\rho )\overline{g_1(\sigma )}\frac{W_0(\rho )W_0(\sigma )}{\overline{W_1(\rho )}W_1(\sigma )}]\phi _\rho ^{},\phi _\sigma ^{}_{N,N}\}.\hfill \end{array}$$

Then from Propositions 5.12, 5.14 and (5.19) we obtain

$$𝐟,𝐠_{L^2(,w(x)dx)}=𝐟,𝐠_{}$$

by a straightforward calculation. ∎

So far we only considered the integral transform $``$ in the case that $`\rho ^2+\frac{1}{4}`$ is in the continuous spectrum of the difference operator $`\mathrm{\Lambda }`$. In the next subsection we consider the discrete spectrum of $`\mathrm{\Lambda }`$.

### 5.4. Discrete spectrum

From (5.8) it follows that for $`\mathrm{}(\rho )<0`$ we have $`\mathrm{\Phi }_\rho (x)L^2(,w(x)dx)`$. So if $`c(\rho )=0`$ and $`\mathrm{}(\rho )<0`$, we find from Proposition 5.9 that $`\phi _\rho (x)=c(\rho )\mathrm{\Phi }_\rho (x)`$, and therefore $`\phi _\rho (x)L^2(,w(x)dx)`$.

There are two possible cases for $`c(\rho )=0`$, cf. Theorem 4.1:

1. $`k_2k_1+\frac{1}{2}<0`$, then $`\rho =i(k_2k_1+\frac{1}{2}+n)`$, $`n=0,\mathrm{},n_0`$, where $`n_0`$ is the largest nonnegative integer such that $`k_2k_1+\frac{1}{2}+n_0<0`$,
2. $`k_1+k_2\frac{1}{2}<0`$, then $`\rho =i(k_1+k_2\frac{1}{2})`$.

Case (ii) does not occur for $`k_1>1`$, which is needed for convergence of the $`{}_{3}{}^{}F_{2}^{}`$-series of $`\phi _\rho (x+i)`$. However for $`k_11`$ we use expression (5.6) for $`\mathrm{\Phi }_\rho (x)`$ (which still converges if $`k_11`$) and $`\phi _\rho (x)=c(\rho )\mathrm{\Phi }_\rho (x)`$. We see that the $`{}_{3}{}^{}F_{2}^{}`$-series becomes a $`{}_{2}{}^{}F_{1}^{}`$-series of unit argument, and then, with Gauss’s summation formula, we find that in case (ii) we have $`\mathrm{\Phi }_\rho (x)=e^{2\varphi x}p(x)`$. Observe that case (i) and case (ii) exclude each other.

First we consider case (i). For $`\rho _n=i(k_2k_1+\frac{1}{2}+n)`$, $`0nn_0`$, we denote $`\phi _\rho (x)`$ by $`\phi _{\rho _n}(x)`$. We show that $`\phi _{\rho _n}(x)`$ is orthogonal to $`\phi _{\rho _m}(x)`$ and $`\phi _{\rho _m}^{}(x)`$ for $`nm`$. Note that $`\phi _{\rho _n}`$ is given by a terminating series, cf. (5.4).

###### Proposition 5.21.

For $`m,n=0,\mathrm{},n_0`$

$$\begin{array}{cc}& \phi _{\rho _n},\phi _{\rho _m}=\delta _{nm}\left(2\pi i\underset{\rho =\rho _n}{\mathrm{Res}}W_0(\rho )\right)^1\hfill \\ & =\delta _{nm}\frac{e^{2t(\varphi \frac{\pi }{2})}\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(2k_12k_21)}{\mathrm{\Gamma }(k_1k_2+it)\mathrm{\Gamma }(k_1k_2it)\mathrm{\Gamma }(2k_11)}\frac{n!(2k_22k_1+n+1)_n(22k_1)_n}{(2k_2)_n(2k_22k_1+2)_{2n}}.\hfill \end{array}$$

###### Proof.

Writing out the explicit expressions (5.4) for $`\phi _{\rho _n}(x)`$ and $`\phi _{\rho _m}(x)`$ gives

$$\begin{array}{cc}\hfill _{}\phi _{\rho _n}(x)\phi _{\rho _m}^{}(x)& w(x)dx\hfill \\ \hfill =& e^{2t(\varphi \frac{\pi }{2})}\frac{1}{2\pi }_{\mathrm{}}^{\mathrm{}}{}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{n,2k_22k_1+n+1,k_2ix}{2k_2,k_2k_1+it+1}.;1)\hfill \\ & \times _3F_2(.\genfrac{}{}{0.0pt}{}{m,2k_22k_1+m+1,k_2+ix}{2k_2,k_2k_1it+1}.;1)\left|\frac{\mathrm{\Gamma }(k_2+ix)}{\mathrm{\Gamma }(k_1+it+ix)}\right|^2dx\hfill \\ \hfill =& e^{2t(\varphi \frac{\pi }{2})}\underset{k=0}{\overset{n}{}}\underset{l=0}{\overset{m}{}}\frac{(n)_k(2k_22k_1+n+1)_k(m)_l(2k_22k_1+m+1)_l}{k!(2k_2)_k(k_2k_1+it+1)_kl!(2k_2)_l(k_2k_1it+1)_l}\hfill \\ & \times \frac{1}{2\pi }_{\mathrm{}}^{\mathrm{}}\frac{\mathrm{\Gamma }(k_2ix+k)\mathrm{\Gamma }(k_2+ix+l)}{\mathrm{\Gamma }(k_1itix)\mathrm{\Gamma }(k_1+it+ix)}𝑑x.\hfill \end{array}$$

The integral inside the sum can be evaluated by \[22, §3.3.4\]

$$\frac{1}{2\pi i}_i\mathrm{}^i\mathrm{}\frac{\mathrm{\Gamma }(a+s)\mathrm{\Gamma }(cs)}{\mathrm{\Gamma }(b+s)\mathrm{\Gamma }(ds)}𝑑s=\frac{\mathrm{\Gamma }(a+c)\mathrm{\Gamma }(b+dac1)}{\mathrm{\Gamma }(ba)\mathrm{\Gamma }(dc)\mathrm{\Gamma }(b+d1)},$$

where $`\mathrm{}(a+cbd)<1`$ and the path of integration separates the poles of $`\mathrm{\Gamma }(a+s)`$ from the poles of $`\mathrm{\Gamma }(cs)`$. Note that the convergence condition $`2k_22k_1+k+l<1`$ is satisfied in case (i) and $`k,ln_0`$. Now we find for the double sum for $`nm`$

$$\begin{array}{c}e^{2t(\varphi \frac{\pi }{2})}\frac{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(2k_12k_21)}{\mathrm{\Gamma }(k_1k_2+it)\mathrm{\Gamma }(k_1k_2it)\mathrm{\Gamma }(2k_11)}\hfill \\ \hfill \times \underset{k=0}{\overset{n}{}}\frac{(n)_k(2k_22k_1+n+1)_k}{k!(2k_22k_1+2)_k}\underset{l=0}{\overset{m}{}}\frac{(m)_l(2k_22k_1+m+1)_l(2k_2+k)_l}{l!(2k_2)_l(2k_22k_1+k+2)_l}.\end{array}$$

The sum over $`l`$ is a terminating $`{}_{3}{}^{}F_{2}^{}`$-series, which can be evaluated by the Pfaff-Saalschütz theorem

$${}_{3}{}^{}F_{2}^{}(.\genfrac{}{}{0.0pt}{}{m,2k_22k_1+m+1,2k_2+k}{2k_2,2k_22k_1+k+2}.;1)=\{\begin{array}{cc}0,\hfill & k<m,\hfill \\ \frac{(1)^mm!(22k_1)_m}{(2k_2)_m(2k_22k_1+m+2)_m},\hfill & k=m.\hfill \end{array}$$

So we find for $`nm`$

$$\begin{array}{c}_{}\phi _{\rho _n}(x)\phi _{\rho _m}^{}(x)w(x)𝑑x=\hfill \\ \hfill \delta _{nm}\frac{e^{2t(\varphi \frac{\pi }{2})}\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(2k_12k_21)}{\mathrm{\Gamma }(k_1k_2+it)\mathrm{\Gamma }(k_1k_2it)\mathrm{\Gamma }(2k_11)}\frac{n!(2k_22k_1+n+1)_n(22k_1)_n}{(2k_2)_n(2k_22k_1+n+2)_n(2k_22k_1+2)_n}.\end{array}$$

Note that from the condition $`k_2k_1+\frac{1}{2}+n<0`$ follows that this expression is positive in case $`n=m`$. For $`nm`$ we find the same result by interchanging the summations over $`k`$ and $`l`$. A straightforward calculation shows that the expression found is equal to

$$\delta _{nm}\left(2\pi i\underset{\rho =\rho _n}{\mathrm{Res}}W_0(\rho )\right)^1.$$

###### Proposition 5.22.

For $`m,n=0,\mathrm{},n_0`$

$$\begin{array}{cc}\hfill \phi _{\rho _n}^{},\phi _{\rho _m}=& \delta _{nm}\left(2\pi i\underset{\rho =\rho _n}{\mathrm{Res}}W_1(\rho )\right)^1\hfill \\ \hfill =& \delta _{nm}\frac{e^{2t(\varphi \frac{\pi }{2})}\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(2k_12k_21)}{\mathrm{\Gamma }(k_1k_2+it)\mathrm{\Gamma }(k_1k_2it)\mathrm{\Gamma }(2k_11)}\hfill \\ & \times \frac{(1)^nn!(k_2k_1+it+1)_n(2k_22k_1+n+1)_n(22k_1)_n}{(k_2k_1it+1)_n(2k_2)_n(2k_22k_1+2)_{2n}}.\hfill \end{array}$$

###### Proof.

From the first formula on page $`142`$ in  we find

$$\phi _{\rho _n}(x)=(1)^n\frac{(k_2k_1it+1)_n}{(k_2k_1+it+1)_n}\phi _{\rho _n}^{}(x).$$

Then the explicit expression follows from Proposition 5.21. A straightforward calculation shows that the explicit expression is equal to the residue at $`\rho =\rho _n`$ of $`W_1(\rho )`$. ∎

A similar calculation is used for case (ii). Recall from the beginning of this subsection that in this case $`\phi _{\rho _c}(x)=e^{2\varphi x}c(\rho _c)p(x)`$.

###### Proposition 5.23.

Let $`\rho _c=i(k_1+k_2\frac{1}{2})`$, then

$$\begin{array}{cc}\hfill \phi _{\rho _c},\phi _{\rho _c}& =\left(2\pi i\underset{\rho =\rho _c}{\mathrm{Res}}W_0(\rho )\right)^1\hfill \\ & =e^{2t(\varphi \frac{\pi }{2})}\frac{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(12k_12k_2)}{\mathrm{\Gamma }(12k_1)}\left|\frac{\mathrm{\Gamma }(k_2k_1+it+1)}{\mathrm{\Gamma }(k_1+k_2+it)\mathrm{\Gamma }(1k_1k_2+it)}\right|^2.\hfill \end{array}$$

###### Proof.

The proof is similar to the proof of Proposition 5.21. ∎

###### Proposition 5.24.

Let $`\rho _c=i(k_1+k_2\frac{1}{2})`$, then

$$\begin{array}{cc}\hfill \phi _{\rho _c}^{},\phi _{\rho _c}& =\left(2\pi i\underset{\rho =\rho _c}{\mathrm{Res}}W_1(\rho )\right)^1\hfill \\ & =e^{2t(\varphi \frac{\pi }{2})}\frac{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(12k_12k_2)\mathrm{\Gamma }(k_2k_1it+1)}{\mathrm{\Gamma }(12k_1)\mathrm{\Gamma }(k_1+k_2it)\mathrm{\Gamma }(1k_1k_2it)\mathrm{\Gamma }(k_1k_2+it))}.\hfill \end{array}$$

###### Proof.

We use Euler’s reflection formula to write $`p(x)`$ in terms of $`\mathrm{\Gamma }`$-functions, then we have

$$\phi _{\rho _c}^{},\phi _{\rho _c}=e^{2t(\varphi \frac{\pi }{2})}\overline{c(\rho _c)}^2\frac{1}{2\pi }_{}\frac{\mathrm{\Gamma }(k_2+ix)\mathrm{\Gamma }(k_2ix)\mathrm{\Gamma }(k_1itix)}{\mathrm{\Gamma }(k_1+it+ix)\mathrm{\Gamma }(1k_1itix)\mathrm{\Gamma }(1k_1itix)}𝑑x.$$

We use a special case of \[25, (4.5.1.2)\] to evaluate the integral;

$$\begin{array}{cc}\hfill \frac{1}{2\pi i}& _i\mathrm{}^i\mathrm{}\frac{\mathrm{\Gamma }(a+s)\mathrm{\Gamma }(bs)\mathrm{\Gamma }(cs)}{\mathrm{\Gamma }(d+s)\mathrm{\Gamma }(es)\mathrm{\Gamma }(fs)}𝑑s=\hfill \\ & \frac{\mathrm{\Gamma }(a+b)\mathrm{\Gamma }(a+c)}{\mathrm{\Gamma }(da)\mathrm{\Gamma }(a+e)\mathrm{\Gamma }(a+f)}_3F_2(.\genfrac{}{}{0.0pt}{}{a+b,a+c,1+ad}{a+e,a+f}.;1),\hfill \end{array}$$

where $`\mathrm{}(d+e+fabc)>0`$ and the path of integration separates the poles of $`\mathrm{\Gamma }(a+s)`$ from the poles of $`\mathrm{\Gamma }(bs)`$ and $`\mathrm{\Gamma }(cs)`$. We put

$$s=k_2+ix,a=2k_2,b=0,c=k_1k_2it,$$

$$d=k_1+k_2+it,e=1k_1k_2it,f=1k_1k_2it,$$

then we find that the $`{}_{3}{}^{}F_{2}^{}`$-series reduces to a $`{}_{2}{}^{}F_{1}^{}`$-series, which can be evaluated by Gauss’s summation formula. From this the result follows. ∎

Next we show that if $`\rho ^2+\frac{1}{4}`$ is in the discrete spectrum of $`\mathrm{\Lambda }`$, $`\phi _\rho (x)`$ is orthogonal to $`\phi _\sigma (x)`$ and $`\phi _\sigma ^{}(x)`$, if $`\sigma `$ is real.

###### Proposition 5.25.

For $`\sigma [0,\mathrm{})`$ and $`\rho =\rho _n`$, or $`\rho =\rho _c`$, we have

$$\phi _\rho ,\phi _\sigma =0,\phi _\rho ,\phi _\sigma ^{}=0.$$

###### Proof.

By Propositions 5.3 and 5.7

$$\underset{N,M\mathrm{}}{lim}_M^N\phi _\rho (x)\phi _\sigma ^{}(x)w(x)𝑑x=\underset{N,M\mathrm{}}{lim}\frac{[\phi _\rho ,\phi _\sigma ](N)[\phi _\rho ,\phi _\sigma ](M)}{\rho ^2\sigma ^2}.$$

We show that the limit of each Wronskian is zero. Let

$$f(x)=\left(\phi _\rho (x)\phi _\sigma ^{}(xi)\phi _\rho (xi)\phi _\sigma ^{}(x)\right)\alpha _{}(x)w(x).$$

First we consider the asymptotic behaviour of $`f(M)`$ and $`f(N)`$.

For $`\rho =\rho _n`$ and $`0y1`$ we find from Lemmas 5.4, 5.10 and Proposition 5.9

$$f(x+iy)=𝒪(|x|^{\frac{1}{2}k_1+k_2n}),x\pm \mathrm{}.$$

For $`\rho =\rho _c`$ we find

$$f(x+iy)=𝒪(|x|^{2k_1+2k_21}),x\pm \mathrm{}.$$

Here the implied constants do not depend on $`y`$. Then dominated convergence gives the result.

The proof for $`\phi _\rho ,\phi _\sigma ^{}=0`$ runs along the same lines. ∎

###### Remark 5.26.

The explicit calculations in this subsection can be carried out because of the choice (5.3) of the $`i`$-periodic function $`p(x)`$. It is not likely that with another choice for the function $`p(x)`$ all the calculations can be done explicitly. This gives another (heuristic) argument for the choice of $`p(x)`$.

### 5.5. The continuous Hahn integral transform

We combine the results of subsection 5.3 with the results of subsection 5.4.

Let $`k_1,k_2>0`$, $`0<\varphi <\pi `$ and $`t`$. Let $`\phi _\rho (x)`$ be the function given by

$$\phi _\rho (x)=\frac{e^{x(2\varphi \pi )}}{\mathrm{\Gamma }(k_1+ix+it)\mathrm{\Gamma }(k_1ixit)}_3F_2(.\genfrac{}{}{0.0pt}{}{k_2ix,k_2k_1+\frac{1}{2}+i\rho ,k_2k_1+\frac{1}{2}i\rho }{2k_2,k_2k_1+it+1}.;1).$$

We denote

$$\begin{array}{cc}\hfill h(\rho )& =\frac{W_0(\rho )}{W_1(\rho )}=\frac{\mathrm{\Gamma }(\frac{1}{2}+it+i\rho )\mathrm{\Gamma }(\frac{1}{2}+iti\rho )}{\mathrm{\Gamma }(k_1k_2+it)\mathrm{\Gamma }(k_2k_1+it+1)},\hfill \\ \hfill W(\rho )& =\frac{W_0(\rho )}{1|h(\rho )|^2}=\frac{1}{2\pi }e^{t(2\varphi \pi )}\left|\frac{\mathrm{\Gamma }(k_2k_1+\frac{1}{2}+i\rho )^2\mathrm{\Gamma }(k_1k_2+\frac{1}{2}+i\rho )\mathrm{\Gamma }(k_1+k_2\frac{1}{2}+i\rho )}{\mathrm{\Gamma }(2k_2)\mathrm{\Gamma }(k_2k_1+it+1)\mathrm{\Gamma }(2i\rho )}\right|^2,\hfill \\ \hfill w(x)& =\frac{1}{2\pi }e^{(2x+t)(2\varphi \pi )}|\mathrm{\Gamma }(k_1+it+ix)\mathrm{\Gamma }(k_2+ix)|^2.\hfill \end{array}$$

Let $``$ be the Hilbert space given by

$$=\overline{\text{span}\left\{𝐪_𝐦\right|m_0\}},$$

where $`𝐪_𝐦`$ is as in Lemma 5.18. For functions $`𝐟=\left(\begin{array}{c}f_1\\ f_2\end{array}\right)`$, the inner product on $``$ is given by:

(i) For $`k_1+k_1\frac{1}{2}0`$, $`k_2k_1+\frac{1}{2}0`$

$$𝐟,𝐠_{}=_0^{\mathrm{}}\left(\begin{array}{c}g_1(\rho )\\ g_2(\rho )\end{array}\right)^{}\left(\begin{array}{cc}1& h(\rho )\\ \overline{h(\rho )}& 1\end{array}\right)\left(\begin{array}{c}f_1(\rho )\\ f_2(\rho )\end{array}\right)W(\rho )𝑑\rho .$$

(ii) For $`k_1+k_2\frac{1}{2}<0`$, $`\rho _c=i(k_1+k_2\frac{1}{2})`$,

$$\begin{array}{cc}\hfill 𝐟,𝐠_{}=_0^{\mathrm{}}\left(\begin{array}{c}g_1(\rho )\\ g_2(\rho )\end{array}\right)^{}& \left(\begin{array}{cc}1& h(\rho )\\ \overline{h(\rho )}& 1\end{array}\right)\left(\begin{array}{c}f_1(\rho )\\ f_2(\rho )\end{array}\right)W(\rho )d\rho \hfill \\ & +\left(\begin{array}{c}g_1(\rho _c)\\ g_2(\rho _c)\end{array}\right)^{}\left(\begin{array}{cc}1& h(\rho _c)\\ \overline{h(\rho _c)}& 1\end{array}\right)\left(\begin{array}{c}f_1(\rho _c)\\ f_2(\rho _c)\end{array}\right)\mathrm{\hspace{0.17em}2}\pi i\underset{\rho =\rho _c}{\mathrm{Res}}W(\rho ).\hfill \end{array}$$

(iii) For $`k_2k_1+\frac{1}{2}<0`$, $`\rho _n=i(k_2k_1+\frac{1}{2}+n)`$, $`n=0,\mathrm{},n_0`$, where $`n_0`$ is the largest integer such that $`i\rho _{n_0}<0`$,

$$\begin{array}{cc}\hfill 𝐟,𝐠_{}=_0^{\mathrm{}}& \left(\begin{array}{c}g_1(\rho )\\ g_2(\rho )\end{array}\right)^{}\left(\begin{array}{cc}1& h(\rho )\\ \overline{h(\rho )}& 1\end{array}\right)\left(\begin{array}{c}f_1(\rho )\\ f_2(\rho )\end{array}\right)W(\rho )d\rho \hfill \\ & +\underset{n=0}{\overset{n_0}{}}\left(\begin{array}{c}g_1(\rho _n)\\ g_2(\rho _n)\end{array}\right)^{}\left(\begin{array}{c}f_1(\rho _n)\\ f_2(\rho _n)\end{array}\right)\pi i\underset{\rho =\rho _n}{\mathrm{Res}}W(\rho )(1|h(\rho )|^2).\hfill \end{array}$$

Observe that for $`m>n`$ we have $`𝐪_𝐦(\rho _n)=\mathrm{𝟎}`$. For $`mn`$ it follows from the way $`q_m`$ is calculated in Proposition 5.18, that $`q_m(\rho _n)=\overline{h(\rho _n)}\overline{q_m(\rho _n)}`$.

For a continuous function $`𝐟`$ we define the linear operator $`:L^2(,w(x)dx)`$ by

$$(𝐟)(x)=𝐟,\left(\begin{array}{c}\phi _\rho ^{}(x)\\ \phi _\rho (x)\end{array}\right)_{}.$$

We call $``$ the continuous Hahn integral transform.

###### Theorem 5.27.

The continuous Hahn integral transfrom $`:L^2(,w(x)dx)`$ is unitary and its inverse is given by

$$\left(^1g\right)(\rho )=_{}g(x)\left(\begin{array}{c}\phi _\rho ^{}(x)\\ \phi _\rho (x)\end{array}\right)w(x)𝑑x.$$

###### Proof.

For case (i) this follows from Propositions 5.16, 5.19 and 5.20.

For case (ii) we only have to check that Propositions 5.12 and 5.14 still hold with the discrete mass point in $`\rho =\rho _c`$ added to the integral. From Propositions 5.23, 5.24 and 5.25 we find

$$\begin{array}{cc}\hfill f(\rho _c)\phi _{\rho _c},\phi _\sigma =& \{\begin{array}{cc}0,\hfill & \sigma [0,\mathrm{}),\hfill \\ \left(2\pi i\underset{\sigma =\rho _c}{\mathrm{Res}}W_0(\sigma )\right)^1f(\rho _c),\hfill & \sigma =\rho _c,\hfill \end{array}\hfill \\ \hfill f(\rho _c)\phi _{\rho _c}^{},\phi _\sigma =& \{\begin{array}{cc}0,\hfill & \sigma [0,\mathrm{}),\hfill \\ \left(2\pi i\underset{\sigma =\rho _c}{\mathrm{Res}}W_1(\sigma )\right)^1f(\rho _c),\hfill & \sigma =\rho _c.\hfill \end{array}\hfill \end{array}$$

Now the proof for case (ii) is completely analogous to the proof of case (i).

For case (iii) injectivity and surjectivity of $``$ can be proved in the same way as case (i). We check that $``$ is an isometry. The continuous part follows from Proposition 5.20, so we only have to check for the discrete part. We write out $`𝐪_𝐤,𝐪_𝐥_{L^2(,w(x)dx}`$, $`k,l_0`$, for the discrete part of $`𝐪_𝐤`$ and $`𝐪_𝐥`$. From Propositions 5.21, 5.22 and $`q_k(\rho _n)=\overline{h(\rho _n)}\overline{q_k(\rho _n)}`$ we find, for $`n,m=0,\mathrm{},n_0`$,

$$\begin{array}{cc}\hfill _{}& \underset{n=0}{\overset{n_0}{}}\left(\phi _{\rho _n}(x)\overline{q_k(\rho _n)}+\phi _{\rho _n}^{}(x)q_k(\rho _n)\right)\pi i\underset{\rho =\rho _n}{\mathrm{Res}}W_0(\rho )\hfill \\ & \times \underset{m=0}{\overset{n_0}{}}(\phi _{\rho _m}(x)\overline{q_l(\rho _m)}+\phi _{\rho _m}^{}(x)q_l(\rho _m))\pi i\underset{\rho =\rho _m}{\mathrm{Res}}W_0(\rho )w(x)dx\hfill \\ \hfill =& \underset{n,m=0}{\overset{n_0}{}}(\overline{q_k(\rho _n)q_l(\rho _m)}\phi _{\rho _n},\phi _{\rho _m}^{}+\overline{q_k(\rho _n)}q_l(\rho _m)\phi _{\rho _n},\phi _{\rho _m}+q_k(\rho _n)\overline{q_l(\rho _m)}\phi _{\rho _n}^{},\phi _{\rho _m}^{}\hfill \\ & +q_k(\rho _n)q_l(\rho _m)\phi _{\rho _n}^{},\phi _{\rho _m})(\pi i)^2\underset{\rho =\rho _n}{\mathrm{Res}}W_0(\rho )\underset{\rho =\rho _m}{\mathrm{Res}}W_0(\rho )\hfill \\ \hfill =& \underset{n=0}{\overset{n_0}{}}\left(q_k(\rho _n)\overline{q_l(\rho _n)}+\overline{q_k(\rho _n)}q_l(\rho _n)\right)\pi i\underset{\rho =\rho _n}{\mathrm{Res}}W_0(\rho ).\hfill \end{array}$$

Here we recognize the discrete part of the inner product $`𝐪_𝐤,𝐪_𝐥_{}`$. Combined with Propositions 5.20 and 5.25 this shows that $``$ acts isometric on the basis elements $`𝐪_𝐤`$. By linearity $``$ extends to an isometry. ∎

The continuous Hahn integral transform in case (i) corresponds exactly to the integral transform we found in §4.3 by formal computations.

###### Remark 5.28.

Let us denote the operator $`\mathrm{\Lambda }`$ by $`\mathrm{\Lambda }(k_1,k_2,t)`$, let $`w(x)=w(x;k_1,k_2,t)`$, and let $`T_t`$ denote the shift operator. Observe that $`w(x+t;k_2,k_1,t)=w(x;k_1,k_2,t)`$. It is clear that $`L^2(,w(x;k_1,k_2,t)dx)`$ is invariant under the action of $`T_t`$. A short calculation shows that $`T_t\mathrm{\Lambda }(k_1,k_2,t)T_t=\mathrm{\Lambda }(k_2,k_1,t)`$, so $`\phi _\rho (x+t;t,k_2,k_1,\varphi )`$ is an eigenfunction of $`\mathrm{\Lambda }(k_1,k_2,t)`$ for eigenvalue $`\rho ^2+\frac{1}{4}`$. Going through the whole machinery of this section again, then gives another spectral measure of $`\mathrm{\Lambda }(k_1,k_2,t)`$, namely the one we found with $`k_1k_2`$, $`k_2k_1`$ and $`tt`$.

Finally we compare the spectrum of the difference operator $`\mathrm{\Lambda }`$ with the tensor product decomposition in Theorem 4.1. The discrete term in case (ii) in this section corresponds to one complementary series representation in the tensor product decomposition in Theorem 4.1. Case (iii) does not occur in Theorem 4.1, since it is assumed that $`k_1k_2`$. If we no longer assume this, the discrete terms in case (iii) correspond to a finite number of positive discrete series representations.

The finite number of negative discrete series in Theorem 4.1 can be obtained as described in Remark 5.28. In order to obtain the Clebsch-Gordan coefficients in this case from the summation formula in Theorem 3.1, we need to consider different overlap coefficients for the continuous series representations. Let $`v_n(x;\lambda ,\epsilon ,\varphi ):=u_n(x;\lambda ,\epsilon ,\pi \varphi )`$, where $`u_n`$ is the Meixner-Pollaczek function as defined by (2.5). Then

$$\left(\begin{array}{c}\stackrel{~}{v}_{\rho ,\epsilon }(x)\\ \stackrel{~}{v}_{\rho ,\epsilon }^{}(x)\end{array}\right)=\underset{n=\mathrm{}}{\overset{\mathrm{}}{}}\left(\begin{array}{c}v_n(x;\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )\\ v_n^{}(x;\frac{1}{2}+i\rho ,\epsilon ,\pi \varphi )\end{array}\right)e_n,$$

is a generalized eigenvector of $`\pi ^{\rho ,\epsilon }(X_\varphi )`$, see \[14, §4.4.11\]. From Theorem 3.1, with $`(k_1,k_2,x_1,x_2,p)(k_2,k_1,x_2,x_1,p)`$ we find the Clebsch-Gordan coefficients for the eigenvector $`\left(\begin{array}{c}\stackrel{~}{v}_{\rho ,\epsilon }(x_1x_2)\\ \stackrel{~}{v}_{\rho ,\epsilon }^{}(x_1x_2)\end{array}\right)`$, and these Clebsch-Gordan coefficients are multiples of continuous Hahn functions $`\phi _\rho (x_1;x_2x_1,k_2,k_1,\varphi )`$ and $`\phi _\rho ^{}(x_1;x_2x_1,k_2,k_1,\varphi )`$. In this case the discrete mass points in the measure for the continuous Hahn transform correspond to one complementary series representation, or a finite number of negative discrete series representations