text:
# NON-GAUSSIAN SIGNATURE INDUCED BY THE SZ EFFECT OF GALAXY CLUSTERS

## 1 Introduction

One of the major goals of cosmology is to understand the origin of the initial density perturbations. Do they come from inflation$`^\mathrm{?}`$ or from topological defects$`^\mathrm{?}`$? One way to answer this question is to make a statistical analysis of the CMB anisotropies. In fact, inflation predicts a Gaussian distribution of the primary anisotropies whereas the topological defects predict a non-Gaussian distribution. However, not only topological defects produce non-Gaussian signals. There are other astrophysical sources of non-gaussianity such as gravitational lensing (cf. F. Bernardeau’s contribution in this proceedings) and the secondary anisotropies such as the Sunyaev-Zel’dovich effect$`^\mathrm{?}`$ of galaxy clusters.
In this context, we have developed a method for the statistical analysis based on the wavelet decomposition of a signal. The method has been tested on an arbitrary set of Gaussian and non-Gaussian maps and then applied to CMB simulated maps to investigate the effect of the SZ contribution on the statistical signature of the anisotropies.

## 2 Detection method

Our method is based on a wavelet analysis of an image. That is, rather than analysing the signal in the direct space, we analyse the associated coefficients in the wavelet decomposition. The wavelet analysis can be viewed as a convolution of the signal by band pass filters: a scaling function similar to a low pas filter and a wavelet function similar to a high pass filter. The low pass filter will give an image with a lower resolution and the high pass will give the associated details. We have used the multi-scale wavelet analysis$`^\mathrm{?}`$ to decompose an input signal into a series of successive low resolution images and their associated detail images. At each level of the decomposition, the reference image has a resolution reduced by a factor of two and only this reference low pass image is decomposed at each level. This is the principle of the dyadic decomposition. We have chosen an anti-symmetric wavelet$`^\mathrm{?}`$ function similar to a first derivative operator because we expect that the non-Gaussian signatures arise from sharp gradients in the signal. Such a function is particularly sensitive to gradients and will detect them easily. The high pass filter applied to both directions of the image gives at each level three detail images corresponding to vertical, horizontal and diagonal details. The features of interest, particularly the statistical signatures, are studied at each scale (or level) for each type of details. We have qualified and tested our detection method on a set of Gaussian and non-Gaussian maps (100 of each) having the same power spectrum. This condition allows us to attribute the differences that are detected between the Gaussian and non-Gaussian maps to nothing but their statistical nature.

Our detection method$`^\mathrm{?}`$ is based on the three following major steps. First, we perform the wavelet decomposition of both 100 Gaussian maps and 100 non-Gaussian maps. We thus obtain at each decomposition level the wavelet coefficients associated with the vertical, horizontal and diagonal details. We also define the multi-scale gradient as the sum of the squared coefficients associated with the vertical and horizontal details. The non-Gaussian signature can be detected through the third or fourth order moment of the distribution, respectively the skewness and the kurtosis. In the following, we focus only on the excess of kurtosis because our signals are very moderately skewed. The second step of the method is to compute for each Gaussian and non-Gaussian map and at each level the excess of kurtosis of the wavelet coefficients associated with the corresponding details. We check that contrary to the Gaussian maps, the excess of kurtosis of the non-Gaussian signal is always centred around a non-zero value. We also note that the dispersion around the mean excess of kurtosis increases at high decomposition levels even for the Gaussian maps, due to the lack of coefficients at these levels. Therefore, in the following, we restrict our study to the first three levels.

The last step of the analysis is to quantify the detectability of the non-Gaussian signature. This is done by comparing the probability distribution functions (PDF) of the different processes. We compute the median excess of kurtosis of the 100 non-Gaussian maps at each decomposition scale and estimate the probability that it belongs to the PDF of the Gaussian maps. A low probability favours of a non-Gaussian signal whereas a high probability indicates a Gaussian process. We can also perform a more global comparison of the PDFs through the Kolmogorov-Smirnov test$`^\mathrm{?}`$. It gives, with a very good accuracy, the probability for a distribution to be different from a Gaussian.

## 3 Application to CMB

We apply our method to simulated CMB data including primary and secondary anisotropies. Our goal is to estimate the statistical non-Gaussian contamination induced by the secondary anisotropies. Therefore, we use an inflation model that generates Gaussian distributed primary anisotropies to which we add the simulated$`^\mathrm{?}`$ contributions due to the Sunyaev-Zel’dovich (thermal and kinetic) effect of galaxy clusters.

We have simulated 100 statistical realisations of the resulting CMB maps and performed the multi-scale decomposition. Following our proposed method, we compute the excess of kurtosis at different scales for the coefficients associated with the diagonal, vertical and horizontal details (Table 1).
At the first three decomposition scales, the excess of kurtosis is very large due to the SZ contribution. We also note, that the computations using the diagonal details are more sensitive to non-gaussianity and thus more powerful in detecting it. In fact, the galaxy clusters exhibit very peaked profiles or even point-like behaviour. The diagonal details are very sensitive to symmetric profiles. We find that the SZ effect a major source of non-gaussianity among the secondary effects.

## 4 Effects of the instrumental configurations

We apply our statistical discriminators to test for non-gaussianity within the context of the representative instrumental configuration of the future Planck Surveyor satellite for CMB observations.

We use the same astrophysical contributions as those described above (primary and SZ). The difference lies in the fact that the maps are convolved with a 6 arcminute Gaussian beam. We also take into account the expected Gaussian noise of Planck ($`(\delta T/T)_{rms}\mathrm{2.\hspace{0.17em}10}^6`$ per 1.5 arcminute pixel). The convolution by a 6 arcminute beam suppresses the power at the corresponding scale (Scale I) and affects the second decomposition scale whereas the third is not significantly altered. At the third decomposition scale, we find for the multi-scale gradient $`k=0.62_{0.60}^{+1.43}`$. Whereas we find for the horizontal and vertical details, and for the diagonal details respectively, $`k=0.07_{0.08}^{+0.11}`$ and $`0.16\pm 0.10`$. In order to quantify the detectability of non-gaussianity in the Planck-like configuration, we generate Gaussian distributed maps with same power spectrum as the studied signal. We plot (Fig. 1) the PDF of the Gaussian (dashed line) and non-Gaussian (solid line) processes. We derive the probability that the median excess of kurtosis measured on the “real sky” belongs to the Gaussian process. Using the multi-scale gradient, we find that the probability of detecting non-gaussianity is 71.9% at the second decomposition scale. There is no significant detection elsewhere. Whereas using the coefficients of the diagonal details, the probability of detecting a non-Gaussian signature at the third scale is 94.5%. We apply the K-S test to the distribution of the excess of kurtosis for the diagonal details and find a probability of 96.6% of detecting non-gaussianity. Since the comparison of the two distributions using the K-S test is very sensitive to departures from gaussianity. It thus gives better results on the detection of the non-Gaussian signature.

## 5 Conclusions

In the present study, we investigate the statistical signature induced by the SZ effect of galaxy clusters when the primary anisotropies result from an inflationary scenario and are thus Gaussian distributed. We use discriminators based on the statistical properties of the coefficients in a four level wavelet decomposition. In our study, we find that the SZ effect of clusters generates a very large non-Gaussian signature that dominates by far all other secondary anisotropies. We apply our statistical tests to a Planck-like configuration in order to estimate the capabilities of the satellite to detect the non-Gaussian signature of the SZ effect. In this case, we detect unambiguously the non-Gaussian signature at the third decomposition scale ($`12`$ arcminutes), the first ($`3`$ arcminutes) and second ($`6`$ arcminutes) scales being affected by the beam convolution.

## References


format: text

text:
# Error-correcting codes and image restoration with multiple stages of dynamics

## I INTRODUCTION

The corruption of signals by noise is a common problem encountered in information processing. To retrieve signals from messages corrupted during the transmission through noisy channels, various error-correcting codes have been proposed . In particular, the error-correction mechanism of a class of parity-checking codes can be considered as the search for thermodynamically stable states of a Hamiltonian constructed in terms of the message bits . These codes have been demonstrated to saturate the Shannon information bound in the limit that each encoded bit checks the parity of an infinitely large number of message bits . While in practice, each encoded bit can only check the parity of a finite number of message bits, these codes still maintain a very low bit error probability.

The need to retrieve signals from corrupted messages is also inherent in image restoration . Although parity-checking bits may not be explicitly introduced for the task, prior knowledge about the images plays a similar role. For example, the smoothness of real-world images provides a mechanism for checking the pixel values in comparison with those of their neighbors. A corresponding Hamiltonian, consisting of a ferromagnetic bias to reflect the smoothening tendency, can be constructed in terms of the image pixels. Modern techniques of image restoration based on Markov random fields correspond to the search for thermodynamically stable states of the Hamiltonian system, using methods such as simulated annealing .

In a recent paper, we have shown that the problems of error-correcting codes and image restoration can be formulated in a unified framework . In both tasks, the choice of the so-called hyperparameters is an important factor in determining their performances. Hyperparameters refer to the coefficients of the various interactions appearing in the Hamiltonian of the tasks. In error correction, they determine the statistical significance given to the parity-checking terms and the received bits. Similarly in image restoration, they determine the statistical weights given to the prior knowledge and the received data. It was shown, by the use of inequalities, that the optimal choice of the hyperparameters correspond to the Maximum Posterior Marginal (MPM) method, where there is a match between the source and model priors. The choice of these values correspond to the Nishimori point in the space of hyperparameters . It is equivalent to a thermodynamic process at finite temperature, and the task performance is better than the Maximum A Posteriori probability (MAP) method, where the values of the hyperparameters are taken to infinity, equivalent to a zero temperature process. Furthermore, from the analytic solution of the infinite-range model and the Monte Carlo simulation of finite-dimensional models, it was shown that an inappropriate choice of the hyperparameters can lead to a rapid degradation of the tasks.

In fact, hyperparameter estimation has been the subject of many previous studies , a recently popular one using the “evidence framework” . However, if the prior models the source poorly, no hyperparameters can be reliable . Even if they can be estimated accurately through steady-state statistical measurements, they may fluctuate when interfered by bursty noise sources in communication channels. Hence it is important to devise decoding or restoration procedures which are robust against the uncertainties in hyperparameter estimation.

In this paper we propose the technique of selective freezing as a method to increase the tolerance to uncertainties in hyperparameter estimation. The technique has been studied for pattern reconstruction in neural networks, where it led to an improvement in the retrieval precision, a widening of the basin of attraction, and a boost in the storage capacity . The idea is best illustrated for Ising bits or pixels with binary states $`\pm 1`$, though it can be easily generalized to other cases. In a finite temperature thermodynamic process, the Ising variables keep moving under thermal agitation. Some of them have smaller thermal fluctuations than the others, implying that they are more certain to stay in one state than the other. This stability implies that they have a higher probability to stay in the correct state for error-correction or image restoration tasks, even when the hyperparameters are not optimally tuned. It may thus be interesting to separate the thermodynamic process into two stages. In the first stage we select those relatively stable bits or pixels whose time-averaged states have a magnitude exceeding a certain threshold. In the second stage we subsequently fix (or freeze) them in the most probable thermodynamic states (for Ising variables this corresponds to the sign of the time-averaged state). Thus these selectively frozen bits or pixels are able to provide a more robust assistance to the less stable bits or pixels in their search for the most probable states. The selective freezing procedure reduces to the usual finite-temperature decoding or restoration process if all bits or pixels are frozen (since nothing happens in the second stage), or no bits or pixels are frozen (since the second stage is merely a continuation of the equilibration process of the first stage).

The two-stage thermodynamic process can be studied analytically in the mean-field model, which provides a qualitative guide to the behavior of more realistic cases of lower dimensions. However, it is necessary to give a remark about the theoretical approach. That is, as far as we have tried, the analytical solution has been inaccessible by the more conventional replica method. Rather, we have to use the cavity method to obtain the equations for the order parameters. In particular, the cavity method leads to the appearance of a term called the trans-susceptibility, which correctly describes the effects of the thermodynamics of the first stage on that of the second.

The paper is organized as follows. In Section II we briefly review the formulation of error-correcting codes and image restoration in a unified framework. In Sections III and IV, we consider the mean-field model for error-correcting codes and image restoration respectively. We derive the equations for the order parameters of the two-stage thermodynamics using the cavity method, and present numerical results illustrating the robustness of selective freezing against uncertainties in hyperparameter estimation. We further demonstrate that even when the noise model changes without the receiver/restoration agent realizing the change (i.e. it makes a wrong estimation of the prior), the task performance is still robust. For the more realistic cases of lower dimensions, simulation results illustrate the relevance of the infinite-range model in providing qualitative guidance. The conclusion is given in Section V.

## II FORMULATION

Consider an information source which generates data represented by a set of Ising spins $`\{\xi _i\}`$, where $`\xi _i=\pm 1`$ and $`i=1,\mathrm{},N`$. The data is generated according to the source prior $`P_s(\{\xi _i\})`$. For error-correcting codes transmitting unbiased messages, all sequences are equally probable and $`P_s(\{\xi \})=2^N`$. For images with smooth structures, the prior consists of ferromagnetic Boltzmann factors, which increase the tendencies of the neighboring spins to stay at the same spin states, that is,

$$P_s(\{\xi \})=\frac{1}{Z(\beta _s)}\mathrm{exp}\left(\frac{\beta _s}{z}\underset{ij}{}\xi _i\xi _j\right).$$
(1)
Here $`ij`$ represents pairs of neighboring spins, $`z`$ is the valency of each site, and the partition function $`Z(\beta _s)`$ is given by

$$Z(\beta _s)=\mathrm{Tr}_\xi \mathrm{exp}\left(\frac{\beta _s}{z}\underset{ij}{}\xi _i\xi _j\right).$$
(2)
The data is coded by constructing the codewords, which are the products of $`p`$ spins $`J_{i_1\mathrm{}i_p}^0=\xi _{i_1}\mathrm{}\xi _{i_p}`$ for appropriately chosen sets of of indices $`\{i_1,\mathrm{},i_p\}`$, the choice of which determines the type of code. Each spin may appear in a number of $`p`$-spin codewords; the number of times of appearance is called the valency $`z_p`$. The Sourlas code  is equivalent to the infinite-range model in which all possible codewords of $`p`$ spins are chosen from $`N`$ spins. On the other hand, the Kabashima-Saad code  consists of combinations in which each spin appears in a random pre-selection of $`z_p`$ codewords. For conventional image restoration, codewords with only $`p=1`$ are transmitted, corresponding to the pixels in the image; the inclusion of terms with $`p>1`$, and their positive effects on restoring the original image, have also been discussed in . For simplicity, we restrict ourselves to the case of a single non-vanishing value of $`p`$ with $`p2`$, and $`p=1`$.

When the signal is transmitted through a noisy channel, the output consists of the sets $`\{J_{i_1\mathrm{}i_p}\}`$ and $`\{\tau _i\}`$, which are the corrupted versions of $`\{J_{i_1\mathrm{}i_p}^0\}`$ and $`\{\xi _i\}`$ respectively. In the binary symmetric channel, the outputs $`J_{i_1\mathrm{}i_p}`$ are equal to $`J_{i_1\mathrm{}i_p}^0`$ with probabilities $`p_J`$ and $`1p_J`$ respectively, and $`\tau _i`$ equal to $`\xi _i`$ with probabilities $`p_\tau `$ and $`1p_\tau `$ respectively. Thus

$$P_{\mathrm{out}}(\{J\},\{\tau \}|\{\xi \})\mathrm{exp}\left(\beta _JJ_{i_1\mathrm{}i_p}\xi _{i_1}\mathrm{}\xi _{i_p}+\beta _\tau \tau _i\xi _i\right),$$
(3)
where

$$\beta _J=\frac{1}{2}\mathrm{ln}\frac{1p_J}{p_J}\mathrm{and}\beta _\tau =\frac{1}{2}\mathrm{ln}\frac{1p_\tau }{p_\tau }.$$
(4)
The first summation in the exponent of Eq. (3) extends over an appropriate set of the indices $`(i_1,\mathrm{},i_p)`$.

The Gaussian channel is defined by, for a given sequence $`\{\xi _i\}`$,

$$P_{\mathrm{out}}(\{J\},\{\tau \}|\{\xi \})\mathrm{exp}\left(\frac{1}{2J^2}(J_{i_1\mathrm{}i_p}J_0\xi _{i_1}\mathrm{}\xi _{i_p})^2\frac{1}{2\tau ^2}(\tau _ia\xi _i)^2\right).$$
(5)
$`J_0`$ and $`a`$ are the strengths of the signals to be fed into the channel, and $`J^2`$ and $`\tau ^2`$ are the variances of the noise. We note that by letting $`\beta _J`$ and $`\beta _\tau `$ to be $`J_0/J^2`$ and $`a/\tau ^2`$ respectively, the input-dependent terms of Eq. (5) reduce to those of Eq. (3), which therefore can be regarded as the noise model for both binary symmetric and Gaussian channels.

According to Bayesian statistics, the posterior probability that the source sequence is $`\{\sigma \}`$, given the outputs $`\{J\}`$ and $`\{\tau \}`$, takes the form

$$P(\{\sigma \}|\{J\},\{\tau \})P_{\mathrm{out}}(\{J\},\{\tau \}|\{\sigma \})P_s(\{\sigma \}).$$
(6)
Using Eq. (3) and (1), we have

$$P(\{\sigma \}|\{J\},\{\tau \})\mathrm{exp}\left(\beta _JJ_{i_1\mathrm{}i_p}\sigma _{i_1}\mathrm{}\sigma _{i_p}+\beta _\tau \tau _i\sigma _i+\frac{\beta _s}{z}\underset{ij}{}\sigma _i\sigma _j\right).$$
(7)
It often happens that the receiver at the end of the noisy channel does not have precise information on $`\beta _J`$, $`\beta _\tau `$ or $`\beta _s`$. One then has to estimate these parameters. If the receiver estimates $`\beta `$, $`h`$ and $`\beta _m`$ for $`\beta _J`$, $`\beta _\tau `$ and $`\beta _s`$ respectively, then the mean of the posterior distribution of $`\sigma _i`$ is equal to the thermal average

$$\sigma _i=\frac{\mathrm{Tr}\sigma _ie^{H\{\sigma \}}}{\mathrm{Tr}e^{H\{\sigma \}}},$$
(8)
where the Hamiltonian is given by

$$H\{\sigma \}=\beta J_{i_1\mathrm{}i_p}\sigma _{i_1}\mathrm{}\sigma _{i_p}h\tau _i\sigma _i\frac{\beta _m}{z}\underset{ij}{}\sigma _i\sigma _j.$$
(9)
One then regards $`\mathrm{sgn}\sigma _i`$ as the $`i`$th bit of the decoded/restored information.

To reduce the sensitivity of the decoding/restoration process to the uncertainties in parameter estimation, we propose a two-stage process of selective freezing instead of the one-stage thermodynamic process implied by Eq. (8). In the first stage the spins evolve thermodynamically as prescribed in Eq. (8), and the thermal averages $`\sigma _i`$ of the spins are monitored. We may relate $`\sigma _i`$ to an effective field $`H_i`$ by $`\sigma _i=\mathrm{tanh}H_i`$. Spins with larger magnitudes of $`\sigma _i`$ correspond to larger magnitudes of $`H_i`$. They are more likely to agree with the correct message or image bit, and are less likely to change signs even when the hyperparameters vary. Their relative stability can be used to assist the less stable spins to boost their robustness against hyperparameter uncertainties. Hence we select those spins with $`|\sigma _i|`$ exceeding a given threshold $`\theta `$, and freeze them in the second stage of the thermodynamics. The average of the spin $`\stackrel{~}{\sigma }_i`$ in the second stage is then given by

$$\stackrel{~}{\sigma }_i=\frac{\mathrm{Tr}\stackrel{~}{\sigma }_i\underset{j}{}\left[\mathrm{\Theta }\left(\sigma _j^2\theta ^2\right)\delta _{\stackrel{~}{\sigma }_j,\mathrm{sgn}\sigma _j}+\mathrm{\Theta }\left(\theta ^2\sigma _j^2\right)\right]e^{\stackrel{~}{H}\{\stackrel{~}{\sigma }\}}}{\mathrm{Tr}_j\left[\mathrm{\Theta }\left(\sigma _j^2\theta ^2\right)\delta _{\stackrel{~}{\sigma }_j,\mathrm{sgn}\sigma _j}+\mathrm{\Theta }\left(\theta ^2\sigma _j^2\right)\right]e^{\stackrel{~}{H}\{\stackrel{~}{\sigma }\}}},$$
(10)
where $`\mathrm{\Theta }`$ is the step function, $`\stackrel{~}{H}\{\stackrel{~}{\sigma }\}`$ is the Hamiltonian for the second stage, and has the same form as Eq. (9) in the first stage. To increase the flexibility in the process, the parameters $`\beta `$, $`h`$ and $`\beta _m`$ can be replaced by $`\stackrel{~}{\beta }`$, $`\stackrel{~}{h}`$ and $`\stackrel{~}{\beta }_m`$ respectively in the second stage. One then regards $`\mathrm{sgn}\stackrel{~}{\sigma }_i`$ as the $`i`$th spin of the decoding/restoration process.

The most important quantity in selective freezing is the overlap of the decoded/restored bit $`\mathrm{sgn}\stackrel{~}{\sigma }_i`$ and the original bit $`\xi _i`$ averaged over the output probability and the spin distribution. This is given by

$$M_{\mathrm{sf}}=\underset{\xi }{}𝑑J𝑑\tau P_s(\{\xi \})P_{\mathrm{out}}(\{J\},\{\tau \}|\{\xi \})\xi _i\mathrm{sgn}\stackrel{~}{\sigma }_i.$$
(11)
Following Appendix A of , we can prove the following inequality

$$M_{\mathrm{sf}}M(\beta =\beta _J,h=\beta _\tau ,\beta _m=\beta _s),$$
(12)
where the right hand side is the overlap of the single-stage dynamics when the model parameters $`\beta `$, $`h`$ and $`\beta _m`$ match the source parameters $`\beta _J`$, $`\beta _\tau `$ and $`\beta _s`$ respectively. Hence selective freezing cannot outperform the single-stage process if the hyperparameters can be estimated precisely. However, we remark that the purpose of selective freezing is rather to provide a relatively stable performance when the hyperparameters cannot be estimated precisely. This cannot be revealed from the inequality, but will be confirmed by the analytic and simulation results in Sections III and IV.

## III THE INFINITE-RANGE MODEL FOR ERROR-CORRECTING CODES

Let us now suppose that the output of the transmission channel consists of only the set of $`p`$-spin interactions $`\{J_{i_1\mathrm{}i_p}\}`$. The Hamiltonian (9) then becomes

$$H\{\sigma \}=\beta \underset{i_1<\mathrm{}<i_p}{}J_{i_1\mathrm{}i_p}\sigma _{i_1}\mathrm{}\sigma _{i_p},$$
(13)
where we have set $`\beta _m=0`$ for the case that all messages are equally probable.

Analytical solutions for the overlap are in general unavailable. We therefore consider the infinite-range model in which the exchange interactions are present for all possible pairs of sites in the Hamiltonian of Eq. (13).

To consider the transition between error-free and errored regimes, we are interested in the noise model in which $`J_{i_1\mathrm{}i_p}`$ is Gaussian with mean $`p!j_0\xi _{i_1}\mathrm{}\xi _{i_p}/N^{p1}`$ and variance $`p!J^2/2N^{p1}`$. Since all messages are equally probable, we can apply a gauge transformation $`\sigma _i\sigma _i\xi _i`$ and $`J_{i_1\mathrm{}i_p}J_{i_1\mathrm{}i_p}\xi _{i_1}\mathrm{}\xi _{i_p}`$ to (13), and arrive at an equivalent $`p`$-spin model with a ferromagnetic bias, where

$$P(J_{i_1\mathrm{}i_p})=\left(\frac{N^{p1}}{\pi J^2p!}\right)^{1/2}\mathrm{exp}\left[\frac{N^{p1}}{J^2p!}\left(J_{i_1\mathrm{}i_p}\frac{p!}{N^{p1}}j_0\right)^2\right].$$
(14)
The Nishimori point for this model is located at $`\beta =2j_0/J^2`$.

The infinite-range model is exactly solvable using mean-field theoretical techniques for disordered systems such as the replica or cavity method . Here we use the cavity method because of its more transparent physical interpretation, and some obstacles encountered in the use of the replica method.

The cavity method uses a self-consistency argument to consider what happens when a spin is added or removed from the system. The central quantity in this method is the cavity field, which is the local field of a spin when it is added to the system, assuming that the exchange couplings act only one-way from the system to the new spin (but not from the spin back to the system). Since the exchange couplings feeding the new spin have no correlations with the system, the cavity field becomes a Gaussian variable in the limit of large valency.

### A Average spin in the first stage

We start with the so-called “clustering property” for mean-field systems ,

$$\sigma _{i_1}\mathrm{}\sigma _{i_p}=\sigma _{i_1}\mathrm{}\sigma _{i_p},$$
(15)
where $``$ represents thermodynamic averages. As shown in Appendix A, the clustering property enables us to express the thermal averages of a spin in terms of the cavity field, say, for spin 1,

$$\sigma _1=\mathrm{tanh}\beta h_1;h_1=\underset{1<j_2<\mathrm{}<j_p}{}J_{1j_2\mathrm{}j_p}\sigma _{j_2}^{\backslash 1}\mathrm{}\sigma _{j_p}^{\backslash 1},$$
(16)
where the superscript $`\backslash 1`$ denotes the thermal averages for a Hamiltonian in which $`\sigma _1`$ and the associated exchange interactions are absent, but otherwise identical to Eq. (13). Thus $`h_1`$ is the cavity field obeying a Gaussian distribution, whose mean and variance are $`pj_0m^{p1}`$ and $`pJ^2q^{p1}/2`$ respectively, where $`m`$ and $`q`$ are the magnetization and Edwards-Anderson order parameter respectively, given by

$$m\frac{1}{N}\underset{i}{}\sigma _i\mathrm{and}q\frac{1}{N}\underset{i}{}\sigma _i^2.$$
(17)
It is convenient to write

$$\beta h_i=\widehat{m}+\sqrt{\widehat{q}}u_i,$$
(18)
where

$$\widehat{m}=p\beta j_0m^{p1}\mathrm{and}\widehat{q}=\frac{p}{2}\beta ^2J^2q^{p1},$$
(19)
and $`u_i`$ is a Gaussian variable with mean 0 and variance 1.

### B Order parameters in the first stage

Applying self-consistently the cavity argument to all terms in Eq. (17), we can obtain self-consistent equations for $`m`$ and $`q`$:

$`m`$ $`=`$ $`{\displaystyle Du\mathrm{tanh}G},`$ (20)
$`q`$ $`=`$ $`{\displaystyle Du\mathrm{tanh}^2G},`$ (21)
where $`Dudue^{u^2/2}/\sqrt{2\pi }`$ is the Gaussian measure, $`G=\widehat{m}+\sqrt{\widehat{q}}u`$. The overlap for the one-stage decoding process is given by

$$M\frac{1}{N}\underset{i}{}\mathrm{sgn}\sigma _i=\mathrm{erf}\frac{\widehat{m}}{\sqrt{2\widehat{q}}}.$$
(22)
Now we consider selective freezing. If we introduce a freezing threshold $`\theta `$ so that all spins with $`\sigma _i^2>\theta ^2`$ are frozen, then the freezing fraction $`f`$ is given by

$$f\frac{1}{N}\underset{i}{}\mathrm{\Theta }\left(\sigma _i^2\theta ^2\right)=1\frac{1}{2}\mathrm{erf}\frac{u_+}{\sqrt{2}}+\frac{1}{2}\mathrm{erf}\frac{u_{}}{\sqrt{2}},$$
(23)
where $`u_\pm =(\pm u_0\widehat{m})/\sqrt{\widehat{q}}`$ with $`\mathrm{tanh}u_0=\theta `$.

### C Average spin in the second stage

Assuming that the spin $`\stackrel{~}{\sigma }_1`$ is dynamic in the second stage, we can write

$$H\{\stackrel{~}{\sigma }\}H\{\stackrel{~}{\sigma }\}^{\backslash 1}\stackrel{~}{\beta }\underset{1<j_1\mathrm{}<j_{p1}}{}\stackrel{~}{\sigma }_1J_{1j_1\mathrm{}j_{p1}}\underset{s=1}{\overset{p1}{}}\left[\stackrel{~}{\sigma }_{j_s}\mathrm{\Theta }\left(\theta ^2\sigma _{j_s}^2\right)+\mathrm{sgn}\sigma _{j_s}\mathrm{\Theta }\left(\sigma _{j_s}^2\theta ^2\right)\right],$$
(24)
where $`H\{\stackrel{~}{\sigma }\}^{\backslash 1}`$ is the Hamiltonian when spin 1 is completely removed from the system in both stages of the thermodynamic process. Removing spin 1 may cause the thermal averages of other spins to adjust slightly in the first stage. Hence some dynamic spins (with $`\sigma _k^2<\theta ^2`$) may become frozen ones (with $`\sigma _k^2>\theta ^2`$) and vice versa, so that strictly speaking, further terms should be considered in Eq. (24) to account for these secondary effects. For example, if spin $`k`$ is induced to switch from dynamic to frozen (or vice versa) on removal of spin 1, then the Taylor expansion of $`H\{\stackrel{~}{\sigma }\}`$ implies that an extra term

$`\stackrel{~}{\beta }\left(\mathrm{sgn}\sigma _k^{\backslash 1}\stackrel{~}{\sigma }_k\right)\left[\delta (\sigma _k^{\backslash 1}\theta )\delta (\sigma _k^{\backslash 1}+\theta )\right](\sigma _k\sigma _k^{\backslash 1})`$ (25)
$`{\displaystyle \underset{1<j_1\mathrm{}<j_{p1}k}{}}J_{kj_1\mathrm{}j_{p1}}{\displaystyle \underset{s=1}{\overset{p1}{}}}\left\{\stackrel{~}{\sigma }_{j_s}\mathrm{\Theta }\left[\theta ^2(\sigma _{j_s}^{\backslash 1})^2\right]+\mathrm{sgn}\sigma _{j_s}^{\backslash 1}\mathrm{\Theta }\left[(\sigma _{j_s}^{\backslash 1})^2\theta ^2\right]\right\}`$ (26)
should be incorporated in Eq. (24). Here, we have neglected these terms for clarity. Nevertheless, justification a posteriori can be provided for their deletion.

Using a cavity argument similar to Appendix A, we can show that

$$\stackrel{~}{\sigma }_1=\mathrm{tanh}\stackrel{~}{\beta }\left\{\underset{1<j_1\mathrm{}<j_{p1}}{}J_{1j_1\mathrm{}j_{p1}}\underset{s=1}{\overset{p1}{}}\left[\stackrel{~}{\sigma }_{j_s}^{\backslash 1}\mathrm{\Theta }\left(\theta ^2\sigma _{j_s}^2\right)+\mathrm{sgn}\sigma _{j_s}^{\backslash 1}\mathrm{\Theta }\left(\sigma _{j_s}^2\theta ^2\right)\right]\right\}.$$
(27)
However, the effective field on the right hand side of Eq. (27) is still not a cavity field because $`\sigma _{j_s}`$, which are used in the step functions to decide whether the spin $`j_s`$ is dynamic or frozen in the second stage, is different from $`\sigma _{j_s}^{\backslash 1}`$. Hence it may have correlations with spin 1. Taylor expansion of $`\sigma _{j_s}`$ about $`\sigma _{j_s}^{\backslash 1}`$ yields

$`\stackrel{~}{\sigma }_1=\mathrm{tanh}\stackrel{~}{\beta }\{\stackrel{~}{h}_1+{\displaystyle \underset{1jj_1\mathrm{}<j_{p2}}{}}J_{1jj_1\mathrm{}j_{p2}}`$ (28)
$`{\displaystyle \underset{s=1}{\overset{p2}{}}}\left[\stackrel{~}{\sigma }_{j_s}^{\backslash 1}\mathrm{\Theta }\left[\theta ^2(\sigma _{j_s}^{\backslash 1})^2\right]+\mathrm{sgn}\sigma _{j_s}^{\backslash 1}\mathrm{\Theta }\left[(\sigma _{j_s}^{\backslash 1})^2\theta ^2\right]\right]`$ (29)
$`[\mathrm{sgn}\sigma _j^{\backslash 1}\stackrel{~}{\sigma }_j^{\backslash 1}][\delta (\sigma _j^{\backslash 1}\theta )\delta (\sigma _j^{\backslash 1}+\theta )](\sigma _j\sigma _j^{\backslash 1})\},`$ (30)
where $`\stackrel{~}{h}_1`$ is the generic cavity field which is now completely uncorrelated with spin 1. It is given by

$$\stackrel{~}{h}_1=\underset{1<j_1\mathrm{}<j_{p1}}{}J_{1j_1\mathrm{}j_{p1}}\underset{s=1}{\overset{p1}{}}\left\{\stackrel{~}{\sigma }_{j_s}^{\backslash 1}\mathrm{\Theta }\left[\theta ^2(\sigma _{j_s}^{\backslash 1})^2\right]+\mathrm{sgn}\sigma _{j_s}^{\backslash 1}\mathrm{\Theta }\left[(\sigma _{j_s}^{\backslash 1})^2\theta ^2\right]\right\}.$$
(31)
To evaluate the difference $`\sigma _j\sigma _j^{\backslash 1}`$ appearing in Eq. (30), we have to apply the cavity method a second time, by comparing the changes when both spins 1 and $`j`$ are removed. This is done in Appendix B and the result is

$$\sigma _j\sigma _j^{\backslash 1}=\left(\beta \mathrm{sech}^2\beta h_j^{\backslash 1}\right)\left(h_{j1}\mathrm{tanh}\beta h_1^{\backslash j}\right),$$
(32)
where

$$h_{1j}=h_{j1}=\underset{1jk_1\mathrm{}<k_{p2}}{}J_{1jk_1\mathrm{}k_{p2}}\sigma _{k_1}^{\backslash 1j}\mathrm{}\sigma _{k_{p2}}^{\backslash 1j}.$$
(33)
When Eqs. (32-33) are substituted into Eq. (30), the significant contribution comes from the terms which pair up $`J_{1jj_1\mathrm{}j_{p2}}`$ and $`J_{1jk_1\mathrm{}k_{p2}}`$. The various terms appearing in the summation over $`jj_1<\mathrm{}<j_{p2}`$ involve thermal averages in the absence of spins 1 or $`j`$. We assume that the effects of removing a spin is negligible (which can be shown to be equivalent to the replica symmetric approximation in the replica method ). Then replacing the components of the terms by their mean values, and counting that $`N^{p2}/(p2)!`$ terms appearing in the summation over $`j_1<\mathrm{}<j_{p2}`$, we arrive at

$`\stackrel{~}{\sigma }_1=\mathrm{tanh}\stackrel{~}{\beta }\{\stackrel{~}{h}_1+{\displaystyle \frac{p}{2}}(p1)J^2{\displaystyle \frac{1}{N}}{\displaystyle \underset{j}{}}[\delta (\sigma _j\theta )\delta (\sigma _j+\theta )][\mathrm{sgn}\sigma _j\stackrel{~}{\sigma }_j]`$ (34)
$`\left(\beta \mathrm{sech}^2\beta h_j\right)\left(r^{p2}\mathrm{tanh}\beta h_1\right)\},`$ (35)
where $`r`$ is the order parameter describing the spin correlations of the two thermodynamic stages:

$$r\frac{1}{N}\underset{i}{}\sigma _i\left\{\stackrel{~}{\sigma }_i\mathrm{\Theta }\left[\theta ^2\sigma _i^2\right]+\mathrm{sgn}\sigma _i\mathrm{\Theta }\left[\sigma _i^2\theta ^2\right]\right\}.$$
(36)
Eq. (35) can be simplified by introducing the trans-susceptibility $`\chi _{tr}`$, which describes the response of a spin in the second stage to variations of the cavity field in the first stage, namely

$$\chi _{tr}\frac{1}{N}\underset{i}{}\frac{\stackrel{~}{\sigma }_i}{h_i}.$$
(37)
Since $`\stackrel{~}{\sigma }_i`$ equals $`\mathrm{sgn}h_i`$ for $`\mathrm{tanh}^2\beta h_i>\theta ^2`$, and $`\mathrm{tanh}\beta \stackrel{~}{h}_i`$ otherwise, we get

$$\chi _{tr}=\frac{1}{N}\underset{i}{}\left[\delta (\sigma _i\theta )\delta (\sigma _i+\theta )\right]\left[\mathrm{sgn}\sigma _i\stackrel{~}{\sigma }_i\right]\beta \mathrm{sech}^2\beta h_i.$$
(38)
Eq. (35) can thus be simplified to

$$\stackrel{~}{\sigma }_1=\mathrm{tanh}\stackrel{~}{\beta }\left\{\stackrel{~}{h}_1+\frac{p}{2}(p1)J^2r^{p2}\chi _{tr}\mathrm{tanh}\beta h_1\right\}.$$
(39)
### D Order parameters in the second stage

The cavity field $`\stackrel{~}{h}_1`$ in the second stage is a Gaussian variable. Its mean and variance are $`pj_0\stackrel{~}{m}^{p1}`$ and $`pJ^2\stackrel{~}{q}^{p1}/2`$ respectively, where $`\stackrel{~}{m}`$ and $`\stackrel{~}{q}`$ are the magnetization and Edwards-Anderson order parameter respectively, given by

$`\stackrel{~}{m}`$ $``$ $`{\displaystyle \frac{1}{N}}{\displaystyle \underset{i}{}}\left[\mathrm{\Theta }(\theta ^2\sigma _i^2)\stackrel{~}{\sigma }_i+\mathrm{\Theta }(\sigma _i^2\theta ^2)\mathrm{sgn}\sigma _i\right],`$ (40)
$`\stackrel{~}{q}`$ $``$ $`{\displaystyle \frac{1}{N}}{\displaystyle \underset{i}{}}\left[\mathrm{\Theta }(\theta ^2\sigma _i^2)\stackrel{~}{\sigma }_i^2+\mathrm{\Theta }(\sigma _i^2\theta ^2)\right].`$ (41)
Furthermore, the covariance between $`h_1`$ and $`\stackrel{~}{h}_1`$ is $`pJ^2r^{p1}/2`$, where $`r`$ is given in Eq. (36).

Algebraic manipulations can be simplified if we write, for $`i=1`$,

$`\beta h_i`$ $`=`$ $`\widehat{m}+\sqrt{\widehat{q}}u_i,`$ (42)
$`\stackrel{~}{\beta }\stackrel{~}{h}_i`$ $`=`$ $`\widehat{\stackrel{~}{m}}+\sqrt{\widehat{\stackrel{~}{q}}}(\eta u_i+\sqrt{1\eta ^2}v_i),`$ (43)
where $`u_i`$ and $`v_i`$ are independent Gaussian variables with mean 0 and variance 1, $`\widehat{m}`$, $`\widehat{q}`$ are given in Eq. (19), and

$`\widehat{\stackrel{~}{m}}=p\stackrel{~}{\beta }j_0\stackrel{~}{m}^{p1},\mathrm{and}\widehat{\stackrel{~}{q}}={\displaystyle \frac{p}{2}}\stackrel{~}{\beta }^2J^2\stackrel{~}{q}^{p1},`$ (44)
$`\widehat{r}={\displaystyle \frac{p}{2}}\beta \stackrel{~}{\beta }J^2r^{p1},\mathrm{and}\eta ={\displaystyle \frac{\widehat{r}}{\sqrt{\widehat{q}\widehat{\stackrel{~}{q}}}}}.`$ (45)
Applying self-consistently the same cavity argument to all terms in Eqs. (40), (41), (36) and (38) and performing the Gaussian average over $`u_i`$ and $`v_i`$, we arrive at the following self-consistent equations for $`\stackrel{~}{m}`$, $`\stackrel{~}{q}`$, $`r`$ and $`\chi _{tr}`$:

$`\stackrel{~}{m}`$ $`=`$ $`{\displaystyle \frac{1}{2}}\mathrm{erf}{\displaystyle \frac{u_+}{\sqrt{2}}}{\displaystyle \frac{1}{2}}\mathrm{erf}{\displaystyle \frac{u_{}}{\sqrt{2}}}+{\displaystyle _u_{}^{u_+}}Du{\displaystyle Dv\mathrm{tanh}L},`$ (46)
$`\stackrel{~}{q}`$ $`=`$ $`1{\displaystyle \frac{1}{2}}\mathrm{erf}{\displaystyle \frac{u_+}{\sqrt{2}}}+{\displaystyle \frac{1}{2}}\mathrm{erf}{\displaystyle \frac{u_{}}{\sqrt{2}}}+{\displaystyle _u_{}^{u_+}}Du{\displaystyle Dv\mathrm{tanh}^2L},`$ (47)
$`r`$ $`=`$ $`\left({\displaystyle _{\mathrm{}}^u_{}}+{\displaystyle _{u_+}^{\mathrm{}}}\right)Du\left|\mathrm{tanh}G\right|+{\displaystyle _u_{}^{u_+}}Du{\displaystyle Dv\mathrm{tanh}G\mathrm{tanh}L},`$ (48)
$`\chi _{tr}`$ $`=`$ $`{\displaystyle \frac{\mathrm{exp}(u_+^2/2)}{J\sqrt{\pi pq^{p1}}}}{\displaystyle Dv(1\mathrm{tanh}L_v^{(+)})}+{\displaystyle \frac{\mathrm{exp}(u_{}^2/2)}{J\sqrt{\pi pq^{p1}}}}{\displaystyle Dv(1+\mathrm{tanh}L_v^{()})},`$ (49)
where

$`L`$ $`=`$ $`\widehat{\stackrel{~}{m}}+\sqrt{\widehat{\stackrel{~}{q}}}(\eta u+\sqrt{1\eta ^2}v)+{\displaystyle \frac{p}{2}}(p1)\stackrel{~}{\beta }J^2r^{p2}\chi _{tr}\mathrm{tanh}G,`$ (50)
$`L_v^{(\pm )}`$ $`=`$ $`\widehat{\stackrel{~}{m}}+\sqrt{\widehat{\stackrel{~}{q}}}(\eta u_\pm +\sqrt{1\eta ^2}v)\pm {\displaystyle \frac{p}{2}}(p1)\stackrel{~}{\beta }J^2r^{p2}\chi _{tr}\theta .`$ (51)
Eqs. (20-21), (46-49) for the order parameters $`m`$, $`q`$, $`\stackrel{~}{m}`$, $`\stackrel{~}{q}`$, $`r`$ and $`\chi _{tr}`$ form a close set of equations. The performance of selective freezing is measured by

$$M_{\mathrm{sf}}\frac{1}{N}\underset{i}{}\left[\mathrm{\Theta }(\theta ^2\sigma _i^2)\mathrm{sgn}\stackrel{~}{\sigma }_i+\mathrm{\Theta }(\sigma _i^2\theta ^2)\mathrm{sgn}\sigma _i\right].$$
(52)
From the above parameters, $`M_{\mathrm{sf}}`$ can be derived as:

$$M_{\mathrm{sf}}=\frac{1}{2}\mathrm{erf}\frac{u_+}{\sqrt{2}}\frac{1}{2}\mathrm{erf}\frac{u_{}}{\sqrt{2}}+_u_{}^{u_+}Du\mathrm{erf}\frac{L_u}{\sqrt{2\stackrel{~}{q}(1\eta ^2)}},$$
(53)
where $`L_u=\widehat{\stackrel{~}{m}}+\sqrt{\widehat{\stackrel{~}{q}}}\eta u+[p(p1)/2]\stackrel{~}{\beta }J^2r^{p2}\chi _{tr}\mathrm{tanh}G`$.

We have also tried to derive the above equations using the replica method. However, in the nearest results that we could find, terms involving the trans-susceptbility are absent, which we believe to be unphysical. Therefore the replica approach to the order parameter equations remain an open question.

We show an example of the case $`p=2`$ and $`j_0=J=1`$ in Fig. 1, where the overlap $`M_{\mathrm{sf}}`$ is plotted as a function of the decoding temperature $`T(=\beta ^1=\stackrel{~}{\beta }^1)`$ for various given values of freezing fraction $`f`$. When $`f=0`$ (no spins frozen) and $`f=1`$ (all spins frozen), the dynamics is equivalent to one with single stage, and the overlap reaches its maximum at the Nishimori point $`T=J^2/2j_0`$ as expected. We observe that the tolerance against variations in $`T`$ is enhanced by selective freezing for certain values of $`f`$.

It is therefore interesting to consider the appropriate values of $`f`$ for the best overlap at a given decoding temperature. Figs. 2(a-f) shows that at high temperatures such as in Figs. 2(a-c), there is a single maximum and its position is fairly independent of temperature, lying around $`f=0.9`$ in the present case. At intermediate temperatures such as in Figs. 2(d-e), there appear two maxima and as temperature changes, there is a discontinuous jump in the maximum position. Fig. 2(f) shows that when the temperature is lower than the Nishimori point ($`T_N=0.5`$), the overlap cannot be improved by selective freezing.

Figure 3 compares the overlap of the one-stage dynamics with that of the best of selective freezing. It shows that when the decoding temperature is mis-determined to be higher than its optimal value at the Nishimori point, selective freezing can provide a fairly robust performance. Furthermore, the choice of the freezing fraction for such robust performance appears to be quite independent of the temperature. The solid line in Fig. 4 locates the position for the best overlap and, as observed from Figs. 2(a-f), lies in the vicinity of $`f0.9`$ for a large range of temperature. The unshaded region in the same figure also indicates that selective freezing leads to an improvement in the overlap over a wide range of the parameter space.

We have also studied the dependence of the overlap on varying the freezing threshold $`\theta `$ rather than the freezing fraction $`f`$. However, Fig. 5 shows that the optimal value of $`\theta `$ has a much larger dependence on the temperature. This is due to the sensitive dependence of the thermal averages of the spins on temperature. At high temperatures, most spins are thermally agitated, and the freezing threshold has to be set to a very low value in order to freeze a given fraction of spins. On the other hand, at low temperatures, most spins are relatively stable, and the freezing threshold has to be set to a very high value in order to keep a given fraction of spins dynamic in the second stage. We conclude that the freezing fraction is a better controlling parameter for the decoding performance.

The advantages of selective freezing are confirmed by Monte Carlo simulations shown in Fig. 6. For one-stage dynamics, the overlap is maximum at the Nishimori point ($`T_N=0.5`$) as expected. However, it deterriorates rather rapidly when the decoding temperature increases. In contrast, selective freezing maintains a more steady performance, especially when $`f=0.9`$.

## IV THE MEAN-FIELD MODEL FOR IMAGE RESTORATION

In conventional image restoration problems, a given degraded image consists of the set of pixels $`\{\tau _i\}`$, but not the set of exchange interactions $`\{J_{i_1,\mathrm{},i_p}\}`$. On the other hand, effective restoration requires the introduction of a model prior distribution of the pixels for smooth images. In this case the Hamiltonian corresponds to that of a random field Ising model,

$$H\{\sigma \}=h\underset{i}{}\tau _i\sigma _i\frac{\beta _m}{z}\underset{ij}{}\sigma _i\sigma _j.$$
(54)
In mean-field systems, each pixel $`i`$ has an extensive valency. The pixels $`\tau _i`$ are the degraded versions of the source pixels $`\xi _i`$, corrupted by noise which, for convenience, is assumed to be Gaussian with mean $`a\xi _i`$ and variance $`\tau ^2`$, i.e.

$$P(\tau _i|\xi _i)=\frac{\mathrm{exp}\left[\frac{1}{2\tau ^2}(\tau _ia\xi _i)^2\right]}{\sqrt{2\pi \tau ^2}}.$$
(55)
In turn, the source pixels satisfy the prior distribution in Eq. (1). Applying the cavity argument for mean-field systems, the prior distribution becomes factorizable,

$$P(\xi _i)=\frac{\mathrm{exp}(\beta _sm_0\xi _i)}{2\mathrm{cosh}\beta _sm_0},$$
(56)
where $`m_0=\mathrm{tanh}\beta _sm_0`$. The order parameter in the first stage is given by

$$m\frac{1}{N}\underset{i}{}\sigma _i=\frac{1}{2\mathrm{cosh}\beta _sm_0}\underset{\xi =\pm 1}{}\mathrm{exp}(\beta _sm_0\xi )Dx\mathrm{tanh}U,$$
(57)
where $`U=\beta _mm+ha\xi +h\tau x`$. The overlap for the one-stage restoration process is given by

$$M\frac{1}{N}\underset{i}{}\xi _i\mathrm{sgn}\sigma _i=\frac{1}{2\mathrm{cosh}\beta _sm_0}\underset{\xi =\pm 1}{}\mathrm{exp}(\beta _sm_0\xi )\xi \mathrm{erf}\frac{\beta _mm+ha\xi }{\sqrt{2}h\tau }.$$
(58)
Next we consider selective freezing in the second stage with a freezing threshold $`\theta `$. The freezing fraction is given by

$$f\frac{1}{N}\underset{i}{}\mathrm{\Theta }\left(\sigma _i^2\theta ^2\right)=\frac{1}{2\mathrm{cosh}\beta _sm_0}\underset{\xi =\pm 1}{}\mathrm{exp}(\beta _sm_0\xi )\left[1\frac{1}{2}\mathrm{erf}\frac{u_+(\xi )}{\sqrt{2}}+\frac{1}{2}\mathrm{erf}\frac{u_{}(\xi )}{\sqrt{2}}\right],$$
(59)
where $`u_\pm (\xi )=(\pm u_0\beta _mmha\xi )/h\tau `$ with $`\mathrm{tanh}u_0=\theta `$. The order parameter of the second stage is given by

$`\stackrel{~}{m}`$ $``$ $`{\displaystyle \frac{1}{N}}{\displaystyle \underset{i}{}}\left[\mathrm{\Theta }(\theta ^2\sigma _i^2)\stackrel{~}{\sigma }_i+\mathrm{\Theta }(\sigma _i^2\theta ^2)\mathrm{sgn}\sigma _i\right]`$ (60)
$`=`$ $`{\displaystyle \frac{1}{2\mathrm{cosh}\beta _sm_0}}{\displaystyle \underset{\xi =\pm 1}{}}\mathrm{exp}(\beta _sm_0\xi )\left[{\displaystyle \frac{1}{2}}\mathrm{erf}{\displaystyle \frac{u_+(\xi )}{\sqrt{2}}}{\displaystyle \frac{1}{2}}\mathrm{erf}{\displaystyle \frac{u_{}(\xi )}{\sqrt{2}}}+{\displaystyle _{u_{}(\xi )}^{u_+(\xi )}}Dx\mathrm{tanh}L\right],`$ (61)
where $`L=\beta _m\stackrel{~}{m}+ha\xi +h\tau x`$. The overlap for selective freezing is given by

$`M_{\mathrm{sf}}`$ $``$ $`{\displaystyle \frac{1}{N}}{\displaystyle \underset{i}{}}\xi _i\left[\mathrm{\Theta }(\theta ^2\sigma _i^2)\mathrm{sgn}\stackrel{~}{\sigma }_i+\mathrm{\Theta }(\sigma _i^2\theta ^2)\mathrm{sgn}\sigma _i\right]`$ (62)
$`=`$ $`{\displaystyle \frac{1}{2\mathrm{cosh}\beta _sm_0}}{\displaystyle \underset{\xi =\pm 1}{}}\mathrm{exp}(\beta _sm_0\xi )\xi \mathrm{erf}{\displaystyle \frac{g(\beta _m\stackrel{~}{m})+ha\xi }{\sqrt{2}h\tau }},`$ (63)
where

$$g(\beta _m\stackrel{~}{m})=\{\begin{array}{ccccc}\beta _mmu_0& & & & \beta _m\stackrel{~}{m}<\beta _mmu_0,\\ \beta _m\stackrel{~}{m}& & & & \beta _mmu_0<\beta _m\stackrel{~}{m}<\beta _mm+u_0,\\ \beta _mm+u_0& & & & \beta _m\stackrel{~}{m}>\beta _mm+u_0.\end{array}$$
(64)
We note that since the spin-glass interaction is absent in this case, there are no trans-susceptibility effects. This is unlike the case of error-correcting codes, in which $`\chi _{tr}`$ is nonzero when $`J`$ is nonzero.

The three cases of the function $`g(\beta _m\stackrel{~}{m})`$ in Eq. (64) correspond to three situations. When $`\beta _m\stackrel{~}{m}<\beta _mmu_0`$, all the dynamic spins in the second stage have negative thermodynamic averages and therefore take the value $`1`$ in the two-stage restoration process. This is equivalent to a one-stage restoration process in which all spins with thermodynamic averages above the threshold $`+\theta `$ are frozen to $`+1`$, and to $`1`$ otherwise. Similarly, when $`\beta _m\stackrel{~}{m}>\beta _mm+u_0`$, all the dynamic spins in the second stage have positive thermodynamic averages. Only when $`\beta _mmu_0<\beta _m\stackrel{~}{m}<\beta _mm+u_0`$, do we have the dynamic spins frozen to partly $`+1`$ and partly $`1`$.

We can consider the condition for the optimal performance $`M_{\mathrm{sf}}`$ of selective freezing. For a given distribution of data and noise, $`g(\beta _m\stackrel{~}{m})`$ is the only adjustable parameter in Eq. (63), playing the same role as the adjustable parameter $`\beta _mm`$ for one-stage dynamics in Eq. (58). In the space of $`h`$ and $`\beta _m`$, the performance is optimal along the line $`h/\beta _\tau =\beta _mm/\beta _sm_0`$ for one-stage dynamics  ($`\beta _\tau =a/\tau ^2`$ for Gaussian noise). Analogously, there exists a line of optimal performance defined by $`h/\beta _\tau =g(\beta _m\stackrel{~}{m})/\beta _sm_0`$ for selective freezing.

An example of the lines of optimal performance is shown in Fig. 7. It is interesting to note the kinks for certain freezing fractions. They correspond to transitions of cases in which the dynamic spins are partially or completely frozen to $`\pm 1`$.

A comparison of Eqs. (58) and (63) shows that selective freezing performs as well as one-stage dynamics, but cannot outperform it. Nevertheless, selective freezing provides a rather stable performance when the hyperparameters cannot be estimated precisely. In image restoration, the usual practice is to choose a fixed ratio of $`\beta _m/h`$. Fig. 8 confirms this stability along the line of operation with $`\beta _m/h`$ set to the optimal ratio $`\beta _s/\beta _\tau `$. Note especially that the lines with $`f=0.7`$ and $`0.9`$ attain a nearly optimal value of $`M_{\mathrm{sf}}`$ over a wide range of parameters. The kink at $`f=0.9`$ is, again, due to the appearance of the $`1`$ frozen dynamic spins (to the right of the kink).

The stable performance of selective freezing can be partly explained by the proximity of the lines of optimal performance with the line of operation which, as discussed in , is an important factor in hyperparameter estimation. This is illustrated by the optimal lines for small values of $`f`$ near the Nishimori point $`(T_m,h)=(1.05^1,1)`$ in Fig. 7.

However, the advantage of selective freezing does not only rely on the fortuitous combination of parameters. Even when the parameters are not chosen optimally, selective freezing still maintains a rather robust performance. For example, along the line of optimal performance for $`f=0.9`$ in Fig. 7, the bending at the kink only causes a modest reduction in the overlap $`M_{\mathrm{sf}}`$ in Fig. 8.

To study the robustness of the performance of selective freezing, we model a situation common in modern communication channels carrying multimedia traffic, which are often bursty in nature. Since burstiness results in intermittent interferences, we consider a noise with two Gaussian components, each with its own characteristics. A random fraction $`f_1`$ of the pixels are influenced by Gaussian noise with signal strength $`a_1`$ and noise variance $`\tau _1^2`$. The rest of the pixels have strength $`a_2`$ and noise variance $`\tau _2^2`$. Hence the distribution of the degraded pixels are

$$P(\tau _i|\xi _i)=f_1\frac{\mathrm{exp}\left[\frac{1}{2\tau _1^2}(\tau _ia_1\xi _i)^2\right]}{\sqrt{2\pi \tau _1^2}}+f_2\frac{\mathrm{exp}\left[\frac{1}{2\tau _2^2}(\tau _ia_2\xi _i)^2\right]}{\sqrt{2\pi \tau _2^2}},$$
(65)
where $`f_2=1f_1`$. The equations for the order parameters can be generalized from the single component case in a straightforward manner.

A case of interest is that the restoration agent operates on the assumption of the characteristics of the majority component of the channel, say the first component. Hence it operates at the ratio $`\beta _m/h=\beta _s\tau _1^2/a_1`$. Suppose the Gaussian noise is partly interrupted to take the characteristics of the second component, but the operation parameters cannot be adjusted soon enough, then there will be a degradation of the quality of the restored images. In the example in Fig. 9, the reduction of the overlap $`M_{\mathrm{sf}}`$ for selective freezing is much more modest than the one-stage process ($`f=0`$).

An alternative situation is that the restoration agent is able to detect the changes in the average signal strengths and noise variance, but still operates on the assumption of a single-component Gaussian channel. Suppose that such simple statistics as $`\mathrm{sgn}\tau _i`$, $`\tau _i`$ and $`\tau _i^2`$ are accessible. Then the parameters $`m_0^{}`$, $`a^{}`$ and $`\tau ^{}`$ estimated by the restoration agent are obtained, for $`\tau _1=\tau _2=\tau `$, from the solutions of

$`m_0^{}\mathrm{erf}{\displaystyle \frac{a^{}}{\sqrt{2}\tau ^{}}}`$ $`=`$ $`\mathrm{sgn}\tau _i=m_0\left[f_1\mathrm{erf}{\displaystyle \frac{a_1}{\sqrt{2}\tau _1}}+f_2\mathrm{erf}{\displaystyle \frac{a_2}{\sqrt{2}\tau _2}}\right],`$ (66)
$`m_0^{}a^{}`$ $`=`$ $`\tau _i=m_0\left[f_1a_1+f_2a_2\right],`$ (67)
$`a^2+\tau ^2`$ $`=`$ $`\tau _i^2=f_1(a_1^2+\tau _1^2)+f_2(a_2^2+\tau _2^2),`$ (68)
and $`\beta _s^{}=\mathrm{tanh}^1m_0^{}/m_0^{}`$. Using these estimated parameters, the performances in Fig. 10 improve over their counterparts based on only the majority component in Fig. 9. Still, one-stage restoration cannot avoid the performance drop when $`h`$ vanishes, whereas correspondingly, selective freezing has a much more gentle drop in performance.

It is interesting to study the more realistic case of two-dimensional images, since we have so far presented analytical results for the mean field model only. As confirmed by the results for Monte carlo simulations in Fig. 11, the overlaps of selective freezing are much more steadier than that of the one-stage dynamics when the decoding temperature changes. This steadiness is most remarkable for a freezing fraction of $`f=0.9`$.

## V DISCUSSIONS

We have introduced a multistage technique for error-correcting codes and image restoration, in which the information extracted from the former stage can be used selectively to improve the performance of the latter one. While the overlap $`M_{\mathrm{sf}}`$ of the selective freezing is bounded by the optimal performance of the one-stage dynamics derived in , it has the advantage of being tolerant to uncertainties in hyperparameter estimation. The performance is especially steady when the fraction of frozen spins, rather than the threshold of their thermodynamic averages, is fixed in the process. This is confirmed by both analytical and simulational results for mean-field and finite-dimensional models. As an example, we have illustrated its advantage of robustness when the noise distribution is composed of more than one Gaussian components, such as in the case of modern communication channels supporting multimedia applications.

We found that selective freezing is most useful when more than one hyperparameters have to be estimated, as illustrated by the example of image restoration, where both $`\beta _m`$ and $`h`$ have to be estimated. In the example of error-correcting codes discussed in Section III, there is only one hyperparameter $`T_m`$, and it is found that selective freezing has performance advantages only when $`T_m`$ is chosen above the Nishimori point. However, more than one hyperparameters are often present in practical applications.

Selective freezing can be generalized to more than two stages, in which spins that remain relatively stable in one stage are progressively frozen in the following one. It is expected that the performance can be even more robust.

While the multistage process described here has a robust performance, it does not raise the critical temperature or the critical noise level for the existence of the ordered phase. Nor can it widen the basin of attraction for the ordered phase. Other multistage processes, proposed in  for neural networks, may be able to achieve this. This remains an area for further research.

We have made progress in the theoretical treatment of multistage processes using the cavity method. It allows the thermal averages of spins to be expressed in terms of the cavity fields. Since a cavity field is uncorrelated with the spin in consideration, it can in turn be expressed in terms of the means and covariances of the spin averages, thereby arriving at a set of self-consistent equations for the order parameters. In particular, there appears a trans-susceptibility term since variations of the cavity field in the first stage are correlated with the spin average in the second stage due to the selective nature of the freezing process in the second stage. However, for the ordered phase considered in this paper, the effects of the trans-susceptibility term is not too large except near the phase boundary.

On the other hand, we have a remark about the basic assumption of the cavity method, namely that the addition or removal of a spin causes a small change in the system describable by a perturbative approach. In fact, adding or removing a spin may cause the thermal averages of other spins to change from below to above the thresholds $`\pm \theta `$ (or vice versa). This change, though often small, induces a non-negligible change of the thermal averages from fractional values to the frozen values of $`\pm 1`$ (or vice versa) in the second stage. The perturbative analysis of these changes is only approximate. The situation is reminiscent of similar instabilities in other disordered systems such as the perceptron, and are equivalent to Almeida-Thouless instabilities in the replica method . A full treatment of the problem would require the introduction of a rough energy landscape , or the replica symmetry breaking ansatz in the replica method . Nevertheless, previous experiences on disordered systems showed that the corrections made by a more complete treatment may not be too large in the ordered phase. For example, corresponding analytical and simulational results in Figs. 1 and 6 respectively are close to each other.

In practical implementations of error-correcting codes, algorithms based on belief-propagation methods, rather than Monte Carlo methods, are often employed . It has recently been shown that such decoded messages converge to the solutions of the TAP equations in the corresponding thermodynamic system . Again, the performance of these algorithms are sensitive to the estimation of hyperparameters. We propose that the selective freezing procedure has the potential to make these algorithms more robust.

Incidentally, multistage dynamics has also been applied in the recently popular turbo codes . Messages are coded in sequences with two possible permutations and at each iterative stage, the information derived from decoding one sequence is fed to the other in the form of external fields for each bit. The techniques developed in the present context can be used to study this iterative process.

###### Acknowledgements.

KYMW wishes to thank Tokyo Institute of Technology for hospitality. HN is grateful to Hong Kong University of Science and Technology for hospitality. This work was partially supported by research grant HKUST6157/99P of the Research Grant Council of Hong Kong.

## A Thermal averages of spins

In this appendix we derive Eq. (16) starting from the clustering property Eq. (15). For convenience we illustrate the derivation for $`p=2`$. We separate the Hamiltonian into two parts, one does not contain $`\sigma _1`$ and the other does. Hence

$$H=H^{\backslash 1}\beta \underset{j>1}{}J_{1j}\sigma _1\sigma _j.$$
(A1)
Thus the thermal average can be written as

$$\sigma _1=\frac{\mathrm{Tr}^{\backslash 1}e^{H^{\backslash 1}}\mathrm{Tr}_1\sigma _1\mathrm{exp}\left(\beta \sigma _1\underset{j}{}J_{1j}\sigma _j\right)/\mathrm{Tr}^{\backslash 1}e^{H^{\backslash 1}}}{\mathrm{Tr}^{\backslash 1}e^{H^{\backslash 1}}\mathrm{Tr}_1\mathrm{exp}\left(\beta \sigma _1_jJ_{1j}\sigma _j\right)/\mathrm{Tr}^{\backslash 1}e^{H^{\backslash 1}}}.$$
(A2)
Expanding the exponential function in the denominator and tracing over $`\sigma `$, we get

$$\mathrm{Den}.=2\underset{n\mathrm{even}}{}\frac{\beta ^n}{n!}\underset{j_1\mathrm{}j_n}{}J_{1j_1}\mathrm{}J_{1j_n}\sigma _{j_1}\mathrm{}\sigma _{j_n}^{\backslash 1}.$$
(A3)
Next, we use the clustering property to factorize the thermal average $`\sigma _{j_1}\mathrm{}\sigma _{j_n}^{\backslash 1}`$. For the coupling distribution specified by Eq. (14), only two kinds of contributions are significant in the summation over the indices $`j_1\mathrm{}j_n`$. In the first kind, an index $`j`$ remains distinct from the rest, contributing a factor of $`J_{1j}\sigma _j^{\backslash 1}`$. In the second kind, two indices become paired up. However, when $`j`$ and $`k`$ pair up, the thermal average $`\sigma _j\sigma _k^{\backslash 1}`$ becomes 1 instead of $`(\sigma _j^{\backslash 1})^2`$. Hence the additional contribution due to the pairing is $`J_{1j}^2[1(\sigma _j^{\backslash 1})^2]`$. Other than these, the contributions due to the pairing of three or more indices are smaller by factors of $`N`$.

The denominator can now be considered a summation over $`n`$ and $`m`$, which are respectively the total number of indices and the number of pairs of paired indices appearing in a term. The number of such terms is $`n!/m!2^m(n2m)!`$. Hence

$$\mathrm{Den}.=2\underset{n\mathrm{even}}{}\underset{m=0}{\overset{n/2}{}}\frac{\beta ^n}{n!}\frac{n!}{m!2^m(n2m)!}\left[\underset{j}{}J_{1j}\sigma _j^{\backslash 1}\right]^{n2m}\left\{\underset{j}{}J_{1j}^2[1(\sigma _j^{\backslash 1})^2]\right\}^m,$$
(A4)
which can be simplified to

$$\mathrm{Den}.=2\mathrm{exp}\left\{\frac{1}{2}\beta ^2\underset{j}{}J_{1j}^2[1(\sigma _j^{\backslash 1})^2]\right\}\mathrm{cosh}\left\{\beta \underset{j}{}J_{1j}\sigma _j^{\backslash 1}\right\}.$$
(A5)
Similarly, the numerator can be written as

$$\mathrm{Num}.=2\mathrm{exp}\left\{\frac{1}{2}\beta ^2\underset{j}{}J_{1j}^2[1(\sigma _j^{\backslash 1})^2]\right\}\mathrm{sinh}\left\{\beta \underset{j}{}J_{1j}\sigma _j^{\backslash 1}\right\}.$$
(A6)
Substituting Eq. (A5) and (A6) into Eq. (A2), we arrive at Eq. (16).

## B Change in thermal averages on removal of a spin

In this appendix we derive Eq. (32). For convenience we illustrate the derivation for $`p=2`$. We separate the Hamiltonian into four parts: (a) does not contain spins 1 and $`j`$, (b) contains only spins 1 and $`j`$, (c) contains spin 1 but not $`j`$, (d) contains spin $`j`$ but not 1. This yields

$$H=H^{\backslash 1j}\beta J_{1j}\sigma _1\sigma _j\beta \underset{k1j}{}J_{k1}\sigma _k\sigma _1\beta \underset{k1j}{}J_{kj}\sigma _k\sigma _j.$$
(B1)
The thermal average of $`\sigma _j`$ can then be written as

$$\sigma _j=\frac{\mathrm{Tr}_{1j}\mathrm{Tr}^{\backslash 1j}e^H\sigma _j/\mathrm{Tr}^{\backslash 1j}e^{H^{\backslash 1j}}}{\mathrm{Tr}_{1j}\mathrm{Tr}^{\backslash 1j}e^H/\mathrm{Tr}^{\backslash 1j}e^{H^{\backslash 1j}}}.$$
(B2)
Using the mean-field technique developed in Appendix A, the denominator can be written as

$`\mathrm{Den}.=\mathrm{Tr}_{1j}\mathrm{exp}\{\beta J_{1j}\sigma _1\sigma _j`$ $`+`$ $`\beta {\displaystyle \underset{k1j}{}}\sigma _k^{\backslash 1j}(J_{k1}\sigma _1+J_{kj}\sigma _j)`$ (B3)
$`+`$ $`{\displaystyle \frac{1}{2}}\beta ^2{\displaystyle \underset{k1j}{}}[1(\sigma _k^{\backslash 1j})^2](J_{k1}\sigma _1+J_{kj}\sigma _j)^2\}.`$ (B4)
After collecting terms and discarding negligible ones,

$$\mathrm{Den}.=\mathrm{Tr}_{1j}\mathrm{exp}\{\beta \sigma _1\underset{k1j}{}J_{1k}\sigma _k^{\backslash 1j}+\beta \sigma _j\underset{k1j}{}J_{jk}\sigma _k^{\backslash 1j}+\beta J_{1j}\sigma _1\sigma _j+\beta ^2(1q)J^2\}.$$
(B5)
Together with a similar manipulation of the numerator, we obtain

$$\sigma _j=\mathrm{tanh}\beta \left(h_j^{\backslash 1}+J_{j1}\mathrm{tanh}\beta h_1^{\backslash j}\right),$$
(B6)
whose Taylor expansion yields

$$\sigma _j=\sigma _j^{\backslash 1}+\left(\beta \mathrm{sech}^2\beta h_j^{\backslash 1}\right)\left(J_{j1}\mathrm{tanh}\beta h_1^{\backslash j}\right),$$
(B7)
which becomes Eq. (32) for the case $`p=2`$.


text:
# The polynomial property (V)

## Abstract.

Given Banach spaces $`E`$ and $`F`$, we denote by $`𝒫(^kE,F)`$ the space of all $`k`$-homogeneous (continuous) polynomials from $`E`$ into $`F`$, and by $`𝒫_{wb}(^kE,F)`$ the subspace of polynomials which are weak-to-norm continuous on bounded sets. It is shown that if $`E`$ has an unconditional finite dimensional expansion of the identity, the following assertions are equivalent: (a) $`𝒫(^kE,F)=𝒫_{wb}(^kE,F)`$; (b) $`𝒫_{wb}(^kE,F)`$ contains no copy of $`c_0`$; (c) $`𝒫(^kE,F)`$ contains no copy of $`\mathrm{}_{\mathrm{}}`$; (d) $`𝒫_{wb}(^kE,F)`$ is complemented in $`𝒫(^kE,F)`$. This result was obtained by Kalton for linear operators. As an application, we show that if $`E`$ has Pełczyński’s property (V) and satisfies $`𝒫(^kE)=𝒫_{wb}(^kE)`$ then, for all $`F`$, every unconditionally converging $`P𝒫(^kE,F)`$ is weakly compact. If $`E`$ has an unconditional finite dimensional expansion of the identity, then the converse is also true.

###### Key words and phrases:

Weakly continuous polynomial, unconditionally converging polynomial, weakly compact polynomial, weakly unconditionally Cauchy series, property (V)  The first named author was supported in part by DGICYT Grant PB 97–0349 (Spain)  The second named author was supported in part by DGICYT Grant PB 96–0607 (Spain)   file ppv.tex

Given two Banach spaces $`E`$ and $`F`$, we denote by $`𝒫(^kE,F)`$ the space of all $`k`$-homogeneous (continuous) polynomials from $`E`$ into $`F`$, and by $`𝒫_{wb}(^kE,F)`$ the subspace of polynomials which are weak-to-norm continuous on bounded sets. This subspace has been studied by many authors: see, for instance, . Clearly, every polynomial in $`𝒫_{wb}(^kE,F)`$ takes bounded sets into relatively compact sets. Observe that $`𝒫(^1E,F)=(E,F)`$, the space of (linear bounded) operators from $`E`$ into $`F`$, and that $`𝒫_{wb}(^1E,F)=𝒦(E,F)`$, the space of compact operators. The spaces $`𝒫(^0E,F)`$ and $`𝒫_{wb}(^0E,F)`$ may be identified with $`F`$.

Kalton studied in  the structure of the space $`𝒦(E,F)`$. In the present paper, we obtain versions of his results for the space $`𝒫_{wb}(^kE,F)`$, showing that $`𝒫_{wb}(^kE,F)`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$ if and only if either $`E`$ contains a complemented copy of $`\mathrm{}_1`$ or $`F`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$. We also prove that, for $`E`$ having an unconditional finite dimensional expansion of the identity, the following assertions are equivalent: (a) $`𝒫(^kE,F)=𝒫_{wb}(^kE,F)`$; (b) $`𝒫_{wb}(^kE,F)`$ contains no copy of $`c_0`$; (c) $`𝒫(^kE,F)`$ contains no copy of $`\mathrm{}_{\mathrm{}}`$; (d) $`𝒫_{wb}(^kE,F)`$ is complemented in $`𝒫(^kE,F)`$.

As an application, we prove that, if $`E`$ has property (V) (the definitions are given below) and $`𝒫(^kE)=𝒫_{wb}(^kE)`$, then every $`k`$-homogeneous unconditionally converging polynomial on $`E`$ is weakly compact. If $`E`$ has an unconditional finite dimensional expansion of the identity, then the converse is also true.

Throughout, $`E`$ and $`F`$ will denote Banach spaces, $`B_E`$ is the closed unit ball of $`E`$ and $`S_E`$ is the unit sphere of $`E`$; $`E^{}`$ will be the dual of $`E`$. The set of natural numbers is denoted by $``$. As usual, $`(e_n)`$ stands for the unit vector basis of $`c_0`$.

A formal series $`x_n`$ in $`E`$ is weakly unconditionally Cauchy (w.u.C., for short) if, for every $`\varphi E^{}`$, we have $`|\varphi (x_n)|<+\mathrm{}`$. Equivalent definitions may be seen in \[7, Theorem V.6\]. The series is unconditionally convergent if every subseries converges. Equivalent definitions may be seen in \[8, Theorem 1.9\].

A polynomial $`P𝒫(^kE,F)`$ is unconditionally converging  if, for each w.u.C. series $`x_n`$ in $`E`$, the sequence $`\left(P\left(_{i=1}^nx_i\right)\right)_n`$ is convergent in $`F`$. The space of all unconditionally converging polynomials is denoted by $`𝒫_{uc}(^kE,F)`$. This class has been very useful for obtaining polynomial characterizations of Banach space properties (see ). We say that $`P𝒫(^kE,F)`$ is (weakly) compact if $`P(B_E)`$ is relatively (weakly) compact in $`F`$. Every weakly compact polynomial is unconditionally converging. For the general theory of polynomials on Banach spaces, we refer to .

To each polynomial $`P𝒫(^kE,F)`$ we can associate a unique symmetric $`k`$-linear mapping $`\widehat{P}:E\times \stackrel{(k)}{\mathrm{}}\times EF`$ so that $`P(x)=\widehat{P}(x,\mathrm{},x)`$ and an operator $`T_P:E𝒫(^{k1}E,F)`$ given by $`T_P(x)(y):=\widehat{P}(x,y,\stackrel{(k1)}{\mathrm{}},y)`$. It is well known that $`P𝒫_{wb}(^kE,F)`$ if and only if $`T_P`$ is compact \[3, Theorem 2.9\].

Denote by $`_s(E,𝒫(^{k1}E,F))`$ the space of all operators $`C:E𝒫(^{k1}E,F)`$ such that

$$(C(x))^{}(y_1,\mathrm{},y_{k1})=(C(y_1))^{}(x,y_2,\mathrm{},y_{k1})(x,y_1,\mathrm{},y_{k1}E),$$

where $`(C(x))^{}`$ stands for the symmetric $`(k1)`$-linear mapping associated to $`C(x)`$.

###### Proposition 1.

The mapping $`T:𝒫(^kE,F)_s(E,𝒫(^{k1}E,F))`$ given by $`T(P)=T_P`$ is a surjective linear isomorphism.

Proof. Clearly, $`T`$ is well defined, linear and injective. Since

$$T_P=\underset{xB_E}{sup}T_P(x)=\underset{x,yB_E}{sup}\widehat{P}(x,y,\stackrel{(k1)}{\mathrm{}},y)\widehat{P}\frac{k^k}{k!}P$$

\[26, Theorem 2.2\], we have that $`T`$ is continuous. To see that it is surjective, take $`C_s(E,𝒫(^{k1}E,F))`$, and define $`A:E\times \stackrel{(k)}{\mathrm{}}\times EF`$ by $`A(y_1,\mathrm{},y_k):=(C(y_1))^{}(y_2,\mathrm{},y_k)`$, and $`P(x)=A(x,\stackrel{(k)}{\mathrm{}},x)`$. Then

$$T_P(x)(y)=A(x,y,\stackrel{(k1)}{\mathrm{}},y)=C(x)(y)(x,yE).$$

Hence, $`T_P=C`$. $`\mathrm{}`$

The subspace of all operators in $`_s(E,𝒫(^{k1}E,F))`$ which are compact (resp. weakly compact) will be denoted by $`𝒦_s(E,𝒫(^{k1}E,F))`$ (resp. $`𝒲_s(E,𝒫(^{k1}E,F))`$). Given $`C𝒦_s(E,𝒫(^{k1}E,F))`$, the symmetry of $`C`$ easily implies that $`C(E)𝒫_{wb}(^{k1}E,F)`$.

The proof of \[5, Proposition 5.3\] gives:

###### Proposition 2.

For $`nm`$, the space $`𝒫(^mE,F)`$ (resp. $`𝒫_{wb}(^mE,F)`$) is isomorphic to a complemented subspace of $`𝒫(^nE,F)`$ (resp. $`𝒫_{wb}(^nE,F)`$).

###### Theorem 3.

The space $`𝒫_{wb}(^kE,F)`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$ if and only if either $`F`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$ or $`E`$ contains a complemented copy of $`\mathrm{}_1`$.

Proof. If $`𝒫_{wb}(^kE,F)`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$, a fortiori the space $`𝒦(E,𝒫_{wb}(^{k1}E,F))`$ contains it. Therefore \[23, Theorem 4\], either $`E`$ contains a complemented copy of $`\mathrm{}_1`$ or $`𝒫_{wb}(^{k1}E,F)`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$. Repeating the process, we conclude that either $`E`$ contains a complemented copy of $`\mathrm{}_1`$ or $`𝒫_{wb}(^0E,F)F`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$.

Conversely, if $`F`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$, since $`F𝒫_{wb}(^0E,F)`$ is isomorphic to a subspace of $`𝒫_{wb}(^kE,F)`$ (Proposition 2), we obtain that $`𝒫_{wb}(^kE,F)`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$. If $`E`$ contains a complemented copy of $`\mathrm{}_1`$, then $`E^{}`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$; since $`E^{}=𝒫_{wb}(^1E)`$ is isomorphic to a subspace of $`𝒫_{wb}(^1E,F)`$, which is in turn isomorphic to a subspace of $`𝒫_{wb}(^kE,F)`$, we obtain that the latter contains a copy of $`\mathrm{}_{\mathrm{}}`$. $`\mathrm{}`$

The proof of \[23, Lemma 2\] yields:

###### Lemma 4.

Assume $`E`$ is separable, $`𝒫_{wb}(^kE,F)`$ is complemented in $`𝒫(^kE,F)`$, and an operator $`\mathrm{\Phi }:\mathrm{}_{\mathrm{}}𝒫(^kE,F)`$ is given with the following properties:

(a) $`\mathrm{\Phi }(e_n)𝒫_{wb}(^kE,F)`$ for all $`n`$;

(b) the subset $`\{\mathrm{\Phi }(\xi )(x):\xi \mathrm{}_{\mathrm{}},xE\}F`$ is separable.

Then, for every infinite subset $`M`$, there exists an infinite subset $`M_0M`$ with $`\mathrm{\Phi }(\xi )𝒫_{wb}(^kE,F)`$ for all $`\xi \mathrm{}_{\mathrm{}}(M_0)`$.

###### Lemma 5.

Suppose $`E`$ contains a complemented copy of $`\mathrm{}_1`$. Then $`𝒫_{wb}(^kE,F)`$ is uncomplemented in $`𝒫(^kE,F)`$ for every $`F`$ and $`k`$ $`(k>1)`$.

Proof. As in \[23, Lemma 3\], we can reduce the problem to the case $`E=\mathrm{}_1`$.

Fix $`vS_F`$ and define the operator

$$\mathrm{\Phi }:\mathrm{}_{\mathrm{}}𝒫(^k\mathrm{}_1,F)$$

by

$$\mathrm{\Phi }(\xi )(x)=\underset{i=1}{\overset{\mathrm{}}{}}\xi _ix_i^kv\text{for }\xi =(\xi _i)_{i=1}^{\mathrm{}}\mathrm{}_{\mathrm{}}\text{ and }x=(x_i)_{i=1}^{\mathrm{}}\mathrm{}_1.$$

Since

$$\mathrm{\Phi }(\xi )=\underset{xB_\mathrm{}_1}{sup}\underset{i=1}{\overset{\mathrm{}}{}}\xi _ix_i^kv\xi \underset{xB_\mathrm{}_1}{sup}\underset{i=1}{\overset{\mathrm{}}{}}\left|x_i^k\right|=\xi ,$$

$`\mathrm{\Phi }`$ is continuous (easily, it is even an isometric embedding).

We claim that $`\mathrm{\Phi }(\xi )𝒫_{wb}(^k\mathrm{}_1,F)`$ if and only if $`\xi c_0`$. Indeed, let

$$T_{\mathrm{\Phi }(\xi )}:\mathrm{}_1𝒫(^{k1}\mathrm{}_1,F)$$

be the associated operator given by

$$T_{\mathrm{\Phi }(\xi )}(x)(y)=\underset{i=1}{\overset{\mathrm{}}{}}\xi _ix_iy_i^{k1}v\text{for }x=(x_i),y=(y_i)\mathrm{}_1.$$

Since, for $`nm`$,

$`\left(T_{\mathrm{\Phi }(\xi )}(e_n)T_{\mathrm{\Phi }(\xi )}(e_m)\right)(y)=`$
$`T_{\mathrm{\Phi }(\xi )}(e_ne_m)(y)={\displaystyle \underset{i=1}{\overset{\mathrm{}}{}}}\xi _i(\delta _{in}\delta _{im})y_i^{k1}v=\left|\xi _ny_n^{k1}\xi _my_m^{k1}\right|,`$
we have

$$\mathrm{max}\{|\xi _n|,|\xi _m|\}T_{\mathrm{\Phi }(\xi )}(e_n)T_{\mathrm{\Phi }(\xi )}(e_m)|\xi _n|+|\xi _m|,$$

it follows that $`T_{\mathrm{\Phi }(\xi )}`$ is compact if and only if $`\xi c_0`$ (see \[7, Exercise VII.5\]).

By Lemma 4, there is an infinite $`M`$ such that $`\mathrm{\Phi }(\xi )𝒫_{wb}(^k\mathrm{}_1,F)`$ for all $`\xi \mathrm{}_{\mathrm{}}(M)`$, which contradicts the above claim. $`\mathrm{}`$

In the linear case $`(k=1)`$, Kalton needs to assume that $`F`$ is infinite dimensional for the validity of Lemma 5 \[23, Lemma 3\]. Taking $`k>1`$, we can drop this condition. Observe that, if $`k=1`$ and $`dim(F)<\mathrm{}`$, we have

$$𝒫_{wb}(^1E,F)=𝒦(E,F)=(E,F)=𝒫(^1E,F),$$

so the conclusion of the Lemma is not true.

An unconditional finite dimensional expansion of the identity for a Banach space $`E`$ is a sequence of finite dimensional operators $`A_n:EE`$ such that for each $`xE`$,

$$x=\underset{n=1}{\overset{\mathrm{}}{}}A_n(x)$$

unconditionally. This condition is a bit more general than having an unconditional basis .

###### Lemma 6.

Suppose $`E`$ has an unconditional finite dimensional expansion of the identity and let $`P𝒫(^kE,F)`$. Then there is a w.u.C. series $`P_i`$ in $`𝒫_{wb}(^kE,F)`$ such that, for all $`xE`$, $`P(x)=_{m=1}^{\mathrm{}}P_m(x)`$ unconditionally.

Proof. There is a sequence $`(A_i)𝒦(E,E)`$ such that, for every $`xE`$, we have $`x=_{i=1}^{\mathrm{}}A_i(x)`$ unconditionally. Then,

$`P\left({\displaystyle \underset{i=1}{\overset{n}{}}}A_i(x)\right)`$ $`=`$ $`{\displaystyle \underset{i_1,\mathrm{},i_k=1}{\overset{n}{}}}\widehat{P}(A_{i_1}(x),\mathrm{},A_{i_k}(x))`$
$`=`$ $`{\displaystyle \underset{m=1}{\overset{n}{}}}\left({\displaystyle \underset{\mathrm{max}\{i_1,\mathrm{},i_k\}=m}{}}\widehat{P}(A_{i_1}(x),\mathrm{},A_{i_k}(x))\right)`$
$`=`$ $`{\displaystyle \underset{m=1}{\overset{n}{}}}P_m(x)`$
where $`P_m𝒫_{wb}(^kE,F)`$.

Choosing finite subsets $`I_1,\mathrm{},I_k`$ of integers, we have

$`{\displaystyle \underset{i_1I_1,\mathrm{},i_kI_k}{}}\widehat{P}(A_{i_1}(x),\mathrm{},A_{i_k}(x))`$ $`=`$ $`\widehat{P}({\displaystyle \underset{i_1I_1}{}}A_{i_1}(x),\mathrm{},{\displaystyle \underset{i_kI_k}{}}A_{i_k}(x))`$
$``$ $`\widehat{P}{\displaystyle \underset{i_1I_1}{}}A_{i_1}(x)\mathrm{}{\displaystyle \underset{i_kI_k}{}}A_{i_k}(x).`$
Hence, the series

$$\underset{i_1,\mathrm{},i_k=1}{\overset{\mathrm{}}{}}\widehat{P}(A_{i_1}(x),\mathrm{},A_{i_k}(x))$$

is unconditionally convergent for all $`xE`$ \[8, Theorem 1.9\]. Therefore, $`P(x)=_{m=1}^{\mathrm{}}P_m(x)`$ unconditionally.

Moreover, by the uniform boundedness principle \[26, Theorem 2.6\], we have

$$\underset{Ifinite}{sup}\underset{iI}{}P_i<\mathrm{}.$$

So $`P_i`$ is w.u.C. in $`𝒫_{wb}(^kE,F)`$ \[7, Theorem V.6\]. $`\mathrm{}`$

###### Theorem 7.

Suppose $`E`$ has an unconditional finite dimensional expansion of the identity and let $`k`$ $`(k>1)`$. Then the following assertions are equivalent:

(a) $`𝒫(^kE,F)=𝒫_{wb}(^kE,F)`$;

(b) $`𝒫_{wb}(^kE,F)`$ contains no copy of $`c_0`$;

(c) $`𝒫(^kE,F)`$ contains no copy of $`\mathrm{}_{\mathrm{}}`$;

(d) $`𝒫_{wb}(^kE,F)`$ is complemented in $`𝒫(^kE,F)`$.

Proof. (a) $``$ (c): Assume $`𝒫(^kE,F)=𝒫_{wb}(^kE,F)`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$. By Theorem 3, either $`E`$ contains a complemented copy of $`\mathrm{}_1`$ or $`F`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$. Lemma 5 implies that $`E`$ contains no complemented copy of $`\mathrm{}_1`$, so $`F`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$. Take a normalized basic sequence $`(x_i)E`$ and a bounded sequence of coefficient functionals $`(\varphi _n)E^{}`$ ($`\varphi _i(x_j)=\delta _{ij}`$). Define $`P𝒫(^kE,\mathrm{}_{\mathrm{}})`$ by $`P(x):=\left(\varphi _n(x)^k\right)_n`$. Then, for $`ij`$, we have $`P(x_i)P(x_j)|\varphi _i(x_i)^k\varphi _i(x_j)^k|=1`$. Hence, denoting by $`J:\mathrm{}_{\mathrm{}}F`$ an isomorphism, we get $`JP𝒫(^kE,F)\backslash 𝒫_{wb}(^kE,F)`$, a contradiction.

(c) $``$ (b): The proof is the same of the linear case \[23, Theorem 6\] with slight modifications.

(b) $``$ (a): Take $`P𝒫(^kE,F)`$. Consider the w.u.C. series $`P_i`$ given by Lemma 6 with $`P_i𝒫_{wb}(^kE,F)`$. Since this space contains no copy of $`c_0`$, the series is unconditionally convergent \[7, Theorem V.8\]. Clearly, its sum must be $`P`$. Since the space $`𝒫_{wb}(^kE,F)`$ is closed, we conclude that $`P𝒫_{wb}(^kE,F)`$.

(a) $``$ (d) is trivial.

(d) $``$ (a): If $`𝒫_{wb}(^kE,F)`$ is complemented in $`𝒫(^kE,F)`$, Lemma 5 implies that $`E`$ contains no complemented copy of $`\mathrm{}_1`$. Suppose there is $`P𝒫(^kE,F)\backslash 𝒫_{wb}(^kE,F)`$. By Lemma 6, we can find a sequence $`(P_i)𝒫_{wb}(^kE,F)`$ so that $`P(x)=_{i=1}^{\mathrm{}}P_i(x)`$ unconditionally for each $`xE`$, and $`P_i`$ is w.u.C. but is not unconditionally convergent since $`P𝒫_{wb}(^kE,F)`$. Hence, we can find $`ϵ>0`$, and an increasing sequence $`(m_j)`$ of integers such that, for each $`j`$, the polynomial

$$C_j:=\underset{i=m_j+1}{\overset{m_{j+1}}{}}P_i$$

satisfies $`C_j>ϵ`$.

Define now $`\mathrm{\Phi }:\mathrm{}_{\mathrm{}}𝒫(^kE,F)`$ by $`\mathrm{\Phi }(\xi )(x)=\xi _jC_j(x)`$ for $`\xi =(\xi _j)_{j=1}^{\mathrm{}}\mathrm{}_{\mathrm{}}`$ and $`xE`$. Since the series $`P_i(x)`$ is unconditionally convergent, so is the series $`\xi _jC_j(x)`$. The set $`\{\mathrm{\Phi }(\xi )(x):\xi \mathrm{}_{\mathrm{}},xE\}`$ is contained in the closed linear span of $`\{C_j(E):j\}`$ which is separable by the compactness of $`C_j`$. On the other hand, $`\mathrm{\Phi }(e_j)=C_j𝒫_{wb}(^kE,F)`$ for all $`j`$. By Lemma 4, there is an infinite subset $`M`$ such that $`\mathrm{\Phi }(\xi )𝒫_{wb}(^kE,F)`$ for each $`\xi \mathrm{}_{\mathrm{}}(M)`$.

Therefore, for each $`\xi \mathrm{}_{\mathrm{}}(M)`$, the series $`\xi _jC_j`$ is weak subseries convergent. The Orlicz-Pettis theorem then implies that it is unconditionally convergent. In particular,

$$\underset{\begin{array}{c}jM\\ j\mathrm{}\end{array}}{lim}C_j=0,$$

a contradiction. $`\mathrm{}`$

Observe that the unconditional finite dimensional expansion of the identity is used only in (b) $``$ (a) and (d) $``$ (a).

In the linear case $`(k=1)`$, the restriction $`dim(F)=\mathrm{}`$ is required for the validity of Theorem 7 \[23, Theorem 6\].

###### Remark 8.

In order to highlight the difference between Theorem 3 and Theorem 7, let us consider the spaces $`𝒫(^k\mathrm{}_p,\mathrm{}_q)`$ and $`𝒫_{wb}(^k\mathrm{}_p,\mathrm{}_q)`$ for $`1<p,q<\mathrm{}`$ and $`k>1`$.

If $`kq>p`$, then the space $`𝒫(^k\mathrm{}_p,\mathrm{}_q)`$ contains a copy of $`\mathrm{}_{\mathrm{}}`$. Indeed, define $`J:\mathrm{}_{\mathrm{}}𝒫(^k\mathrm{}_p,\mathrm{}_q)`$ by

$$J(\xi )(x)=\left(\xi _ix_i^k\right)_{i=1}^{\mathrm{}}.$$

Since, for all $`n`$,

$$|\xi _n|=J(\xi )(e_n)J(\xi )=\underset{xB_\mathrm{}_p}{sup}J(\xi )(x)\xi ,$$

we get that $`J`$ is an isometric embedding.

However, the space $`𝒫_{wb}(^k\mathrm{}_p,\mathrm{}_q)`$ contains no copy of $`\mathrm{}_{\mathrm{}}`$ since it is separable. In fact, it is the norm closure of the space of all finite type polynomials generated by the mappings of the form $`\varphi ^ny`$, for $`\varphi (\mathrm{}_p)^{}`$, $`y\mathrm{}_q`$ \[4, Proposition 2.7\].

On the other hand, if $`\xi c_0`$, then $`J(\xi )`$ is in the closure of the finite type polynomials, so $`J(\xi )𝒫_{wb}(^k\mathrm{}_p,\mathrm{}_q)`$. Hence, this space contains $`J(c_0)`$, an isometric copy of $`c_0`$.

Recall moreover that, if $`kq<p`$, then the space $`𝒫(^k\mathrm{}_p,\mathrm{}_q)`$ is reflexive \[1, 4.3\].

Every polynomial $`P𝒫(^kE,F)`$ has an extension, called the Aron-Berner extension, to a polynomial $`\overline{P}𝒫(^kE^{},F^{})`$ (see ). If $`P𝒫_{wb}(^kE)`$, then $`\overline{P}𝒫(^kE^{})`$ is weak-star continuous on bounded sets .

Recall that a Banach space has property (V), introduced in , if every unconditionally converging operator on $`E`$ is weakly compact. Every $`C(K)`$ space has property (V) .

We shall need the following result:

###### Theorem 9.

 The following assertions are equivalent:

(a) The space $`E`$ has property (V);

(b) for all $`F`$ and $`k`$, the Aron-Berner extension of every $`P𝒫_{uc}(^kE,F)`$ is $`F`$-valued;

(c) There is $`k`$ such that, for all $`F`$, the Aron-Berner extension of every $`P𝒫_{uc}(^kE,F)`$ is $`F`$-valued.

Easily, if a polynomial is weakly compact, then its Aron-Berner extension is $`F`$-valued .

###### Theorem 10.

For $`k`$, consider the assertions:

(a) $`E`$ has property (V) and $`𝒫(^kE)=𝒫_{wb}(^kE)`$;

(b) for each $`F`$, every $`P𝒫_{uc}(^kE,F)`$ is weakly compact.

Then (a) $``$ (b). If, moreover, $`E`$ has an unconditional finite dimensional expansion of the identity, then (b) $``$ (a).

Proof. (a) $``$ (b): Given $`P𝒫_{uc}(^kE,F)`$, by Theorem 9, the range of its Aron-Berner extension $`\overline{P}`$ is contained in $`F`$. Take a net $`(x_\alpha )B_E`$. We can assume that $`(x_\alpha )`$ is weak Cauchy and so it converges in the weak-star topology to some $`zE^{}`$.

Let $`\psi F^{}`$. Then $`\psi P𝒫_{wb}(^kE)`$ and so,

$$\psi P(x_\alpha )=\psi \overline{P}(x_\alpha )=\overline{\psi P}(x_\alpha )\overline{\psi P}(z)=\psi \overline{P}(z).$$

Therefore, the net $`(P(x_\alpha ))`$ is weakly convergent to $`\overline{P}(z)F`$. So, $`P(B_E)`$ is relatively weakly compact.

(b) $``$ (a): By the comment preceding this Theorem, and by Theorem 9, (b) implies that $`E`$ has property (V). Also, every polynomial in $`𝒫(^kE,\mathrm{}_1)`$ is compact. From this, we obtain that $`𝒫(^kE)`$ contains no copy of $`c_0`$ \[10, Corollary 8\]. A fortiori, $`𝒫_{wb}(^kE)`$ contains no copy of $`c_0`$. Since $`E`$ has an unconditional finite dimensional expansion of the identity, we conclude from Theorem 7 that $`𝒫_{wb}(^kE)=𝒫(^kE)`$. $`\mathrm{}`$

We do not know if the condition on the existence of an unconditional finite dimensional expansion of the identity may be removed from Theorem 10. In fact, if (b) is satisfied, since $`𝒫(^{k1}E)`$ contains no copy of $`c_0`$ and $`E`$ has property (V), we have

$`𝒫(^kE)=_s(E,𝒫(^{k1}E))=𝒲_s(E,𝒫(^{k1}E)),`$
$`𝒫_{wb}(^kE)=𝒦_s(E,𝒫(^{k1}E)).`$
So, we only have to show that $`𝒦_s(E,𝒫(^{k1}E))=𝒲_s(E,𝒫(^{k1}E))`$. There are many conditions on $`E`$ (apart from the existence of an unconditional finite dimensional expansion of the identity) that imply this equality , and it is not known if there are Banach spaces $`X`$, $`Y`$ such that $`𝒦(X,Y)𝒲(X,Y)`$ while $`𝒦(X,Y)`$ contains no copy of $`c_0`$ .

Recall that the condition $`𝒫(^kE)=𝒫_{wb}(^kE)`$ implies that $`E`$ contains no copy of $`\mathrm{}_1`$ .

It is proved in  that if $`𝒫(^kE)=𝒫_{wb}(^kE)`$ and $`E`$ has property (u) then, for all $`F`$, every $`P𝒫_{uc}(^kE,F)`$ is weakly compact. Part (a) $``$ (b) of Theorem 10 is stronger than the result of . The latter cannot be applied, for instance, to the space $`c_0\widehat{}_\pi c_0`$ which fails property (u)  while it does have property (V) . For the definition of property (u) and its relationship to property (V), see . Recall in particular that, if $`E`$ has property (u) and contains no copy of $`\mathrm{}_1`$, then $`E`$ has property (V) \[27, Proposition 2\].