\section{Introduction}\label{sec:Intro}
Iterative methods based on projections of the solution onto a Krylov subspace are often used to solve large-scale linear ill-posed problems \cite{RegParamItr,Chung2008,Novati2013,Bazan2010,Hochstenbach2010,projRegGenForm,Gazzola2015}, \cite[Chap. 6]{RankDeff}, \cite[Chap. 6]{HansenInsights}. Such large problems arise in a variety of applications including image deblurring \cite{Nagy2004,Yuan2007,SpecFiltBook,projRegGenForm,Chung2008} and machine learning \cite{Ong2004,Martens2010,Freitas2005,Ide2007}. Krylov methods iteratively project the solution onto a low-dimensional subspace and solve the resulting small-scale problem using standard procedures such as the QR decomposition \cite{LSQR}. These methods are therefore attractive for the solution of large-scale problems that cannot be solved directly as well as for problems perturbed by noise, since the projection onto a Krylov subspace possesses a regularizing effect \cite{Hnetynkova2009}. Accurate solution using iterative procedures requires  stopping the process close to the optimal stopping iteration. In this paper we develop a general stopping criterion for Krylov subspace regularization, which we particularly apply to the Golub-Kahan bidiagonalization (GKB), also frequently referred to as Lanczos bidiagonalization \cite{Golub1965}.

Consider the problem
\begin{equation}\label{eq:Ax=b} Ax=b,\end{equation}
where the matrix \(A\in\mathbb{R}^{m\times n}\) is large and ill-conditioned and the data vector \(b=b_{true} + n\) constitutes the true data \(b_{true}\) perturbed by an additive white noise vector \(n\). We are interested in approximating the least-squares solution of the problem
\begin{equation}\label{eq:LSsolution}
\min_x ||b_{true}- Ax||^2
\end{equation}
without knowledge of \(b_{true}\). This can be done by minimizing the projected least-squares (PLS) problem
\begin{equation}\label{eq:GKBprob}
\min_x ||b-Ax||^2,\quad \text{such that}\quad x\in\mathcal{K}_k(A^TA,A^Tb),
\end{equation}
where
\begin{equation}\label{eq:KrylovSubspace}
\mathcal{K}_k(A^TA,A^Tb)=\text{span}\{A^Tb,\ldots,\left(A^TA\right)^{k-1}A^Tb\},
\end{equation}
is the Krylov subspace generated using the GKB process for each iteration \(k\).
The regularizing effect of the projection onto \(\mathcal{K}_k(A^TA,A^Tb)\) then dampens the noise in \(b\), allowing us to approximate \(x_{true}\) \cite{RegParamItr,Hnetynkova2009}. The reason for this regularization effect is that in the initial iterations the basis vectors spanning the Krylov subspace are smooth and so is the projected solution. However, for large iteration numbers \(k\) the accuracy of the projected solution decreases as the basis vectors become corrupted by noise. By applying GKB to \ref{eq:Ax=b}, we thus obtain a sequence of iterates \(\{x^{(k)}\}\) whose error \(||x^{(k)}-x_{true}||\) initially decreases with increasing iterations and then goes up sharply. This behavior of GKB is termed semi-convergence \cite[Sect. 6.3]{RankDeff}. It is therefore crucial to develop a reliable stopping rule by which to terminate the iterative solution of \ref{eq:GKBprob} before noise contaminates the solution. For this purpose, a number of stopping criteria have been proposed including the L-curve \cite{RegParamItr,LCurve}, the generalized cross validation (GCV) \cite{RegParamItr}, \cite[Sect. 7.4]{RankDeff} and the discrepancy principle \cite{RegParamItr}. However, both the GCV and the L-curve methods are inaccurate for determination of the stopping iteration in a significant percentage of cases, as has been demonstrated in \cite[Sect. 7.2]{Hansen2006} for the former and below in \ref{sec:NumEx} for the latter. The discrepancy principle, on the other hand, requires \emph{a priori} knowledge of the noise level and is highly sensitive to it \cite[Sect. 4.1.2]{Bauer2011}. Recently, a new stopping criterion called the normalized cumulative periodogram (NCP), which is based on a whiteness measure  of the residual  vector \(r^{(k)}=b-Ax^{(k)}\), was proposed in \cite{Hansen2006}. While this method outperforms the above-mentioned alternatives, we nevertheless show that its results are inconsistent for some of our numerical problems.

Estimation of the stopping iteration using the GCV or the L-curve method requires projection of the solution onto the subspace \(\mathcal{K}_k(A^TA,A^Tb)\), which depends on the noisy data vector \(b\). In contrast to the original problem \ref{eq:Ax=b} where the noise is entirely contained within data vector \(b\), while coefficient matrix \(A\) is noiseless, the coefficient matrix in the projected problem is contaminated by noise. It is thus nontrivial to generalize standard parameter-choice methods to the problem \ref{eq:GKBprob}, as their usage may result in suboptimal solutions due to the fact that they do not account for noise in the coefficient matrix. To overcome this problem, we estimate the optimal stopping iteration for GKB using the Data Filtering (DF) method originally proposed and briefly discussed in \cite{Levin2016}. Using the DF method the stopping iteration is selected as the one for which the distance \(||\widehat b-Ax^{(k)}||\) between the filtered data \(\widehat b\) and the data reconstructed from the iterated solution \(Ax^{(k)}\) is either minimal or levels-off. In \cite{Levin2016} the filtered data \(\widehat b\approx b-n\) is obtained by separating the noise from the data using the so-called Picard parameter \(k_0\), above which the coordinates of the data in the basis of the left singular vectors of \(A\) are dominated by noise. The approximation of the unperturbed data \(\widehat b\) is then obtained by setting the coordinates of \(b\) in that basis to zero for \(k>k_0\).

Computing the SVD of \(A\) for large-scale problems is not feasible and the specific basis in which the authors of \cite{Levin2016} achieve a separation between signal and noise is therefore unavailable. To overcome this problem we propose to replace the SVD basis used in \cite{Levin2016} by the basis of the discrete Fourier transform (DFT) and show that we can achieve a similar separation of signal from noise in one and in two dimensions. It is well known however, that the DFT assumes the signal to be periodic, and applying it to a non-periodic signal results in artifacts in the form of fictitious high-frequency components that cannot be distinguished from the noise in the DFT basis. We prevent these high-frequency components from appearing by using the Periodic Plus Smooth (PPS) decomposition \cite{Moisan2011}, which allows us to write the signal as a sum, \(b=p+s\), of a periodic component \(p\) and a smooth component \(s\). The periodic component is compatible with the periodicity assumption of the DFT and does not produce high-frequency artifacts, while the smooth component does not need to be filtered. We then have to filter only the Fourier coefficients of the periodic component.

For two-dimensional problems, the Fourier coefficients of the data require a vectorization prior to estimation of the Picard parameter. The coefficients are usually vectorized by order of increasing spatial frequency \cite{Hansen2006}. Here we propose an alternative vectorization, ordered by increasing value of the product of the spatial frequencies in each dimension, which enables a more accurate and consistent estimation of the Picard parameter. We demonstrate that such ordering is equivalent to the sorting of a Kronecker product of two vectors of spatial frequencies and stems from a corresponding Kronecker product structure of the two-dimensional DFT matrix. This approach is also analogous to reordering the SVD of a separable blur as discussed in e.g. \cite[Sect. 4.4.2]{SpecFiltBook}. We demonstrate that a filter based on the proposed ordering performs similarly to or outperforms its spatial frequencies-based counterpart in all our numerical examples, allowing termination of the iterative process closer to the optimal iteration. The new filtering procedure is simple and effective, and can be used independently of the iterative inversion algorithm.

Hybrid methods, which replace the semi-convergent PLS problem \ref{eq:GKBprob} with a convergent alternative, have received significant attention in recent years  \cite{RegParamItr,Chung2008,Novati2013,Hochstenbach2010,Bazan2010,Gazzola2015,Chung2015}. These methods combine Tikhonov regularization with a projection onto \(\mathcal{K}_k(A^TA,A^Tb)\), replacing problem \ref{eq:GKBprob} with the projected Tikhonov problem
\begin{equation}\label{eq:TikhMinProb} \min_x ||b-Ax||^2 + \lambda^2||Lx||^2\quad \text{such that}\quad x\in\mathcal{K}_k(A^TA,A^Tb),\end{equation}
where \(L\) is a regularization matrix and \(\lambda\) is a regularization parameter that controls the smoothness of the solution. In this paper, we follow the authors of \cite{RegParamItr,Chung2008,Novati2013,Hochstenbach2010,Bazan2010,Gazzola2015} by considering only the \(L=I\) case. We would like to note, however, that a generalization to case \(L\neq I\) was developed and discussed in \cite{Hochstenbach2010}. The advantage of hybrid methods is that given an accurate choice of the value \(\lambda\) at each iteration, the error in the solution of \ref{eq:TikhMinProb} stabilizes for large iterations, contrary to the least-squares problem \ref{eq:GKBprob} for which the solution error has a minimum. However, appropriately choosing the regularization parameter \(\lambda\) at each iteration is a difficult task, since the coefficient matrix in the projected problem becomes contaminated by noise, as in the PLS problem. Hence, standard parameter-choice methods such as the GCV cannot be na\"{\i}vely applied to problem \ref{eq:TikhMinProb}. In practice, the GCV indeed fails to stabilize the iterations in a large number of cases as reported in \cite{Chung2008} and \cite[Sect. 5.1.1]{Bazan2010}. To overcome this problem, the authors of \cite{Chung2008} proposed the weighted GCV (W-GCV) method which incorporates an additional free weight parameter, chosen adaptively at each iteration and shown to significantly improve the performance of the method. We demonstrate however, using several numerical examples, that the results of the W-GCV method are still suboptimal.

This paper is organized as follows. In Sect. \ref{sec:DirectReg} we summarize results from the Tikhonov regularization of \ref{eq:Ax=b} that we extend to the PLS problem \ref{eq:GKBprob}. In Sect. \ref{sec:DFTfilter}, we present our filtering technique, based on the Picard parameter in the DFT basis for one- and two-dimensional problems. In Sect. \ref{sec:GKBinvert}, we present the GKB algorithm and formulate our stopping criterion. In this section we also discuss hybrid methods that combine projection with Tikhonov regularization. Finally, in Sect. \ref{sec:NumEx} we give numerical examples which demonstrate the performance of the proposed stopping criterion, and compare it to the L-curve, NCP and W-GCV methods.

\section{Tikhonov regularization}\label{sec:DirectReg}
We begin with a description of our parameter-choice method, detailed in \cite{Levin2016}, for standard direct Tikhonov regularization of \ref{eq:Ax=b} using the Picard parameter, which represent the starting point of our derivation. The Tikhonov regularization method solves the ill-posed problem \ref{eq:Ax=b} by replacing it with the related, well-posed counterpart
\begin{equation}\label{eq:TikhProb} \min_x ||b-Ax||^2 + \lambda^2||x||^2 \implies \left(A^TA+\lambda^2I\right)x=b.\end{equation}
The solution of \ref{eq:TikhProb} can be written in a convenient form, using the singular value decomposition (SVD) of \(A\), given by
\begin{equation}\label{eq:SVDofA} A = U\Sigma V^T,\end{equation}
where \(U\in\mathbb{R}^{m\times m}\) and \(V\in\mathbb{R}^{n\times n}\) are orthogonal matrices. For simplicity, let the \(j\)th columns of \(U\) and \(V\) be denoted by \(u_j\) and \(v_j\), respectively, and the \(j\)th singular value of \(A\) by \(\sigma_j\). Furthermore, let \(\beta_j= u_j^Tb\) be the \(j\)th Fourier coefficient of \(b\) with respect to \(\{u_j\}_{j=1}^m\) and let \(\nu_j = u_j^Tn\) be the Fourier coefficients of the noise. Then, the solution of \ref{eq:TikhProb} can be written as
\begin{equation}\label{eq:TikhSoln}
x(\lambda) = \sum_{j=1}^{m}\frac{\sigma_j}{\sigma_j^2+\lambda^2}\beta_j v_j.
\end{equation}
It can be shown that in order for solution \ref{eq:TikhSoln} to represent a good approximation to the true solution \(x_{true}\) for some \(\lambda\), the problem must satisfy the discrete Picard condition (DPC) \cite{DPC}. The DPC requires that the sequence of Fourier coefficients of the true data \(\{u_j^Tb_{true}\}=\{\beta_j-\nu_j\}\) decays faster than the singular values \(\{\sigma_j\}\) which, by the ill-conditioning of \(A\), decay relatively quickly. Therefore, the DPC implies that \(\beta_j-\nu_j \approx 0\), or equivalently, \(\beta_j\approx \nu_j\), for \(j\geq k_0\) from some index \(k_0\), termed the Picard parameter, on. In other words, the coefficients of \(b\) with indices larger than \(k_0\) are dominated by noise, while coefficients with smaller indices are dominated by the true data.

To estimate \(\lambda\) we can rewrite \ref{eq:LSsolution} as
\begin{equation}\label{eq:LSsolution2}
\min_\lambda ||b_{true}- Ax(\lambda)||^2,
\end{equation}
but since \(b_{true}\) is not known we suggest to replace it with the filtered field \(b_{true}\approx \hat b\), as in the DF method \cite{Levin2016}. The DF method sets the regularization parameter \(\lambda\) for \ref{eq:TikhProb} to be the minimizer of the distance function
\begin{equation}
\label{eq:DistNorm}
\min_\lambda ||\widehat{b}-Ax(\lambda)||^2,
\end{equation}
between the data \(Ax(\lambda)\) reconstructed from the solution \ref{eq:TikhProb}, and the filtered data \(\widehat b\). To obtain the filtered data \(\widehat b\) we remove the noise-dominated coefficients from the expansion of \(b\) in basis \(\{u_j\}\) so that
\begin{equation}\label{eq:PicFiltSVD}
\widehat b = \sum_{j=1}^{k_0-1}\beta_ju_j.
\end{equation}
The Picard parameter \(k_0\), can be found by detection of the levelling-off of the sequence
\begin{equation}\label{eq:VxDefn}
V(k) = \frac{1}{m-k+1}\sum_{j=k}^m \beta_j^2,
\end{equation}
which is shown to decrease on average until it levels-off at \(V(k_0)\simeq s^2\), where \(s^2\) is the variance of the white noise. The detection is done by setting \(k_0\) to be the smallest index satisfying
\begin{equation}\label{eq:PicIndCondSVD}
\frac{|V(k+h)-V(k)|}{V(k)} \leq \varepsilon,
\end{equation}
for some step size \(h\) and a bound on the relative change \(\varepsilon\). The above estimation of \(k_0\) is stated to be robust to changes in \(h\) and \(\varepsilon\), with the values \(\varepsilon\in[10^{-3},10^{-1}]\) and \(h\in[\lfloor\frac{m}{100}\rfloor,\lceil\frac{m}{10}\rceil]\) working consistently well \cite{Levin2016}.

Unfortunately, for large-scale problems, computing the SVD of \(A\) given by \ref{eq:SVDofA} is unfeasible in general and therefore the above separation of noise from signal in the SVD basis is unobtainable. In the next section, we propose a similar filtering procedure for \(b\), which utilizes the basis of the DFT instead of SVD. This basis satisfies an analog of the DPC and is effective for large-scale applications.

\section{The DFT filter}\label{sec:DFTfilter}
In this section we replace the SVD basis discussed in Sect. \ref{sec:DirectReg} with the DFT basis. Since computing the Fourier coefficients with respect to the DFT basis can be done efficiently, the proposed procedure remains computationally cheap even for large-scale problems.
We begin by noting that the true data \(b_{true}\) is generally smooth and therefore is dominated by low-frequencies. This is true for cases such as image deblurring problems and problems arising from discretization of integral equations, where the coefficient matrix \(A\) has a smoothing effect and hence, \(b_{true} = Ax_{true}\) is smooth even if \(x_{true}\) is not, see \cite{Hansen2006,Hansen2008} \cite[Sect. 5.6]{SpecFiltBook}. Furthermore, the SVD basis \(\{u_j\}\) is usually similar to that of the DFT,as shown in \cite{Hansen2006}, where the authors demonstrate that vectors \(u_j\) corresponding to small indices \(j\) are well represented by just the first few Fourier modes. In contrast, vectors \(u_j\) with large indices \(j\) are shown to include significant contributions from high frequency Fourier modes. These observations suggest that we can replace the SVD basis with the Fourier basis, so that the role of the decreasing singular values in the ordering of the basis vectors is played by the increasing Fourier frequencies. For our procedure to be valid we expect the DFT coefficients of \(b_{true}\) to satisfy an analog of the DPC and therefore to decay to zero as the frequency increases.

For an image \(B\) of size \(M\times N\) we use the two-dimensional DFT
\begin{equation}\label{eq:DFT2D}
\text{DFT2}[B] = \mathcal{F}_M^*B\overline{\mathcal{F}}_N,
\end{equation}
where
\begin{equation}\label{Eq:DFTmat} \left(\mathcal{F}_m\right)_{j,k} = \frac{1}{\sqrt{m}}e^{i2\pi(j-1)(k-1)/m},\end{equation}
is the unitary DFT matrix of size \(m\times m\), \(\overline{X}\) denotes complex conjugation, and \(i=\sqrt{-1}\). Note that \ref{eq:DFT2D} reduces to the one-dimensional DFT if \(N=1\). The data vector \(b\) in \ref{eq:Ax=b} is then obtained by vectorizing the matrix \(B\) by stacking its columns one upon the other so that \(b=\text{vec}(B)\), where \(\text{vec}(\cdot)\) denotes the above vectorization scheme and \(m=MN\) is the resulting length of \(b\). However, the Fourier coefficients found in \ref{eq:DFT2D} cannot be used directly for our purposes because a na\"{\i}ve application of DFT to a non-periodic signal causes artifacts in the frequency domain. Specifically, DFT assumes that the data to be transformed is periodic and therefore application of the DFT to a non-periodic data leads to discontinuities at the boundaries. In the frequency domain, these discontinuities take the form of large high-frequency coefficients \cite{Moisan2011}. Thus Fourier coefficients of smooth but non-periodic true data do not satisfy the DPC as we require. To circumvent this difficulty, we propose to use the Periodic Plus Smooth (PPS) decomposition introduced in \cite{Moisan2011}. The PPS decomposition decomposes an image into a sum
\begin{equation}\label{eq:PPS} B = P + S,\end{equation}
of a periodic component \(P\) very similar to the original one but that smoothly decays towards its boundaries, and a smooth component \(S\) that is nonzero mainly at the boundaries to compensate for the decaying \(P\).
To compute the PPS of \(B\), we define
\begin{equation}\label{eq:PerGapImg}
\begin{aligned}
&V_1(j,k) = \left\{\begin{array}{cc} B(M-j+1,k)-B(j,k), & \text{if } j=1 \text{ or } j=M,\\ 0, & \text{otherwise},\end{array}\right.,\\
&V_2(j,k) = \left\{\begin{array}{cc} B(j,N-k+1)-B(j,k), & \text{if } k=1 \text{ or } k=N,\\ 0, & \text{otherwise},\end{array}\right.,
\end{aligned}
\end{equation}
and \(V = V_1 + V_2\). Then, the two-dimensional DFT of the smooth component \(S\) is given by
\begin{equation}\label{eq:DFTS}
\text{DFT2}[S](j,k) = \left\{\begin{array}{cc} 0, & \text{if } j=k=1,\\ \frac{\text{DFT2}[V](j,k)}{2\cos\left(\frac{2\pi(j-1)}{M}\right)+2\cos\left(\frac{2\pi(k-1)}{N}\right)-4}, & \text{otherwise},\end{array}\right.
\end{equation}
which can be inverted to obtain \(S\) and \(P=B-S\). The subsequent filtering procedure uses only the periodic component \(P\), which contains all the noise as \(S\) is always smooth.

In order to filter the vectorized Fourier coefficients
\begin{equation}\label{eq:VecFourierCoeffs2D} \beta = \text{vec}\left(\text{DFT2}[P]\right),
\end{equation}
by using the Picard parameter, as in Sect. \ref{sec:DirectReg}, we first have to rearrange \(\beta\) so that the first coefficients correspond to the true data while the last are dominated by noise. In \cite{Hansen2006} the coefficients were arranged in order of increasing spatial frequency. Specifically, the basis of the two dimensional Fourier transform is a plane wave given by
\begin{equation}\label{eq:2DFTbasis}
f((j,k),(s,l)) = \exp\left\{-i2\pi \left[\frac{(j-1)(k-1)}{M}+\frac{(s-1)(l-1)}{N}\right]\right\} = \exp\left\{-i2\pi \mathbf{k}\cdot\mathbf{x}\right\},
\end{equation}
where \(\mathbf{k}=\left(\frac{j-1}{M},\frac{s-1}{N}\right)^T\) is the frequency vector, \(j=0,1,...,M\), \(s=0,1,...,N\) and \(\mathbf{r}=(k-1,l-1)^T\) is the spatial vector. The components of \ref{eq:VecFourierCoeffs2D} are arranged in order of increasing magnitude of the spatial frequency \(\mathbf{k}\), given by \(|\mathbf{k}|^2=(j-1)^2/M^2+(s-1)^2/N^2\). We refer to this ordering as the elliptic ordering since the contours of the spatial frequency function \(|\mathbf{k}|^2\) are ellipses centered about the zero frequency. However, use of this arrangement in our numerical experiments causes some results to be highly suboptimal.

An alternative ordering of the Fourier coefficients allows us to overcome this problem. Specifically, we utilize the Kronecker product structure of the two-dimensional Fourier transform, which can be written as a matrix-vector multiplication with \(b\) as
\begin{equation}\label{eq:DFT2kron}
\text{vec}\left(\text{DFT2}[B]\right) = \left(\mathcal{F}^{(2)}_{M,N}\right)^*b.
\end{equation}
Here
\begin{equation}\label{eq:2DForuierMat}
\mathcal{F}^{(2)}_{M,N} = \mathcal{F}_M\otimes\mathcal{F}_N
\end{equation}
is the 2D Fourier transform matrix and '\(\otimes\)' denotes the Kronecker product defined as
\begin{equation}\label{eq:KronDefn} A\otimes B = \left(
                   \begin{array}{ccc}
                     a_{1,1}B & \cdots & a_{1,n}B \\
                     \vdots & \ddots & \vdots \\
                     a_{m,1}B & \cdots & a_{m,n}B \\
                   \end{array}
                 \right).\end{equation}
In view of \ref{eq:2DForuierMat}, we suggest to reorder the Fourier coefficients \(\beta\) in \ref{eq:VecFourierCoeffs2D} according to the ordering permutation \(\pi\) which arranges the Kronecker product
\begin{equation}\label{eq:freqVec}
\mathbf{f}^{(2)}_{M,N} = \mathbf{f}_M\otimes \mathbf{f}_N\in\mathbb{R}^m,
\end{equation}
in increasing order, where \(\mathbf{f}_M\in\mathbb{R}^M\) and \(\mathbf{f}_N\in\mathbb{R}^N\) are the vectors representing the ordered absolute frequencies of \(\mathcal{F}_M\) and \(\mathcal{F}_N\) respectively. Note that since the frequencies in the two-dimensional Fourier transform are shifted so that the zero frequency is located at the corner of the image, the vectors \(\mathbf{f}_N\) and \(\mathbf{f}_M\) in \ref{eq:freqVec} also need to be correspondingly shifted.

The vector \(\mathbf{f}^{(2)}_{M,N}\) whose components are products of the absolute values of spatial frequencies is not ordered and in a component-wise form \ref{eq:freqVec} is given by
\begin{equation}\label{eq:freqVecCompWise}
\left(\mathbf{f}^{(2)}_{M,N}\right)_{M(j-1)+s}=\frac{1}{m}\left\{\begin{array}{ll}
(j-1)(s-1), & 1\leq j \leq \lfloor\frac{N}{2}\rfloor,\ 1\leq s\leq \lfloor\frac{M}{2}\rfloor,\\
(N-j+1)(s-1), & \lfloor\frac{N}{2}\rfloor < j \leq N,\ 1\leq s\leq \lfloor\frac{M}{2}\rfloor,\\
(j-1)(M-s+1), & 1 \leq j \leq \lfloor\frac{N}{2}\rfloor,\ \lfloor\frac{M}{2}\rfloor < s\leq M,\\
(N-j+1)(M-s+1), & \lfloor\frac{N}{2}\rfloor < j \leq N,\ \lfloor\frac{M}{2}\rfloor < s\leq M, \end{array}\right.
\end{equation}
where as above \(m=MN\). We then construct the permutation \(\pi\) such that \(\mathbf{f}^{(2)}_{M,N}(\pi(1:m))\) appears in increasing order and rearrange the coefficients \ref{eq:VecFourierCoeffs2D} to obtain \(\beta\mapsto \beta(\pi(1:m))\). We term this ordering the hyperbolic ordering since the contours of the function \ref{eq:freqVecCompWise} are hyperbolas centered about zero frequency (see \ref{fig:Masks}).

The above rearrangement using the ordering permutation \(\pi\) is analogous to the rearrangement of the SVD decomposition of a separable blur
\begin{equation}\label{eq:SepBlur} A = A_1\otimes A_2.\end{equation}
where \(A_1\in\mathbb{R}^{N\times N}\), \(A_2\in\mathbb{R}^{M\times M}\) and \(A\in\mathbb{R}^{m\times m}\) \cite[Sect. 2]{Hansen2008}. Letting \(A_1 = U_1\Sigma_1V_1^T\) and \(A_2 = U_2\Sigma_2V_2^T\) be the SVD of \(A_1\) and \(A_2\), the SVD of \(A\) \ref{eq:SVDofA} can be written as
\begin{equation}
A = \left(\underbrace{U_1\otimes U_2}_{= U}\right)\left(\underbrace{\Sigma_1\otimes \Sigma_2}_{= \Sigma}\right)\left(\underbrace{V_1\otimes V_2}_{= V}\right)^T.
\end{equation}
As in the case of the two-dimensional Fourier transform \ref{eq:2DForuierMat}, even though the singular values of \(A_1\) and \(A_2\) are ordered, those of \(A\) are not \cite[Sect. 4.4.2]{SpecFiltBook}, \cite{Hansen2008}. To be able to use the filter described in Sect. \ref{sec:DirectReg} we must reorder the entries of \(U\), \(\Sigma\) and \(V\) according to decreasing singular values using the ordering permutation \(\pi\) as in \ref{eq:freqVec}.

Once the Fourier coefficients are rearranged we proceed according to the procedure in Sect.  \ref{sec:DirectReg}. Specifically, we form the sequence \(\{V(k)\}_{k=1}^m\) defined in \ref{eq:VxDefn}, estimate the Picard parameter using \ref{eq:PicIndCondSVD} and set to zero the Fourier coefficients with indices larger than \(k_0\) to form the vector \(\widehat\beta(\pi(1:m)) = (\beta(\pi(1)),\ldots,\beta(\pi(k_0-1)),\underbrace{0,\ldots,0}_{m-k_0+1})^T\). We then invert \ref{eq:VecFourierCoeffs2D} using \(\widehat\beta\) instead of \(\beta\) to obtain the filtered periodic component \(\widehat P\) and the filtered image \(\widehat B = \widehat P + S\). The filtered data vector is then obtained as \(\widehat b = \text{vec}(\widehat B)\).

Note that dropping the last coefficients of the data using the two orderings discussed above can also be interpreted as applying one-parameter windows of different shapes in the Fourier domain. Specifically, the elliptic ordering of \cite{Hansen2006} corresponds to setting to zero the Fourier coefficients outside of an ellipse, whereas the hyperbolic ordering corresponds to doing the same outside a hyperbola. This is illustrated in \ref{fig:Masks} where we plot the Fourier transform coefficients of a \(256\times 256\) image with \(k_0=10^4\) for each of the orderings. Viewed from this perspective, the proposed filtering algorithm simply applies a window depending on the parameter \(k_0\) to the DFT of the perturbed image. The difference between our approach and the approach of \cite{Hansen2006} is in the chosen shape of the window.

\section{Iterative inversion using the GKB}\label{sec:GKBinvert}
In this section, we consider the solution of the ill-posed problem \ref{eq:Ax=b} using the GKB algorithm \cite{Golub1965}. This algorithm approximates the subspace, spanned by the first \(k\) largest right singular vectors of \(A\) with the first \(k\) basis vectors of the Krylov subspace \(\mathcal{K}_k(A^TA,A^Tb)\) (see \cite[sect. 6.3.2]{RankDeff} and \cite[sect. 6.3.1]{HansenInsights}). After \(k\) iterations (\(1\leq k\leq n\)), the GKB algorithm yields two matrices with orthonormal columns \(W_k\in\mathbb{R}^{n\times k}\) and \(Z_{k+1}\in\mathbb{R}^{m\times(k+1)}\), and a lower bidiagonal matrix \(B_k\in\mathbb{R}^{(k+1)\times k}\) with the structure
\begin{equation}\label{eq:BkDefn} B_k = \left(
     \begin{array}{cccc}
       \varrho_1 &   &   &   \\
       \theta_2 & \varrho_2 &   &   \\
         & \theta_3 & \ddots &   \\
         &   & \ddots & \varrho_k \\
         &   &   & \theta_{k+1} \\
     \end{array}
   \right),\end{equation}
such that
\begin{equation}
\label{eq:LanczosRels}\begin{aligned} &AW_k = Z_{k+1}B_k,\\ &A^TZ_{k+1} = W_kB_k^T + \varrho_{k+1}w_{k+1}e_{k+1}^T,\\ &Z_{k+1}\theta_1e_1 = b.
\end{aligned}
\end{equation}
The GKB algorithm is summarized in \ref{alg:GKB}. We perform a reorthogonalization at each step of the algorithm to ensure that the columns of \(W_k\) and \(Z_{k+1}\) remain orthogonal.
\begin{algorithm}
\caption{Golub-Kahan Bidiagonalization (GKB)}\label{alg:GKB}
\begin{algorithmic}
\INPUT{\(A,b,k\)}\Comment{Coefficient matrix \(A\), data vector \(b\) and number of iterations \(k\)}
\OUTPUT{\(W_k,Z_{k+1},B_k\)}
\LineComment{Initialization:}
\State \(\theta_1 \gets ||b||, \quad z_1\gets b/\theta_1\)
\State \(\varrho_1 \gets ||A^Tz_1||, \quad w_1\gets A^Tz_1/\varrho_1\)
\For{\(j=1,2,\ldots,k\)}
    \State \(p_j \gets Aw_{j}-\varrho_jz_j\)
    \State \(p_j\gets \left(I-Z_jZ_j^T\right)p_j\) \Comment{Reorthogonalization step}
    \State \(\theta_{j+1} = ||p_j||, \quad z_{j+1}\gets p_j/\theta_{j+1}\)
    \State \(q_j \gets A^Tz_{j+1}-\theta_{j+1}w_j\)
    \State \(q_j\gets \left(I-W_jW_j^T\right)q_j\) \Comment{Reorthogonalization step}
    \State \(\varrho_{j+1} = ||q_j||, \quad w_{j+1}\gets q_j/\varrho_{j+1}\)
    \LineComment{Update output matrices:}
    \State \(W_j \gets \left(W_{j-1},\ w_j\right), \quad Z_{j+1} \gets \left(Z_{j},\ z_{j+1}\right)\)
    \State \(B_j \gets \left(\begin{array}{cc} \left(\begin{array}{c} B_{j-1} \\ 0 \end{array}\right), & \left(\begin{array}{c} \mathbf{0} \\ \varrho_j \\ \theta_{j+1} \end{array}\right)\end{array}\right)\)
\EndFor
\end{algorithmic}
\end{algorithm}
It can be shown that the columns of \(W_k\) span the Krylov subspace \ref{eq:KrylovSubspace} and that we can achieve a regularizing effect by projecting the solution onto this subspace \cite[Sect. 2.1]{Gazzola2015}.

Choosing a solution from the column space of \(W_k\), such that \(x=W_ky\), and using the relations \ref{eq:LanczosRels}, we can rewrite the residual norm in \ref{eq:GKBprob} as
\begin{equation}\rho(k) = ||b-Ax^{(k)}||^2 = ||U_{k+1}\left(\theta_1e_1-B_ky\right)||^2,\end{equation}
and, since \(U_{k+1}\) has orthonormal columns,
\begin{equation}\label{eq:rewriteLS}
\rho(k) = ||\theta_1e_1-B_ky||^2.
\end{equation}
Hence, solving the PLS problem \ref{eq:GKBprob} amounts to minimizing \ref{eq:rewriteLS}, which is small-scale and can be solved using standard procedures such as the QR decomposition of \(B_k\) as in the LSQR algorithm \cite{LSQR}.

In the absence of noise, the GKB algorithm is terminated once \(\varrho_{k}=0\) or \(\theta_{k+1}=0\).
However, the solution of the PLS problem \ref{eq:rewriteLS} exhibits semi-convergence, whereby the error in the iterates \(||x_{true}-x^{(k)}||\) first decreases as \(k\) increases and then increases sharply well before the above condition is met. This is due to the fact that the columns of \(W_k\) contain increasing levels of noise, as described in \cite{Hnetynkova2009}. Hence, at early iterations the columns are almost noiseless and the solution gets closer to the true one, while at later iterations the solution becomes contaminated by noise. It is thus crucial to appropriately terminate the iterations before the noise becomes dominant.

Usually, standard methods like the GCV or the L-curve are used for estimation of the optimal stopping iteration. However the GCV method assumes that the noise is additive and is fully contained in the data vector \(b\), while the coefficient matrix \(A\) is noiseless. That is indeed the case in the original, large-scale problem \ref{eq:Ax=b} but not in the PLS problem of minimizing \ref{eq:rewriteLS}. In the PLS problem the projected data vector \(Z_{k+1}^Tb=\theta_1e_1\) is a noiseless scaled standard basis vector. The projected coefficient matrix \(B_k = Z_{k+1}^TAW_k\) however, is generated by the Algorithm \ref{alg:GKB} from the noisy data vector \(b\) and depends on the columns of \(W_k\) and \(Z_{k+1}\). Therefore, the derivation of the standard form of the GCV function and the proof of its optimality, such as the ones given in \cite[Thm. 1]{GCV2} no longer apply. The justification for the L-curve method seems to hold for the projected problem, but, as we show in the numerical examples section, for many cases it is far from optimal.

In this paper we propose to use the DF method developed in \cite{Levin2016} to stop the iterative process. The DF method uses the distance between the filtered data \(\widehat b\) and the data reconstructed from the \(k\)th iterate \(Ax^{(k)}\) to characterize the quality of the iterated solution \(x^{(k)}\). Writing the distance as
\begin{equation}\label{eq:DistNormGKB}
\widehat f(k) = ||\widehat b - Ax^{(k)}||^2,
\end{equation}
and using the methods of Sect. \ref{sec:DFTfilter} to obtain the filtered data \(\widehat b\) we expect  \(\widehat f(k)\) to have a global minimum at or near the optimal iteration. However, \(\widehat f(k)\) may also have local minima, and so we must continue the iterations beyond a potential minimum of \ref{eq:DistNormGKB} to ensure that the function \(\widehat f(k)\) continues to increase. In addition, for problems with very small noise levels, the filter of Sect. \ref{sec:DFTfilter} may not change the data vector \(b\) sufficiently, in which case \(\widehat f(k)\) will not have a minimum. Instead, \(\widehat f(k)\) will flatten after the optimal iteration, since the solution will not become significantly contaminated. To account for all the above behaviors, we propose to terminate the iterations once the relative change in \ref{eq:DistNormGKB} is small enough
\begin{equation}\label{eq:GKBcond}
\frac{\widehat f(k)-\widehat f(k+1)}{\widehat f(k)} \leq \delta,\quad \text{for \(p\) consecutive iterations}
\end{equation}
for some small bound \(\delta>0\). We emphasize that the numerator in \ref{eq:GKBcond} is not an absolute value and becomes negative if \(\widehat f(k)\) starts to increase. Since we choose \(\delta>0\), the condition \ref{eq:GKBcond} is automatically satisfied if \(\widehat f(k)\) has a minimum. Otherwise the iterations are terminated once \(\widehat f(k)\) becomes very flat. From our experience, the algorithm works well for \(\delta\in [10^{-2}, 10^{-4}]\) and \(p \geq 5\). Finally, we observe that since a discrete iteration number (\(k\in\{1,\ldots,n\}\)) acts as a regularization parameter, the ability of the reconstructed data \(Ax^{(k)}\) in \ref{eq:DistNormGKB} to get closer to the filtered data is limited, making the minimum of \ref{eq:DistNormGKB} less sensitive to the quality of the filtered data \(\widehat b\). Furthermore, the discrete regularization parameter does not limit the accuracy of the solution in practice. As we demonstrate in the numerical examples below, the additional regularization of hybrid methods is unnecessary and only slightly improves the solution, if at all.

\subsection{Regularizing the projected problem}\label{sec:TikhRegProj}
In this section, we briefly discuss hybrid methods for the solution of \ref{eq:Ax=b}, which combine projection onto a Krylov subspace with Tikhonov regularization and are the main competitors to our approach. Instead of attempting to terminate the GKB process at an optimal iteration, hybrid methods replace the PLS problem \ref{eq:GKBprob} with the Tikhonov minimization problem \ref{eq:TikhMinProb}. Using the relations \ref{eq:LanczosRels} and \(x=W_ky\), as in Sect. \ref{sec:GKBinvert}, problem \ref{eq:TikhMinProb} can be rewritten as
\begin{equation}\label{eq:MinTikhGKB} \min_y ||\theta_1e_1-B_ky||^2 + \lambda^2 ||y||^2,\end{equation}
or as the normal equation
\begin{equation}\label{Eq:TikhNormalEq0}
\left(B_k^TB_k + \lambda^2 I\right)y = B_k^T\theta_1e_1
\end{equation}
which has the solution
\begin{equation}\label{Eq:TikhNormalEq}
y_\lambda = \left(B_k^TB_k + \lambda^2 I\right)^{-1}B_k^T\theta_1e_1.
\end{equation}
The solution \ref{Eq:TikhNormalEq} can be rewritten, similarly to Sect. \ref{sec:DirectReg}, using the SVD of \(B_k\) as
\begin{equation}\label{eq:SVDbk}
B_k = U_k\Sigma_k V_k^T,
\end{equation}
where \(U_k\in \mathbb{R}^{(k+1)\times (k+1)}\) and \(V_k\in\mathbb{R}^{k\times k}\) are orthogonal and \(\Sigma_k\in\mathbb{R}^{(k+1)\times k}\) has the structure
\begin{equation}\label{eq:SigmaStruct}
\Sigma_k = \left(\begin{array}{c} \text{diag}\{\sigma_1,\ldots,\sigma_k\} \\ \mathbf{0}^T \end{array}\right),
\end{equation}
with the singular values \(\{\sigma_j\}\) arranged in decreasing order.
 Due to the structure of \(B_k\) in \ref{eq:BkDefn} and the fact that \(\varrho_j,\; \theta_j\; >0\) for all relevant iterations, we have \(\text{rank}(B_k)=k\) and \(\sigma_j >0\) for all \(j\leq k\). Denoting the \(j\)th columns of \(U_k\) and \(V_k\) as \(u^{(k)}_j\) and \(v^{(k)}_j\) respectively, the Tikhonov solution \ref{Eq:TikhNormalEq} can be written similarly to \ref{eq:TikhSoln} as
\begin{equation}\label{eq:TikhSolnGKB} y_{\lambda} = \theta_1\sum_{j=1}^k \frac{\sigma^{(k)}_ju_j^{(k)}(1)}{\left(\sigma^{(k)}_j\right)^2+\lambda^2}v^{(k)}_j,\end{equation}
where \(u^{(k)}_j(1)=e_1^Tu^{(k)}_j\).

By appropriately choosing \(\lambda\) at each iteration we can, in theory, filter out noise added to the least-squares solution in \ref{eq:GKBprob} at higher iterations and thus, stabilize the error and make the final solution independent of the stopping iteration. Regularization in hybrid methods may also filter out noise that is not filtered by projection alone. Nevertheless we argue that this additional filtering has a negligible effect and that choosing \(\lambda\) appropriately at each iteration presents a significant challenge. Specifically, determination of \(\lambda\) for hybrid methods is usually done using standard procedures originally developed for direct regularization, the most popular of which is the GCV \cite{splines,GCV2}. These procedures assume a noiseless coefficient matrix \(A\) \ref{eq:MinTikhGKB}, whereas the hybrid methods project the solution into a noise-dependent Krylov space, thus also contaminating the projected coefficient matrix, similarly to the PLS problem. Therefore, the GCV method is not expected to produce optimal solutions for hybrid methods, as was indeed demonstrated in \cite[Sect. 4]{Chung2008}. The W-GCV method proposed in \cite{Chung2008} attempts to overcome the above shortcoming of the GCV by introducing an additional free parameter and choosing it adaptively at each iteration. However, as we show in the numerical examples in Sect. \ref{sec:NumEx}, the W-GCV method still produces suboptimal results in many cases.

In all of our numerical examples we observe that the minimum errors achievable using PLS and hybrid methods are almost identical and therefore any additional filtering achievable by hybrid methods is negligible. To explain this we note that at early iterations, the vectors spanning the Krylov subspace \ref{eq:KrylovSubspace}, and hence also the solutions projected onto them, are typically very smooth and do not contain noise \cite{Hnetynkova2009}. Therefore, little to no regularization is required at this stage and we expect \(\lambda\approx 0\), making the Tikhonov problem \ref{eq:TikhMinProb} equivalent to the least-squares problem \ref{eq:GKBprob}. Only after the basis vectors spanning \ref{eq:KrylovSubspace} become contaminated by noise does the solution require regularization. At this stage the optimal regularization parameter \(\lambda\) increases to some non-negligible, noise-dependent value that keeps the error of the regularized solution approximately constant, while the error of the unregularized PLS solution increases sharply. We demonstrate in the following section, using several numerical examples, that the optimal solution of the hybrid method occurs while the noise that contaminates it is very small and so in most typical cases the optimal solutions of \ref{eq:GKBprob} and \ref{eq:TikhMinProb} are very close to each other. This was also demonstrated in the numerical examples in  \cite[Sect. 4]{Chung2015} and \cite{Chung2008}. Therefore, in most practical problems the only significant advantage of hybrid methods over simple GKB stopping criteria is in the ability to stabilize the iterations, making them less sensitive to the stopping iteration. This also implies that having a reliable stopping criterion for PLS obviates the need for a hybrid method in these cases.

\section{Numerical examples}\label{sec:NumEx}
In this section we demonstrate the performance of the proposed method using seven test problems from the \texttt{Matlab} toolbox \texttt{RestoreTools} \cite{Nagy2004}: \texttt{satellite}, \texttt{GaussianBlur440}, \texttt{AtmosphericBlur50}, \texttt{Grain}, \texttt{Text}, \texttt{Text2} and \texttt{VariantMotionBlur\_large}. Each of these problems includes a different blur, \(A\), and a different image, \(x_{true}\), to reconstruct. To generate the data vector \(b\), we form the true data, \(b_{true} = Ax_{true}\) and perturb it with white Gaussian noise of variance \(s^2=\alpha\max|b_{true}|^2\) where the noise level \(\alpha\) takes on three values for each problem \(\alpha\in\{10^{-2},10^{-4},10^{-6}\}\). We apply the inversion procedure to each test problem 100 times, each time using a different noise realization.

To implement our stopping criterion, we set \(h=\lceil\frac{m}{100}\rceil\) and \(\varepsilon=10^{-2}\) in \ref{eq:PicIndCondSVD} and also \(p=5\) and \(\delta=2\times10^{-3}\) in \ref{eq:GKBcond}. As mentioned above, however, our numerical results are robust and wouldn't change much for a wide range of \(h\), \(\varepsilon\), \(p\) and \(\delta\). In the numerical tests we compare our stopping criterion with the L-curve criterion \cite{RegParamItr,LCurve,Calvetti2000}, \cite[Chap. 7]{RankDeff}, the NCP method for the PLS problem \cite{Hansen2006,Rust2008}, and the hybrid W-GCV method \cite{Chung2008}. The L-curve method constitutes finding the point of maximum curvature on the so-called L-curve, defined as \((||r^{(k)}||,\; ||x^{(k)}||)\), where \(r^{(k)}=b-Ax^{(k)}\) is the residual vector. To do so we use the function \verb!l_corner! from Hansen's \texttt{Regularization Tools} toolbox \cite{RegTools}. Using the L-curve method, we terminate the iterations once the chosen iteration number either stays the same or decreases for \(p=5\) consecutive iterations, signifying that the corner of the L-curve is found.

The NCP method is based on calculating a whiteness measure of the residual vector \(r^{(k)}\) at each iteration \cite{Hansen2006}. The stopping iteration is chosen as the one at which the residual vector \(r^{(k)}\) most resembles white noise, as follows. The vector \(r^{(k)}\) is reshaped into an \(M\times N\) matrix \(R^{(k)}\) satisfying \(r^{(k)} = \text{vec}\left(R^{(k)}\right)\), and the quantity \(\widehat R = |\text{DFT2}(R^{(k)})|\) is defined to be the absolute value of its two-dimensional Fourier transform. Since \(R^{(k)}\) is a real valued signal, it follows that \(\widehat R\) is symmetric about \(q_1 = \lfloor M/2 \rfloor + 1\) and \(q_2 = \lfloor N/2 \rfloor +1\), so that \(\widehat R_{j,k} = \widehat R_{M-j,k}\) for \(2\leq j\leq q_1-1\) and \(\widehat R_{j,k} = \widehat R_{j,N-k}\) for \(2\leq k\leq q_2-1\). Consequently, only the first quarter of \(\widehat R\), which can be written as \(\widehat T = \widehat R(1:q_1,1:q_2)\) using \texttt{Matlab} notation, is required for the analysis. Vector \(\widehat t\) is then obtained by vectorizing \(\widehat T\) using the elliptical parametrization defined in Sect. \ref{sec:DFTfilter}. The NCP of \(R^{(k)}\) is defined as the vector \(c(R^{(k)})\) of length \(q_1q_2-1\) with components\footnote{In \cite{Hansen2006}, the authors assume the problem is square so that \(M=N\) and \(q_1=q_2\).}
\begin{equation}\label{eq:NCPdefn}
c(R^{(k)})_j = \frac{||\widehat t(2:j+1)||_1}{||\widehat t(2:q_1q_2)||_1},
\end{equation}
where the dc component of \(R^{(k)}\) is not included in the NCP. We note that it is argued in \cite{Rust2008} that the NCP should include the dc component of \(R^{(k)}\) since it captures the deviation from zero mean white noise. However, we found no difference between the two definitions in practice and therefore we shall follow \cite{Hansen2006} and disregard the dc component as in \ref{eq:NCPdefn}. It is shown in \cite{Hansen2006,Rust2008} that for white noise, the NCP should be a straight line from 0 to 1 represented by the vector with components \(s_j = j/(q_1q_2-1)\). Therefore, the whiteness measure is defined as the distance
\begin{equation}
\label{eq:NCPfunc}
N(k) = ||s-c(R^{(k)})||_1.
\end{equation}
The iterations are terminated once \ref{eq:NCPfunc} reaches its global minimum, signifying that the residual vector at the chosen iteration is the closest to white noise. To implement this method, we compute function \ref{eq:NCPfunc} at each iteration and terminate the iterations once the norm \ref{eq:NCPfunc} increases for \(p=5\) consecutive iterations, just as we do with our own method. We then choose the solution corresponding to the global minimum of the computed \(N(k)\).

In contrast to the above methods, the W-GCV is a hybrid method and solves the Tikhonov problem \ref{eq:TikhMinProb}. It is based on introducing a free parameter to the GCV criterion, as discussed in Sect. \ref{sec:TikhRegProj} and in \cite{Chung2008}. To implement the W-GCV, we use the \texttt{HyBR\_modified} routine provided in the \texttt{RestoreTools} package \cite{Nagy2004} as \texttt{x = HyBR\_modified(A,B,[],HyBRset('Reorth','on'),1)}. Note that we use the reorthogonalization option of the \verb!HyBR_modified! routine to make a fair comparison to our \ref{alg:GKB} that employs full reorthogonalization.

We measure the quality of a solution by computing its Mean-Square Deviation (MSD), defined as
\begin{equation}\label{eq:MSD} \text{MSD} = \frac{||x_{true} - x||^2}{||x_{true}||^2},\end{equation}
where \(x\) is a solution. We then define the optimal solutions to the PLS problem \ref{eq:GKBprob} and to the hybrid problem \ref{eq:TikhMinProb} as the ones minimizing the MSD to each problem.
We present the results of our simulations as boxplots of the resulting MSD values in \ref{fig:Boxplots_1-3} and \ref{fig:Boxplots_4-6}. The boxplots divide the data into quartiles with the boxes spanning the middle 50\% of the data, called the interquartile range and the vertical lines extending from the boxes span 150\% of the interquartile range above and below it. Anything outside this interval is considered an outlier and is marked with a '+'. Each box also contains a horizontal line marking the median of the data.

Based on the results presented in \ref{fig:Boxplots_1-3} and \ref{fig:Boxplots_4-6} we can make the following observations:
\begin{enumerate}
    \item The hyperbolic ordering with the DF method performs similarly to or better than the corresponding elliptic ordering in all examples.
    \item The DF method with hyperbolic ordering performs similarly to or outperforms the L-curve, NCP and W-GCV methods in all examples without exception.
    \item The DF method with elliptic ordering failed to produce acceptable solutions for the \texttt{Text2} problem with \(\alpha=10^{-4}\). Contrary to the other examples where the distance function \ref{eq:DistNormGKB} with this ordering has a minimum, in this example it has neither a minimum nor even an inflection point and therefore the right stopping iteration could not be found with this ordering.

    \item The optimal MSD values for the PLS problem \ref{eq:GKBprob} and the projected Tikhonov problem \ref{eq:TikhMinProb} are almost identical in all examples, as expected from the discussion in \ref{sec:TikhRegProj}.
\end{enumerate}
 Overall, we can conclude that the DF criterion with hyperbolic ordering for estimation of optimal stopping iteration is accurate, robust and outperforms state-of-the-art methods.

\begin{figure}[!p]
    \centering
    \hspace*{\fill}
    \begin{subfigure}[b]{0.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{MaskCirc.pdf}
    \end{subfigure}
    \hfill
    \begin{subfigure}[b]{0.45\textwidth}
        \centering
        \includegraphics[width=\textwidth]{MaskKron.pdf}
    \end{subfigure}
    \hspace*{\fill}
\caption{Effect of the filtering procedure introduced in Sect. \ref{sec:DFTfilter} on an image of size \(256\times256\) by the elliptic ordering of \cite{Hansen2006} and the hyperbolic ordering \ref{eq:freqVec} in Fourier domain. The Picard parameter for both methods is \(k_0=10^4\). The zero frequency component is placed at the center of the image.}\label{fig:Masks}
\end{figure}

 \begin{figure}[!p]
    \centering
    \includegraphics[width=\textwidth]{box_1-4.pdf}
    \caption{Boxplots of the MSD values obtained for the PLS problem \ref{eq:GKBprob} and the Tikhonov regularized problem \ref{eq:TikhMinProb} with the methods: (1) DF with hyperbolic ordering (DF-h), (2) DF with elliptic ordering (DF-e), (3) L-curve, (4) NCP, (5) W-GCV, (6) minimum MSD for PLS problem (PLS\(_{OPT}\)), (7) minimum MSD solution for Tikhonov problem (Tikh\(_{OPT}\)). The problems presented are \emph{First row}: \texttt{satellite}; \emph{Second row}: \texttt{GaussianBlur440}; \emph{Third row}: \texttt{AtmosphericBlur50}; \emph{Fourth row}: \texttt{Grain}. The noise levels are \emph{First column}: \(\alpha=10^{-2}\); \emph{Second column}: \(\alpha=10^{-4}\); \emph{Third column}: \(\alpha=10^{-6}\).}\label{fig:Boxplots_1-3}
\end{figure}
\begin{figure}[!p]
    \centering
    \includegraphics[width=\textwidth]{box_4-7.pdf}
    \caption{Boxplots of the MSD values obtained for the PLS problem \ref{eq:GKBprob} and the Tikhonov regularized problem \ref{eq:TikhMinProb} with the methods: (1) DF with hyperbolic ordering (DF-h), (2) DF with elliptic ordering (DF-e), (3) L-curve, (4) NCP, (5) W-GCV, (6) minimum MSD for PLS problem (PLS\(_{OPT}\)), (7) minimum MSD solution for Tikhonov problem (Tikh\(_{OPT}\)). The problems presented are \emph{First row}: \texttt{Text}; \emph{Second row}: \texttt{Text2}; \emph{Third row}: \texttt{VariantMotionBlur\_large}. The noise levels are \emph{First column}: \(\alpha=10^{-2}\); \emph{Second column}: \(\alpha=10^{-4}\); \emph{Third column}: \(\alpha=10^{-6}\).}\label{fig:Boxplots_4-6}
\end{figure}

\bibliographystyle{plain}


\section{Model}
The Hamiltonian describing the aforementioned model is given by
\begin{equation}
H = \sum_{i=3}^{L-2}(b_i + b_i^\dagger ) \cdot \left(\mathcal{N}^3_i+
\mathcal{N}^2_i \right)
\label{ham}
\end{equation}
where {$L$ is the number of sites;}
$b$ and $b^\dagger$ are the usual annihilation and creation
operators ($\hbar=1$); the operators
$\mathcal{N}^2_i = \sum_P n_{\alpha}n_{\beta}\bar n_{\gamma} \bar n_{\delta}$ and
$\mathcal{N}^3_i=\sum_{P'} n_{\alpha} n_{\beta}  n_{\gamma} \bar n_{\delta}$
($n=b^\dagger b$, $\bar n = 1-n$, the indices $\alpha, \beta, \gamma,
\delta$ label the four neighbouring sites)
count the population present in the four neighbouring sites (the sum
runs on every possible permutation $P$ and $P'$ of the positions of
the $n$ and $\bar n$ operators) and $\mathcal{N}^2$ ($\mathcal{N}^3$)
gives the null operator if the population is different
from two (three), the identity otherwise. For classical states, as for
example an initial random configuration of dead and alive states,
the Hamiltonian~(\ref{ham}) is, at time zero, $H_{Active}= b_i + b_i^\dagger $
on the sites with two or three alive neighbours
and $H_{Hibernate}= 0$ otherwise. If the Hamiltonian would remain
constant, every active site would oscillate forever while the
hibernated ones would stand still. On the contrary
as soon as the evolution starts, the state evolves into a
superposition of possible classical configurations, resulting in a
complex dynamics as shown below and the interaction between sites
starts to play a role.
{Thus, the Hamiltonian introduced in Eq.~(\ref{ham}) induces
a quantum dynamics that resembles the rules of the GoL: a site with less
than two or more than three alive neighbouring sites "freezes'' while, on
the contrary, it ``lives''. The difference with the classical game --
connected to the reversibility of quantum dynamics --
is that ``living'' means oscillating with a typical timescale between
two possible classical states (see e.g. Fig.~\ref{rule}.}

\begin{figure}
  \begin{center}
    \includegraphics[width=4cm]{./fig3.eps}
    \caption{Schematic representation of a one-dimensional
      time-evolution of the discretized population
      $\mathcal{D}_i(t)$ of a
      ``blinker'' (case A of Fig.~\ref{timeev}).
      From left to right the states of subsequent generations are
      sketched.
      Empty (blue) squares are ``dead'' sites, coloured (red)
      ones are ``alive''}
    \label{blink}
  \end{center}
\end{figure}
\section{Dynamics} To study the quantum GoL dynamics we employ the time
dependent Density Matrix Renormalization group (DMRG). Originally
developed to investigate condensed matter
systems, the DMRG and its time dependent extension have been proven to
be a very powerful method to numerically investigate many-body quantum
systems~\cite{white,daley,revDMRG, dechiara}. As it is possible to use it efficiently only in
one-dimensional systems, we concentrate to the one-dimensional version
of the Hamiltonian~(\ref{ham}): the operators  $\mathcal{N}^2$
and $\mathcal{N}^3$ count the populated sites on the nearest-neighbour
and next-nearest-neighbour sites and thus $\alpha=i-2, \beta=i-1, \gamma=i+1,
\delta=i+2$. Note that it has been shown that the main
statistical properties of the classical GoL are the same in
both two- and one-dimensional versions~\cite{oned}.

{To describe the system dynamics we introduce different quantities
that characterise in some detail the system evolution.}
We first concentrate on the population dynamics, measuring the
expectation values of the number operator at every site $\langle
n_i(t) \rangle$. This clearly gives a picture of the ``alive''
and ``dead'' sites as a function of time, as it gives the probability
of finding a site in a given state when measured. That is, if we observe
the system at some final time $T_f$ we will find dead or alive sites
according to these probabilities. In Fig.~\ref{timeev} we show three
typical evolutions (leftmost pictures): configuration $A$
corresponds to a ``blinker'' where two couples of nearest-neighbour
sites oscillate regularly between dead and alive states {(a schematic
representation of the resulting dynamics of the discretised population
 $\mathcal{D}_i(t)$ is reproduced also in Fig.~\ref{blink})};
configuration $B$ is
a typical overcrowded scenario where twenty-four ``alive'' sites
disappear leaving behind only some residual activity; finally a
typical initial random configuration ($C$) is shown.
Notice that in all configurations it is possible to identify the
behaviour of the wave function tails that propagate and generate
interference effects. These effects can be highlighted by computing
the visibility of the dynamics, the maximum
variation of the populations within subsequent generations,
defined as:
\begin{equation}
v_i(t)= |\max_{t'} n_i(t') - \min_{t'} n_i(t')|;
t'\in[t-\frac{T}{2};t+\frac{T}{2}];
\end{equation}
{that is, the visibility at time $t$ reports the maximum variation of the
population in the time interval of length $T$ centered around $t$.}
The visibility clearly follows the preceding dynamics (see
Fig.~\ref{timeev}, second column) and identifies the presence
of ``activity'' in every site.

\begin{figure}
  \begin{center}
     \includegraphics[width=4cm]{./avpop.eps}
     \includegraphics[width=4cm]{./avpopinf.eps}
\\
     \includegraphics[width=4cm]{./avdiv.eps}
     \includegraphics[width=3.9cm]{./avdivinf.eps}
    \caption{Left: Average population $\rho(t)$ (upper)
      and diversity $\Delta(t)$ (lower) as a function of time {for
      different initial population density $\rho_0$}.
    Right: Equilibrium average population $\rho$ (upper) and diversity
    $\Delta$ (lower) for the quantum (blue squares) and classical (red
    circles) GoL {as a function of the initial population density
    $\rho_0$}. Simulations are performed with a t-DMRG at third
    order, Trotter step $\delta t=10^{-2}$, truncation dimension
    $m=30$, size $L=32$,  averaged over up to thirty
    different initial configurations.}
    \label{stat}
  \end{center}
\end{figure}

To stress the connections and comparisons with the original GoL we
introduce a classical figure of merit (shown in the third column of
Fig.~\ref{timeev}):  we report a
discretized version of the populations as a function of time
($\mathcal{D}_i(t)=1$ for $n_i(t) > 0.5$ and $\mathcal{D}_i(t)=0$
otherwise). {Notice that $\mathcal{D}_i(t)$ gives the most
probable configuration of the system after a measurement
on every site in the basis $\{|0\rangle, |1\rangle\}$.}
{Thus, we recover} a ``classical'' view of the
quantum GoL with the usual definition of site status. For example,
configuration $A$ is a ``blinker'' that changes status at every
generation {(see Fig.\ref{timeev} and \ref{blink})}.
More complex configurations appear in the other two
cases. The introduction of the discretized populations $\mathcal{D}_i$
can also be viewed as a new definition of ``alive'' and ``dead''
sites from which we could have started from the very beginning to
introduce a stochastic component as done in~\cite{SMgol}.
This quantity allows analysis to be performed as usually done on
the classical GoL and to stress the similarities between the quantum and
the classical GoL.
Following the
literature to quantify such complexity, we compute the clustering
function
$\mathcal{C}(\ell,t)$ that gives the number of clusters of
neighbouring ``alive'' sites of size $\ell$ as a function of time~\cite{oned}.
{For example, the function $\mathcal{C}(\ell,t)$ for a uniform
distribution of ``alive'' sites would be simply $\mathcal{C}(L)=1$ and
zero otherwise while a random pattern would result in a random cluster
function.} This function characterises the complexity of the evolving patterns,
e.g. it is oscillating between zero- and two-size clusters for the
initial condition $A$, while it is much more complex for the random
configuration $C$ (see Fig.~\ref{timeev}, rightmost column).

\section{Statistics}
To characterise the statistical properties of the quantum GoL we study
the time evolution of different initial random configurations as a
function of the initial density of alive sites. {We concentrate
on two macroscopic quantities: the density of the sites
that if measured would with higher probability result in ``alive'' states
\begin{equation}
\rho(t)=\sum_i \mathcal{D}_i(t)/L;
\end{equation}
and the diversity
\begin{equation}
\Delta(t)= \sum_\ell\mathcal{C}(\ell,t),
\end{equation}
the number of different cluster sizes that are present
in the systems, that quantifies the complexity of the
generated dynamics~\cite{oned, SMgol}.} Typical results,
averaged over different initial configurations, are shown in
Fig.~\ref{stat} (left). As it can be clearly seen
the system equilibrates and the density of states as well as the
diversity reach a steady value. This resembles the typical
behaviour of the classical GoL where any typical initial random
configuration eventually equilibrates to a stable configuration.
Moreover, we compare the quantum GoL with a classical {reversible}
version of GoL corresponding to that introduced here:
at every step a cell changes its status if and only if
within the first four neighbouring cells only three or two are alive.
Notice that, the evolution being unitary and thus reversible,
the equilibrium state locally changes with time, however the
macroscopic quantities reach their equilibrium values {that
depend} non trivially only on the initial population density.
{In fact, for the classical game, we
were able to check that the final population density is independent
of the
system size while the final diversity scales as $L^{1/2}$ (up
to $2^{10}$ sites, data not shown).}
Moreover, the {time needed to reach equilibrium}
 is almost independent of the system
size and initial population density.
{These results on the scaling of classical system properties
support the conjecture that our findings for the quantum case
will hold in general},
while performing the analysis for bigger system sizes is highly demanding.
A detailed analysis of the size scaling of the system properties
will be presented elsewhere.
In Fig.~\ref{stat} we report the final (equilibrium)
population density (right upper) and diversity (right lower)
as a function of the initial population density for both the classical
and the quantum GoL for
systems of $L=32$ cells. The equilibrium population density $\rho$
is a non linear function of the initial one $\rho_0$ in both cases:
the classical one has an initial linear dependence up to half-filling
where a plateau is present up to the final convergence to unit filling
for $\rho_0=1$. Indeed, the all-populated configuration is
a stable system configuration. The quantum GoL follows a
similar behaviour, with a more complex
pattern. Notice that here a first signature of quantum behaviour is
present: the steady population density reached by the quantum
GoL is always smaller than its classical counterpart. This is
probably due to the fact that the evolution is not completely captured
by this classical quantity: the sites with population below half
filling, i.e. the tails of the wave functions, are described as
unpopulated by $\mathcal{D}_i$. However, this missing population
plays a role in the evolution: within the overall superposition of
basis states, a part of the probability density (corresponding to the
states where the sites are populated)
undergoes a different evolution than the classical one.
In general, the quantum system is effectively more populated than
the classical $\rho$ indicates. This difference in the
quantum and classical dynamics is even more evident in the dependence of the
equilibrium diversity on the initial population density
$\rho_0$. In the classical case the maximum diversity is slightly
above three: on average, in the
steady state, there are no more than about three different cluster
sizes present in the system independently of the initial
configuration. On the contrary --in the quantum case-- the
maximal diversity is about four, increasing the information content
(the complexity) generated by the evolution by about $10-20\%$.

{These findings are a signature of the difference between quantum and
classical GoL. In particular} we have shown that the quantum GoL has a
higher capacity of generating diversity than the {corresponding}
classical one. This property arises from the possibility of having quantum
superpositions of states of single sites. Whether purely quantum
correlations (entanglement) play a crucial role is under
investigation. Similarly, as there is some arbitrariness
in our definition of the quantum GoL, the investigation of
possible variations is left for future work.

The investigation presented here fits perfectly as
a subject of study for quantum simulators, like for example cold atoms
in optical lattices. Indeed, the five-body Hamiltonian~(\ref{ham})
can be written in {pseudo spin-one-half operators (Pauli matrices)} and
thus it can be simulated along the lines
presented in~\cite{cirac}. In particular, these simulations would give
access to investigations in two and three dimensions that are
not feasible by means of t-DMRG~\cite{revDMRG}.

In conclusion we note that this is {one of the few
available simulations of a many-body quantum game scalable in the number
of sites~\cite{qgames,mbqg,chen04}}. With a straightforward generalisation
(adding more than one possible strategy defined in
Eq.~(\ref{ham})) one could study also different
many-player quantum games. This approach will allow
different issues {to be studied} related
to many-player quantum games such as the
appearance of new equilibria and their thermodynamical
properties.
Moreover, the approach introduced here shows that one might
investigate many different aspects of many-body quantum systems with
the tools developed in the field of complexity and dynamical systems:
In particular,
the relations with Hamiltonian quantum cellular automata
in one dimension and quantum games~\cite{qgames,nagaj}.
Finally, the search for the possible existence of self-organised
criticality in these systems along the lines of similar investigations
in the classical GoL~\cite{SOC}, if successful, would be the
first manifestation of such effect in a quantum system and might
have intriguing implications in quantum gravity~\cite{smolin,gupta}.

After completing this work we became aware of another work on the same
subject~\cite{arXive}.

We { acknowledge}
interesting discussions and support by R.~Fazio and M.B.~Plenio,
the SFB-TRR21, the EU-funded projects
AQUTE, PICC for funding, the BW-Grid for computational
resources, and the PwP project for the t-DMRG code (www.dmrg.it).


\section{Introduction}
Decentralization, blockchain technology, and token-based economics are all concepts incorporated into Web3, a new iteration of the World Wide Web \cite{ww}. The origin of Web3 can be traced to Tim Berners-Lee \cite{1}, the founder of the World Wide Web and the first person to summarize and discuss the idea of Web3, which he originally referred to as the "Semantic Web". However, Web3 is currently defined by a set of principles focused on decentralization, user ownership of data, and cryptocurrency \cite{3}. Berners-Lee identifies the initial challenge of creating a system that allows knowledge sharing with no central governance or forced commonalities that make sharing inherently tricky. This idea was partially delivered by introducing search engines and globally accepted guidelines on structuring data so that it can be indexed \cite{2}, which helped organize data and make it more accessible. This combination of "do-gooding" developers and growing search engines led to enormous amounts of shared data which is now accessible to a wide audience \cite{2}.

Web3, or the semantic web, has evolved from the idea of growing the Internet out of this inherent paradox of an encyclopedia without a set organization to creator or user ownership of the knowledge and assets that collectively comprise the Internet. The solution, as identified by Berners-Lee, is the ability to pass data along with the rules and logic that govern it from source to source while attempting to lose as little information as possible \cite{2}. The implementation of this solution is now contained in the public blockchains that are the foundation of cryptocurrencies and smart contracts \cite{a5}. This in theory, resolves the limitation of needing developers to create knowledge in good faith to be able to be shared and indexed because of the ability to pass logic across the Internet. Berners-Lee also highlights that the benefits of this idea are not waiting for an astronomical breakthrough in technology where "future software agents…can navigate the wealth of its rich representations." Instead, it can be gradually realized as systems provide more detailed information \cite{2}, and the community makes a conscious effort to adopt those core principles of knowledge sharing enablement and user ownership.
With the connection between systems through smart contracts, the benefits of a Web3 world can begin to be realized as greater change comes over time. This fundamental move to Web3 is not synonymous with a technology upgrade but rather aligned with a growing Internet culture focused on data sharing and ownership that can enable computers to process this data better for the benefit of the wider community. As Berners-Lee puts it, "think of the web today as turning all the documents in the world into one big book, then think [Web3] as turning all the data into one big database, or one big mathematical formula. Web3 is not a separate web; it is a new form of information adding one more dimension to the diversity of the one web we have today" \cite{2}. Simply put, Web1 was read-only. Web2 was read-write. Web3 is read-write-own \cite{4}.

\section{Foundation of Web3 and Enablement}\label{ExSurv}
Web3, as mentioned by Berners-Lee, is not going to hinge on a technological breakthrough, although there is evidence already of creative libraries that enable Web3 ideas. Web3 is an ideological shift in how the Internet of data is constructed that can be enabled with existing technology to produce benefits today. The enablement of Web3 will depend on the transition to Web3 libraries and concepts by knowledge and data creators. The foundational items that currently comprise the Web3 framework are blockchain networks (decentralized \cite{a6} but connected nodes), Web3 libraries, emerging specialized languages (Solidity), identity stores (wallets), smart contracts, and specialized service providers \cite{5}.

\subsection{Blockchains (Bitcoin, Ethereum, etc.)}
The public blockchains are the core pipelines for all transactions and interactions that occur within Web3 applications. They are the public ledgers of record, decentralized and immutable \cite{a7}. A global permissionless database that provides the ability to track the ownership chain of any asset or piece of data that exists on the blockchain at any time by anyone. While made popular by cryptocurrencies, their utility extends to any form of data. This is about Facebook or any other social network moving your profile to a blockchain where you have complete control to grant, revoke, or even sell your data rather than giving it away for the utility of using the social network. In Web2, the network controlled and held all the value. The principles of Web3 dictate that the data holds all the value, not the network. Web3 is an ideological shift for the Internet, just as Web2 was for Web1.
As users interact across Web3 and its associated frameworks, there is a functional need to house the necessary transactions and interactions. The Web3 components which provide this service and enable blockchains to function are known as nodes. Without nodes, the applications cannot communicate; thus, the decentralization of applications becomes voided. Nodes facilitate the tracking of data and are the multiple storehouses that exist to fight against data loss threats \cite{5} and have additional copies of the interaction. This also makes transactions verifiable and immutable because this would be the corroborating source for other Web3 applications.
\subsection{Web3 Libraries}
The Web3 libraries provide a set of application programming interfaces to bridge blockchains and smart contracts, enabling the creation of a new class of applications, commonly known as Decentralized Applications (aka DApps). By their very definition, Web3 applications utilize a blockchain. Ethereum is the most common blockchain used by DApps as Ethereum was created to support application development and has a governance model specifically crafted to be developer friendly. Web3.js, Next.js, Ether.js, and Truffle Suite \cite{5} are commonly used Web3 application development libraries, which are all JavaScript based. Cloudflare provides an example of this interaction here, which hosts an open-source NFT project to explain Web3 and blockchain \cite{6}.

   \begin{figure}
    \centering
    \includegraphics[width=10cm]{p2.png}
    \caption{	System targets from a user-centric explainable AI framework \cite{5}}
    \label{fig:life}
\end{figure}
\subsection{Identity Stores (Wallets)}
	With ownership being at the heart of the Web3 principles, immutable identities are a requirement. Wallets are applications that store and protect the identity used for interacting with blockchain and facilitate the literal transaction. This wallet is the “final authority of your data” and can be any entity \cite{6}. Transactions are totaled here, and all the interactions with knowledge will also be tracked here.

\subsection{Smart Contracts}
	A smart contract \cite{SINGH2020101654} is akin to what Berners-Lee was talking about when passing the rules along with the knowledge. Smart contracts act as a conditional mechanism to make something happen that is fully transparent and separate from any intermediary to fulfill a transaction. Smart contracts rely on nodes to validate the data of a wallet account and pass in the parameters to provide the output of a smart contract condition. Essentially, smart contracts are codes that reflect the logic of the transaction two parties are executing. Every interaction with a smart contract is then recorded in a ‘block’ on the ‘chain’ by the node executing the contract \cite{7}. These would act as the access points of knowledge sharing permission or a point of passing the universal logic.
	These concepts have contributed to the early landscape of Web3 and are critical cogs to a functioning network of decentralized data points, knowledge sharing, and registered owners.

\subsection{Virtual Reality Systems}
Virtual reality has evolved beyond games today, and people use it for art, tourism, and industrial purposes \cite{v}. Ticketing or fees must be paid for this purpose, and a blockchain is an excellent solution for completely secure payments \cite{v,g}. Blockchains provide a distributed cryptographic ledger that cannot be trusted. There is no need for a trusted authority to verify a party's interaction \cite{a8,a9}. Depending on how users log in and their access rights, there are different types of blockchain, including private, public, and consortium. Generally, blockchain-based virtual reality platforms use a blockchain for supporting Web3 so that anyone can easily participate. Virtual reality in the blockchain process involves hashing all the information the user provides, then encrypting the private key on the user's device and generating a digital signature. On a peer-to-peer (P2P) network, transaction data is sent to peers along with a digital signature. Using the public key, network members decrypt the device by comparing its hash against the hash of the transaction data.  Due to the widespread use of Web3 technology, concerns have been raised about information theft, hacking, dissemination, and copying. Consequently, many companies are using virtual reality, because they are unable to limit supply. To support virtual reality in the middle of Web3, a blockchain is an ideal solution for creating decentralization, security, and transparency \cite{a10,a11}. The data recorded in the virtual reality system cannot be altered, and its use can be easily traced. As a result of utilizing blockchain technology, security is increased, and the user experience is improved. Virtual reality security and trust can be improved by utilizing blockchain.

\section{A Web3 Developer}
	Web3 as a concept has created a market for developers with skillsets that can enable and manage the core functions of knowledge and data-sharing-centric network. With Web3 being tied heavily to the current cryptocurrency infrastructure that is aligned with the core principles of Web3, there has been a large move of users contributing to the most significant public GitHub repositories involving the Web3 development stack, according to Richard MacManus at the New Stack \cite{8} due to the interest stirred by the recent popularity of cryptocurrency. This population of around 18,000 developers is likely an underestimate of the active base, as there is no information on the true amount of private proprietary Web3 developers. Consensys, the makers of popular Web3 development tools and services, reported 350,000 active developers using its Infura blockchain development platform in November 2021 \cite{9}. By April 2022 – the time of the creation of this paper – this number had grown to 430,000 active developers \cite{10}. According to the New Stack, this figure is a “drop in the bucket,” with the likelihood of this population growing more in the coming years as this market has the potential to rapidly expand if the market for decentralized and interconnected applications becomes a wider norm for software and technology companies.
	In this market, a Web3 developer does not look vastly different from other developers, with the difference being experienced in the core concepts of blockchain, digital wallets, smart contracts, and the Web3 libraries mostly written in JavaScript and Web3 service providers. Familiarity with the core libraries is important in creating the core functions of a Web3 app which is no different from other fields of software development and relies on knowledge of some of the most popular languages to create this interaction, namely JavaScript \cite{5}. The one exception to this assertion is in the area of smart contracts, which are usually written in a specialized language in order to optimize performance. Some of these languages are Solidity, Rust, Yul, Vyper, and JavaScript. This area is also not restricted in any sense, with open-source libraries available to whoever wants to interact and use them. Web3 development looks to involve more of a fundamental mindset shift in how applications and functions need to interact and the creation of readable scripts that other applications can tap into as portable nodes rather than querying a central server, very similar to querying cloud services. The ability to communicate with the recording system is essential in being part of this Web3 environment because all interactions across the environment rely on an audit log with specific conditions that need to be passed, sometimes referred to as a ledger – the blockchain. Experience with data security is also a key trait for Web3 developers based on the general infrastructure that Web3 is founded on, passing data around transactions through ledgers and conditional function codes known as smart contracts. This interaction is different from most other central-based systems, where logging is always a reference point back to internal systems through various inputs rather than a wider network of checks that are highly portable and accessible via a blockchain network. Security on a blockchain is enforced at every entry and execution point on the chain. If you do not meet the security requirements of the blockchain, your application will not be able to execute smart contracts or write to the blockchain. The governing body of the blockchain sets the security standards, or the DAO – Decentralized Autonomous Organization – their published bylaws.
	Fundamentally, Web3 developers look very much like other software developers today but work with specialized concepts and tools specific to blockchain technologies. Demand will continue to grow for Web3 developers. The population of around 430,000 will also enable these first-comers to make potentially ground-breaking contributions to the space and push Web3 forward. The market will continue to grow for some time, and the developers looking for new challenges and the likes of the ’do-gooding’ developers that helped push Web2 are most likely going to join in the coming years. The continual fascination with blockchain as a technology and its applications will also be a part of this growing market with estimates as high 34.1\% CAGR (compounded annual growth rates) over the next 10 years \cite{11}. With blockchain being closely tied to Web3 as an enabler, this will create more developers capable of pushing the Web3 principles in other areas outside the core areas right now of banking, cybersecurity, and cryptocurrency. Developers wishing to explore a shift in their careers can find many helpful free resources, from job boards \cite{12} to guides to learning Web3 technologies \cite{13,14}.
	\section{Web3 Impact}
	In order to understand Web3, we must first understand the place of the web in the network architecture, as shown in Fig \ref{2} In this way, real-world applications can help us better understand web functionality.
	   \begin{figure}
    \centering
    \includegraphics[width=8cm]{w1.png}
    \caption{The place of web in the network architecture }
    \label{2}
\end{figure}

	Web3 aims to be an evolution into an open, permissionless, decentralized environment for the Internet that will have widened the scope of knowledge and data accessibility and fundamentally shifted the economics upon adoption \cite{15}. Whether this adoption is contingent on technology or ideology, these principles will hold true on the overall impact in both the market and the internet ecosystem. The core impact will be on how computers will interact with other computers for the benefit of users and who owns the outcome. This will create smarter searches and better application of smart technology to put the ownership of the outcomes in the hands of the users and creators rather than the intermediaries of Web2.
	Web3 will be structured as open-source code, not reliant on the big middlemen that enable accessing of content today, such as Google, Microsoft, and Facebook as examples \cite{15}. That is until Big Tech creates their own blockchains, makes them public, and incentivizes developers with airdrops backed by the company's stock to use their tools and services, making the developers owners of the company. The development of Web3 has become transparent to allow software developers to utilize the growing library of publicly accessible Web3 packages and libraries to build the ecosystem as soon as information becomes available, accelerating the velocity of evolution.
	This availability of the infrastructure will enable users to create this Web3 ecosystem. A core piece of this idea is around permissionless technology where interactions are not reliant on trusted third parties to connect users, whether through a search engine or transaction machine \cite{15}. The goal will be to make this feasible by "blockchain-like" infrastructure, if not the same libraries that blockchains use today. The structure acts as its intermediaries, so peer computers can cut out the requirement of an intermediary by using universal logic and conditions to have transferable data ledgers to track transactions and share knowledge based on rules within the node through smart contracts. This helps to protect against the concern of data theft by the current infrastructure of Web2 that enables intermediaries to capture your data and use it for their purposes, often monetizing it in ways the creators of the data might not agree with. Allowing the rules and conditions to be transferable and available without going through an intermediary can also increase execution speed, transparency, and equitable ownership.
	The reliance on large sole sources in the form of data warehouses that maintain knowledge is most likely to be disrupted by the principles of knowledge within Web3. Adopting Web3 as the infrastructure creates more areas where data is stored, which mitigate major data-loss threats, keeps data behind rules not maintained by one guardian, and keeps data from being hidden and controlled by middlemen that disperse the data not necessarily to the benefit of the user base \cite{15}. Decentralization of data storage is a core feature of Web3 that will allow users to create more content and follow a universal logic to share information rather than conforming to an intermediary that acts as a gatekeeper that pushes a framework requirement to best share out the work \cite{15}. Lessening the effort required to create new content accessible can make the whole process of knowledge sharing and value creation a lot faster than it is today through the use of Web3 principles and technology.
\section{Web3 Risks}
	With the emergence of this new Web3 ecosystem which is projected to have a large swath of influence, there are some inherent risks based on the core infrastructure of Web3 and blockchain technology as a whole \cite{16}.
\subsection{Lack of Regulation and Oversight}
	Today, blockchain technology interfaces within the Web3 landscape are essentially unregulated, with a general lack of understanding by most regulators. There are no written laws or advisory boards overseeing how this ecosystem operates, which poses a risk of bad actors and bad faith interactions occurring. This open space of operation has regulatory boards struggling to define even the structure of what are ’entities’ within this operation, such as how even to define what a “blockchain-enabled organization” is in comparison to other companies \cite{17}. Blockchains are operated by Decentralized Autonomous Organizations (DAO’s) with varying and non-standard governance models by common business standards \cite{a12}. DAOs are organizations represented by rules encoded as a transparent computer program, controlled by the organization members, and not influenced by a central government \cite{18}.
	Transactions pose another risk based purely on the global scope of how Web3 can interact, with a central point not existing in any of these ecosystems. For knowledge sharing of particular materials, the jurisdiction of regulation is unclear because it can exist in many different environments and tracking a geographic origin might be impossible. Countries, regions, and governing bodies have different regulations for more traditional knowledge sharing that would be disrupted by how Web3 and blockchain operate without a framework to govern this. With services being provided through a Web3 environment, there are also financial implications in how those are served and to what body. It is unreasonable to believe that because there is not one singular regional source that governing bodies will allow transactions to be monitored/taxed without oversight for a long period of time. This would include how bad faith transactions would work because currently, there is no clear operating system to govern and ensure people are behaving appropriately beyond the bylaws of the DAOs. Traditional ‘scams’, insider trading, and various schemes can be adjudicated through justice systems that are not connected to or cover the framework of Web3’s blockchains. This new business model of promoting peer-to-peer transactions now puts the risks with users instead of what used to be managed by central intermediaries, which is an inherent risk of working with blockchain technology \cite{19}.
	For example, a current blockchain/cryptocurrency scheme that exists (which does not directly tie into Web3 and the semantic web but provides an overview of potential risks) is referred to as a crypto pump and dump, which is parallel to insider trading on big events. An organization will promote a cryptocurrency they may have a large stake in to push the value up considerably before cashing out and typically crashing a currency with new entrants bearing the brunt of the losses. Because most governing bodies do not consider cryptocurrency as a security, let alone a commodity or actual currency, there is no legal precedent for doing anything about this practice that is parallel to an illegal activity \cite{20}.
	As Web3 is adopted and operates for a wider audience, regulation boards should move closer to providing guidelines and direction to ensure the safety of consumers and businesses from bad actors \cite{a13}. This will also be a big pillar of credibility to this area of what exists now as the gray area with few legal definitions.
\subsection{Portability of Illegal Items}
Web3, as professed in the previous material, is focused on allowing knowledge to be shared without a required framework where information can be easily queried and passed along with the rules that define it. The ecosystem now exists within a connected framework where each node houses the same universal information that makes traceability hard. This means bad actors can use these properties to create material that is easily accessible by the people who want to view it and hidden from users who do not want to view it through a major component of Web3 as the smart contract. Illegal media could more easily exist in a Web3 framework due to this decentralization that allows creators to be more inclined to release more of this knowledge and data due to the inherent security provided by a decentralized and conditionally accessed piece of knowledge.
	As a concept, the portability of illegal items might make other internet content creators and providers hesitant to begin the full-on shift based on the services they plan to make available, especially with no plan/guideline in place to protect against these questionable practices that, in parallel mediums are considered illegal \cite{a14}. This gap has a market in waiting, so potentially by the time Web3 is truly ready to be embraced, companies and services could be available to help combat this issue with their own version of a secure smart contract and ledger. This could also be in combination with wider efforts of regulators to put in place guidelines on the operation and have a better understanding of this space operates.

\subsection{Accuracy of Information}
	Now that, in the context of Web3, content is easily transmissible and not contained to a central source, there lies a risk with the rapid movement of disinformation \cite{21,a13i}. The ability to send and receive information fast via a Web3 framework has issues in highlighting good or bad information. Current state, content creators and providers have the opportunity to monitor this information and correct items because it is sourced in their own central area. Within a Web3 framework, information can pass way faster with no restrictions and no oversight. This issue is prevalent today through other semi-decentralized applications such as WhatsApp, which has had rampant issues in combating disinformation in its own encrypted messaging platform \cite{21}.

\subsection{Privacy}
	With security at the forefront as a benefit for the Web3 framework, there is still a vague gap regarding data privacy and how interactions will be saved not only for a user's wallet but also within the network \cite{20}. If an interaction is logged within a ledger and that ledger exists everywhere, what sort of information does the wallet provide to get in, and will users be able to understand those implications? This could mean now intermediaries do not control personal data but potentially their peers. With data being free flowing in this environment, there seems to be larger hurdles to surpass for data privacy, primarily when sources are not centered in regions but exist across networks. A user-focused approach for Web3 empowers users and the internet to be focused on them, but that does not provide them the education on how their information is going to be processed by each unique system, and this especially means regulators will not understand this either\cite{20,a15}.
	Web2 policies around data privacy do not account for this framework and will need to be updated to account for the new capabilities of the framework to process information and store it universally. This alone will put the full onus on service providers to ensure they store this information correctly, which is not sufficient without regulatory oversight to provide clear direction on what that means. There is danger in allowing companies to govern themselves because ensuring that you are constantly doing the right thing does not always fit with the plan, as we can review countless firm practices that have put consumers and users in harm's way.
	\subsection{Future Roadmap}
Our digital world faces new technologies ( e.g., Software Defined Network (SDN) \cite{t}, Network functions virtualization (NFV) \cite{a1}, Artificial intelligence (AI) \cite{Ekramifard2020}, Internet of things (IoT)  \cite{a2}, and Blockchain \cite{a4}) that directly and indirectly affect human life in education, business, industry, and the military. Combined technologies have also improved construction, energy, and resource efficiency \cite{a1,a2}. Web3 \cite{w} is one of today's hottest, most attractive, and most exciting technologies.

	We anticipate that research on Web3 will move towards quantum, IoT, AI, and secure cloud-assisted areas. The reason behind this anticipation is the existence of trends toward the application of security, Quantum, AI, IoT, and secure cloud-assisted environments are interesting areas for academic and industry sections \cite{a17}. Many companies are investing in Web3 applications and building their products to support them.
   \begin{figure*}
    \centering
    \includegraphics[width=18cm]{w4}
    \caption{	Combination of Web3 with other technologies }
    \label{3}
\end{figure*}
Moreover, Fig. \ref{3} summarizes the widespread involvement of AI and  IoT, Quantum with Web3, for the future. Indeed, it shows that inspired AI supported Web3 and the ability to Combination of Web3 with other technologies. It can prove these areas will move toward Web3 quickly.
\subsection{Conclusion}
Fast forward 10 years – all developers are Web3 developers of some form. The definition of "full stack" will evolve. Front-end developers will remain focused on the user experience but may have to deal with additional complexities of data coming from blockchains or sending data for smart contract execution. Back-end developers will still deal with server or cloud application-based logic, but the new complexities of blockchain and smart contract dependencies will require new skill development. Blockchain and smart contract layers will be added to the definition of "full stack", creating new areas of specialization and opportunities. With each blockchain transaction being assigned a very specific value in terms of the transaction itself and the value of the data contained in the blocks, security will become a very acute focus and a specialization unto itself. Web3 security will be focused on the security of the blockchain and the smart contracts, the points where the value is created and stored. Security practices will become ever more integrated into the code of the applications, increasing the requirement of developers to obtain new skills and demonstrate secure coding and development practices. The inherent risks posed by the current state of Web3 technologies create opportunities for growth and value creation. Likely, the jobs boom that occurred as the industry began to understand what Web2 meant in the context of Web1 will be dwarfed by the jobs boom Web3 will generate. The fundamental economic shifts in the ideology of Web3 alone will trigger a tectonic shift we have never seen before. This, coupled with the scarcity of resources the industry expects over the next decade, even without considering the impact of Web3, will provide developers and other roles in software engineering a tremendous opportunity to generate value and wealth \cite{22,a21,23}.

\bibliographystyle{IEEEtran}