\section{Introduction}

The Regularity lemma of Szemer\'edi \cite{Sz} has proved to be a
very useful tool in graph theory. It was initially developed as an
auxiliary lemma to prove a long standing conjecture of Erd\H{o}s and
Tur\'{a}n\cite{ET} on arithmetic progressions, which stated that
sequences of integers with positive upper density must contain
arbitrarily long arithmetic progressions. Now the Regularity Lemma
by itself has become an important tool and found numerous other
applications (see \cite{KSSS}). Based on the Regularity Lemma and
the Blow-up Lemma \cite{KSS5} the Regularity method has been
developed that has been quite successful in a number of applications
in graph theory (e.g. \cite{GRSS1}). However, one major disadvantage
of these applications and the Regularity Lemma is that they are
mainly theoretical, they work only for astronomically large graphs
as the Regularity Lemma can be applied only for such large graphs.
Indeed, to find the $\varepsilon$-regular partition in the
Regularity Lemma, the number of vertices must be a tower of 2's with
height proportional to $\varepsilon^{-5}$. Furthermore, Gowers
demonstrated \cite{G} that a tower bound is necessary.

The basic content of the Regularity Lemma could be described by
saying that every graph can, in some sense, be partitioned into
random graphs. Since random graphs of a given edge density are much
easier to treat than all graphs of the same edge-density, the
Regularity Lemma helps us to carry over results that are trivial for
random graphs to the class of all graphs with a given number of
edges. We are especially interested in harnessing the power of the
Regularity Lemma for clustering data. Graph partitioning methods for
clustering and segmentation have become quite popular in the past
decade because of representative ease of data with graphs and the
strong theoretical underpinnings that accompany the same.

In this paper we propose a general methodology to make the
Regularity Lemma more useful in practice. To make it truly
applicable, instead of constructing a provably regular partition we
construct an {\em approximately} regular partition. This partition
behaves just like a regular partition (especially for graphs
appearing in practice) and yet it does not require the large number
of vertices as mandated by the original Regularity Lemma. Then this
approximately regular partition is used for performing clustering.
We call the resulting new clustering technique {\em Regularity
clustering}. We present comparisons with standard clustering methods
such as $k$-means and spectral clustering and the results are very
encouraging.

To present our attempt and the results obtained, the paper is
organized as follows: In section \ref{prior} we discuss briefly some
prior attempts to apply the Regularity Lemma in practical settings
and place our work in contrast to those. In Section \ref{clust} we
discuss clustering in general and also present a popular spectral
clustering algorithm that is used later on the reduced graph. We
also point out what are the possible ways to improve its running
time. In Section \ref{not} we give some definitions and general
notation. In Section \ref{reg} we present two constructive versions
of the Regularity Lemma (the original lemma was non-constructive).
Furthermore, in this section we point out the various problems
arising when we attempt to apply the lemma in real-world
applications. In Section \ref{mod} we discuss how the constructive
Regularity Lemmas could be modified to make them truly applicable
for real-world problems where the graphs typically are much smaller,
say have a few thousand vertices only. In Section \ref{app} we show
how this Practical Regularity partitioning algorithm can be applied
to develop a new clustering technique. In Section \ref{test}, we
present an extensive empirical validation of our method. Section
\ref{future} is spent in discussing the various possible future
directions of work.

\section{Prior Applications of the Regularity Lemma}\label{prior}

As we discussed above so far the Regularity Lemma has been ``well
beyond the realms of any practical applications" \cite{Abel}, the
existing applications have been theoretical, {\em mathematical}. The
only practical application attempt of the Regularity Lemma to the
best of our knowledge is by Sperotto and Pelillo \cite{SP}, where
they use the Regularity Lemma as a pre-processing step. They give
some interesting ideas on how the Regularity Lemma might be used,
however they do not give too many details. Taking leads from some of
their ideas we give a much more thorough analysis of the
modifications needed in order to make the lemma applicable in
practice. Furthermore, they only give results for using the
constructive version by Alon {\em et al} \cite{AD}, here we
implement the version proposed by Frieze and Kannan \cite{FK} as
well. We also give a far more extensive empirical validation; we use
12 datasets instead of 3.

\section{Clustering}\label{clust}

Out of the various modern clustering techniques, spectral clustering
has become one of the most popular. This has happened due to not
only its superior performance over the traditional clustering
techniques, but also due to the strong theoretical underpinnings in
spectral graph theory and its ease of implementation. It has many
advantages over the more traditional clustering methods such as
$k$-means and expectation maximization (EM). The most important is
its ability to handle datasets that have arbitrary shaped clusters.
Methods such as $k$-means and EM are based on estimating explicit
models of the data. Such methods fail spectacularly when the data is
organized in very irregular and complex clusters. Spectral
clustering on the other hand does not work by estimating explicit
models of the data but does so by analysing the spectrum of the
Graph Laplacian. This is useful as the top few eigenvectors can
unfold the data manifold to form meaningful clusters.

In this work we employ spectral clustering on the reduced graph (which is an essence
of the original graph),
even though any other pairwise clustering method could be used. The
algorithm that we employ is due to Ng, Jordan and Weiss \cite{NJW}.
Despite various advantages of spectral clustering, one major problem
is that for large datasets it is very computationally intensive. And
understandably this has received a lot of attention recently. As originally stated,
the spectral clustering pipeline has two main bottlenecks: First, computing the
affinity matrix of the pairwise distances between datapoints, and second, once we have the
affinity matrix the finding of the eigendecomposition. Many ways have been
suggested to solve these problems more efficiently. One approach is not to
use an all-connected graph but a k-nearest neighbour graph in which
each data point is typically connected to $\log{n}$ neighboring
datapoints(where $n$ is the number of data-points). This
considerably speeds up the process of finding the affinity matrix,
however it has a drawback that by taking nearest neighbors we might
miss something interesting in the global structure of the data. A
method to remedy this is the Nystr\"{o}m method which takes a random
sample of the entire dataset (thus preserving the global structure
in a sense) and then doing spectral clustering on this much smaller
sample. The results are then extended to all other points in the
data set \cite{FowlChung}.

Our work is quite different from such methods. The speed-up is
primarily in the second stage where eigendecomposition is to be
done. The original graph is represented by a reduced graph which is
much smaller and hence eigendecomposition of this reduced graph can significantly
ease the computational load. Further work on a practical variant
of the sparse Regularity Lemma could be useful in a speed-up in the
first stage, too.

\section{Notation and Definitions}\label{not}

Below we introduce some notation and definitions for describing the
Regularity Lemma and our methodology.

Let $G = (V,E)$ denote a graph, where $V$ is the set of vertices and
$E$ is the set of edges. When $A, B$ are disjoint subsets of $V$,
the number of edges with one endpoint in $A$ and the other in $B$ is
denoted by $e(A,B)$. When $A$ and $B$ are nonempty, we define the
{\em density} of edges between $A$ and $B$ as $d(A,B) =
\frac{e(A,B)}{|A||B|}$. The most important concept is the following.

\begin{definition}\hspace{-6pt}{\bf.\ }
The bipartite graph $G=(A,B,E)$ is $\varepsilon$-{\em regular} if
for every $X\subset A$, $Y\subset B$ satisfying: $\
|X|>\varepsilon|A|,\ |Y|>\varepsilon|B|,$ we have
$|d(X,Y)-d(A,B)|<\varepsilon,$ otherwise it is $\varepsilon$-{\em
irregular}.
\end{definition}

\noindent Roughly speaking this means that in an $\varepsilon$-regular bipartite
graph the edge density between {\em any} two relatively large
subsets is about the same as the original edge density. In effect
this implies that all the edges are distributed almost uniformly.

\begin{definition}\hspace{-6pt}{\bf.\ }\label{defn}
A partition $P$ of the vertex set $V=V_0\cup V_1\cup \ldots \cup
V_k$ of a graph $G = (V,E)$ is called an {\em equitable partition}
if all the classes $V_i, 1\leq i\leq k$, have the same cardinality.
$V_0$ is called the exceptional class.
\end{definition}

\begin{definition}\hspace{-6pt}{\bf.\ }\label{potential}
For an equitable partition $P$ of the vertex set $V=V_0\cup V_1\cup
\ldots \cup V_k$ of $G = (V,E)$, we associate a measure called the
{\em index} of $P$ (or the potential) which is defined by
$$ind(P) = \frac{1}{k^2}\sum_{s=1}^k \sum_{t=s+1}^k d(C_s,C_t)^2.$$
\end{definition}
This will measure the progress towards an $\varepsilon$-regular partition.

\begin{definition}\hspace{-6pt}{\bf.\ }
An equitable partition $P$ of the vertex set $V=V_0\cup V_1\cup
\ldots \cup V_k$ of $G = (V,E)$ is called $\varepsilon$-{\em
regular} if $|V_0| < \varepsilon |V| $ and all but $\varepsilon k^2$
of the pairs $(V_i,V_j)$ are $\varepsilon$-regular where $1 \leq i <
j \leq k$.
\end{definition}
With these definitions we are now in a position to state the
Regularity Lemma.

\section{The Regularity Lemma}\label{reg}

\begin{theorem}[Regularity Lemma \cite{Sz}]\hspace{-6pt}{\bf.\ }\label{p1}
For every positive $\varepsilon > 0$ and positive integer $t$ there is an
integer $T = T(\varepsilon,t)$ such that every graph with $n > T$ vertices
has an $\varepsilon$-regular partition into $k+1$ classes, where $t \leq k
\leq T$.
\end{theorem}

In applications of the Regularity Lemma the concept of the {\em
reduced graph} plays an important role.

\begin{definition}\hspace{-6pt}{\bf.\ }\label{reduced}
Given an $\varepsilon$-regular partition of a graph $G = (V, E)$ as
provided by Theorem \ref{p1}, we define the {\em reduced graph}
$G^R$ as follows. The vertices of $G^R$ are associated to the
classes in the partition and the edges are associated to the
$\varepsilon$-regular pairs between classes with density above $d$.
\end{definition}

The most important property of the reduced graph is that many
properties of $G$ are inherited by $G^R$. Thus $G^R$ can be treated
as a representation of the original graph $G$ albeit with a much
smaller size, an ``essence'' of $G$. Then if we run any algorithm on
$G^R$ instead of $G$ we get a significant speed-up.

\subsection{Algorithmic Versions of the Regularity Lemma} \label{RegAlgo}

The original proof of the Regularity Lemma \cite{Sz} does not give a
method to construct a regular partition but only shows that one must
exist. To apply the Regularity Lemma in practical settings, we need
a constructive version. Alon {\em et al.} \cite{AD} were the first
to give an algorithmic version. Since then a few other algorithmic
versions have also been proposed \cite{FK}, \cite{KRT}. Below we
present the details of the Alon {\em et al.} algorithm.

\subsubsection{Alon {\em et al.} Version}

\begin{theorem}[Algorithmic Regularity Lemma \cite{AD}]\hspace{-6pt}{\bf.\ }\label{al}
For every $\varepsilon > 0$ and every positive integer $t$ there is an
integer $T = T(\varepsilon, t)$ such that every graph with $n > T$ vertices
has an $\varepsilon$-regular partition into $k + 1$ classes, where $t \le k
\le T$. For every fixed $\varepsilon > 0$ and $t \ge 1$ such a partition
can be found in $O(M(n))$ sequential time, where $M(n)$ is the time
for multiplying two $n$ by $n$ matrices with $0, 1$ entries over the
integers. The algorithm can be parallelized and implemented in
$NC^1$.
\end{theorem}

This result is somewhat surprising from a computational complexity
point of view since as it was proved in \cite{AD} that the
corresponding decision problem (checking whether a given partition
is $\varepsilon$-regular) is co-NP-complete. Thus the search problem
is easier than the decision problem. To describe this algorithm, we
need a couple of lemmas.

\begin{lemma}[Alon {\em et al.} \cite{AD}]\hspace{-6pt}{\bf.\ }\label{lnew1}
Let $H$ be a bipartite graph with equally sized classes $|A| = |B| =
n$. Let $2n^{-1/4} < \varepsilon <\frac{1}{16}$. There is an $O(M(n))$
algorithm that verifies that $H$ is $\varepsilon$-regular or finds two
subset $A' \subset A$, $B' \subset B$, $|A'| \ge
\frac{{\varepsilon}^4}{16}n$, $|B'| \ge \frac{{\varepsilon}^4}{16}n$, such that
$|d(A, B) - d(A', B')| \ge \varepsilon^4$. The algorithm can be
parallelized and implemented in $NC^1$.
\end{lemma}

This lemma basically says that we can either verify that the pair is
$\varepsilon$-regular or we provide certificates that it is not. The
certificates are the subsets $A', B'$ and they help to proceed to
the next step in the algorithm. The next lemma describes the
procedure to do the refinement from these certificates.

\begin{lemma}[Szemer\'edi \cite{Sz}]\hspace{-6pt}{\bf.\ }\label{lnew2}
Let $G = (V,E)$ be a graph with $n$ vertices. Let $P$ be an
equitable partition of the vertex set $V=V_0\cup V_1\cup \ldots \cup
V_k$. Let $\gamma >0$ and let $k$ be a positive integer such that
$4^k > 600\gamma^{-5}$. If more than $\gamma k^2$ pairs $(V_s,
V_t)$, $1 \le s < t \le k$, are $\gamma$-irregular then there is an
equitable partition $Q$ of $V$ into $1 + k4^k$ classes, with the
cardinality of the exceptional class being at most $|V_0| +
\frac{n}{4^k}$ and such that $ind(Q) > ind(P) + \frac{\gamma^5}{20}.$
\end{lemma}

This lemma implies that whenever we have a partition that is not
$\gamma$-regular, we can refine it into a new partition which has a
better index (or potential) than the previous partition. The
refinement procedure to do this is described below.

{\bf Refinement Algorithm:} {\em Given a $\gamma$-irregular
equitable partition $P$ of the vertex set $V=V_0\cup V_1\cup \ldots
\cup V_k$ with $\gamma = \frac{\varepsilon^4}{16}$, construct a new partition $Q$.\\
For each pair $(V_s, V_t)$, $1 \leq s < t \leq k$, we apply Lemma
\ref{lnew1} with $A=V_s$, $B=V_t$ and $\varepsilon$. If $(V_s, V_t)$
is found to be $\varepsilon$-regular we do nothing. Otherwise, the
certificates partition $V_s$ and $V_t$ into two parts (namely the
certificate and the complement). For a fixed $s$ we do this for all
$t\not= s$. In $V_s$, these sets define the obvious equivalence
relation with at most $2^{k-1}$ classes, namely two elements are
equivalent if they lie in the same partition part for every $t\not=
s$. The equivalence classes will be called atoms. Set $m = \lfloor
\frac{|V_i|}{4^k}\rfloor$, $1 \le i \le k$. Then we construct our
new partition $Q$ by choosing a maximal collection of pairwise
disjoint subsets of $V$ such that every subset has cardinality $m$
and every atom $A$ contains exactly $\lfloor \frac{|A|}{m}\rfloor$
subsets; all other vertices are put in the exceptional class. The
collection $Q$ is an equitable partition of $V$ into at most
$1+k4^k$ classes and the cardinality of its exceptional class is at
most $|V_0| + \frac{n}{4^k}$. }

Now we are ready to present the main algorithm.

{\bf Regular Partitioning Algorithm:} {\em Given a graph $G$ and
$\varepsilon$, construct a $\varepsilon$-regular partition.
\begin{enumerate}
\item {\bf Initial partition:} Arbitrarily divide the vertices of $G$ into an equitable partition $P_1$ with classes $V_0, V_1, \ldots, V_b$, where
$|V_1| = \lfloor \frac{n}{b} \rfloor$ and hence $|V_0| < b$. Denote
$k_1 = b$.
\item {\bf Check regularity:} For every pair $(V_s,V_t)$ of $P_i$, verify if it is $\varepsilon$-regular or find
$X \subset V_s, Y \subset V_t, |X| \ge \frac{\varepsilon^4}{16}|V_s|,|Y|
\ge \frac{\varepsilon^4}{16}|V_t|$, such that $|d(X,Y) - d(V_s,V_t)| \ge
\varepsilon^4$.
\item {\bf Count regular pairs:} If there are at most $\varepsilon k_i^2$ pairs that are not verified as $\varepsilon$-regular, then halt. $P_i$ is an $\varepsilon$-regular partition.
\item {\bf Refinement:} Otherwise apply the Refinement Algorithm and
Lemma \ref{lnew2}, where $P = P_i, k = k_i, \gamma =
\frac{\varepsilon^4}{16}$, and obtain a partition $Q$ with $1 +
k_i4^{k_i}$ classes.
\item {\bf Iteration:} Let $k_{i+1} = k_i4^k_i, P_{i+1} = Q, i = i+1$, and go to step 2.
\end{enumerate}
}

Since the index cannot exceed $1/2$, the algorithm must halt after
at most $\lceil 10\gamma^{-5} \rceil$ iterations (see \cite{AD}).
Unfortunately, in each iteration the number of classes increases to
$k4^k$ from $k$. This implies that the graph $G$ must be indeed
astronomically large (a tower function) to ensure the completion of
this procedure. As mentioned before, Gowers \cite{G} proved that
indeed this tower function is necessary in order to guarantee an
$\varepsilon$-regular partition for {\em all} graphs. The size requirement
of the algorithm above makes it impractical for real world
situations where the number of vertices typically is a few thousand.

\subsubsection{Frieze-Kannan Version}

The Frieze-Kannan constructive version is quite similar to the
above, the only difference is how to check regularity of the pairs
in Step 2. Instead of Lemma \ref{lnew1}, another lemma is used based
on the computation of singular values of matrices. For the sake of
completeness we present the details.

\begin{lemma}[Frieze-Kannan \cite{FK}]\hspace{-6pt}{\bf.\ }\label{singularFK}
Let $W$ be an $R \times C$ matrix with $|R| = p$ and $|C| = q$ and
$W_\infty\leq 1$ and $\gamma$ be a positive real.

\begin{enumerate}
\item[a] If there exist $S \subseteq R, T \subseteq C $ such that $|S| \geq \gamma p, |T| \geq \gamma q$ and $|W(S,T)| \geq \gamma |S| |T|$
then $\sigma_1(W) \geq \gamma^3 \sqrt{pq}$ (where $\sigma_1$ is the
first singular value).
\item[b] If $\sigma_1(W) \geq \gamma \sqrt{pq}$ then there exist  $S \subseteq R, T \subseteq C $ such that $|S| \geq \gamma'p, |T| \geq \gamma'q$
and $W(S,T) \geq \gamma' |S| |T|$, where $\gamma' =
\frac{\gamma^3}{108}$. Furthermorem $S,T$ can be constructed in
polynomial time.
\end{enumerate}

\end{lemma}

Combining Lemmas \ref{lnew2} and \ref{singularFK}, we get an
algorithm for finding an $\varepsilon$-regular partition, quite
similar to the Alon {\em et al.} version \cite{AD}, which we present
below:

{\bf Regular Partitioning Algorithm (Frieze-Kannan):} {\em Given a
graph $G$ and $\varepsilon}\def\ffi{\varphi$, construct a $\varepsilon}\def\ffi{\varphi$-regular partition.
\begin{enumerate}
\item {\bf Initial partition:} Arbitrarily divide the vertices of $G$ into an equitable partition $P_1$ with classes $V_0, V_1, \ldots, V_b$, where
$|V_1| = \lfloor \frac{n}{b} \rfloor$ and hence $|V_0| < b$. Denote
$k_1 = b$.
\item {\bf Check regularity:} For every pair $(V_s,V_t)$ of $P_i$, compute $\sigma_1(W_{r,s})$. If the pair $(V_r, V_s)$ are not $\varepsilon$-regular
then by Lemma \ref{singularFK} we obtain a proof that they are not
not $\gamma = \varepsilon^9/108$-regular.
\item {\bf Count regular pairs:} If there are at most $\varepsilon}\def\ffi{\varphi k_i^2$ pairs that produce proofs of non $\gamma$-regularity, then halt. $P_i$ is an $\varepsilon}\def\ffi{\varphi$-regular partition.
\item {\bf Refinement:} Otherwise apply the Refinement Algorithm and
Lemma \ref{lnew2}, where $P = P_i, k = k_i, \gamma =
\frac{\varepsilon}\def\ffi{\varphi^9}{108}$, and obtain a partition $P'$ with $1 +
k_i4^{k_i}$ classes.
\item {\bf Iteration:} Let $k_{i+1} = k_i4^{k_i}, P_{i+1} = P', i = i+1$, and go to step 2.
\end{enumerate}
}

This algorithm is guaranteed to finish in at most
$\varepsilon^{-45}$ steps with an $\varepsilon$-regular partition.

\section{Modifications to the Constructive Version}\label{mod}

We see that even the constructive versions are not directly
applicable to real world scenarios. We note that the above
algorithms have such restrictions because their aim is to be
applicable to {\em all} graphs. Thus, to make the regularity lemma
truly applicable we would have to give up our goal that the lemma
should work for {\em every} graph and should be content with the
fact that it works for {\em most} graphs. To ensure that this
happens, we modify the Regular Partitioning Algorithm(s) so that
instead of constructing a regular partition, we find an {\em
approximately} regular partition, which should be much easier to
construct. We have the following 3 major modifications to the
Regular Partitioning Algorithm.

{\bf Modification 1:} We want to decrease the cardinality of atoms
in each iteration. In the above Refinement Algorithm the cardinality
of the atoms may be $2^{k-1}$, where $k$ is the number of classes in
the current partition. This is because the algorithm tries to find
all the possible $\varepsilon$-irregular pairs such that this
information can then be embedded into the subsequent refinement
procedure. Hence potentially each class may be involved with up to
$(k-1)$ $\varepsilon$-irregular pairs. One way to avoid this problem
is to bound this number. To do so, instead of using all the
$\varepsilon$-irregular pairs, we only use some of them.
Specifically, in this paper, for each class we consider at most one
$\varepsilon$-irregular pair that involves the given class. By doing
this we reduce the number of atoms to at most $2$. We observe that
in spite of the crude approximation, this seems to work well in
practice.

{\bf Modification 2:} We want to bound the rate by which the class
size decreases in each iteration. As we have at most $2$ atoms for
each class, we could significantly increase $m$ used in the
Refinement Algorithm as $m = \frac{|V_i|}{l}$, where a typical value
of $l$ could be $3$ or $4$, much smaller than $4^k$. We call this
user defined parameter $l$ the refinement number.

{\bf Modification 3:} Modification 2 might cause the size of the
exceptional class to increase too fast. Indeed, by using a smaller
$l$, we risk putting $\frac{1}{l}$ portion of all vertices into
$V_0$ after each iteration. To overcome this drawback, we
``recycle'' most of $V_0$, i.e. we move back most of the vertices
from $V_0$. Here is the modified Refinement Algorithm.

{\bf Modified Refinement Algorithm:} {\em Given a $\gamma$-irregular
equitable partition $P$ of the vertex set $V=V_0\cup V_1\cup \ldots
\cup V_k$ with $\gamma = \frac{\varepsilon^4}{16}$ and refinement
number $l$, construct a new partition $Q$.\\
For each pair $(V_s, V_t)$, $1 \leq s < t \leq k$, we apply Lemma
\ref{lnew1} with $A=V_s$, $B=V_t$ and $\varepsilon$. For a fixed $s$
if $(V_s, V_t)$ is found to be $\varepsilon$-regular for all $t\not=
s$ we do nothing, i.e. $V_s$ is one atom. Otherwise, we select one
$\varepsilon$-irregular pair $(V_s, V_t)$ randomly and the
corresponding certificate partitions $V_s$ into two atoms. Set $m =
\lfloor \frac{|V_i|}{l}\rfloor$, $1 \le i \le k$. Then first we
choose a maximal collection $Q'$ of pairwise disjoint subsets of $V$
such that every member of $Q'$ has cardinality $m$ and every atom
$A$ contains exactly $\lfloor \frac{|A|}{m}\rfloor$ members of $Q'$.
Then we unite the leftover vertices in each $V_s$, if there are at
least $m$ vertices then we select one more subset of size $m$ from
these vertices, we add these sets to $Q'$ and finally we add all
remaining vertices to the exceptional class, resulting in the
partition $Q$. The collection $Q$ is an equitable partition of $V$
into at most $1+lk$ classes. }

\noindent Now, we present our modified Regular Partitioning
Algorithm. There are three main parameters to be selected by the
user: $\varepsilon$, the refinement number $l$ and $h$, the minimum
class size when we must halt the refinement procedure. The parameter
$h$ is used to ensure that if the class size has gone too small then
the procedure should not continue.

{\bf Modified Regular Partitioning Algorithm (or the Practical
Regularity Partitioning Algorithm):} {\em Given a graph $G$ and
parameters $\varepsilon$, $l$, $h$, construct an approx.
$\varepsilon$-regular partition.
\begin{enumerate}
\item {\bf Initial partition:} Arbitrarily divide the vertices of $G$ into an equitable partition $P_1$ with classes $V_0, V_1, \ldots, V_l$, where
$|V_1| = \lfloor \frac{n}{l} \rfloor$ and hence $|V_0| < l$. Denote
$k_1 = l$.
\item {\bf Check size and regularity:} If $|V_i| < h$, $1\leq i \leq k$, then halt.
Otherwise for every pair $(V_s,V_t)$ of $P_i$, verify if it is $\varepsilon$-regular or find
$X \subset V_s, Y \subset V_t, |X| \ge \frac{\varepsilon^4}{16}|V_s|,|Y|
\ge \frac{\varepsilon^4}{16}|V_t|$, such that $|d(X,Y) - d(V_s,V_t)| \ge
\varepsilon^4$.
\item {\bf Count regular pairs:} If there are at most $\varepsilon k_i^2$ pairs that are not verified as $\varepsilon$-regular, then halt. $P_i$ is an $\varepsilon$-regular partition.
\item {\bf Refinement:} Otherwise apply the Modified Refinement Algorithm,
where $P = P_i, k = k_i, \gamma = \frac{\varepsilon^4}{16}$, and obtain a partition $Q$ with $1 + lk_i$ classes.
\item {\bf Iteration:} Let $k_{i+1} = l k_i, P_{i+1} = Q, i = i+1$, and go to step 2.
\end{enumerate}
}

The Frieze-Kannan version is modified in an identical way.

\section{Application to Clustering}\label{app}

To make the regularity lemma applicable in clustering settings, we
adopt the following two phase strategy (as in \cite{SP}):
\begin{enumerate}
\item {\bf Application of the Practical Regularity Partitioning Algorithm:} In the first
stage we apply the Practical Regularity partitioning algorithm as
described in the previous section to obtain an approximately regular
partition of the graph representing the data. Once such a partition
has been obtained, the reduced graph as described in Definition
\ref{reduced} could be constructed from the partition.

\item {\bf Clustering the Reduced Graph:} The reduced graph as constructed above
would preserve most of the properties of the original graph (see
\cite{KSSS}). This implies that any changes made in the reduced
graph would also reflect in the original graph. Thus, clustering the
reduced graph would also yield a clustering of the original graph.
We apply spectral clustering (though any other pairwise clustering
technique could be used, e.g. in \cite{SP} the dominant-set
algorithm is used) on the reduced graph to get a partitioning and
then project it back to the higher dimension. Recall that vertices
in the exceptional set $V_0$ are leftovers from the refinement
process and must be assigned to the clusters obtained. Thus in the
end these leftover vertices are redistributed amongst the clusters
using a k-nearest neighbor classifier to get the final grouping.

\end{enumerate}

\section{Empirical Validation}\label{test}
In this section we present extensive experimental results to
indicate the efficacy of this approach by employing it for
clustering on a number of benchmark datasets. We also compare the
results with spectral clustering in terms of accuracy. We also
report results that indicate the amount of compression obtained by
constructing the reduced graph. As discussed later, the results also
directly point to a number of promising directions of future work.
We first review the datasets considered and the metrics used for
comparisons.

\subsection{Datasets and Metrics Used} \label{datasets}
The datasets considered for empirical validation were taken from the
University of California, Irvine machine learning repository
\cite{UCI}. A total of 12 datasets were used for validation. We
considered datasets with real valued features and associated labels
or ground truth. In some datasets (as described below) that had a
large number of real valued features, we removed categorical
features to make it easier to cluster. Unless otherwise mentioned,
the number of clusters was chosen so as to equal to the number of
classes in the dataset (i.e. if the number of classes in the ground
truth is 4, then the clustering results are for k = 4). An attempt
was made to pick a wide variety of datasets, i.e. with integer
features, binary features, synthetic datasets and of course real
world datasets with both very high and small dimensionality.

The following datasets were considered (for details about the
datasets see \cite{UCI}): (1) Red Wine (R-Wine) and (2) White Wine
(W-Wine), (3) The Arcene dataset (Arcene), (4) The Blood Transfusion
Dataset (Blood-T), (5) The Ionosphere dataset (Ionos), (6) The
Wisconsin Breast cancer dataset (Cancer), (7) The Pima Indian
diabetes dataset (Pima), (8) The Vertebral Column dataset
(Vertebral-1), the second task (9) (Vertebral-2) is considered as
another dataset, (10) The Steel Plates Faults Dataset (Steel), (11)
The Musk 2 (Musk) dataset and (12) Haberman's Survival (Haberman)
data.

Next we discuss the metric used for comparison with other clustering
algorithms. For evaluating the quality of clustering, we follow the
approach of \cite{WuScho} and use the cluster accuracy as a measure.
The measure is defined as: $$\displaystyle Accuracy = 100*
\biggl(\frac{\sum_{i=1}^{n} \delta (y_i, map(c_i))}{n} \biggr ), $$
where $n$ is the number of data-points considered, $y_i$ represents
the true label (ground truth) while $c_i$ is the obtained cluster
label of data-point $x_i$. The function $\delta(y,c)$ equals 1 if
the true and the obtained labels match ($y=c$) and 0 if they don't.
The function $map$ is basically a permutation function that maps
each cluster label to the true label. An optimal match can be found
by using the Hungarian Method for the assignment problem
\cite{Kuhn}.

\subsection{Case Study} \label{casestudy}
Before reporting comparative results on benchmark datasets, we first
consider one dataset as a case study. While experiments reported in
this case study were carried on all the benchmark datasets
considered, the purpose here is to illustrate the investigations
conducted at each stage of application of the regularity lemma. An
auxiliary purpose is also to underline a set of guidelines on what
changes to the practical regularity partitioning algorithm proved to
be useful.

For this task we consider the Red Wine dataset which has 1599
instances with 11 attributes each, the number of classes involved is
six. It must be noted though that the class distribution in this
dataset is pretty skewed (with the various classes having 10, 53,
681, 638, 199 and 18 datapoints respectively), this makes clustering
this dataset quite difficult when k = 6. We however consider both k
= 6 and k = 3 to compare results with spectral clustering.

Recall that our method has two meta-parameters that need to be user
specified (or estimated by cross-validation): $\varepsilon$ and $l$.
Note that $h$ is usually decided so that it is at least as big as
$\frac{1}{\varepsilon}$. The first set of experiments thus explore
the accuracy landscape of regularity clustering spanned over these
two parameters. We consider 25 linearly spaced values of
$\varepsilon$ between 0.15 and 0.50. The refinement number $l$, as
noted in Section \ref{mod}, can not be too large. Since it can only
take integer values, we consider six values from 2 to 7. For the
sake of comparison, we also obtain clustering results on the same
dataset with spectral clustering with self tuning \cite{selftuned}
(both using all connected and k-nearest neighbour graph versions)
and k-means clustering. Figure \ref{fig:CaseStudy} gives the
accuracy of the Regularity Clustering on a grid of $\varepsilon$ and
$l$. Even though this plot is only for exploratory purposes, it
shows that the accuracy landscape is in general much better than the
accuracy obtained by spectral clustering for this dataset.

\begin{figure}
\centering
\includegraphics[width=2in]{AlonK6}%
\hspace{0.5in}%
\includegraphics[width=2in]{AlonK3}
\caption{Accuracy Landscape for Regularity Clustering on the Red Wine Dataset for different values of $\varepsilon$ and refinement size $l$ (with k = 6 on the left and k = 3 on the right). The Plane cutting through in blue represents accuracy by running self-tuned spectral clustering using the fully connected similarity graph.}
\label{fig:CaseStudy}
\end{figure}

An important aspect of the Regularity Clustering method is that by
using a modified constructive version of the Regularity Lemma we
obtain a much reduced representation of the original data. The size
of the reduced graph depends both on $\varepsilon$ and $l$. However,
in our observation it is more sensitive to changes to $l$ and
understandably so. From the grid for $\varepsilon$ and $l$ we take
three rows to illustrate the obtained sizes of the reduced graph
(more precisely, the dimensions of the affinity matrix of the
reduced graph). We compare these numbers with the original dataset
size. As we note in the results over the benchmark datasets in
section \ref{benchmark}, this compression is quite big in larger
datasets.

\begin{table}
\caption{Reduced Graph Sizes. Original Affinity Matrix size : 1599 $\times$ 1599}
\centering

\begin{tabular}{| c | c | c | c | c | c | c |}
\hline \hline
\backslashbox{{\bf $\varepsilon$}}{{\bf $l$}} & {\bf 2} & {\bf 3} & {\bf 4} & {\bf 5} & {\bf 6} & {\bf 7} \\ \hline
{\bf 0.15} & 16 $\times$ 16 & 27 $\times$ 27 & 27 $\times$ 27 & 27 $\times$ 27  & 36 $\times$ 36 & 49 $\times$ 49 \\ \hline
{\bf 0.33} & 49 $\times$ 49 & 49 $\times$ 49 & 66 $\times$ 66 & 66 $\times$ 66  & 66 $\times$ 66 & 66 $\times$ 66 \\ \hline
{\bf 0.50} & 66 $\times$ 66 & 66 $\times$ 66  & 66 $\times$ 66 & 66 $\times$ 66 & 66 $\times$ 66 & 66 $\times$ 66 \\ \hline
\end{tabular}
\label{redgraphsizes}
\end{table}

The proof of the Regularity Lemma is using a potential function, the
index of the partition defined earlier in Definition
\ref{potential}. In each refinement step the index increases
significantly. Surprisingly this remains true in our modified
refinement algorithm when the number of partition classes is not
increasing as fast as in the original version, see Table
\ref{potentialincrease}. Another interesting observation is that if
we take $\varepsilon$ sufficiently high, we do get a regular
partition in just a few iterations. A few examples where this was
noticed in the Red Wine dataset are mentioned in Table
\ref{regpartitions}.

\begin{table}
\caption{Illustration of Increase in Potential}
\centering
\begin{tabular}{| c | c | c | c | c |}
\hline \hline
\backslashbox{{ \bf ($\varepsilon$,$l$)}}{{\bf $ind(P)$}} & {\bf $ind(P_1)$} & {\bf $ind(P_2)$} & {\bf $ind(P_3)$} & {\bf $ind(P_4)$} \\ \hline
{\bf 0.15, 2} & 0.1966 & 0.2892 & 0.3321 & 0.3539  \\ \hline
{\bf 0.33, 2} & 0.1966 & 0.2883 & 0.3321 & 0.3683   \\ \hline
{\bf 0.50, 2} & 0.1965 & 0.2968 & 0.3411 & 0.3657    \\ \hline
\end{tabular}
\label{potentialincrease}
\end{table}

\begin{table}
\caption{Regular Partitions with req. no. of regular pairs and actual no. present}
\centering
\begin{tabular}{| c | c | c | c |}
\hline \hline
($\varepsilon$, $l$) & \# for $\varepsilon$-regularity & \# of Reg. Pairs & \# Iterations \\ \hline
{\bf 0.6, 2} & 1180 & 1293 & 6 \\ \hline
{\bf 0.7, 6} & 352 & 391 & 2 \\ \hline
{\bf 0.7, 7} & 506 & 671  & 2 \\ \hline
\end{tabular}
\label{regpartitions}
\end{table}

Finally, before reporting results we must make a comment on
constructing the reduced graph. The reduced graph was defined in
Definition \ref{reduced}. But note that there is some ambiguity in
our case when it comes to constructing the reduced graph. The
reduced graph $G^R$ is constructed such that the vertices correspond
to the classes in the partition and the edges are associated to the
$\varepsilon$-regular pairs between classes with density above $d$.
However, in many cases the number of regular pairs is quite small
(esp. when $\varepsilon$ is small) making the matrix too sparse,
making it difficult to find the eigenvectors. Thus for technical
reasons we added all pairs to the reduced graph. We contend that
this approach works well because the classes that we consider (and
thus the densities between them) are obtained after the modified
refinement procedure and thus enough information is already embedded
in the reduced graph.

\subsection{Clustering Results on Benchmark Datasets}\label{benchmark}

In this section we report results on a number of datasets described
earlier in Section \ref{datasets}. We do a five fold
cross-validation on each of the datasets, where a validation set is
used to learn the meta-parameters for the data. The accuracy
reported is the average clustering quality on the rest of the data
after using the learned parameters from the validation set. We use a
grid-search to learn the meta-parameters. Initially a coarse grid is
initialized with a set of 25 linearly spaced values for
$\varepsilon$ between 0.15 and 0.50 (we don't want $\varepsilon$ to
be outside this range). For $l$ we simply pick values from 2 to 7
simply because that is the only practical range that we are looking
at.

\begin{table}
\caption{Clustering Results on UCI Datasets. Regular1 and Regular2
represent the results by the versions due to Alon {\em et al.} and
Frieze-Kannan, respectively. Spect1 and Spect2 give the results for
spectral clustering with a k-nearest neighbour graph and a fully
connected graph, respectively. Follow the text for more details.}
\centering
\begin{tabular}{llllllll}
\hline\noalign{\smallskip}
{\bf Dataset} & {\bf \# Feat.} & {\bf Comp.} &  {\bf Regular1} &  {\bf Regular2} & {\bf Spect1} & {\bf Spect2} & {\bf k-means}\\
\noalign{\smallskip}
\hline
\noalign{\smallskip}
R-Wine & 11 & 1599-49 & {\bf 47.0919} & {\bf 46.8342} & 23.9525 & 23.9524 & 23.8899  \\
W-Wine & 11 & 4898-125 &{\bf 44.7509} &{\bf 44.9121} & 23.1319  & 20.5798  & 23.8465   \\
Arcene & 10000 & 200-9 & {\bf 68} & {\bf 68} & 61  & 62  & 59  \\
Blood-T & 4 & 748-49 & {\bf 76.2032} & {\bf 75.1453} & 65.1070  & 66.2331  & 72.3262   \\
Ionos & 34 & 351-25 & {\bf 74.0741} & {\bf 74.6787} & 70.0855  & 70.6553  & 71.2251   \\
Cancer & 9 & 683-52 & 93.5578 & 93.5578 & {\bf 97.2182}   & 97.2173   & 96.0469   \\
Pima & 8 & 768-52 & {\bf 65.1042 } & {\bf 64.9691} &  51.5625  & 60.8073 & 63.0156   \\
Vertebral-1 & 6 & 310-25 &67.7419 & 67.8030 & {\bf 74.5161} & 71.9355 & 67.0968    \\
Vertebral-2 & 6 & 310-25 &{\bf 70 } &{\bf 69.9677} & 49.3948 & 48.3871  & 65.4839   \\
Steel & 27 & 1941-54 & {\bf 42.5554} & {\bf 43.0006} & 29.0057   & 34.7244  &  29.7785  \\
Musk & 166 & 6598-126 & {\bf 84.5862} & {\bf 81.4344} & 53.9103   & 53.6072  &  53.9861  \\
Haberman & 3 & 306-16 & {\bf 73.5294} & {\bf 70.6899} & 52.2876   & 51.9608  &  52.2876  \\
\hline
\end{tabular}\label{UCIResultTable}
\end{table}

We compare our results with a fixed $\sigma$ spectral clustering
with both a fully connected graph (Spect2) and a k-nearest neighbour
graph (Spect1). For the sake of comparison we also include results
for k-means on the entire dataset. We also report results on the
compression that was achieved on each dataset in Table
\ref{UCIResultTable} (The compression is indicated in the format x-y
where x represents one dimension of the adjacency matrix of the
dataset and y of the reduced graph).

In the results we observe that the Regularity Clustering method, as
indicated by the clustering accuracies is quite powerful; it gave
significantly better results in 10 of the 12 datasets. It was also
observed that the regularity clustering method did not appear to
work very well in synthetic datasets. This seems understandable
given the quasi-random aspect of the Regularity method. We also
report that the results obtained by the Alon {\em et al.} and by the
Frieze-Kannan versions are virtually identical, which is not
surprising.

\section{Future Directions} \label{future}

We believe that this work opens up a lot of potential research
problems. First and foremost would be establishing theoretical
results for quantifying the approximation obtained by our
modifications to the Regularity Lemma. Also, the original Regularity
Lemma is applicable only while working with dense graphs. However,
there are sparse versions of the Regularity Lemma. These sparse
versions could be used in the first phase of our method such that
even sparse graphs (k-nearest neighbor graphs) could be used for
clustering, thus enhancing its practical utility even further.

A natural generalization of pairwise clustering methods leads to
hypergraph partitioning problems \cite{Bulo}, \cite{Zhou}. There are
a number of results that extend the Regularity Lemma to hypergraphs
\cite{ChungHyper}, \cite{GowersHyper}, \cite{RodlHyper2}. It is thus
natural that our methodology could be extended to hypergraphs and
then used for hypergraph clustering.

In final summary, our work gives a way to harness the Regularity
Lemma for the task of clustering. We report results on a number of
benchmark datasets which strongly indicate that the method is quite
powerful. Based on this work we also suggest a number of possible
avenues for future work towards improving and generalizing this
methodology.


\section{Introduction}

Recently the subject of quantum channels has catalysed the
attention for its usefulness in foundational issues as well as
in technological applications (for a recent review, see
\cite{CGLM14}).
Formally a quantum channel is a completely positive and trace
preserving map acting on the set of states (density operators)
living in a Hilbert space. Since any physical process involves a
state change, it can be regarded as a quantum channel mapping
the initial (input) state to the final (output) state. As such
it can be characterized in terms of its information transmission
capability. This implies the use of entropic functionals among
which the minimum output entropy plays a dominant role.
In fact it is related to the minimum amount of noise inherent to
the channel, since it quantifies the minimum uncertainty
occurring at the output of a channel when inputting pure states.
More precisely, the output entropy measures the entanglement of
the input pure state with the environment. Being this latter not
accessible, such entanglement induces loss of quantum coherence
and thus injection of noise at the channel output. Clearly, low
values of entanglement, i.e., of output entropy, correspond to
low communication noise.
As a consequence, the study of output entropy yields useful
insights about channel capacities. In particular, an upper bound
on the classical capacity can be derived from a lower bound on
the output entropy of multiple channel uses \cite{King}.

When studying quantum channels a dichotomy between discrete and
continuous channels usually appears. The formers act on states
living in finite dimensional Hilbert space. In contrast the
latter act on states living in infinite dimensional Hilbert
space. This is reflected in the possibility of using discrete or
continuous variables where to encode classical information.
Among continuous quantum channels attention has been almost
exclusively devoted to Gaussian quantum channels,
that is channels mapping Gaussian input states into Gaussian
output ones \cite{Oleg}. The reason is that they are easily
implementable at experiment level and moreover they also handy
at theoretical level.
For these channels the minimum output entropy was largely
investigated \cite{Vittorio1} and then showed that actually
their classical capacity is achieved through states minimizing
the output entropy \cite{Vittorio2}.

Here, we go beyond the restriction of Gaussianity of continuous
quantum channels and propose a model of non-Gaussian quantum
channel that stems from the combination of two physically
relevant processes that occur in open quantum systems, namely
amplitude damping and dephasing. We then analytically find input
states approaching zero output entropy, while respecting the
input energy constraint. They consist in the superposition of
two number states the farthest away one from the other. In
truncated Hilbert space, we find that beside superposition of
two number states, the so-called binomial states \cite{binomial}
can be optimal depending on the value of channels parameters. We
support this latter results by numerical investigations.

The paper is organized as follows. In Section \ref{sec:mod} we
introduce the model and then we show the existence of
optimal input states achieving zero output entropy in Section \ref{sec:optinput}.
Subsequently, in Section \ref{sec:spacetruncation}, we restrict
our attention to truncated Hilbert space and we conjecture about
the optimality of binomial states, beside superposition of two
number states, and we give numerical evidences of this idea.
Section \ref{sec:conclu} is for concluding remarks.

\vspace{-0.8cm}

\section{The model}\label{sec:mod}

Let us start considering the Hilbert space $L^2(\mathbb{R})$
associated to a single bosonic mode
with ladder operator $a,a^\dag$.

In the framework of dynamical maps,
a typical example of Gaussian process is provided by the
amplitude damping effect
described by the master equation \cite{openq}
\begin{equation*}
\frac{d}{dt}\varrho=2 a \varrho a^{\dag}-a^\dag a\varrho-\varrho
a^\dag a=:\mathcal{L}_{AD}(\varrho),
\end{equation*}
for the density operator $\varrho$.
In contrast, a typical example of non-Gaussian process is
provided by the purely dephasing effect
described by the master equation \cite{openq}
\begin{equation*}
\frac{d}{dt}\varrho=2a^\dag a\varrho a^\dag a -(a^\dag a)^2
\varrho-\varrho (a^\dag a)^2=:\mathcal{L}_{PD}(\varrho).
\end{equation*}
In order to interpolate between these two regimes we are going
to consider the following dynamics
\begin{equation}\label{master}
\frac{d}{dt}\varrho=(1-\epsilon)\mathcal{L}_{AD}(\varrho)+\epsilon\mathcal{L}_{PD}(\varrho),
\end{equation}
with $\epsilon\in[0,1]$. It is easy to see that
\begin{equation*}
\mathcal{L}_{AD}(\mathcal{L}_{PD}(\varrho))=\mathcal{L}_{PD}(\mathcal{L}_{AD}(\varrho)).
\end{equation*}
Therefore we can write the formal solution of \eqref{master} as
\begin{equation}\label{prodmaps}
\varrho(t)=e^{(1-\epsilon)t\mathcal{L}_{AD}}e^{\epsilon
t\mathcal{L}_{PD}}\varrho(0).
\end{equation}
Actually this map can be regarded as a
quantum channel $\Phi_{\epsilon,t}$ (depending on the parameters
$\epsilon$ and $t$) mapping
\begin{equation}\label{channel}
\varrho(0)\mapsto\varrho(t)=\Phi_{\epsilon,t}(\varrho(0))=\sum_{j,k=0}^\infty
E_{jk} \varrho(0) E_{jk}^\dag,
\end{equation}
where $E_{jk}$ are the Karus operators \cite{CGLM14}.
In view of \eqref{prodmaps}
\begin{equation*}
E_{jk}=A_jP_k,
\end{equation*}
where $A_j$ are the amplitude damping Kraus operators
\cite{Liu04}
\begin{equation}\label{krausA}
A_j=\sum_{l=j}^\infty \sqrt{
l \choose j } \left[ 1-f(\epsilon,t) \right]^{(l-j)/2}
\left[f(\epsilon,t)\right]^{j/2} |l-j\rangle\langle l |,
\end{equation}
with $f(\epsilon,t):=1-e^{-2(1-\epsilon)t}$,
and $P_k$ are the phase damping Kraus operators \cite{Liu04}
\begin{equation}\label{krausP}
P_k=\sum_{l=0}^\infty \sqrt{\frac{(2 l^2 \epsilon t)^k}{k!}}
e^{-l^2 \epsilon t} |l\rangle\langle l|.
\end{equation}
In Eqs.\eqref{krausA} and \eqref{krausP} it is used the Fock
basis $\{ |l\rangle\}_{l\in \mathbb{N}_0}$ representation.
Expanding $\varrho(0)$ in the same basis as
$\varrho(0)=\sum_{m,n=0}^{\infty}C_{m,n}(0)|m\rangle\langle n|$ and
considering the channel in \eqref{channel}, we obtain
\begin{equation}\label{outexp}
\varrho(t)=\sum_{m,n=0}^{\infty}C_{m,n}(t)|m\rangle\langle n|,
\end{equation}
with
\begin{equation}\label{coeff}
\begin{aligned}
C_{m,n}(t)&=e^{-Y_{m,n}(\epsilon)t}\\
&\times\sum_{l=0}^{\infty}C_{_{m+l,n+l}}(0)
\left[{m+l \choose l}{n+l \choose l}\right]^{\frac{1}{2}} f^l,
\\
\end{aligned}
\end{equation}
in which $Y_{m,n}(\epsilon):=(1-\epsilon)(m+n)+\epsilon (m-n)^2$.
Equation \eqref{coeff} is also the solution of the following
recursive relation
\begin{equation}\label{recursive}
\begin{aligned}
\dot{C}_{m,n}(t)&=2(1-\epsilon)\sqrt{(m+1)(n+1)}C_{m+1,n+1}(t)\\&-Y_{m,n}(\epsilon)C_{m,n}(t),
\end{aligned}
\end{equation}
which is obtainable from the master equation \eqref{master}.

When dealing with quantum channels acting on the set of states
living in an infinite dimensional Hilbert space,
it is customary to
employ the constraint of fixed average
input energy, that is
\begin{equation}\label{inenergy}
{\rm Tr}\left(\varrho(0) a^\dag a\right)=N.
\end{equation}

\vspace{-0.8cm}
\section{Minimizing output entropy}\label{sec:optinput}

The output entropy of the quantum channel $\Phi$ in
Eq.\eqref{channel} is the von Neumann entropy of the output
state, namely
\begin{equation}
S\left(\Phi_{\epsilon,t}\left(\rho\right)\right)
:= -{\rm Tr}\left[ \Phi_{\epsilon,t}\left(\rho\right)
\log_2\left(\Phi_{\epsilon,t}\left(\rho\right)\right)\right].
\end{equation}
In order to quantify the noise inherent to the quantum channel
$\Phi_{\epsilon,t}$
we look for its minimal output entropy and call the state with
minimum output entropy the optimal input state.

The following Theorem states the existence of states with zero
output entropy.
\begin{Theorem}\label{Theorem}
Input states
\begin{equation}\label{kappa}
|\kappa_{\alpha}\rangle=\sqrt{1-\frac{N}{K}}|0\rangle+\sqrt{\frac{N}{K}}e^{i\alpha
K}|K\rangle, \quad K\in\mathbb{N},
\end{equation}
with $\alpha\in\mathbb{R}$, respect the input energy constraint \eqref{inenergy} and satisfy
\begin{equation}
\lim_{K\to\infty}S\left(\Phi_{\epsilon,t}\left(|\kappa_{\alpha}\rangle\langle\kappa_{\alpha}|\right)\right)=0,
\end{equation}
for all values of $\epsilon$ and $t$.
\end{Theorem}

\proof{
First we note that all the states $|\kappa_{\alpha}\rangle$ have the
same output entropy due to the covariance property
of the channel under unitary transformations
\begin{equation*}\label{calU}
U\in\mathcal{U}:=\bigg\{\sum_n e^{i\alpha n} |n\rangle\langle n|
\,\Big|\, \alpha\in\mathbb{R}, n\in\mathbb{N}_0\bigg\}.
\end{equation*}
Therefore, we prove the theorem for $|\kappa_{0}\rangle$.
Using Eq.\eqref{channel}, the corresponding output reads
\begin{equation*}
\begin{aligned}
&\Phi_{\epsilon,t}\left(|\kappa_0\rangle\langle\kappa_0|\right)=\left(1-\frac{N}{K}(1-f^K)\right)|0\rangle\langle
0|\\
&+\sqrt{\frac{N}{K}\left(1-\frac{N}{K}\right)}(1-f)^{K}e^{-\epsilon
K^2t}
\left(|0\rangle\langle K|+|K\rangle\langle 0|\right)\\
&+\frac{N}{K}\sum_{m=1}^K {K \choose m
}(1-f)^mf^{K-m}|m\rangle\langle m|.
\end{aligned}
\end{equation*}
The matrix form of this output state is block-diagonal, so the
eigenvalues can be easily found as
\begin{equation*}
\begin{aligned}
\lambda_{0,K}&=\frac{1}{2}
\left(A+B\pm\sqrt{(A-B)^2+4C^2}\right),\nonumber\\
\lambda_m&=\frac{N}{K} {K \choose m }(1-f)^mf^{K-m}, \qquad
m=1,\ldots, K-1,
\end{aligned}
\end{equation*}
with
\begin{equation*}
\begin{aligned}
A&:=1-\frac{N}{K}\left(1-f^K\right),\nonumber\\
B&:=\frac{N}{K}\left(1-f\right)^K,\nonumber\\
C&:=\sqrt{\frac{N}{K}\left(1-\frac{N}{K}\right)}(1-f)^Ke^{-\epsilon
K^2 t}.
\end{aligned}
\end{equation*}
It is easy to see that all the eigenvalues approach zero for
$K\rightarrow \infty$, except $\lambda_0$ that approaches one.
Therefore the input state \eqref{kappa}, while satisfying the
input energy constraint, leads to a zero output entropy. More
precisely, its output entropy results:
\begin{eqnarray}\label{Soutkappa}
&&S\left(\Phi_{\epsilon,t}\left(|\kappa_0\rangle\langle\kappa_0|\right)\right)=-\lambda_{0}\log_2\lambda_{0}
-\lambda_{K}\log_2\lambda_{K}\nonumber\\
&&-\frac{N}{K} \left[1-f^K-(1-f)^K\right]
\log_2\left(\frac{N}{K}\right)\nonumber\\
&&+\frac{N}{K}\left[f^K\log_2(f^K)+(1-f)^K\log_2\left((1-f)^K\right)\right]\nonumber\\
&&+\frac{N}{K}\frac{1}{2}\log_2\left(2\pi e Kf(1-f)\right)+{\cal
O}\left(\frac{1}{K}\right).
\end{eqnarray}
Now fixing $\mathcal{E}>0$ we should find
$\mathcal{K}\in\mathbb{N}$ such that
$S\left(\Phi_{\epsilon,t}\left(|\kappa_0\rangle\langle\kappa_0|\right)\right)<\mathcal{E}$
for $K>\mathcal{K}$.
To this end we first find an upper bound for \eqref{Soutkappa}.
Since projective measurements increase entropy \cite{MOhya}, we
have the inequality
\begin{equation*}
S\left(\Phi_{\epsilon,t}\left(|\kappa_0\rangle\langle\kappa_0|\right)\right)
\le H\left(p_{\epsilon,t}(n)\right)
\end{equation*}
where the r.h.s. is the Shannon entropy of the probability mass
function $p_{\epsilon,t}(n):=\langle
n|
\Phi_{\epsilon,t}\left(|\kappa_0\rangle\langle\kappa_0|\right)
|n\rangle$.
Explicitly the latter reads
\begin{equation*}
p_{\epsilon,t}(n)=\left\{
\begin{array}{ccc}
1-\frac{N}{K}(1-f^K)& &n=0\cr\cr
 \frac{N}{K}{K \choose n }(1-f)^mf^{K-n}& &n=1,\ldots, K
\end{array}
\right..
\end{equation*}
As a consequence
\begin{equation*}
\begin{aligned}
H\left(p_{\epsilon,t}(n)\right)&=-\left[1-\frac{N}{K}\left(1-f^K\right)\right]\log_2\left[1-\frac{N}{K}\left(1-f^K\right)\right]
\nonumber\\
&-\frac{N}{K}\log_2\left(\frac{N}{K}\right)+\frac{N}{K}f^K\log_2\left(\frac{N}{K}f^K\right)\nonumber\\
&+\frac{1}{2}\frac{N}{K}\log_2\left(2\pi e K f(1-f)\right)+{\cal
O}\left(\frac{1}{K}\right).
\end{aligned}
\end{equation*}
Using the inequality $-x\log x<\sqrt{x(1-x)}$ we then get
\begin{equation*}
\begin{aligned}
H\left(p_{\epsilon,t}(n)\right)&\leq \sqrt{
\left(1-\frac{N}{K}\left(1-f^K\right)\right)\frac{N}{K}\left(1-f^K\right)}
\nonumber\\
&+\sqrt{\frac{N}{K}\left(1-\frac{N}{K}\right)}
+\frac{1}{2}\frac{N}{K}\sqrt{2\pi e
Kf\left(1-f\right)}\nonumber\\
&\leq
2\sqrt{\frac{N}{K}}+\sqrt{\frac{N}{K}\frac{\pi}{2}eNf(1-f)}.
\end{aligned}
\end{equation*}
By imposing that the above r.h.s. becomes smaller than $\mathcal
E$, it follows
\begin{equation*}
\mathcal{K}=\left\lceil \frac{N}{{\cal
E}^2}\left(2+\sqrt{\frac{\pi}{2}eNf(1-f)}\right)^2 \right\rceil.
\end{equation*}}

\vspace{-0.8cm}

\section{Space Truncation}\label{sec:spacetruncation}

In the previous Section we showed that the input states
\eqref{kappa} give zero output entropy for
$K\rightarrow\infty$. However, if we truncate the Hilbert space
to a finite value of $K$, it is not guaranteed that these
states are still optimal. Finding the optimal input
state under that condition is the aim of this Section.
To start with, we introduce a class of states knows as binomial
states \cite{binomial}
\begin{equation}\label{Binomial}
|B\rangle_{M,\mu}:=\sum_{n=0}^M\beta_{n}|n\rangle,\;\;\;
\beta_{n}:=\left[{M \choose
n}\mu^n(1-\mu)^{M-n}\right]^{\frac{1}{2}},
\end{equation}
with parameters $M\in\mathbb{N}$ and $\mu\in[0,1]$. The binomial
state \eqref{Binomial} reduces to the number state $|0\rangle$
for $\mu=0$ and to the number state $|M\rangle$ for $\mu=1$.
In contrast, in the limit $\mu\rightarrow 0$,
$M\rightarrow \infty$, and $\mu M=\alpha\in\mathbb{R}$ the
binomial state approaches the coherent state $|\alpha\rangle$.

The energy constraint \eqref{inenergy} yields the relation
\begin{equation*}\label{binave}
{\rm Tr}\left( |B\rangle_{M,\mu}\langle B| a^\dag a\right)=M\mu
= N.
\end{equation*}
Furthermore, inserting the coefficients $\beta_n$ of \eqref{Binomial} into \eqref{coeff} we get
the explicit expression
of the output density operator representation in the Fock basis
\begin{equation*}\label{BinoOut}
\begin{aligned}
&\Phi_{\epsilon,t}(|B\rangle_{M,\mu}\langle
B|)=\sum_{m,n=0}^Me^{-Y_{m,n}(\epsilon)t}
\left(\frac{\mu}{1-\mu}\right)^{\frac{m+n}{2}}\\
&\times \sum_{l=0}^{M-\max\{m,n\}}
\left[ {M \choose m+l} {m+l \choose m}{M \choose n+l} {n+l
\choose n}\right]^{\frac{1}{2}}\\
& \times \left(\mu f \right)^l
\left(1-\mu\right)^{M-l} |m\rangle\langle n|.
\end{aligned}
\end{equation*}
Here we numerically evaluate the output entropy for binomial
input states with average energy $N$. Once $N$ is fixed we still
have the freedom to vary $\mu$ and $M$ in a way that $\mu M=N$.
Since $\mu\leq 1$, for fixed $N$, we
increase $M$ from $\lceil N \rceil$ to $K$,
in order to find the minimum value of
$S\left({\Phi_{\epsilon,t}\left(|B\rangle_{M,\mu}\langle
B|\right)}\right)$. From here on, when we refer to the binomial state $|B\rangle$,
we mean the one which has minimum output entropy among other
possible binomial states with average energy $N$.

 \begin{figure}[t]
\centering

   \includegraphics[scale=0.4]{Bino_Magic_N06}

 \centering
\caption{(Color Online) Output entropy for input state $|B\rangle$ (Blue dashed line) and
for $|\kappa_0\rangle$ (solid magenta line)
 input states with $N=0.6$ ant  $t=0.5$ (top), $t=1.5$ (bottom).}
\label{Bino_Magic}
\end{figure}

Figure \ref{Bino_Magic} shows the output entropy of the state
$|B\rangle$ in \eqref{Binomial} (Blue dashed line)
and of the state $|\kappa_0\rangle$ in \eqref{kappa}
(Magenta solid line) versus $\epsilon$ for $N=0.6$ at $t=0.5$
(top) and $t=1.5$ (bottom). Here $4$-dimensional Hilbert space is
considered. As can be argued from these figures, the output entropy
of $|B\rangle$ remains smaller than the output entropy of
$|\kappa_0\rangle$ (for any value of $\epsilon$)
until $t$ reaches a threshold $t_*$. Then, for $t>t_*$ the state with less
output entropy can be either $|B\rangle$ or $|\kappa\rangle$
depending on the value of $\epsilon$ (see also Fig.\ref{tstar}).

To have an estimation of $t_*$, we first point out that our
numerical analysis shows that the output entropy of
$|B\rangle$ and $|\kappa_0\rangle$ cross each other at large
values of $\epsilon$ where the optimal value of $M$ is 1. In such a case
the output state of $|B\rangle$ lives in a two
dimensional subspace and its output entropy turns out to be
\begin{equation*}
\begin{aligned}
S_B&=-\sum_{j=1}^2\mu_j\log(\mu_j),\\
\mu_{1,2}&:=\frac{1\pm\sqrt{(1-2N(1-f))^2+4N(1-N)e^{-2t}}}{2}.
\end{aligned}
\end{equation*}
Then solving the equation
$S(\Phi_{\epsilon,t}\left(|\kappa_0\rangle\langle\kappa_0|)\right)=S_B, $
we can find the value of $t_*$.

To do the similar calculation for any given $N$, we have
numerically found that the optimal value of $M$ is
$\lceil N\rceil$. Therefore the output entropy of
$|B\rangle_{M,\mu}$, with $M=\lceil N\rceil$ and $\mu=N/M$,
should be found and equated to
$S(\Phi_{\epsilon,t}\left(|\kappa_0\rangle\langle\kappa_0|)\right)$
in order to get $t_*$.

 \begin{figure}[t]
\centering
   \includegraphics[width=0.4\textwidth]{2dsurf}
 \centering
 \caption{ Curve in the $\epsilon,t$ plane where $S(\Phi_{\epsilon,t}(|B\rangle\langle B|))=
 S(\Phi_{\epsilon,t}(|\kappa_0\rangle\langle \kappa_0|))$  for $N=0.6$. On the left (resp. on the right) of the
 curve it is $S(\Phi_{\epsilon,t}(|B\rangle\langle B|))<
 S(\Phi_{\epsilon,t}(|\kappa_0\rangle\langle \kappa_0|))$
 (resp. $S(\Phi_{\epsilon,t}(|B\rangle\langle B|))>
 S(\Phi_{\epsilon,t}(|\kappa_0\rangle\langle \kappa_0|))$).
 The horizontal dashed line represents the value of $t_*$.
 }
\label{tstar}
\end{figure}

After having compared the behaviour of the output entropy for
inputs of the kind \eqref{kappa} and \eqref{Binomial}, we
formulate the following conjecture.

\begin{conjecture}\label{conj:bin+magic}
In a truncated Hilbert space of dimension $K+1$, the
minimal output entropy of the quantum channel \eqref{channel} is
achieved either by binomial states of Eq.\eqref{Binomial} or by
states $|\kappa_{\alpha}\rangle$ of Eq. \eqref{kappa},
depending on the values of $\epsilon$ and $t$.
\end{conjecture}
To support this Conjecture we perform a uniform random search over all pure input states in the finite dimensional Hilbert space. The restriction to search only among pure states is motivated by the following Lemmas.

\begin{lemma}
Given a self adjoint operator $H: \mathbb{C}^{K+1} \to \mathbb{C}^{K+1}$,
we can always decompose a density operator $\rho$ on $\mathbb{C}^{K+1}$ satisfying a linear constraint ${\rm Tr}(\rho H)=N$, in terms of pure states  $|\psi_k\rangle$
satisfying the same constraint, i.e. ${\rm Tr}(|\psi_k\rangle\langle\psi_k| H)=N$.
\end{lemma}

\proof{
Consider the spectral decomposition of $H=\sum \limits _j  h _j |j\rangle\langle j|$.
An arbitrary density
operator represented in the $H$ eigenvectors basis
\begin{equation}\label{rho1}
\rho =\sum \limits _{i,j} r _{i,j} |i\rangle\langle j|, \quad r_{j,j}>0,\quad\sum \limits _j r _{j,j}=1,
\end{equation}
satisfies the constrain if ${\rm Tr}(\rho H)=\sum \limits _j h_j r_{{j,j}}=N$.
Decomposing $\rho$ in terms of pure states we have
\begin{equation}\label{rho2}
\rho =\sum \limits _k p _k |\psi_k\rangle\langle \psi_k|,\quad p _k>0,\quad  \sum \limits _k p _k=1,
\end{equation}
Comparing Eqs.\eqref{rho1} and \eqref{rho2}, we find that
$\sum \limits _k p _k|\langle \psi_k|j\rangle |^2=r _{{j,j}}$.
If we take
\begin{equation}\label{condi}
|\langle \psi_k|j\rangle|^2=r_{{j,j}}, \quad \forall k,
\end{equation}
it will result
\begin{equation*}
{\rm Tr} ( |\psi_k\rangle\langle \psi_k| H)=\sum \limits _j h_j r_{{j,j}} =N, \quad \forall k.
\end{equation*}
Hence it is enough to determine the $|\psi_k\rangle$s from the condition \eqref{condi} to get a decomposition
of $\rho$ in terms of pure states satisfying the same constraint. This is always possible, actually in infinite many ways.
Additionally we have the freedom in choosing the $p_k$s.}

\begin{lemma}
The minimum output entropy of a quantum channel $\Phi$ acting on states $\rho$ on
$\mathbb{C}^{K+1}$ satisfying the energy constraint \eqref{inenergy} is achieved on pure states.
\end{lemma}

\proof{
Assume that the minimum output entropy is achieved by the input state $\rho$ satisfying the energy constraint. Decomposing it in terms of pure states that satisfy the same energy constraint
$\rho=\sum_k p_k |\psi_k\rangle\langle \psi_k|$, and using the concavity of von Neumann entropy \cite{MOhya}, we have
\begin{eqnarray*}
S(\Phi(\rho))&=&S\big(\sum_k p_k \Phi\left(|\psi_k\rangle\langle \psi_k|\right)\big)\\
&\geq&\sum_k p_k S\left(\Phi\left(|\psi_k\rangle\langle \psi_k|\right)\right).
\end{eqnarray*}
In the decomposition, let us denote the pure state with minimum output entropy by $|\psi_*\rangle$. Therefore we have:
\begin{equation*}
S\left(\Phi(\rho)\right)\geq S\left(\Phi\left(|\psi_*\rangle\langle \psi_*|\right)\right),
\end{equation*}
that is, the optimal input state must be pure.
}

To generate random pure input states in $K+1$-dimensional
Hilbert space, we employ the following parametrization
\begin{equation*}
\begin{aligned}
|\psi\rangle&=\sum_{n=0}^{K} \nu_n |n\rangle,\\
\nu_0&=\cos\theta_K,\quad
\nu_{n>0}=e^{i\phi_n}\cos\theta_{K-n}\prod_{l=K-n+1}^{K}\sin\theta_l.
\end{aligned}
\end{equation*}
Then, according to \cite{Karol}, it is enough to generate $\phi_{n\geq 1}\in[0,2\pi)$ from
a uniform distribution $p(\phi_{n\geq 1})=\frac{1}{2\pi}$ and random
independent variables $\xi_n$ distributed uniformly in $[0,1]$
for $n=1,\ldots, K$ defining
\begin{equation*}
\theta_n:=\arcsin(\xi_n^{\frac{1}{2n}}).
\end{equation*}
However, due to the energy constraint \eqref{inenergy},
we should consider states satisfying
 $\sum_{n=0}^{K} n |\nu_n|^2=N$.
This imposes a functional relation among $\theta_n$s and so
among $\xi_n$s, which can be written as:
 $\xi_{K}=g(\xi_1,\xi_2,\ldots\xi_{K-1};N)$.
Therefore we should generated $K-1$ random variables with the
following modified probability distribution function
\begin{equation*}
\begin{aligned}
\tilde{p}(\xi_1,\ldots, \xi_{K-1})
={\cal C}\int d\xi_{K} p(\xi_1,\ldots,\xi_{K})
\delta\left(\xi_{K}-g\right),
\end{aligned}
\end{equation*}
being ${\cal C}$ a normalization factor and
$p(\xi_1,\ldots,\xi_{K})$ the probability distribution
function for the variables $\xi_1,\ldots,\xi_{K}$.
Since these are chosen independently and with a standard uniform
distribution in $[0,1]$, we conclude that we should generate
$\xi_1,\ldots,\xi_{K-1}$ according to
$ \tilde{p}(\xi_1,\ldots, \xi_{K-1})=p(\xi_1,\ldots, \xi_{K-1})=1$,
and pick $\xi_{K}$ as
\begin{equation*}
\begin{aligned}
&\xi_{K}=g(\xi_1,\ldots, \xi_{K-1};N)\\
&=\frac{N}{\left[1+\xi_{K-1}^{1/{(K-1)}}\left(1+\xi_{K-2}^{1/(K-2)}\left(1+\cdots\xi_2^{1/2}\left(1+\xi_1\right)\right)\right)\right]}.\nonumber
\end{aligned}
\end{equation*}
In our $4$-dimensional example with $N=0.6$ the search over
$10^5$ states, generated as explained above, confirms the
statement of Conjecture \ref{conj:bin+magic}.

\vspace{-0.8cm}

\section{Conclusion}\label{sec:conclu}

We have opened an avenue for studying,
from an information theoretic point of view,
continuous quantum channels beyond the usual restriction of
Gaussianity.
Actually we have proposed a model of non-Gaussian quantum
channel that stems from a master equation accounting for two
processes, amplitude damping and dephasing.
Its physical relevance relies on the fact that
amplitude damping and dephasing are
applied in many concrete discussions to model noise of quantum
information processing with single mode light field, vibration
phonon mode, or excitonic wave, see e.g. \cite{Lidar}.

Then, the first question that arises is how much the introduced
channel deviates from Gaussianity. Arguably this depends on the
parameter $\epsilon$, however an exact quantification would be
in order, maybe in a fashion similar to what has been done for
non-Gaussian states \cite{Konrad}. This could also shed light on
the choice of optimal input states for communication tasks.
Here we found input states approaching zero output
entropy, while respecting the input energy constraint.
They consist in the superposition of two number states the
farthest away one from the other. In truncated Hilbert space,
the minimum output entropy remains finite and optimal input
states are conjectured to be binomial states beside
superposition of two number states, depending on the values of
the channel's parameters. This is corroborated by numerical
results. The study performed in truncated Hilbert space is
justified by the fact that in realistic physical situations is
hard to fully exploit the infinite dimensionality of the space
$L^2(\mathbb{R})$.

As further development one could address the issue of additivity
of output entropy for two copies of the channel and then
eventually of multiple copies. This would be motivated by the
additivity of the classical capacity
deriving from the additivity of the minimum output entropy
\cite{Shor}.

Although challenging,
the introduced map leaves concrete hopes for characterizing its
(product states) classical capacity which implies finding the
optimal input ensemble of states maximizing the Holevo chi
quantity \cite{Holevo}.

\vspace{-0.8cm}

\acknowledgments
S. M. would like to thank the Sharif University of Technology for
kind hospitality during the final stage of this work.