\section{Introduction}

\IEEEPARstart{T}{ext} in images contains
valuable information and is exploited in many
content-based image and video applications, such as content-based web
image search, video information retrieval, mobile based text analysis and
recognition~\cite{Zhong2000, Doermann2000, Weinman2009, Yin2011, Chew2011}.
Due to complex background, variations of  font,
size, color and orientation, text in natural scene images
has to be robustly detected before being recognized or retrieved.

Existing methods for scene text detection can  roughly be
categorized into three groups: sliding window based
methods~\cite{derivative_feature, adaboost_text, Kim2003},
connected component based methods~\cite{Epshtein, structure-partition, color-clustering},
and hybrid methods~\cite{pan}.
Sliding window based methods, also called as region based methods, engage a sliding window to search
for possible texts in the image and then use machine learning
techniques to identify texts.
These methods tend to be slow as the image has to be
processed in multiple scales.
Connected component based methods extract character candidates from
images by connected component analysis followed by grouping
character candidates into text; additional checks may be
performed to remove false positives.
The hybrid method presented by Pan et al.~\cite{pan} exploits a
region detector to detect text candidates and extracts
connected components as character candidates by local
binarization; non-characters are eliminated with a Conditional Random Fields~(CRFs)~\cite{crf} model,
 and characters can finally be grouped into text.
More recently, Maximally Stable Extremal Region (MSER) based methods, which actually fall into the family of
connected component based methods but use MSERs~\cite{mser} as character candidates, have
become the focus of several recent projects~\cite{icdar2011, edge_mser, head_mounted,
  real_time, pruned_search, neumann_method, mser2013}.

Although the MSER based method is the winning method of the benchmark data, i.e., ICDAR
2011 Robust Reading Competition~\cite{icdar2011}
and has reported promising
performance, there remains several problems to be addressed.
First, as the MSER algorithm detects a large number of non-characters,
most of the character candidates need to be removed
before further processing. The existing methods for MSERs
pruning~\cite{head_mounted, real_time},  on one hand, may still have room for further improvement in terms of the accuracies;  on the other hand, they tend to be slow because
of the computation of complex features.
Second, current approaches~\cite{head_mounted, real_time, pan} for
text candidates construction, which can be categorized as
rule based and clustering based methods, work well but
are still not sufficient;
rule based methods generally require hand-tuned parameters,
which is time consuming and error pruning;
the clustering based method~\cite{pan} shows good performance but it is
complicated by incorporating a second stage processing
after minimum spanning tree clustering.

In this paper, we propose a robust and accurate MSER based scene text detection
method.
First, by exploring the hierarchical structure of MSERs and
adopting simple features, we designed a fast and accurate MSERs
pruning algorithm;
the number of character candidates to be processed is
significantly reduced with a high accuracy.
Second, we propose a novel self-training distance metric learning algorithm
that can learn distance weights and clustering threshold simultaneously and automatically;
character candidates are clustered into text candidates
by the single-link clustering algorithm using the learned parameters.
Third, we propose to use a character classifier to
estimate the posterior probabilities of text candidates
corresponding to non-text and remove text candidates with
high probabilities.
Such elimination helps to train a more powerful text
classifier for identifying text.
By integrating the above ideas, we built an accurate and robust
scene text detection system.
The system is evaluated on the benchmark ICDAR 2011 Robust Reading
Competition dataset and achieved an $f$ measure of 76\%. To our best knowledge, this result ranks the first on all the reported performance and is much higher the current best performance of 71\%.
We also validate our method on the multilingual (include Chinese and English) dataset used in ~\cite{pan}. With an $f$ measure of 74.58\%, our system  significantly outperforms  the competitive method~\cite{pan} that achieves only 65.2\%.
An online demo of our proposed scene text detection system
  is available at \emph{\url{http://kems.ustb.edu.cn/learning/yin/dtext}}.

The rest of this paper is organized as follows. Recent MSER
based scene text detection methods
are reviewed in Section~\ref{section:related_work}.
Section~\ref{section:system} describes the proposed scene text detection method.
Section~\ref{section:experimental_results} presents the experimental results of the proposed system
on ICDAR 2011 Robust Reading Competition dataset and a
multilingual (include Chinese and English) dataset.
Final remarks are presented in Section~\ref{section:conclusion}.

\section{Related Work}
\label{section:related_work}
As described above,  MSER based methods have demonstrated very promising performance in many real projects. However, current MSER based methods still have some
key limitations, i.e., they may suffer from  large number of non-characters candidates in detection and also
insufficient text candidates construction algorithms. In this section, we review the MSER
based methods with the focus on these two problems. Other scene text detection methods can be referred to in some
survey papers~\cite{survey04,survey05,survey08}. A recently published MSER based method can be referred to in
 Shi et al.~\cite{mser2013}.

The main advantage of MSER based methods over traditional
connected component based methods may root in the usage of MSERs as
character candidates.
Although the MSER algorithm can detect most characters even when
the image is in low quality (low resolution, strong noises,
low contrast, etc.), most of the detected character candidates correspond
to non-characters.
Carlos et al.~\cite{head_mounted} presented a MSERs pruning
algorithm that contains two steps: (1) reduction of linear
segments and (2) hierarchical filtering.
The first stage reduces linear segments in the MSER tree into
one node by maximizing the \emph{border energy} function;
the second stage walks through the tree in a depth-first
manner and eliminates nodes by checking them against a
cascade of filters: \emph{size, aspect ratio, complexity,
  border energy and texture}.
Neumann and Matas~\cite{real_time} presented a two stage algorithm for
Extremal Regions (ERs) pruning.
In the first stage, a classifier trained from
incrementally computable descriptors (\emph{area, bounding
  box, perimeter, Euler number and horizontal crossing}) is used to
estimate the class-conditional probabilities
$p(r|\mbox{chracter})$ of ERs; ERs corresponding to
local maximum of probabilities in the
ER inclusion relation are selected.
In the second stage, ERs passed the first stage are
classified as characters and non-characters using more
complex features. As most of the MSERs correspond to
non-characters, the purpose of using cascading filters and
incrementally computable descriptors in these above two methods is
to deal with the computational complexity caused by the high
false positive rate.

Another challenge of MSER based methods, or more generally,
CC-based methods and hybrid methods, is how to group
character candidates into text candidates.
The existing methods for text candidates construction fall
into two general approaches: rule-based~\cite{edge_mser,
  head_mounted, real_time} and clustering-based methods~\cite{pan}.
Neumann and Matas~\cite{real_time} grouped character
candidates using the text line constrains, whose basic
assumption is that characters in a word can be fitted by one or
more top and bottom lines.
Carlos et al.~\cite{head_mounted} constructed a fully connected
graph over character candidates; they filtered edges by
running a set of tests (edge angle, relative position
and size difference of adjacent character candidates) and
used the remaining connected subgraphs as text candidates.
Chen et al.~\cite{edge_mser} pairwised character candidates as
clusters by putting constrains on stroke width and height
difference; they then
exploited a straight line to fit to the centroids of
clusters and declared a line as text candidate if it
connected three or more  character candidates.
The clustering-based method presented by Pan et al.~\cite{pan} clusters character
candidates into a tree using the minimum spanning tree
algorithm with a learned distance metric~\cite{yin-liu:metric-2009};
text candidates are constructed by cutting off between-text
edges with an energy minimization model.
The above rule-based methods generally require hand-tuned
parameters, while the clustering-based method
is complicated by the incorporating of the
post-processing stage, where one has to specify the
energy model.

\section{Robust Scene Text Detection}
\label{section:system}
In this paper, by incorporating several key improvements over
traditional MSER based methods, we propose a novel MSER based
scene text detection method, which finally leads to significant performance improvement over the other competitive methods.
The structure of the proposed system, as well as the sample result
of each stage is presented in
Figure~\ref{fig:system_overview}.
The proposed scene text detection method includes the
following stages:

1)  \emph{Character candidates extraction}.
character candidates are extracted using the MSER algorithm;
most of the non-characters are reduced by the proposed MSERs
pruning algorithm using the strategy of minimizing
regularized variations. More details are presented in Section~\ref{section:mser_extraction}.

2)  \emph{Text candidates construction}.
distance weights and threshold are learned simultaneously
using the proposed metric learning algorithm; character
candidates are clustered into text candidates by the
single-link clustering algorithm using the learned
parameters. More details are presented in Section~\ref{section:region_construction}.

3)  \emph{Text candidates elimination}.   the
posterior probabilities of text candidates corresponding to
non-text are measured using the character classifier and
text candidates with high probabilities for non-text are removed.
More details are presented in Section~\ref{section:character_classifier}.

4)  \emph{Text candidates classification}.   text
candidates corresponding to true text are identified by the
text classifier. An AdaBoost
classifier is trained to decide whether an text candidate corresponding
to true text or not~\cite{Yin12}.
As characters in the same text tend to have similar features,
the uniformity of character candidates'
features are used as text candidate's features to train the classifier.

\begin{figure}
  \centering
  \includegraphics[width=0.4\textwidth]{system-crop.pdf}
  \caption{Flowchart of the proposed system and the corresponding experimental
    results after each step of a sample image.
    Text candidates are labeled by blue bounding rectangles;
    character candidates identified as characters are
    colored green, others red.
  }
  \label{fig:system_overview}
\end{figure}

In order to measure the performance of the proposed system
using the ICDAR 2011 competition dataset, text candidates
identified as text are further partitioned into words
by classifying inner character distances into character
spacings and word spacings using an AdaBoost classifier~\cite{Yin12}.
The following features are adopted: spacing aspect ratio,
relative width difference between left and right
neighbors, number of character candidates in the text
candidate.

\subsection{Letter Candidates Extraction}
\label{section:mser_extraction}
\subsubsection{Pruning Algorithm Overview}
The MSER algorithm is able to detect almost all
characters even when the image is in low quality.
However, as shown in Figure~\ref{fig:mser_tree_origin}, most of the detected
character candidates correspond to non-characters and
should be removed before further processing.
Figure~\ref{fig:mser_tree_origin} also shows that the detected characters forms
a tree, which is quite useful for designing  the
pruning algorithm.
 In real world situations, as characters cannot be
``included'' by or ``include''
other characters, it is safe to remove children once the
parent is known to be a character, and vice versa.
The parent-children elimination is a safe operation
because characters are preserved after the operation.
By reduction, if the MSER tree is pruned by applying
parent-children elimination operation
recursively in a depth-first manner, we are still in safe
place and characters are preserved.
As shown in Figure~\ref{fig:mser_tree_tree_accumulation},
the above algorithm will end up with a set of
disconnected nodes containing all the characters.
The problem with the above algorithm is that it is expensive to
identify character.
Fortunately, rather than identifying the character,
the choice between parent and children can be
made by simply choosing the one that is more likely to be
characters,
which can be estimated by
the proposed regularized variation scheme.
Considering different situations in MSER trees,
we design two versions of the parent-children
elimination method,
namely the \emph{linear reduction} and
\emph{tree accumulation} algorithm.
Non-character regions are eliminated by the linear
reduction and tree accumulation algorithm using the strategy
of minimizing regularized variations.
Our experiment on ICDAR 2011 competition training set shows that
more than 80\% of character candidates are eliminated using
the proposed pruning algorithm.

In the following sections, we first introduce the
concept of variation and explain why variations need to be
regularized. Then we introduce the linear reduction and
tree accumulation algorithm. Finally we present the
complexity analysis for the proposed algorithms.

\subsubsection{Variation and Its Regularization }
According to Matas et al.~\cite{mser},
an ``extremal region'' is a connected component of an image
whose pixels have either higher or lower intensity than its
outer boundary pixels~\cite{vlfeat, detector_compare}.
Extremal regions are extracted by applying a set of
increasing intensity levels to the gray scale image.
When the intensity level increases,
a new extremal region is extracted by
accumulating pixels of current level and joining
lower level extremal regions~\cite{head_mounted};
when the top level is reached, extremal regions of the whole image are extracted as
a rooted tree.
The variation of an extremal region is defined as follows.
Let $R_l$ be an extremal region, $B(R_l) = (R_l, R_{l+1}, ...,
R_{l+\Delta})$
($\Delta$ is an parameter)  be the branch of the tree rooted at $R_l$,
the variation (instability) of $R_l$ is defined as
\begin{equation}
  v(R_l) = \frac{|R_{l+\Delta} - R_l|}{|R_l|}.
\end{equation}
An extremal region $R_l$ is a maximally stable extremal region if its
variation is lower than (more stable) its parent $R_{l-1}$ and
child $R_{l+1}$~\cite{vlfeat, mser}.
Informally, a maximally stable extremal region is an extremal region
whose size remains virtually unchanged over a range of
intensity levels~\cite{real_time}.

As MSERs with lower variations have sharper
borders
and are more likely to be characters,
one possible strategy may be used by the
parent-children elimination operation is to select parent or children
based on who have the lowest variation.
However, this strategy alone will not work because
MSERs corresponding to characters may not
necessarily have lowest variations.
Consider a very common situation depicted in Figure~\ref{fig:situations}.
The children of the MSER tree in Figure~\ref{fig:situation1} correspond to
characters while the parent of the MSRE tree in
Figure~\ref{fig:situation2} corresponds to
character.
The ``minimize variation'' strategy cannot deal with this
situation because either parent or children may
have the lowest variations.
However, our experiment shows that this limitation can be easily fixed by variation
regularization, whose  basic idea is to penalize variations of
MSERs with too large or too small aspect ratios.
Note that we are not requiring characters to have the lowest
variations globally, a lower variation in a parent-children
relationship suffices for our algorithm.

Let $\mathcal{V}$ be the variation and $a$ be the aspect
ratio of a MSER,
the aspect ratios of characters
are expected to fall in $[a_{min}, a_{max}]$, the
regularized variation is defined as
\begin{equation}
\mathcal{V} =
\begin{cases}
  \mathcal{V} - \theta_1 (a - a_{max}) & \text{if } a > a_{max} \\
  \mathcal{V} - \theta_2 (a_{min} - a) & \text{if } a < a_{min} \\
  \mathcal{V}                      & \text{otherwise}
\end{cases}\;,
\end{equation}
where $\theta_1$ and $\theta_2$ are penalty parameters.
Based on experiments on the training dataset,
these parameters are set as
$\theta_1 = 0.03, \theta_2 = 0.08, a_{max} = 1.2 \text{ and }
a_{min} = 0.7$.

\begin{figure}[htb!]
  \centering
  \begin{subfigure}[b]{0.11\textwidth}
    \centering
    \includegraphics[width=\textwidth]{situation1.png}
    \caption{}
    \label{fig:situation1}
  \end{subfigure}
  \begin{subfigure}[b]{0.11\textwidth}
    \centering
    \includegraphics[width=\textwidth]{situation2.png}
    \caption{}
    \label{fig:situation2}
  \end{subfigure}
  \caption{Character correspondence in MSER trees.
    (a) A MSER tree whose children corresponds to characters;
    (b) a MSER tree whose parent corresponds to character.
  }
  \label{fig:situations}
\end{figure}

\begin{figure}[htb!]
  \centering

  \begin{subfigure}[b]{0.09\textwidth}
    \centering
    \includegraphics[trim = 60mm 14mm 0mm 107mm, clip,
    width=\textwidth]{mser_tree_origin.png}

    \caption{}
    \label{fig:mser_tree_origin}
  \end{subfigure}
  \begin{subfigure}[b]{0.09\textwidth}
    \centering
    \includegraphics[trim = 60mm 14mm 0mm 107mm, clip, width=\textwidth]{mser_tree_var.png}

    \caption{}
    \label{fig:mser_tree_var}
  \end{subfigure}
  \begin{subfigure}[b]{0.09\textwidth}
    \centering
    \includegraphics[trim = 60mm 14mm 0mm 107mm, clip, width=\textwidth]{mser_tree_var_recalc.png}

    \caption{}
    \label{fig:mser_tree_regularized_var}
  \end{subfigure}
  \begin{subfigure}[b]{0.09\textwidth}
    \centering
    \includegraphics[trim = 60mm 0mm 0mm 35mm, clip, width=\textwidth]{mser_tree_linear_reduction.png}

    \caption{}
    \label{fig:mser_tree_linear_reduction}
  \end{subfigure}
  \begin{subfigure}[b]{0.09\textwidth}
    \centering
    \includegraphics[width=\textwidth]{mser_tree_tree_accumulation2.png}

    \caption{}
    \label{fig:mser_tree_tree_accumulation}
  \end{subfigure}

  \caption{MSERs pruning.
    (a) MSER tree of a text segment;
    (b) MSERs colored according to variations, as
    variations increase, MSERs are colored from green to
    yellow then to red;
    (c) MSERs colored according to regularized
    variations;
    (d) MSER tree after linear reduction;
    (e) character candidates after tree accumulation.
  }\label{fig:mser_reduction}
\end{figure}

Figure~\ref{fig:mser_tree_var} shows a MSER tree colored
according to variation.
As variation increases, the color changes from green to
yellow then to red.
The same tree colored according to regularized variation is
shown in Figure~\ref{fig:mser_tree_regularized_var}.
The MSER tree in Figure~\ref{fig:mser_tree_regularized_var}
are used in our linear reduction (result presented in Figure~\ref{fig:mser_tree_linear_reduction}) and tree accumulation
algorithm (result presented in Figure~\ref{fig:mser_tree_tree_accumulation}).
Notice that ``variation'' in the following sections refer to
``regularized variation''.

\subsubsection{Linear Reduction}

The linear reduction algorithm is used in situations where
MSERs has only one child.
The algorithm chooses from parent and child the one with the
minimum variation and discards the other.

This procedure is applied across the whole tree recursively.
The detailed algorithm is presented in
Figure~\ref{fig:linear_reduction}.
Given a MSER tree, the procedure
returns the root of the processed
tree whose linear segments are reduced.
The procedure works as follows.
Given a node $t$, the procedure checks the number of
children of $t$; if $t$ has no children, returns $t$ immediately;
if $t$ has only one child,
get the root $c$ of child tree by first applying
the linear reduction procedure to the child tree; if $t$ has a
lower variation compared with $c$, link $c$'s children to $t$
and return $t$; otherwise we return $c$; if $t$ has more than
one children, process these children using linear reduction
and link the resulting trees to $t$ before returning $t$.
Figure~\ref{fig:mser_tree_linear_reduction} shows the
resulting MSER tree
after applying linear reduction to the tree shown in
Figure~\ref{fig:mser_tree_regularized_var}.
Note that in the resulting tree
all linear segments are reduced and non-leaf nodes always
have more than one children.

\begin{figure}[htb!]

  \begin{algorithmic}[1]
    \Procedure{Linear-Reduction}{$T$}
    \If{nchildren[$T$] = 0}
    \State \textbf{return} $T$
    \ElsIf{nchildren[$T$] = 1}
    \State $c$ $\gets$ {\Call{Linear-Reduction}{child[$T$]}}
    \If{var[$T$] $\leq$ var[$c$]}
    \State{link-children($T$, children[$c$])}
    \State \textbf{return} $T$
    \Else
    \State \textbf{return} $c$
    \EndIf
    \Else\Comment{nchildren[$T$] $\geq$ 2}
    \For{ \textbf{each}  $c$  $\in$ children[$T$]}
    \State link-children($T$, {\Call{Linear-Reduction}{$c$}})
    \EndFor
    \State \textbf{return} $T$
    \EndIf
    \EndProcedure
  \end{algorithmic}
  \caption{The linear reduction algorithm.}
  \label{fig:linear_reduction}
\end{figure}

\subsubsection{Tree Accumulation}
The tree accumulation algorithm is used when MSERs has
more than one child.
Given a MSER tree, the procedure returns a set of
disconnected nodes.
The algorithm works as follows.
For a given node $t$, tree accumulation checks the number of
$t$'s children; if $t$ has no children, return $t$ immediately; if $t$
has more than two children, create an empty set $C$ and
append the result of applying tree accumulation to
$t$'s children to $C$;
if one of the nodes in $C$ has a lower variation
than  $t$'s variation, return $C$, else
discard $t$'s children and return $t$.
Figure~\ref{fig:mser_tree_tree_accumulation} shows the result of
applying tree accumulation to the tree shown in
Figure~\ref{fig:mser_tree_linear_reduction}.
Note that the final result is a set of disconnected nodes
containing all the characters in the original MSER tree.

\begin{figure}[htb!]
  \begin{algorithmic}[1]
    \Procedure{Tree-Accumulation}{$T$}
    \If{nchildren[$T$] $\geq$ 2}
    \State $C \gets \emptyset$
    \For{ \textbf{each} $c$ $\in$ children[$T$]}
    \State $C \gets C \ \cup $ {\Call{Tree-Accumulation}{$c$}}
    \EndFor
    \If{var[$T$] $\leq$ min-var[$C$]}
    \State discard-children($T$)
    \State \textbf{return} $T$
    \Else
    \State \textbf{return} $C$
    \EndIf
    \Else\Comment{nchildren[$T$] = 0}
    \State \textbf{return} $T$
    \EndIf
    \EndProcedure
  \end{algorithmic}
  \caption{The tree accumulation algorithm.}
  \label{fig:tree-accumulation}
\end{figure}

\subsubsection{Complexity Analysis}
\label{section:complexity_analysis}
The linear reduction and tree accumulation algorithm
effectively visit each nodes in the MSRE tree and do simple
comparisons and pointer manipulations, thus the complexity
is linear to the number of tree nodes.
The computational complexity of the variation regularization
is mostly due to the calculations of MSERs' bounding rectangles,
which is up-bounded by the number of pixels in the image.

\subsection{Text Candidates Construction}
\label{section:region_construction}

\subsubsection{Text Candidates Construction Algorithm
  Overview}
Text candidates are constructed by clustering character
candidates using the single-link
clustering algorithm~\cite{clustering}.
Intuitively, single-link clustering produce clusters that
are elongated~\cite{clustering_review} and thus is
particularly suitable for the text candidates construction task.
Single-link clustering belongs to the family of hierarchical
clustering; in hierarchical clustering,
each data point is initially treated as a
singleton cluster and clusters are successively merged
until all points have been merged into a single remaining
cluster.
In the case of single-link clustering, the two clusters
whose two closest
members have the smallest distance are merged in each step.
A distance threshold can be specified such that the clustering
progress is terminated when the distance between nearest
clusters exceeds the threshold.
The resulting clusters of single-link algorithm form
a hierarchical cluster tree or cluster forest if termination
threshold is specified.
In the above algorithm, each data point represent a
character candidate and \emph{top level} clusters in the
final hierarchical cluster tree (forest) correspond to
text candidates.

The problem is of course to determine the distance
function and threshold for the single-link algorithm.
We use the weighted sum of features as the distance
function.
Given two data points $u,v$, let $x_{u,v}$ be the feature
vector characterizing the similarity between $u$ and $v$, the distance
between $u$ and $v$ is defined as
\begin{equation}
  d(u,v;w) = w^T x_{u,v},
  \label{eq:metric00}
\end{equation}
where $w$, the feature weight vector together with the
distance threshold, can be learned using the proposed distance metric learning
algorithm.

In the following subsections, we first introduce the feature
space $x_{u,v}$, then detail the proposed metric learning algorithm
and finally present the empirical analysis on the proposed algorithm.

\subsubsection{Feature Space}
The feature vector $x_{u,v}$ is used to describe
the similarities between data points $u$ and $v$.
Let $x_u, y_u$ be the coordinates of top left corner of $u$'s bounding rectangle,
$h_u, w_u$ be the height and width of the bounding
rectangle of $u$,
$s_u$ be the stroke width of $u$,
$c1_u, c2_u, c3_u$ be the average three channel color value of $u$,
feature vector ${x}_{u,v}$ include the
following features:

\begin{itemize}
\item Spatial distance
  \begin{align*}
    abs(x_u + 0.5 h_u - x_v - 0.5 w_u) / \max(w_u , w_v).
  \end{align*}

\item Width and height differences
  \begin{align*}
    & abs(w_u - w_v) / \max(w_u , w_v), \\
    & abs(h_u - h_v) / \max(h_u , h_v).
  \end{align*}

\item Top and bottom alignments
  \begin{align*}
    & \arctan(\frac{abs(y_u - y_v)} { abs(x_u + 0.5 h_u - x_v - 0.5 w_u)}), \\
    & \arctan(\frac{abs(y_u  + h_u - y_v - h_v)} { abs(x_u + 0.5 h_u - x_v - 0.5 w_u)}).
  \end{align*}

\item Color difference
  \begin{align*}
    \sqrt{(c1_u - c1_v)^2 + (c2_u -
      c2_v)^2 + (c3_u - c3_v)^2}.
  \end{align*}

\item Stroke width difference
  \begin{align*}
    abs(s_u - s_v)
    / \max(s_u, s_v).
  \end{align*}
\end{itemize}

\subsubsection{Distance Metric Learning}

There are a variety of distance metric learning methods~\cite{huang1, huang2, huang3}.
More specifically, many clustering algorithms rely on a good distance metric
over the input space.
One task of semi-supervised clustering is to learn a
distance metric that satisfies the labels or constrains in the
supervised data given the clustering
algorithm~\cite{integrating,xing_metric,klein}.
The strategy of metric learning  is to the learn distance
function by minimizing
distance between point pairs in $\mathcal{C}$ while maximizing
distance between point pairs in $\mathcal{M}$,
where $\mathcal{C}$ specifies pairs of points in different
clusters and $\mathcal{M}$ specifies pairs of points in
the same cluster.
In  single-link clustering, because clusters are formed by merging
smaller clusters, the final resulting clusters will form a
binary hierarchical cluster tree, in which non-singleton
clusters have exactly two direct subclusters.
It is not hard to see that the following
property holds for \emph{top level} clusters: given the termination threshold $\epsilon$,
it follows that distances between each top level cluster' subclusters
are less or equal to $\epsilon$
and distances between data pairs in different top level clusters are
great than $\epsilon$, in which the distance between clusters
is that of the two closest members in each cluster.
This property of single-link clustering enables us to design
a learning algorithm that can learn the distance function and
threshold simultaneously.

Given the top level cluster
set $\{C_k\}_{k=1}^{m}$, we randomly initialize feature weights
$w$ and set $\mathcal{C}$ and $\mathcal{M}$ as
\begin{align}
  & \mathcal{C} = \{(\hat{u}_k,
  \hat{v}_k)= \operatornamewithlimits{\arg min}_{u \in C_k, v \in C_{-k}}d(u,v;w)\}_{k=1}^{m},
  \label{eq:cannot-link} \\
  & \mathcal{M} = \{(u_k^*, v_k^*) =
  \operatornamewithlimits{\arg min}_{u \in C_k^1, v \in C_k^2} d(u,v;w)\}_{k=1}^{m},
  \label{eq:must-link}
\end{align}
where $C_{-k}$ is the set of points excluding points in
$C_k$, $C_k^1$ and $C_k^2$ are direct subclusters of $C_k$.
Suppose $\epsilon$ is specified as the single-link
clustering termination threshold. By the definition of
single-link clustering, we must have
\begin{align}
  & d(u,v;w) > \epsilon \text{ for all } (u,v) \in
  \mathcal{C},
  \label{eq:cannot-link-constain}\\
  & d(u,v;w) \leq \epsilon \text{ for all } (u,v) \in
  \mathcal{M}.
  \label{eq:must-link-constain}
\end{align}
The above equations show that $\mathcal{C}$ and
$\mathcal{M}$ can be corresponded as the positive and
negative sample set of a classification problem, such that
feature weights and threshold can be learned by minimizing
the classification error.
As we know, the logistic regression loss is the traditional loss used in classification
with a high and stable performance.
By adopting the objective function of logistic regression,
we define the following objective function

\begin{align}
 \label{eq:logistic_regression1}
  J(\theta: \mathcal{C}, \mathcal{M}) = & \frac{-1}{2m} ( \sum_{(u,v) \in \mathcal{C}} \log(h_{\theta}(x'_{u,v})) \\ \notag
   & + \sum_{(u,v) \in \mathcal{M}} \log(1-h_{\theta}(x'_{u,v})) ),
\end{align}
where
\begin{align}
  & h_{\theta}(x'_{u,v}) = 1/(1+\exp(-\theta^T x'_{u,v})), \\ \nonumber
  & \theta = \left(
    \begin{array}{c}
      -\epsilon \\ \nonumber
      w
    \end{array}
  \right),\\ \nonumber
  & x'_{u,v} =
  \left(
    \begin{array}{c}
      1 \\ \nonumber
      x_{u,v}
    \end{array}
  \right).
\end{align}
The feature weights $w$ and
threshold $\epsilon$ can be learned simultaneously
by minimizing the objective function $J(\theta: \mathcal{M},
\mathcal{C})$ with respect to current assignment of $\mathcal{C}$ and $\mathcal{M}$
\begin{equation}
\label{eq:logistic_regression2}
\theta^* = \operatornamewithlimits{\arg min}_{\theta} J(\theta: \mathcal{C}, \mathcal{M})
\end{equation}
Minimization of the above objective function is a
typical nonlinear optimization problem and can be solved by
classic gradient optimization methods~\cite{elements_sl_book}.

Note that in the above learning scheme,
initial values for $w$ have to be
specified in order to generate set $\mathcal{C}$ and
$\mathcal{M}$ according to Equation~\eqref{eq:cannot-link}
and~\eqref{eq:must-link}.
For this reason, we design an iterative optimization algorithm in
which each iteration involves two successive steps
corresponding to assignments of $\mathcal{C}, \mathcal{M}$
and optimization with respect to $\mathcal{C}, \mathcal{M}$.
We call our algorithm as ``\emph{self-training distance metric learning}''.
Pseudocode for this learning algorithm is presented in
Figure~\ref{fig:metric-learning}.
Given the top level cluster set $\{C_k\}_{k=1}^{m}$, the
learning algorithm find an optimized $\theta$
such that the objective function $J(\theta:\mathcal{C},
\mathcal{M})$ is minimized with respect to $\mathcal{C}, \mathcal{M}$.
In this algorithm, initial value for $\theta$ is set before
the iteration begins; in the first stage of the iteration $\mathcal{M}$ and
$\mathcal{C}$ are update according to Equation~\eqref{eq:cannot-link}
and~\eqref{eq:must-link} with respect to current assignment
of $\theta$; in the second stage, $\theta$ is updated by
minimizing the objective function with respect to the
current assignment of $\mathcal{C}$ and $\mathcal{M}$.
This two-stage optimization is then repeated until convergence
or the maximum number of iterations is exceeded.
\begin{figure}[htb!]
  \begin{algorithmic}
    \State{\textbf{Input:} labeled clusters set $\{C_k\}_{k=1}^{m}$}
    \State{\textbf{Output:} optimized $\theta$ such that
      objective function $J$ is minimized}
    \State{\textbf{Method:}}
    \State{randomly initialize $\theta$}
    \Repeat
    \State{\textbf{stage1}: update $\mathcal{M}$ and
      $\mathcal{C}$ with respect to $\theta$ }
    \State{\textbf{stage2}: $\theta \gets
      \operatornamewithlimits{\arg min}_{\theta}J(\theta: \mathcal{C}, \mathcal{M})$}
    \Until{convergence or reach iteration limitation}
  \end{algorithmic}
  \caption{The self-training distance metric learning algorithm.}
  \label{fig:metric-learning}
\end{figure}

Similar to most self-training algorithms, convergence of the proposed algorithm is not guaranteed
because the objective function is not assured to decrease in
stage one. However, self-training algorithms have demonstrated their success in many applications. In our case, we find that our algorithm can usually generate very good performance
after a very small number of iterations, typically in 5 iterations. This phenomenon will be investigated in the next subsection.

\subsubsection{Empirical Analysis}
We perform an empirical analysis on the proposed  distance metric learning algorithm.
We labeled in the ICDAR 2011 competition dataset $466$ text candidates corresponding to true
text in the  training set, 70\% of which used as
training data, 30\% as validation data.
In each iteration of the algorithm,
cannot-link set $\mathcal{C}$ and must-link set $\mathcal{M}$ are updated
in step one by generating cannot-link point pairs and must-link
point pairs from true text candidates
in every image in the training dataset;
the objective function are optimized using the L-BFGS
method~\cite{LBFGS} and the parameters are updated in stage two.
Performance of the learned distance weights and threshold in
step two is evaluated on the validation dataset in each
iteration.

As discussed in the previous section, the algorithm may
or may not converge due to different initial values
of the parameters.
Our experiments show that the learned parameters almost always
have a very low error rate on the validation set after the first several iterations and no major
improvement is observed in the continuing iterations.
As a result, whether the algorithm converge or not has no great
impact on the performance of the learned parameters.

We plot the value of the objective
function after stage one and stage two in each iteration of
two instance (correspond to a converged one and not
converged one) of the
algorithm in Figure~\ref{fig:objective}.
The corresponding error rates of the learned parameters on
the validation set in each iteration are plotted in
Figure~\ref{fig:error-rate}.
Notice that  value of the objective function and
validation set error rate dropped immediately after the first several iterations.
Figure~\ref{fig:error-rate} shows that the learned
parameters have different error rates due to different
initial value, which suggests to run
the algorithm several times to get the satisfactory parameters.
The parameters for the single-link clustering algorithm in our scene text detection system are chosen
based on the performance on the validation set.

\begin{figure}[htb!]
  \centering
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[width=\textwidth]{objective-crop.pdf}

    \caption{}
    \label{fig:objective}
  \end{subfigure}
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[width=\textwidth]{error_rate-crop.pdf}

    \caption{}
    \label{fig:error-rate}
  \end{subfigure}
  \caption{
    Objective function value (a) and validation set error
    rate of learned parameters (b)
    after stage one and stage two in each iteration of two instance  of the metric
    learning algorithm; the red line corresponds to the
    not converged instance and the blue line corresponds to
    the converged instance.
  }
  \label{situations}
\end{figure}

\subsection{Text Candidates Elimination}
\label{section:character_classifier}
Using the text candidates construction algorithm proposed in
Section~\ref{section:region_construction},
our experiment in ICDAR 2011 competition training set shows
that only 9\%
of the text candidates correspond to true
text.
As it is hard to train an effective text classifier using such
unbalanced dataset, most of the non-text candidates need to
be removed before training the classifier.
We propose to use a character classifier to estimate the
posterior probabilities of text candidates corresponding to
non-text and remove text candidates with high probabilities for non-text.

The following features are used to train the character classifier.
Smoothness,  defined as the average difference of adjacent
boundary pixels' gradient directions, stroke width features, including the average stroke width and stroke width variation,
height, width, and aspect ratio.
Characters with small aspect ratios such as ``i'', ``j''
and ``l'' are treated as
negative samples, as it is very uncommon that some words
comprise many small aspect ratio characters.

Given a text candidate $T$, let $O(m,n;p)$ be the observation
that there are $m$ ($m \in \mathbb{N}, m \geq 2$) character
candidates in $T$, of which $n$ ($n \in \mathbb{N}, n \leq m$)
are classified as non-characters by a character classifier of
precision $p$ ($0 < p < 1$).
The probability of the observation conditioning on $T$
corresponding to text and non-text are $P(O(m,n;p) |
\text{text}) = p^{m - n} (1 - p)^{n}$ and $P(O(m,n;p) |
\text{non-text}) = (1 - p)^{m - n} p^{n}$ respectively.
Let $P(\text{text})$ and $P(\text{non-text})$ be the prior
probability of $T$ corresponding to text and non-text.
By applying Bayes' rule, the posterior probability of
$T$ corresponding to non-text given the observation is
\begin{align}
  P(\text{non-text} | &O(m,n;p)) = \nonumber \\
  & \frac{P(O(m,n;p) | \text{non-text}) P(\text{non-text})}{P(O(m,n;p))},
\end{align}
where $P(O(m,n;p))$ is the probability of the observation
\begin{align}
  P(O(m,n;p)) &= P(O(m,n;p) | \text{text}) P(\text{text})
  \nonumber \\
  &\qquad {} +  P(O(m,n;p) | \text{non-text})
  P(\text{non-text}),
\end{align}
The candidate region is rejected if
\begin{equation}
  P(\text{non-text} |O(m,n;p)) \geq \varepsilon,
\end{equation}
where $\varepsilon$ is the threshold.

Our experiment shows that text candidates of different sizes
tend to have different probability of being text.
For example, on the ICDAR training set, 1.25\% of text candiates
of size two correspond to text, while 30.67\% of text
candidates of size seven correspond to text, which suggests
to set different priors for text candidates of different
size.
Given a text candidates $T$ of size $s$, let $N_s$ be the total
number of text candidates of size $s$, $N_s^*$ be the number
of text candidates of size $s$ that correspond to text,
we estimate the prior of $T$ being text as $P_s(\text{text}) =
{N_s^*} / {N_s}$, and the prior of $T$ being non-text as
$P_s(\text{non-text}) = 1 - P_s(\text{text})$.
These priors are computed based on statistics on the ICDAR
training dataset.

\begin{figure}[htb!]
  \centering
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[width=\textwidth]{RegionEliminationStatF-crop.pdf}
    \caption{}
    \label{fig:precision_recall}
  \end{subfigure}
  \begin{subfigure}[b]{0.24\textwidth}
    \centering
    \includegraphics[width=\textwidth]{RegionEliminationStat-crop.pdf}
    \caption{}
    \label{fig:elimination}
  \end{subfigure}
  \caption{Performance of different $\varepsilon$ on the
    validation set.
    (a) Precision, recall and $f$ measure of text
    classification task;
    (b) ratio of preserved text samples, ratio of eliminated
    non-text samples and ratio of text samples.
  }\label{fig:varepsilon_perfomance}
\end{figure}

To find the appreciate $\varepsilon$, we used 70\% of ICDAR
training dataset to train the character classifier and text classifier,
the remaining 30\% as validation set to test the performance
of different $\varepsilon$.
Figure~\ref{fig:precision_recall} shows the precision,
recall and $f$ measure of text candidates classification
task on the validation set.
As $\varepsilon$ increases, text candidates are more
unlikely to be eliminated, which results in the increase of
recall value. In the scene text detection task, recall is
preferred over precision, until $\varepsilon = 0.995$ is
reached, where a major decrease of $f$ measure occurred,
which can be explained by the sudden decrease of ratio of
text samples (see Figure~\ref{fig:elimination}).
Figure~\ref{fig:elimination} shows that at $\varepsilon =
0.995$, $92.95\%$ of text are preserved, while $95.25\%$ of
non-text are eliminated.

\section{Experimental Results}
\label{section:experimental_results}

\footnote{
  An online demo of the proposed scene text detection system
  is available at \emph{\url{http://kems.ustb.edu.cn/learning/yin/dtext}.}
}

In this section, we presented the experimental results of
the proposed scene text detection method on two publicly
available benchmark datasets, ICDAR 2011 Robust
Reading Competition dataset
\footnote{The ICDAR 2011 Robust
Reading Competition dataset is available at
\emph{\url{http://robustreading.opendfki.de/wiki/SceneText}}.}
and the multilingual dataset
\footnote{The multilingual dataset is available at \emph{\url{http://liama.ia.ac.cn/wiki/projects:pal:member:yfpan}}.}
provided by Pan et al.~\cite{pan}.

\subsection{Experiments on ICDAR 2011 Competition Dataset }
\label{section:icdar2011}

The ICDAR 2011 Robust Reading Competition (Challenge 2: Reading Text in
Scene Images) dataset ~\cite{icdar2011} is a widely used
dataset for benchmarking scene text detection algorithms.
The dataset contains $229$ training images
and $255$ testing images.
The proposed system is trained on the
training set and evaluated on the testing set.

It is worth noting that the evaluation scheme of ICDAR 2011
competition is not the same as of ICDAR 2003 and ICDAR
2005.
The new scheme, the \emph{object count/area} scheme
proposed by Wolf et al.~\cite{Wolf_Jolion_2006}, is more
complicated but offers
several enhancements over the old scheme.
Basically, these two scheme use the notation of precision,
recall and $f$ measure that is defined as
\begin{align}
  & recall = \frac{\sum_{i=1}^{|G|} match_G(G_i)}{|G|}, \\
  & precision = \frac{\sum_{j=1}^{|D|} match_D(D_j)}{|D|}, \\
  & f = 2 \frac{recall \cdot precision}{recall + precision},
\end{align}
where $G$ is the set of groundtruth rectangles and $D$ is
the set of detected rectangles.
In the old evaluation scheme, the matching functions are defined
as
\begin{align}
  & match_G(G_i) = \max_{j = 1...|D|} \frac{2 \cdot area(G_i
    \cap D_j)}{area(G_i) + area(D_j)}, \\
  & match_D(D_j) = \max_{i = 1...|G|} \frac{2 \cdot area(D_j
    \cap G_i)}{area(D_j) + area(G_i)}.
\end{align}
The above matching functions only consider one-to-one matches
between groundtruth and detected rectangles, leaving room
for ambiguity between detection quantity and
quality~\cite{Wolf_Jolion_2006}.
In the new evaluation scheme, the
matching functions are redesigned considering detection quality
and different matching situations (one-to-one matchings,
one-to-many matchings and many-to-one matchings) between groundtruth
rectangles and detected rectangles, such that the detection
quantity and quality can both be observed using the new
evaluation scheme.
The evaluation software DetEval \footnote
{DetEval is available at \emph{\url{http://liris.cnrs.fr/christian.wolf/software/deteval/index.html}}.}
used by ICDAR 2011 competition is available online and free to use.

The performance of our system, together with Neumann and Matas'
method~\cite{real_time}, a very recent MSER based method by
Shi et al.~\cite{mser2013}
and some of the top
scoring methods
(Kim's method, Yi's method, TH-TextLoc system and Neumann's method)
from ICDAR 2011 Competition
are presented in
Table~\ref{table:performance}.
As can be seen from Table~\ref{table:performance},
our method produced much better recall, precision and
$f$ measure over other methods on this dataset.
It is worth noting that the first four methods in
Table~\ref{table:performance} are all MSER based methods and
Kim's method is the winning method of ICDAR 2011 Robust
Reading Competition.
Apart from the detection quality, the proposed system offers
speed advantage over some of the listed methods.
The average processing speed of the proposed system on a
Linux laptop with Intel (R) Core (TM)2 Duo 2.00GHZ CPU is 0.43s
per image.
The processing speed of Shi et al.'s method~\cite{mser2013}
on a PC with Intel (R) Core (TM)2 Duo 2.33GHZ CPU is 1.5s
per image.
The average processing speed of Neumann and Matas'
method~\cite{real_time} is 1.8s per image on a ``standard
PC''.
Figure~\ref{fig:samples_icdar} shows some text detection
examples by our system on ICDAR 2011 dataset.

\begin{figure}[htb!]
  \centering
  \includegraphics[width=0.5\textwidth]{samples_icdar.pdf}
  \caption{Text detection examples on the ICDAR 2011
    dataset.
    Detected text by our system are labeled using red rectangles.
    Notice the robustness against low contrast, complex
    background and font variations.}
  \label{fig:samples_icdar}
\end{figure}

\begin{table}[htb!]
  \centering
  \caption{Performance ($\%$) comparison of text detection algorithms on ICDAR 2011 Robust Reading Competition dataset.}
  \label{table:performance}
  \begin{tabular} {|c|c|c|c|}
    \hline
    Methods                            & Recall         & Precision      & $f$            \\ \hline
    \textbf{Our Method}               & \textbf{68.26} & \textbf{86.29} & \textbf{76.22} \\ \hline
    Shi et al.'s method~\cite{mser2013} & 63.1          & 83.3           & 71.8           \\ \hline
    Kim's Method (not published)      & 62.47          & 82.98          & 71.28          \\ \hline
    Neumann and Matas~\cite{real_time} & 64.7           & 73.1           & 68.7           \\ \hline
    Yi's Method                       & 58.09          & 67.22          & 62.32          \\ \hline
    TH-TextLoc System                 & 57.68          & 66.97          & 61.98          \\ \hline
    Neumann's Method                  & 52.54          & 68.93          & 59.63          \\ \hline
  \end{tabular}
\end{table}

To fully appreciate the benefits of
\emph{text candidates elimination}
and \emph{the MSERs pruning algorithm},
we further profiled the proposed system on this dataset using the following
schemes (see Table~\ref{table:component_profile})

1) \textbf{Scheme-I}, no text candidates elimination
performed. As can be seen from
Table~\ref{table:component_profile}, the absence of text
candidates elimination results in a major decrease in
precision value.
The degradation can be explained by the fact that
large number of non-text are
passed to the text candidates classification stage without
being eliminated.

2) \textbf{Scheme-II}, using default parameter setting~\cite{vlfeat} for the
MSER extraction algorithm.
The MSER extraction algorithm is controlled by several
parameters~\cite{vlfeat}: $\Delta$ controls how the variation is
calculated; maximal variation $v_{+}$ excludes too unstable MSERs;
minimal diversity $d_{+}$ removes duplicate MSERs by measuring the size difference
between a MSER and its parent.
As can be seen from Table~\ref{table:component_profile},
compared with our parameter setting ($\Delta = 1, v_+=0.5, d_+=0.1$),
the default parameter setting ($\Delta = 5, v_+=0.25,
d_+=0.2$) results in a major decrease in recall value.
The degradation can be explained by two reasons:
(1) the MSER algorithm is not able to detect some
low contrast characters (due to $v_+$), and
(2) the MSER algorithm tends to miss some regions that are
more likely to be characters (due to $\Delta$ and $d_+$).
Note that the speed loss (from 0.36 seconds to 0.43 seconds)
is mostly due to the MSER detection algorithm itself.

\begin{table}[htb!]
  \centering
  \caption{Performance (\%) of the proposed method due to
    different components}
  \label{table:component_profile}
  \begin{tabular} {|c|c|c|c|c|}
    \hline
    Component          & Recall & Precision & $f$   & Speed (s) \\ \hline
    Overall system     & 68.26  & 86.29     & 76.22 & 0.43      \\ \hline
    \textbf{Scheme-I}  & 65.57  & 77.49     & 71.03 & 0.41      \\ \hline
    \textbf{Scheme-II} & 61.63  & 85.78     & 71.72 & 0.36      \\ \hline
  \end{tabular}
\end{table}

\subsection{Experiments on Multilingual Dataset}
The multilingual (include Chinese and English, see
Figure~\ref{fig:samples_multilingual}) dataset was
initially published by Pan et al.~\cite{pan} to evaluate the
performance of their scene text detection system.
The training dataset contains $248$ images and the testing
dateset contains $239$ images.
As there are no apparent spacing between Chinese word,
this multilingual dataset only provides groundtruths for
text lines.
We hence evaluate the text line detection performance of
our system without further partitioning text into words.
Figure~\ref{fig:samples_multilingual} shows some scene text
detection examples by our system on this dataset.

\begin{table}[htb!]
  \centering
  \caption{Performance ($\%$) comparison of text detection
    algorithms on the multilingual dataset.
    Speed of Pan et al.'s method is profiled on a
    PC with Pentium D 3.4GHz CPU.

  }
  \label{table:performance_multilingual}
  \begin{tabular} {|c|c|c|c|c|}
    \hline
    Methods                       & Recall  & Precision & $f$     & Speed (s)  \\ \hline
    \textbf{Scheme-III}                    & {63.23} & {79.38}   & {70.39} & {0.22} \\ \hline
    \textbf{Scheme-IV}                   & {68.45} & {82.63}   & {74.58} & {0.22} \\ \hline
    Pan et al.'s method~\cite{pan} & 65.9    & 64.5      & 65.2    & 3.11   \\ \hline
  \end{tabular}
\end{table}

\begin{figure}[htb!]
  \centering
  \includegraphics[width=0.5\textwidth]{samples_multilingual.pdf}
  \caption{Text detection examples on the multilingual
    dataset. Detected text by our system are labeled using red rectangles.

  }
  \label{fig:samples_multilingual}
\end{figure}

The performance of our system (include \textbf{Scheme-III} and
\textbf{Scheme-IV}) and Pan et al.'s method~\cite{pan} is presented in
Table~\ref{table:performance_multilingual}.
The evaluation scheme in ICDAR 2003 competition (see
Section~\ref{section:icdar2011}) is used for fair comparison.
The main difference between \textbf{Scheme-III} and
\textbf{Scheme-IV} is that
the character classifier in the first scheme is trained on
the ICDAR 2011 training set while the character classifier in
the second scheme is trained on the multilingual training set
(character features for training the classifier are the same).
The result comparison between \textbf{Scheme-III} and
\textbf{Scheme-IV} in
Table~\ref{table:performance_multilingual}
shows that the performance of the proposed system is
significantly improved because of the incorporating of the
Chinese-friendly character classifier.
The basic implication of this improvement is that the
character classifier has a significant
impact on the performance of the overall system, which
offers another advantage of the proposed system: the
character classifier can be trained on desired dataset until
it is accurate enough and be plugged into the system and
the overall performance will be improved.
Table~\ref{table:performance_multilingual} also shows the
advantages of the proposed method over Pan et al.'s method
in detection quality and speed.

\section{Conclusion}
\label{section:conclusion}
This paper presents a new MSER based scene text
detection method.
Several key improvement over traditional methods have been
proposed.
We propose a fast and accurate MSERs pruning algorithm that
enables us to detect most the characters even when the image is
in low quality.
We propose a novel self-training distance metric learning algorithm that can
learn distance weights and threshold simultaneously;
text candidates are constructed by clustering character
candidates by the single-link algorithm using the learned
parameters.
We propose to use a character classifier to estimate the
posterior probability of text candidate corresponding to non-text and
eliminate text candidates with high probability for non-text, which helps
to build a more powerful text classifier.
By integrating the above ideas, we built a robust scene text
detection system that exhibited superior performance over
state-of-the-art methods on both the ICDAR 2011 Competition dataset and a multilingual dataset.

\ifCLASSOPTIONcompsoc

  \section*{Acknowledgments}
\else

  \section*{Acknowledgment}
\fi

The research was partly supported by National Basic Research Program of China (2012CB316301)
and National Natural Science Foundation of China (61105018, 61175020).

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\bibliographystyle{IEEEtran}


\section{Introduction}
Network models of stock market have attracted a large attention in theoretical and applied
research. Different graph structures related with stock market network are considered in the literature \cite{Boginski_2006}. One of such graph structure, maximum spanning tree (MST), is a popular tool in market network analysis. Many papers are devoted to the use of MST for particular stock markets (see recent papers  \cite{Walle_2018}, \cite{Sensoya_2014}, \cite{Wang_2012} and exhaustive bibliography in \cite{Marti_Arxive}). However, much less attention is payed in the literature to the estimation of uncertainty of obtained results. One particular way to measure uncertainty is related with bootstrap technique applied to the observed data \cite{Tumminello_2007}. In the present paper we suggest a general theoretical framework to measure uncertainty of MST identification. We study uncertainty in the framework of the concept of random variable network \cite{Kalyagin_2020}.

Random variable network (RVN) is a pair $(X,\gamma)$, where $X=(X_1,X_2,\ldots,X_N)$ is a random vector and $\gamma$ is a measure of similarity between pairs of random variables. This concept allows to introduce the {\it true MST} associated with RVN. We call true MST the maximum spanning tree in the complete weighted graph $(V,\Gamma)$, where $V=\{1,2,\ldots,N\}$ is the set of nodes (vertices), and $\Gamma=(\gamma_{i,j})$ is the matrix of weights, $\gamma_{i,j}=\gamma(X_i,X_j)$, $i,j=1,2,\ldots,N$, $i \neq j$, $\gamma_{i,j}=0$ for $i=j$. To model distribution of the vector $X$ we use a large class of elliptical distributions, which is widely used in applied finance \cite{Gupta_2013}. To measure similarity between stock's we consider different correlation networks for the stock's returns: Pearson correlation network, Fechner correlation network, and Kendall correlation network. Pearson correlation is most used in market network analysis. We show in the paper that for elliptical distributions the true  MST in Fechner and Kendall correlation networks are the same as the true MST in Pearson correlation network for Gaussian distribution. This fact gives a theoretical basis for correct comparison of uncertainty of MST identification algorithms in different networks.

Uncertainty of MST identification in our setting is related with the difference between true MST and MST identified from observations. To assess uncertainty of MST identification we analyze different error rates known in multiple testing and binary classification. We argue that the most appropriate error rate for MST identification is the well known False Discovery Rate (FDR). In our case FDR is the proportion of false edges (non correctly identified edges) in MST. We investigate FDR of Kruskal algorithm for MST identification  and show that reliability of MST identification is different in three correlation networks. We emphasize that for Pearson correlation network the FDR essentially depends on distribution of stock returns. We prove that for  Fechner correlation network the FDR is non sensitive to the assumption on stock's return distribution. New and surprising phenomena are discovered for   Kendall correlation network. Our experiments show that FDR of Kruskal algorithm for MST identification in Kendall correlation network weakly depend on distribution and at the same time the value of FDR is almost the best in comparison with MST identification in other networks. These facts are important in practical applications.

The paper is organized as follows. In Section \ref{Basic definitions and notations} we present necessary definitions and notations. In Section \ref{Connection} we prove that MST is the same in three correlation networks for a large class of elliptical distributions. In Section \ref{Uncertainty of MST} we discuss measures of uncertainty of MST identification. The Section \ref{Kruskal algorithm for MST identification} is devoted to the description of Kruskal algorithms for MST identification in different correlation networks. In Section \ref{Robustness of Kruskal algorithm} we prove robustness of Kruskal algorithm of MST identification in Fechner correlation network. In Section \ref{Reliability of Kruskal algorithm} we present the results of numerical investigation of reliability of Kruscal algorithm in different correlation networks. The Section \ref{Concluding remarks} summarizes the main results of the paper and discusses a further research.

\section{Basic definitions and notations.}\label{Basic definitions and notations}

Random variable network is a pair $(X,\gamma)$, where $X=(X_1,\ldots,X_N)$ is a random vector, and $\gamma$ is a pairwise measure of similarity between random variables.
One can consider different random variable networks associated with different distributions of the random vector $X$ and different measures of similarity $\gamma$. For example, the Gaussian Pearson correlation network is the random variable network, where $X$ has a multivariate Gaussian distribution and $\gamma$ is the Pearson correlation. On the same way one can consider the  Gaussian partial correlation network, the Gaussian Kendall correlation network, the Student Pearson correlation network and so on.

The random variable network generates a network model. Network model for random variable network $(X, \gamma)$ is the complete weighted graph $(V,\Gamma)$ with $N$ nodes , where $V=\{1,2,\ldots,N\}$ is the set of nodes, $\Gamma=(\gamma_{i,j})$ is the matrix of weights, $\gamma_{i,j}=\gamma(X_i,X_j)$. The spanning tree in the network model $(V,\Gamma)$ is a connected graph $(V,E)$ without cycles. Weight of the  spanning tree $(V,E)$ is the sum of weights of its edges $\sum_{(i,j) \in E} \gamma_{i,j}$. Maximum spanning tree (MST) is the spanning tree with maximal weight. In what follows we consider MST as unweighted  graph. MST obtained in this way will be called {\it true MST} or {\it MST in true network model}. There are known many algorithms to calculate the minimum spanning tree in an undirected weighted graph \cite{Gross_2006}. All of them can be easily transformed onto algorithms to calculate the maximum spanning tree. In this paper we use classical Kruscal algorithm:

\vskip 2mm
\noindent
{\bf Kruskal algorithm for calculation of the true MST:} the Kruskal algorithm calculates the collection of edges $MST$ of maximum spanning tree in the network model $(V,\Gamma)$ by the following steps
\begin{itemize}
\item Sort the edges of the complete weighted graph $(V,\Gamma)$ into decreasing order by weights $\gamma_{i, j}$.
\item Add the first edge to $MST$.
\item Add the next edge to $MST$ if and only if it does not form a cycle in the current $MST$.
\item If $MST$ has $(N-1)$ edges, stop and output $MST$ . Otherwise go to the previous step.
\end{itemize}

We consider three correlation networks, Pearson correlation network, Fechner correlation network, and Kendall correlation network with elliptical distribution of the vector $X$.
Pearson correlation network is a random variable network with Pearson correlation as the measure of similarity $\gamma=\gamma^P$
\begin{equation}\label{Pearson measure}
\gamma^P_{i,j}=\gamma^P(X_i,X_j)=\frac{Cov(X_i,X_j)}{\sqrt{Cov(X_i,X_i)}\sqrt{Cov(X_j,X_j)}}
\end{equation}
Fechner correlation network is a random variable network with Fechner correlation as the measure of similarity $\gamma=\gamma^{Fh}=2 \gamma^{Sg}-1$
where $\gamma^{Sg}$ is so-called sign similarity
\begin{equation}\label{Sign similarity}
\gamma^{Sg}_{i,j}=\gamma^{Sg}(X_i,X_j)=P\{(X_i-E(X_i))(X_j-E(X_j))>0\}
\end{equation}
\begin{equation}\label{Fechner correlation}
\gamma^{Fh}_{i,j}=2\gamma^{Sg}_{i,j}-1
\end{equation}
Kendall correlation network is a random variable network with Kendall correlation as the measure of similarity $\gamma=\gamma^{Kd}$
\begin{equation}\label{Kendall correlation}
\gamma^{Kd}_{i,j}=\gamma^{Kd}(X_i,X_j)=2P\{(X_i^{(1)}-X_i^{(2)})(X_j^{(1)}-X_j^{(2)})>0\}-1
\end{equation}
where $(X_i^{(1)}, X_j^{(1)})$, $(X_i^{(2)}, X_j^{(2)})$ are two independent random vectors with the same distribution as the vector $(X_i,X_j)$ (see \cite{Kruskal_1958}).

Random vector $X$ belong to the class of  elliptically contoured distributions (elliptical distributions) if its  density function has the form  \cite{Anderson_2003}:
\begin{equation}\label{density_of_elliptical_distribution}
f(x; \mu, \Lambda)=|\Lambda|^{-\frac{1}{2}}g\{(x-\mu)'\Lambda^{-1}(x-\mu)\}
\end{equation}
where $\Lambda=(\lambda_{i,j})_{i,j=1,2,\ldots,N}$ is positive definite symmetric matrix, $g(x)\geq 0$, and
$$
\int_{-\infty}^{\infty}\ldots\int_{-\infty}^{\infty}g(y'y)dy_1 dy_2\cdots dy_N=1
$$
This class includes in particular multivariate Gaussian distribution
$$
f_{Gauss}(x)=\frac{1}{(2\pi)^{N/2}|\Lambda|^{\frac{1}{2}}}e^{-\frac{1}{2}(x-\mu)'\Lambda^{-1}(x-\mu)}
$$
and multivariate Student distribution with $\nu$ degree of freedom
$$
f_{Student}(x)=\frac{\Gamma\left(\frac{\nu+N}{2}\right)}{\Gamma\left(\frac{\nu}{2}\right)\nu^{N/2}\pi^{N/2}}|\Lambda|^{-\frac{1}{2}}\left[1+\frac{(x-\mu)'\Lambda^{-1}(x-\mu)}{\nu}\right]^{-\frac{\nu+N}{2}}
$$
The class of elliptical distributions is a natural generalization of the class of Gaussian distributions. Many properties of Gaussian distributions have analogs for elliptical distributions, but this class is much larger, in particular it includes distributions with heavy tails. For detailed investigation of elliptical distributions see  \cite{Fang_1990}, \cite{Anderson_2003}, \cite{Gupta_2013}. It is known that if $E(X)$ exists then $E(X)=\mu$. One important property of elliptical distributions is the connection between covariance matrix of the vector $X$ and the matrix $\Lambda$. Namely, if covariance matrix exists one has
\begin{equation}\label{Covariance_Lambda}
\sigma_{i,j}=Cov(X_i, X_j)=C \cdot \lambda_{i,j}
\end{equation}
where
$$
C=\frac{2\pi^{\frac{1}{2}N}}{\Gamma(\frac{1}{2}N)}\int_0^{+\infty}r^{N+1}g(r^2)dr
$$
In particular, for Gaussian distribution one has $Cov(X_i, X_j)=\lambda_{i,j}$. For multivariate Student distribution with $\nu$ degree of freedom ($\nu>2$) one has $\sigma_{i,j}=\nu/(\nu-1)\lambda_{i,j}$.

\section{Connection between random variable networks}\label{Connection}
There is a connection between  three networks for the vector $X$ with elliptical distribution with the same matrix $\Lambda$. Let $\Lambda$ be a fixed  positive definite matrix of dimension $(N \times N)$. Denote by $K(\Lambda)$ the class of distributions such that its  density function has the form (\ref{density_of_elliptical_distribution}). The following statement holds.
\begin{thm}
Let $X \in K(\Lambda)$. If covariance matrix of $X$ exists, then the true MST in Pearson, Fechner, and Kendall correlation networks is the same for any network and any distribution of the vector $X$. This MST coincides with the true MST for multivariate Gaussian distribution with the covariance matrix $\Lambda$.
\end{thm}

\noindent
{\bf Proof.}  Let $X$ be a random vector with  elliptical distribution (\ref{density_of_elliptical_distribution}) with the matrix $\Lambda$ and the function $g(u)$. The relation (\ref{Covariance_Lambda}) implies that
$$
\gamma^P(X_i,X_j)=\displaystyle \frac{\lambda_{i,j}}{\sqrt{\lambda_{i,i}\lambda_{j,j}}},
$$
that is $\gamma^P(X_i,X_j)$ does not depend on the function $g(u)$ and are defined by the matrix $\Lambda$ only. We will prove that this is true for Fechner and Kendall correlations too. This fact is proved for the sign similarities $\gamma_{i,j}^{Sg}$ in \cite{Kalyagin_2017}, Lemma 1 and Lemma 2. Therefore it is true for Fechner correlations $\gamma^{Fh}_{i,j}=2\gamma_{i,j}^{Sg}-1$. Moreover it is proved in \cite{Kalyagin_2017} that
$$
\gamma_{i,j}^{Sg}=\displaystyle \frac{1}{2}+\frac{1}{\pi}\arcsin (\gamma^P_{i,j})
$$
For Kendall correlations consider two  independent random vectors $X^{(1)}$, $X^{(2)}$  with the same distribution as the vector $X$. It can be easy proved that in this case the random vector $(X^{(1)}-X^{(2)})$ has elliptical distribution \cite{Lindskog_2003}. Calculation of the covariance matrix for this vector implies
$$
Cov(X^{(1)}_i-X^{(2)}_i, X^{(1)}_j-X^{(2)}_j)=2 Cov(X_i,X_j)=2C\lambda_{i,j}
$$
Therefore
$$
\gamma_{i,j}^{Kd}=2\gamma^{Sg}(X^{(1)}_i-X^{(2)}_i,X^{(1)}_j-X^{(2)}_j)-1=\displaystyle \frac{2}{\pi}\arcsin(\frac{\lambda_{i,j}}{\sqrt{\lambda_{i,i}\lambda_{j,j}}})
$$
It implies that Kendall correlations don't depend on the function $g(u)$.
Moreover the following relations  hold for any distribution from the class $K(\Lambda)$:
$$
\gamma^{Fh}_{i,j}=\gamma^{Kd}_{i,j}=\displaystyle \frac{2}{\pi}\arcsin(\gamma^P_{i,j}).
$$

To calculate the true MST in each of three networks one can use Kruskal algorithm. The first step of the algorithm is to sort the edges of the complete weighted graph $(V,\Gamma)$ into decreasing order by weights $\gamma_{i, j}$. Note, that $\gamma^{Fh}_{i,j}$, $\gamma^{Kd}_{i,j}$ are obtained from  $\gamma^P_{i,j}$ by increasing function. It means that the first step of the Kruskal algorithm will give the same edge ordering for all three networks. Next steps of the algorithm depends only on this ordering  and does not depend on a particular values of the weights of edges. Therefore the true maximum spanning tree (true MST) is the same in all networks for any distribution of the vector $X \in K(\Lambda)$. True MST for multivariate Gaussian distribution with the covariance matrix $\Lambda$ is a particular case of such MST.

This statement gives a basis for a correct comparison of reliability of Kruskal algorithm for MST identification in different correlation networks.

\noindent
{\bf Remark:} The relation between Kendall and Pearson correlation for elliptical distributions is known \cite{Fang_2002}, \cite{Lindskog_2003}. We give here a
sketch of proof by the sake of completeness.

\section{Uncertainty of MST identification in random variable network}\label{Uncertainty of MST}
The main problem under discussion in this paper is a reliability of the identification of the true MST from observations. Let $(X,\gamma)$ be a random variable network and $(V,\Gamma)$ be the associated network model. True maximum spanning tree (true MST) is the spanning tree in  $(V,\Gamma)$ with maximal weight.  Let $X(t)$, $t=1,2,\ldots, n$ be a sample from distribution of $X$. Denote by $x(t)$ observed value of the random vector $X(t)$. Sample space is defined by the matrices $x=(x_j(t)) \in R^{N \times n}$.  We define the decision space as the space of all adjacency matrices $S$ of the spanning trees in $(V,\Gamma)$:
$$
{\cal{D}}=\{S : \ S \in R^{N \times N}, S \ \mbox{ is adjacency matrix for a spanning tree in} \ (V,\Gamma) \}
$$
Any MST identification algorithm $\delta=\delta(x)$ is a map from the sample space $R^{N \times n}$ to the decision space $\cal{D}$.
Quality of an identification algorithm $\delta$ is related with the difference between true maximum spanning tree  and the spanning tree given by $\delta$ which can be evaluated by a loss function $w(S,Q)$ where $S=(s_{i,j})$ is the true decision and $Q=(q_{i,j})$ is the decision given by $\delta$.
Uncertainty of an identification algorithm $\delta$ is then measured by the expected value of the loss function, which is known as the risk function
\begin{equation}\label{risk_function}
Risk(S; \delta)= \sum_{Q \in \cal{D}} w(S,Q)P(\delta=Q)
\end{equation}
The choice of the loss function is an important point for uncertainty evaluation. To discuss an appropriate choice of the loss function for MST identification we consider the following tables familiar in binary classification. Table \ref{s_ij_q_ij} illustrates Type I and Type II errors for the individual edge $(i,j)$. It represents all possible cases for different values of $s_{i,j}$ and $q_{i,j}$. Value $0$ means that the edge $(i,j)$ is not included in the MST,
value $1$ means that the edge $(i,j)$ is  included in the MST. We associate the case $s_{i,j}=0$, $q_{i,j}=1$ with Type I error (false edge inclusion), and we associate the case $s_{i,j}=1$, $q_{i,j}=0$ with Type II error (false edge non inclusion).

\begin{table}[h!]
\begin{center}
  \begin{tabular}{|c|c|c|}
\hline
  &  &  \\
$q_{i,j}$   $s_{i,j}$ & 0 & 1 \\
  &  &  \\
\hline
  &  &  \\
0 & \ edge is not included correctly & Type II error \  \\
  &  &  \\
\hline
  &  &  \\
1 & \ Type I error & edge is included correctly \ \\
  &  &  \\
\hline
  \end{tabular}
	\end{center}
 \caption{Type I (false edge inclusion) and Type II (false edge exclusion) errors for the edge $(i,j)$}\label{s_ij_q_ij}
 \end{table}

Table \ref{TN_TP} represents the numbers of Type I errors (False Positive), number of Type II errors (False Negative), and numbers of correct decisions (True Positive and True Negative). This table has a specific properties for the numbers of errors in MST identification. First, number of edges in any spanning tree is equal to $(N-1)$,
that is $FP+TP=N-1$, $FP+TN=M-(N-1)$, where $M=C^2_N$. Second, one false included edge implies one false excluded edge and vice versa, that is $FP=FN$. In addition one has $0 \leq FP \leq N-1$, $M-2(N-1) \leq TN \leq M-(N-1)$.

 \begin{table}[h!]
\begin{center}
  \begin{tabular}{|c|c|c|c|}
\hline
  &  &  & \\
$Q \ S$ & 0 in $S$ & 1 in $S$ & Total\\
  &  &  & \\
\hline
  &  &  & \\
0 in $Q$ &  TN & FN  &  number of 0 in $Q$ \\
  &  &  & \\
\hline
  &  &  & \\
1 in $Q$ & FP & TP & number of 1 in $Q$ \ \\
  &  &  & \\
\hline
  &  &  & \\
Total & number of 0 in $S$ & number of 1 in $S$ & $N(N-1)/2$ \\
  &  &  & \\
\hline
  \end{tabular}
\end{center}
 \caption{Numbers of Type I and Type II errors for MST identification}\label{TN_TP}
 \end{table}

Now we discuss the choice of the loss and risk functions appropriate for the MST identification by observations. The most simple loss function is
$$
w_{Simple}(S,Q)=
\left \{
\begin{array}{lll}
1 & if & S \neq Q \\
0 & if & S = Q \\
\end{array}
\right.
$$
The associated risk is the probability of the false decision $Risk(S; \delta)=P(\delta(x) \neq S)$. For MST identification it is the same as
FWER (Family Wise Error Rate), known in multiple hypotheses testing. FWER is the probability of at least one Type I error \cite{Hochberg_1987}, \cite{Bretz_2011}, \cite{Lehmann_2005}. The true MST is correctly identified if and only if $FP=FN=0$. This measure of uncertainty takes into account only the fact of correct identification of MST (no errors) and it does not take into account the number of errors. Moreover, one can show by simulations, that the probability of correct decision for MST identification is very small even if the number of observations is big \cite{Kalyagin_2014_1}.

Another error rates such as Conjunctive Power (CPOWER) and Disjunctive Power (DPOWER), known in multiple hypotheses testing, are related with the Type II errors \cite{Bretz_2011}. In the case of MST identification these error rates are connected with FWER. In particular one has $CPOWER=FWER$. Therefore it does not give a new measure of uncertainty.

Considered measures of uncertainty  don't take into account the numbers of errors. In multiple hypotheses testing there are error rates which take into account the numbers of errors:  Per-Family Error Rate (PFER), Per-Comparison Error Rate (PCER), Average Power (AVE), or True Positive Rate (TPR). PFER is defined as the expected  number of Type I errors. Associated loss function can be defined as $w_{PFER}=FP$. PCER is defined by $PCER= PFER/M$, $M=N(N-1)/2$.  Loss function for the Average Power (AVE) is defined by $w_{AVE}=(TP/(FN+TP))$. In binary classification $Risk_{AVE}$ is related with True Positive Rate (TPR), or Sensitivity, or Recall.

For  MST identification all these uncertainty characteristics are related with False Discovery Rate (FDR).
FDR is defined by the loss function $w_{FDR}=(FP/(FP+TP))$. One has in our case $PFER=(N-1)FDR$, $PCER=2FDR/N$, $AVE=1-FDR$, $TPR=Recall=Precision=1-FDR$.
In addition, one has
$$
0 \leq TPR \leq 1, \ \ 0 \leq FPR \leq \frac{FP}{M-(N-1)}=\frac{2}{N-2}.
$$

Another measure of error in binary classification is Accuracy (ACC), or proportion of correct decisions. It is defined  by the following loss function $w_{ACC}=(TP+TN)/M$, $M=N(N-1)/2$. This measure is related with FDR by the formula
$$
ACC= \displaystyle 1- \frac{4FDR}{N}
$$
ACC is not well appropriate for MST  identification because for a large $N$ ACC is close to 1, independently of the number of errors.

Taking into account the above discussion we argue that FDR is an appropriate measure of uncertainty for MST identification. Note that for MST identification FDR is the proportion of false edges (non correctly identified edges) in MST.

\section{Kruskal algorithm for MST identification}\label{Kruskal algorithm for MST identification}

Let $(X,\gamma)$ be the random variables network where $X=(X_1,\ldots,X_N)$ be the random vector and $\gamma$ be the pairwise measure of dependence. Let $x_i(t),i=1,\ldots,N;t=1,\ldots,n$ be the observations of $X$ and $\hat{\gamma}_{i,j}$ be the estimations of the $\gamma_{i,j}$ constructed by observations $x_i(t),i=1,\ldots,N;t=1,\ldots,n$, $\hat{\Gamma}=(\hat{\gamma}_{i,j})$. Kruskal  algorithm for MST identification can be described as follows.

\vskip 2mm
\noindent
{\bf Kruskal algorithm for MST identification by observations:} the Kruskal algorithm calculates the collection of edges $\hat{MST}$ of maximum spanning tree in the network model $(V,\hat{\Gamma})$ by the following steps
\begin{itemize}
\item Sort the edges of the complete weighted graph $(V,\hat{\Gamma})$ into decreasing order by weights $\hat{\gamma}_{i,j}$.
\item Add the first edge to $\hat{MST}$.
\item Add the next edge to $\hat{MST}$ if and only if it does not form a cycle in the current $\hat{MST}$.
\item If $\hat{MST}$ has $(N-1)$ edges, stop and output $\hat{MST}$ . Otherwise go to the previous step.
\end{itemize}

Kruskal algorithm for MST identification in Pearson correlation network uses the classical estimations of Pearson correlations (sample Pearson correlations):
\begin{equation}\label{sample_pearson_measure}
 \hat{\gamma}^P_{i,j}=r_{i,j}=\frac{\sum_{t=1}^n(x_i(t)-\overline{x_i})(x_j(t)-\overline{x_j})}{\sqrt{\sum_{t=1}^n(x_i(t)-\overline{x_i})^2\sum_{t=1}^n(x_j(t)-\overline{x_j})^2}}
\end{equation}

\vskip 2mm
Kruskal algorithm for MST identification in Fechner correlation network uses the following estimations of Fechner correlations (sample Fechner correlations):
$$
\displaystyle \hat{\gamma}^{Fh}_{i,j}=2 \hat{\gamma}^{Sg}_{i,j}-1
$$
where $\hat{\gamma}^{Sg}_{i,j}$ are estimations of sign similarities. These estimations are given by
\begin{equation}\label{sample_sign_measure}
\hat{\gamma}^{Sg}_{i,j}=\frac{1}{n}\sum_{t=1}^nI_{i,j}(t)
\end{equation}
with
$$
I_{i,j}(t)=\left\{\
							\begin{array}{ll}
							  0,& (x_i(t)-\overline{x_i})(x_j(t)-\overline{x_j})\leq 0\\
								1,& (x_i(t)-\overline{x_i})(x_j(t)-\overline{x_j})>0\\
							\end{array}
						\right.
$$
where
$$
\overline{x_i}=\displaystyle \frac{1}{n}\sum_{t=1}^n x_i(t), \ \ i=1,2,\ldots,N
$$
In the case when the vector of means $\mu$ is known one can calculate $I_{i,j}(t)$ by
$$
I_{i,j}(t)=\left\{\
							\begin{array}{ll}
							  0,& (x_i(t)-\mu_i)(x_j(t)-\mu_j)\leq 0\\
								1,& (x_i(t)-\mu_i)(x_j(t)-\mu_j)>0\\
							\end{array}
						\right.
$$

\vskip 2mm
Kruskal algorithm for MST identification in Kendall correlation network uses the following estimations of Kendall correlations
\begin{equation}\label{sample_kendall_measure}
 \hat{\gamma}^{Kd}_{i,j}=\frac{1}{n(n-1)}\sum_{t=1}^n\sum_{\begin{array}{l}s=1\\s\neq t \end{array}}^n I^{Kd}_{i,j}(t,s)
\end{equation}
where
$$
I^{Kd}_{i,j}(t,s)=\left\{\
									\begin{array}{ll}
									  1, & (x_i(t)-x_i(s))(x_j(t)-x_j(s))\geq 0\\
										-1, & (x_i(t)-x_i(s))(x_j(t)-x_j(s))<0
									\end{array}
							 \right.
$$

\section{Robustness of Kruskal algorithm in Fechner correlation network}\label{Robustness of Kruskal algorithm}
Uncertainty of Kruskal algorithm for MST identification depends on the chosen correlation network. From one side, for any $X \in K(\Lambda)$ Kruskal algorithms in different correlation networks  identify the same true MST. From the other side, error in the identification can be different. In this Section we state and prove an interesting property of Kruskal algorithm for MST identification in Fechner correlation network. This property can be associated with robustness of the algorithm. Indeed, robustness in general is associated with non sensitivity of  an algorithm to the change of some parameters. This is the case of Kruskal algorithm for MST identification in Fechner correlation network. More precisely the following statement is true:

\begin{thm}
Let $X \in K(\Lambda)$ and the vector of means $\mu$ be known. Then FDR of Kruscal algorithm for MST identification in Fechner correlation network is the same for any vector $X$.
\end{thm}
This means that FDR as a risk function does not depend on distribution from the class $K(\Lambda)$ (distribution free risk function).

\noindent
{\bf Proof.} Proof is based on the results from our publication \cite{Kalyagin_2017}. First step of the Kruskal algorithm for MST identification in Fechner correlation network is to sort the edges of the complete weighted graph $(V,\hat{\Gamma}^{Fh})$ into decreasing order by weights $\hat{\gamma}^{Fh}_{i,j}$.
One has
$$
\displaystyle \hat{\gamma}^{Fh}_{i,j}=2 \hat{\gamma}^{Sg}_{i,j}-1
$$
Therefore, the first step of the algorithm is equivalent to sort $\hat{\gamma}^{Sg}_{i,j}$ in decreasing order.
It is proved in \cite{Kalyagin_2017} (Theorem 2) that the joint distribution of statistics $\hat{\gamma}^{Sg}_{i,j}$ (in the paper they are denoted by $T^{Sg}_{i,j}$) is the same for any $X \in K(\Lambda)$. It implies that the probability of any ordering of $\hat{\gamma}^{Sg}_{i,j}$ does not depend on distribution of the vector $X \in K(\Lambda)$. $\hat{MST}$, obtained by Kruskal algorithm of identification is completely  defined by ordering of $\hat{\gamma}^{Sg}_{i,j}$. Therefore, any such ordering generates the same numbers FP, FN, TP and TN. It implies that  the distribution of the loss function $w_{FDR}=(FP/(FP+TP))$ is the same for any $X \in K(\Lambda)$ and the theorem follows.

\section{Reliability of Kruskal algorithm in  different correlation networks}\label{Reliability of Kruskal algorithm}

It the section we study by numerical simulations reliability (uncertainty) of Kruskal algorithm for MST identification in three correlation networks: Pearson correlation network, Fechner correlation network, and Kendall correlation network for stock market returns. The results of numerical experiments show that reliability of MST identification is different in three networks, despite the fact that true MST is the same. It is shown  that for Pearson correlation network the FDR of MST identification essentially depends on distribution of stock returns. For Fechner correlation network we observe that the FDR is non sensitive to the assumption on stock's return distribution in accordance with theoretical result of the robustness of Kruskal algorithm. New and surprising phenomena are discovered for   Kendall correlation network. Our experiments show that FDR of Kruskal algorithm for MST identification in Kendall correlation network weakly depend on distributions of the vector $X \in K(\Lambda)$ and at the same time the value of FDR is almost the best in comparison with MST identification in other networks. This needs a further investigation.

Our experiments are organized as follows. We take the real data of stock returns from a stock market. Using these data we estimate vector of means and  correlation matrix for the stock returns. These estimations are fixed as true vector of means $\mu$ and matrix $\Lambda$ for the random vectors $X$ from the class $K(\Lambda)$ of elliptical distributions (in our experiments, we use correlation matrix as the matrix $\Lambda$). To make our conclusions more general we consider networks of different sizes.

To study how FDR of Kruskal algorithm for MST identification depends on distribution from the class $K(\Lambda)$ we consider the  family of distributions from this class with the following densities
$$
f_{\epsilon}(x)=(1-\epsilon) f_{Gauss}(\mu,\Lambda)+\epsilon f_{Student}(\mu,\Lambda), \ \ \epsilon \in[0,1]
$$
Here for $f_{Student}(\mu,\Lambda)$ we fix the parameter $\nu=3$. For $\epsilon=0$ we have the multivariate Gaussian distribution, and for $\epsilon=1$ we have the multivariate Student distribution. Other distributions are a mixture of these two distributions. The computational scheme is the following
\begin{itemize}
\item For a given covariance (correlation) matrix $\Lambda$ calculate true MST.
\item Generate a sample of the size $n$ from distribution with density $f_{\epsilon}(x)$.
\item For each correlation network use Kruskal algorithm to identify MST by observations.
\item Compare true MST and MST identified by Kruskal algorithm and calculate FDR for the sample
\item Repeat last three steps $S$ times and make average of FDR's to evaluate the expected value of FDR.
\end{itemize}

\vskip 2mm
\noindent
{Experiment 1.} Consider the following $N=10$ stocks from USA stock market: A (Agilent Technologies Inc), AA (Alcoa Inc), AAP (Advance Auto Parts Inc), AAPL (Apple Inc), AAWW (Atlas Air Worldwide Holdings Inc), ABAX (Abaxis Inc), ABD (ACCO Brands Corp), ABG (Asbury Automotive Group Inc), ACWI (iShares MSCI ACWI Index Fund), ADX (Adams Express Company). We estimate the parameters $\mu$ and $\Lambda$ by the data for the 250 observations started from November 2010. Associated matrix of Pearson correlations is
$$
\left(
  \begin{array} {cccccccccc} \label{arr:weightMatrix}
               1.0000&    0.7220&    0.4681&    0.4809&    0.6209&    0.5380&    0.6252&    0.6285&    0.7786&    0.7909\\
           0.7220&    1.0000&    0.4395&    0.5979&    0.6381&    0.5725&    0.6666&    0.6266&    0.8583&    0.8640\\
            0.4681&    0.4395&    1.0000&    0.3432&    0.3468&    0.2740&    0.4090&    0.4016&    0.4615&    0.4832\\
            0.4809&    0.5979&    0.3432&    1.0000&    0.4518&    0.4460&    0.4635&    0.4940&    0.6447&    0.6601\\
            0.6209&    0.6381&    0.3468&    0.4518&    1.0000&    0.5640&    0.5994&    0.5369&    0.7170&    0.7136\\
           0.5380&    0.5725&    0.2740&    0.4460&    0.5640&    1.0000&    0.4969&    0.4775&    0.6439&    0.6242\\
            0.6252&    0.6666&    0.4090&    0.4635&    0.5994&    0.4969&    1.0000&    0.6098&    0.7161&    0.7158\\
           0.6285&    0.6266&    0.4016&    0.4940&    0.5369&    0.4775&    0.6098&    1.0000&    0.6805&    0.6748\\
            0.7786&    0.8583&    0.4615&    0.6447&    0.7170&    0.6439&    0.7161&    0.6805&    1.0000&    0.9523\\
            0.7909&    0.8640&    0.4832&    0.6601&    0.7136&    0.6242&    0.7158&    0.6748&    0.9523&    1.0000
        \end{array}
    \right)
$$
True MST, obtained from this matrix is given by Fig. \ref{True_MST_N_10}.

\begin{figure}[h!]
\centering
\includegraphics*[width=1.00\textwidth]{MST.png}
\caption{True MST, $N=10$}\label{True_MST_N_10}
\end{figure}

The results of FDR evaluation for Kruskal algorithm of MST identification in three networks are presented in Tables \ref{FDR_N10_n10},\ref{FDR_N10_n100},\ref{FDR_N10_n1000}. Analysis of the results shows that for $n=10$ and $n=100$ all algorithms for MST identification have weak reliability in terms of FDR. The results of Table \ref{FDR_N10_n1000} show that 1-2 edges in identified MST are different from edges in true MST.

Interesting results were obtained for Kendall correlation network. Namely the obtained results shows that quality of MST identification in Kendall correlation network is close to the quality of MST identification in Pearson network for Gaussian distribution and are better than obtained results of MST identification in Pearson network for Student distribution. This is valid for $n=1000$ too. Besides one can see strong dependence on distribution of FDR for Kruskal algorithm of MST identification in Pearson correlation network and stability of FDR for Kruskal algorithm of MST identification in Fechner correlation network.  From the other side, FDR for Kruskal algorithm of MST identification in Kendall correlation network is almost stable, weakly depending on distribution.

\vskip 2mm
\noindent
{Experiment 2.} Consider $N=50$ stocks from NASDAQ stock market with largest trade volume for the year 2014. Parameters $\mu$ and $\Lambda$ are estimated by  250 observations for 2014.
The tables \ref{FDR_N50_n1000} and \ref{FDR_N50_n10000} present the results of FDR evaluation for $n=1000$ and $n=10000$ for three networks. The results are almost the same as for Experiment 1. One can see strong dependence on distribution of FDR for Kruskal algorithm of MST identification in Pearson correlation network and stability of FDR for Kruskal algorithm of MST identification in Fechner correlation network.  From the other side, FDR for Kruskal algorithm of MST identification in Kendall correlation network is almost stable, weakly depending on distribution. Reliability of MST identification in this network is almost the best with respect to other networks.

\begin{table}[h!]
\begin{center}
  \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|}
\hline
 measure,$\epsilon $ & 0    & 0.1  & 0.2  & 0.3  & 0.4  & 0.5  & 0.6  & 0.7  & 0.8  & 0.9  & 1    \\ \hline
 Pearson 						 & 0.66 & 0.67 & 0.67 & 0.67 & 0.68 & 0.68 & 0.67 & 0.68 & 0.69 & 0.68 & 0.69 \\ \hline
 Fechner             & 0.65 & 0.64 & 0.64 & 0.64 & 0.64 & 0.64 & 0.64 & 0.64 & 0.64 & 0.63 & 0.64 \\ \hline
 Kendall             & 0.65 & 0.65 & 0.66 & 0.66 & 0.66 & 0.67 & 0.67 & 0.67 & 0.67 & 0.66 & 0.66 \\ \hline

  \end{tabular}
\end{center}
 \caption{False discovery rate. N=10, n=10}\label{FDR_N10_n10}
 \end{table}

\begin{table}[h!]
\begin{center}
  \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|}
\hline
 measure,$\epsilon $ & 0    & 0.1  & 0.2  & 0.3  & 0.4  & 0.5  & 0.6  & 0.7  & 0.8  & 0.9  & 1    \\ \hline
 Pearson 						 & 0.37 & 0.40 & 0.40 & 0.41 & 0.43 & 0.45 & 0.46 & 0.48 & 0.48 & 0.50 & 0.52 \\ \hline
 Fechner             & 0.52 & 0.53 & 0.54 & 0.53 & 0.53 & 0.53 & 0.53 & 0.54 & 0.53 & 0.53 & 0.53 \\ \hline
 Kendall             & 0.41 & 0.40 & 0.41 & 0.41 & 0.42 & 0.42 & 0.42 & 0.44 & 0.44 & 0.44 & 0.44 \\ \hline

  \end{tabular}
\end{center}
 \caption{False discovery rate. N=10, n=100}\label{FDR_N10_n100}
 \end{table}

\begin{table}[h!]
\begin{center}
  \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|}
\hline
 measure,$\epsilon $ & 0    & 0.1  & 0.2  & 0.3  & 0.4  & 0.5  & 0.6  & 0.7  & 0.8  & 0.9  & 1    \\ \hline
 Pearson 						 & 0.15 & 0.17 & 0.19 & 0.21 & 0.22 & 0.23 & 0.26 & 0.29 & 0.30 & 0.33 & 0.34 \\ \hline
 Fechner             & 0.34 & 0.34 & 0.33 & 0.33 & 0.33 & 0.33 & 0.33 & 0.33 & 0.33 & 0.34 & 0.33 \\ \hline
 Kendall             & 0.17 & 0.17 & 0.17 & 0.17 & 0.18 & 0.18 & 0.18 & 0.18 & 0.19 & 0.20 & 0.20 \\ \hline

  \end{tabular}
\end{center}
 \caption{False discovery rate. N=10, n=1000}\label{FDR_N10_n1000}
 \end{table}

\begin{table}[h!]
\begin{center}
  \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|}
\hline
 measure,$\epsilon $ & 0    & 0.1  & 0.2  & 0.3  & 0.4  & 0.5  & 0.6  & 0.7  & 0.8  & 0.9  & 1    \\ \hline
 Pearson 						 & 0.23 & 0.26 & 0.29 & 0.34 & 0.37 & 0.34 & 0.42 & 0.44 & 0.46 & 0.50 & 0.52 \\ \hline
 Fechner             & 0.41 & 0.39 & 0.40 & 0.40 & 0.39 & 0.40 & 0.40 & 0.41 & 0.41 & 0.40 & 0.41 \\ \hline
 Kendall             & 0.25 & 0.25 & 0.25 & 0.26 & 0.27 & 0.27 & 0.28 & 0.27 & 0.28 & 0.29 & 0.28 \\ \hline

  \end{tabular}
\end{center}
 \caption{False discovery rate. N=50, n=1000}\label{FDR_N50_n1000}
 \end{table}

\begin{table}[h!]
\begin{center}
  \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|}
\hline
 measure,$\epsilon $ & 0    & 0.1  & 0.2  & 0.3  & 0.4  & 0.5  & 0.6  & 0.7  & 0.8  & 0.9  & 1    \\ \hline
 Pearson 						 & 0.08 & 0.10 & 0.13 & 0.14 & 0.16 & 0.18 & 0.21 & 0.25 & 0.24 & 0.28 & 0.32 \\ \hline
 Fechner             & 0.14 & 0.13 & 0.14 & 0.13 & 0.14 & 0.14 & 0.14 & 0.14 & 0.13 & 0.13 & 0.14 \\ \hline
 Kendall             & 0.09 & 0.08 & 0.08 & 0.09 & 0.08 & 0.08 & 0.09 & 0.09 & 0.09 & 0.08 & 0.10 \\ \hline

  \end{tabular}
\end{center}
 \caption{False discovery rate. N=50, n=10000}\label{FDR_N50_n10000}
 \end{table}

\section{Concluding remarks}\label{Concluding remarks}

The main advantage of the proposed framework to measure uncertainty of algorithms for MST identification is that it allows to make a correct comparison of the uncertainty for different networks and for a large class of distributions. Peculiarities of Pearson, Fechner and Kendall correlation networks for elliptical distributions were emphasized in the paper on the base of this approach. It was observed that Kendall correlation network looks the most appropriate for MST identification. This phenomena will be a subject for further investigations.