\section{Approach}
\vspace{-0.1in}

Our goal is to find a discriminative set of parts or patches that co-occur in many of the positively labeled images in the same configuration. We address this goal in two steps.
First, we find a set of patches that are discriminative, i.e., they tend to occur mainly in positive images. Second, we use an efficient approach to find co-occurring configurations of pairs of such patches. Our approach easily extends beyond pairs; for simplicity and to retain configurations that occur frequently enough, we here restrict ourselves to pairs.

\textbf{Discriminative candidate patches.} For identifying discriminative patches, we begin with a construction similar to that of \citet{song-icml2014}.
Let $\mathcal{P}$ be the set of positively labeled images. Each image $I$ contains candidate boxes $\{b_{I,1}, \ldots, b_{I,m}\}$ found via selective search \cite{selectivesearch}. For each $b_{I,i}$, we find its closest matching neighbor $b_{I',j}$ in each other image $I'$ (regardless of the image label).
The $K$ closest of those neighbors form the neighborhood $\mathcal{N}(b_{I,i})$, the remaining ones are discarded.

Discriminative patches will have neighborhoods mainly within images in $\mathcal{P}$, i.e., if $\mathcal{B}(\mathcal{P})$ is the set of all patches from images in $\mathcal{P}$, then $\mathcal{N}(b) \cap \mathcal{B}(\mathcal{P}) \approx K$. To identify a small, diverse and representative set of such patches,
like \cite{song-icml2014}, we construct a bipartite graph $\mathcal{G} = (\mathcal{U},\mathcal{V},\mathcal{E})$, where both $\mathcal{U}$ and $\mathcal{V}$ contain copies of $\mathcal{B}(\mathcal{P})$.
Each patch $b \in \mathcal{V}$ is connected to the copy of its nearest neighbors in $\mathcal{U}$ -- these will be $K$ or less, depending on whether the $K$ nearest neighbors of $b$ occur in $\mathcal{B}(\mathcal{P})$ or in negatively labeled images. The most representative patches maximize the covering function
\begin{equation}
  \label{eq:cover}
  F(S) = | \Gamma(S) |,
\end{equation}
where $\Gamma(S) = \{u \in \mathcal{U} \mid (b,u) \in \mathcal{E} \text{ for some } b \in S\}$ is the neighborhood of $S \subseteq \mathcal{V}$ in the bipartite graph. Figure~\ref{fig:g_c} shows a cartoon illustration.
The function $F$ is monotone and submodular, and the $C$ maximizing elements (for a given $C$) can be selected greedily~\cite{nemhauser1978}.

However, if we aim to find part configurations, we must select multiple, jointly informative patches per image. Patches selected to merely maximize coverage can still be redundant, since the most frequently occurring ones are often highly overlapping. A straightforward modification would be to treat highly overlapping patches as identical. This identification would still admit a submodular covering model as in Equation~\eqref{eq:cover}. But, in our case, the candidate patches are very densely packed in the image, and, by transitivity, we would have to make all of them identical. In consequence, this would completely rule out the selection of more than one patch in an image and thereby prohibit the discovery of any co-occurring configurations.

\begin{figure}
  \centering
  \begin{tabular}{l@{\hspace{35pt}}c}
  \multirow{3}{*}{
  \includegraphics[width=0.55\textwidth]{figures/figure1/clusters.pdf}} &
  \vspace{1pt} \\
  &
  \includegraphics[width=0.25\textwidth]{figures/figure1/g_c2.pdf}
  \\

  & \vspace{2pt}
  \end{tabular}
  \caption{Left: bipartite graph $\mathcal{G}$ that defines the utility function $F$ and identifies discriminativeness; right: graph $\mathcal{G}_C$ that defines the diversifying independence constraints $\mathcal{M}$. We may pick $C_1$ (yellow) and $C_3$ (green) together, but not $C_2$ (red) with any of those, since it is redundant. If we identify overlapping patches in $\mathcal{G}$ and thus the covering $F$, then we would only ever pick one of $C_1$, $C_2$ and $C_3$, and no characteristic configurations could be identified.}
  \label{fig:g_c}
\end{figure}
\vspace{-0.1cm}

Therefore, we take a different approach, differing from \cite{song-icml2014} whose goal is to identify single patches, and not part-based configurations. We constrain our selection such that no two patches $b, b' \in \mathcal{V}$ can be picked whose neighborhoods overlap by more than a fraction of $\theta$. By overlap, we mean that the patches in the neighborhoods of $b, b'$ overlap significantly (they need not be identical).
This notion of diversity is reminiscent of NMS and similar to that in~\cite{doersch-siggraph2012}, but we here phrase and analyze it as a constrained submodular optimization problem.
Our constraint can be expressed in terms of a different graph $\mathcal{G}_C = (\mathcal{V}, \mathcal{E}_C)$ with nodes $\mathcal{V}$. In $\mathcal{G}_C$, there is an edge between $b$ and $b'$ if their neighborhoods overlap prohibitively, as illustrated in Figure~\ref{fig:g_c}. Our family of feasible solutions is
\begin{equation}
  \label{eq:constrainedcov}
  \mathcal{M} = \{S \subseteq V \mid \forall\, b,b' \in S \text{ there is no edge } (b,b') \in \mathcal{E}_C\}.
\end{equation}
In other words, $\mathcal{M}$ is the family of all independent sets in $\mathcal{G}_C$. We aim to maximize
\begin{align}
  \max\nolimits_{S\subseteq \mathcal{V}} F(S) \quad \text{s.t. } S \in \mathcal{M}.
\end{align}
This problem is NP-hard. We solve it approximately via the following greedy algorithm. Begin with $S^0 = \emptyset$, and, in iteration $t$, add $b \in \argmax_{b \in \mathcal{V} \setminus S} |\Gamma(b) \setminus \Gamma(S^{t-1})|$. As we add $b$, we delete all of $b$'s neighbors in $\mathcal{G}_C$ from $\mathcal{V}$. We continue until $\mathcal{V} = \emptyset$. If the neighborhoods $\Gamma(b)$ are disjoint, then this algorithm amounts to the following simplified scheme: we first sort all $b \in \mathcal{V}$ in non-increasing order by their degree $\Gamma(b)$, i.e., their number of neighbors in $\mathcal{B}(\mathcal{P})$, and visit them in this order. We always add the currently highest $b$ in the list to $S$, then delete it from the list, and with it all its immediate (overlapping) neighbors.
The following lemma states an approximation factor for the greedy algorithm, where $\Delta$ is the maximum degree of any node in $\mathcal{G}_C$.
\begin{lemma}\label{lem:bound}
  The solution $S_g$ returned by the greedy algorithm is a $1/(\Delta+2)$ approximation for Problem~\eqref{eq:constrainedcov}: $F(S_g) \geq \tfrac{1}{\Delta+2}F(S^*)$. If $\Gamma(b) \cap \Gamma(b') = \emptyset$ for all $b,b' \in \mathcal{V}$, then the worst-case approximation factor is $1/(\Delta+1)$.
\end{lemma}
The proof relies on phrasing $\mathcal{M}$ as an intersection of matroids.
\begin{definition}[Matroid]
A matroid $(\mathcal{V}, \mathcal{I}_k)$ consists of a ground set $\mathcal{V}$ and a family $\mathcal{I}_k \subseteq 2^{\mathcal{V}}$ of ``independent sets'' that satisfy three axioms: (1) $\emptyset \in \mathcal{I}_k$; (2) downward closedness: if $S \in \mathcal{I}_k$ then $T \in \mathcal{I}_k$ for all $T \subseteq S$; and (3) the exchange property: if $S, T \in \mathcal{I}_k$ and $|S| < |T|$, then there is an element $v \in T\setminus S$ such that $S \cup \{v\} \in \mathcal{I}_k$.
\end{definition}
\begin{proof}\emph{(Lemma~\ref{lem:bound})}
  We will argue that Problem~\eqref{eq:constrainedcov} is the problem of maximizing a monotone submodular function subject to the constraint that the solution lies in the intersection of $\Delta+1$ matroids. With this insight, the approximation factor of the greedy algorithm for submodular $F$ follows from \cite{fisher78} and that for non-intersecting $\Gamma(b)$ from \cite{jenkyns76}, since in the latter case the problem is that of finding a maximum weight vector in the intersection of $\Delta+1$ matroids.

It remains to argue that $\mathcal{M}$ is an intersection of matroids.
Our matroids will be partition matroids (over the ground set $\mathcal{V}$) whose independent sets are of the form $\mathcal{M}_k = \{S \mid |S \cap e| \leq 1, \text{ for all } e \in E_k\}$. To define those, we partition the edges in $\mathcal{G}_C$ into disjoint sets $E_k$, i.e., no two edges in $E_k$ share a common node. The $E_k$ can be found by an edge coloring -- one $E_k$ and $\mathcal{M}_k$ for each color $k$. By Vizing's theorem \cite{vizing64}, we need at most $\Delta+1$ colors.
The matroid $\mathcal{M}_k$ demands that for each edge $e \in E_k$, we may only select one of its adjacent nodes. All matroids together say that for any edge $e \in \mathcal{E}$, we may only select one of the adjacent nodes, and that is the constraint in Equation~\eqref{eq:constrainedcov}, i.e. $\mathcal{M} = \bigcap_{k=1}^{\Delta+1} \mathcal{M}_k$. We do not ever need to explicitly compute $E_k$ and $\mathcal{M}_k$; all we need to do is check membership in the intersection, and this is equivalent to checking whether a set $S$ is an independent set in $\mathcal{G}_C$, which is done by the deletions in the algorithm.
\vspace{-0.1cm}
\end{proof}

From the constrained greedy algorithm, we obtain a set $S \subset \mathcal{V}$ of discriminative patches. Together with its neighborhood $\Gamma(b)$, each patch $b \in \mathcal{V}$ forms a representative cluster.
Figure~\ref{fig:clusters} shows some example patches derived from the labels ``aeroplane'' and ``motorbike''. The discovered patches intuitively look like ``parts'' of the objects, and are frequent but sufficiently different.

\begin{figure*}[t]
\centering
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/aeroplane/c_1_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/aeroplane/c_1_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/aeroplane/c_2_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/aeroplane/c_2_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/aeroplane/c_3_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/aeroplane/c_3_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/aeroplane/c_4_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/aeroplane/c_4_1.png}\\
\vspace{0.15cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/motorbike/c_1_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/motorbike/c_1_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/motorbike/c_2_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/motorbike/c_2_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/motorbike/c_3_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/motorbike/c_3_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/motorbike/c_4_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/motorbike/c_4_1.png}\\
\vspace{0.15cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/cat/c_1_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/cat/c_1_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/cat/c_2_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/cat/c_2_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/cat/c_3_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/cat/c_3_1.png}\hspace{0.03cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/cat/c_4_0.png}\hspace{-0.12cm}
\includegraphics[width=0.12\textwidth, height=1.2cm]{figures/clusters/cat/c_4_1.png}\\
\vspace{-0.05in}
\caption{Examples of discovered patch ``clusters'' for aeroplane, motorbike, and cat.  The discovered patches intuitively look like object parts, and are frequent but sufficiently different.}
\label{fig:clusters}
\vspace{-0.1in}
\end{figure*}

\textbf{Finding frequent configurations.}
The next step is to find frequent configurations of co-occurring clusters, e.g., the head patch of a person on top of the torso patch, or a bicycle with visible wheels.
A ``configuration'' consists of patches from two clusters $C_i, C_j$, their relative location, and their viewpoint and scale. In practice, we give preference to pairs that by themselves are very relevant and maximize a weighted combination of co-occurrence count and coverage $\max\{\Gamma (C_i), \Gamma (C_j)\}$.

All possible configurations of all pairs of patches amount to too many to explicitly write down and count. Instead, we follow an efficient procedure for finding frequent configurations. Our approach is inspired by \cite{li-cvpr2012}, but does not require any supervision.
We first find configurations that occur in at least two images. To do so, we consider each pair of images $I_1$, $I_2$ that have at least two co-occurring clusters. For each correspondence of cluster patches across the images, we find a corresponding transform operation (translation, rescale, viewpoint change). This results in a point in a 4D transform space, for each cluster correspondence. We quantize this space into $B$ bins.
Our candidate configurations will be pairs of cluster correspondences $((b_{I_1,1}, b_{I_2,1}),(b_{I_1,2},b_{I_2,2})) \in (C_i \times C_i), (C_j \times C_j)$ that fall in the same bin, i.e., share the same transform, and have the same relative location. Between a given pair of images, there can be multiple such pairs of correspondences. We keep track of those via a multi-graph $\mathcal{G}_P = (\mathcal{P}, \mathcal{E}_P)$ that has a node for each image $I \in \mathcal{P}$. For each correspondence $((b_{I_1,1}, b_{I_2,1}),(b_{I_1,2},b_{I_2,2})) \in (C_i \times C_i), (C_j \times C_j)$, we draw an edge $(I_1,I_2)$ and label it by the clusters $C_i, C_j$ and the common relative position. As a result, there can be multiple edges $(I_1,I_j)$ in $\mathcal{G}_P$ with different edge labels.

The most frequently occurring configuration can now be read out by finding the largest connected component in $\mathcal{G}_P$ induced by retaining only edges with the same label. We use the largest component(s) as the characteristic configurations for a given image label (object class). If the component is very small, then there is not enough information to determine co-occurrences, and we simply use the most frequent single cluster. (This may also be determined by a statistical test.)
The final single ``correct'' localization will be the smallest bounding box that contains the full configuration.

\section{Conclusion}
\textbf{Conclusion.}
We presented a novel weakly-supervised object detection method that discovers frequent configurations of discriminative visual patterns.  We showed that the discovered configurations provide more accurate spatial coverage of the full object and provide a way to generate useful hard negatives.  Together, these lead to state-of-the-art weakly-supervised detection results on the challenging PASCAL VOC dataset.

\section{Experiments}
\vspace{-0.1in}

In this section, we analyze (1) detection performance of the models trained with the discovered configurations, and (2) impact of the discovered hard negatives on detection performance.

\textbf{Implementation details.}
We employ a recent region based detection framework \cite{rcnnTR, song-icml2014} and use the same fc7 features from the CNN model \cite{decafTR} on region proposals \cite{selectivesearch} throughout the experiment.  For discriminative patch discovery, we use $K=|\mathcal{P}|/2, \theta =K/20$.  For correspondence detection, we discretize the 4D transform space of \{$x$: relative horizontal shift, $y$: relative vertical shift, $s$: relative scale, $p$: relative aspect ratio\} with $\Delta x = 30~px, \Delta y = 30 ~px, \Delta s = 1 ~px/px, \Delta p = 1 ~px/px$. We choose this binning scheme by visually examining few qualitative examples so that scale, aspect ratio agreement between the two paired instances are more strict, while their translation agreement is more loose in order to handle deformable objects. More details regarding be transform space binning can be found in \cite{parikh-cvpr08}.

\textbf{Discovered configurations.}
Figure \ref{fig:cluster_visualization_figure} qualitatively illustrates discovered configurations (green and yellow boxes) and foreground estimates (magenta boxes) that have high degree in graph $\mathcal{G}_P$ for all classes in the PASCAL dataset.  Our method consistently finds meaningful combinations such as a wheel and body of bicycles, face and torso of people, locomotive basement and upper body parts of trains/buses, window and body frame of cars, etc.  Some failures include cases where the algorithm latches onto different objects co-occurring in consistent configurations such as the lamp and sofa combination (right column, second row from the bottom in Figure \ref{fig:cluster_visualization_figure}).

\textbf{Weakly-supervised object detection.}
Following the evaluation protocol of the PASCAL VOC dataset, we report detection results on the PASCAL \emph{test} set using detection average precision. For a direct comparison with the state-of-the-art weakly-supervised object detection method \cite{song-icml2014}, we do not use the extra instance level annotations such as \emph{pose, difficult, truncated} and restrict the supervision to the image level object presence annotations.  Table \ref{tab:detection-full} compares our detection results against two baseline methods \cite{siva1,song-icml2014} which report the result on the full dataset.  As shown in Table \ref{tab:detection-full}, our method improves detection performance on the majority of the classes (consistent improvement on rigid man-made object classes). It is worth noting that our method shows significant improvement on the person class (arguably most important category in the PASCAL dataset). Figure \ref{fig:detection_images} shows some example high scoring detection results on the test set.

\begin{figure*}[htbp]
\centering
\includegraphics[width=0.25\textwidth, height=2cm]{figures/detection_images/aeroplane_008950.pdf}\hspace{0.1cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/detection_images/bicycle_009866.pdf}\hspace{0.1cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/detection_images/car_009243.pdf}\hspace{0.1cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/detection_images/motorbike_009361.pdf}\hspace{0.1cm}
\includegraphics[width=0.245\textwidth, height=2cm]{figures/detection_images/car_009675.pdf}\hspace{0.15cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/detection_images/person_009612.pdf}\hspace{0.1cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/detection_images/person_009583.pdf}\hspace{0.1cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/detection_images/bicycle_009198.pdf}\hspace{0.1cm}
\caption{Example detections on test set. Green: our method, Red: \cite{song-icml2014}}
\label{fig:detection_images}
\end{figure*}
\vspace{-0.2cm}

\textbf{Impact of discovered hard negatives.}
To analyze the effect of our discovered hard negatives, we compare to two baseline cases: (1) not adding any negative examples from positives images (2) adding image regions around the foreground estimate as conventionally implemented in fully supervised object detection algorithms \cite{pedro2008, rcnnTR}. We use the criterion from \cite{rcnnTR}, where all image regions in positive images with overlap score (intersection area over union area with respect to foreground regions) less than $0.3$ are used as ``neighboring'' negative image regions on positive images.  Table \ref{tab:hard-negatives} shows the effect of our hard negative examples in terms of detection average precision, for all classes (mAP).  The experiment shows that adding ``neighboring negative regions'' does not lead to noticeable improvement over not adding any negative regions from positive images, while adding our automatically discovered hard negative regions improves the detection performance more substantially.

\begin{figure*}[h!]
\centering
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/aeroplane_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/aeroplane_rank_00002.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/bicycle_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/bicycle_rank_00002.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/bird_rank_00004.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/bird_rank_00002.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/boat_rank_00003.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/boat_rank_00004.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/bottle_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/bottle_rank_00002.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/bus_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/bus_rank_00002.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/car_rank_00003.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/car_rank_00004.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/cat_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/cat_rank_00002.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/chair_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/chair_rank_00002.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/cow_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/cow_rank_00002.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/diningtable_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/diningtable_rank_00003.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/dog_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/dog_rank_00002.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/horse_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/horse_rank_00002.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/motorbike_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/motorbike_rank_00002.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/person_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/person_rank_00002.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/pottedplant_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/pottedplant_rank_00002.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/sheep_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/sheep_rank_00002.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/sofa_rank_00003.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/sofa_rank_00002.pdf}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/train_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/train_rank_00002.pdf}\hspace{0.2cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/tvmonitor_rank_00001.pdf}\hspace{-0.12cm}
\includegraphics[width=0.23\textwidth, height=2cm]{figures/results/tvmonitor_rank_00002.pdf}
\caption{Example configurations that have high degree in graph $\mathcal{G}_P$.  The green and yellow boxes show the discovered discriminative visual parts, and the magenta box shows the bounding box that tightly fits their configuration.}
\label{fig:cluster_visualization_figure}
\vspace{-0.05in}
\end{figure*}

\newcommand{\fontseries{b}\selectfont}{\fontseries{b}\selectfont}

\begin{table*}[t]
\footnotesize
\centering
\renewcommand{\arraystretch}{1.0}
\renewcommand{\tabcolsep}{0.35mm}
\begin{tabular}{l *{21}{c}}
\toprule
 & ~aero & bike & bird & boat & bottle & bus & car & cat & chair & cow & table & dog & horse & mbike & pson & plant & sheep & sofa & train & tv & ~mAP\\
\midrule
\midrule
\cite{siva1} & ~13.4 & 44.0 & 3.1 & 3.1 & 0.0 & 31.2 & 43.9 & 7.1 & 0.1 & 9.3 & 9.9 & 1.5 & \fontseries{b}\selectfont 29.4 & 38.3 & 4.6 & 0.1 & 0.4 & 3.8 & 34.2 & 0.0 & ~ 13.9\\
\midrule
\cite{song-icml2014} & ~27.6 & 41.9 & 19.7 & 9.1 & 10.4 & 35.8 & 39.1 & \fontseries{b}\selectfont 33.6 & 0.6 & 20.9 & 10.0 & \fontseries{b}\selectfont 27.7 & \fontseries{b}\selectfont 29.4 & 39.2 & 9.1 & \fontseries{b}\selectfont 19.3 & 20.5 & \fontseries{b}\selectfont 17.1 & 35.6 & 7.1 & ~22.7\\
\midrule
\midrule
ours + SVM & ~31.9 & 47.0 & 21.9 & 8.7 & 4.9 & 34.4 & 41.8 & 25.6 & 0.3 & 19.5 & \fontseries{b}\selectfont 14.2 & 23.0 & 27.8 & 38.7 & \fontseries{b}\selectfont 21.2 & 17.6 & \fontseries{b}\selectfont 26.9 & 12.8 & \fontseries{b}\selectfont 40.1 & 9.2 & ~23.4\\
\midrule
ours + LSVM & ~\fontseries{b}\selectfont 36.3 & \fontseries{b}\selectfont 47.6 & \fontseries{b}\selectfont 23.3 & \fontseries{b}\selectfont 12.3 & \fontseries{b}\selectfont 11.1 & \fontseries{b}\selectfont 36.0 & \fontseries{b}\selectfont 46.6 & 25.4 & \fontseries{b}\selectfont 0.7 & \fontseries{b}\selectfont 23.5 & 12.5 & 23.5 & 27.9 & \fontseries{b}\selectfont 40.9 & 14.8 & 19.2 & 24.2 & \fontseries{b}\selectfont 17.1 & 37.7 & \fontseries{b}\selectfont 11.6 & ~\fontseries{b}\selectfont 24.6\\
\bottomrule
\end{tabular}
\caption{Detection average precision (\%) on full PASCAL VOC 2007 test set.}
\label{tab:detection-full}
\vspace{-0.1in}
\end{table*}

\begin{table}[t]
\footnotesize
\centering
\begin{tabular}{l *{4}{c}}
\toprule
 & ~~w/o hard negatives & ~~neighboring hard negatives & ~~discovered hard negatives\\
\midrule
\midrule
ours + SVM & 22.5 & 22.2 & \fontseries{b}\selectfont 23.4\\
\midrule
ours + LSVM & 23.7 & 23.9 & \fontseries{b}\selectfont 24.6\\
\bottomrule
\end{tabular}
\caption{Effect of our hard negative examples on full PASCAL VOC 2007 test set.}
\label{tab:hard-negatives}
\vspace{-0.05in}
\end{table}

\section{Introduction}
\vspace{-0.1in}

The growing amount of sparsely and noisily labeled image data promotes the need for robustly learning detection methods that can cope with a minimal amount of supervision. A prominent example of this scenario is the abundant availability of labels at the image level (i.e., whether a certain object is present in the image or not), while detailed annotations of the exact location of the object are tedious and expensive and, consequently, scarce. Learning methods that handle image-level labels circumvent the need for such detailed annotations and therefore have the potential to
effectively utilize the massive and ever-growing textually annotated visual data available on the Web. In addition, such weakly supervised methods can be more robust than fully supervised ones if the detailed annotations are noisy or ill-defined.

Motivated by these developments, recent work has explored learning methods that decreasingly rely on strong supervision. Early ideas for weakly supervised detection~\cite{weber,fergus03} paved the way by successfully learning part-based object models, albeit on simple object-centric datasets (e.g., Caltech-101). Since then, a number of approaches \cite{pandey,siva2012defence,song-icml2014} have attempted to learn models on more realistic and challenging data sets that feature large intra-category appearance variations and background clutter. To cope with those difficulties, these methods typically generate multiple candidate regions and retain the ones that occur most frequently in the positively labeled images. However, due to intra-category variations and deformations, the identified (single) patches often correspond to only a part of the object, such as a human face instead of the entire body. Such mislocalizations are a frequent problem for weakly supervised detection methods. Figure~\ref{fig:detection_images} illustrates some examples.

Mislocalization and too large or too small bounding boxes are problematic in two respects. First, they obviously affect the accuracy of the detection method. Detection is commonly phrased as multiple instance learning and addressed with non-convex optimization methods that alternatingly guess the location of the objects as positive examples (since the true location is unknown) and train a detector based on those guesses. Such methods are therefore heavily influenced by good initial localizations.
Second, a common approach is to train the detector in stages, while adding informative ``hard'' negative examples to the training data. If we are not given accurate true object localizations in the training data, these hard examples must be derived from the detections identified in earlier rounds, and these initial detections may only use image-level annotations. The higher the accuracy of the initial localizations, the more informative is the augmented training data -- and this is key to the accuracy of the final learned model.

In this work, we address the issue of mislocalizations by identifying characteristic, discriminative \emph{configurations} of multiple patches (rather than a single one). This part-based approach is motivated by the observation that automatically discovered single ``discriminative'' patches often correspond to object parts. In addition, wrong background patches (e.g., patches of water or sky) occur throughout the positive images, but do not re-occur in typical configurations.
In particular, we propose an effective method that takes as input a set of images with labels of the form ``the object is present''/``not present'', and automatically identifies characteristic part configurations of the given object.

To identify such co-occurrences, we use two main criteria. First, useful patches are \emph{discriminative}, i.e., they occur in many positively labeled images, but not in the negatively labeled ones. To identify such patches, we use a discriminative covering formulation similar to \cite{song-icml2014}. However, our end goal is to discover multiple patches in each image that represent different parts, i.e., they may be close but may not be overlapping too much. In covering formulations, one may discourage overlaps by saying that for two overlapping regions, one ``covers'' the other, i.e., they are treated as identical. But this is a transitive relation, and the density of possible regions in detection would imply that all regions are identical, strongly discouraging the selection of more than one part per image. Partial covers face the challenge of scale invariance. Hence, we take a different approach and formulate an independence constraint.  This second criterion ensures that we select regions that may be close but non-redundant and not fully overlapping. We show that this constrained selection problem corresponds to maximizing a submodular function subject to a matroid intersection constraint, which leads to approximation algorithms with theoretical worst-case bounds.
Given candidate parts identified by those two criteria, we effectively find frequently co-occurring configurations that take into account relative position, scale and viewpoint.

We demonstrate multiple benefits of the discovered configurations. First, we observe that combinations of patches can produce more accurate spatial coverage of the full object, especially when the most discriminative pattern corresponds to an object part.  Second, any overlapping region between the co-occurring visual patterns is likely to cover a part (but not the full) of the object of interest (see intersecting regions between green and yellow boxes in Figure~\ref{fig:cluster_visualization_figure}); thus, they can be used to generate very informative hard negatives for training, and those can reduce localization errors at test time.

In short, our main contribution is a novel weakly-supervised object detection method that automatically discovers frequent configurations of discriminative visual patterns and exploits them for training  more robust object detectors. In our experiments on the challenging PASCAL VOC dataset, we find the inclusion of our discriminative, automatically detected configurations to outperform all state-of-the-art methods.

\subsection{Related work}
\vspace{-0.05in}

\textbf{Weakly-supervised object detection.} Training object detectors is usually done in a fully-supervised fashion using tight bounding box annotations that cover the object of interest (e.g.,~\cite{pedro-dpm}).  To reduce laborious bounding box annotation costs, recent weakly-supervised approaches~\cite{weber,fergus03,pandey,siva2012defence,song-icml2014} train detectors using binary object-presence labels without any object-location information.

Early efforts~\cite{weber,fergus03} focused on simple datasets that have a single prominent object in each image (e.g., Caltech-101).  More recent approaches~\cite{pandey,siva2012defence,song-icml2014} focus on the more challenging PASCAL dataset, which contains multiple objects in each image and large intra-category appearance variations.  Of these, \citet{song-icml2014} achieve state-of-the-art results by finding discriminative image patches that occur frequently in the positive images but rarely in the negative images using deep Convolutional Neural Network (CNN) features~\cite{krizhevsky-nips2012} and a submodular cover formulation. We use a similar approach to identify discriminative patches. But, contrary to \cite{song-icml2014} who assume patches to contain entire objects, we allow patches to contain full objects or merely object \emph{parts}. Thus, we aim to automatically piece together those patches to produce better full-object estimates.  To this end, we augment the covering formulation and identify patches that are both representative and explicitly mutually different.
We will see that this leads to more robust object estimates and further allows our system to intelligently select ``hard negatives'' (mislocalized objects), both of which improve detection performance.

\textbf{Visual data mining.} Existing approaches discover high-level object categories~\cite{sivic-iccv05,grauman-cvpr06,faktor-eccv2012}, mid-level patches~\cite{singh-eccv2012,doersch-siggraph2012,juneja-cvpr2013}, or low-level foreground features~\cite{ff-ijcv} by grouping similar visual patterns (i.e., images, patches, or contours) according to their texture, color, shape, etc.  Recent methods~\cite{doersch-siggraph2012,juneja-cvpr2013} use weakly-supervised labels to discover discriminative visual patterns.  We use related ideas, but formulate the problem as a submodular optimization over matroids, which leads to approximation algorithms with theoretical worst-case guarantees.
Covering formulations have also been used in \cite{barinova12,chen14}, but after running a trained object detector.  An alternative discriminative approach, but less scalable than covering, uses spectral methods \cite{zou13}.

\textbf{Modeling co-occurring visual patterns.}  Modeling the spatial and geometric relationship between co-occurring visual patterns (objects or object-parts) often improves visual recognition performance~\cite{weber,fergus03,sivic-cvpr2004,quack-iccv2007,zhang-cvpr2009,ff-ijcv,pedro-dpm,farhadi-cvpr2011,singh-eccv2012,li-cvpr2012}.  Co-occurring patterns are usually represented as doublets~\cite{singh-eccv2012}, higher-order constellations~\cite{weber,fergus03} or star-shaped models~\cite{pedro-dpm}.  Among these, our work is most inspired by~\cite{weber,fergus03}, which learns part-based models using only weak-supervision. However, we use more informative features and a different formulation, and show results on more difficult datasets.
Our work is also related by~\cite{li-cvpr2012}, which discovers high-level object compositions (``visual phrases''~\cite{farhadi-cvpr2011}) using ground-truth bounding box annotations.  In contrast to~\cite{li-cvpr2012}, we aim to discover part compositions to represent full objects and do so without any bounding box annotations.


\section{Introduction}

\object{HD 150136} is the brightest member of the \object{NGC 6193} cluster in the \object{Ara OB1} association and a known non-thermal radio-emitter \citep{ben2006}, i.e.\ an object where particles are accelerated to relativistic energies \citep[see][and references therein]{debrev}. This distinctive feature was the reason for a decade-long observational effort to clarify the nature and properties of HD~150136. Recently, \citet[][hereafter \citetalias{MGS12}]{MGS12} showed that HD~150136 is a triple hierarchical system consisting of an inner binary, with an O3V((f$^{\star}$))-O3.5V((f$^+$)) primary star  and an O5.5-6V((f)) secondary star  on a $P_\mathrm{in} \approx 2.67$~d orbit,  and a third physically bound O6.5-7V((f)) companion on a $P_\mathrm{out} \approx 8$- to 15-year orbit. With a total mass to be estimated around 130~$M_{\odot}$, HD~150136 is  one of the most massive multiple O-star systems known. It is also the closest one to Earth to harbour an O3 star.

Given the range of possible orbital periods for the outer system, \citet{MGS12} estimated a probable separation on the plane of the sky between the inner pair and the third component of roughly 10 to 40~milli-arcsec (mas). Here, we report  on extended spectroscopic monitoring that allows us to obtain the first orbital solution of the outer system. We also report on the very first interferometric detection of the wide system using the Very Large Telescope Interferometer (VLTI).

\begin{table}
\caption{Journal of the new and archival spectroscopic observations.}
\label{tab: spectro}
\centering
\begin{tabular}{c r r r r}
\hline \hline
HJD           & $v_1$ \hspace*{3mm} & $v_2$ \hspace*{3mm} & $v_3$ \hspace*{3mm} \\
$-$2\,450\,000 & (km\,s$^{-1}$) & (km\,s$^{-1}$) & (km\,s$^{-1}$) \\
\hline
\vspace*{-1mm}\\
\multicolumn{4}{c}{2000 UVES observation}\\
\vspace*{-1mm}\\
1726.5548 & 173.5 & $-$368.8 &  29.4      \\
\vspace*{-1mm}\\
\multicolumn{4}{c}{2008 FEROS observation}\\
\vspace*{-1mm}\\
4658.5184 & $-$107.5 & 128.2 &  27.0      \\
\vspace*{-1mm}\\
\multicolumn{4}{c}{2011 FEROS observations}\\
\vspace*{-1mm}\\
5642.916 &	$-$182.9&        268.4&	 $-$6.9\\
5696.790 &	$-$208.8&        298.5&	 $-$4.0\\
5696.907 & 	$-$191.8&        269.4&	 $-$7.7\\
5697.897 &	   186.5&     $-$337.0&	 $-$6.4\\
5699.583 &      $-$198.9&        272.9&	 $-$6.4\\
5699.588 &      $-$191.7&        268.9&	 $-$6.7\\
\vspace*{-1mm}\\
\multicolumn{4}{c}{2012 FEROS observations}\\
\vspace*{-1mm}\\
6048.661 &	   148.6&     $-$274.7&	 $-$23.7\\
6048.758 &	   120.7&     $-$221.2&	 $-$27.4\\
6048.899 &	    62.0&     $-$150.9&	 $-$29.5\\
6049.597 &	$-$215.8&	 306.1&	 $-$18.0\\
6049.715 &	$-$221.1&	 314.8&	 $-$18.0\\
6049.924 &	$-$208.2&	 280.1&	 $-$22.3\\
6050.666 &	   119.1&     $-$223.4&	 $-$26.2\\
6050.843 &	   167.7&     $-$310.1&	 $-$25.7\\
6051.641 &	    33.6&     $-$114.2&	 $-$22.1\\
6051.864 &	$-$107.5&	 128.2&	 $-$21.0\\
6052.646 &	$-$192.7&	 268.9&	 $-$20.4\\
6052.781 &	$-$152.4&	 219.6&	 $-$24.9\\
6052.909 &	$-$109.9&	 107.7&	 $-$27.6\\
6053.660 &	   188.6&     $-$344.1&	 $-$27.6\\
6053.778 &	   187.8&     $-$350.7&	 $-$28.3\\
6053.937 &	   168.4&     $-$308.4&	 $-$26.1\\
6054.649 &	$-$152.4&	 190.2&	 $-$26.5\\
6054.758 &	$-$188.9&	 240.8&	 $-$23.4\\
\hline
\end{tabular}
\end{table}

\section{Observations and data reduction}

\subsection{Spectroscopy}
We used the FEROS spectrograph mounted at the MPG/ESO 2.2m telescope at La Silla (Chile) to obtain  new high-resolution optical spectra of HD~150136 that supplement the FEROS data analysed in \citetalias{MGS12}. Eighteen FEROS spectra were obtained during a eight-night run in May 2012 (PI: Mahy), providing a continuous wavelength coverage from 3700 to 9200\AA\ at a resolving power of 48\,000. The data were processed  as described in \citetalias{MGS12}.  In addition,  we searched the ESO archives for complementary data. We retrieved six FEROS spectra from 2011 (PI: Barb\'a; 087.D-0946(A)), one FEROS spectrum from 2008 (PI: Barb\'a; 079.D-0564(B)) and one UVES spectrum from July 2000 (PI: Roueff; 065.I-0526(A)). The latter two spectra provide additional constraints on the tertiary component, but not on the systemic velocity of the inner pair and were therefore  used only to calculate the orbital solution of the wider pair.

The  disentangling procedure described in \citetalias{MGS12} for triple systems was applied to the 2011 and 2012 sets of spectra. We also reprocessed the complete data set, using our cross-correlation technique on all data from 1999 to 2012 to consistently measure the radial velocities (RVs) of the HD~150136 components. The RV values corresponding to the new observational epochs are given in Table~1 along with the journal of the 2011 and 2012 observations.

\subsection{Long baseline interferometry}

\begin{figure*}
\centering
   \includegraphics[width=18cm]{interfero.pdf}
   \caption{Calibrated visibilities (left panel) and closure phases (middle panel) from PIONIER in August 2012, overlaid with the best fit binary model (red solid lines). The right panel shows the $\chi^2$ map in the vicinity of the best-fit solution.
           }
      \label{fig:interfero}
\end{figure*}

Interferometric data were obtained in June and August 2012 with the PIONIER combiner \citep[][]{Le-Bouquin:2011}
and the Auxiliary Telescopes (ATs) at the VLTI \citep{Haguenauer:2010}.
Fringes were dispersed over three spectral channels across the H-band (1.50 - 1.80\,$\mu$m).
The ATs were located in configuration A1-G1-K0-I1, providing six projected baselines
ranging from 30 to 120~m and a maximum angular resolution of 2~mas in the H-band. Data were reduced
and calibrated with the \texttt{pndrs} package \citep[][]{Le-Bouquin:2011}. Calibration stars were
chosen in the JMMC Stellar Diameters Catalog \citep{Lafrasse:2010uq}.  The closure phases and
visibilities were modelled with two unresolved sources because the expected diameters of
the individual components ($<0.1$\ mas) as well as the separation of the inner pair ($<0.2$\ mas)
are largely unresolved by the longest VLTI baselines. The data show no evidence that these assumptions
may be wrong ($\chi^2_\mathrm{red}\approx1.2$). We used the
\texttt{LITpro}\footnote{\tiny\url{www.jmmc.fr/litpro}} software \citep{Tallon-Bosc:2008}
to extract the best-fit binary parameters, namely the flux ratio (assumed to be constant across the H-band)
and the astrometric separation.  The closure phases were also analysed independently using the
method presented in \citet{Absil:2011}, and the results agreed excellently.

 HD\,150136 was clearly resolved on both dates with a slight decrease in separation between June and August (Table~\ref{tab: interfero}).  Figure~\ref{fig:interfero}
shows the data obtained in August 2012 overlaid with the best-fit binary model.
 The final accuracy is dominated by a 2\%\ uncertainty
in the wavelength calibration of PIONIER.
Adopting a distance of $1.32 \pm 0.12$~kpc \citep{HeH77}, the measured angular separations translate into projected distances of
$12.2\pm 1.1$ and $11.4\pm 1.1$~AU for the June and August observations.
The 1$\sigma$\ error-bars are dominated by the 10\%\ uncertainty on the distance.

\begin{table}
\caption{Interferometric best-fit measurements and 1$\sigma$\ error-bars.}
\label{tab: interfero}
\centering
\begin{tabular}{l l l l }
\hline\hline  & & \multicolumn{2}{c}{Observing date} \\
Parameter & Unit & 2012-06-10 & 2012-08-15 \\
\hline
\vspace*{-3mm}\\
HJD &($-$2\,450\,000)             & \hspace*{4mm} 6088.595 &\hspace*{4mm} 6154.573 \\
$(f_3/f_{1+2})_{1.65\mu}$ &         & \hspace*{2.5mm}$0.24 \pm 0.02$ & \hspace*{2.5mm}$0.24 \pm 0.02$   \\
$\delta$E               & (mas)   &               $-7.96 \pm 0.16$ & $-6.98 \pm 0.14$ \\
$\delta$N               & (mas)   &               $-4.73 \pm 0.09$ & $-5.13 \pm 0.10$ \\
$r$                     & (mas)   & \hspace*{2.5mm}$9.26 \pm 0.19$ & \hspace*{2.5mm}$8.66\pm0.17$ \\
$\theta$                & (\degr) & \hspace*{0.7mm}$239.2 \pm 1.0$ & \hspace*{0.7mm}$233.7\pm 1.1$ \\
$\chi^2_\mathrm{red}$    &         & \hspace*{8mm}  1.2             & \hspace*{8mm}   1.4 \\
\vspace*{-3mm}\\
\hline
\end{tabular}
\end{table}

\section{Orbital properties}

\begin{table}
\caption{Best-fit circular orbital solutions of the inner pair for the 2011 and 2012 campaigns.}
\label{tab: RV12}
\centering
\begin{tabular}{l l l l }
\hline\hline
Parameter & Unit & Primary & Secondary \\
\hline
\vspace*{-1mm}\\
\multicolumn{4}{c}{ 2011 SB2 solution }\\
\vspace*{-1mm}\\
$P_\mathrm{in}$ & (d)       & \multicolumn{2}{c}{ 2.67454 (fixed) } \\
$T_0$ &(HJD$-$2\,450\,000) & \multicolumn{2}{c}{ 1327.136 $\pm$ 0.006} \\
$M_2/M_1$  &               & \multicolumn{2}{c}{ \hspace*{5.5mm}0.623 $\pm$ 0.008} \\

$\gamma$ &(km\,s$^{-1}$)       &                $-15.7 \pm 2.4$ &                $-12.1 \pm 3.2$ \\
$K$ &(km\,s$^{-1}$)            & \hspace*{.7mm}$208.3 \pm 2.3$ & \hspace*{.7mm}$334.5 \pm 3.6$ \\
r.m.s. &(km\,s$^{-1}$)         & \multicolumn{2}{c}{\hspace*{3mm} 6.2} \\
\vspace*{-1mm}\\
\multicolumn{4}{c}{ 2012 SB2 solution} \\
\vspace*{-1mm}\\
$P_\mathrm{in}$ &(d)        & \multicolumn{2}{c}{ 2.67454 (fixed) } \\
$T_0$ &(HJD$-$2\,450\,000) & \multicolumn{2}{c}{1327.151 $\pm$ 0.005 } \\
 $M_2/M_1$ &               & \multicolumn{2}{c}{ \hspace*{5.5mm}0.630 $\pm$ 0.007 } \\

$\gamma$& (km\,s$^{-1}$)       & $-20.4 \pm 2.0$ & $-12.1  \pm 2.8 $ \\
$K$ &(km\,s$^{-1}$)            & \hspace*{.7mm}$214.4 \pm 2.5$ & \hspace*{.7mm}$340.4  \pm 3.9 $ \\
r.m.s.& (km\,s$^{-1}$)         & \multicolumn{2}{c}{ \hspace*{3mm}13.1} \\

\hline
\end{tabular}
\end{table}

\subsection{The close pair}
We used the Li{\`e}ge Orbital Solution Package\footnote{LOSP is developed and maintained by H. Sana and is available at http://www.science.uva.nl/$\sim$hsana/losp.html. The algorithm is based on the generalization of the SB1 method of \citet{wol67} to the SB2 case  along the lines described in \citet{rau00} and \citet{san06a}.} (LOSP) to compute the orbital solutions of the inner pair during the 2011 and 2012 campaigns (Table~\ref{tab: RV12}). This provided a measurement of the systemic velocity of the inner pair at these epochs. We fixed the orbital period to the same value as that of \citetalias{MGS12}, i.e.\ 2.67454~days. For both campaigns, the mass ratio, the semi-amplitudes, and the projected semi-major axis are similar to those found in Paper ~I. The difference between the systemic velocities of the primary and of the secondary components is larger for the 2012 solution but, globally, both orbital solutions of Table~\ref{tab: RV12} agree with each other and with the solution determined in \citetalias{MGS12}.

\subsection{The third companion}

\subsubsection{Spectroscopic orbital solution}
We determined the systemic velocities of the inner system  during  the individual short-term campaigns with more than one spectrum since 1999. These values are put in perspective with the RVs of the tertiary component in Fig.~\ref{fig: RV3}. There is a clear, anti-correlated periodic motion of the systemic velocity of the close binary system and of the tertiary component.  We performed a Fourier analysis on the tertiary RVs using the Heck-Manfroid-Mersch method \citep[][refined by \citealt{gos01}]{hec85}. The highest peak in the periodogram indicates a period of about $3335 \pm 260$~days. We adopted this value as our initial guess and used LOSP to refine the period estimate. We computed  an SB1 orbital solution from the RVs of the tertiary alone and an SB2 solution using the tertiary RVs and the systemic velocities of the inner pair ($\gamma_{12}$). The two solutions are in excellent agreement and the SB2 RV solution is given in Table~\ref{tab: spectro2}. The reliability of the RV-only solution, especially its eccentricity and the semi-amplitudes of the RV curves, is unfortunately limited by a lack of sampling around periastron.

\begin{table}
\caption{Best-fit LOSP RV solution of the wide system.}
\label{tab: spectro2}
\centering
\begin{tabular}{l l l l}
\hline\hline
            &            & \multicolumn{2}{c}{LOSP RV solution}   \\
Parameter    & Unit      & Inner pair       & Tertiary            \\
\hline
\vspace*{-3mm}\\
$P_\mathrm{out}$ &(d)     & \multicolumn{2}{c}{ $2980 \pm 71$ }    \\
$e$             &        & \multicolumn{2}{c}{ \hspace*{3.5mm}$0.60 \pm 0.14$}   \\
$\omega$ &(\degr)              & \multicolumn{2}{c}{ $259.2\pm 5.8$}    \\
$T$ &(HJD$-$2\,450\,000) & \multicolumn{2}{c}{ \hspace*{1.7mm}$1193 \pm 104$}    \\
$M_3/M_{1+2}$ &           & \multicolumn{2}{c}{ \hspace*{3.5mm}$0.31 \pm 0.07$}   \\
$\gamma$& (km\,s$^{-1}$)         & $-21.6 \pm 2.7$ & $-16.1 \pm  4.9$   \\
$K$ &(km\,s$^{-1}$)              & \hspace*{2.2mm}$16.7 \pm 4.5$ & \hspace*{2.2mm}$53.6 \pm 14.4$   \\
r.m.s.& (km\,s$^{-1}$)           & \multicolumn{2}{c}{ \hspace*{3.mm}5.2}               \\
\hline
\end{tabular}
\end{table}

\subsubsection{Simultaneous RV and astrometric orbital solution}
The separations measured by PIONIER in June and August 2012 are relatively small and indicate an eccentricity at the upper end of the confidence interval of the RV-only solution of Table~\ref{tab: spectro2}. We therefore proceeded to simultaneously adjust the RV and astrometric measurements of the outer pair.  We minimized the  squared differences between the measurements and the model
\begin{eqnarray}
\chi^2& = & \sum \left( \frac{\gamma_{12} - \gamma_{12}^\mathrm{mod}}{\sigma_{12}       } \right)^2
        +   \sum \left( \frac{      v_{3} -       v_{3}^\mathrm{mod}}{\sigma_{3}        } \right)^2 \nonumber \\
      & + & \sum \left( \frac{ \delta E  -  \delta E^\mathrm{mod}}{\sigma_{\delta E} } \right)^2
        +   \sum \left( \frac{ \delta N - \delta N^\mathrm{mod}}{\sigma_{\delta N}} \right)^2
\end{eqnarray}
 using a Levenberg-Marquardt method, adopting the RV-only solution of Table~\ref{tab: spectro2} as a starting point. Based on the residuals of the tertiary  and the systemic RVs around the best-fit RV curves, we adopted $\sigma_3=3.4$ and $\sigma_{12}=6.8$~km\,s$^{-1}$\ as typical uncertainties on $v_3$ and $\gamma_{12}$.  The three-dimensional orbital solution  converges towards a higher eccentricity (although still within error-bars), hence towards larger RV curve semi-amplitudes, than the RV-only solution. All other parameters remain unchanged. The best-fit combined solution is given in Table~\ref{tab: final} and is shown in Fig.~\ref{fig: RV3}.

To estimate the uncertainties, we performed Monte-Carlo (MC) simulations. We randomly drew input RVs and astrometric positions around the best-fit orbital solution at epochs corresponding to our observing dates. We used normal distributions with  standard deviations corresponding to the $1\sigma$ measurement uncertainties. We also included the uncertainty on the distance by drawing the distance from a normal distribution centered on 1320~pc and with a standard deviation of 120~pc. We recomputed the best-fit orbital solution and inclination using 1000 simulated data sets and we constructed the distributions of the output parameters. The medians of the distributions match the best-fit values of Table~\ref{tab: final} very well. We used the distances between the median and the 0.16 and 0.84 percentiles as uncertainty estimates. The upper and lower error-bars agree within 10\%\ for all parameters; Table~\ref{tab: final} provides the average of both values.

\begin{table}
\caption{Best-fit simultaneous RV and astrometric orbital solution of the wide system. The corresponding RV curves and projected orbit on the plane of sky are displayed in Fig.~\ref{fig: RV3}. }
\label{tab: final}
\centering
\begin{tabular}{llll}
\hline\hline
           &             & \multicolumn{2}{c}{Combined solution}\\
Parameter  & Unit        & Inner pair       & Tertiary  \\
\hline
\vspace*{-3mm}\\
$P_\mathrm{out}$ &(d)     & \multicolumn{2}{c}{ $3008 \pm 54$ }    \\
$e$             &        & \multicolumn{2}{c}{\hspace*{3.5mm}$0.73\pm0.05$ }    \\
$\omega$ &(\degr)              & \multicolumn{2}{c}{               $250.7\pm2.9$}     \\
$T$ &(HJD$-$2\,450\,000) & \multicolumn{2}{c}{                 $1241\pm38$}   \\

$M_3/M_{1+2}$   &         & \multicolumn{2}{c}{ \hspace*{3.5mm}$0.32\pm0.08$}  \\
$i_\mathrm{out}$ &(\degr) & \multicolumn{2}{c}{ $107.9\pm3.0$}     \\
$\Omega$ &(\degr)        & \multicolumn{2}{c}{ \hspace*{1.5mm}$-60\pm10$}     \\
\\
$\gamma$ &(km\,s$^{-1}$)         & $-19.9 \pm 2.2$ & $-21.5 \pm  2.0$   \\
$K$ &(km\,s$^{-1}$)              & \hspace*{2.3mm}$25.8 \pm 7.6$ & \hspace*{2.3mm}$79.8 \pm  11$   \\
$a$ &(A.U.)              & \hspace*{4mm}$5.1  \pm 1.2$ & \hspace*{2.3mm}$15.8 \pm  0.6$   \\
$M$ &($M_{\odot}$)             & \hspace*{3.2mm}$102   \pm 16$  & \hspace*{5mm}$ 33 \pm  12$   \\
\\
$\chi^2_\mathrm{red}$     &    & \multicolumn{2}{c}{\hspace*{3mm} 0.94}               \\
\vspace*{-3mm}\\
\hline
\end{tabular}
\end{table}

\section{Discussion}

\subsection{Physical properties}

Using the best-fit three-dimensional orbit, absolute mass estimates are $M_3=33\pm12$~$M_{\odot}$\ and $M_{1+2}=102\pm16$~$M_{\odot}$\ for the third companion and the total mass of the inner pair, respectively. Combining the total absolute mass of the inner pair with the spectroscopic minimum masses of \citetalias{MGS12} (table 4), we constrain the inclination of the inner binary to values of $49.6\pm3.6$\degr. This value is compatible with the absence of eclipse in the inner system. Best masses for the primary and secondary are thus $M_1=62.6\pm10.0$ and $M_2=39.5\pm6.3$~$M_{\odot}$. The agreement with the expected masses given the estimated spectral types is remarkably good \citep{MSH05}.  The dynamical masses also agree within errors with the evolutionary masses obtained in \citetalias{MGS12}: $70.6_{-9.1}^{+11.4}$, $36.2_{-1.6}^{+5.0}$ and $27.0_{-3.5}^{+3.0}$~$M_{\odot}$\ for components 1, 2, and 3, respectively. The measured flux-ratio in the H-band, $(f_3/f_{1+2})_{1.65\mu}= 0.24$, also agrees well with the expected H-band flux-ratio of 0.26 given the component spectral types \citep{MaP06}, which
  confirms the main-sequence nature of the tertiary object \citepalias[see discussion in][]{MGS12}.

\subsection{Non-thermal emission} \label{sect: nt}

The present study allows us to clarify to some extent the considerations on the origin of the non-thermal radio emission. As argued in Paper\,I, the synchrotron radio emission probably comes from the colliding winds in the wide orbit. The radius of the photosphere ($\tau = 1$) for the primary (whose wind dominates the free-free opacity in the system) is expected to be at most 850\,$R_{\odot}$\ (at $\lambda=20$\,cm), strongly suggesting that the stagnation point of the collision zone must be located farther away (see Paper\,I). For the tertiary star, we assume two typical values for the mass loss rate: $\mathrm{\dot M}_{cl}$\,=\,10$^{-7}$\,$M_{\odot}$\,yr$^{-1}$ \citep[{\it classical} value,][]{muijres2012} and $\mathrm{\dot M}_{ww}$\,=\,10$^{-9}$\,$M_{\odot}$\,yr$^{-1}$ (more representative of the {\it weak-wind} case). We also adopt a terminal velocity of 2500\,km\,s$^{-1}$, corresponding to $2.6 \times v_\mathrm{esc}$ and $v_\mathrm{esc} \approx 970$~km\,s$^{-1}$\ for O6.5-7~V stars \citep{muijres2012}.

On these bases, we estimate that, in the classical-wind case, the stagnation point is located at about 260 and 1630\,$R_{\odot}$\ from the tertiary at periastron and at apastron. In the weak-wind case, these distances reduce to about 30 and 200\,$R_{\odot}$. Using the same approach as described in Paper\,I, we find that the extension of the radio photosphere of the tertiary star is shorter than the distance between the tertiary component and the stagnation point of the colliding winds, whatever the wavelength and whatever the assumption about the nature of the stellar wind (classical or weak).

In the classical-wind case, however, the stagnation point very close to periastron could be located below the radio photosphere at longer wavelengths (13 and 20\,cm) where HD\,150136 is significantly detected, but the observations reported by \citet{ben2006} were performed in December 2003, which is far from periastron passage according to our ephemeris. In the weak-wind case,  the stagnation point is always located significantly outside the photosphere of the dense primary wind, even at periastron, and at all wavelengths. These facts are in agreement with the rather high flux densities measured between 3 and 20~cm, as reported by \citet{ben2006}.
   \begin{figure}
   \centering
      \includegraphics[width=\columnwidth]{rv4.pdf}
      \caption{Main panel: Evolution of the third component RVs (triangles) and of the systemic velocity of the O+O inner system (squares) as a function of time. The best-fit RV curves are overlaid (Table~\ref{tab: final}). Insert: PIONIER astrometric points and projection of the best-fit relative orbit on the plane of the sky. The dashed line shows the line of nodes. } \label{fig: RV3}
   \end{figure}

As discussed in \citet{deb2012} for a similar system, the variability in the synchrotron radio flux from a colliding-wind binary most likely comes from two changing factors: stellar separation and free-free absorption. Following the discussion above, absorption is unlikely to dominate. Any temporal change of the  HD\,150136 flux densities would therefore be mainly attributed to the varying stellar separation along the eccentric wide orbit. One may therefore expect the radio flux density to reach its maximum very close to periastron, with a minimum at apastron.

A radio monitoring of the wide orbit in HD\,150136 (so far non-existent) is required to validate this scenario and to achieve a more detailed description of the relevant non-thermal processes. Given the expected location of the stagnation point, the wind interaction zone may be resolvable from the stars by very large baseline radio interferometric facilities. Imaging the wind-wind collision at several epochs would help to distinguish between a weak-wind, as possibly suggested by the CMFGEN analysis in \citetalias{MGS12}, and a normal wind for the tertiary star.

\section{Summary}

We reported the very first interferometric detection of the outer companion in the hierarchical triple system HD~150136. Combining the interferometric measurements, new and archival spectroscopy with data from \citetalias{MGS12}, we obtained the first  three-dimensional orbital solution of the wider system. The best-fit solution indicates a 8.2-yr period and a high eccentricity ($e \mathrel{\hbox{\rlap{\hbox{\lower3pt\hbox{$\sim$}}}\hbox{\raise2pt\hbox{$>$}}}} 0.7$). The accuracy of the PIONIER interferometric measurements allowed us to constrain the inclination of the outer orbit, and from there on,  of the inner pair to within a few degrees only.

We constrained the masses of the three stars of the system to 63, 40, and 33~$M_{\odot}$\ for the O3-3.5~V, O5.5-6~V, and O6.5-7~V components. In particular, this is the first direct measurement of the mass of an early main-sequence O star. We showed that the obtained dynamical masses agree within errors with the evolutionary masses estimated in ~\citetalias{MGS12}.

Although relative error-bars remain at the 15\%-level for masses of the inner pair components and at the 30\%-level for the tertiary mass, spectroscopic observations around the next periastron passage in 2015 and further  interferometric monitoring over a significant fraction of the orbit may provide direct mass measurements with uncertainties of a couple of per cent only. It will also allow for an accurate independent measurement of the distance to the system. This constitutes a must-do to accurately test high-mass star evolutionary models and to investigate the reality and origin of the mass discrepancy in more details \citep[e.g.\ ][]{HKV92, WeV10}.

As for other non-thermal emitters (\object{HD 93250}, \citealt{SLBDB11}; \object{HD 167971}, \citealt{deb2012}), the present study also shows that long baseline interferometry is ideally suited to resolve the components of O-type non-thermal radio emitters. The significant advances on the three-dimensional description of the orbits in HD\,150136 is a solid base for future investigations to understand the non-thermal emission and particle acceleration processes at work in massive multiple systems.

\begin{acknowledgements}
 PIONIER is funded by the Universit\'e Joseph Fourier (UJF, Grenoble) through its Poles
TUNES and SMING and the vice-president of research, the Institut de
Plan\'etologie et d'Astrophysique de Grenoble, the ``Agence Nationale pour la Recherche''
with the program ANR EXOZODI, and the Institut National des Sciences de l'Univers (INSU)
with the programs ``Programme National de Physique Stellaire'' and ``Programme National de Plan\'etologie''.
The integrated optics beam combiner is the result of a collaboration between IPAG and CEA-LETI based
on CNES R\&T funding. The authors thank all the people involved in the VLTI project.
The use of Yassine Damerdji's orbital code is also warmly acknowledged.
This study made use of the Smithsonian/NASA Astrophysics Data System (ADS), of the Centre de
Donn\'ees astronomiques de Strasbourg (CDS) and of the Jean-Marie Mariotti Center (JMMC).
Some calculations and graphics were performed with the freeware \texttt{Yorick}.
\end{acknowledgements}


\section{Introduction}

Rogue (freak) waves can be described as high amplitude waves with a height bigger than $2-2.2$ times the significant waveheight in a wavefield. Their studies have become extensive in recent years \cite{Akhmediev2009b, Akhmediev2009a, bayindir2016, Akhmediev2011}. The research has emerged with the investigation of one of the simplest nonlinear models, which is the nonlinear Schr\"{o}dinger equation (NLSE)\cite{Akhmediev2009b}. Discovery of the unexpected rogue wave solutions of the NLSE resulted in seminal studies of rogue wave dynamics, such as in Ref.\cite{Akhmediev2009b}. Their existence is not necessarily restricted to optical media \cite{FirstOpticalRW}, they can also be observed in hydrodynamics, Bose-Einstein condensation, acoustics and finance, just to name a few \cite{Akhmediev2009b, bayindir2016}. It is natural to expect that in a medium whose dynamics are described by the NLSE and NLSE like equations, rogue waves can also emerge.
In this study we consider optical rogue waves for which analyzing the dynamics, shapes and statistics of rogue wavy optical fields are crucially important to satisfy certain power and communication constraints.

On the other hand, quantum Zeno dynamics \cite{MisraSudarshan,Facchi2008JPA}, which is the inhibition of the evolution of an unstable quantum state by appropriate frequent observations during a time interval has attracted an intense attention in quantum science, usually for protecting the quantum system from decaying due to inevitable interactions with its environment. It emerged that the observation alters the evolution of an atomic particle, even can stop it \cite{HarochePRL2010, HarochePRA2012, HarocheNP2014}. Duan and Guo showed that the dissipation of two particles can be prevented \cite{Guo1998PRL,Guo1998PRA}, Viola and Lloyd proposed a dynamical suppression of decoherence of qubit systems \cite{Lloyd1998PRA}, Maniscalco et al. proposed a strategy to fight against the decoherence of the entanglement of two atoms in a lossy resonator \cite{Maniscalco2008PRL}, Nourmandipour et al. studied Zeno and anti-Zeno effects on the entanglement dynamics of dissipative qubits coupled to a common bath \cite{Nourmandipour2016JOSAB}, and Bernu et al. froze the coherent field growth in a cavity \cite{Bernu2008PRL}.
Very recently, Facchi et al. studied the large-time limit of the quantum Zeno effect \cite{Facchi17JMP}.

Quantum Zeno dynamics can also be used for realizing controlled operations and creating entanglement. Creation of entanglement is a major issue in quantum information science, requiring controlled operations such as CNOT gates between qubits, which is usually a demanding task. As the number of qubits exceeds two, multipartite entanglement emerges in inequivalent classes such as GHZ, W and cluster states, which cannot be transformed into each other via local operations and classical communications. The preparation of multipartite entangled states -especially W states- require not only even more controlled operations but also novel methods \cite{OzaydinW1, OzaydinW2, OzaydinW3, OzaydinW4, OzaydinW5}. Wang et al. proposed a collective threshold measurement scheme for creating bipartite entanglement, avoiding the difficulty of applying CNOT gates or performing Bell measurements \cite{Wang2008PRA}, which can be extended to multipartite entangled states. Chen et al. proposed to use Zeno dynamics for generation of W states robust against decoherence and photon loss \cite{Chen2016OptComm} and Barontini et al. experimentally demonstrated the deterministic generation of W states by quantum Zeno dynamics \cite{Barontini2015Science}. Nakazoto et al. further showed that purifying quantum systems is possible via Zeno-like measurements \cite{Nakazoto20013PRL}.

Optical analogue of the quantum Zeno effect has been receiving an increasing attention.
Yamane et al. reported Zeno effect in optical fibers \cite{Yamane2001OptComm}.
Longhi proposed an optical lattice model including tunneling-coupled waveguides for the observation of the optical Zeno effect \cite{Longhi2006PRL}.
Leung and Ralph proposed a distillation method for improving the fidelity of optical Zeno gates \cite{Leung2006PRA}.
Biagioni et al. experimentally demonstrated the optical Zeno effect by scanning tunneling optical microscopy \cite{Biagioni2008OptExp}.
Abdullaev et al. showed that it is possible to observe the optical analog of not only linear but also nonlinear quantum Zeno effects in a simple coupler and they further proposed a setup for the experimental demonstration of these effects \cite{Abdullaev2011PRA}.
McCusker et al. utilized quantum Zeno effect for the experimental demonstration of the interaction-free all-optical switching \cite{Kumar2013PRL}.
Thapliyal et al. studied quantum Zeno and anti-Zeno effects in nonlinear optical couplers \cite{Thapliyal2016PRA}.

In this paper we numerically investigate the optical analogue of quantum Zeno dynamics of the rogue waves that are encountered in optics. With this motivation, in the second section of this paper we review the NLSE and the split-step Fourier method for its numerical solution. We also review a procedure applied to wavefunction to model the Zeno dynamics of an observed system. In the third section of this paper, we analyze the  Zeno dynamics of the Akhmediev breathers, Peregrine and Akhmediev-Peregrine soliton solutions of the NLSE, which are used as models to describe the rogue waves. We show that frequent measurements of the wave inhibits the movement of the wave in the observation domain for each of these types of rogue waves. We also analyze the spectra of the rogue waves under Zeno dynamics and discuss the effect of observation frequency on the rogue wave profile and on the probability of lingering of the wave in the observation domain. In the last section we conclude our work and summarize future research tasks.

\section{Nonlinear Schr\"{o}dinger Equation and Zeno Effect}

It was shown that all the features of linear quantum mechanics can be reproduced by NLSE \cite{Richardson2014PRA}, and quantum NLSE can accurately describe quantum optical solitons in photonic waveguides with Kerr nonlinearity \cite{Carter1987PRL,Drummond1987JOSAB,Lai1989PRA844,Lai1989PRA854}.
The bosonic matter wave field for weakly interacting ultracold atoms in a Bose-Einstein condensate, evolves according to quantum NLSE \cite{LeggettRMP2001,MorschRMP2006}.
Many nonlinear phenomena observed in fiber optics are generally studied in the frame of the NLSE \cite{Akhmediev2009b}. Optical rogue waves are one of those phenomena and rational rogue wave soliton solutions of the NLSE are accepted as accurate optical rogue wave models \cite{Akhmediev2009b}. In order the analyze the Zeno dynamics of rogue waves, we consider the nondimensional NLSE given as
\begin{equation}
i\psi_t + \frac{1}{2} \psi_{xx} +  \left|\psi \right|^2 \psi =0,
\label{eq01}
\end{equation}
where $x$ and $t$ are the spatial and temporal variables, respectively, $i$ is the imaginary number, and $\psi$ is the complex amplitude. It is known that the NLSE given by Eq.(\ref{eq01}) admits many different types of analytical solutions. Some of these solutions are reviewed in the next section of this paper. For arbitrary wave profiles, where the analytical solution is unknown, the NLSE can be numerically solved by a split-step Fourier method (SSFM), which is one of the most commonly used forms of the spectral methods. Similar to other spectral methods, the spatial derivatives are calculated using spectral techniques in SSFM. Some applications of the spectral techniques can be seen in Refs.\cite{bay2009, Bay_arxNoisyTun, BayTWMS2016, bay_cssfm, Agrawal, Bay_arxNoisyTunKEE, demiray, Bay_arxEarlyDetectCS, Karjadi2010, Karjadi2012, Bay_cssfmarx, bayindir2016nature, Bay_arxChaotCurNLS, BayPRE1, BayPRE2, Bay_CSRM} and their more comprehensive analysis can be seen in Refs.\cite{canuto, trefethen}.

 The temporal derivatives in the governing equations is calculated using time integration schemes such as Adams-Bashforth and Runge-Kutta, etc. \cite{canuto, trefethen, demiray}. However, SSFM uses an exponential time stepping function for this purpose. SSFM is based on the idea of splitting the equation into two parts, namely the linear and the nonlinear parts. Then time stepping is performed starting from the initial conditions. In a possible splitting we take the first part of the NLSE as
\begin{equation}
i\psi_t= -\left| \psi \right|^2\psi
\label{eq02}
\end{equation}
which can exactly be solved as
\begin{equation}
\tilde{\psi}(x,t_0+\Delta t)=e^{i \left| \psi(x,t_0)\right|^2  \Delta t}\ \psi_0,
\label{eq03}
\end{equation}
where $\Delta t$ is the time step and $\psi_0=\psi(x,t_0)$ is the initial condition. The second part of the NLSE can be written as
\begin{equation}
i\psi_t=- \frac{1}{2} \psi_{xx}.
\label{eq04}
\end{equation}
Using a Fourier series expansion we obtain
 \begin{equation}
\psi(x,t_0+\Delta t)=F^{-1} \left[e^{-i k^2 /2 \Delta t}F[\tilde{\psi}(x,t_0+\Delta t) ] \right],
\label{eq05}
\end{equation}
where $k$ is the wavenumber \cite{bay_cssfm, Agrawal}. Substituting Eq.(\ref{eq03}) into Eq.(\ref{eq05}), the final form of the SSFM becomes
 \begin{equation}
\psi(x,t_0+\Delta t)=F^{-1} \left[e^{-i k^2 /2\Delta t}F[ e^{i \left| \psi(x,t_0)\right|^2 \Delta t}\ \psi_0 ] \right].
\label{eq06}
\end{equation}
Starting from the initial conditions, the time integration of the NLSE can be done by the SSFM. Two fast Fourier transform (FFT) operations per time step are needed for this form of the SSFM. The time step is selected as $\Delta t=10^{-3}$, which does not cause a stability problem. The number of spectral components are taken as $M=2048$ in order to use the FFT routines efficiently.

Although it is known that the decay of an atomic particle can be inhibited by Zeno dynamics, it remains an open question whether the rogue waves in the quantized optical fields in the frame of the NLSE can be stopped by Zeno dynamics. In this paper we analyze the Zeno dynamics of such rogue waves by using the SSFM reviewed above. Although analytical solution of the NLSE is known and used as initial conditions in time stepping of SSFM, after a positive Zeno measurement the wavefunction becomes complicated thus numerical solution is needed.
Recently a theoretical wavefunction formulation of the quantum Zeno dynamics is proposed in Ref.\cite{PorrasFreeZeno}, used in Refs.\cite{Porras_Zenotunnelmimick, Porras_diffractionspread} and experimentally tested in Ref.\cite{Porras_zeno_clss_opt}. In this formulation, after a positive measurement the particle is found in the observation domain of $[-L,L]$ with a wavefunction of $\psi_T(x,t)=\psi(x,t)\textnormal{rect}(x/L)/\sqrt{P}$ where $P=\int_{-L}^L \left| \psi \left(x,t \right) \right|^2 dx$, and $\textnormal{rect}(x/L)=1$ for $-L \leq x \leq L$, and $0$ elsewhere \cite{PorrasFreeZeno}. Between two successive positive measurements, the wave evolves according to NLSE. This cycle can be summarized as
\begin{equation}
\begin{split}
\psi_T &  \left(x,\frac{(n-1)t}{N} \right) \stackrel{evolve}{\rightarrow} \psi \left(x,\frac{nt}{N} \right) \stackrel{measure}{\rightarrow} \\
&    \psi_T \left(x,\frac{nt}{N} \right)= \psi \left(x,\frac{nt}{N} \right) \frac{\textnormal{rect(x/L)}}{\sqrt{P_N^{n}}}
\label{eq07}
\end{split}
\end{equation}
where $n$ is the observation index, $N$ is the number of observations \cite{PorrasFreeZeno}, and
 \begin{equation}
P_N^{n}=\int_{-L}^L \left| \psi \left(x,\frac{nt}{N} \right) \right|^2 dx.
\label{eq08}
\end{equation}
The cumulative probability of finding the wave in the observation domain becomes \cite{PorrasFreeZeno}
 \begin{equation}
P_N=\prod_{n=1}^N P_N^{n}.
\label{eq09}
\end{equation}
Using the momentum representation of the linear Schr\"{o}dinger equation and analogy of optical wave dynamics of Fabry-Perot resonator, an analytical derivation of the lingering probability of an atomic particle in the interval of $[-L,L]$ after $n^{th}$ measurement is given as
 \begin{equation}
P_N^{n}\approx 1-0.12 \left(\frac{4}{\pi} \right)^2  \left(\frac{2 \pi t}{N} \right)^{3/2}
\label{eq10}
\end{equation}
in \cite{PorrasFreeZeno}. After $N$ measurements the cumulative probability of finding the particle in the observation domain becomes
 \begin{equation}
P_N \approx \left(1-0.12 \left(\frac{4}{\pi} \right)^2  \left(\frac{2 \pi t}{N} \right)^{3/2} \right)^N
\label{eq11}
\end{equation}
which can further be simplified using Newton's binomial theorem \cite{PorrasFreeZeno}. The reader is referred to Ref.\cite{PorrasFreeZeno} for the details of the derivation of these relations. We compare the analytical relations given in Eqs.(\ref{eq10})-(\ref{eq11}) with the numerical probability calculations in the next section of this paper.

\section{Results and Discussion}

\subsection{Freezing Akhmediev Breathers}
In order to study the Zeno dynamics of rogue waves we first consider the Akhmediev breather (AB) solution of the NLSE given in Eq.(\ref{eq01}). It is known that the NLSE admits a solution in the form of
\begin{equation}
\psi_{AB}=\left[1+\frac{2(1-2a) \cosh{(bt)}+ i b \sinh{(bt)}}{\sqrt{2a} \cos{(\lambda x)}-\cosh{(bt)} }  \right] \exp{[it]}
\label{eq12}
\end{equation}
where $a$ is a free parameter, $\lambda=2 \sqrt{1-2a}$ and $b=\sqrt{8a(1-2a)}$ \ \cite{AkhmedievBreather, AkhmedievBreatherExp}. This solution is known as AB and plays an essential role in describing the modulation instability mechanism and rogue wave generation. Experiments have demonstrated that the ABs can exist in optical fibers \cite{AkhmedievBreatherExp}. The triangularization of the Fourier spectrum, i.e. the triangular supercontinuum generation, plays a key role in the generation and early detection mechanisms of rogue waves. Thus, we also examine the spectra of ABs under Zeno effect.

In Fig.~\ref{fig1}, 3D plots of the AB for $a=0.45$ and its Fourier spectrum are depicted in the first row. This AB is freely evolving for the time interval of $t=[-5,5]$. In the second row of Fig.~\ref{fig1}, we present the AB under Zeno effect and its Fourier spectrum. This AB evolved freely  in the time interval of $t=[-5,0]$ and it was continuously subjected to Zeno observations in the time interval of $t=[0,5]$ within $L=[-7.5,7.5]$. For a better visualization of the effect of Zeno observations, the wave profile is not normalized in this figure.
\begin{figure}[ht!]
\begin{center}
   \includegraphics[width=3.4in]{fig1.pdf}
  \end{center}
\caption{\small Zeno dynamics of an Akhmediev breather with $a=0.45$ a) freely evolving wave b) its Fourier spectrum c) continuously observed Akhmediev breather in $[-7.5,7.5]$ during $t=[0-5]$  d) its Fourier spectrum.}
  \label{fig1}
\end{figure}

\begin{figure}[htb!]
\begin{center}
   \includegraphics[width=3.4in]{fig2.pdf}
  \end{center}
\caption{\small Zeno dynamics of an Akhmediev breather with $a=0.45$ a) freely evolving wave b) its Fourier spectrum c) continuously observed Akhmediev breather in $[-5,5]$ during $t=[0-5]$  d) its Fourier spectrum.}
  \label{fig2}
\end{figure}

Next we apply the similar procedure to the AB, but with a narrower region of observation which is selected as $L=[-5,5]$ to illustrate the effects of narrowing the observation domain as depicted in Fig.~\ref{fig2}. Due to Fourier duality principle, the AB frozen in this fashion has a wider spectrum compared to its counterpart presented in Fig.~\ref{fig1}.

Depending on the width of the Zeno observation domain, it is possible to freeze the whole AB and thus its triangular spectrum during the observation time.

\subsection{Freezing Peregrine Soliton}
Rogue waves of the NLSE is considered to be in the form of rational soliton solutions \cite{Akhmediev2009b}. The simplest rational soliton solution of the NLSE is the Peregrine soliton \cite{Peregrine, Kibler}. It is given by
\begin{equation}
\psi_1=\left[1-4\frac{1+2it}{1+4x^2+4t^2}  \right] \exp{[it]}
\label{eq13}
\end{equation}
where $t$ and $x$ denotes the time and space, respectively \cite{Akhmediev2009b}. This solution can be recovered as the limiting case of the AB when the period of the solution tends to infinity. It has been shown that Peregrine soliton is only a first order rational soliton solution of the NLSE \cite{Akhmediev2009b}. Higher order rational soliton solutions of the NLSE and a hierarchy of obtaining those rational solitons based on Darboux transformations are given in \cite{Akhmediev2009b}. Throughout many simulations \cite{Akhmediev2009b, Akhmediev2009a, Akhmediev2011} and some experiments \cite{Kibler}, it has been confirmed that rogue waves can be in the form of the first (Peregrine) and higher order rational soliton solutions of the NLSE.

\begin{figure}[htb!]
\begin{center}
   \includegraphics[width=3.4in]{fig3.pdf}
  \end{center}
\caption{\small Zeno dynamics of a Peregrine soliton a) freely evolving wave b) its Fourier spectrum c) continuously observed Peregrine soliton in the interval of $[-7.5,7.5]$ during $t=[0-5]$  d) its Fourier spectrum.}
  \label{fig3}
\end{figure}

Similar to the AB case, in order to analyze the Zeno dynamics of the Peregrine soliton we apply the procedure described by Eqs.(\ref{eq07})-(\ref{eq09}). In Fig.~\ref{fig3}, plots of the Peregrine soliton and its Fourier spectrum are shown in the first row. This Peregrine soliton evolved freely during the time interval of $t=[-5,5]$. In the second row of Fig.~\ref{fig3}, we present the Peregrine soliton inhibited by Zeno observations and its corresponding Fourier spectrum. In this plot the Peregrine soliton evolved freely during the temporal interval of $t=[-5,0]$ and it was continuously subjected to Zeno observations in the time interval of $t=[0,5]$ within $L=[-7.5,7.5]$. For a better visualization of the effect of Zeno observations, the wave profile is not normalized in this figure as before. Again depending on the width of the Zeno observation domain, it is possible to freeze the Peregrine soliton wholly or partially and thus its triangular spectrum during the observation time.

\begin{figure}[htb!]
\begin{center}
   \includegraphics[width=3.4in]{fig4.pdf}
  \end{center}
\caption{\small Probability density (unnormalized) after N intermediate measurements of the Peregrine soliton in the interval $[-0.8,0.8]$ for a time $t=n/N=2$.}
  \label{fig4}
\end{figure}

\begin{figure}[htb!]
\begin{center}
   \includegraphics[width=3.4in]{fig5.pdf}
  \end{center}
\caption{\small The probability of finding the Peregrine soliton in the interval $[-0.8,0.8]$ for a time of $t=n/N=2$ after N intermediate measurements.}
  \label{fig5}
\end{figure}

In Fig.~\ref{fig4}, we present the unnormalized wave profile with different observation numbers ($N$). For a larger number of intermediate measurements, the decay of Peregrine soliton is inhibited for longer times compared to the case of fewer Zeno observations. This is due to the fact that for a smaller number of observations the evolution time between two successive observations increases. Thus the wave profile diffuses in this relatively large temporal interval. The constant background level of unity is diminishes since the probability vanishes in the unobserved regions of the evolution domain for a positive measurement.

In Fig.~\ref{fig5}, we present the normalized probabilities of finding the wave in the observation domain for a time interval of $t=[0, 2]$. The dashed lines show the probabilities obtained analytically by Eqs.(\ref{eq10})-(\ref{eq11}) whereas the continuous lines represent numerical results. Since the numerical results are obtained using the NLSE and the analytical distributions given by Eqs.(\ref{eq10})-(\ref{eq11}) rely on the assumption that the particle's motion is governed by linear Schr\"{o}dinger equation, some discrepancies appear between two results, where the discrepancies are less for more frequent observations as expected.

\subsection{Freezing Akhmediev-Peregrine Soliton}

Second order rational soliton solution of the NLSE is Akhmediev-Peregrine soliton \cite{Akhmediev2009b}, which is considered to be a model for rogue waves with higher amplitude than the Peregrine soliton. The formula of Akhmediev-Peregrine soliton is given as
\begin{equation}
\psi_2=\left[1+\frac{G_2+it H_2}{D_2}  \right] \exp{[it]}
\label{eq14}
\end{equation}
where
\begin{equation}
G_2=\frac{3}{8}-3x^2-2x^4-9t^2-10t^4-12x^2t^2
\label{eq14}
\end{equation}
\begin{equation}
H_2=\frac{15}{4}+6x^2-4x^4-2t^2-4t^4-8x^2t^2
\label{eq16}
\end{equation}
and
\begin{equation}
\begin{split}
D_2=\frac{1}{8} & [ \frac{3}{4}+9x^2+4x^4+\frac{16}{3}x^6+33t^2+36t^4 \\
& +\frac{16}{3}t^6-24x^2t^2+16x^4t^2+16x^2t^4 ]
\label{eq17}
\end{split}
\end{equation}
where $t$ is the time and $x$ is the space parameter \cite{Akhmediev2009b}. Using Darboux transformation formalism this soliton can be obtained using the Peregrine soliton as the seed solution \cite{Akhmediev2009b}. Many numerical simulations also confirm that rogue waves in the NLSE framework can also be in the form of Akhmediev-Peregrine soliton \cite{Akhmediev2011, Akhmediev2009b, Akhmediev2009a}. However, to our best knowledge an experimental verification of this soliton still does not exist.

\begin{figure}[htb!]
\begin{center}
   \includegraphics[width=3.4in]{fig6.pdf}
  \end{center}
\caption{\small Zeno dynamics of an Akhmediev-Peregrine soliton a) freely evolving wave b) its Fourier spectrum c) continuously observed Akhmediev-Peregrine soliton in the interval of $[-7.5,7.5]$ during $t=[0-5]$  d) its Fourier spectrum.}
  \label{fig6}
\end{figure}

As in the AB and Peregrine soliton cases, in order to analyze the Zeno dynamics of the Akhmediev-Peregrine soliton we apply the procedure described by Eqs.(\ref{eq07})-(\ref{eq09}). Similarly, in Fig.~\ref{fig6}, the Akhmediev-Peregrine soliton, which evolved freely during the time interval of $t=[-5,5]$, and its Fourier spectrum are plotted in the first row. In the second row of Fig.~\ref{fig6}, the Akhmediev-Peregrine soliton inhibited by Zeno observations and its corresponding Fourier spectrum are depicted. In this plot the Akhmediev-Peregrine soliton evolved freely during the temporal interval of $t=[-5,0]$ and continuous Zeno observations took place time within $L=[-7.5,7.5]$ during the time interval of $t=[0,5]$.

\begin{figure}[htb!]
\begin{center}
   \includegraphics[width=3.4in]{fig7.pdf}
  \end{center}
\caption{\small Probability density after N intermediate measurements of the Akhmediev-Peregrine soliton in the interval $[-1.7,1.7]$ for a time $t=n/N=2$.}
  \label{fig7}
\end{figure}
Again unnormalized wave profile is depicted in this figure for a better visualization of the effect of Zeno observations on the Akhmediev-Peregrine soliton.  Again depending on the width of the Zeno observation domain, it is possible to freeze the Akhmediev-Peregrine soliton wholly or partially and thus its triangular spectrum can be preserved during the observation time.

In Fig.~\ref{fig7}, the unnormalized wave profiles for different observation numbers ($N$) are shown. Similar to the previous cases, for a larger number of intermediate measurements, the decay of Akhmediev-Peregrine soliton is inhibited for longer times compared to the case of fewer Zeno observations due to shorter diffusion time between two positive Zeno observations.  Additionally, the peak as well as two dips of the Akhmediev-Peregrine soliton can be preserved during measurements depending on the length of the observation domain. Similarly, since the probability vanishes in the unobserved regions of the evolution domain for a positive measurement, the constant background level of unity is diminishes.

\begin{figure}[h!]
\begin{center}
   \includegraphics[width=3.4in]{fig8.pdf}
  \end{center}
\caption{\small The probability of finding the Akhmediev-Peregrine soliton in the interval $[-0.8,0.8]$ for a time of $t=n/N=2$ after N intermediate measurements.}
  \label{fig8}
\end{figure}
In Fig.~\ref{fig8}, we present the normalized probabilities of finding the wave in the observation domain for a time interval of $t=[0, 2]$. The dashed lines show the probabilities obtained analytically by Eqs.(\ref{eq10})-(\ref{eq11}) whereas the continuous lines represent numerical results. As before, since the numerical results are obtained using the NLSE and the analytical distributions given by Eqs.(\ref{eq10})-(\ref{eq11}) rely on the assumption that the particle's motion is governed by linear Schr\"{o}dinger equation, some discrepancies appear between two results, where the discrepancies are less for more frequent observations as expected. However the discrepancies are slightly higher compared to the Zeno dynamics of the Peregrine soliton depicted in Fig.~\ref{fig5} due to increased steepness of the Akhmediev-Peregrine soliton.

\section{Conclusion}

In this paper we have numerically investigated the Zeno dynamics of optical rogue waves in the frame of the standard NLSE. In particular, we have analyzed the Zeno dynamics of the Akhmediev breathers, Peregrine and Akhmediev-Peregrine soliton solutions of the NLSE, which are considered as accurate rogue wave models. We have showed that frequent measurements of the rogue wave inhibits its movement in the observation domain for each of these solutions. We have analyzed the spectra of the rogue waves to observe the supercontinuum generation under Zeno observations. Fourier as well as the wavelet spectra of rogue waves under Zeno dynamics may give some clue about the application time (position) of the Zeno observations to freeze the emergence or decay of rogue waves. This would especially be important for the Zeno dynamics of stochastic wavefields which produce rogue waves. We have also analyzed the effect of observation frequency on the rogue wave profile and on the probability of freezing the wave in the observation domain. We have showed that the rogue wave shape can be preserved for longer times and the probability of the freezing the rogue wave increases for more frequent observations, for all three types of rogue waves considered.

The revival dynamics, that is the dynamics after Zeno observations are ceased, of rogue waves will be a part of future work. The analysis of statistical distributions and the shapes of rogue waves in the stochastic fields after Zeno observation remains as an problem which needs further attention as well.
We believe that the results presented herein may open new insights in modelling the dynamics of standing and propagating optical rogue waves.
More specifically, the procedure analyzed in this paper in the frame of the standard NLSE can be used to advance the fields of optical communications and vibrations with special applications which include but are not limited to freezing and steering the rogue and other wave types, avoiding their breaking, imposing a time delay by Zeno effect. The procedure analyzed in this paper in the frame of the standard NLSE can also be extended to model the Zeno dynamics of the many other fascinating nonlinear phenomena which can be used to advance the optical science and technology.

\section*{Acknowledgment}
FO is funded by Isik University Scientific Research Funding Agency under Grant Number: BAP-15B103.