\subsubsection*{\bibname}}

\usepackage{parskip}
\input{math}
\usepackage{xspace}
\usepackage{ifthen}
\usepackage{todonotes}

\usepackage{hyperref}
\hypersetup{
    linkbordercolor=cyan,
    urlbordercolor=magenta,
    citebordercolor=green,
}

\newcommand{\topic}[1]{\vspace{1mm}\noindent\textbf{#1}}
\newcommand{\meanstd}[3][]{
    \ifthenelse{ \equal{#1}{} }
    {#2 {\scriptstyle \pm #3}}
    {#2 {\scriptstyle \pm #3}\times10^{#1}}
}
\newcommand{GsKL\xspace}{GsKL\xspace}

\newcommand{D}{D}
\newcommand{\normpdf}[3]{\mathcal{N}\left({#1}; {#2}, {#3 } \right)}
\newcommand{x}{x}
\newcommand{\mathbf{X}}{\mathbf{X}}
\newcommand{\mathbf{y}}{\mathbf{y}}
\newcommand{\psi}{\psi}
\newcommand{C}{C}
\newcommand{\sigma}{\sigma}

\begin{document}

\twocolumn[

\aistatstitle{Parallel MCMC Without Embarrassing Failures}

\aistatsauthor{Daniel Augusto de Souza$^1$, Diego Mesquita$^{2,3}$, Samuel Kaski$^{2,4}$, Luigi Acerbi$^5$}

\aistatsaddress{ $^1$University College London  $^2$Aalto University $^3$Getulio Vargas Foundation\\ $^4$University of Manchester $^5$University of Helsinki\\ \texttt{\small daniel.souza.21@ucl.ac.uk, diego.mesquita@fgv.br, samuel.kaski@aalto.fi,  luigi.acerbi@helsinki.fi} }]

\runningauthor{Daniel Augusto de Souza, Diego Mesquita, Samuel Kaski, Luigi Acerbi}

\begin{abstract}
\emph{Embarrassingly parallel} Markov Chain Monte Carlo (MCMC) exploits parallel computing to scale Bayesian inference to large datasets by using a two-step approach. First, MCMC is run in parallel on (sub)posteriors defined on data partitions. Then, a server combines local results. While efficient, this framework is very sensitive to the quality of subposterior sampling. Common sampling problems such as missing modes or misrepresentation of low-density regions are amplified  -- instead of being corrected -- in the combination phase, leading to catastrophic failures.
In this work, we propose a novel combination strategy to mitigate this issue. Our strategy, Parallel Active Inference (PAI), leverages Gaussian Process (GP) surrogate modeling and active learning. After fitting GPs to subposteriors, PAI (i) shares information between GP surrogates to cover missing modes; and (ii) uses active sampling to individually refine subposterior approximations.  We validate PAI in challenging benchmarks, including heavy-tailed and multi-modal posteriors and a real-world application to computational neuroscience. Empirical results show that PAI succeeds where previous methods catastrophically fail, with a small communication overhead.
\end{abstract}

\section{INTRODUCTION}

Markov Chain Monte Carlo (MCMC) methods have become a gold standard in Bayesian statistics \citep{gelman2013bayesian,Carpenter2017}.
However, scaling MCMC methods to large datasets is challenging due to their sequential nature and that they typically require many likelihood evaluations, implying repeated sweeps through the data.
Various approaches that leverage distributed computing have been proposed to mitigate these limitations \citep{Angelino+others:2016, Robert2018}.
In general, we can split these approaches between those that incur constant communication costs and those requiring frequent interaction between server and computing nodes \citep{vehtari2020expectation}.

\emph{Embarrassingly parallel} MCMC~\citep{Neiswanger2014} is a popular class of methods which employs a divide-and-conquer strategy to sample from a target posterior, requiring only a single communication step. For dataset $\mathcal{D}$ and model parameters $\theta \in \mathbb{R}^D$, suppose we are interested in the  Bayesian posterior $\post \propto p(\theta) p(\mathcal{D} | \theta)$, where $p(\theta)$ is the prior and $p(\mathcal{D} | \theta)$ the likelihood. Embarrassingly parallel methods begin by splitting the data $\mathcal{D}$ into $K$ smaller partitions $\mathcal{D}_1, \ldots, \mathcal{D}_K$ so that we can rewrite the posterior as
\vspace{-0.25em}
\begin{equation}\label{eq:pde}
    \post \propto \prod_{k=1}^{K} p(\theta)^{1/K} p(\data_k | \theta) \equiv
    \prod_{k=1}^{K} p_k(\theta).
\vspace{-0.2em}
\end{equation}
Next, an MCMC sampler is used to draw samples $\samples_k$ from each \emph{subposterior} $p_k(\theta)$, for $k=1\ldots K$, in parallel. Then, the computing nodes send the local results to a central server, for a final aggregation step. These local results are either the samples themselves or approximations $q_1, \ldots, q_K$ built using them.

Works in embarrassingly parallel MCMC mostly focus on combination strategies.
\citet{Scott2016} employ a weighted average of subposterior samples.
\citet{Neiswanger2014} propose using multivariate-normal surrogates  as well as non-parametric and semi-parametric forms.
\citet{Wang2015} combine subposterior samples into a hyper-histogram with random partition trees.
\citet{Nemeth2018} leverage density values computed during MCMC to fit Gaussian process (GP) surrogates to log-subposteriors.
\citet{Mesquita2019} use subposterior samples to fit normalizing flows and apply importance sampling to draw from their product.

Despite these advances, parallel MCMC suffers from an unaddressed limitation: its dependence on high-quality subposterior sampling.
This requirement is especially difficult to meet when subposteriors are multi-modal or heavy-tailed, in which cases MCMC chains often visit only a subset of modes and may underrepresent low-density regions.
Furthermore, the surrogates $(q_k)_{k=1}^{K}$ built only on local MCMC samples might match poorly the true subposteriors if not carefully tuned.

\paragraph{Outline and contributions.}  We first discuss the failure modes of parallel MCMC (Section \ref{sec:embarrassing}).
Drawing insight from this discussion, Section \ref{sec:method} proposes a novel GP-based solution, Parallel Active Inference (PAI).
After fitting the subposterior surrogates, PAI shares a subset of samples between computing nodes to prevent mode collapse.
PAI also uses active learning to refine low-density regions and avoid catastrophic model mismatch.
Section \ref{sec:experiments} validates our method on challenging benchmarks and a real-world neuroscience example.
Finally, Section \ref{sec:related_works} reviews related work and Section \ref{sec:discussion} discusses strengths and limitations of PAI.

\section{EMBARRASSINGLY PARALLEL MCMC: HOW CAN IT FAIL?}
\label{sec:embarrassing}

\begin{figure*}[tb!]
\centering
\includegraphics[width=0.9\linewidth]{figures/illustrations/f1_fixed2.pdf}
  \caption{{\bf Failure modes of embarrassingly parallel MCMC.} \textbf{A--C}. Each column illustrates a distinct failure type described in Section \ref{sec:failures}. For each column, the top rows show two subposteriors $p_k(\theta)$ (black dashed line: ground truth; blue circles: MCMC samples), and the bottom row shows the full posterior $p(\theta|\mathcal{D})$ with the approximate posterior combined using the method specified in the legend (orange line). These failure modes are general and not unique to the displayed algorithms (see Appendix \ref{supp:failure_modes} %
  for details and further explanations).
  }
  \label{fig:toy}
\end{figure*}

We recall the basic structure of a generic embarrassingly parallel MCMC algorithm in Algorithm \ref{alg:basic}.
This schema has major failure modes that we list below, before discussing potential solutions. We also illustrate these pathologies in Fig \ref{fig:toy}.
In this paper, for a function $f$ with scalar output and a set of points $\samples = \{s_1, \ldots, s_N \}$, we denote with $f(\samples) \equiv \{f(s_1), \ldots, f(s_N)\}$.

\begin{algorithm}[h!]
\caption{Generic embarrassingly parallel MCMC}\label{alg:basic}
\begin{algorithmic}[1] \small
\INPUT Data partitions $\data_1, \ldots, \data_K$; prior $p(\theta)$; likelihood function $p(\data|\theta)$.
\ParFor{$1 \ldots K$} \Comment{Parallel steps}
\State $\samples_k \leftarrow $ MCMC samples from $p_k(\theta) \propto p(\theta)^{1/K} p(\data_k | \theta)$
\State build subposterior model $q_k(\theta)$ from $\samples_k$
\EndParFor
\State Combine: $q(\theta) \propto \prod_{k=1}^K q_k(\theta) $ \Comment{Centralized step}
\end{algorithmic}
\end{algorithm}

\subsection{Failure modes}
\label{sec:failures}

\paragraph{I: Mode collapse.}

It is sufficient that \emph{one} subposterior $q_k$ misses a mode for the combined posterior $q$ to lack that mode (see Fig \ref{fig:toy}A). While dealing with multiple modes is an open problem for MCMC, here the issue is exacerbated by the fact that a single failure propagates to the final solution. A back-of-the-envelope calculation shows that even if the chance of missing a mode is small, $\varepsilon > 0$, the probability of mode collapse in the combined posterior is $\approx (1 - \varepsilon)^K$ making it a likely occurrence for sufficiently large $K$.

\begin{tcolorbox}[colback=red!5!white,colframe=red!75!black]
\textbf{Insight 1:}
For multimodal posteriors, mode collapse is almost inevitable unless the computing nodes can exchange information about the location of important posterior regions.
\end{tcolorbox}

\paragraph{II: Catastrophic model mismatch.}

Since the $q_k$ are approximations of the true subposteriors $p_k$, small deviations between them are expected -- this is not what we refer to here. Instead, an example of \emph{catastrophic} model mismatch is when a simple parametric model such as a multivariate normal is used to model a multimodal posterior with separate modes (see Section \ref{sec:experiments}).
Even nonparametric methods can be victims of this failure.
For example, GP surrogates are often used to model nonparametric deviations of the log posterior from a parametric `mean function'. While these models can well represent multimodal posteriors, care is needed to avoid grossly mismatched solutions in which a $q_k$ `hallucinates' posterior mass due to an improper placement of the GP mean function (Fig \ref{fig:toy}B).

\begin{tcolorbox}[colback=red!5!white,colframe=red!75!black]
\textbf{Insight 2:} We cannot take subposterior models at face value. Reliable algorithms should check and refine the $q_k$'s to avoid catastrophic failures.
\end{tcolorbox}

\paragraph{III: Underrepresented tails.} This effect is more subtle than the failure modes listed above, but it contributes to accumulating errors in the estimate of the combined posterior. The main issue here is that, by construction, MCMC samples and subposterior models based on these samples focus on providing information about the high-posterior-mass region of the subposterior. However, different subposteriors may overlap only in their tail regions (Fig \ref{fig:toy}C), implying that the tails and the nearby `deep' regions of each subposterior might actually be the most important in determining the exact shape of the combined posterior.

\begin{tcolorbox}[colback=red!5!white,colframe=red!75!black]
 \textbf{Insight 3:} Subposterior models built only from MCMC samples (and their log density) can miss important information about the tails and nearby regions of the subposterior which would also contribute to the combined posterior.
\end{tcolorbox}

\subsection{Past solutions}
\label{sec:past}

Since there is no guarantee that $q$ approximates well the posterior $p$, \citet{Nemeth2018} refine $q$ with an additional parallel step, called Distributed Importance Sampler (DIS). DIS uses $q$ as a proposal distribution and draws samples $\samples \sim q$, that are then sent back for evaluation of the log density $\log p_k(\samples)$ at each parallel node. The true log density $\log p(\samples) = \sum_k \log p_k(\samples)$ is then used as a target for \emph{importance sampling/resampling}  \citep{robert2013monte}. Technically, this step makes the algorithm not `embarrassingly parallel' anymore, but the prospect of fixing catastrophic failures outweighs the additional communication cost. However, DIS does not fully solve the issues raised in Section \ref{sec:failures}. Notably, importance sampling will not help recover the missing regions of the posterior if $q$ does not cover them in the first place. DIS can help in some model mismatch cases, in that `hallucinated' regions of the posterior will receive near-zero weights after the true density is retrieved.

\subsection{Proposed solution}
\label{sec:solutions}

Drawing from the insights in Section \ref{sec:failures}, we propose two key ideas to address the blind spots of embarrassingly parallel MCMC. Here we provide an overview of our solution, which is described in detail in Section \ref{sec:method}. The starting point is modeling subposteriors via Gaussian process surrogates (Fig \ref{fig:sols}A).

\paragraph{Sample sharing.} We introduce an additional step in which each node shares a selected subset of MCMC samples with the others (Fig \ref{fig:sols}B). This step provides sufficient information for local nodes to address mode collapse and underrepresented tails. While this communication step makes our method not strictly `embarrassingly' parallel, we argue it is necessary to avoid posterior density collapse. Moreover, existing methods already consider an extra communication step \citep{Nemeth2018}, as mentioned in Section \ref{sec:past}.

\paragraph{Active learning.} We use \emph{active learning} as a general principle whenever applicable. The general idea is to select points that are informative about the shape of the subposterior, minimizing the additional communication required.
Active learning is used here in multiple steps: when selecting samples from MCMC to build the surrogate model (as opposed to thinning or random subsampling); as a way to choose which samples from other nodes to add to the current surrogate model of each subposterior $q_k$ (only informative samples are added); to actively sample \emph{new} points to reduce uncertainty in the local surrogate $q_k$ (Fig \ref{fig:sols}C). Active learning contributes to addressing both catastrophic model mismatch and underrepresented tails.

Combined, these ideas solve the failure modes    discussed previously (Fig \ref{fig:sols}D).

\begin{figure*}[t!]
\centering
\includegraphics[width=0.95\linewidth]{figures/illustrations/f2_2.pdf}
\vspace{-1em}
  \caption{{\bf Parallel active inference (PAI).} \textbf{A.} Each log subposterior $\log p_k(\theta)$ (black dashed line) is modeled via Gaussian process surrogates (orange dashed line: mean GP; shaded area: 95\% confidence interval) trained on MCMC samples $\mathcal{S}_k$ (blue circles) and their log-density $\log p_k(\mathcal{S}_k)$. Here, MCMC sampling on the second subposterior has missed a mode. \textbf{B.} Selected subsets of MCMC samples are shared across nodes, evaluated locally and added to the GP surrogates. Here, sample sharing helps finding the missing mode in the second subposterior, but the GP surrogate is now highly uncertain outside the samples. \textbf{C.} Subposteriors are refined by actively selecting new samples (stars) that resolve uncertainty in the surrogates. \textbf{D.} Subposteriors are combined into the full approximate log posterior (orange line); here a perfect match to the true log posterior (black dashed line).
  \label{fig:sols}
  }
\end{figure*}

\section{PARALLEL ACTIVE INFERENCE}
\label{sec:method}

In this section, we present our framework, which we call Parallel Active Inference (PAI), designed to address the issues discussed in Section \ref{sec:embarrassing}. The steps of our method are schematically explained in Fig \ref{fig:sols} and the detailed algorithm is provided in Appendix \ref{supp:algorithm}.

\subsection{Subposterior modeling via GP regression}
\label{sec:gpsubposteriors}

As per standard embarrassingly parallel algorithms, we assume each node computes a set of samples $\samples_k$ and their log density, $\log p_k(\samples_k)$, by running MCMC on the subposterior $p_k$.
We model each log-subposterior $\lq_k(\theta) \equiv \log q_k(\theta)$ using GP regression (Fig \ref{fig:sols}A; see \cite{rasmussen2006gaussian,Nemeth2018,gortler2019visual} and Appendix \ref{supp:gps} for more information).
We say that a GP surrogate model is trained on $\samples_k$ as a shorthand for $\left(\samples_k, \log p_k(\samples_k) \right)$.

When building the GP model of the subposterior, it is not advisable to use all samples $\samples_k$ because: (1) exact inference in GPs scales cubically in the number of training points (although see \cite{wang2019exact}); (2) we want to limit communication costs when sharing samples between nodes; (3) there is likely high redundancy in $\samples_k$ about the shape of the subposterior. \citet{Nemeth2018} simply choose a subset of samples by `thinning' a longer MCMC chain at regular intervals. Instead, we employ \emph{active subsampling} as follows.

First, we pick an initial subset of $\ninit$ samples $\samples_k^{(0)} \subset \samples_k$, that
we use to train an initial GP (details in Appendix \ref{sec:step1}). Then, we iteratively select points $\theta^\star$ from $\samples_k$ by maximizing the \emph{maximum interquantile range} (MAXIQR) acquisition function \citep{jarvenpaa2021parallel}:
\begin{equation} \label{eq:maxiqr}
 \begin{split}
\theta^\star
    &= \arg\max_\theta \left\{ e^{m(\theta; \samples_k^{(t)})} \text{sinh}  \left(u \cdot s(\theta; \samples_k^{(t)})\right)\right\},
\end{split}
\end{equation}
where $m(\theta; \samples_k^{(t)})$ and $s(\theta; \samples_k^{(t)})$ are, respectively, the posterior latent mean and posterior latent standard deviation of the GP at the end of iteration $t \geq 0$;
and $\sinh(z) = (\exp(z) -\exp(-z))/2$ for $z \in \mathbb{R}$ is the hyperbolic sine.
Eq. \ref{eq:maxiqr} promotes selection of points with high posterior density for which the GP surrogate is also highly uncertain, with the trade-off controlled by $u > 0$, where larger values of $u$ favor further exploration. In each iteration $t+1$, we greedily select a batch of $n_\text{batch}$ points at a time from $\samples_k \setminus \samples_k^{(t)}$ using a batch version of MAXIQR \citep{jarvenpaa2021parallel}. We add the selected  points to the current training set, $\samples_k^{(t+1)}$, and retrain the GP after each iteration (see Appendix \ref{sec:step1}).
After $T$ iterations, our procedure yields a  subset of points $\subsamples_k \equiv \samples_k^{(T)} \subseteq \samples_k$ that are highly informative about the shape of the subposterior.

\subsection{Sample sharing}
\label{sec:samplesharing}

In this step, each node $k$ shares the selected samples $\subsamples_k$ with all other nodes (Fig \ref{fig:sols}B). Thus, node $k$ gains access to the samples $\subsamples_{\setminus k} \equiv \bigcup_{j \neq k} \subsamples_j$. Importantly, $\subsamples_{\setminus k}$ might contain samples from relevant
subposterior regions that node $k$ has has not explored.
As discussed in Section \ref{sec:gpsubposteriors}, for efficiency we avoid adding \emph{all} points $\subsamples_{\setminus k}$ to the current GP surrogate for subposterior $k$.
Instead, we add a sample $\theta^\star \in \subsamples_{\setminus k}$ to the GP training set only if the prediction of the current GP surrogate deviates from the true subposterior $\log p_k(\theta^\star)$ in a significant way (see Appendix \ref{sec:step2} for details).
After this step, we obtain an expanded set of points $\subsamplestwo_k$ that includes information from all the nodes, minimizing the risk of mode collapse (see Section \ref{sec:failures}).

\subsection{Active subposterior refinement}
\label{sec:activerefinement}

So far, the GP models have been trained using selected subsets of samples from the original MCMC runs. In this step, we refine the GP model of each subposterior by sampling \emph{new} points (Fig \ref{fig:sols}C). Specifically, each node $k$ actively selects new points by optimizing the MAXIQR acquisition function (Eq. \ref{eq:maxiqr}) over $\mathcal{X} \subseteq \mathbb{R}^D$ (see Appendix \ref{sec:step3}). New points are selected greedily in batches of size $n_\text{batch}$, retraining the GP after each iteration.
This procedure yields a refined set of points $\subsamplesthree_k$ which includes new points that better pinpoint the shape of the subposterior, reducing the risk of catastrophic model mismatch and underrepresented tails. The final log-subposterior surrogate model $\lq_k$ is the GP trained on $\subsamplesthree_k$.

\subsection{Combining the subposteriors}
\label{sec:combining}

Finally, we approximate the full posterior $\log \post = \sum_{k=1}^K \log p_k(\theta)$ by combining all subposteriors together (Fig \ref{fig:sols}D). Since each log-subposterior is approximated by a GP, the approximate full log-posterior is a sum of GPs and itself a GP, $\lq(\theta) = \sum_{k=1}^K \lq_k(\theta)$. Note that $\lq(\theta)$, being a GP, is still a distribution over functions. We want then to obtain a point estimate for the (unnormalized) posterior density corresponding to $\exp \lq(\theta)$. One choice is to take the posterior mean, which leads to the expectation of a log-normal density \citep{Nemeth2018}. We prefer a robust estimate and use the posterior median instead \citep{jarvenpaa2021parallel}. Thus, our estimate is
\begin{equation} \label{eq:sumgp}
    q(\theta) \propto \exp\left\{\sum_{k=1}^K m_k(\theta; \subsamplesthree_k) \right\}.
\end{equation}
In low dimension ($D = 1,2$), Eq. \ref{eq:sumgp} can be evaluated on a grid. In higher dimension,
one could sample from $q(\theta)$ using MCMC methods such as NUTS \citep{hoffman2014no}, as done by \cite{Nemeth2018}. However, $q(\theta)$ is potentially multimodal which does not lend itself to easy MCMC sampling.
Alternatively, \citet{acerbi2018variational} runs variational inference on $q(\theta)$ using as variational distribution a mixture of Gaussians with a large number of components. Finally, for moderate $D$, importance sampling/resampling with an appropriate (adaptive) proposal would also be feasible.

As a final optional step, after combining the subposteriors into the full approximate posterior $q(\theta)$, we can refine the solution using \emph{distributed importance sampling} (DIS) as proposed by \citet{Nemeth2018} and discussed in Section \ref{sec:past}.

\subsection{Complexity analysis}

Similarly to conventional embarrassingly parallel MCMC, we can split the cost of running PAI into two main components. The first consists of local costs, which involve computations happening at individual computing nodes (i.e., model fitting and active refinement). The second are global (or aggregation) costs, which comprise communication and sampling from the combined approximate posterior.

\subsubsection{Model fitting}

After sampling their subposterior, each computing node $k$ has to fit the surrogate model on the subset of their samples, $\subsamples_k$. These subsets are designed such that their size is $\mathcal{O}(D)$ (see Appendix \ref{supp:algorithm}). Thus, the cost of fitting the surrogate GP models in each of the $K$ computing nodes is $\mathcal{O}(D^3)$. The same applies for $\subsamplestwo_k$ and $\subsamplesthree_k$.

\subsubsection{Communication costs}
Traditional embarrassingly parallel MCMC methods only require two global communication steps: (1) the central server splits the $N$ observations among $K$ computing nodes; (2) each node sends $S$ subposterior samples of dimension $D$ back to the server, assuming nodes draw the same number of samples. Together, these steps amount to $\mathrm{O}(N + K S D)$ communication cost.

PAI imposes another communication step, in which nodes share their subposterior samples and incurring $\mathrm{O}(K S D)$ cost. Furthermore, supposing PAI acquires $A$ active samples to refine each subposterior, the cost of sending local results to servers is increased by $\mathrm{O}(KAD)$. PAI also incur a small additional cost for sending back the value of the GP hyperparameters $\mathrm{O}(KD)$. In sum, since usually $A \ll S$, the asymptotic communication cost of PAI is equivalent to traditional methods.

\subsubsection{Active refinement costs}
Active learning involves GP training and optimization of the acquisition function, but only a small number of likelihood evaluations. Thus, under the embarrassingly parallel MCMC assumption that likelihood evaluations are costly (e.g., due to large datasets), active learning is relatively inexpensive \citep{acerbi2018variational}. More importantly, as shown in our ablation study in Appendix \ref{sec:ablation}, this step is crucial to avoid the pathologies of embarrassingly parallel MCMC

\subsubsection{Sampling complexity}
Sampling from the aggregate approximate posterior $q(\theta)$ only requires evaluating the GP predictive mean for each subposterior and does not require access to the data or all samples. The sampling cost is linear in the number of subposteriors $K$ and the size of each GP $\mathcal{O}(D)$. Even if $K$ is chosen to scale as the size of the actual data, each GP only requires a small training set, making them comparably inexpensive.

\begin{figure*}[t!]
\centering
\includegraphics%
[height=8cm]%
{figures/experiments/4modes_33.pdf}
\vspace{-1em}
  \caption{\textbf{Multi-modal posterior.} Each panel shows samples from the combined approximate posterior (red) against the ground truth (blue). With exception of PAI, all methods completely miss at least one high-density region. Moreover, PAI is the only method that does not assign mass to regions without modes.}
  \label{fig:exp_4modes}
\end{figure*}

\section{EXPERIMENTS}
\label{sec:experiments}

We evaluate PAI on a series of target posteriors with different challenging features. Subsection \ref{subsec:multi_modal} shows results for a posterior with four distinct modes, which is prone to mode collapse (Fig \ref{fig:toy}A).
Subsection \ref{subsec:heavy_tails} targets a posterior with heavy tails, which can lead to underrepresented tails (Fig \ref{fig:toy}C).
Subsection \ref{subsec:rare_events} uses a rare event model to gauge how well our method performs when the true subposteriors are drastically different.
Finally, Subsection \ref{subsec:neuro} concludes with a real-world application to a model and data from computational neuroscience~\citep{acerbi2018bayesian}.
We provide implementation details in Appendix \ref{appendix:implementation} and source code is available at \url{https://github.com/spectraldani/pai}.

\paragraph{Algorithms.} We compare basic PAI and PAI with the optional distributed importance sampling step (PAI-DIS) against six popular and state-of-the-art (SOTA) embarrassingly parallel MCMC methods: the parametric, non-parametric and semi-parametric methods by \citet{Neiswanger2014}; PART \citep{Wang2015}; and two other GP-surrogate methods \citep{Nemeth2018}, one using a simple combination of GPs (GP) and the other using the distributed importance sampler (GP-DIS; see Section \ref{sec:past}).

\paragraph{Procedure.}
For each problem, we randomly split the data in equal-sized partitions and divide the target posterior into $K$ subposteriors (Eq. \ref{eq:pde}). We run MCMC separately on each subposterior using Stan with multiple chains \citep{Carpenter2017}. The same MCMC output is then processed by the different algorithms described above, yielding a combined approximate posterior for each method.
To assess the quality of each posterior approximation, we compute the mean marginal total variation distance (MMTV), the 2-Wasserstein (W2) distance, and the Gaussianized symmetrized Kullback-Leibler (GsKL\xspace) divergence between the approximate and the true posterior, with each metric focusing on different features.
For each problem, we computed ground-truth posteriors using numerical integration (for $D \le 2$) or via extensive MCMC sampling in Stan \citep{Carpenter2017}.
For all GP-based methods (PAI, PAI-DIS, GP, GP-DIS), we sampled from the potentially multimodal combined GP (Eq. \ref{eq:sumgp}) using importance sampling/resampling with an appropriate proposal.
We report results as mean $\pm$ standard deviation across ten runs in which the entire procedure was repeated with different random seeds. For all metrics, lower is better, and the best (statistically significant) results for each metric are reported in bold. See Appendix \ref{supp:evaluation} for more details.

\subsection{Multi-modal posterior}
\label{subsec:multi_modal}

\paragraph{Setting.}
In this synthetic example, the data consist of $N=10^{3}$ samples $y_1, \ldots, y_N$ drawn from the following hierarchical model:
\begin{equation*}
\label{eq:multimodal}
\begin{split}
    \theta \sim p(\theta) & = \mathcal{N}(0, \sigma_p^2 \mathbb{I}_2)\\
    y_1, \ldots, y_N \sim p(y_n | \theta) & =  \sum_{i=1}^{2}\frac{1}{2}\mathcal{N}\left(P_i(\theta_i), \sigma^2_l\right)
\end{split}
\end{equation*}
where $\theta \in \mathbb{R}^2$, $\sigma_p=\sigma_l=1/4$ and $P_i$'s are second-degree polynomial functions. By construction, the target posterior $p(\theta | y_1, \dots, y_N) \propto p(\theta) \prod_{n=1}^N p(y_n | \theta)$ is multimodal with four modes. We run parallel inference on $K=10$ equal-sized partitions of the data. We provide more details regarding $P_1, P_2$ in Appendix \ref{supp:models}.

\paragraph{Results.} Fig \ref{fig:exp_4modes} shows the output of each parallel MCMC method for a typical run, displayed as samples from the approximate combined posterior overlaid on top of the ground-truth posterior. Due to MCMC occasionally missing modes in subposterior sampling, the combined posteriors from all methods except PAI lack at least one mode of the posterior (\emph{mode collapse}, as seen in Fig \ref{fig:toy}A).
Other methods also often inappropriately distribute mass in low-density regions (as seen in Fig \ref{fig:toy}B). In contrast, PAI accurately recovers all the high-density regions of the posterior achieving a near-perfect match. Table \ref{tab:multi-modal} shows that PAI consistently outperforms the other methods in terms of metrics.

\begin{table}[h!]
    \caption{\textbf{Multi-modal posterior.}

    }
    \centering
\resizebox{\columnwidth}{!}{
    \begin{tabular}{l c c c }
    \toprule
    \bf{Model}  &  \bf{MMTV} & \bf{W2} & \bf{GsKL\xspace}\\
    \toprule
    Parametric&
        $\meanstd{0.89}{0.12}$&
        $\meanstd{1.08}{0.33}$&
        $\meanstd[2]{8.9}{11}$\\
    Semi-param.&
        $\meanstd{0.81}{0.09}$&
        $\meanstd{1.08}{0.12}$&
        $\meanstd[1]{5.6}{1.3}$\\
    Non-param.&
        $\meanstd{0.81}{0.09}$&
        $\meanstd{1.12}{0.09}$&
        $\meanstd[1]{5.0}{1.8}$\\
    PART&
        $\meanstd{0.55}{0.09}$&
        $\meanstd{1.06}{0.33}$&
        $\meanstd[2]{7.3}{14}$\\
    GP&
        $\meanstd{0.93}{0.16}$&
        $\meanstd{1.01}{0.36}$&
        $\meanstd[4]{1.2}{1.3}$\\
    GP-DIS&
        $\meanstd{0.87}{0.18}$&
        $\meanstd{1.04}{0.34}$&
        $\meanstd[16]{4.8}{14}$\\
    \midrule
    \textbf{PAI}&
        $\mathbf{\meanstd{0.037}{0.011}}$&
        $\mathbf{\meanstd{0.028}{0.011}}$&
        $\mathbf{\meanstd[-4]{1.6}{1.7}}$\\
    \textbf{PAI-DIS}&

        $\mathbf{\meanstd{0.034}{0.019}}$&
        $\mathbf{\meanstd{0.026}{0.008}}$&
        $\mathbf{\meanstd[-5]{3.9}{2.4}}$\\

    \bottomrule
    \end{tabular}
}
    \label{tab:multi-modal}
\end{table}

\paragraph{Large datasets.} To illustrate the computational benefits of using PAI for larger datasets, we repeated the same experiment in this section but with $10^5$ data points in each of the $K=10$ partitions. Remarkably, even for this moderate-sized dataset, we notice a $6\times$ speed-up -- decreasing the total running time from 6 hours to 57 minutes, (50 for subposterior sampling + 7 from PAI; see Appendix  \ref{appendix:large_dataset}).
Overall, PAI's running time is in the same order of magnitude as the previous SOTA \citep[e.g.][]{Wang2015, Nemeth2018}.
However, only PAI returns correct results while other methods fail.

\begin{figure}[t!]
\centering
\includegraphics%
[width=0.9\linewidth]%
{figures/experiments/warped_33.pdf}
\vspace{-1em}
    \caption{\textbf{Warped Student's t.} Top: Log marginal posterior for $\theta_1$. Bottom: Log posterior density. Thanks to active sampling, PAI better captures details in the depths of the tails. }
  \label{fig:warped}
\end{figure}

\subsection{Warped Student's t}
\label{subsec:heavy_tails}

\paragraph{Setting.} We now turn to a synthetic example with heavy tails. Consider the following hierarchical model:
\begin{equation*}
\begin{split}
    \theta \sim p(\theta) & =  \mathcal{N}(0, \sigma^2_p \mathbb{I}_2)\\
    y_1, \ldots, y_N \sim p(y_n | \theta) & = \mathrm{StudentT}\left(\nu, \theta_1 + \theta_2^2, \sigma_l^2\right)
\end{split}
\end{equation*}
where $\theta \in \mathbb{R}^2$, $\nu=5$ is the degrees of freedom of the Student's $t$-distribution, $\sigma_p=1$, and $\sigma_l=\sqrt{2}$. This model is a heavy-tailed variant of the Warped Gaussian model studied in earlier work, e.g., \citet{Nemeth2018, Mesquita2019}.
 As before, we generate $N=10^{3}$ samples and split the data into $K=10$ partitions for parallel inference.

\paragraph{Results.} Fig \ref{fig:warped} shows the full posterior and the marginal posterior for $\theta_1$ obtained using the two best-performing methods without DIS refinement, PAI and GP (see Table \ref{tab:warped-gaussian}). While PAI(-DIS) is very similiar to GP(-DIS) in terms of metrics, Fig \ref{fig:warped} shows that, unlike GP(-DIS), PAI accurately captures the far tails of the posterior which could be useful for downstream tasks, avoiding failure mode III (Fig \ref{fig:toy}C).

\begin{table}[h]
    \caption{\textbf{Warped Student's t.}}
    \centering
\resizebox{\columnwidth}{!}{
    \begin{tabular}{l c c c }
    \toprule
    \bf{Model}  &  \bf{MMTV} & \bf{W2} & \bf{GsKL\xspace}\\
    \toprule
    Parametric&
        $\meanstd{0.51}{0.01}$&
        $\meanstd{0.71}{0.07}$&
        $\meanstd[0]{1.9}{0.1}$\\
    Semi-param.&
        $\meanstd{0.50}{0.03}$&
        $\meanstd{0.57}{0.05}$&
        $\meanstd[1]{1.1}{0.2}$\\
    Non-param.&
        $\meanstd{0.51}{0.02}$&
        $\meanstd{0.59}{0.03}$&
        $\meanstd[1]{1.2}{0.2}$\\
    PART&
        $\meanstd{0.66}{0.07}$&
        $\meanstd{0.78}{0.09}$&
        $\meanstd[2]{1.2}{0.7}$\\
    GP&
        $\mathbf{\meanstd{0.015}{0.003}}$&
        $\mathbf{\meanstd{0.003}{0.002}}$&
        $\meanstd[-4]{4.5}{10.5}$\\
    GP-DIS&

        $\meanstd{0.018}{0.004}$&
        $\mathbf{\meanstd{0.002}{0.001}}$&
        $\meanstd[-5]{6.6}{5.8}$\\
    \midrule
    \textbf{PAI}&
        $\mathbf{\meanstd{0.015}{0.003}}$&
        $\mathbf{\meanstd{0.002}{0.001}}$&
        $\mathbf{\meanstd[-5]{1.2}{0.8}}$\\
    {PAI-DIS}&

        ${\meanstd{0.018}{0.003}}$&
        $\mathbf{\meanstd{0.002}{0.001}}$&
        $\meanstd[-5]{3.8}{3.4}$\\
    \bottomrule

    \end{tabular}
}
    \label{tab:warped-gaussian}
\end{table}

\subsection{Rare categorical events}
\label{subsec:rare_events}

\paragraph{Setting.} To evaluate how our method copes with heterogeneous subposteriors, we run parallel inference for a synthetic example with Categorical likelihood and $N=10^3$ discrete observations split among three classes. To enforce heterogeneity, we make the first two classes rare (true probability $\theta_1 = \theta_2 = 1/N$) and the remaining class much more likely (true probability $\theta_3 = (N-2)/N$). Since we split the data into $K=10$ equal-sized parts, some of them will not contain even a single rare event. We perform inference over $\theta \in \Delta^2$ (probability 2-simplex) with a symmetric Dirichlet prior with concentration parameter $\alpha = 1/3$.

\paragraph{Results.}  Fig \ref{fig:rarecat} shows the samples from the combined approximate posterior for each method. In this example, PAI-DIS matches the shape of the target posterior extremely well, followed closely by GP-DIS (see also Table \ref{tab:rare}). Notably, even standard PAI (without the DIS correction) produces a very good approximation of the posterior -- a further display of the ability of PAI of capturing fine details of each subposterior, particularly important here in the combination step due to the heterogeneous subposteriors. By contrast, the other methods end up placing excessive mass in very-low-density regions (PART, Parametric, GP) or over-concentrating (Non-parametric, Semi-parametric).

\begin{table}[h]
    \caption{\textbf{Rare categorical events.}

    }
    \centering
\resizebox{\columnwidth}{!}{
    \begin{tabular}{l c c c }
    \toprule
    \bf{Model}  &  \bf{MMTV} & \bf{W2} & \bf{GsKL\xspace}\\
    \toprule
    Parametric&
        $\meanstd{0.26}{0.14}$&
        $\meanstd{0.15}{0.19}$&
        $\meanstd[0]{1.1}{1.4}$\\
    Semi-param.&
        $\meanstd{0.49}{0.21}$&
        $\meanstd{0.27}{0.23}$&
        $\meanstd[0]{3.5}{3.4}$\\
    Non-param.&
        $\meanstd{0.43}{0.17}$&
        $\meanstd{0.19}{0.25}$&
        $\meanstd[0]{2.8}{3.9}$\\
    PART&
        $\meanstd{0.31}{0.14}$&
        $\meanstd{0.08}{0.13}$&
        $\meanstd[-1]{8.6}{10}$\\
    GP&
        $\meanstd{0.16}{0.09}$&
        $\meanstd{0.04}{0.07}$&
        $\meanstd[-1]{3.5}{4.8}$\\
    GP-DIS&

        $\meanstd{0.011}{0.002}$&
        $\meanstd[-4]{6.3}{0.9}$&
        $\meanstd[-4]{1.1}{1.5}$\\
    \midrule
    PAI&
        ${\meanstd{0.028}{0.027}}$&
        ${\meanstd{0.001}{0.002}}$&
        ${\meanstd[-3]{8.0}{16}}$\\
    \textbf{PAI-DIS}&

        $\mathbf{\meanstd{0.009}{0.002}}$&
        $\mathbf{\meanstd[-4]{5.4}{0.8}}$&
        $\mathbf{\meanstd[-5]{4.3}{2.1}}$\\
    \bottomrule

    \end{tabular}
}
    \label{tab:rare}
\end{table}

\begin{figure*}[t!]
\centering
\includegraphics%
[height=7cm]%
{figures/experiments/rarecat_33_wider.pdf}
\vspace{-1em}
  \caption{\textbf{Rare categorical events.} Each ternary plot shows  samples from the combined approximate posterior (red) on top of the true posterior (blue). Note that the panels are zoomed in on the relevant corner of the probability simplex. Of all methods, PAI is the one that best captures the shape of the posterior. }
  \label{fig:rarecat}
\end{figure*}

\subsection{Multisensory causal inference}
\label{subsec:neuro}

\paragraph{Setting.}
\emph{Causal inference} (CI) in multisensory perception denotes the process whereby the brain decides whether distinct sensory cues come from the same source, a commonly studied problem in computational and cognitive neuroscience \citep{kording2007causal}.
Here we compute the posterior for a 6-parameter CI model given the data of subject S1 from \citep{acerbi2018bayesian} (see Appendix \ref{supp:models} for model details).
The fitted model is a proxy for a large class of similar models that would strongly benefit from parallelization due to likelihoods that do not admit analytical solutions, thus requiring costly numerical integration.
For this experiment, we run parallel inference over $K = 5$ partitions of the $N = 1069$ observations in the dataset.

\paragraph{Results.} Table \ref{tab:neuroscience} shows the outcome metrics of parallel inference. Similarly to the rare-events example, we find that PAI-DIS obtains an excellent approximation of the true posterior, with GP-DIS performing about equally well (slightly worse on the GsKL\xspace metric). Despite lacking the DIS refinement step, standard PAI performs competitively, achieving a reasonably good approximation of the true posterior (see Appendix \ref{supp:models}). All the other methods perform considerably worse; in particular the GP method without the DIS step is among the worst-performing methods on this example.

\begin{table}[h]
    \caption{\textbf{Multisensory causal inference.}}
    \centering
\resizebox{\columnwidth}{!}{
    \begin{tabular}{l c c c }
    \toprule
    \bf{Model}  &  \bf{MMTV} & \bf{W2} & \bf{GsKL\xspace}\\
    \toprule
    Parametric&
        $\meanstd{0.40}{0.05}$&
        $\meanstd{4.8}{0.6}$&
        $\meanstd[1]{1.2}{0.4}$\\
    Semi-param.&
        $\meanstd{0.68}{0.07}$&
        $\meanstd{9.7}{9.6}$&
        $\meanstd[1]{5.6}{3.2}$\\
    Non-param.&
        $\meanstd{0.26}{0.02}$&
        $\meanstd{0.52}{0.14}$&
        $\meanstd[0]{5.3}{0.3}$\\
    PART&
        $\meanstd{0.24}{0.04}$&
        $\meanstd{1.5}{0.5}$&
        $\meanstd[0]{8.0}{5.4}$\\
    GP&
        $\meanstd{0.49}{0.25}$&
        $\meanstd{17}{23}$&
        $\meanstd[1]{6.3}{8.9}$\\
    GP-DIS&
        $\mathbf{\meanstd{0.07}{0.03}}$&
        $\mathbf{\meanstd{0.16}{0.07}}$&
        $\meanstd[-1]{8.7}{14}$\\
    \midrule
    {PAI}&
        ${\meanstd{0.16}{0.04}}$&
        ${\meanstd{0.56}{0.21}}$&
        ${\meanstd[0]{2.0}{1.7}}$\\
    \textbf{PAI-DIS}&
        $\mathbf{\meanstd{0.05}{0.04}}$&
        $\mathbf{\meanstd{0.14}{0.13}}$&
        $\mathbf{\meanstd[-1]{2.9}{3.6}}$\\

    \bottomrule
    \end{tabular}
} \vspace{-0.5em}
    \label{tab:neuroscience}
\end{table}

\section{RELATED WORKS}
\label{sec:related_works}

While the main staple of embarrassingly parallel MCMC is being a divide-and-conquer algorithm, there are other methods that scale up MCMC using more intensive communication protocols. For instance, \citet{Ahn2014} propose a distributed version of stochatic gradient Langevin dynamics \citep[SGLD,][]{Welling+Teh:2011} that constantly passes around the chain state to computing nodes, making updates only based on local data. However, distributed SGLD tends to diverge from the posterior when the communications are limited, an issue highlighted by recent work \citep{ElMekkaoui2021, Vono2022}. Outside the realm of MCMC, there are also works proposing expectation propagation as a framework for inference on partitioned data \citep{vehtari2020expectation, bui2018partitioned}.

Our method, PAI, builds on top of related work on GP-based surrogate modeling and active learning for log-likelihoods and log-densities. Prior work used GP models and active sampling to learn the intractable marginal likelihood \citep{osborne2012active,gunter2014sampling} or the posterior \citep{kandasamy2015bayesian,wang2017adaptive,jarvenpaa2021parallel}. Recently, the framework of Variational Bayesian Monte Carlo (VBMC) was introduced to simultaneously compute both the posterior and the marginal likelihood \citep{acerbi2018variational,acerbi2019exploration,acerbi2020variational}. PAI extends the above works by dealing with  partitioned data in the embarrassingly parallel setting, similarly to \citet{Nemeth2018}, but with the key addition of active learning and other algorithmic improvements.

\section{DISCUSSION}
\label{sec:discussion}

In this paper, we first exposed several potential major failure modes of existing embarrassingly parallel MCMC methods. We then proposed a solution with our new method, \emph{parallel active inference} (PAI), which incorporates two key strategies: sample sharing and active learning. On a series of challenging benchmarks, we demonstrated that `vanilla' PAI is competitive with current state-of-the-art parallel MCMC methods and deals successfully with scenarios (e.g., multi-modal posteriors) in which all other methods catastrophically fail. When paired with an optional refinement step (PAI-DIS), the proposed method consistently performs on par with or better than state-of-the-art. Our results show the promise of the proposed strategies to deal with the challenges arising in parallel MCMC. Still, the solution is no silver bullet and several aspects remain open for future research.

\subsection{Limitations and future work}

The major limitation of our method, a common problem to surrogate-based approaches, is scalability to higher dimensions. Most GP-based approaches for Bayesian posterior inference are limited to up to $\sim 10$ dimensions, see e.g. \citealp{acerbi2018variational,acerbi2020variational,jarvenpaa2021parallel}. Future work could investigate methods to scale GP surrogate modeling to higher dimensions, for example taking inspiration from high-dimensional approaches in Bayesian optimization (e.g., \citealp{kandasamy2015high}).

More generally, the validity of any surrogate modeling approach hinges on the ability of the surrogate model to faithfully represent the subposteriors. Active learning helps, but model mismatch in our method is still a potential issue that hints at future work combining PAI with more flexible surrogates such as GPs with more flexible kernels \citep{wilson2013gaussian} or deep neural networks \citep{Mesquita2019}. For the latter, obtaining the uncertainty estimates necessary for active learning would involve Bayesian deep learning techniques (e.g., \citealp{maddox2019simple}).

As discussed before, our approach is not `embarrassingly' parallel in that it requires a mandatory global communication step in the sample sharing part (see Section \ref{sec:samplesharing}). The presence of additional communication steps seem inevitable to avoid catastrophic failures in parallel MCMC, and has been used before in the literature (e.g., the DIS step of \citealp{Nemeth2018}). Our method affords an optional final refinement step (PAI-DIS) which also requires a further global communication step. At the moment, there is no automated diagnostic to determine whether the optional DIS step is needed. Our results show that PAI already performs well without DIS in many cases. Still, future work could include an analysis of the GP surrogate uncertainty to recommend the DIS step when useful.

\section*{Acknowledgments}

This work was supported by the Academy of Finland (Flagship programme: Finnish Center for Artificial Intelligence FCAI and grants 328400, 325572) and UKRI (Turing AI World-Leading Researcher Fellowship, EP/W002973/1). We also acknowledge the computational resources provided by the Aalto Science-IT Project from Computer Science IT.

{
\small
\bibliographystyle{plainnat}


\section{Introduction}
The image description generation (IDG) problem concerns about generating a natural language description that transcribes an input image. Over the years, tremendous effort has been dedicated to developing models that are descriptive. However, little effort is dedicated to generating descriptions that are \emph{stylish} (e.g.\ romantic, lyric, etc). Even for the handful of stylish IDG models that exist, they only have a loose control over the style. Ideally, a stylish IDG model should allow users to flexibly control over the generated descriptions as shown in Fig \ref{fig:Introduction}. Such a model would be useful for increasing user engagement in applications requiring human interaction such as chatbot and social media sharing.

A naive approach to tackle the stylish IDG problem is to collect new corpora of paired images and descriptions for training. However, this is expensive. For each style that we wish to generate, we have to ask human annotators to write the romantic descriptions for each image in the training dataset.

In this paper, we propose a controllable stylish IDG model. Our model is jointly trained with a paired unstylish image description corpus (source domain) and a monolingual corpus of the specific style (target domain). In this setting, our model can learn to generate various styles without collecting new paired data in the target domain. Our main contribution is to show that the layer normalization can be used to disentangle language styles from the content of source and target domains via a small tweak. This design enables us to use the shared content to generate descriptions that are more relevant to the image as well as control the style by plugging in a set of style-specific parameters. We refer this mechanism as Domain Layer Normalization (DLN) since we treat each style as the target domain in the domain transfer setting

We conduct an extensive experimental evaluation to validate the proposed approach using both subjective and objective performance metrics. We evaluate our model on four different styles, including fairy tale, romance, humor, and country song lyrics style (lyrics). Experiment results show that our model generates stylish descriptions that are more preferred by human subjects. It also outperforms prior works on the objective performance metrics.
\begin{figure}[t!]
    \centering
    \includegraphics[scale=0.26]{a3i_fig1.pdf}
    \caption{An ideal IDG can generate stylish descriptions for the given image. The generated descriptions should relate to the image content with different language styles.}
    \label{fig:Introduction}
\end{figure}
\section{Related Works}
\noindent \textbf{Visual style transfer.} Image style transfer has been widely studied in computer vision. Gatys et al.~\cite{gatys2015neural} synthesize a new stylish image by recombining image content with style features extracted from different images. Dumoulin et al.~\cite{45832} propose to learn the style embedding of visual artistic style by conditioning on the parameter of batch normalization~\cite{43442}. Huang et al.~\cite{huang2017arbitrary} use adaptive instance norm. More recent approaches use the generative adversarial network (GAN)~\cite{NIPS2014_5423} to align and transfer images from different domains. Liu et al.~\cite{NIPS2016_6544} employ weight-sharing assumption to learn the shared latent code between two domains and further propose translation stream in~\cite{NIPS2017_6672} to encourage the same image in two domains to be mapped into common latent code. While our method is similar to these works in high level, the discrete property of language required new model design.

\noindent \textbf{Language style transfer.} Supervised learning can be used to generate various linguistic attribute (e.g., different sentiments and different degrees of descriptiveness), but it requires a significant amount of labeled data. Many recent works assume there exist a share content space and a latent style vector between two non-parallel corpora for unsupervised language style transfer. Shen et al.~\cite{shen2017style} propose an encoder-decoder structure with adversarial training to learning this space. Following the same line, Melnyk et al.~\cite{melnyk2017improved} introduce content preservation loss and classification loss to improve the transfer performance. Fu et al.~\cite{fu2018style} propose to use a multi-decoder for different styles and a discriminator to learn a shared content code. Zhang et al.~\cite{zhang2018shaped} also use similar structure by using shared and private encoder-decoder. In a recent work, Prabhumoye et al.~\cite{prab2018style} introduce to ground the sentence in translation model, then apply adversarial training to get the desired style. What differs us from prior works is that we require generated stylish descriptions to match the visual content. Moreover, the style transferred in our work is more abstract instead of explicit styles such as sentiment, gender, or authorship in previous works.

\noindent\textbf{Image description generation.} Several works have been proposed to generate image descriptions by using paired image description data~\cite{vinyals2015show,krause2016paragraphs,liang2017recurrent}. To increase the naturalness and diversity of generated descriptions, Dai et al.~\cite{dai2017towards} apply adversarial training approach to train an evaluator to score the quality of generated descriptions. Chen et al.~\cite{chen2017show} propose an adversarial training procedure to adapt image captioning style using unpaired images and captions. A new objective is proposed in~\cite{dai2017contrastive} to enhance the distinctiveness of generated captions. On the other hand, there exist a few works proposed to enhance the attractiveness and style of the generated descriptions. Zhu et al.~\cite{zhu2015aligning} align the book and the corresponding movie release to a story-like description of the visual content. However, this method does not preserve the visual content. Matthews et al.~\cite{mathews2016senticap} propose the switch RNN to generate caption with positive and negative sentiments, which requires word level supervision and might not be able to scale. Recently, Gan et al.~\cite{gan2017semantic} investigate to generate tag-dependent caption by extending the weight matrix of LSTM to consider tag information. The following work StyleNet~\cite{gan2017stylenet} explores to decomposes LSTM matrix to incorporate the style information. One key difference is that we leverage an arbitrary stylish monolingual corpus that is not paired with any image dataset as target corpus instead of using paired images with stylish ground truth. The most similar to our work is~\cite{mathews2018semstyle}, the major differences are that we do not exploit the language features such as POS tag of corpus and we do not pre-process the target corpus to make it similar to the source one. Our approach is end to end with minimal pre-process of target corpus.

\section{Unsupervised Stylish Image Description Generation}
The goal of stylish Image Description Generation (IDG) is to generate a natural language description $d_T$ in space $\mathcal{D}_T$ given an image $I$ in the image space $\mathcal{I}$. The style of the description is implicitly captured in the description space $\mathcal{D}_T$, where we use subscript $T$ to emphasize the target style. There exist two settings for learning a stylish IDG model.
\paragraph{\bf Supervised stylish IDG.}  In supervised stylish IDG, we are given a training dataset $\mathbb{D}=\{(I^{(n)},d_T^{(n)}), n=1,...,N\}$, where each sample $(I^{(n)},d_T^{(n)})$ is a pair of image and its target stylish description sampled from the joint distribution $p(\mathcal{I},\mathcal{D}_T)$. The goal is to learn the conditional distribution $p(\mathcal{D}_T|\mathcal{I})$ using $\mathbb{D}$ so that we can generate \mbox{stylish image descriptions for an input image.}
\paragraph{\bf Unsupervised stylish IDG.}  In unsupervised stylish IDG, we are given two training datasets $\mathbb{D}_S$ and $\mathbb{D}_T$. $\mathbb{D}_S=\{(I^{(n)},d_S^{(n)}), n=1,...,N_S\}$ consists of pairs of image and its description $(I^{(n)},d_S^{(n)})$ sampled from $p(\mathcal{I},\mathcal{D}_S)$, where $S$ is referred to as the source domain which is typically unstylish. $\mathbb{D}_T=\{(d_T^{(n)}), n=1,...,N_T\}$ is a dataset of target stylish descriptions $d_T^{(n)}$ sampled from $p(\mathcal{D}_T)$, where the corresponding images are not available. Hence, the learning task is considered as unsupervised. The goal of unsupervised stylish IDG is to learn the conditional distribution $p(\mathcal{D}_T|\mathcal{I})$ using $\mathbb{D}_S$ and $\mathbb{D}_T$.

Unsupervised stylish IDG is an ill-posed problem since it is about learning the conditional distribution $p(\mathcal{D}_T|\mathcal{I})$ without using samples from the joint distribution $p(\mathcal{I},\mathcal{D}_T)$. Therefore, learning an unsupervised stylish IDG function is difficult without leveraging some useful assumptions. However, under the unsupervised setting, training data collection is greatly simplified: one could pair a general image description dataset (e.g., the MS-COCO dataset~\cite{lin2014microsoft}) with an existing corpus of the target style (e.g., some romantic novels) for learning. A solution to the unsupervised problem could enable many stylish image description generation applications.

\subsection{Unsupervised Stylish IDG via Domain Layer Norm}
\label{sec:framework}
\begin{figure*}[t]
    \centering
    \includegraphics[width=0.75\textwidth]{assumption_v4.pdf}
    \caption{We make several assumptions to deal with the challenging unsupervised stylish image description generation problem. We first assume there exists a shared latent space $\mathcal{Z}$ so that a latent code $\mathbf{z}\in\mathcal{Z}$ can be mapped to the source description space $\mathcal{D}_S$ and the target stylish description space $\mathcal{D}_T$ via $G_S$ and $G_T$. We also assume there exists a stylish image description embedding function $E_T$ that can map a stylish description to a latent code. Finally, we assume there exists an image embedding function $E_I$ that can map an image to a latent code.
    Once these functions are learned from data, we can generate a stylish image description for an image by applying $E_I$ and $G_T$ sequentially.
    }
    \label{fig::asssumption}
\end{figure*}
\paragraph{\bf Assumptions.}
To deal with the ill-posed unsupervised stylish IDG problem, we make several assumptions illustrated in Figure~\ref{fig::asssumption}. We first assume that there exists a latent space $\mathcal{Z}$ providing a common ground to effectively map to and from the image space $\mathcal{I}$, the source description space $\mathcal{D}_S$, and the target stylish description space $\mathcal{D}_T$.
From latent space to description space, we assume that there exists a source description generation function $G_S(\mathbf{z})\in\mathcal{D}_S$ and a target stylish description generation function $G_T(\mathbf{z})\in\mathcal{D}_T$.
From non-latent space to latent pace, we assume that there exist an image encoder $E_I(I)\in\mathcal{Z}$ and a target description encoder $E_T(d_T)\in\mathcal{Z}$.
Our goal is to learn the generation functions ($G_T$ and $G_S$) and the encoding functions ($E_I$ and $E_T$) from the unsupervised stylish IDG training data $\mathbb{D}_S$ and $\mathbb{D}_T$. Note that this is a challenging learning task if $G_T$ and $G_S$ is completely independent of each other. Hence, we assume that $G_T$ and $G_S$ share the ability to describe the same factual content but with different styles.
Once these functions are learned,  we can simply first encode the image $I$ to a latent code using $E_I$ and then using $G_T$ to generate a stylish image description. In other words, the stylish image description is given by $G_T(E_I(I))$. We model the conditional distribution as $p(\mathcal{D}_T|\mathcal{I})=\delta(G_T(E_I(I)))$, where $\delta$ is the delta function.
\begin{figure*}[t]
    \centering
    \includegraphics[width=0.78\textwidth]{model_archicompo_0518.pdf}
    \caption{The $E_{I}$ and $E_{T}$ map the image and the target stylish description to a shared latent space. Both $G_{S}$ and $G_{T}$ share all weights except the layer norm parameters to capture the similar content in two domains. To disentangled the style factor, we employ different sets of layer norm parameters denoted as $\{\boldsymbol{g}_{S},\boldsymbol{b}_{S}\}$ and $\{\boldsymbol{g}_{T},\boldsymbol{b}_{T}\}$ for source and target domain during training.}
    \label{fig::model_archi}
\end{figure*}
Inspired by the success of deep learning, we model both of the generation and encoding functions using deep networks.
Specifically, we model $E_I$ using a deep convolutional neural network (CNN)~\cite{NIPS2012_4824} and model $E_T$, $G_T$, and $G_S$ using recurrent neural network as illustrated in Figure~\ref{fig::model_archi}. We also use Skip-Thought Vectors (STV)~\cite{kiros2015skip} to model $E_T$. For $G_T$ and $G_S$, we use Layer Normalized Long Short Term Memory unit (LN-LSTM) as their recurrent module~\cite{ba2016layer,hochreiter1997long}.
\begin{figure}[t]
    \centering
    \includegraphics[scale=0.18]{a3i_model_cell_tmp.pdf}
    \caption{Inside the LN-LSTM cell (left) and the operation of layer normalization (right).}
    \label{fig:DLNdetail}
\end{figure}
\paragraph{\bf Training sketch.}
With the source domain dataset $\mathbb{D}_S$, we can train $\boldsymbol{z}_S=E_I(I)$ and $d_S=G_S(\boldsymbol{z}_S)$ jointly by solving the supervised IDG learning task, where $\boldsymbol{z}_S$ is the learned latent representation in the source domain. On the other hand, with the target domain dataset $\mathbb{D}_T$, we can train $\boldsymbol{z}_T=E_T(d_T)$ and $d_T=G_T(\boldsymbol{z}_T)$ jointly by solving an unsupervised description reconstruction learning task, where $\boldsymbol{z}_T$ is the learned latent representation in the target domain. To ensure that the latent space is shared (i.e., $\boldsymbol{z}_T\in \mathcal{Z}$ and $\boldsymbol{z}_S\in \mathcal{Z}$), we further assume that the generation functions $G_S$ and $G_T$ share most of their parameters.

\paragraph{\bf Domain Layer Norm.}
Specifically, we assume $G_S$ and $G_T$ share all the parameters except those in their layer norm parameters~\cite{ba2016layer}. In other words, the domain description generators ($G_S$ and $G_T$) only defer in the layer norm parameters. We refer this weight-sharing scheme as the Domain Layer Norm (DLN) scheme. The intuition behind DLN is to encourage the shared weight to capture the factual content between two domains while the differences (i.e., styles) are captured in layer norm parameters. This design helps $G_T$ generate descriptions that are related to the image content even without the supervision of the corresponding images in training.

\noindent\textbf{Training $E_I$ and $G_S$ via Supervised IDG.}
The goal of supervised image description generation is to learn $p(\mathcal{D}_S|\mathcal{I})$ by using $\mathbb{D}_S$. The $G_S$ consists of an embedding matrix $\boldsymbol{\theta_W}$ that maps input text $x_k$ to a vector $\boldsymbol{e}_k$, an LN-LSTM module, and an output matrix $\boldsymbol{\theta_V}$ that maps hidden state to predicted token $\boldsymbol{\hat{y}}$. Formally,
\begin{align}\label{eq:1
(\boldsymbol{\hat{y}}_{k+1}, \boldsymbol{h}_{k+1}) = G_S(\boldsymbol{e}_k,\boldsymbol{h}_{k})~,\\ \boldsymbol{\hat{y}}_{k+1} = \boldsymbol{\theta_V}^T\boldsymbol{h}_{k}~,\\
\boldsymbol{e}_k = \boldsymbol{\theta_W}^T\boldsymbol{1}\{x_k\}~,\\
\boldsymbol{e}_{-1}= E_I(I), \boldsymbol{h}_{-1} = \boldsymbol{0}~,
\end{align}
where $\boldsymbol{h}_{k}$ is the hidden feature in the LN-LSTM, $k\in\{-1\ldots m-1\}$ is time step of description with length $m$, and $\boldsymbol{1}\{\}$ denotes the operator for one-hot encoding. To train the network, we minimize the sum of cross-entropy of correct words as follows,
\begin{align}
    \mathcal{L}_{S} &= -\sum_{k=1}^{m}\textrm{log}(\boldsymbol{1}\{x_k\}^T\boldsymbol{\hat{y}}_k)~,
\end{align}
where $x_k$ is the $k^{th}$ word in the ground truth sentence.

\noindent\textbf{Training $E_T$ and $G_T$ via Stylish Image Description Reconstruction.}
The $G_T$ contains the LN-LSTM module, the same output matrix and embedding matrix used in $G_S$. Formally,
\begin{align}\label{eq:3}
(\boldsymbol{\hat{y}}_{k+1}, \boldsymbol{h}_{k+1}) = G_T(\boldsymbol{e}_k,\boldsymbol{h}_{k})~,\\ \boldsymbol{\hat{y}}_{k+1} = \boldsymbol{\theta_V}^T\boldsymbol{h}_{k}~,\\
\boldsymbol{e}_k = \boldsymbol{\theta_W}^T\boldsymbol{1}\{d^k_T\}~,\\
\boldsymbol{e}_{-1} = E_T(d_T)~,\\
\boldsymbol{h}_{-1} = \boldsymbol{0}~,
\end{align}

where $d_T$ is the target style image description. To train the network, we minimize the reconstruction error as follows,
\begin{align}
    \mathcal{L}_{T} &= -\sum_{k=1}^{m}\textrm{log}(\boldsymbol{1}\{d^k_T\}^T\boldsymbol{\hat{y}}_k)~,
\end{align}
where $d^k_T$ is the $k^{th}$ word in the target style image description.

\noindent\textbf{Relating $G_S$ and $G_T$ via Domain Layer Norm.}
We relate $G_{S}$ and $G_{T}$ by sharing all weights except layer norm parameters in the LN-LSTM. Details inside the LN-LSTM are shown in Fig \ref{fig:DLNdetail}, where the layer norm operation (LN) is applied to each gate of LSTM. Take the input gate as an example:
\begin{align}\label{eq:4
\boldsymbol{\hat{i}}_k &= \textrm{LN}(\boldsymbol{i}_k), \boldsymbol{i}_k = \boldsymbol{\theta}_{ie}\boldsymbol{e}_k+ \boldsymbol{\theta}_{ih}\boldsymbol{h}_{k-1}~,
\end{align}
where $\boldsymbol{\hat{i}}_k$ and $\boldsymbol{i}_k$ are the normalized and unnormalized input gates, $\boldsymbol{\theta}_{ie}$, $\boldsymbol{\theta}_{ih}$ are two projection matrices that map the embedding vector and the previous hidden state into the same dimension. The LN operation converts any input $\boldsymbol{a}$ to a normalized output $\boldsymbol{\hat{a}}$ as follows,
\begin{align}
\label{eq:5
    \boldsymbol{\hat{a}} &= \frac{\boldsymbol{g}}{\sigma} \odot (\boldsymbol{a} - \mu)) + \boldsymbol{b}~,\\
    \mu &= \frac{1}{p_{h}}\sum_{i=1}^{i=p_{h}}a_{i}~,\\
    \sigma &= \sqrt{\frac{1}{p_{h}}\sum_{i=1}^{p_{h}}(a_{i} - \mu)^2}~,
\end{align}
where $a_i$ denotes the $i^{th}$ entry in the vector $\boldsymbol{a}$, $p_h$ is the dimention of the input $\boldsymbol{a}$, $\mu$ and $\sigma$ are the mean and standard deviation of the input $\boldsymbol{a}$, $\boldsymbol{g}$ and $\boldsymbol{b}$ are scaling and shifting vectors (i.e., layer norm parameters) learned from the data.

We train the whole network by jointly minimizing the supervised IDG loss $\mathcal{L}_{S}$ and the unsupervised image description reconstruction loss $\mathcal{L}_{T}$ subject to the architectural constraint set to $G_S$ and $G_T$ as below, where $\lambda$ is a hyperparameter.
\begin{multline}\label{eq:6}
    \mathcal{L}(\boldsymbol{\theta}_{E_I} \boldsymbol{\theta}_{G_S},\boldsymbol{\theta}_{E_T},\boldsymbol{\theta}_{G_T}) = \lambda \mathcal{L}_{S}(\boldsymbol{\theta}_{E_I},\boldsymbol{\theta}_{G_S})\\
    + (1 - \lambda)\mathcal{L}_{T}(\boldsymbol{\theta}_{E_T},\boldsymbol{\theta}_{G_T})~.
\end{multline}
\paragraph{\bf Extension to New Target Styles.}
Given a model with parameters $\boldsymbol{\theta_{V}}$, $\boldsymbol{\theta_{W}}$, $\boldsymbol{\theta}_{E_I}$, and $\boldsymbol{\theta}_{G_S}$, pre-trained on a pair of the source and one target domain, we aim to adapt it to a new target domain (i.e, style) by enlarging $\boldsymbol{\theta_{V}}$ and $\boldsymbol{\theta_{W}}$ to $\boldsymbol{\theta}^{\prime}_{\boldsymbol{V}}$ and $\boldsymbol{\theta}^{\prime}_{\boldsymbol{W}}$ to accommodate new vocabulary and finetuning the remaining parameters to $\boldsymbol{\theta}^{\prime}_{E_I}$, $\boldsymbol{\theta}^{\prime}_{E_T}$, $\boldsymbol{\theta}^{\prime}_{G_S}$ and $\boldsymbol{\theta}^{\prime}_{G_T}$. Hence, we define a new loss function as:
\begin{multline}\label{eq:7}
    \mathcal{L}(\boldsymbol{\theta}^{\prime}_{E_{I}}, \boldsymbol{\theta}^{\prime}_{G_S},\boldsymbol{\theta}^{\prime}_{E_T},\boldsymbol{\theta}^{\prime}_{G_T}) = \lambda_1\mathcal{L}_{S}(\boldsymbol{\theta}^{\prime}_{E_I},\boldsymbol{\theta}^{\prime}_{G_S})\\ + (1 -\lambda_1)\mathcal{L}_{T}(\boldsymbol{\theta}^{\prime}_{E_T},\boldsymbol{\theta}^{\prime}_{G_T}) + \lambda_2R(\boldsymbol{\theta}^{\prime}_{E_{I}},\boldsymbol{\theta}^{\prime}_{\boldsymbol{W}}, \boldsymbol{\theta}^{\prime}_{\boldsymbol{V}})~,
\end{multline}
where $\lambda_1$ and $\lambda_2$ are hyperparameters.
The regularization term $R(\boldsymbol{\theta}^{\prime}_{E_{I}},\boldsymbol{\theta}^{\prime}_{\boldsymbol{W}}, \boldsymbol{\theta}^{\prime}_{\boldsymbol{V}})=\lVert \boldsymbol{\theta}^{\prime}_{E_{I}} - \boldsymbol{\theta}_{E_{I}}\rVert_{2} + \lVert \boldsymbol{\theta}^{\prime}_{\boldsymbol{W}} - \boldsymbol{\theta_{W}} \rVert_{2} + \lVert \boldsymbol{\theta}^{\prime}_{\boldsymbol{V}} - \boldsymbol{\theta_{V}} \rVert_{2}$ is used to prevent new weights from deviating the pretrained model. This encourages the adapted model to keep the information learned during the pretrained phase.
We use pretrained $\boldsymbol{\theta}_{E_I}$ and $\boldsymbol{\theta}_{G_S}$ as initialization of $\boldsymbol{\theta}^{\prime}_{E_I}$ and $\boldsymbol{\theta}^{\prime}_{G_S}$. For $\boldsymbol{\theta}^{\prime}_{G_T}$, we share all parameters in $\boldsymbol{\theta}^{\prime}_{G_S}$ except the layer norm parameters. $\boldsymbol{\theta}^{\prime}_{E_T}$ is trained from scratch. Note that we do not update the source domain layer norm parameters since we do not need to learn source style.
\section{Experiment}
We conduct two experiments to evaluate our proposed method. First, we demonstrate that our method can generate stylish descriptions based on paired image and unstylish description in the source domain and a stylish monolingual corpus that is not paired with any image dataset in the target domain. Then, we demonstrate the flexibility of our DLN to progressively include new styles one by one in the second experiment. The implementation details are in the supplementary.
\subsection{Evaluation Setting}
\noindent\textbf{Datasets.} We use paragraphs released in~\cite{krause2016paragraphs} (VG-Para) as our source domain dataset. We do not use caption dataset such as MS-COCO because we found captions are less stylish when transfer to target style domain. We use pre-split data which contain 14575, 2489 and 2487 for training, validation and testing.  For target dataset, we use humor and romance novel collections in BookCorpus~\cite{zhu2015aligning}. We also collect country song lyrics and fairy tale to show that our method is effective on corpora with different syntactic structures and word usage. More details can be found in supplementary materials.\\
\noindent\textbf{Baselines.} We compare our method with four baselines: StyleNet~\cite{gan2017stylenet}, Neural Story Teller (NST)~\cite{kiros2015skip}, DLN-RNN and Random. Stylenet generates stylish descriptions in an end-to-end way but with paired image and stylish ground truth description. NST breaks down the task into two steps, which first generate unstylish captions then apply style shift techniques to generate stylish descriptions. DLN-RNN uses the same framework as DLN with only difference in using simple recurrent neural network. Random samples the the same number of nouns as that in the unstylished ground truth from the corresponding vocabulary of target domain. Although a concurrent work ~\cite{mathews2018semstyle} that attempts to solve similar task as ours, the major differences are we do not exploit linguistic features and pre-process the target corpus to facilitate the training. Moreover, it is not sure whether the concurrent work can be applied to other styles or even multiple styles as it only makes a step toward generating sentences with romantic style.\\
\noindent\textbf{Metrics of semantic relevance.} As there is no ground truth sentences for stylish image descriptions in unpaired setting, the conventional n-gram based metrics such as BLEU~\cite{papineni2002bleu}, METEOR~\cite{denkowski2014meteor} and CIDEr~\cite{vedantam2015cider} cannot be applied. It is also not suitable to calculate these metrics between stylish sentences and the unstylished ground truth because the goal of stylish description generation is to change the word usage while preserve certain semantic relevance between the stylish description and images.

We propose content similarity to evaluate the semantic relevance between generated stylish sentences and the unstylished ground truth. To calculate content similarity, we define $C_{S}$ as the set of nouns in the ground truth (source domain), and $C^{\prime}_{S}$ as the union between $C_{S}$ and synonyms for each noun in $C_{S}$, for the model may describe the same object with different words (e.g., cup and mug). Similar logic is applied to $C_{T}$ and $C^{\prime}_{T}$ in the generated description (target domain). We calculate:
\begin{align}
\label{eq:8}
p &= \frac{|C_T \cap C^{\prime}_{S}|}{|C_T|} & r = \frac{|C_S \cap C^{\prime}_T|}{|C_S|},~
\end{align}
We take the f-score of the $p$ and $r$ as the content similarity score. The overall content similarity score is averaged over the testing data. This is because we assume stylish descriptions should at least contain objects which appear in the image. We also report SPICE~\cite{anderson2016spice} score, which calculate the f-score of semantic tuples between untylished ground truth and the generated stylish descriptions. The final score is average over all testing data.\\
\noindent\textbf{Metrics of stylishness.} We use transfer accuracy to evaluate the stylishness of our generated description. The transfer accuracy is widely used in language style transfer task~\cite{shen2017style,melnyk2017improved,fu2018style}. It measures how often do descriptions have labels of target style on test dataset based on a pre-trained style classifier. We follow the definition of transfer accuracy in~\cite{fu2018style}, which is
\begin{equation}
\mathcal{T} =
\begin{cases}
1 & \text{if $s > 0.5$} \\
0 & \text{if $s \leq 0.5$}
\end{cases}
\end{equation}
where $s$ is the output probability score of the classifier. We define $R_{T} = \frac{N_{vt}}{N_{vs}}$ as our transfer accuracy, which is the fraction of number of testing $N_{vs}$ data in source domain and number of testing data that correctly transfer description with target style $N_{vt}$. The final score is average over all testing data.\\
\noindent\textbf{Human evaluation.} The difficulty in generating stylish sentence in unpaired setting is to remain semantic relevance. Therefore, we conduct a human study on Amazon Mechanical Turk (AMT) independently for each methods to judge the semantic relevance between image and description. For each model, we randomly sample 100 images then generate stylish descriptions for each style. Two workers are asked to vote the semantic relevance with following prompt: Given an image and a paragraph from the book (Our stylish corpus), how well does the paragraph content relate to objects in the image. Workers are forced to vote from unrelated to related. The criteria for eligible workers are having at least 100 successful HITs with 70\% acceptance rate. The total number of HIT is 2400. For each HIT, the order of options is randomized. Workers are forced to vote and all responses are counted without aggregation.
\begin{table*}[t!]
  \centering
  \begin{tabular}{lc|ccccccccc}
    \toprule
    Model  & Data & CS & S & T & $p$ & $r$ & $n_{p}$ & $n_{r}$\\
    \midrule
     NST~\cite{kiros2015skip} & Lyrics& 0.037 & 0.016  & 100\% & 0.041 & 0.044 & 0.68 & 0.75 \\
     StyleNet~\cite{gan2017stylenet}&Lyrics& 0.033  & 0.014 & 100\%    &0.038 & 0.038 &0.57 &0.67 \\
     Random              & Lyrics & 0.008 & 0.002 & 55.2\% & 0.007 & 0.012 &0.13 &0.09  \\
     DLN-RNN                  & Lyrics &0.072  & 0.030  &100\%  & \textbf{0.101} & 0.069 &\textbf{1.65} &1.17  \\
     DLN  &Lyrics &\textbf{0.083}& \textbf{0.033} &99.2\% & 0.080& \textbf{0.115} &1.25 & \textbf{1.92} \\
    \midrule
     NST~\cite{kiros2015skip} & Romance& 0.088 & 0.039 & 100\% & 0.087 & 0.113 &\textbf{1.57} &1.90\\
     StyleNet~\cite{gan2017stylenet}   &Romance& 0.012 & 0.005 & 100\% & 0.032 &0.001 & 0.11&0.14  \\
     Random              & Romance& 0.005& 0.002&  100\% &0.004 &0.001 &0.07 &0.05 \\
     DLN-RNN                  & Romance& 0.083& 0.034  &   94.3\%     & 0.078 & 0.125      &1.27 &0.71 \\
     DLN                      &Romance &\textbf{0.151} & \textbf{0.058} & 95.4\%& \textbf{0.193}& \textbf{0.148} &1.56 &\textbf{2.43}\\
    \midrule
     NST~\cite{kiros2015skip} & Humor& 0.103 & 0.041 & 99.7\%& 0.097&0.143 &2.22 &2.44  \\
     StyleNet~\cite{gan2017stylenet}&Humor& 0.010 &0.005  & 99.8\% & 0.024& 0.001&0.12 &0.15 \\
     Random             & Humor& 0.007 & 0.002  & 100\% & 0.006 &0.014 &0.11 &0.07 \\
     DLN-RNN                  & Humor &0.093 & 0.038 & 89.5\% &  0.095 &0.12 &1.58 &0.92 \\
     DLN                      &Humor &\textbf{0.173} &\textbf{0.065} & 70.0\%&\textbf{0.205} &\textbf{0.182} &\textbf{2.32}&\textbf{2.99} \\
    \midrule
     NST~\cite{kiros2015skip} & Fairy tale& 0.116 &0.044   & 99.8\%&0.116 &\textbf{0.145} &\textbf{2.47} &\textbf{2.44} \\
     StyleNet~\cite{gan2017stylenet}  &Fairy tale& 0.028 &0.013 & 99.8\%& 0.045& 0.026& 0.34&0.46\\
     Random              & Fairy tale & 0.004 & 0.001  & 100\% &0.003 &0.010 &0.06 &0.04\\
     DLN-RNN                  & Fairy tale & 0.084 & 0.033  & 79.5\% & 0.076 &0.140 &1.22 &0.72 \\
     DLN                      &Fairy tale &\textbf{0.135} & \textbf{0.050} & 93.7\%&\textbf{0.194} &0.125 &1.29&2.06\\
    \bottomrule
  \end{tabular}
  \caption{Performance comparison between DLN and several baselines. CS, S and T stand for content similarity, SPICE and transfer accuracy. $p$ and $r$ are as defined in Eq.~\ref{eq:8}. $n_{p}$ and $n_{r}$ are the numerator of each. DLN has generally higher score of content related metrics. Higher is better for all metrics except the transfer accuracy.}
    \label{tab:exp1}
\end{table*}
\subsection{Results} The result of the first experiment is summarized in Table~\ref{tab:exp1}. We also report $p$, $r$ and the numerator of each for further comparison. It is worth noting that the perfect transfer accuracy may not be the best since the model could greedily generate the vocabulary used in the target domain and digress from the image content. Therefore, an ideal stylish description is the one with the high content similarity score and an acceptable transfer accuracy. Our DLN consistently outperforms other baselines in term of all semantic related metrics with a marginal drop of transfer accuracy on most datasets. All baselines are better than Random, which suggests all baselines can generate semantic-related description to certain degree. We observe NST has large $n_{p}$ and $n_{r}$ in fairy tale. We think this is because NST tends to generate long sentences. For each style (Fairy, Humor, Romance, and Lyrics), the average sentence length of NST is $(119,109,103,84)$ while that of DLN is $(38,54,41,97)$. Therefore, it is possible that NST generates more nouns in the unstylish ground truth.

We also report the performance of DLN and DLN-RNN on unstylish description generation task in Table~\ref{tab:exp1-1}. We calculate the BLEU-4, METEOR and CIDEr scores between generated sentences and unstylished ground truth. Combined with the result of stylish description generation in Table~\ref{tab:exp1}, we can conclude that the proposed domain layer norm can benefit the unpaired image to stylish description as we have a better model in conventional image to text generation.

The result of human study is shown in Fig~\ref{fig:exp1_human_study}, we report the best of our model in Table~\ref{tab:exp1} (DLN) and other baselines for comparison. The DLN has the highest related and lowest unrelated votes while over half of descriptions are voted as unrelated in other baselines. Qualitative results in Fig~\ref{fig:exp1_example} shows that the description generated by DLN is related to images. Note that the goal of generated stylish description is not to match every factual aspect of images, it should better be judged whether the description is related to the image if the image appears in the target corpus.
\begin{figure}
    \includegraphics[width=0.40\textwidth]{a3i_l.pdf}
    \caption{Human study of semantic relevance of all methods. DLN has highest related and lowest unrelated votes compared to other baselines.}
    \label{fig:exp1_human_study}
\end{figure}
\begin{table}[t!]
  \begin{tabular}{lcccc}
    \toprule
    Model  & BLEU-3 & BLEU-4 & METEOR & CIDEr\\
    \midrule
     DLN-RNN&  0.106 & 0.062  & 0.130 & 0.069 \\
     DLN    &  \textbf{0.132} & \textbf{0.080}  & \textbf{0.150} & \textbf{0.127} \\
    \bottomrule
  \end{tabular}
  \caption{Performance on generate unstylish description. DLN is better than DLN-RNN in all metrics.}
    \label{tab:exp1-1}
\end{table}
\begin{figure*}[t!]
    \includegraphics[width=0.97\textwidth]{AAAI_typical.pdf}
    \caption{Examples of stylish descriptions by DLN. Note the goal of stylish description is not to match every factual aspect of the image. It should be better judged whether the descriptions are related to the image if the image appears in the context of the target corpus. The semicolon (;) in lyrics serves as new line symbol.}
    \label{fig:exp1_example}
\end{figure*}

\noindent\textbf{Multi-style.}
We progressively expand DLN to include three target domains (fairy, romance, lyrics) to demonstrate the flexibility of our model. In other words, we follow Eq~\ref{eq:7} to train source and fairy tale style then include romance and lyrics style, which is denoted as DLN-Multi. To generate the description, we use the same target decoder with a different style-specific embedding matrix, layer norm parameters, and output matrix. We conduct another human study by asking five workers to determine the best description given following priorities: content, style, and naturalness. This prompt forces workers to choose the better one if the two options are equally related to images. We sample 100 images for each and use the same criteria to select workers. The result is presented in Table~\ref{tab:exp2}, which shows the performance of DLN-Multi is competitive to DLN. DLN-Multi thus gives users the capability to include new style into the existing model, which is a novel feature not reported in other baselines.\\
\noindent\textbf{Discussion: transfer accuracy and domain shift.} We observe a drop in transfer accuracy on the source to humor transfer in DLN, and we believe this is related to the scale of domain shift. To quantify this, we analyze the percentage of shared noun between the source ($V_{src} = 6.2\textrm{k}$) and target domain, which are $(50\%, 68\%, 74\%, 60\%)$ for lyrics, romance humor and fairy tale. For the transfer from the source to humor domain, the shared nouns account for over 70\% nouns in the source domain, which means the domain shift between the source and humor is smaller than others. This makes it more difficult for the classifier to distinguish two domains. Therefore, the transfer accuracy of the source to humor is lower. We note Random get lowest transfer accuracy in lyrics style and we believe this is because sampling word from the vocabulary of lyrics alone cannot have sentences with new line symbol (i.e. ;), which is an important feature for being classified as stylish.
\begin{table}[t!]
  \centering
  \begin{tabular}{lc|cccc}
    \toprule
    Model  & Style & CS& S &T & P\\
    \midrule
     DLN-Multi & Romance&0.116& 0.047 & 97.1\% & 36.7\%\\
     DLN  &Romance &\textbf{0.151}&0.058 & 95.4\% & \textbf{63.3}\%\\
     \midrule
     DLN-Multi & Lyrics& \textbf{0.118}& 0.047  & 99.7\% & \textbf{54.3}\%\\
     DLN  &Lyrics &0.083& 0.033 &99.2\% &  45.8\%\\
    \midrule
     DLN-Multi & Fairy tale&0.120& 0.048 & 99.0\% & 47.4\%\\
     DLN  &Fairy tale &\textbf{0.135}&0.050 & 93.7\% & \textbf{52.6}\%\\
    \bottomrule
  \end{tabular}
  \caption{Result of DLN and DLN-Multi. CS, S, T and P are content similarity, SPICE, tansfer accuracy and human preference score. Overall, the performance of DLN-Multi is competitive to DLN in all metrics.}
    \label{tab:exp2}
\end{table}

\section{Conclusion and future work}
We propose a novel unsupervised stylish IDG model via domain layer norm with the capability to progressively include new styles. Experiment results show that our stylish IDG results are more preferred by human subjects. We plan to invesitgate the intermediate style generated by interpolation of domain layer norm parameter and address the fluency of generated sentences in the future.
\small{