\section{Introduction}
In recent years, vehicular technology has attracted significant attention from the automotive and telecommunication industries, leading to the emergence of vehicle-to-everything (V2X) communications for improving road safety, traffic management services and driving comfort.
V2X supported by the sixth generation (6G) is envisioned to be a key enabler of future connected autonomous vehicles \cite{9779322}. Although its transformative benefits for leveraging intelligent transportation systems, V2X still face several technical issues mainly related to performance and security.

The integration of sensing and communication (ISAC) has emerged very recently as a revolutionary element of 6G that could potentially help enabling adaptive learning and intelligent decision-making in future V2X applications.
The combination of sensing and communication allows vehicles to perceive their surroundings better, predict manoeuvres from nearby users and make intelligent decisions, thus paving the way toward a safer transportation system \cite{9665433}.
Modernized vehicles are augmented with various types of sensors divided into exteroceptive to observe their surrounding environment and proprioceptive to observe their internal states.
The former like GPS, Lidar, and Cameras are conveyed to improve situational awareness, while latter sensors, such as steering, pedal, and wheel speed, convey to improve self-awareness.

While sensing the environment, vehicles can exchange messages that assist in improving situational- and self-awareness and in coordinating maneuvers with other vehicles.
Those messages like the basic safety (BSMs) and cooperative awareness messages (CAMs) are composed of transmitting vehicle's states such as position and velocity and other vehicles' states in the vicinity. Vehicles might use their sensors, such as cameras and Lidar, to detect road users (e.g., pedestrians), which can be communicated with other road users via the V2X messages to improve the overall performance. However, V2X communication links carrying those messages are inherently vulnerable to malicious attacks due to the open and shared nature of the wireless spectrum among vehicles and other cellular users \cite{8336901}. For instance, a jammer in the vicinity might alter the information to be communicated to nearby vehicles/users or can intentionally disrupt communication between a platoon of vehicles making the legitimate signals unrecognizable for on-board units (OBUs) and/or road side units (RSUs) that endanger vehicular safety
\cite{8553649}.

In addition, the integrity of GPS signals and the correct acquisition of navigation data to compute position, velocity and time information is critical in V2X applications for their safe operation. However, since civil GPS receivers rely on unencrypted satellite signals, spoofers can easily replicate them by deceiving the GPS receiver to compute falsified positions \cite{9226611}.
Also, the long distance between satellites and terrestrial GPS receivers leads to an extremely weak signal that can be easily drowned out by a spoofer.
Thus, GPS sensors' vulnerability to spoofing attacks poses a severe threat that might be causing vehicles to be out of control or even hijacked and endanger human life \cite{9881548}.
Therefore, GPS spoofing attacks and jamming interference needs to be controlled and detected in real-time to reach secured vehicular communications allowing vehicles to securely talk to each other and interact with the infrastructure (e.g., roadside terminals, base stations) \cite{9860410}.

Existing methods for GPS spoofing detection include GPS signal analysis methods and GPS message encryption methods \cite{9845684}. However, the former requires the ground truth source during the detection process, which is not always possible to collect. In contrast, the latter involves support from a secured infrastructure and advanced computing resources on GPS receivers, which hinders their adoption in V2X applications. On the other hand, existing methods for jammer detection in vehicular networks are based on analysing the packet drop rate as in \cite{9484071}, making it difficult to detect an advanced jammer manipulating the legitimate signal instead of disrupting it.
In this work, we propose a method to jointly detect GPS spoofing and jamming attacks in the V2X network. A coupled generalized dynamic Bayesian network (C-GDBN) is employed to learn the interaction between RF signals received by the RSU from multiple vehicles and their corresponding trajectories. This integration of vehicles' positional information with vehicle-to-infrastructure (V2I) communications allows semantic learning while mapping RF signals with vehicles' trajectories and enables the RSU to jointly predict the RF signals it expects to receive from the vehicles from which it can anticipate the expected trajectories.

The main contributions of this paper can be summarized as follows: \textit{i)} A joint GPS spoofing and jamming detection method is proposed for the V2X scenario, which is based on learning a generative interactive model as the C-GDBN. Such a model encodes the cross-correlation between the RF signals transmitted by multiple vehicles and their trajectories, where their semantic meaning is coupled stochastically at a high abstraction level. \textit{ii)} A cognitive RSU equipped with the acquired C-GDBN can predict and estimate vehicle positions based on real-time RF signals. This allows RSU to evaluate whether both RF signals and vehicles' trajectories are evolving according to the dynamic rules encoded in the C-GDBN and, consequently, to identify the cause (i.e., a jammer attacking the V2I or a spoofer attacking the satellite link) of the abnormal behaviour that occurred in the V2X environment. \textit{iii)} Extensive simulation results demonstrate that the proposed method accurately estimates the vehicles' trajectories from the predicted RF signals, effectively detect any abnormal behaviour and identify the type of abnormality occurring with high detection probabilities.
To our best knowledge, this is the first work that studies the joint detection of jamming and spoofing in V2X systems.

\section{System model and problem formulation}
The system model depicted in Fig.~\ref{fig_SystemModel}, includes a single cell vehicular network consisting of a road side unit (RSU) located at $\mathrm{p}_{R}=[{x}_{R},{y}_{R}]$, a road side jammer (RSJ) located at $\mathrm{p}_{J}=[{x}_{J},{y}_{J}]$, a road side spoofer (RSS) located at $\mathrm{p}_{s}=[{x}_{s},{y}_{s}]$ and $N$ vehicles moving along multi-lane road in an urban area. The time-varying positions of the $n$-th vehicle is given by $\mathrm{p}_{n,t}=[{x}_{n,t},{y}_{n,t}]$ where $n \in N$. Among the $K$ orthogonal subchannels available for the Vehicle-to-Infrastructure (V2I) communications, RSU assigns one V2I link to each vehicle. Each vehicle exchanges messages composed of the vehicle's state (i.e., position and velocity) with RSU through the $k$-th V2I link by transmitting a signal $\textrm{x}_{t,k}$ carrying those messages at each time instant $t$ where $k \in K$. We consider a reactive RSJ that aims to attack the V2I link by injecting intentional interference to the communication link between vehicles and RSU to alter the transmitted signals by the vehicles. In contrast, the RSS purposes to mislead the vehicles by spoofing the GPS signal and so registering wrong GPS positions. RSU aims to detect both the spoofer on the satellite link and the jammer on multiple V2I links in order to take effective actions and protect the vehicular network.
The joint GPS spoofing and jamming detection problem can be formulated as the following ternary hypothesis test:
\begin{equation}
    \begin{cases}
        \mathcal{H}_{0}: \mathrm{z}_{t,k} = \mathrm{g}_{t,k}^{nR} \mathrm{x}_{t,k} + \mathrm{v}_{t,k}, \\
        \mathcal{H}_{1}: \mathrm{z}_{t,k} = \mathrm{g}_{t,k}^{nR} \mathrm{x}_{t,k} + \mathrm{g}_{t,k}^{JR} \mathrm{x}_{t,k}^{j} + \mathrm{v}_{t,k}, \\
        \mathcal{H}_{2}: \mathrm{z}_{t,k} = \mathrm{g}_{t,k}^{nR} \mathrm{x}_{t,k}^{*} + \mathrm{v}_{t,k},
    \end{cases}
\end{equation}
where $\mathcal{H}_{0}$, $\mathcal{H}_{1}$ and $\mathcal{H}_{2}$ denote three hypotheses corresponding to the absence of both jammer and spoofer, the presence of the jammer, and the presence of the spoofer, respectively. $\textrm{z}_{t,k}$ is the received signal at the RSU at $t$ over the $k$-th V2I link, $\textrm{g}_{t,k}^{nR}$ is the channel power gain from vehicle $n$ to the RSU formulated as: $\textrm{g}_{t,k}^{nR} = \alpha_{t,k}^{nR} \mathrm{h}_{t,k}^{nR}$, where $\alpha_{t,k}^{nR}$ is the large-scale fading including path-loss and shadowing modeled as \cite{8723178}: $\alpha_{t,k}^{nR}=G\beta d_{t,nR}^{-\gamma}$.
\begin{figure}[t!]
    \centering
    \includegraphics[height=5.3cm]{Figures/SystemModel_V1.pdf}
    \caption{An illustration of the system model.}
    \label{fig_SystemModel}
\end{figure}
$G$ is the pathloss constant, $\beta$ is a log normal shadow fading random variable, $d_{t,nR}=\sqrt{({x}_{n,t}-x_{R})^{2}+({y}_{n,t}-y_{R})^{2}}$ is the distance between the $n$-th vehicle and the RSU. $\gamma$ is the power decay exponent and
$\mathrm{h}_{t,k}$ is the small-scale fading component distributed according to $\mathcal{CN}(0,1)$. In addition, $\mathrm{x}_{t,k}$ is the desired signal transmitted by the $n$-th vehicle, and $\mathrm{v}_{t,k}$ is an additive white Gaussian noise with variance $\sigma_{n}^{2}$. $\mathrm{x}_{t,k}^{J}$ is the jamming signal, $\mathrm{x}_{t,k}^{*}$ is the spoofed signal (i.e., the signal that carries the bits related to the wrong GPS positions), $\mathrm{g}_{t,k}^{JR} = \alpha_{t,k}^{JR} \mathrm{h}_{t,k}^{JR}$ is the channel power gain from RSJ to RSU where $\alpha_{t,k}^{JR}=G\beta d_{t,JR}^{-\gamma}$ such that $d_{t,JR}=\sqrt{({x}_{J}-x_{R})^{2}+({y}_{J}-y_{R})^{2}}$.
We assume that the channel state information (CSI) of V2I links is known and can be estimated at the RSU as in \cite{8345717}.
The RSU is equipped with an RF antenna which can track the vehicles' trajectories after decoding the received RF signals. RSU aims to learn the interaction between the RF signals received from multiple vehicles and their corresponding trajectories.

\section{Proposed method for joint detection of GPS spoofing and jamming}

\subsection{Environment Representation}
The RSU is receiving RF signals from each vehicle and tracking its trajectory (which we refer to as GPS signal) by decoding and demodulating the received RF signals.
The Generalized state-space model describing the $i$-th signal evolvement at multiple levels embodies the following equations:
\begin{equation} \label{eq_discreteLevel}
    \mathrm{\Tilde{S}_{t}}^{(i)} = \mathrm{f}(\mathrm{\Tilde{S}_{t-1}}^{(i)}) + \mathrm{\tilde{w}}_{t},
\end{equation}
\begin{equation} \label{eq_continuousLevel}
    \mathrm{\Tilde{X}_{t}}^{(i)} = \mathrm{A} \mathrm{\Tilde{X}_{t-1}}^{(i)} + \mathrm{B} \mathrm{U}_{\mathrm{\Tilde{S}_{t}}^{(i)}} + \mathrm{\tilde{w}}_{t},
\end{equation}
\begin{equation} \label{eq_observationLevel}
    \mathrm{\Tilde{Z}_{t}}^{(i)} = \mathrm{H} \mathrm{\Tilde{X}_{t}}^{(i)} + \mathrm{\tilde{v}}_{t},
\end{equation}
where $i \in \{$RF, GPS$\}$ indicates the type of signal received by the RSU. The transition system model defined in \eqref{eq_discreteLevel} explains the evolution of the discrete random variables $\mathrm{\Tilde{S}_{t}}^{(i)}$ representing the clusters of the RF (or GPS) signal dynamics, $\mathrm{f}(.)$ is a non linear function of its argument and the additive term $\mathrm{\tilde{w}}_{t}$ denotes the process noise. The dynamic model defined in \eqref{eq_continuousLevel} explains the RF signal dynamics evolution or the motion dynamics evolution of the $n$-th vehicle, where $\mathrm{\Tilde{X}_{t}}^{(i)}$ are hidden continuous variables generating sensory signals, $\mathrm{A} \in \mathbb{R}^{2d}$ and $\mathrm{B} \in \mathbb{R}^{2d}$ are the dynamic and control matrices, respectively, and $\mathrm{U}_{\mathrm{\Tilde{S}_{t}}^{(i)}}$ is the control vector representing the dynamic rules of how the signals evolve with time. The measurement model defined in \eqref{eq_observationLevel} describes dependence of the sensory signals $\mathrm{\Tilde{Z}_{t}}^{(i)}$ on the hidden states $\mathrm{\Tilde{X}_{t}}^{(i)}$ that is parametrized by the measurement matrix $\mathrm{B} \in \mathbb{R}^{2d}$ where $d$ stands for the data dimensionality and $\mathrm{\tilde{v}}_{t}$ is a random noise.

\subsection{Learning GDBN}
The hierarchical dynamic models defined in \eqref{eq_discreteLevel}, \eqref{eq_continuousLevel} and \eqref{eq_observationLevel} are structured in a Generalized Dynamic Bayesian Network (GDBN) \cite{9858012} as shown in Fig.~\ref{fig_GDBN_CGDBN}-(a) that provides a probabilistic graphical model expressing the conditional dependencies among random hidden variables and observable states. The generative process explaining how sensory signals have been generated can be factorized as:
\begin{equation} \label{eq_generative_process}
\begin{split}
    \mathrm{P}(\mathrm{\tilde{Z}}_{t}^{(i)}, \mathrm{\tilde{X}}_{t}^{(i)}, \mathrm{\tilde{S}}_{t}^{(i)}) = \mathrm{P}(\mathrm{\tilde{S}}_{0}^{(i)}) \mathrm{P}(\mathrm{\tilde{X}}_{0}^{(i)}) \\ \bigg[ \prod_{t=1}^{\mathrm{T}} \mathrm{P}(\mathrm{\tilde{Z}}_{t}^{(i)}|\mathrm{\tilde{X}}_{t}^{(i)}) \mathrm{P}(\mathrm{\tilde{X}}_{t}^{(i)}|\mathrm{\tilde{X}}_{t-1}^{(i)}, \mathrm{\tilde{S}}_{t}^{(i)}) \mathrm{P}(\mathrm{\tilde{S}}_{t}^{(i)}|\mathrm{\tilde{S}}_{t-1}^{(i)}) \bigg],
\end{split}
\end{equation}
where $\mathrm{P}(\mathrm{\tilde{S}}_{0}^{(i)})$ and $\mathrm{P}(\mathrm{\tilde{X}}_{0}^{(i)})$ are initial prior distributions, $\mathrm{P}(\mathrm{\tilde{Z}}_{t}^{(i)}|\mathrm{\tilde{X}}_{t}^{(i)})$ is the likelihood, $\mathrm{P}(\mathrm{\tilde{X}}_{t}^{(i)}|\mathrm{\tilde{X}}_{t-1}^{(i)}, \mathrm{\tilde{S}}_{t}^{(i)})$ and $\mathrm{P}(\mathrm{\tilde{S}}_{t}^{(i)}|\mathrm{\tilde{S}}_{t-1}^{(i)})$ are the transition densities describing the temporal and hierarchical dynamics of the generalized state-space model.
The generative process defined in \eqref{eq_generative_process} indicates the cause-effect relationships the model impose on the random variables $\mathrm{\tilde{S}}_{t}^{(i)}$, $\mathrm{\tilde{X}}_{t}^{(i)}$ and $\mathrm{\tilde{Z}}_{t}^{(i)}$ forming a chain of causality describing how one state contributes to the production of another state which is represented by the link $\mathrm{\tilde{S}}_{t}^{(i)} \rightarrow \mathrm{\tilde{X}}_{t}^{(i)} \rightarrow \mathrm{\tilde{Z}}_{t}^{(i)}$.

The RSU starts perceiving the environment using a static assumption about the environmental states evolution by assuming that sensory signals are only subject to random noise. Hence, RSU predicts the RF signal (or vehciles trajectory) using the following simplified model:
$\mathrm{\tilde{X}}_{t}^{(i)} = \mathrm{A} \mathrm{\tilde{X}}_{t-1}^{(i)} + \mathrm{\tilde{w}}_{t}$,
that differs from \eqref{eq_continuousLevel} in the control vector $\mathrm{U}_{\mathrm{\Tilde{S}_{t}}^{(i)}}$ which is supposed to be null, i.e., $\mathrm{U}_{\mathrm{\Tilde{S}_{t}}^{(i)}} = 0$ as the dynamic rules explaining how the environmental states evolve with time are not discovered yet.
Those rules can be discovered by exploiting the generalized errors (GEs), i.e., the difference between predictions and observations. The GEs projected into the measurement space are calculated as:
$\tilde{\varepsilon}_{\mathrm{\tilde{Z}}_{t}^{(i)}}^{} = \mathrm{\tilde{Z}}_{t}^{(i)} - \mathrm{H} \mathrm{\tilde{X}}_{t}^{(i)}$.
Projecting $\tilde{\varepsilon}_{\mathrm{\tilde{Z}}_t}^{}$ back into the generalized state space can be done as follows:
\begin{equation}\label{GE_continuousLevel_initialModel}
    \tilde{\varepsilon}_{\mathrm{\tilde{X}}_t}^{(i)} = \mathrm{H}^{-1}\tilde{\varepsilon}_{\mathrm{\tilde{Z}}_{t}^{(i)}}^{}=\mathrm{H}^{-1}(\mathrm{\tilde{Z}}_{t}^{(i)}-\mathrm{H}\mathrm{\tilde{X}}_{t}^{(i)}) = \mathrm{H}^{-1}\mathrm{\tilde{Z}}_{t}^{(i)} - \mathrm{\tilde{X}}_{t}^{(i)}.
\end{equation}
The GEs defined in \eqref{GE_continuousLevel_initialModel} can be grouped into discrete clusters in an unsupervised manner by employing the Growing Neural Gas (GNG). The latter produces a set of discrete variables (clusters) denoted by:
$\mathbf{\tilde{S}^{(i)}}=\{\mathrm{\tilde{S}}_{1}^{(i)},\mathrm{\tilde{S}}_{2}^{(i)},\dots,\mathrm{\tilde{S}}_{M_{i}}^{(i)}\}$,
where $M_{i}$ is the total number of clusters and each cluster $\mathrm{\tilde{S}}_{m}^{(i)} \in \mathbf{\tilde{S}^{(i)}}$ follows a Gaussian distribution composed of GEs with homogeneous properties, such that $\mathrm{\tilde{S}}_{m}^{(i)} \sim \mathcal{N}(\tilde{\mu}_{\mathrm{\tilde{S}}_{m}^{(i)}}=[\mu_{\tilde{S}_{m}^{(i)}}, \Dot{\mu}_{\tilde{S}_{m}^{(i)}}], \Sigma_{\mathrm{\tilde{S}}_{m}^{(i)}})$.
\begin{figure}[t!]
    \begin{center}
        \begin{minipage}[b]{.40\linewidth}
        \centering
           \includegraphics[width=2.5cm]{Figures/GDBN.pdf}
        \\[-1.0mm]
        {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{.50\linewidth}
            \centering
            \includegraphics[width=5.0cm]{Figures/C_GDBN.pdf}

            {\scriptsize (b)}
        \end{minipage}
        \caption{(a) The GDBN. (b) The coupled GDBN (C-GDBN) composed of two GDBNs representing the two signals received at the RSU where their discrete hidden variables are stochastically coupled.}
        \label{fig_GDBN_CGDBN}
    \end{center}
\end{figure}
The dynamic transitions of the sensory signals among the available clusters can be captured in a time-varying transition matrix ($\Pi_{\tau}$) by estimating the time-varying transition probabilities $\pi_{ij}=\mathrm{P}(\mathrm{\tilde{S}}_{t}^{(i)}=i|\mathrm{\tilde{S}}_{t-1}^{(i)}=j, \tau)$ where $\tau$ is the time spent in $\mathrm{\tilde{S}}_{t-1}^{(i)}=j$ before transition to $\mathrm{\tilde{S}}_{t}^{(i)}=i$.

\subsection{Learning Coupled GDBN (C-GDBN)}
The learning procedure described in the previous section can be executed for each signal type, i.e., RF and GPS. After learning a separated GDBN model for each signal type, we analyse the interaction behaviour between RF signal and GPS signal received at the RSU by tracking the cluster firing among $\mathbf{\tilde{S}^{(1)}}$ and $\mathbf{\tilde{S}^{(2)}}$ during a certain experience. Such an interaction can be encoded in a Coupled GDBN (C-GDBN) as shown in Fig.\ref{fig_GDBN_CGDBN}-(b) composed of the two GDBNs representing the two signals where their hidden variables at the discrete level are stochastically coupled (in $\mathrm{\tilde{C}}_{t}{=}[\mathrm{\tilde{S}}_{t}^{(1)},\mathrm{\tilde{S}}_{t}^{(2)}]$) as those variables are uncorrelated but have coupled means.
The interactive matrix $\Phi \in \mathbb{R}^{M_{1},M_{2}}$ encodes the firing cluster pattern allowing to predict the GPS signal from RF signal is defined as follows:
\begin{equation} \label{interactiveTM_fromRFtoGPS}
\Phi =
        \begin{bmatrix}
            \mathrm{P}(\mathrm{\Tilde{S}_{1}}^{(2)}|\mathrm{\Tilde{S}_{1}}^{(1)}) & \mathrm{P}(\mathrm{\Tilde{S}_{2}}^{(2)}|\mathrm{\Tilde{S}_{1}}^{(1)}) & \dots & \mathrm{P}(\mathrm{\Tilde{S}_{M_{2}}}^{(2)}|\mathrm{\Tilde{S}_{1}}^{(1)}) \\
            \mathrm{P}(\mathrm{\Tilde{S}_{1}}^{(2)}|\mathrm{\Tilde{S}_{2}}^{(1)}) & \mathrm{P}(\mathrm{\Tilde{S}_{2}}^{(2)}|\mathrm{\Tilde{S}_{2}}^{(1)}) & \dots & \mathrm{P}(\mathrm{\Tilde{S}_{M_{2}}}^{(2)}|\mathrm{\Tilde{S}_{2}}^{(1)}) \\
            \vdots & \vdots & \ddots & \vdots \\
            \mathrm{P}(\mathrm{\Tilde{S}_{1}}^{(2)}|\mathrm{\Tilde{S}_{M_{1}}}^{(1)}) & \mathrm{P}(\mathrm{\Tilde{S}_{2}}^{(2)}|\mathrm{\Tilde{S}_{M_{1}}}^{(1)}) & \dots & \mathrm{P}(\mathrm{\Tilde{S}_{M_{2}}}^{(2)}|\mathrm{\Tilde{S}_{M_{1}}}^{(1)})
        \end{bmatrix}.
\end{equation}

\subsection{Joint Prediction and Perception}
RSU starts predicting the RF signals it expects to receive from each vehicle based on a Modified Markov Jump Particle Filter (M-MJPF) \cite{9858012} that combines Particle filter (PF) and Kalman filter (KF) to perform temporal and hierarchical predictions. Since the acquired C-GDBN allows predicting a certain signal's dynamic evolution based on another's evolution, it requires an interactive Bayesian filter capable of dealing with more complicated predictions. To this purpose, we propose to employ an Interactive M-MJPF (IM-MJPF) on the C-GDBN. The IM-MJPF consists of a PF that propagates a set of $L$ particles equally weighted, such that $\{\mathrm{\tilde{S}}_{t,l}^{(1)}, \mathrm{W}_{t,l}^{(1)}\}{\sim}\{\pi(\mathrm{\tilde{S}}_{t}^{(1)}), \frac{1}{L}\}$, where $\mathrm{\tilde{S}}_{t,l}^{(1)}$, $l \in L$ and $(.^{(1)})$ is the RF signal type. In addition, RSU relies on $\Phi$ defined in \eqref{interactiveTM_fromRFtoGPS} to predict $\mathrm{\tilde{S}}_{t}^{(2)}$ realizing the discrete cluster of vehicle's trajectory starting from the predicted RF signal according to: $\{\mathrm{\tilde{S}}_{t}^{(2)},\mathrm{W}_{t,l}^{(2)}\}{\sim} \{\Phi(\mathrm{\tilde{S}}_{t,l}^{(1)}){=}\mathrm{P}(.|\mathrm{\tilde{S}}_{t,l}^{(1)}), \mathrm{W}_{t,l}^{(2)}\}$. For each predicted discrete variable $\mathrm{\tilde{S}}_{t,l}^{(i)}$, a multiple KF is employed to predict multiple continuous variables which guided by the predictions at the higher level as declared in \eqref{eq_continuousLevel} that can be represented probabilistically as $\mathrm{P}(\mathrm{\tilde{X}}_{t}^{(i)}|\mathrm{\tilde{X}}_{t-1}^{(i)}, \mathrm{\tilde{S}}_{t}^{(i)})$. The posterior probability that is used to evaluate expectations is given by:
\begin{multline} \label{piX}
    \pi(\mathrm{\tilde{X}}_{t}^{(i)})=\mathrm{P}(\mathrm{\tilde{X}}_{t}^{(i)},\mathrm{\tilde{S}}_{t}^{(i)}|\mathrm{\tilde{Z}}_{t-1}^{(i)})= \\ \int \mathrm{P}(\mathrm{\tilde{X}}_{t}^{(i)}|\mathrm{\tilde{X}}_{t-1}^{(i)}, \mathrm{\tilde{S}}_{t}^{(i)}) \lambda(\mathrm{\tilde{X}}_{t-1}^{(i)})d\mathrm{\tilde{X}}_{t-1}^{(i)},
\end{multline}
where $\lambda(\mathrm{\tilde{X}}_{t-1}^{(i)}){=}\mathrm{P}(\mathrm{\tilde{Z}}_{t-1}^{(i)}|\mathrm{\tilde{X}}_{t-1}^{(i)})$.
The posterior distribution can be updated (and so representing the updated belief) after having seen the new evidence $\mathrm{\tilde{Z}}_{t}^{(i)}$ by exploiting the diagnostic message $\lambda(\mathrm{\tilde{X}}_{t}^{(i)})$ in the following form: $\mathrm{P}(\mathrm{\tilde{X}}_{t}^{(i)}, \mathrm{\tilde{S}}_{t}^{(i)}|\mathrm{\tilde{Z}}_{t}^{(i)}) {=} \pi(\mathrm{\tilde{X}}_{t}^{(i)})\lambda(\mathrm{\tilde{X}}_{t}^{(i)})$. Likewise, belief in discrete hidden variables can be updated according to: $\mathrm{W}_{t,l}^{(i)}{=}\mathrm{W}_{t,l}^{(i)}\lambda (\mathrm{\tilde{S}}_{t}^{(i)})$ where:
$\lambda (\mathrm{\tilde{S}}_{t}^{(i)}) {=} \lambda (\mathrm{\Tilde{X}}_{t}^{(i)})\mathrm{P}(\mathrm{\Tilde{X}}_{t}^{(i)}|\mathrm{\tilde{S}}_{t}^{(i)}) {=} \mathrm{P}(\mathrm{\tilde{Z}}_{t}^{(i)}|\mathrm{\Tilde{X}}_{t}^{(i)})\mathrm{P}(\mathrm{\Tilde{X}}_{t}^{(i)}|\mathrm{\tilde{S}}_{t}^{(i)})$.

\subsection{Joint GPS spoofing and jamming detection}
RSU can evaluate the current situation and identify if V2I is under attack, or the satellite link is under spoofing based on a multiple abnormality indicator produced by the IM-MJPF. The first indicator calculates the similarity between the predicted RF signal and the observed one, which is defined as:
\begin{equation}\label{eq_CLA1}
    \Upsilon_{\mathrm{\tilde{X}}_{t}^{(1)}} = -ln \bigg( \mathcal{BC} \big(\pi(\mathrm{\tilde{X}}_{t}^{(1)}),\lambda(\mathrm{\tilde{X}}_{t}^{(1)}) \big) \bigg),
\end{equation}
where $\mathcal{BC}(.){=}\int \sqrt{\pi(\mathrm{\tilde{X}}_{t}^{(1)}),\lambda(\mathrm{\tilde{X}}_{t}^{(1)}})d\mathrm{\tilde{X}}_{t}^{(1)}$ is the Bhattacharyya coefficient.
The second indicator calculates the similarity between the predicted GPS signal (from the RF signal) and the observed one after decoding the RF signal which is defined as:
\begin{equation}\label{eq_CLA2}
    \Upsilon_{\mathrm{\tilde{X}}_{t}^{(2)}} = -ln \bigg( \mathcal{BC} \big(\pi(\mathrm{\tilde{X}}_{t}^{(2)}),\lambda(\mathrm{\tilde{X}}_{t}^{(2)}) \big) \bigg),
\end{equation}
where $\mathcal{BC}(.){=}\int \sqrt{\pi(\mathrm{\tilde{X}}_{t}^{(2)}),\lambda(\mathrm{\tilde{X}}_{t}^{(2)}})d\mathrm{\tilde{X}}_{t}^{(2)}$.
Different hypotheses can be identified by the RSU to understand the current situation whether there is: a jammer attacking the V2I link, or a spoofer attacking the link between the satellite and the vehicle or both jammer and spoofer are absent according to:
\begin{equation}
    \begin{cases}
        \mathcal{H}_{0}: \text{if} \ \ \Upsilon_{\mathrm{\tilde{X}}_{t}^{(1)}} < \xi_{1} \ \text{and} \ \Upsilon_{\mathrm{\tilde{X}}_{t}^{(2)}} < \xi_{2}, \\
        \mathcal{H}_{1}: \text{if} \ \ \Upsilon_{\mathrm{\tilde{X}}_{t}^{(1)}} \geq \xi_{1} \ \text{and} \ \Upsilon_{\mathrm{\tilde{X}}_{t}^{(2)}} \geq \xi_{2}, \\
        \mathcal{H}_{2}: \text{if} \ \ \Upsilon_{\mathrm{\tilde{X}}_{t}^{(1)}} < \xi_{1} \ \text{and} \ \Upsilon_{\mathrm{\tilde{X}}_{t}^{(2)}} \geq \xi_{2},
    \end{cases}
\end{equation}
where $\xi_{1} = \mathbb{E}[\Bar{\Upsilon}_{\mathrm{\tilde{X}}_{t}^{(1)}}] + 3\sqrt{\mathbb{V}[\Bar{\Upsilon}_{\mathrm{\tilde{X}}_{t}^{(1)}}]}$, and $\xi_{2} = \mathbb{E}[\Bar{\Upsilon}_{\mathrm{\tilde{X}}_{t}^{(2)}}] + 3\sqrt{\mathbb{V}[\Bar{\Upsilon}_{\mathrm{\tilde{X}}_{t}^{(2)}}]}$. In $\xi_{1}$ and $\xi_{2}$, $\Bar{\Upsilon}_{\mathrm{\tilde{X}}_{t}^{(1)}}$ and $\Bar{\Upsilon}_{\mathrm{\tilde{X}}_{t}^{(2)}}$ stand for the abnormality signals during training (i.e., normal situation when jammer and spoofer are absent).

\subsection{Evaluation metrics}
In order to evaluate the performance of the proposed method to jointly detect jammer and GPS spoofer, we adopt the jammer detection probability ($\mathrm{P}_{d}^{j}$) and the spoofer detection probability ($\mathrm{P}_{d}^{s}$), respectively, which are defined as:
\begin{equation}
    \mathrm{P}_{d}^{j} = \mathrm{Pr}(\Upsilon_{\mathrm{\tilde{X}}_{t}^{(1)}}\geq \xi_{1}, \Upsilon_{\mathrm{\tilde{X}}_{t}^{(2)}} \geq \xi_{2}|\mathcal{H}_{1}),
\end{equation}
\begin{equation}
    \mathrm{P}_{d}^{s} = \mathrm{Pr}(\Upsilon_{\mathrm{\tilde{X}}_{t}^{(1)}}< \xi_{1}, \Upsilon_{\mathrm{\tilde{X}}_{t}^{(2)}} \geq \xi_{2}|\mathcal{H}_{2}).
\end{equation}
Also, we evaluate the accuracy of the proposed method in predicting and estimating the vehicles' trajectories and the expected RF signals by adopting the root mean square error (RMSE) defined as:
\begin{equation}
    RMSE = \sqrt{ \frac{1}{T} \sum_{t=1}^{T}\bigg( \mathrm{\tilde{Z}}_{t}^{(i)}-\mathrm{\tilde{X}}_{t}^{(i)} \bigg)^{2} },
\end{equation}
where $T$ is the total number of predictions.

\section{Simulation Results}
In this section, we evaluate the performance of the proposed method to jointly detect the jammer and the spoofer using extensive simulations. We consider $\mathrm{N}=2$ vehicles interacting inside the environment and exchanging their states (i.e., position and velocity) with the RSU. The vehicles move along predefined trajectories performing various maneuvers which are picked from the \textit{Lankershim} dataset proposed by \cite{5206559}. The dataset depicts a four way intersection and includes about $19$ intersection maneuvers. RSU assigns one subchannel realizing the V2I link for each vehicle over which the vehicles' states are transmitted. The transmitted signal carrying the vehicle's state and the jamming signal are both QPSK modulated.
The simulation settings are: carrier frequency of $2$GHz, BW${=}1.4$MHz, cell radius of $500$m, RSU antenna height and gain is $25$m and $8$ dBi, receiver noise figure of $5$dB, vehicle antenna height and gain is $1.5$m and $3$dBi, vehicle speed is $40$Km/h, V2I transmit power is $23$dBm, jammer transmit power ranging from $20$dBm to $40$dBm, SNR of $20$dB, path loss model ($128.1{+}37.6log d$), Log-normal shadowing with $8$dB standard deviation and a fast fading channel following the Rayleigh distribution.
\begin{figure}[ht!]
    \begin{center}
        \begin{minipage}[b]{.55\linewidth}
        \centering
            \includegraphics[width=5.0cm]{Results/ObservedTrajectories_reference}
        \\[-1.5mm]
        {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{.49\linewidth}
            \centering
            \includegraphics[width=4.9cm]{Results/ObservedRFsignal_Veh1_reference}
            \\[-1.5mm]
            {\scriptsize (b)}
        \end{minipage}
        \begin{minipage}[b]{.49\linewidth}
            \centering
            \includegraphics[width=4.9cm]{Results/ObservedRFsignal_Veh2_reference}
            \\[-1.5mm]
            {\scriptsize (c)}
        \end{minipage}
        \caption{An example visualizing the received RF signals from the two vehicles and the corresponding trajectories: (a) Vehicles' trajectories, (b) received RF signal from vehicle 1, (c) received RF signal from vehicle 2.}
        \label{fig_receivedRFsignalandTrajectory}
    \end{center}
\end{figure}
\begin{figure}[t!]
    \begin{center}
        \begin{minipage}[b]{.49\linewidth}
            \centering
                \includegraphics[width=4.5cm]{Results/clusters_trajectory_veh1}
            \\[-1.5mm]
            {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{.49\linewidth}
            \centering
                \includegraphics[width=4.5cm]{Results/clusters_trajectory_veh2}
            \\[-1.5mm]
            {\scriptsize (b)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.5cm]{Results/clusters_RFsignal_veh1}
            \\[-1.5mm]
            {\scriptsize (c)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.5cm]{Results/clusters_RFsignal_veh2}
            \\[-1.5mm]
            {\scriptsize (d)}
        \end{minipage}

        \caption{GNG output after clustering the generalized errors obtained from different experiences: (a) clustered trajectory of vehicle 1, (b) clustered trajectory of vehicle 2, (c) clustered RF signal received from vehicle 1, (d) clustered RF signal received from vehicle 2.}
        \label{fig_GNG_of_receivedRFsignalandTrajectory}
    \end{center}
\end{figure}

The RSU aims to learn multiple interactive models (i.e., C-GDBN models) encoding the cross relationship between the received RF signal from each vehicle and its corresponding trajectory. These models allow the RSU to predict the trajectory the vehicle will follow based on the received RF signal and evaluate whether the V2I is under jamming attacks or the satellite link is under spoofing. It is to note that the RSU is receiving only the RF signals from the two vehicles and obtaining their positions after decoding the RF signals. Thus, the RSU should be able to evaluate if the received RF signals are evolving according to the dynamic rules learned so far and if the vehicles are following the expected (right) trajectories to decide whether the V2I links are really under attack or whether the satellite link is under spoofing.

Fig.~\ref{fig_receivedRFsignalandTrajectory}-(a) illustrates an example of the interaction between the two vehicles performing a particular manoeuvre, and Fig.~\ref{fig_receivedRFsignalandTrajectory}-(b) shows the received RF signals by the RSU from the two vehicles. At the beginning of the learning process, RSU performs predictions according to the simplified model defined in \eqref{eq_continuousLevel} where $\mathrm{U}_{\mathrm{\Tilde{S}_{t}}^{(i)}} {=} 0$.
After obtaining the generalized errors as pointed out in \eqref{GE_continuousLevel_initialModel}, RUS clusters those errors using GNG to learn two GDBN models encoding the dynamic rules of how the RF signal and the GPS signal evolve with time, respectively, as showed in Fig.~\ref{fig_GNG_of_receivedRFsignalandTrajectory} and Fig.~\ref{fig_graphicalRep_transitionMatrices}. RSU can couple the two GDBNs by learning the interactive transition matrix that is encoded in a C-GDBN as shown in Fig.~\ref{fig_interactiveMatrices}.
\begin{figure}[t!]
    \begin{center}
        \begin{minipage}[b]{.49\linewidth}
            \centering
                \includegraphics[width=4.5cm]{Results/graphTransition_Trajectory_veh1}
            \\[-1.5mm]
            {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{.49\linewidth}
            \centering
                \includegraphics[width=4.5cm]{Results/graphTransition_Trajectory_veh2}
            \\[-1.5mm]
            {\scriptsize (b)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.5cm]{Results/graphTransition_RFsignal_veh1}
            \\[-1.5mm]
            {\scriptsize (c)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.5cm]{Results/graphTransition_RFsignal_veh2}
            \\[-1.5mm]
            {\scriptsize (d)}
        \end{minipage}
        \caption{Graphical representation of the transition matrices (TM): (a) TM related to the trajectory of vehicle 1, (b) TM related to the trajectory of vehicle 2, (c) TM related to the RF signal received from vehicle 1, (d) TM related to the RF signal received from vehicle 2.}
        \label{fig_graphicalRep_transitionMatrices}
    \end{center}
\end{figure}
\begin{figure}[t!]
    \begin{center}
        \begin{minipage}[b]{.49\linewidth}
            \centering
                \includegraphics[width=3.8cm]{Results/interactiveMatrix_RFtoGPS_Neu5_veh1}
            \\[-1.0mm]
            {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=3.8cm]{Results/interactiveMatrix_RFtoGPS_Neu25_veh1}
            \\[-1.0mm]
            {\scriptsize (d)}
        \end{minipage}

        \caption{Interactive transition matrix defined in \eqref{interactiveTM_fromRFtoGPS} using different configurations: (a) $\mathrm{M_{1}}=5$, $\mathrm{M_{2}}=5$, (b) $\mathrm{M_{1}}=25$, $\mathrm{M_{2}}=25$.}
        \label{fig_interactiveMatrices}
    \end{center}
\end{figure}
\begin{figure}[t!]
    \begin{center}
        \begin{minipage}[b]{.49\linewidth}
        \centering
            \includegraphics[width=4.9cm]{Results/RF_situation1_best_veh1}
        \\[-1.5mm]
        {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.9cm]{Results/RF_situation1_worst_veh1}
            \\[-1.5mm]
            {\scriptsize (b)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.9cm]{Results/RF_situation1_best_veh2}
            \\[-1.5mm]
            {\scriptsize (c)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.9cm]{Results/RF_situation1_worst_veh2}
            \\[-1.5mm]
            {\scriptsize (d)}
        \end{minipage}
        \caption{An example visualizing the predicted and observed RF signals transmitted by the 2 vehicles using different configurations. Predicted RF signal from: (a) vehicle 1 using $\mathrm{M_{1}}{=}5$, $\mathrm{M_{2}}{=}5$, (b) vehicle 1 using $\mathrm{M_{1}}{=}25$, $\mathrm{M_{2}}{=}25$, (c) vehicle 2 using $\mathrm{M_{1}}{=}5$, $\mathrm{M_{2}}{=}5$, (d) vehicle 2 using $\mathrm{M_{1}}{=}25$, $\mathrm{M_{2}}{=}25$.}
            \label{fig_situation1_PredictedRF}
    \end{center}
\end{figure}

\begin{figure}[t!]
    \begin{center}
        \begin{minipage}[b]{.49\linewidth}
        \centering
            \includegraphics[width=4.8cm]{Results/GPSfromRF_situation1_best}
        \\[-1.0mm]
        {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.8cm]{Results/GPSfromRF_situation1_worst}
            \\[-1.0mm]
            {\scriptsize (b)}
        \end{minipage}
        %
        \caption{An example visualizing the predicted and observed trajectories of two vehicles interacting in the environment. (a) $\mathrm{M_{1}}{=}5$, $\mathrm{M_{2}}{=}5$, (b) $\mathrm{M_{1}}{=}25$, $\mathrm{M_{2}}{=}25$.}
            \label{fig_situation1_VehiclesTrajectories}
    \end{center}
\end{figure}

\begin{figure}[ht!]
    \begin{center}
        \begin{minipage}[b]{.49\linewidth}
        \centering
            \includegraphics[width=4.8cm]{Results/rmse_on_trajectory}
        \\[-1.0mm]
        {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{0.49\linewidth}
            \centering
            \includegraphics[width=4.8cm]{Results/rmse_on_RFSignal}
            \\[-1.0mm]
            {\scriptsize (b)}
        \end{minipage}
        \caption{The average RMSE after testing different experiences and examples of: (a) trajectories and (b) RF signals.}
            \label{fig_rmse_onTraj_onSig}
    \end{center}
\end{figure}

Fig.~\ref{fig_situation1_PredictedRF} illustrates an example comparing between predicted RF signals and observed ones based on two different configurations in learning the interactive matrix (as shown in Fig.~\ref{fig_interactiveMatrices}). Also, Fig.~\ref{fig_situation1_VehiclesTrajectories} illustrates an example comparing between the predicted and observed trajectories of the two vehicles using the two interactive matrices depicted in Fig.~\ref{fig_interactiveMatrices}. From Fig.~\ref{fig_situation1_PredictedRF} and Fig.~\ref{fig_situation1_VehiclesTrajectories} we can see that using an interactive matrix with less clusters allows to perform better predictions compared to that with more clusters. This can be validated by observing Fig.~\ref{fig_rmse_onTraj_onSig} that illustrates the RMSE values versus different number of clusters related to the two models representing the dynamics of the received RF signals and the vehicles' trajectories. It can be seen that as the number of clusters increases the RMSE error increases, since adding more clusters decreases the firing probability that explains the possibility to be in one of the $M_{2}$ clusters of the second model conditioned in being in a certain cluster of the first model.

Fig.~\ref{fig_exNormal_Spoofed_JammedTrajectories} illustrates an example of vehicle's trajectory under normal situation (i.e., jammer and spoofer are absent), under jamming attacks and under spoofing attacks. Also the figure shows the predicted trajectory which should follow the same dynamic rules learned during a normal situation. After that, we implemented the IM-MJPF on the learned C-GDBN to perform multiple predictions, i.e., to predict the RF signal that the RSU is expecting to receive from a certain vehicle and the corresponding trajectory that the vehicle is supposed to follow. IM-MJPF through the comparison between multiple predictions and observations, produces multiple abnormality signals as defined in \eqref{eq_CLA1} and \eqref{eq_CLA2} which are used to detect the jammer and the spoofer.

Fig.~\ref{fig_abnormalitySignals_JammerSpoofer} illustrates the multiple abnormality signals related to the example shown in Fig.~\ref{fig_exNormal_Spoofed_JammedTrajectories}. We can observe that the abnormal signals related to both RF signal (Fig.~\ref{fig_abnormalitySignals_JammerSpoofer}-(a)) and trajectory (Fig.~\ref{fig_abnormalitySignals_JammerSpoofer}-(b)) are below the threshold under normal situations. This proves that RSU learned the correct dynamic rules of how RF signals and trajectories evolve when the jammer and spoofer are absent (i.e., under normal situations). Also, we can see that the RSU can notice a high deviation on both the RF signal and the corresponding trajectory due to a jamming interference from what it has learned so far by relying on the abnormality signals. In contrast, we can see that under spoofing attacks, RSU notice a deviation only on the trajectory and not on the RF signal since the spoofer has affected only the positions without manipulating the RF signal. In addition, it is obvious how the proposed method allows the RSU to identify the type of abnormality occurring and to explain the cause of the detected abnormality (i.e., understanding if it was because of a jammer attacking the V2I link or a spoofer attacking the satellite link).
\begin{figure}[t!]
    \centering
    \includegraphics[width=6.5cm]{Results/trajectories_underJamming_andSpoofing}

    \caption{Vehicle's trajectory under: normal situation, jamming and spoofing.}
    \label{fig_exNormal_Spoofed_JammedTrajectories}
\end{figure}
\begin{figure}[t!]
    \begin{center}
        \begin{minipage}[b]{.92\linewidth}
        \centering
            \includegraphics[height=2.6cm]{Results/abnSignal_onRF}
        \\[-1.5mm]
        {\scriptsize (a)}
        \end{minipage}
        \begin{minipage}[b]{.92\linewidth}
            \centering
            \includegraphics[height=2.6cm]{Results/abnSignal_onGPS}
            \\[-1.5mm]
            {\scriptsize (b)}
        \end{minipage}
        %
        \caption{Abnormality Signals related to the example shown in Fig.\ref{fig_exNormal_Spoofed_JammedTrajectories}: (a) abnormality indicators related to the RF signal, (b) abnormality indicators related to the trajectory.}
            \label{fig_abnormalitySignals_JammerSpoofer}
    \end{center}
\end{figure}
\begin{figure}[t!]
    \centering
    \includegraphics[height=3.2cm]{Results/Detection_Probability_RFfromGPS_versusPj}
    \caption{Detection probability ($\mathrm{P_{d}}$) versus jammer's power ($\mathrm{P_{J}}$) using different number of clusters $\mathrm{M}_{2}$.}
    \label{fig_jammerDetectionProb}
\end{figure}
\begin{figure}[t!]
    \centering
    \includegraphics[height=3.2cm]{Results/spoofingDetectionProbability_falseAlarm_versusM2}
    \caption{Spoofing detection probability ($\mathrm{P}_{d}^{s}$) and spoofing false alarm ($\mathrm{P}_{f}^{s}$) versus the number of clusters $\mathrm{M}_{2}$.}
    \label{fig_spooferDetectionProb}
\end{figure}

Fig.~\ref{fig_jammerDetectionProb} shows the overall performance of the proposed method in detecting the jammer by testing many situations and examples and by considering different jamming powers which ranges from $20$dBm to $40$dBm. It can be seen that the proposed method is able to detect the jammer with high probabilities (near $1$) and by considering low and high jamming powers. Also, the figure compares the performance in detecting the jammer by varying the number of clusters ($M_{2}$).
Fig.~\ref{fig_spooferDetectionProb} shows the overall performance of the proposed method in detecting the spoofer by testing different different examples of driving maneuvers. It can be seen that the RSU is able to detect the spoofer with high detection probability and null false alarm versus different number of clusters.

\section{Conclusion}
A joint detection method of GPS spoofing and jamming attacks is proposed. The method is based on learning a dynamic interactive model encoding the cross-correlation between the received RF signals from multiple vehicles and their corresponding trajectories. Simulation results show the high effectiveness of the proposed approach in jointly detecting the GPS spoofer and jammer attacks.
Subsequent work will extend the system model to consider more than two vehicles with different channel conditions and various modulation schemes to evaluate the effectiveness of the proposed method.

\bibliographystyle{IEEEtran}


\section{Introduction}
Canonical correlation analysis (CCA) is a powerful tool to integrate two data matrices \cite{klami2013bayesian,sun2008least,yang2017canonical,cai2016kernel,wang2017novel}, which has been comprehensively used in many diverse fields. Given two matrices $\bm{X}\in \mathbb{R}^{n\times p}$ and $\bm{Y}\in \mathbb{R}^{n\times q}$ from the same samples, CCA is used to find two sparse canonical vectors $\bm{u}$ and $\bm{v}$ to maximize the correlation between $\bm{X}\bm{u}$ and $\bm{Y}\bm{v}$. However, in many real-world problems like those in bioinformatics \cite{witten2009penalized,mizutani2012relating,le2009sparse,fang2016joint,yoshida2017sparse}, the number of variables in each data matrix is usually much larger than the sample size. The classical CCA leads to non-sparse canonical vectors which are difficult to interpret in biology. To conquer this issue, a large number of sparse CCA models \cite{witten2009penalized,mizutani2012relating,le2009sparse,fang2016joint,yoshida2017sparse,parkhomenko2009sparse,witten2009extensions,asteris2016simple,chu2013sparse,hardoon2011sparse,gao2015minimax} have been proposed by using regularized penalties (\emph{e.g.}, LASSO and $L_0$-norm) to obtain sparse canonical vectors for variable selection. Parkhomenko \emph{et al.} \cite{parkhomenko2009sparse} first proposed a Sparse CCA (SCCA) model using LASSO ($L_1$-norm) penalty to genomic data integration. $\mbox{L}\hat{e}$ Cao \emph{et al.} \cite{le2009sparse} further proposed a regularized CCA with Elastic-Net penalty for a similar task. Witten \emph{et al.} \cite{witten2009penalized} proposed the Penalized matrix decomposition (PMD) algorithm to solve the Sparse CCA with two penalties: LASSO and Fused LASSO to integrate DNA copy number and gene expression from the same samples/individuals. Furthermore, a large number of generalized LASSO regularized CCA models have been proposed to consider prior structural information of variables \cite{lin2013group,virtanen2011bayesian,chen2012structure,chen2012structured,du2016structured}. For example, Lin \emph{et al.} \cite{lin2013group} proposed a Group LASSO regularized CCA to explore the relationship between two types of genomic data sets. If we consider a pathway as a gene group, then these gene pathways form an overlapping group structure \cite{jacob2009group}. Chen \emph{et al.} \cite{chen2012structured} developed an overlapping group LASSO regularized CCA model to employ such group structure.

These existing sparse CCA models can find two sparse canonical vectors with a small subset of variables across all samples (Fig.1(a)). However, many real data such as the cancer genomic data show distinct heterogeneity \cite{dai2015breast,chen2016integrative}. Thus, the current CCA models fail to consider such heterogeneity and cannot directly identify a set of sample-specific correlated variables. To this end, we propose a novel Sparse weighted CCA (SWCCA) model, where weights are used for regularizing different samples with a typical penalty (\emph{e.g.}, LASSO and $L_0$-norm) (Fig.1(b)). In this way, SWCCA can not only select two variable sets, but also select a sample set (Fig.1 (b)). We further adopt an efficient alternating iterative algorithm to solve $L_0$ (or $L_1$) regularized SWCCA model. We apply $L_0$-SWCCA and related ones onto two simulated datasets and two real biological data to demonstrate its efficiency in capturing correlated variables across a subset of samples.
\begin{figure}[ht]
  \centering
  \includegraphics[width=1\linewidth]{figures/fig1.pdf}
  \caption{Illustration of the difference between SWCCA and SCCA. (a) SCCA is used to extract two sparse canonical vectors ($\bm{u}$ and $\bm{v}$) to measure the association of two matrices; (b) SWCCA is used to consider two subset of sample-related sparse canonical vectors. The weights ($\bm{w}$) are used for regularizing different samples  in SWCCA. SWCCA can not only obtain two sparse canonical vectors, but also identify a set of samples based on those non-zero elements of $\bm{w}$.}
\end{figure}
\section{$\bm{L_0}$-regularized SWCCA}
Here, we assume that there are two data matrices $\bm{X}\in \mathbb{R}^{n\times p}$ ($n$ samples and $p$ variables) and $\bm{Y} \in \mathbb{R}^{n\times q}$ ($n$ samples and $q$ variables) across a same set of samples. The classical CCA seeks two components ($\bm{u}$ and $\bm{v}$) to maximize the correlation between linear combinations of variables from the two data matrices as Eq.(1).
\begin{equation}
 \rho = \frac{\bm{u}^{\rm{T}} \bm{\Sigma}_{xy}\bm{v}}{\sqrt{(\bm{u}^{\rm{T}}\bm{\Sigma}_{x}\bm{u})(\bm{v}^{\rm{T}}\bm{\Sigma}_{y}\bm{v})}}
\end{equation}
If $\bm{X}$ and $\bm{Y}$ are centered, we obtain the empirical covariance matrices $\bm{\Sigma}_{xy} = \frac{1}{n}\bm{X}^{\rm{T}}\bm{Y}$, $\bm{\Sigma}_{x} = \frac{1}{n}\bm{X}^{\rm{T}}\bm{X}$ and $\bm{\Sigma}_{y} = \frac{1}{n}\bm{Y}^{\rm{T}}\bm{X}$. Thus we have the following equivalent criterion as Eq.(2).
\begin{equation}
 \rho = \frac{\bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\bm{Y}\bm{v}} {\sqrt{(\bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\bm{X}\bm{u})(\bm{v}^{\rm{T}}\bm{Y}^{\rm{T}}\bm{Y}\bm{v})}}
\end{equation}
Obviously, $\rho$ of (2) is invariant to the scaling of $\bm{u}$ and $\bm{v}$. Thus, maximizing criterion (2) is equivalent to solve the following constrained optimization problem as Eq.(3).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{u},\bm{v}} & \bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\bm{Y}\bm{v} \\
    \mbox{s.t.}& \bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\bm{X}\bm{u} =  1, \bm{v}^{\rm{T}}\bm{Y}^{\rm{T}}\bm{Y}\bm{v} = 1
  \end{array}
\end{equation}
Previous studies \cite{witten2009penalized,witten2009extensions} have shown that considering the covariance matrix ($\frac{1}{n}\bm{X}^{\rm{T}}\bm{X}$, $\frac{1}{n}\bm{Y}^{\rm{T}}\bm{Y}$) as diagonal one can obtain better results. For this reason, Asteris \emph{et al.} \cite{asteris2016simple} assume that $\bm{X}^{\rm{T}}\bm{X} = \bm{I}$ and $\bm{Y}^{\rm{T}}\bm{Y} = \bm{I}$, and the $L_0$-regularized Sparse CCA ($L_0$-SCCA) (also called ``diagonal penalized CCA'') model can be presented as Eq.(4).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{u},\bm{v}} & \bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\bm{Y}\bm{v}\\
    \mbox{s.t.}& \|\bm{u}\|_0\leq k_u, \|\bm{v}\|_0\leq k_v\\
               & \bm{u}^{\rm{T}}\bm{u} = \bm{v}^{\rm{T}}\bm{v} = 1
  \end{array}
\end{equation}
where $\|\bm{u}\|_0$ is the $L_0$-norm penalty function, which returns to the number of non-zero entries of $\bm{u}$.
Asteris \emph{et al.}\cite{asteris2016simple}  applied a projection strategy to solve $L_0$-SCCA. Let $\bm{A}=\bm{X}^{\rm{T}}\bm{Y}$, then the model of Eq.(4) is equivalent to rank-one $L_0$-SVD model \cite{min2015novel}.

Let $\bm{a} = \bm{X}\bm{u}$ and $\bm{b} = \bm{Y}\bm{v}$, then the objective function $\bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\bm{Y}\bm{v} = \sum_{i=1}^n a_ib_i$. To consider the different contribution for samples, we modify the objective function of Eq.(4) to be $\sum_{i=1}^n w_i(a_ib_i)$ with $\bm{w}=[w_1,w_2,\cdots,w_n]^{\rm{T}}$. Thus, we obtain a new objective function as Eq.(5).
\begin{equation}
  \sum_{i=1}^n w_i(a_ib_i) = \bm{u}\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v}
\end{equation}
Furthermore, we also force $\bm{w}$ to be sparse to select a limited number of samples. Finally we propose a $L_0$-regularized SWCCA ($L_0$-SWCCA) model as Eq.(6).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{u},\bm{v},\bm{w}} & \bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v}\\
    \mbox{s.t.} & \|\bm{u}\|_0\leq k_u, \|\bm{v}\|_0\leq k_v, \|\bm{w}\|_0\leq k_w\\
                & \bm{u}^{\rm{T}}\bm{u} = \bm{v}^{\rm{T}}\bm{v} = \bm{w}^{\rm{T}}\bm{w} = 1
  \end{array}
\end{equation}
where $\mbox{diag}(\bm{w})$ is a diagonal matrix and $\mbox{diag}(\bm{w})_{ii} = w_i$. If $\mbox{diag}(\bm{w}) = \frac{1}{\sqrt{n}}\bm{I}$, then $L_0$-SWCCA reduces to $L_0$-SCCA.

\section{Optimization}
In this section, we design an alternating iterative algorithm to solve (6) by using a sparse projection strategy. We start with the sparse projection problem corresponding to the sub-problem of (6) with fixed $\bm{v}$ and $\bm{w}$ as Eq.(7).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{u}} & \bm{u}^{\rm{T}}\bm{z} \\
    \mbox{s.t.}& \bm{u}^{\rm{T}}\bm{u} =  1, \|\bm{u}\|_0 \leq k
  \end{array}
\end{equation}
For a given column vector $\bm{z} \in \mathbb{R}^{p \times 1}$ and $k\leq p$, we define a sparse project operator $\Pi(\cdot,k)$ as Eq.(8).
\begin{eqnarray}
  [\Pi(\bm{z},k)]_i =
  \begin{cases}
    z_i, &\mbox{if}~i \in \mbox{support}(\bm{z},k)\cr
    0, &\mbox{otherwise}
  \end{cases}
\end{eqnarray}
where $\mbox{support}(\bm{z},k)$ is defined as a set of indexes corresponding to the largest $k$ absolute values of $\bm{z}$. For example, if $\bm{z}=[-5, 3, 5, 2, -1]^{\rm{T}}$, then $\Pi(\bm{z},3) = [-5, 3, 5, 0, 0]^{\rm{T}}$.

{\bf Theorem 1}\quad  The solution of problem (7) is
\begin{equation}
  \bm{u}^* =  \frac{\Pi(\bm{z},k)}{\|\Pi(\bm{z},k)\|_2}
\end{equation}

Note that $\|\cdot\|_2$ denotes the Euclidean norm. We can prove the Theorem~1 by contradiction (we omit the proof here). Based on Theorem~1, we design an alternating iterative approach to solve Eq.(6).

i) Optimizing $\bm{u}$ with fixed $\bm{v}$ and $\bm{w}$.
Fix $\bm{v}$ and $\bm{w}$ in Eq.(6), let $\bm{z}_u =\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v}$, then Eq.(6) reduces to as Eq.(10).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{u}} & \bm{u}^{\rm{T}}\bm{z}_u \\
    \mbox{s.t.}& \bm{u}^{\rm{T}}\bm{u} =  1, \|\bm{u}\|_0 \leq k_u
  \end{array}
\end{equation}
Based on the Theorem~1, we obtain the update rule of $\bm{u}$ as Eq.(11).
\begin{equation}
   \bm{u} \leftarrow \frac{\Pi(\bm{z}_u,k_u)}{\|\Pi(\bm{z}_u,k_u)\|_2}
\end{equation}

ii) Optimizing $\bm{v}$ with fixed $\bm{u}$ and $\bm{w}$.
Fix $\bm{u}$ and $\bm{w}$ in Eq.(6), let $\bm{z}_v=\bm{Y}^{\rm{T}}\mbox{diag}(\bm{w})\bm{X}\bm{u}$, then Eq.(6) reduces to as Eq.(12).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{v}} & \bm{v}^{\rm{T}}\bm{z}_v \\
    \mbox{s.t.}& \bm{v}^{\rm{T}}\bm{v} =  1, \|\bm{v}\|_0 \leq k_v
  \end{array}
\end{equation}
Similarly, we obtain the update rule of $\bm{v}$ as Eq.(13).
\begin{equation}
   \bm{v} \leftarrow \frac{\Pi(\bm{z}_v,k_v)}{\|\Pi(\bm{z}_v,k_v)\|_2}
\end{equation}

iii) Optimizing $\bm{w}$ with fixed $\bm{u}$ and $\bm{v}$.
Fix $\bm{u}$ and $\bm{v}$ in Eq.(6), then Eq.(6) reduces to as Eq.(14).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{w}} & \bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v}\\
    \mbox{s.t.}& \bm{w}^{\rm{T}}\bm{w} =  1, \|\bm{w}\|_0 \leq k_u
  \end{array}
\end{equation}
Let $\bm{t}_1 = \bm{Xu}$, $\bm{t}_2 = \bm{Yu}$ and $\bm{z}_w = \bm{t}_1\odot \bm{t}_2$ where `$\odot$' denotes point multiplication which is equivalent to `.*' in Matlab, then we have
$\bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v} =\bm{t}_1\mbox{diag}(\bm{w})\bm{t}_2 = (\bm{t}_1\odot\bm{w})^{\rm{T}}\bm{t}_2 = \bm{w}^{\rm{T}}(\bm{t}_1\odot\bm{t}_2) = \bm{w}^{\rm{T}}\bm{z}_w$. Thus, problem (14) reduces to as Eq.(15).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{w}} & \bm{w}^{\rm{T}}\bm{z}_w\\
    \mbox{s.t.}& \bm{w}^{\rm{T}}\bm{w} =  1, \|\bm{w}\|_0 \leq k_w
  \end{array}
\end{equation}
Similarly, we obtain the update rule of $\bm{w}$ as Eq.(16).
\begin{equation}
   \bm{w} \leftarrow \frac{\Pi(\bm{z}_w,k_w)}{\|\Pi(\bm{z}_w,k_w)\|_2}
\end{equation}
Finally, combining (11), (13) and (16), we propose the following alternating iterative algorithm to solve problem (6) as Algorithm 1.

\begin{figure*}[htbp]
  \centering
  \includegraphics[width=1\linewidth]{figures/fig2.pdf}
  \caption{Results of the synthetic data 1. (a), (f) and (e) denote true $\bm{u}$, $\bm{v}$ and $\bm{w}$; (b), (g) and (j) denote estimated $\bm{u}$, $\bm{v}$ and $\bm{w}$ by $L_0$-SWCCA; (c) and (h) denote estimated $\bm{u}$ and $\bm{v}$ by $L_0$-SCCA; (d) and (i) denote stimated $\bm{u}$ and $\bm{v}$ by PMD.}
\end{figure*}

\begin{algorithm}[h]
\caption{\textbf{$\bm{L_0}$-SWCCA}.} \label{alg:Framwork1}
\begin{algorithmic}[1]
\REQUIRE $\bm{X}\in \mathbb{R}^{n\times p}$, $\bm{Y}\in \mathbb{R}^{n\times q}$, $k_u$, $k_v$ and $k_w$.
\ENSURE $\bm{u}$, $\bm{v}$, and $\bm{w}$.
\STATE Initial  $\bm{u}$, $\bm{v}$, and $\bm{w}$
\REPEAT
\STATE Update $\bm{u}$ according to Eq.(11)
\STATE Update $\bm{v}$ according to Eq.(13)
\STATE Update $\bm{w}$ according to Eq.(16)
\UNTIL convergence of $\bm{u}$, $\bm{v}$, and $\bm{w}$.
\end{algorithmic}
\end{algorithm}

{\bf Terminating Condition:} We can set different stop conditions to control the iterations. For example, the update length of $\bm{u}$, $\bm{v}$, and $\bm{w}$ are smaller than a given threshold (\emph{i.e.}, $\|\bm{u}^k - \bm{u}^{k+1}\|_2^2<10^{-6}$, $\|\bm{v}^k - \bm{v}^{k+1}\|_2^2<10^{-6}$ and $\|\bm{w}^k - \bm{w}^{k+1}\|_2^2<10^{-6}$), or the maximum number of iterations is a given number (\emph{e.g.}, 1000), or the change of objective value is less than a give threshold.

{\bf Computation Complexity:} The complexity of matrix multiplication with one $n\times p$ matrix and another one $n \times q$ is $\mathcal{O}(npq)$. To reduce the computational complexity of $\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v}$, we note that $\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v} = \bm{X}^{\rm{T}}(\mbox{diag}(\bm{w})\bm{Y}\bm{v}) = \bm{X}^{\rm{T}}[(\bm{Y}\bm{v})\odot\bm{w}]$. Let $\bm{t}_1 = \bm{Y}\bm{v}$, $\bm{t}_2 = \bm{t}_1\odot\bm{w}$ and $\bm{t}_3 = \bm{X}^{\rm{T}}\bm{t}_2$. Thus, the complexity of $\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v}$ is $\mathcal{O}(nq+n+np)$. Similarly, we can see that the complexity of $\bm{Y}^{\rm{T}}\mbox{diag}(\bm{w})\bm{X}\bm{u}$ is $\mathcal{O}(np+n+nq)$, and the complexity of $(\bm{Xu})\odot(\bm{Yv})$ is $\mathcal{O}(nq+np+n)$.
In Algorithm 1, we need to obtain the largest $k$ absolute values of a given vector $\bm{z}$ of size $p\times 1$ [\emph{i.e.}, $\Pi(\bm{z},k)$]. We adopt a linear time selection algorithm called Quick select (QS) algorithm to compute $\Pi(\bm{z},k)$, which applies a divide and conquer strategy, and the average time complexity of QS algorithm is $\mathcal{O}(p)$. Thus, the entire time complexity of Algorithm 1 is $\mathcal{O}(Tnp+Tnq)$, where $T$ is the number of iterations for convergence. In general, $T$ is a small number.

{\bf Convergence Analysis:} Similar with Theorem 1 in Ref. \cite{sun2015multi}, also see e.g., \cite{bolte2014proximal}, we can prove that Algorithm 1 converges globally to a critical point (we omit the proof here).
\section{Experiments}
\subsection{Synthetic data 1}
Here we generate the first synthetic data matrices $\bm{X}$ and $\bm{Y}$ with $n = 50$, $p = 100$ and  $q = 80$ using the following two steps:

Step 1: Generate two canonical vectors $\bm{u}$, $\bm{v}$ and a weighted vector $\bm{w}$ as Eq.(17).
\begin{equation}
  \begin{array}{rl}
    \bm{u}~~= & [r(1,30),r(0,70)]^{\rm{T}}\\
    \bm{v}~~= & [N(20),r(0,20),N(10),r(0,30)]^{\rm{T}}\\
    \bm{w}~~= & [r(1,30),r(0,20)]^{\rm{T}}\\
  \end{array}
\end{equation}
where $r(a,n)$ denotes a row vector of size $n$, whose elements are equal to $a$, $N(m)$ denotes a row vector of size $m$, whose elements are randomly sampled from a standard normal distribution.

Step 2: Generate two input matrices $\bm{X}$ and $\bm{Y}$ as Eq.(18).
\begin{equation}
  \begin{array}{rl}
    \bm{X}~~= & \bm{w}\bm{u}^{\rm{T}} + \bm{\epsilon}_x \\
    \bm{Y}~~= & \bm{w}\bm{v}^{\rm{T}} + \bm{\epsilon}_y
  \end{array}
\end{equation}
where the elements of $\bm{\epsilon}_x$ and $\bm{\epsilon}_y$ are randomly sampled from a standard normal distribution.

We evaluate the performance of $L_0$-SWCCA with the above synthetic data and compare its performance with the typical sparse CCA, including $L_0$-SCCA \cite{asteris2016simple} and PMD \cite{witten2009penalized} with $L_1$-penalty. For comparison,  we set parameters $k_u = 30$, $k_v = 30$ and $k_w = 30$ for $L_0$-SWCCA; $k_u = 30$, $k_v = 30$ for $L_0$-SCCA; $c_1 = \frac{30}{100}\sqrt{p}$ and $c_2 = \frac{30}{80}\sqrt{q}$ for PMD. Note that $c_1 = c\sqrt{p}$ and $c_2 = c\sqrt{q}$ where $c \in (0, 1)$ for PMD are to approximately control the sparse proportion of the canonical vectors ($\bm{u}$ and $\bm{v}$).

The true and estimated patterns for $\bm{u}$, $\bm{v}$ and $\bm{w}$ in the synthetic data 1 are shown in Fig.2. Compared to PMD, $L_0$-SWCCA and $L_0$-SCCA does fairly well for identifying the local non-zero pattern of the underlying factors (\emph{i.e.}, $\bm{u}$ and $\bm{v}$). However, the two traditional SCCA methods ($L_0$-SCCA and PMD) do not recognize the difference between samples and remove the noisy samples. Interestingly, $L_0$-SWCCA not only discovers the true patterns for $\bm{u}$, $\bm{v}$ (Fig.2(b) and (g)), but also identifies the true non-zero characteristics of samples ($\bm{w}$) (Fig.2(e)). Furthermore, to assess our approach is indeed able to find a greater correlation level between two input matrices, we define the correlation criterion as Eq.(19).
\begin{equation}
\rho = \mbox{cor}((\bm{X}\bm{u})\odot\bm{w}, (\bm{Y}\bm{v})\odot\bm{w})
\end{equation}
where $\mbox{cor}(\cdot)$ is a function to calculate the correlation coefficient of the two vectors. For comparison, we set $\bm{w} = [1, \cdots, 1]^{\rm{T}}$ for $L_0$-SCCA and PMD to compute the correlation criterion.
$L_0$-SWCCA gets the largest $\rho=0.96$ compared to $L_0$-SCCA with $\rho=0.80$ and PMD with $\rho=0.87$ in the above synthetic data. All results show that $L_0$-SWCCA is more effective to capture the latent patterns of canonical vectors than other methods.
\subsection{Synthetic data 2}
Here we apply another way to generate synthetic data matrices $\bm{X}\in \mathbb{R}^{n\times p}$ and $\bm{Y}\in \mathbb{R}^{n\times q}$ with $n = 50$, $p = 100$, and  $q = 80$. The following three steps are used to generate the second synthetic data matrices $\bm{X}$ and $\bm{Y}$:

Step 1: We first generate two zero matrices as Eq.(20).
\begin{equation}
  \begin{array}{rl}
    \bm{X}~~\leftarrow & matrix(0,nrow=n, ncol=p)\\
    \bm{Y}~~\leftarrow & matrix(0,nrow=n, ncol=q)
  \end{array}
\end{equation}

Step 2: We then update two sub-matrices in the $\bm{X}$ and $\bm{Y}$.
\begin{equation}
  \begin{array}{rl}
    \bm{X}[1:30,1:50]~~\leftarrow & 1\\
    \bm{Y}[1:30,1:40]~~\leftarrow & - 1
  \end{array}
\end{equation}

Step 3: We add the Gaussian noise in $\bm{X}$ and $\bm{Y}$.
\begin{equation}
  \begin{array}{rl}
    \bm{X}~~\leftarrow &  \bm{X} +  \bm{\epsilon}_x\\
    \bm{Y}~~\leftarrow &  \bm{Y} +  \bm{\epsilon}_y
  \end{array}
\end{equation}
\begin{figure*}[htbp]
  \centering
  \includegraphics[width=1\linewidth]{figures/fig4.pdf}
  \caption{Results of the synthetic data 2. (a), (f) and (e) denote true $\bm{u}$, $\bm{v}$ and $\bm{w}$; (b), (g) and (j) denote estimated $\bm{u}$, $\bm{v}$ and $\bm{w}$ by $L_0$-SWCCA; (c) and (h) denote estimated $\bm{u}$ and $\bm{v}$ by $L_0$-SCCA; (d) and (i) denote estimated $\bm{u}$ and $\bm{v}$ by PMD.}
\end{figure*}

For simplicity and comparison, we can set true $\bm{u} = [r(1,50),~r(0,50)]^{\rm{T}}$, true $\bm{v} = [r(-1,40),~r(0,40)]^{\rm{T}}$ and true $\bm{w} = [r(1,30),r(0,20)]^{\rm{T}}$ to characterize the patterns of $\bm{X}$ and $\bm{Y}$ (Fig.3(a), (f) and (e)). Similarly, we also apply $L_0$-SWCCA, $L_0$-SCCA \cite{asteris2016simple} and PMD \cite{witten2009penalized} to the synthetic data 2. For comparison, we set parameters $k_u = 50$, $k_v = 40$ and $k_w = 30$ for $L_0$-SWCCA; $k_u = 50$, $k_v = 40$ for $L_0$-SCCA; $c_1 = \frac{50}{100}\sqrt{p}$ and $c_2 = \frac{40}{80}\sqrt{q}$ for PMD.

The true and estimated patterns for $\bm{u}$, $\bm{v}$ and $\bm{w}$ are shown in Fig.3. $L_0$-SWCCA and $L_0$-SCCA are superior to PMD about identifying the latent patterns of canonical vectors $\bm{u}$ and $\bm{v}$ (Fig.3(b), (c), (d), (g), (h) and (i)). However $L_0$-SCCA and PMD fail to remove interference samples. Compared to $L_0$-SCCA and PMD, $L_0$-SWCCA can clearly identify the true non-zero characteristics of samples (Fig.3(e)). Similarly, we also compute the correlation criterion based on the formula (19). We find that $L_0$-SWCCA gets the largest correlation $\rho=0.97$ compared to $L_0$-SCCA with $\rho=0.93$ and PMD with $\rho=0.95$. All results show that our method is more effective to capture the latent patterns of canonical vectors than other ones.

\subsection{Breast cancer data}
We first consider a breast cancer dataset \cite{witten2009penalized,chin2006genomic} consisting of gene expression and DNA copy number variation data across 89 cancer samples. Specifically, the gene expression data $\bm{X}$ and the DNA copy number data $\bm{Y}$ are of size $n \times p$ and  $n \times q$ with $n = 89$, $p = 19672$ and $q = 2149$. We apply SWCCA and related ones to this data to identify a gene set whose expression is strongly correlated with copy number changes of some genomic regions.

In PMD \cite{witten2009penalized}, we set $c_1 = c\sqrt{p}$ and $c_2 = c\sqrt{q}$, where $c \in (0, 1)$ is to approximately control the sparse ratio of canonical vectors. We ensure that the canonical vectors ($\bm{u}$ and $\bm{v}$) extracted by the three methods (PMD, $L_0$-SCCA, and $L_0$-SWCCA) have the same sparsity level for comparison. We first apply PMD in the breast cancer data to obtain two sparse canonical vectors $\bm{u}$ and $\bm{v}$ for each given $c \in (0,1)$. Then, we compute the number of nonzero elements in the above extracted $\bm{u}$ and $\bm{v}$, denoted as $N_u$ and $N_v$. Finally, we set $k_u = N_u$, $k_v = N_v$ in $L_0$-SCCA and $L_0$-SWCCA, and set $k_w =  53 \approx 0.6\times89$ in $L_0$-SWCCA to identify the sample loading $\bm{w}$ with sparse ratio $60\%$.

We adopt two criteria: correlation level defined in formula~(19) and objective value defined in Eq.(5) for comparison. Here we consider different $c$ values (\emph{i.e.}, $0.1,0.2,0.3,0.4,0.5,0.6,0.7$) to  control the different sparse ratio of canonical vectors. We find that, compared to PMD and $L_0$-SCCA, $L_0$-SWCCA does obtain higher correlation level and objective value for all cases (Table 1). Since the `breast cancer data' did not collect any clinical information of patients, it is very difficult to study the specific characteristics of these selected samples. To this end, we also apply our method to another biological data with more detailed clinical information.

{\tabcolsep=2.7pt \footnotesize
\begin{center}
\begin{tabular}{|c|c|c|c|c|c|c|c|}
    \multicolumn{8}{c}{\bf Table 1.~Results on  Correlation level (CL) and Objective}\\

    \multicolumn{8}{c}{\bf value (OV) for different c values.}\\
    \hline
    {\bf CL}   & c=0.1 & c=0.2 & c=0.3 & c=0.4 & c=0.5 & c=0.6 & c=0.7\\
	\hline
    PMD & 0.77 & 0.81 & 0.85 & 0.86 & 0.86 & 0.85 & 0.83\\
	\hline
    $L_0$-SCCA& 0.88 & 0.84 & 0.86 & 0.85 & 0.84 & 0.82 & 0.81\\
	\hline
    $L_0$-SWCCA & 0.99 & 0.98 & 1.00 & 1.00 & 0.99 & 0.97 & 0.99\\
	\hline
     \hline
    {\bf OV}  & c=0.1 & c=0.2 & c=0.3 & c=0.4 & c=0.5 & c=0.6 & c=0.7\\
	\hline
    PMD &253  & 768  & 1390 & 2020 & 2589 & 3056  & 3392\\
	\hline
    $L_0$-SCCA   &371  & 1066 & 1824 & 2467 & 3014 & 3360  & 3520\\
	\hline
    $L_0$-SWCCA &1570 & 3046 & 4960 & 6576 & 7860 & 10375 & 10692\\
	\hline
\end{tabular}
\end{center}}
\subsection{TCGA BLCA data}
Recently, it is a hot topic to study microRNA (miRNA) and gene regulatory relationship from matched miRNA and gene expression data \cite{min2015novel,zhang2011novel}. Here, we apply SWCCA onto the bladder urothelial carcinoma (BLCA) miRNA and gene expression data across 405 patients from TCGA (\url{https://cancergenome.nih.gov/}) to identify a subtype-specific miRNA-gene co-correlation module. To remove some noise miRNAs and genes, we first adapt standard deviation method to extract 200 largest variance  miRNAs and 5000 largest variance genes for further analysis. Finally, we obtain a miRNA expression matrix $\bm{X}\in \mathbb{R}^{405\times200}$, which is standardized for each miRNA, and a gene expression matrix $\bm{Y}\in \mathbb{R}^{405\times5000}$, which is standardized for ecah gene. We apply $L_0$-SWCCA onto BLCA data with $k_u = 10$, $k_v = 200$ and $k_w = 203$ to identify a miRNA set with 10 miRNAs and a gene set with 200 genes and a sample set with 203 patients. We also apply PMD with $c_1=(10/200)\sqrt{p}$, $c_2=(200/5000)\sqrt{q}$ and $L_0$-SCCA with $k_u = 10$ and $k_v = 200$ onto BLCA data for comparison.

\begin{figure}[h]
  \centering
  \includegraphics[width=1\linewidth]{figures/fig6.pdf}
  \caption{Kaplan-Meier survival analysis between the selected patients and the remaining patents based on the $\bm{w}$ estimated by $L_0$-SWCCA. $P$-value is calculated by log-rank test.}
\end{figure}

Similarly, $L_0$-SWCCA obtains the largest correlation level (CL) and objective value (OV) than others ones [(CL, OV): (0.98, 1210) for $L_0$-SWCCA, (0.84, 346) for PMD, (0.86,469) for $L_0$-SCCA], respectively. More importantly, we also analyze characteristics of these selected patients by $L_0$-SWCCA. We find that it is significantly different with respect to patient survival time between the selected 203 patients and the remaining 202 patients with $p$-value $=0.023$ (Fig.4). These results imply that $L_0$-SWCCA can be used to discover BLCA subtype-specific miNRA-gene co-correlation modules.

Furthermore, we also assess whether these identified genes by $L_0$-SWCCA are biologically relevant with BLCA. DAVID (\url{https://david.ncifcrf.gov/}) is used to perform the Gene Ontology (GO) biological processes (BPs) and KEGG pathways enrichment analysis. Several significantly enriched  GO BPs and pathways relating to BLCA are discovered including \emph{GO:0008544:epidermis development} (B-H adjusted $P$-value $=$ 1.1E-12), \emph{hsa00591:Linoleic acid metabolism} (B-H adjusted $p$-value $=$ 4.8E-3), \emph{hsa00590:Arachidonic acid metabolism} (B-H adjusted $p$-value $=$ 3.6E-3) and \emph{hsa00601:Glycosphingolipid biosynthesis-lacto and neolacto series} (B-H adjusted $p$-value $=$ 2.6E-2). Finally, we also examine whether the identified miRNAs by $L_0$-SWCCA are associated with BLCA. Interestingly, in the identified 10 miRNAs by $L_0$-SWCCA, we find that there are six miRNAs (including \emph{hsa-miR-200a-3p, hsa-miR-200b-5p, hsa-miR-200b-3p, hsa-miR-200a-5p, hsa-miR-200c-3p and hsa-miR-200c-5p}) belonging to miR-200 family. Notably, several studies \cite{wiklund2011coordinated,cheng2016mir} have also reported miR-200 family plays key roles in BLCA. All these results imply that the identified miNRA-gene module by $L_0$-SWCCA may help us to find new therapeutic strategy for BLCA.

\section{Extensions}
\subsection{SWCCA with generalized penalties}
We first consider a general regularized SWCCA framework as Eq.(23).
\begin{equation}
  \begin{array}{rl}
    \max_{\bm{u},\bm{v},\bm{w}} & \bm{u}^{\rm{T}}\bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v} \\
                         & - \mathcal{R}_u(\bm{u}) - \mathcal{R}_v(\bm{v})-\mathcal{R}_w(\bm{w})\\
    \mbox{s.t.} & \bm{u}^{\rm{T}}\bm{u} = 1,\bm{v}^{\rm{T}}\bm{v} = 1, \bm{w}^{\rm{T}}\bm{w} = 1
  \end{array}
\end{equation}
where $\mathcal{R}_u(\cdot)$, $\mathcal{R}_v(\cdot)$, $\mathcal{R}_w(\cdot)$ are three regularized functions. For different prior knowledge, we can use different sparsity inducing penalties.

\subsubsection{LASSO regularized SWCCA}
If $\mathcal{R}_u(\bm{u})= \lambda_u\|\bm{u}\|_1$, $\mathcal{R}_v(\bm{v})= \lambda_v\|\bm{v}\|_1$, and $\mathcal{R}_w(\bm{w})= \lambda_w\|\bm{w}\|_1$. We obtain a $L_1$ (Lasso) regularized SWCCA ($L_1$-SWCCA).
Similar to solve Eq.(6), we only need to solve the following problem to solve $L_1$-SWCCA as Eq.(24).
\begin{equation}
  \begin{array}{rl}
    \min_{\bm{u}}&-\bm{u}^{\rm{T}}\bm{z} + \lambda_u\|\bm{u}\|_1\\
    \mbox{s.t.}& \bm{u}^{\rm{T}}\bm{u} =  1
  \end{array}
\end{equation}
where $\bm{z} = \bm{X}^{\rm{T}}\mbox{diag}(\bm{w})\bm{Y}\bm{v}$.
We first replace the constraint $\bm{u}^{\rm{T}}\bm{u} = 1$ with $\bm{u}^{\rm{T}}\bm{u} \leq 1$ and obtain the following the problem as Eq.(25).
\begin{equation}
  \begin{array}{rl}
    \min_{\bm{u}}&-\bm{u}^{\rm{T}}\bm{z} + \lambda_u\|\bm{u}\|_1\\
    \mbox{s.t.}& \bm{u}^{\rm{T}}\bm{u} \leq 1
  \end{array}
\end{equation}
It is easy to see that problem (25) is equivalent to (24). Thus, we can obtain its Lagrangian form as Eq.(26).
\begin{equation}
\mathcal{L}(\bm{u},\lambda_u, \eta_u) = -\bm{u}^{\rm{T}}\bm{z} + \lambda_u\|\bm{u}\|_1 + \eta_u(\bm{u}^{\rm{T}}\bm{u}-1)
\end{equation}
Thus, we can use a coordinate descent method to minimize Eq.(26) and obtain the following update rule of $\bm{u}$ as Eq.(27).
\begin{equation}
\bm{u} = \frac{\mathcal{S}_{\lambda_u}(\bm{z})}{\|\mathcal{S}_{\lambda_u}(\bm{z})\|_2}
\end{equation}
where $\mathcal{S}_{\lambda_u}(\cdot)$ is a soft thresholding operator and $\mathcal{S}_{\lambda_u}(z_i)=\mbox{sign}(|z_i|-\lambda_u)_+$. Based on the above, an alternating iterative strategy can be used to solve $L_1$-SWCCA.
\subsubsection{Group LASSO regularized SWCCA}
If $\mathcal{R}_u(\bm{u})= \lambda_u \sum_l\|\bm{u}^{(l)}\|_2$, $\mathcal{R}_v(\bm{v})= \lambda_v \sum_l\|\bm{v}^{(l)}\|_2$ and $\mathcal{R}_w(\bm{w})=  \lambda_w \sum_l\|\bm{w}^{(l)}\|_2$. Problem (24) reduces to $L_{2,1}$-regularized SWCCA ($L_{2,1}$-SWCCA). Similarly, we should solve the following projection problem as Eq.(28).
\begin{equation}
  \begin{array}{rl}
    \min_{\bm{u}}&-\bm{u}^{\rm{T}}\bm{z} + \lambda_u\sum\limits_l\|\bm{u}^{(l)}\|_2\\
    \mbox{s.t.}& \bm{u}^{\rm{T}}\bm{u} \leq 1
  \end{array}
\end{equation}
Thus, we obtain its Lagrangian form as Eq.(29).
\begin{equation}
\mathcal{L}(\bm{u},\lambda_u, \eta_u) = -\bm{u}^{\rm{T}}\bm{z} + \lambda_u\sum_l\|\bm{u}^{(l)}\|_2 + \eta_u(\bm{u}^{\rm{T}}\bm{u}-1)
\end{equation}
where $\bm{u}^{(l)}$ is the $l$th group of $\bm{u}$. We adopt a block-coordinate descent method \cite{tseng2001convergence} to solve it and obtain the learning rule of $\bm{u}^{(l)}$ ($l = 1,\cdots, L$) as Eq.(30).
\begin{eqnarray}
  \bm{u}^{(l)}=
  \begin{cases}
    \frac{1}{2\eta_u }\bm{z}^{(l)} (1- \frac{\lambda_u}{\|\bm{z}^{(l)}\|_2} ), &\mbox{if}~\|\bm{z}^{(l)}\|_2 > \lambda_u,\cr
    \bm{0}, &\mbox{otherwise}.
  \end{cases}
\end{eqnarray}
By cyclically applying the above updates, we can minimize Eq.(29). Thus, an alternating iterative strategy can be used to solve $L_{2,1}$-SWCCA.

\subsection{Multi-view sparse weighted CCA}
In various scientific fields, multiple view data (more than two views) can be available from multiple sources or diverse feature subsets. For example, multiple high-throughput molecular profiling data by omics technologies can be produced for the same individuals in bioinformatics \cite{li2012identifying,min2015novel,sun2015multi}. Integrating these data together can significantly increase the power of pattern discovery and individual classification.

Here we extend SWCCA to Multi-view SWCCA (MSWCCA) model for multi-view data analysis (Fig.5) as follows:
\begin{eqnarray*}
  \begin{array}{rl}
    \max_{\bm{u}_i, \bm{w}} & \bm{w}^{\rm{T}}\big[\bigodot\limits_{i=1}^M(\bm{X}_i\bm{u}_i)\big] - \sum\limits_{i=1}^M\mathcal{R}_{\bm{u}_i}(\bm{u}_i)-\mathcal{R}_w(\bm{w})\\
    \mbox{s.t.} & \bm{w}^{\rm{T}}\bm{w} = 1, \bm{u}_i^{\rm{T}}\bm{u}_i = 1~\mbox{for}~i = 1,\cdots, M
  \end{array}
\end{eqnarray*}
where $\bigodot_{i=1}^M(\bm{X}_i\bm{u}_i) = (\bm{X}_1\bm{u}_1)\odot(\bm{X}_2\bm{u}_2)\cdots\odot(\bm{X}_M\bm{u}_M)$. When $M=2$, we can see that $\bm{w}^{\rm{T}}[(\bm{X}_1\bm{u}_1)\odot(\bm{X}_2\bm{u}_2)] = \bm{u}_1^{\rm{T}}\bm{X}_1^{\rm{T}}\mbox{diag}(\bm{w})\bm{X}_2\bm{u}_1$ and it reduces to SWCCA. So we can solve MSWCCA in a similar manner with SWCCA.

\begin{figure}[h]
  \centering
  \includegraphics[width=1\linewidth]{figures/fig7.pdf}
  \caption{Illustration of the multi-view sparse weighted CCA designed for integrating multiple data matrices.}
\end{figure}
\section{Conclusion}
In this paper, we propose a sparse weighted CCA framework. Compared to SCCA, SWCCA can reveal that the selected variables are only strongly related to a subset of samples. We develop an efficient alternating iterative algorithm to solve the $L_0$-regularized SWCCA. Our tests using both simulation and biological data show that SWCCA can obtain more reasonable patterns compared to the typical SCCA. Moreover, the key idea of SWCCA is easy to be adapted by other penalties like LASSO and Group LASSO. Lastly, we extend SWCCA to MSWCCA for multi-view situation with multiple data matrices.

\section*{Acknowledgment}
Shihua Zhang and Juan Liu are the corresponding authors of this paper. Wenwen Min would like to thank the support of National Center for Mathematics and Interdisciplinary Sciences, Academy of Mathematics and Systems Science, CAS during his visit.

\small{
\bibliographystyle{named}
\balance