\section{Introduction}\label{sec:intro}

A detailed understanding of a potentially unknown environment plays a fundamental role in mobile robotic applications. Different robots and environments come along with varying requirements for the map building process in terms of accuracy, efficiency and usablility. Common SLAM methods, which attempt to map an environment while simultaneously localizating the robot in it, usually have to find a balance between these properties. Due to the complexity of the task, it is still very challenging to perform SLAM while maintaining a dense map representation. Compared to commonly used sparse map representation, \textit{occupancy grids} have many advantages as they integrate all available information into a single representation which is easy to understand for an operator and also allows for efficient queries.

2D projections of such dense reprenstations have been used extensively for mobile robot navigation tasks. As robots become more agile, scenes more complex and sensors more capable, it is also desireable to adopt these structures for the third dimension. For ground vehicles, so called 2.5D elevation maps as suggested by Herbert et.~al~\cite{herbertmapping} may be sufficient. However, for agile robots with complex kinematics such as legged robots or UAVs, a full 3D representation of the environment is essential.

\begin{figure}[tb]
  \centering
  \includegraphics[width=0.7\columnwidth]{img/vdb_iosb.png}
  \caption{Map of the Fraunhofer IOSB site using the mapping approach. Preprocessing using a custom factor graph approach described in \cite{iosb_mapping}.}
  \label{fig:mapping_iosb}
  \vspace{-0.5cm}
\end{figure}

To store the 3D map memory efficient and with certain complexity guarantees for random access operations, different datastructures have been suggested. The most prominent example for such a 3D map representaion is the OctoMap~\cite{octomap} framework by Hornung et al., working on octrees as hierarchical tree structure. It allows for a memory effiecient storing through efficient pruning and propagating leaf states to higher levels of the tree. OctoMap has been considered state-of-the-art for a long time but as sensors are achieving higher rates with millions of points per second, the \textit{insert} operation of OctoMap is not able to achieve real-time performance. As the octree has a fixed layout, it is also difficult to later increase the volume without performance overhead. One of the most popular frameworks for global consistent mapping is VoxBlox by Oleynikova et al.~\cite{voxblox}. They incrementally build a \textit{Truncated Signed Distance Map} (TSDF)~\cite{sdf} instead of an occupancy voxel grid and reach almost real-time performance on a single core implementation using a hashmap representation instead of a tree as fundamental datastructure.

OpenVDB is a modern framework developed in a computer graphic context. Similar to OctoMap it is based on a hieracical structure but it comes with certain accelerators to support almost constant insertion and read procedures using a B+ tree like structure. The main reason for the improved performance is an advanced indexing and caching system. We therefore leverage OpenVDB as underlying datastructure for our map building approach and present its capabilities in further detail in the upcoming Section~\ref{sec:pipeline}. OpenVDB as flexible backbone allows not only fast insertions but also supports efficient raycasting step samplers and a virtually infinite map size.

Only few works utilize the VDB datastructure as occupancy representation so far. Our work was originally based on~\cite{vdb_fzi} by Besselmann et al. and can be considered a successor with a revised update scheme and optimized insertion procedure. They bring up the idea to integrate data into a temporary grid first to cope with discretization ambiguities which arise when raycasting new data.  In~\cite{vdb-edt}, Zhu et al. present a full framework for occupancy and distance mapping, which also uses a raycast based insertion scheme in order to create the occupancy map. They put the focus on the \textit{Euclidean Distance Transform} step and disregard the map integration itsself. Macenski et~al.~\cite{spatiotemporal} built a spatio-temporal voxel layer on top of OpenVDB. They focus on local dynamic maps and therefore use a sensor frustrum based visibility check instead of raycasting as integration scheme.

In our work we further push the limits of the underlying OpenVDB structure by supporting a flexible multithreaded raycasting insertion scheme into the map supported by additional ray-level hashing to avoid unnecessary operations. Fast merging operations of single \textit{bit-grids} allow for a minimal lock time for modifying the global occupancy grid. Different subsample strategies can be selected to allow for a dynamically adjustable tradeoff between map accuracy and efficiency.

The main contribution of the work can be summarized as follows:
\begin{itemize}
  \item We present a real-time capable and multithreaded dense mapping approach for efficiently creating occupancy maps based on the VDB data structure.
  \item We introduce several optimizations in the integration scheme allowing for a user-definable tradeoff between map accuracy and efficiency.
  \item We conduct benchmarks on different operations and test the whole pipeline in simulation and real environments.
  \item We open-source the codebase and a corresponding wrapper for the Robot Operating System II (ROS II~\cite{ros2}) which enables fast prototyping for mobile robotic applications.
\end{itemize}

The remainder of this work is structured as follows. Section~\ref{sec:pipeline} formalizes the problem of mapping and introduces OpenVDB as underlying datastructure. We also give a detailed overview on the insertion scheme and introduce various optimizations leading to improved performance. In Section~\ref{sec:evaluation} we carry out different experiments to verify the effect of the introduced optimizations. We summarize and conclude the work and discuss potential future improvements in Section~\ref{sec:discussion}.

\section{Mapping Pipeline}\label{sec:pipeline}

In this section, we first formalize the problem before discussing the proposed insertion scheme.

\subsection{Problem Statement}

The main goal of occupancy mapping is to create a map \(\mathcal{M}_{\text{occ}}\) which stores the occupancy probability \(p(x_{i}|s_{1:t}, z_{1:t})\) for each cell \(x_{i} \in \mathcal{M}_{\text{occ}}\) given some sensor measurements \(s_{1:t}\) and the corresponding robot poses \(z_{1:t}\), where \(1{:}t\) denotes the sequence from the start up to time \(t\). We consider a cell to be an obstacle if \(p(x_{i}|s_{1:t},z_{1:t})\) exceeds a certain threshold \( \phi_{\text{occ}}\) and free if it falls below the threshold \(\phi_{\text{free}}\). Note that being marked as \textit{free} is different from not being observed yet.

\subsection{OpenVDB}

\begin{figure}[b]
  \centering
  \includegraphics[width=1\columnwidth]{img/vdb_structure.png}
  \caption{Illustration of the underlying VDB datastructure. Image is adopted from the original publication \cite{vdb}. The height of the tree is typically 4 with one root node (gray) and two internal layers (green, orange) as well as leaf nodes, which store the actual tile values (blue). All nodes in lower levels have a branching factor equal to a power of two. Root and internal nodes store pointers to their respective child nodes. An \textit{active} bitmask for each nodes encodes if subsequent tiles are active or not (gray values).}
  \label{fig:vdb}
\end{figure}

The OpenVDB framework has been introduced by Museth et~al. in 2013 \cite{vdb}. Originally it was designed for computer graphic applications such as rendering animations of complex mesh structures and time-varying sparse volumes such as clouds. Since then, it has been widely adopted to different applications as it allows for flexible modifications to its core structure. At its heart it levereges a B+ tree~\cite{bplustree} variant as main datastructure. This structure is supported by hieracically organized caches to faciliate fast access to inner tree nodes. Such a datastructure is ideal to store sparse voxelized environment representations. The discretization of the space can be adjusted by choosing the size of the leaf nodes accordingly. Other adjustable parameters are the tree depth and branching factors which can further improve the memory footprint depending on the sparsity of the environment. Typical branching factors for the VDB datastructures are very large compared to the octree branching factors, which are typically two in each spatial dimension and thus lead to less shallow trees.
The schematics of the underlying tree structure is depicted in Figure~\ref{fig:vdb}. The fixed height of the tree makes it possible to implement \textit{insert} and \textit{read} operations in constant time on average~\cite{vdb}. It also allows the framework to efficiently utilize typical cache architectures on modern CPUs to further speed up read operations for tiles with spatial proximity. Access to the tree's tile values is implemented via a virtually infinite \textit{index space}, which can be accessed by signed \(32\)-bit integer coordinates, allowing the map to grow in each direction without additional overhead. As noted in Figure~\ref{fig:vdb}, tree nodes additionally store bitsets indicating if subsequent nodes are active or not. This allows for a fast traversal of a sparse volume without the need to visit tile values explicitly. The features of the tree structure are described in further detail in the original work~\cite{vdb}. Moreover, VDB allows for efficient raycasting operations using optimized index space iterators. Therefore, we utilize a raycast based sensor insertion scheme which is described in further detail in the following section.

\subsection{Map Updates}
The proposed structure is not tied to a specific sensor. Typical data comes from a LiDAR sensor, mimicing the raycast operation which is also performed to create the obstacle map. Cells store their occupancy probability in order to reflect not only occupied but also free space. To represent time-dependent updates we faciliate the very commonly used log-odds based update scheme initially formulated by Moravec and Elfes in~\cite{logodds}. Essentially we calculate
\begin{equation}
  \mathcal{M}_{\text{occ}}(x_{i}|s_{1:t})= \mathcal{M}_{\text{occ}}(x_{i}|s_{1:t-1}) + \log\left[\frac{p(x_{i}|s_{1:t})}{1-p(x_{i}|s_{1:t})}\right]
  \label{eq:update}
\end{equation}
in each update step and for each \(x_{i} \in \mathcal{M}_{\text{occ}} \).

Algorithm~\ref{alg:map} gives an abstracted overview on the insertion process. Essentially we do parallel raycasting in coaligned temporary grids, which are merged together in a later step. We first create a coaligned map \(\mathcal{M}_{\text{agg}}\) (line 2), which we later use to aggregate the temporary maps. We then divide the incoming points into different chunks which can be processed in parallel (line 5). Points of each chunk are inserted by calculating their respective end position in world coordinates (line 9) and marching along the ray using OpenVDBs digital differential analyzer (DDA) implementation (lines 19-22) and marking all visited voxels as \textit{active}. The only place where we need to lock the threads is during the merge operation in lines 23-24. This can be done efficiently as we simply XOR the boolean grids together.
As OpenVDB stores the active state of nodes in a fast accessible bitmask, we can now efficiently iterate over all \textit{active} values in our aggregated map \(\mathcal{M}_{\text{agg}}\) (line 25). We increase or decrease the occupancy value following Equation~\ref{eq:update} (lines 27 to 29). If a voxel exceeds or falls below a certain threshold it will be marked as occupied or unoccupied.

We suggest and implement two runtime optimizations, which are roughly based on similar ideas used in Voxblox~\cite{voxblox}, namely a \textit{subsampling} and a \textit{bundling} optimiztation. \textit{Subsampling} (cf.  Figure~\ref{fig:subsample}) uses an additional map \(\mathcal{M}_{\text{sub}}\) which increases the resolution \(\mathcal{M}_{\text{occ}}\) by a \textit{subsampling-factor} \(\delta_{\text{sub}}\). Typically, \(\delta_{\text{sub}}\) is restricted to powers of \(2\), we use \(4\) in most setups. In addition we use a Hashmap \(\mathcal{H}_{\text{sub}}\) (line 3) storing for each cell in \(\mathcal{M}_{\text{sub}}\), if it has already been visited in the current integration step. If this is the case, all subsequent integrations are skipped (lines 12 and 13). This optimization comes with the cost of inaccurate details but can save a lot of integration steps especially in dense environments with large voxel sizes, where a single voxel is hit multiple times.

\let\@latex@error\@gobble
\begin{algorithm}[H]
  \caption{Map Update Scheme}
  \label{alg:map}
  \setstretch{1.30}
  \SetKwFunction{MapUpdate}{MapUpdate}
  \KwIn{Pointcloud $\mathcal{P}$, Sensor origin $\mathbf{o}$, Number of Chunks $c$, Global Occupancy Map $\mathcal{M}_{\text{occ}}$}
  $\mathcal{M}_{\text{sub}} \leftarrow \text{coalign}(\mathcal{M})$ \tcp{Aligned Subsampling Map}
  $\mathcal{M}_{\text{agg}} \leftarrow \text{coalign}(\mathcal{M})$ \tcp{Aligned Aggregation Map}
  Hashmap$\left<\text{coord } x,\text{bool hit}\right> \mathcal{H}_{\text{sub}} \leftarrow [\;]$\;
  Hashmap$\left<\text{int count}, \text{coord } x, \text{bool maxray}\right> \mathcal{H}_{\text{bun}} \leftarrow [\;]$ \;
  $\mathcal{P}_{i} \leftarrow \text{Equal Chunks of } \mathcal{P} \text{ for } i=1..c$\;
  \ParallelForEach{$\mathcal{P}_{i} \in \mathcal{P}$}{
    $\mathcal{M}_{\text{temp}} \leftarrow \text{coalign}(\mathcal{M})$ \tcp{Aligned Temporary Map}
    \ForEach{$\mathbf{p} \in \mathcal{P}_{i}$} {
      $\mathbf{r}_{\text{end}}=\mathbf{o} + (\mathbf{p} - \mathbf{o}) \text{ in } \mathcal{M}_{\text{agg}}$ \;
      $\mathbf{r}_{\text{end\_sub}}=\mathbf{o} + (\mathbf{p} - \mathbf{o}) \text{ in } \mathcal{M}_{\text{sub}}$ \;
      is\_maxray $\leftarrow $ check for max-length ray \;
     \If{$\mathcal{H}_{\text{sub}}[\mathbf{r}_{\text{end\_sub}}]$}{
        \Continue\;
      }
      \eIf{$\mathcal{H}_{\text{bun}}[\mathbf{r}_{\text{end}}].\text{count}>\text{thresh}$}{
        $(\text{count}, \overline{\mathbf{r}_{\text{end}}}, \text{maxray}) = \mathcal{H}_{\text{bun}}[\mathbf{r}_{\text{end}}]$ \;
      }{
        $\mathcal{H}_{\text{bun}}[\mathbf{r}_{\text{end}}]+=(1,\mathbf{r}_{\text{dir}},\text{is\_maxray})$ \;
        \Continue\;
      }
      \Do{$\mathbf{r}_{\text{dda}} \neq \mathbf{r}_{\text{end}}$}{
        $\mathcal{M}_{\text{temp}}[\mathbf{r}_{\text{dda}}].\text{active} = \text{true}$\;
        $\mathbf{r}_{\text{dda}}++$
      }

    }
    \With{MapLock$(\mathcal{M}_{\text{temp}})$}{
      $\mathcal{M}_{\text{agg}} \;|=\; \mathcal{M}_{\text{temp}}$ \;
    }
    }
  \ForEach{active value $x \in \mathcal{M}_{\text{agg}}$}{
    \eIf{$x$}{
      increase occupancy on $\mathcal{M}_{\text{occ}}[x]$ following Eq.~\ref{eq:update}\;

    }{
      reduce occupancy on $\mathcal{M}_{\text{occ}}[x]$ following Eq.~\ref{eq:update}\;

    }
  }
  \KwOut{Updated $\mathcal{M}_{\text{occ}}$}

\end{algorithm}

\begin{figure*}[tb]
  \centering
  \begin{subfigure}[b]{0.35\textwidth}
    \captionsetup{margin={2.6cm,0cm}}
    \centering
    \includegraphics[page=2, width=\textwidth]{img/mapping.pdf}
    \caption{Default Raycasting}
    \label{fig:raycasting}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.2\textwidth}
    \centering
    \includegraphics[page=5, width=\textwidth]{img/mapping.pdf}
    \caption{Parallelization}
    \label{fig:parallel}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.2\textwidth}
    \centering
    \includegraphics[page=4, width=\textwidth]{img/mapping.pdf}
    \caption{Subsample Optimization}
    \label{fig:subsample}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.205\textwidth}
    \centering
    \includegraphics[page=3, width=\textwidth]{img/mapping.pdf}
    \caption{Bundle Optimization}
    \label{fig:bundle}
  \end{subfigure}
  \caption{Scheme of different optimizations on the insertion procedure. In (a), the standard raycasting process is visualized. When enough rays hit a voxel so that \(\phi_{\text{occ}}\) is exceeded, this voxel will be marked as occupied. Figure (b) visualizes the process of dividing rays into different chunks which can then be processed in a multithreaded fashion. In (c) the target space is further subsampled with a customizable subsampling map \(\mathcal{M}_{\text{sub}}\). Rays which hit an already marked subsampled cell (the blue ray) are skipped. In (d) multiple rays are aggregated and averaged if they hit the same target cell. }
  \label{fig:mapping_optimizations}
\end{figure*}
The \textit{bundling} optimization on the other hand bundles multiple rays together. Again, we use a hashmap \(\mathcal{H}_{\text{bun}}\) where we insert incoming ray end points without actually integrating the rays (lines 17 and 18). If a certain threshold is exceeded, we integrate the whole bundle targeting the end cell \(\mathbf{r}_{\text{end}}\) at once. In the map, we additionally store the original end points and if the ray reached its maximum length. During the integration of the bundle this information is used to average the final end point  \(\overline{\mathbf{r}_{\text{end}}}\). Again, this optimization favors environments with a lot of redundant integrations. Figure~\ref{fig:bundle} indicates that only one orange bundle is inserted into the map, even if multiple rays hit the cell (cf. Figure~\ref{fig:raycasting}). Both optimizations can be enabled or disabled individually or together. While enabling the optimizations leads to a deliberately impaired map accuracy, it can be useful as more sensor data can be integrated overall due to the additionally gained performance.
It is worth noticing that it is not necessary to deal with discretization ambiguities introduced by sequential raycasting presented in~\cite{vdb_fzi} as we use the same two step approach as in~\cite{vdb_fzi}: First we activate all visited voxels in a seperate aggregation map  \(\mathcal{M}_{\text{agg}}\) before  we integrate it into the global map \(\mathcal{M}_{\text{occ}}\).

\section{Evaluation}\label{sec:evaluation}

We will now present the results of experiments which we conducted to measure the performance of our proposed methods under different conditions. We first compare different iterations of our method in a set of ablation studies to measure the effect of different optimizations. In a next step we compare the method on typical benchmark sets before we finally conduct some real world experiments by capturing outdoor and indoor scenes of our lab. All experiments are performed using a machine equipped with a 6-core Intel\copyright Core\texttrademark i7-10850H and 32~GB of memory. As hardware platforms to carry out our experiments we use a BostonDynamics Spot equipped with an Ouster OS0-64 LiDAR for outdoor experiments as well as a custom UAV platform with a solid-state LiDAR for indoor experiments (see Figure~\ref{fig:robots} for details).

\begin{figure}[tb]
  \centering
  \begin{subfigure}[b]{0.45\columnwidth}
    \captionsetup{format=hang}
    \centering
    \includegraphics[width=\textwidth]{img/spot.png}
    \caption{BostonDynamics Spot with additional sensors}
    \label{fig:spot}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.45\columnwidth}
    \captionsetup{format=hang}
    \centering
    \includegraphics[width=\textwidth]{img/drone.jpg}
    \caption{Custom UAV platform for indoor usage}
    \label{fig:drone}
  \end{subfigure}
  \caption{The hardware setup used for indoor and outdoor experiments. Both robots are equipped with a solid-state LiDAR (Realsense L515) and provide a localization coming from a multisensor-fusion approach.}
  \label{fig:robots}
  \vspace*{-0.3cm}
\end{figure}

\subsection{Ablation Studies}
\begin{table}[tb]
  \centering
  \caption{Runtime of different variants with 0.1m map resolution and $n$ insertions each. \textit{Random} refers to randomly sampled points in a radius which is $1.2 \cdot \text{ray length}$ as depicted in (b). The \textit{structured} environment (a) refers to a sampling where $(x,y)$ coordinates are sampled randomly but $z$ coordinates are restricted to a small width of $1$m simulating a wall structure.~(*) VDB-BUN and VDB-SUB approaches are only included as rough estimate and do not compare to the other methods, as some rays are not casted, when using bundle or subsampling optimizations. The parallel version VDB-PAR runs on 12 threads and VDB-FMAP is the improved and parallel variant restricted to a single core and with disabled optimizations while VDB-MAP is the VDB based method described in \cite{vdb_fzi}.}
  \label{tab:runtime}
  \begin{tabular}{@{}cl@{}rrrr@{}}
  \toprule
    \multirow{3}{*}{$n$} & \multirow{3}{*}{method} & \multicolumn{4}{c}{runtime [ms]} \\
            & & \multicolumn{2}{c}{ray length 6m} & \multicolumn{2}{c}{ray length 60m} \\
                          & & structure & random & structure & random \\ \midrule
  \multirow{6}{*}{\rotatebox[origin=c]{90}{50\,000}}& OctoMap~\cite{octomap} & 110 & 747 & 11676 & 42\,593 \\
                         & VDB-EDT~\cite{vdb-edt}     & 102 & 106 & 3\,005 & 3\,973\\
                            & VDB-MAP~\cite{vdb_fzi}     & 55 & 74 & $\mathbf{863}$ & $\mathbf{1\,984}$ \\
                            & VDB-FMAP      & 54 & 71 & 906 & 2\,075 \\
                            & VDB-SUB*   & 56 & 75  & 923 & 2\,149 \\
                            & VDB-BUN*      & 31 & 40 & 491 & 1\,083 \\
                            & VDB-PAR    & $\mathbf{10}$ & $\mathbf{20}$ & 1308 & 5\,581 \\ \midrule
    \multirow{6}{*}{\rotatebox[origin=c]{90}{500\,000}} & OctoMap~\cite{octomap} & 796 & 2\,189 & 44\,072 & 223\,672 \\
                         & VDB-EDT~\cite{vdb-edt}      & 957 & 987 & 22\,010 & 28\,684 \\
                          & VDB-MAP~\cite{vdb_fzi}     & 547 & 709  & 6\,042 & 11\,364 \\
                          & VDB-FMAP     & 547 & 671  & 5\,779 & 10\,718 \\
                          & VDB-SUB*  & 288 & 583  & 5\,011 & 10\,998 \\
                          & VDB-BUN*     & 261 & 330  & 2\,202 & 4\,764 \\
                          & VDB-PAR   & $\mathbf{54}$ & $\mathbf{74}$ & $\mathbf{1\,979}$ & $\mathbf{10\,339}$ \\ \bottomrule \\
  \end{tabular}
  \begin{subfigure}[b]{0.23\textwidth}
    \includegraphics[width=\textwidth]{img/structured_env.png}
    \caption{Structured Raycasting}
    \label{fig:structure_raycast}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.23\textwidth}
    \includegraphics[width=\textwidth]{img/random_structure.png}
    \caption{Random Raycasting}
    \label{fig:random_raycast}
  \end{subfigure}
  \vspace*{-0.5cm}
  \end{table}

We first compare different versions of our method with each other and against baselines from other works. We use random insertions of pointclouds with different sizes as a baseline setting. All experimental results are averaged over 5 runs. The results are presented in Table~\ref{tab:runtime}. Even the basis variant using OpenVDB is able to outperform OctoMap by a factor of 2, in multiple settings this advantage grows up to a factor of 30. Interestingly, the parallel integration scheme \textit{VDB-PAR} achieves almost linear speedup for small ray lengths. Figure~\ref{fig:eval_threads} gives further insight into that evaluation. The reason for the performance drop for higher ray lengths is the increasing amount of time required for the \textit{merge} operations as insertion performance stays constant for increasing submap sizes while the merge workload grows cubic. This insight can be derived from Figure~\ref{fig:eval_length}, where the different steps of an integration procedure are measured. Consequently, the best parallelization speedup can be achieved in low-range settings with a lot of points to be integrated. This exactly matches the domain of indoor mapping scenarios using high-resolution solid-state LiDARs as we will show in the next sectione A detailed evaluation of different ray lengths is given in Figure~\ref{fig:eval_full}. OctoMap and the VDB-EDT approach from~\cite{vdb-edt} are outperformed in low range (below 30m) scenarios by almost a magnitude.

The \textit{bundling} optimization VDB-BUN guarantees to save runtime, as the first ray to a specific voxel is skipped in every case. This approximately halves the runtime over all settings. The \textit{subsampling} optimization VDB-SUB on the other hand only applies when many points reside in a small volume. Consequently it saves the most runtime in a setting with many points in a \textit{structured} environment, whereas it is not faster or even slower for different settings.

\begin{table}[bt]
  \caption{Benchmarks on the \textit{cow-and-lady} dataset. \textit{\#Points} denote the amount of processed points over all frames. This varies due to different processing speeds and different temporal alignments bewteen poses and pointcloud. \textit{Time} measures the total integration time and \textit{\#Occupied Voxels} counts the number of occupied voxels after the integration procedure.}
  \label{tab:datasets}
  \renewcommand{\arraystretch}{1.1}
  \centering
  \begin{tabular}{@{}lrrr}
    \toprule
    Name  & \#Points & \makecell{\#Occupied\\ Voxels} & \makecell{Time per \\ frame [ms]} \\ \midrule
    OctoMap~\cite{octomap} &  $0.463 \cdot 10^{9}$ & $0.332 \cdot 10^{6}$ & 388 \\
    VDB-EDT~\cite{vdb-edt} & $0.557 \cdot 10^{9}$ & $0.567 \cdot 10^{6}$ & 357 \\
    VDB-MAP~\cite{vdb_fzi} & $0.518 \cdot 10^{9}$ & $0.536 \cdot 10^{6}$ & 263 \\
    VDB-FMAP & $0.513 \cdot 10^{9}$ & $0.523 \cdot 10^{6}$ & 282 \\
    VDB-BUN & $0.522 \cdot 10^{9}$ & $0.316 \cdot 10^{6}$ & 94 \\
    VDB-SUB & $0.521 \cdot 10^{9}$ & $0.510 \cdot 10^{6}$ & 170 \\
    VDB-PAR & $0.526 \cdot 10^{9}$ & $0.530 \cdot 10^{6}$ & $\mathbf{47}$ \\ \bottomrule
  \end{tabular}
\end{table}
\begin{figure}[tb]
  \centering
  \begin{subfigure}[b]{0.23\textwidth}
    \centering
    \includegraphics[width=\textwidth]{img/sup_rviz.png}
    \caption{VDB-FMAP \((100\%)\)}
    \label{fig:gaz_raycasting}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.23\textwidth}
    \centering
    \includegraphics[width=\textwidth]{img/bun_rviz_ann.png}
    \caption{VDB-BUN \((23.0\%)\)}
    \label{fig:gaz_bundle}
  \end{subfigure}\\[0.3cm]
  \begin{subfigure}[b]{0.23\textwidth}
    \centering
    \includegraphics[width=\textwidth]{img/par_rviz_ann.png}
    \caption{VDB-SUB \((39.7\%)\)}
    \label{fig:gaz_subsample}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{0.23\textwidth}
    \centering
    \includegraphics[width=\textwidth]{img/sim1.png}
    \caption{Indoor lab map \((100\%)\)}
    \label{fig:sim_hall}
  \end{subfigure}
  \caption{Qualitative visualization of different mapping optimization effects on the \textit{cow-and-lady} dataset in (a)-(c) as well as the lab environment captured by an indoor UAV (cf.~\ref{fig:drone}) in (d). The percentage denotes how many of the original points are casted as rays using the respective optimization.}
  \label{fig:mapping_sim}
\end{figure}

\input{img/plots.tex}

\subsection{Benchmark and Real Datasets}
We evaluate the presented methods on the indoor \textit{cow-and-lady} dataset released as part of VoxBlox~\cite{voxblox}. It consists of 2831 depth frames captured by a Microsoft Kinect I as well as corresponding pose data. The results in Table~\ref{tab:datasets} show that VDB based methods outperform previous methods in terms of simple integration performance. For the \textit{bundling} optimization VDB-BUN, the resulting amount of occupied voxels is only half compared to the other variants. This indicates that the map quality is reduced. On the other hand, the \textit{subsampling} strategy VDB-SUB only comes along with a negliglible reduction of occupied voxels. This shows that mostly redundant rays are omitted in the integration procedure while it is almost able to halve the runtime. Again, the parallel version outperforms all other versions without any reduction in accuracy.

Figure~\ref{fig:mapping_sim} examplarily shows an excerpt of the \textit{cow-and-lady} dataset after 500 frames for different integration strategies. There are only few details (marked in red) of quality loss for saving more than \(60\%\) of the integration steps using the subsampling optimization and almost 80\% using the bundled integration. Figures~\ref{fig:mapping_iosb} and~\ref{fig:sim_hall} show that our mapping procedure is capable of producing high quality maps of outdoor and indoor environments using a legged robot or a drone, respectively. Even without preprocessing the poses using a factor graph~\cite{Dellaert17fnt}, the resulting map quality is satisfactory, if the pose drift is not too large (cf.~\ref{fig:sim_hall}).

\section{Conclusion and Discussion}\label{sec:discussion}

We presented an efficient map building approach for mobile robots, especially suitable for agile robots in dynamic environments. The experimental evaluation demonstrates the effectiveness of the approach particularly in indoor environments with low sensor ranges, where the approach outperforms current solutions. The parallel fusion of different maps allows not only fast but also flexible integration of new sensor data into the map. This leaves room for additional improvements such as an extension with dynamic resolution adaption. Maintaining a real-time distance map is also a valuable extension, which could be implemented efficiently using the VDB datastructure. In order to reduce global inconsistencies coming from drifts in the localization, one could couple the VDB representation with a factor graph backend. This has been tested in a prototyped fashion as demonstrated in Figure~\ref{fig:mapping_iosb} but can potentially be improved by a tighter coupling.

\bibliographystyle{IEEEtran}


\section{Proof of the First Zonklar Equation}

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\input{./section/6_Bibliography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/alessandro_betti.jpg}}]{Alessandro Betti}
	Dr. Alessandro Betti received the M.S. degree in Physics and the Ph.D. degree in Electronics Engineering from the University of Pisa, Italy, in 2007 and 2011, respectively. His main field of research was modeling of noise and transport in quasi-one dimensional devices. His work has been published in 10 papers in peer-reviewed journals in the field of solid state electronics and condensed matter physics and in 16 conference papers, presenting for 3 straight years his research at the top International Conference IEEE in electron devices, the International Electron Device Meeting in USA. In September 2015 he joined the company i-EM in Livorno, where he currently works as a Senior Data Scientist developing power generation forecasting, predictive maintenance and Deep Learning models, as well as solutions in the electrical mobility fields and managing a Data Science Team.
\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/Emanuele_Crisostomi}}]{Emanuele Crisostomi}(M'12-SM'16) received the B.Sc. degree in computer science engineering, the M.Sc. degree in automatic control, and the Ph.D. degree in automatics, robotics, and bioengineering from the University of Pisa, Pisa, Italy, in 2002, 2005, and 2009, respectively. He is currently an Associate Professor of electrical engineering in the Department of Energy, Systems, Territory and Constructions Engineering, University of Pisa. He has authored tens of publications in top refereed journals and he is a co-author of the recently published book on ``Electric and Plug-in Vehicle Networks: Optimization and Control'' (CRC Press, Taylor and Francis Group, 2017). His research interests include control and optimization of large scale systems, with applications to smart grids and green mobility networks.

\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/gianluca_paolinelli.jpg}}]{Gianluca Paolinelli}

Gianluca Paolinelli received the bachelor and master degree in electrical engineering from the University of Pisa, Pisa, Italy, in 2014 and 2018 respectively. His research interests included big data analysis and computational intelligence applied in on-line monitoring and diagnostics.
Currently, he is an electrical software engineer focused in the development of electric and hybrid power-train controls for Pure Power Control S.r.l.

\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/antonio_piazzi.jpg}}]{Antonio Piazzi}
	Antonio Piazzi received the M.Sc. degree in electrical engineering from the University of Pisa, Pisa, Italy, 2013. Electrical engineer at i-EM since 2014, he gained his professional experience in the field of renewable energies. His research interests include machine learning and statistical data analysis, with main applications on modeling and monitoring the behaviour of renewable power plants. Currently, he is working on big data analysis applied on hydro power plants signals.
\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/FabrizioRuffini_Linkedin}}]{Fabrizio Ruffini}
	Fabrizio Ruffini received the Ph.D. degree in Experimental Physics from University of Siena, Siena, Italy, in 2013. His research-activity is centered on data analysis, with particular interest in multidimensional statistical analysis. During his research activities, he was at the Fermi National Accelerator Laboratory (Fermilab), Chicago, USA, and at the European Organization for Nuclear Research (CERN), Geneva, Switzerland. Since 2013, he has been working at i-EM as data scientist focusing on applications in the renewable energy sector, atmospheric physics, and smart grids. Currently, he is senior data scientist with focus on international funding opportunities and dissemination activities.
\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/Mauro_Tucci}}]{Mauro Tucci} received the Ph.D.degree in applied electromagnetism from the University of Pisa, Pisa, Italy, in 2008. Currently, he is an Associate Professor with the Department of Energy, Systems, Territory and Constructions Engineering, University of Pisa. His research interests include computational intelligence
and big data analysis, with applications in electromagnetism, non destructive testing, powerline communications, and smart grids.
\end{IEEEbiography}

\end{document}

\section{Introduction}

\subsection{Motivation}
\IEEEPARstart{A}{s} power generation from renewable sources is increasingly seen as a fundamental component in a joint effort to support decarbonization strategies, hydroelectric power generation is experiencing a new golden age. In fact, hydropower has a number of advantages compared to other types of power generation from renewable sources. Most notably,
hydropower generation can be ramped up and down, which provides a valuable source of flexibility for the power grid, for instance, to support the integration of power generation from other renewable energy sources, like wind and solar. In addition, water in hydropower plants' large reservoirs may be seen as an energy storage resource in low-demand periods and transformed into electricity when needed \cite{Helseth2016,Hjelmeland2019}. Finally, for large turbine-generator units, the mechanical-to-electrical energy conversion process can have a combined efficiency of over 90\% \cite{Bolduc2014}. Accordingly, in 2016, around 13\% of the world's consumed electricity was generated from hydropower\footnote{\url{https://www.iea.org/statistics/balances/}}. In addition, hydropower plants have provided for more than 95\% of energy storage for all active tracked storage installations worldwide\footnote{\url{http://www.energystorageexchange.org/projects/data_visualization}}.

In addition to the aforementioned advantages, hydroelectric power plants are also typically characterized by a long lifespan and relatively low operation and maintenance costs, usually around 2.5\% of the overall cost of the plant. However, according to \cite{IHA_db}, by 2030 over half of the world's hydropower plants will be due for upgrade and modernization, or will have already been renovated. Still according to \cite{IHA_db}, the main reason why major works seem around the corner is that most industries in this field wish to adopt best practices in operations and asset management plans, or in other words, share a desire for optimized performance and increased efficiency. In combination with the quick pace of technological innovation in hydropower operations and maintenance, together with the increased ability to handle and manage big amounts of data, a technological revolution of most hydropower plants is expected to take place soon.

\subsection{State of the art}
In hydropower plants, planned periodic maintenance has been for a long time the main, if not the only one, adopted maintenance method. Condition monitoring procedures have been often reserved for protection systems, leading to shutting down the plants when single monitored signals exceeded pre-defined thresholds (e.g., bearings with temperature and vibration protection). In this context, one of the earliest works towards predictive maintenance has been \cite{Jiang2008}, where Artificial Neural Networks (ANNs) were used to monitor, identify and diagnose the dynamic performance of a prototype of system. Predictive maintenance methods obviously require the measurement and storage of all the relevant data regarding the power plant. An example of early digitalization is provided in \cite{Li2009} where a Wide Area Network for condition monitoring and diagnosis of hydro power stations and substations of the Gezhouba Hydro Power Plant (in China) was established. Thanks to measured data, available in real-time, more advanced methods that combine past history and domain knowledge can provide more efficient monitoring services, advanced fault prognosis, short- and long-term prediction of incipient faults, prescriptive maintenance tools, and residual lifetime and future risk assessment. Benefits of this include, among other things, preventing (possibly severe) faults from occurring, avoiding unnecessary replacements of components, more efficient criteria for scheduled maintenance.

The equipment required for predictive maintenance in hydro generators is also described in \cite{Ribeiro2014}, where the focus was to gain the ability to detect and classify faults regarding the electrical and the mechanical defects of the generator-turbine set, through a frequency spectrum analysis. More recent works (e.g., \cite{Selak2014}) describe condition monitoring and fault-diagnostics (CMFD) software systems that use recent Artificial Intelligence (AI) techniques (in this case a Support Vector Machine (SVM) classifier) for fault diagnostics. In \cite{Selak2014} a CMFD has been implemented on a hydropower plant with three Kaplan units. Another expert system has been developed for an 8-MW bulb turbine downstream irrigation hydropower plant in India, as described in \cite{Buaphan2017}.
An online temperature monitoring system was developed in \cite{Milic2013}, and an artificial neural network based predictive maintenance system was proposed in \cite{Chuang2004}. The accuracy of early fault detection system is an important feature for accurate reliability modeling \cite{Khalilzadeh2014}.

\subsection{Paper contribution}
In this paper we propose a novel Key Performance Indicator (KPI) based on an appropriately trained Self-Organizing Map (SOM) for condition monitoring in a hydropower plant. In addition to detecting faulty operating conditions, the proposed indicator also identifies the component of the plant that most likely gave rise to the faulty behaviour. Very few works, as from the previous section, address the same problem, although there is a general consensus that this could soon become a very active area of research \cite{IHA_db}. In this paper we show that the proposed KPI performs better than a standard multivariate process control tool like the Hotelling control chart, over a test period of more than one year (from April 2018 to July 2019).

This paper is organized as follows: Section \ref{case_study} describes more in detail the case study of interest, and the data used to tune the proposed indicator. Section \ref{Methodologies} illustrates the proposed indicator. Also, the basic theory of the Hotelling multivariate control chart is recalled, as it is used for comparison purposes. The obtained results are provided and discussed in Section \ref{Results}. Finally, in Section \ref{Conclusion} we conclude our paper and outline our current lines of research in this topic.

\section{Case study}
\label{case_study}
Throughout the paper, we will refer to two hydroelectric power plants, called plant A and plant B, which have an installed power of 215 MW and 1000 MW respectively. The plants are located in Italy as shown in Figure \ref{fig:HPP_Plant_Soverzene_Presenzano_location}, and both use Francis turbines. Plant A is of type reservoir, while plant B is of type pumped-storage. More details are provided in the following subsections.
\begin{figure}[ht]
	\centering
	\includegraphics[width=0.3\textwidth]{./fig/2_Case_study/italy.png}
	\caption{Location of the two considered hydropower plants in Italy.

	}
	\label{fig:HPP_Plant_Soverzene_Presenzano_location}
\end{figure}
\subsection{Hydropower plants details}
Plant A is located in Northern Italy and consists of four generation units moved by vertical axes Francis turbines, with a power of 60 MVA for each unit. The machinery room is located 500 meters inside the mountain. The plant is powered by two connected basins: the main basin, with a daily regulation purpose with a capacity of 5.9 $\times 10^6$ m$^3$, and a second basin, limited by a dam, with a seasonal regulation capability. The plant is part of a large hydraulic system and it has been operative since 1951. At full power, the plant employs a flow of 88 m$^3/$s, with a net head of 284 m; in nominal conditions (1 unit working 24/7, 3 units working 12/7) the main basin can be emptied in about 24 hours. The 2015 net production was of 594 GWh, serving both the energy and the service market thanks to the storage capability.

Plant B is representative of pumped-storage power plants; power is generated by releasing water from the upper reservoir down to the power plant which contains four reversible 250 MW Francis pump-turbine-generators. After power production, the water is sent to the lower reservoir. The upper reservoir, formed by an embankment dam, is located at an elevation of 643 meters in the Province of Isernia. Both the upper and lower reservoirs have an active storage capacity of 60 $\times 10^6$  m$^3$. The difference in elevation leads to a hydraulic head of 495 meters. The plant has been in operation since 1994, and its net production in 2015 was 60 GWh. This plant is strategical for its pumping storage capability and sells mainly to the services market.
\subsection{Dataset}
The dataset of plant A consists of 630  analog signals with a sampling time of 1 minute. The dataset of plant B consists of 60 analog signals, since a smaller number of sensors is installed in this system. The signals are collected from several plant-components, for instance, the water intake, penstocks, turbines, generators, and HV transformers.
The acquisition system has been in service since the 1$^{st}$ of May 2017. In this work, we used data from the 1$^{st}$ of May 2017 to the 31$^{st}$ of March 2018, as the initial training set. Then, the model was tested online from the 1$^{st}$ of April 2018 until the end of July 2019. During the online testing phase, we  retrained the model every two months to include the most recent data.

As usual in this kind of applications, before starting the training phase, an accurate pre-processing was required to improve the quality of measured data. In particular:
\begin{enumerate}
\item
several measured signals had a large number of not regular data, such as missing, or ``frozen'' samples (i.e., where the signal measured by the sensor does not change in time), values out of physical or operative limits, spikes, and statistical outliers in general;
\item
the training dataset did not contain information about historical anomalies occurred in the plants; similarly, Operation and Maintenance (O\&M) logs were not available.
\end{enumerate}
Accordingly, we first implemented a classic procedure of data cleaning (see for instance \cite{Data_Cleaning_Book}). This procedure has two advantages itself: first of all, it allows any data-based condition monitoring methodology to be tuned upon nominal data corresponding to a correct functioning of the plant; in addition, the plants' operators were informed of those signals whose percentage of regular samples was below a given threshold, so that they could evaluate whether it could be possible to mitigate the noise with which data were recorded, or whether the sensor was actually broken.

The second problem was mainly due to the fact that in many operating plants, the O\&M logs are often not integrated with the historical databases. Thus, it may be subtle to distinguish nominal and faulty behaviours occurred in the past. In this case, we found it useful to iteratively merge analytic results with feedbacks from on-site operators to reconstruct correct sequences of nominal data. In particular, as we shall see in greater detail in the next section, the output of our proposed procedure is a newly proposed KPI, which monitors the functioning of the hydropower plant. In particular, a warning is triggered when the KPI drops below a threshold and an automatic notification is sent to the operator. The operator, guided by the provided warning, checks and possibly confirms the nature of the detected anomaly. Then, the sequence of data during the fault is eventually removed from the log, so that it will not be included in any historical dataset and it will not be used in the future retraining stages.

\section{Methodologies}
\label{Methodologies}
The proposed approach consists in training a self-organizing map (SOM) neural network in order to build a model of the nominal behaviour of the system, using a historical dataset comprehending nominal state observations. The new state observations are then classified as ``in control'' or ``out-of-control'' after comparing their distortion measure to the average distortion measure of the nominal states used during the training, as it is now illustrated in more detail.

\subsection{Self-Organizing Map neural network based Key Performance Indicator}
Self-organizing maps (SOMs) are popular artificial neural network algorithms, belonging to the unsupervised learning category, which have been frequently used in a wide range of applications \cite{Kohonen1990}-\cite{Tuc_2010}. Given a high-dimensional input dataset, the SOM algorithm produces a topographic mapping of the data into a lower-dimensional output set.

The SOM output space consists of a fixed and ordered bi-dimensional grid of cells, identified by an index in the range $1,\dots,D$,  where a distance metric $d(c,i)$ between any two cells of index $c$ and $i$ is defined \cite{Kohonen1990}. Each cell of index $i$ is associated with a model vector $\mathbf{m}_i\in \mathbb{R}^{1 \times n}$ that lies in the same high-dimensional space of the input patterns $\mathbf{r}\in \Delta$, where the matrix $\Delta\in \mathbb{R}^{N \times n}$ represents the training dataset to be analyzed, containing $N$ observations of row vectors $\mathbf{r}\in \mathbb{R}^{1 \times n}$. After the training,  the distribution of the model vectors resembles the distribution of the input data, with the additional feature of preserving the grid topology: model vectors that correspond to neighbouring cells shall be neighbours in the high-dimensional input space as well. When a new input sample $\mathbf{r}$ is  presented to the network, the SOM finds the best matching unit (BMU) $c$, whose model vector $\mathbf{m}_c$ has the minimum Euclidean distance from $\mathbf{r}$:

\[c = \mathbf{arg min}_{i} \{\|   \mathbf{r} -  \mathbf{m}_i\|\}.\]

It is known that the goal of the SOM training algorithm is to minimize the following distortion measure:
\begin{equation}\label{DM_average}
DM_{\Delta}=\frac{1}{N} \sum_{\mathbf{r} \in \Delta} \sum_{i=1}^{D} w_{ci} \|\mathbf{r}-\mathbf{m}_i\|,
\end{equation}
where the function
\begin{equation}
w_{ci}  = exp\left( \frac{-d(c,i)^2}{2 \sigma^2} \right),
\end{equation}
is the neighbourhood function,  $c$ is the BMU corresponding to input sample $\mathbf{r}$, and $\sigma$ is the neighbourhood width.
The distortion measure indicates the capacity of the trained SOM to fit the data maintaining the bi-dimensional topology of the output grid.
The distortion measure relative to a single input pattern $\mathbf{r}$  is computed as:
\begin{equation}\label{DM_single}
DM(\mathbf{r})= \sum_{i=1}^{D} w_{ci} \|\mathbf{r}-\mathbf{m}_i\|,
\end{equation}
from which it follows that $DM_{\Delta}$, as defined in (\ref{DM_average}), is the average of the distortion measures of all the patterns in the training data $\mathbf{r}\in \Delta$.

In order to  assess the condition of newly observed state patterns $\mathbf{r}$  to be monitored, we introduce the following KPI:
\begin{equation}
KPI(\mathbf{r})=\frac{1}{1+\|1- \frac{DM(\mathbf{r} )}{DM_{\Delta}}  \|}.
\end{equation}

Roughly speaking, the rationale behind the previous KPI definition is as follows: if the acquired state  $\mathbf{r}$ corresponds to a normal behaviour, its distortion measure $DM(\mathbf{r}) $ should be similar to the average distortion measure of the nominal training set $DM_{\Delta}$ (which consists of non-faulty states), and the ratio $ \frac{DM(\mathbf{r}) }{DM_{\Delta}} $ should be close to one, which in turn gives a $KPI( \mathbf{r})$ value to be close to one as well. On the other hand, if the acquired state $\mathbf{r}$ actually corresponds to an anomalous behaviour, $DM(\mathbf{r}) $ should substantially differ from $DM_{\Delta}$, leading to values of $KPI( \mathbf{r})$ considerably smaller than one. In this way, values of the KPI near to one indicate a nominal functioning, while smaller values indicate that the plant is going out of control.

A critical aspect is the choice of the threshold to discriminate a correct and a faulty functioning: for this purpose, we compute the average value $\mu_{KPI}$ and the variance $\sigma^2_{KPI}$ of the filtered KPI values of all the points in the training set $\Delta$, and we define the threshold as a lower control limit (LCL) as follows:

\begin{equation}
LCL_{kpi}= \mu_{kpi}-3\sigma_{kpi}.
\end{equation}

If the measured data are noisy, the proposed KPI may present a noisy nature as well. For this purpose, in our work we filtered the KPI using an exponentially weighted average filter over the last 12 hours of consecutive KPI values.

\subsection{The contribution of individual variables to the SOM-based KPI}

When the SOM-based KPI deviates from its nominal pattern, it is desired to identify the individual variables that most contribute to the KPI variation. This allows the operator not only to identify a possible malfunctioning in the hydropower plant, but also the specific cause, or location, of such a malfunctioning. For this purpose, we first calculate an average contribution to DM of individual variables using the data in the nominal dataset $\Delta$. We then compare the contribution of individual variables of newly acquired patterns to the average contribution of nominal training patterns.

For each pattern $\mathbf{r} \in \Delta$, we calculate the following average distance vector $\mathbf{d(r)}\in \mathbb{R}^{1 \times n}$ :

\begin{equation}
\mathbf{d(r)} =\frac{1}{D} \sum_{i=1}^{D} w_{ci} ( \mathbf{r}-\mathbf{m}_i ), \forall \mathbf{r} \in \Delta.
\end{equation}

Then we calculate the vector of squared components of $\mathbf{d(r)}$   normalized to have norm 1, named $\mathbf{d_n(r)} \in \mathbb{R}^{1 \times n}$, as

\begin{equation}\label{vet_dist}
\mathbf{d_n(r)} = \frac{\mathbf{d(r)}\circ \mathbf{d(r)} }{\| \mathbf{d(r)}\|^2}, \forall \mathbf{r} \in \Delta,
\end{equation}
where the symbol $\circ$ denotes the Hadamard (element-wise) product. Finally, we compute the average vector of the normalized squared distance components for all patterns in $\Delta$:

\begin{equation}
\mathbf{d_{n_{\Delta}}} =\frac{1}{N} \sum_{r\in\Delta} \mathbf{d_n(r)} .
\end{equation}

When a new pattern $\mathbf{r}$ is acquired during the monitoring phase, we calculate the following Hadamard ratio:

\begin{equation}
\mathbf{d_n(r)}\div \mathbf{d_{n_{\Delta}}} = [ cr_1 cr_2 \dots cr_n]
\end{equation}

where the contribution ratios $cr_i$ , $i=1\dots n$ represent how individual variables of the new pattern influence the DM compared to their influence in non-faulty conditions. If the new pattern actually corresponds to a non-faulty condition, $cr_i$ takes values close to one. If the new pattern deviates from the nominal behaviour, some of the  $cr_i$ exceed the unitary value. An empirical threshold 1.3 was selected, as a trade-off between false positives and true positives.

\subsection{Hotelling multivariate control chart}
As a term of comparison for our SOM-based KPI indicator, we consider the Hotelling multivariate control chart \cite{Hotelling1947}. While very few works may be found for our hydropower plant application of interest, Hotelling charts are quite popular for multivariable process control problems in general, and we take it as a benchmark procedure for comparison.

The Hotelling control chart performs a projection of the multivariate data to a scalar parameter denoted as $t^2$ statistics,
which is defined as the square of the Mahalanobis distance \cite{Maesschalck2000} between the
observed pattern and the vector containing the mean values of
the variables in nominal conditions. The Hotelling $t^2$ statistics is able to capture the changes in
multivariate data, revealing the deviations from the nominal
behaviour, and for these reasons the
Hotelling control chart is widely used for early detection of
incipient faults,  see for instance\cite{Aparisi2009}.
The construction of the control chart includes two phases: in the
first phase, historical data are analyzed and the control limits
are computed; phase two corresponds to the monitoring of the newly acquired state patterns.
\subsubsection{Phase one}
 Let the nominal historic dataset be represented by the matrix ${{\Delta}} \in {\mathbb{R} ^{N \times n}}$,  containing N observations of row  vectors $ {{\bf{r}}}\in {\mathbb{R} ^{1 \times n}}$. The sample mean vector ${{\boldsymbol{\mu }}}\in {\mathbb{R} ^{1 \times n}}$ of the data  is defined as:
\begin{equation}
\label{mean_vector_phase_one}
{{{\boldsymbol{\mu }}}} = \frac{1}{N}\sum\limits_{\mathbf{r} \in \Delta} {{{\bf{r}}}} .
\end{equation}

The covariance matrix is defined by means of the zero-mean data matrix ${{\Delta}_{0}} \in {\mathbb{R} ^{N \times n}}$:
\begin{equation}
{{\Delta}_{0}} = \Delta- \bf{1} \cdot \boldsymbol{\mu},
\end{equation}
where ${\bf{1}} \in {\mathbb{R} ^{N \times 1}}$ represents a column vector with all entries equal to one.
Then, the covariance matrix ${{\bf{C}}} \in {\mathbb{R} ^{n \times n}}$ of the data is defined as:
\begin{equation}
{{\bf{C}}} = \frac{1}{{N - 1}}{\Delta}_{0}^T{{\Delta}_{0}},
\end{equation}
where  ${\left( {} \right)^T}$ denotes vector transpose  operation. The multivariate statistics  ${{\boldsymbol{\mu }}}$  and $\bf{C}$ represent the multivariate distribution of nominal observations, and we assume that  $\bf{C}$ is full rank.  The scalar ${t^2}$ statistics is defined as a function of a single state pattern ${\bf{r}}\in \mathbb{R} ^{1 \times n}$:
\begin{equation}
\label{t_2_def}
t^2(\bf{r}) = \left( {{{\bf{r}}} - {{\boldsymbol{\mu }}}} \right){\bf{C}}^{ - 1}{\left( {{{\bf{r}}} - {{\boldsymbol{\mu }}}} \right)^T}.\,
\end{equation}

The ${t^2}$ statistics is small when  pattern vector ${\bf{r}}$ represents nominal states, while it increases when the pattern vector ${\bf{r}}$  deviates from the nominal behaviour. In order to define the control limits $UCL$ and $LCL$ of the control chart, in this first phase we calculate the mean value $\mu_{t^2}$ and standard deviation $\sigma_{t^2}$ of the $t^2$ values obtained with all the observations of the historical dataset ${\bf{r}}\in{\Delta}$.  Then we define the safety thresholds as:
\begin{equation}
\label{Safety_Thresholds}
\left\{
\begin{array}{lll}
UCL_{t^2} & = & \mu_{t^2} + 3 \sigma_{t^2}\\
& & \\
LCL_{t^2} & = & max(\mu_{t^2} - 3 \sigma_{t^2}, 0)\\
\end{array}\right..
\end{equation}

\subsubsection{Phase two}
In the second phase, new observation vectors are measured, and the corresponding $t^2$  values are calculated as in Equation (\ref{t_2_def}). The Hotelling control chart may be seen as a monitoring tool that plots the $t^2$  values as consecutive points in time and compares them against the control limits. The process is considered to be ``out of control'',  when the $t^2$ values continuously exceed the control limits.
\section{Results}
\label{Results}
After a prototyping phase, the condition monitoring system has been operating since April 2018 on several components of the plants described in Section \ref{case_study}, with a total of more than 600 input signals. As an example, some of the most critical components are shown in Table \ref{tab:20190208_analyzed_components}. As can be seen from the last column of the table, there is usually a large redundancy of sensors measuring the same, or closely related, signals. The components listed in Table \ref{tab:20190208_analyzed_components} are among the most relevant components for the plant operators, as it is known that their malfunctioning may, in some cases, lead to major failures or plant emergency stops.

\begin{table}[ht!]
	\centering
	\begin{tabular}{l|l|c}
		\hline
		 \textbf{Component name} & \textbf{Measured Signals} & \textbf{Number of sensors} \\
		\hline
		 Generation Units & Vibrations & 34 \\
				\hline
		 HV Transformer & Temperatures & 27 \\
				 &  Gasses levels &  \\
				\hline

		 Turbine & Pressures & 27  \\
				 & Flows &  \\
				 & Temperatures &  \\
				\hline

		 Oleo-dynamic system & Pressures & 20 \\
				  & Temperatures &  \\
				\hline

		 Supports & Temperatures & 54  \\
				\hline
		 Alternator & Temperatures & 43  \\
		\hline
	\end{tabular}
	\vspace{0.05cm}
	\caption{List of main components analyzed for the hydro plants.}
	\label{tab:20190208_analyzed_components}
\end{table}

Since April 2018, our system detected more than $20$ anomalous situations, the full list of whose occurrences is given in Table \ref{occurrences}, with different degrees of severity, defined with plant operators, ranging from ``no action needed'' to ``severe malfunction'', leading to plant emergency stop.

It is worth noting that these events were not reported by the standard condition-monitoring systems operative in the plants, and in some cases would not have been well identified by the multivariate Hotelling control chart either, thus emphasizing the importance of the more sophisticated KPI introduced in this paper. In addition, we also see how the ability to identify non-nominal operations improves over time, as new information is acquired.
\begin{table*}[htbp]
  \centering
     \begin{tabular}{|c|c|c|l|l|l|}

		\hline
		\rowcolor[rgb]{ .949,  .949,  .949} \textbf{Plant Code} & \textbf{Unit ID} & \textbf{Warning Date} & \textbf{Title} & \textbf{Severity Level} & \textbf{Feedback} \\

		\hline
    \rowcolor[rgb]{ .886,  .937,  .855} A     & 2     & 01/25/2018 & Efficiency Parameters & Low   & Weather anomalies effects  \\

		\hline
		\rowcolor[rgb]{ .886,  .937,  .855} A     & 1     & 07/02/2019 & Supports Temperature & Low   & Under investigation \\

		\hline
    \rowcolor[rgb]{ .886,  .937,  .855} A     & 3     & 03/03/2019 & Francis Turbine & Low   & Anomaly related to ongoing maintenance activities\\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 1     & 06/11/2018 & Efficiency Parameters & Medium-Low & Not relevant anomaly \\
		\hline

    \rowcolor[rgb]{ .988,  .894,  .839} A     & 1     & 06/29/2018 & Francis Turbine & Medium-Low & Weather anomalies effects on turbine's components temperature \\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 4     & 06/30/2018 & Efficiency Parameters & Medium-Low & Not relevant anomaly \\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 2     & 08/07/2018 & Francis Turbine & Medium-Low & Weather anomalies effects on turbine's components temperature \\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 2     & 09/14/2018 & Generator Vibrations & Medium-Low & Not relevant anomaly \\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 3     & 01/10/2019 & Generator Temperature & Medium-Low & Under investigation \\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 3     & 01/16/2019 & Transformer Temperature & Medium-Low & Weather anomalies effects on transformer temperature \\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 2     & 03/03/2019 & Generator Temperature & Medium-Low & Anomaly related to ongoing maintenance activities\\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 3     & 06/17/2019 & Generator Temperature & Medium-Low & Under investigation \\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 4     & 06/21/2019 & Generator Temperature & Medium-Low & Weather anomalies effects on generator temperature \\

		\hline
    \rowcolor[rgb]{ .886,  .937,  .855} A     & 1     & 04/22/2018 & HV transformer gasses & Medium-High & Monitoring system anomaly \\

		\hline
    \rowcolor[rgb]{ .886,  .937,  .855} B     & 2     & 06/27/2018 & Generator Vibrations & Medium-High & Data Quality issue \\

		\hline
    \rowcolor[rgb]{ .886,  .937,  .855} A     & 1     & 10/29/2018 & Generator Vibrations & Medium-High & Data Quality issue \\

		\hline
    \rowcolor[rgb]{ .886,  .937,  .855} A     & 3     & 11/20/2018 & Generator Vibrations & Medium-High & Under investigation \\

		\hline
    \rowcolor[rgb]{ .886,  .937,  .855} A     & 1     & 03/24/2019 & Francis Turbine & Medium-High & Under investigation \\

		\hline
    \rowcolor[rgb]{ .886,  .937,  .855} A     & 2     & 04/07/2019 & Oleo-Dynamic System & Medium-High & Under investigation \\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} B     & 2     & 10/01/2018 & Generator Temperature & High  & Sensor anomaly on block channel signal\\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} B     & 2     & 11/01/2018 & Generator Temperature & High  & Sensor anomaly on block channel signal\\

		\hline
    \rowcolor[rgb]{ .988,  .894,  .839} A     & 2     & 03/01/2019 & HV transformer gasses & High  & Under investigation \\
		\hline
    \end{tabular}%
		\caption{List of anomalous behaviours that have been noticed during 16 months of test on the two hydropower plants. Lines 14 and 20 correspond to the two faults that have been illustrated in this paper.}
  \label{occurrences}%
\end{table*}%
 We now describe more in detail two different anomalies belonging to the two different plants, as summarized in Table \ref{tab:20190131_SOM_results}.
\begin{table}[ht!]
	\centering
	\begin{tabular}{c|c|c|l}
		\hline
		\textbf{Case Study} &\textbf {Plant}  & \textbf{Warning name}     &\textbf{ Warning date} \\
				\hline
	 1	& B       & Generator Temperature  & 10/01/2018  \\
				\hline
	2	& A       & HV transformer gasses    & 04/22/2018 \\

		\hline

	\end{tabular}%
	\caption{Two failures reported by the predictive system and discussed in detail.}

	\label{tab:20190131_SOM_results}%

\end{table}%

\subsection{Case Study 1: plant B - generator temperature signals}

In October 2018, an anomalous behavior was reported by our system, which indicated an anomaly regarding a sensor measuring the iron temperature of the alternator. It was then observed that the temperature values of such a sensor were higher than usual, as shown in Figure \ref{fig:Temperatura_1_Ferro_Alternatore_2_warning_id1_zoom.}, and also higher than the measurements taken by other similar sensors. However the temperature values did not yet exceed the warning threshold of the condition-monitoring systems already operative in the plant.

From an inspection of the sensor measurements it was possible to establish that the exact day when this anomaly started occurring, the proposed SOM based KPI sharply notified a warning, as shown in Figure \ref{fig:KPI_temp_IDRT_subplot_SOM_T2};  for this case, the time-extension of the training dataset  was nine months, from 1 January 2018 to 1 September 2018. After receiving the warning alert, the plant operators checked the sensor and confirmed the event as a relevant anomaly. In particular, they acknowledged that this was a serious problem, since a further degradation of the measurement could have eventually led to the stop of the generation unit. For this reason, timely actions were taken: operators restored the nominal and correct behavior of the sensor starting on the 12$^{th}$ of October 2018.  The saved costs related to the prevented stopping of the generation unit were estimated in the range between 25 k\euro\ and 100 k\euro.

While also the Hotelling control chart noticed an anomalous behaviour of the sensor in the same time frame, still it would have given rise to several false positives in the past, as shown in Figure \ref{fig:KPI_temp_IDRT_subplot_SOM_T2}.

\begin{figure}[!ht]
	\centering
	\includegraphics[trim=3cm 0cm 2cm 0cm,width=1\linewidth]

	{./fig/4_Results/Presenzano_Generator_v2.eps}
	\caption{Measurement of the temperature sensor in the alternator of plant B. Anomalous values were detected by our system (start warning), timely actions were taken and the correct functioning was restored (end warning)  }
	\label{fig:Temperatura_1_Ferro_Alternatore_2_warning_id1_zoom.}
\end{figure}

\begin{figure*}[!ht]
	\centering
	\includegraphics[trim=3cm 0cm 2cm 0cm,width=\linewidth]{./fig/4_Results/Presenzano_Generator_con_T2_Mauro.eps}

	\caption{Plant A: KPI as a function of time. SOM-based results (top) are compared with $t^2$-based results (bottom).}
	\label{fig:KPI_temp_IDRT_subplot_SOM_T2}
\end{figure*}

\subsection{Case study 2: Plant A - HV Transformer anomaly}

The SOM-based KPI detected  an anomaly on the HV transformer of one of the Generation Units at the beginning of June 2018 as shown in fig. \ref{fig:KPI_hydran_IDCA_subplot_SOM_T2}. After inspection of the operators, they informed us that similar anomalous situations had occurred in the past as well, but had not been tagged as faulty behaviours.  As soon as the time occurrences of the similar past anomalies had been notified by a plant operator, we proceeded to remove the corresponding signals from the training set. Then we recomputed our KPI based on the revised corrected historical dataset, and the KPI retrospectively found out that the ongoing faulty pattern had actually started one month earlier. This updated information was then validated by analyzing the output of an already installed gas monitoring system, that continuously monitors a composite value of various fault gases in ppM (Parts per Million) and tracks the oil-moisture. The gas monitoring system had been measuring increasing values, with respect to the historical ones, since the 22$^{nd}$ of April 2018, but no warning had been generated by that system. In this case, this was not however a critical fault, as the level of gasses in the transformer oil was not exceeding the maximum feasible limit. However, the warning triggered by our system was used to schedule maintenance actions that restored the nominal operating conditions. In addition, the feedback from the plant operators was very useful in order to tune the SOM-based monitoring system, and to increase its early detection capabilities.

In this case, the Hotelling control chart realized of the anomalous behaviour only 20 days after our SOM-based KPI.

\begin{figure*}[!ht]
	\centering
	\includegraphics[trim=4cm 0cm 2cm 0cm,width=\linewidth]
	{./fig/4_Results/Soverzene_Transormer_con_T2.eps}
	\caption{Plant B: KPI as a function of time. SOM-based results (top) are compared with $t^2$-based results (bottom).}
	\label{fig:KPI_hydran_IDCA_subplot_SOM_T2}
\end{figure*}

\section{Conclusion}
\label{Conclusion}
Driven by rapidly evolving enabling technologies, most notably Internet-of-Thing sensors and communication tools, together with more powerful artificial intelligent algorithms, condition monitoring, early diagnostics and predictive maintenance methodologies and tools are becoming some of the most interesting areas of research in the power community. While some preliminary examples can be found in many fields, solar and wind plants being one of them, fewer applications can be found in the field of hydropower plants. In this context, this paper is one of the first examples, at least up to the knowledge of the authors, that results of a newly proposed KPI are validated over a 1-year running test field in two hydropower plants.\\
\newline
This paper provided very promising preliminary results, that encourage further research on this topic. In particular the proposed procedure can not be implemented in a fully unsupervised fashion yet, but some iterations with plant operators still take place when alarm signals are alerted. Also, the proposed condition monitoring strategy is a first step towards fully automatic predictive maintenance schemes, where faults are not just observed but are actually predicted ahead of time, possibly when they are only at an incipient stage. In the opinion of the authors this is a very promising area of research and there is a general interest towards the development of such predictive strategies.

\section{Proof of the First Zonklar Equation}

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\input{./section/6_Bibliography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/alessandro_betti.jpg}}]{Alessandro Betti}
	Dr. Alessandro Betti received the M.S. degree in Physics and the Ph.D. degree in Electronics Engineering from the University of Pisa, Italy, in 2007 and 2011, respectively. His main field of research was modeling of noise and transport in quasi-one dimensional devices. His work has been published in 10 papers in peer-reviewed journals in the field of solid state electronics and condensed matter physics and in 16 conference papers, presenting for 3 straight years his research at the top International Conference IEEE in electron devices, the International Electron Device Meeting in USA. In September 2015 he joined the company i-EM in Livorno, where he currently works as a Senior Data Scientist developing power generation forecasting, predictive maintenance and Deep Learning models, as well as solutions in the electrical mobility fields and managing a Data Science Team.
\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/Emanuele_Crisostomi}}]{Emanuele Crisostomi}(M'12-SM'16) received the B.Sc. degree in computer science engineering, the M.Sc. degree in automatic control, and the Ph.D. degree in automatics, robotics, and bioengineering from the University of Pisa, Pisa, Italy, in 2002, 2005, and 2009, respectively. He is currently an Associate Professor of electrical engineering in the Department of Energy, Systems, Territory and Constructions Engineering, University of Pisa. He has authored tens of publications in top refereed journals and he is a co-author of the recently published book on ``Electric and Plug-in Vehicle Networks: Optimization and Control'' (CRC Press, Taylor and Francis Group, 2017). His research interests include control and optimization of large scale systems, with applications to smart grids and green mobility networks.

\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/gianluca_paolinelli.jpg}}]{Gianluca Paolinelli}

Gianluca Paolinelli received the bachelor and master degree in electrical engineering from the University of Pisa, Pisa, Italy, in 2014 and 2018 respectively. His research interests included big data analysis and computational intelligence applied in on-line monitoring and diagnostics.
Currently, he is an electrical software engineer focused in the development of electric and hybrid power-train controls for Pure Power Control S.r.l.

\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/antonio_piazzi.jpg}}]{Antonio Piazzi}
	Antonio Piazzi received the M.Sc. degree in electrical engineering from the University of Pisa, Pisa, Italy, 2013. Electrical engineer at i-EM since 2014, he gained his professional experience in the field of renewable energies. His research interests include machine learning and statistical data analysis, with main applications on modeling and monitoring the behaviour of renewable power plants. Currently, he is working on big data analysis applied on hydro power plants signals.
\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/FabrizioRuffini_Linkedin}}]{Fabrizio Ruffini}
	Fabrizio Ruffini received the Ph.D. degree in Experimental Physics from University of Siena, Siena, Italy, in 2013. His research-activity is centered on data analysis, with particular interest in multidimensional statistical analysis. During his research activities, he was at the Fermi National Accelerator Laboratory (Fermilab), Chicago, USA, and at the European Organization for Nuclear Research (CERN), Geneva, Switzerland. Since 2013, he has been working at i-EM as data scientist focusing on applications in the renewable energy sector, atmospheric physics, and smart grids. Currently, he is senior data scientist with focus on international funding opportunities and dissemination activities.
\end{IEEEbiography}

\begin{IEEEbiography}[{\includegraphics[width=1in,height=1.25in,clip,keepaspectratio]{./fig/6_Bibliography/Mauro_Tucci}}]{Mauro Tucci} received the Ph.D.degree in applied electromagnetism from the University of Pisa, Pisa, Italy, in 2008. Currently, he is an Associate Professor with the Department of Energy, Systems, Territory and Constructions Engineering, University of Pisa. His research interests include computational intelligence
and big data analysis, with applications in electromagnetism, non destructive testing, powerline communications, and smart grids.
\end{IEEEbiography}

\end{document}