  Attackers have developed ever more sophisticated and intelligent ways to hack
information and communication technology systems. The extent of damage an
individual hacker can carry out upon infiltrating a system is well understood.
A potentially catastrophic scenario can be envisaged where a nation-state
intercepting encrypted financial data gets hacked. Thus, intelligent
cybersecurity systems have become inevitably important for improved protection
against malicious threats. However, as malware attacks continue to dramatically
increase in volume and complexity, it has become ever more challenging for
traditional analytic tools to detect and mitigate threat. Furthermore, a huge
amount of data produced by large networks has made the recognition task even
more complicated and challenging. In this work, we propose an innovative
statistical analysis driven optimized deep learning system for intrusion
detection. The proposed intrusion detection system (IDS) extracts optimized and
more correlated features using big data visualization and statistical analysis
methods (human-in-the-loop), followed by a deep autoencoder for potential
threat detection. Specifically, a pre-processing module eliminates the outliers
and converts categorical variables into one-hot-encoded vectors. The feature
extraction module discard features with null values and selects the most
significant features as input to the deep autoencoder model (trained in a
greedy-wise manner). The NSL-KDD dataset from the Canadian Institute for
Cybersecurity is used as a benchmark to evaluate the feasibility and
effectiveness of the proposed architecture. Simulation results demonstrate the
potential of our proposed system and its outperformance as compared to existing
state-of-the-art methods and recently published novel approaches. Ongoing work
includes further optimization and real-time evaluation of our proposed IDS.
