text: This paper introduces a new similarity measure, the covering similarity, which we formally define for evaluating the similarity between a symbolic sequence and a set of symbolic sequences. A pairwise similarity can also be directly derived from the covering similarity to compare two symbolic sequences. An efficient implementation to compute the covering similarity is proposed which uses a suffix-tree data structure, but other implementations, based on suffix array for instance, are possible and are possibly necessary for handling very large-scale problems. We have used this similarity to isolate attack sequences from normal sequences in the scope of host-based intrusion detection. We have assessed the covering similarity on two well-known benchmarks in the field. In view of the results reported on these two datasets for the state-of-the-art methods, according to the comparative study, we have carried out based on three challenging similarity measures commonly used for string processing, or in bioinformatics, we show that the covering similarity is particularly relevant to address the detection of anomalies in sequences of system calls.
  @cite_38: Due to the advance of information and communication techniques, sharing information through online has been increased. And this leads to creating the new added value. As a result, various online services were created. However, as increasing connection points to the internet, the threats of cyber security have also been increasing. Intrusion detection system(IDS) is one of the important security issues today. In this paper, we construct an IDS model with deep learning approach. We apply Long Short Term Memory(LSTM) architecture to a Recurrent Neural Network(RNN) and train the IDS model using KDD Cup 1999 dataset. Through the performance test, we confirm that the deep learning approach is effective for IDS.
  @cite_36: Payload anomaly detection is a relatively young branch of intrusion detection focusing on the analysis of packet payloads to identify attacks in the application layer. Previous payload anomaly detection systems have used models such as Support Vector Machines and various Markov models. The model proposed in this paper for analysis is the Conditional Random Fields model. Conditional Random Fields have been successfully applied to anomaly detection methods other than payload anomaly detection. In this paper, a prototype payload anomaly detection system is discussed then tested against a public data set. The results of the testing show that the Conditional Random Fields model is a very powerful model for payload anomaly detection.
  @cite_32: Network security has become an essential component of any computer network. Despite significant advances having been made on network-based intrusion prevention and detection, ongoing attacks penetrating network-based security mechanisms have been reported. It is being realized that network-based security mechanisms such as firewalls or intrusion detection systems (IDS) are not effective in detecting certain attacks such as insider attacks and attacks without generating significant network traffic. The trend of network security will be to merge host-based IDS (HIDS) and networkbased IDS (NIDS). This chapter will provide the fundamentals of host-based anomaly IDS as well as their developments. A new architectural framework is proposed for intelligent integration of multiple detection engines. The novelty of this framework is that it provides a feedback loop so that one output from a detection engine can be used as an input for another detection engine. It is also illustrated how several schemes can be derived from this framework. New research topics for future research are discussed.
  @cite_45: An intrusion detection system is now an inevitable part of any computer network. With the ever increasing number and diverse type of attacks, including new and previously unseen attacks, the effectiveness of an intrusion detection system is often subjected to testing. The use of such systems have greatly reduced the threat level, however, the networks and hence the data and services offered by them are far away from the state when they can be considered as secure. In this paper we propose and experimentally validate the use and robustness of 'conditional random fields,' for the task of intrusion detection. We show, experimentally, that the conditional random fields, can be very effective in detecting intrusions when compared with the previously known techniques.
  @cite_50: An anomaly intrusion detection method based on HMM is presented. The system call trace of a UNIX privileged process is passed to a HMM to obtain state transition sequences. Preliminary experiments prove the state transition sequences can express the different mode between normal action and intrusion behaviour in a more stable and simple manner.
  @cite_13: Intrusion detection is well-known as an essential component to secure the systems in Information and Communication Technology (ICT). Based on the type of analyzing events, two kinds of Intrusion Detection Systems (IDS) have been proposed: anomaly-based and misuse-based. In this paper, three-layer Recurrent Neural Network (RNN) architecture with categorized features as inputs and attack types as outputs of RNN is proposed as misuse-based IDS. The input features are categorized to basic features, content features, time-based traffic features, and host-based traffic features. The attack types are classified to Denial-of-Service (DoS), Probe, Remote-to-Local (R2L), and User-to-Root (U2R). For this purpose, in this study, we use the 41 features per connection defined by International Knowledge Discovery and Data mining group (KDD). The RNN has an extra output which corresponds to normal class (no attack). The connections between the nodes of two hidden layers of RNN are considered partial. Experimental results show that the proposed model is able to improve classification rate, particularly in R2L attacks. This method also offers better Detection Rate (DR) and Cost Per Example (CPE) when compared to similar related works and also the simulated Multi-Layer Perceptron (MLP) and Elman-based intrusion detectors. On the other hand, False Alarm Rate (FAR) of the proposed model is not degraded significantly when compared to some recent machine learning methods.
  @cite_12: This paper aims to investigate and identify distinguishable TCP services, that comprise of both attack and normal types of TCP packets, using J48 decision tree algorithm. A predictive model capable of discriminating between normal and abnormal behavior of network traffic is developed by integrating Hidden Markov Model (HMM) technique with anomaly intrusion detection approach for each distinguishable TCP service. The model has been trained for each TCP session of the KDD Cup 1999 dataset using Baum-Welch training (BWT) and Viterbi training (VT) algorithms. Evaluation of the developed HMM model is performed using Forward and Backward algorithms. Results show that the proposed model is able to classify network traffic with approximately 76 to 99 accuracy. The overall performance of model is measured using standard evaluation method ROC curves.
summary: , essentially Hidden Markov Models (HMM) @cite_50 @cite_32 @cite_12 , Conditional random Fields (CRF) @cite_45 @cite_36 or Recurrent Neural Networks (RNN, LSTM, etc.) @cite_13 @cite_38 have been used with apparent success on various intrusion detection tasks, such as payload analysis or Network Layer Intrusion Detection or HIDS. However, the choice of parameters such as the order of the Markovian dependency, number of hidden variables, etc., is often the result of a compromise to avoid over-fitting, and long-term dependency is not necessarily easily modeled.