text: Systems for symbolic event recognition infer occurrences of events in time using a set of event definitions in the form of first-order rules. The Event Calculus is a temporal logic that has been used as a basis in event recognition applications, providing among others, direct connections to machine learning, via Inductive Logic Programming (ILP). We present an ILP system for online learning of Event Calculus theories. To allow for a single-pass learning strategy, we use the Hoeffding bound for evaluating clauses on a subset of the input stream. We employ a decoupling scheme of the Event Calculus axioms during the learning process, that allows to learn each clause in isolation. Moreover, we use abductive-inductive logic programming techniques to handle unobserved target predicates. We evaluate our approach on an activity recognition application and compare it to a number of batch learning techniques. We obtain results of comparable predicative accuracy with significant speed-ups in training time. We also outperform hand-crafted rules and match the performance of a sound incremental learner that can only operate on noise-free datasets. This paper is under consideration for acceptance in TPLP.
  @cite_18: When comparing inductive logic programming (ILP) and attribute-value learning techniques, there is a trade-off between expressive power and efficiency. Inductive logic programming techniques are typically more expressive but also less efficient. Therefore, the data sets handled by current inductive logic programming systems are small according to general standards within the data mining community. The main source of inefficiency lies in the assumption that several examples may be related to each other, so they cannot be handled independently. Within the learning from interpretations framework for inductive logic programming this assumption is unnecessary, which allows to scale up existing ILP algorithms. In this paper we explain this learning setting in the context of relational databases. We relate the setting to propositional data mining and to the classical ILP setting, and show that learning from interpretations corresponds to learning from multiple relations and thus extends the expressiveness of propositional learning, while maintaining its efficiency to a large extent (which is not the case in the classical ILP setting). As a case study, we present two alternative implementations of the ILP system TILDE (Top-down Induction of Logical DEcision trees): TILDEclassic, which loads all data in main memory, and TILDELDS, which loads the examples one by one. We experimentally compare the implementations, showing TILDELDS can handle large data sets (in the order of 100,000 examples or 100 MB) and indeed scales up linearly in the number of examples.
  @cite_22: 
  @cite_21: Decision rules, which can provide good interpretability and flexibility for data mining tasks, have received very little attention in the stream mining community so far. In this work we introduce a new algorithm to learn rule sets, designed for open-ended data streams. The proposed algorithm is able to continuously learn compact ordered and unordered rule sets. The experimental evaluation shows competitive results in comparison with VFDT and C4.5rules.
  @cite_0: 
  @cite_19: Two common characteristics of relational data sets -- concentrated linkage and relational auto-correlation -- can cause traditional methods of evaluation to greatly overestimate the accuracy of induced models on test sets. We identify these characteristics, define quantitative measures of their severity, and explain how they produce this bias. We show how linkage and autocorrelation affect estimates of model accuracy by applying FOIL to synthetic data and to data drawn from the Internet Movie Database. We show how a modified sampling procedure can eliminate the bias.
  @cite_5: Many data sets can be represented naturally as collections of linked objects. For example, document collections can be represented as documents (nodes) connected by citations and hypertext references (links). Similarly, organizations can be represented as people (nodes) connected by reporting relationships, social relationships, and communication patterns (links). This type of data representation is facilitated by the growing availability of object-oriented databases and hypertext systems. The analysis of such data, often called link analysis, is becoming increasingly important in such diverse fields as law enforcement, fraud detection, epidemiology, and information retrieval. One important class of techniques for link analysis involves inductive inference — the construction of predictive relationships from linked data. For example, researchers in information retrieval might construct models to predict whether a particular WWW document is a homepage based on features of other documents to which it is connected (e.g., pages listing publications or family photos). Similarly, law enforcement agents might construct models to predict whether a person is a potential money-launderer based on bank deposits, international travel, business connections, and the arrest records of known associates. Constructing such predictive relationships is an essential part of link analysis. Despite growing data analysis opportunities, relatively little work examines the unique statistical challenges of inductive inference in linked data. This paper examines three such challenges: 1) statistical dependence caused by linked instances; 2) bias introduced by sampling density; and 3) multiple comparisons intensified by feature combinatorics. In general, current systems
  @cite_15: 
  @cite_25: 
  @cite_20: Nowadays, many organizations have relational databases with millions of records and an important question is how to extract information from them. This work proposes HTILDE (Hoeffding TILDE) to handle very large relational databases, based on the Inductive Logic Programming (ILP) system TILDE (Top-down Induction of Logical Decision Trees) and the propositional Very Fast Decision Tree (VFDT) learner. It is an incremental and anytime algorithm that uses the Hoeffding bound to find out the amount of examples that must be considered for choosing the best test for a node. The results show that, compared to TILDE, HTILDE generates theories from very large relational datasets more efficiently without harming their quality measures (F-measure, precision, recall and accuracy). Also, HTILDE learns less complex theories than TILDE.
  @cite_11: There is a large and growing mismatch between the size of the relational data sets available for mining and the amount of data our relational learning systems can process. In particular, most relational learning systems can operate on data sets containing thousands to tens of thousands of objects, while many real-world data sets grow at a rate of millions of objects a day. In this paper we explore the challenges that prevent relational learning systems from operating on massive data sets, and develop a learning system that overcomes some of them. Our system uses sampling, is efficient with disk accesses, and is able to learn from an order of magnitude more relational data than existing algorithms. We evaluate our system by using it to mine a collection of massive Web crawls, each containing millions of pages.
summary: The Hoeffding bound has been used for propositional machine learning tasks on data streams, such as learning decision trees @cite_0 and decision rules @cite_21 , and clustering @cite_15 . However, its usage for learning relational models is limited. One reason is that it requires independence of observations, which cannot always be ensured in relational domains, due to dependencies in the data @cite_5 @cite_19 @cite_11 @cite_22 . An ILP approach that uses the Hoeffding bound for relational learning is @cite_20 , an extension of the system for learning first-order decision trees @cite_25 . These are decision trees where each internal node consists of a conjunction of literals and each leaf is a propositional predicate representing a class. constructs trees by testing conjunctions of literals at each node, using an ILP refinement operator to generate the conjunctions and information gain as the guiding heuristic. extends by using the Hoeffding bound to perform these internal tests on a subset of the training data. To ensure independence of observations, learns from interpretations @cite_18 , a setting, used also by , where each training instance is assumed a disconnected part of the dataset.