
## 3 Augmented Data Mining Live Cycle

The implementations of the knowledge taxonomy are given by the processing system, the (ontological) knowledge basis, the information state, the **meta knowledge by ML models, and the introspective ML models. Thereby, the theory**
combines top-down approaches (i.e., ontological knowledge representation) with bottom-up approaches (i.e., empirical process data model exploitation). The later means information state features aggregation and data mining by combining declarative and procedural knowledge. Metacognitive control is the application of the introspective knowledge gained on the meta-level by **controlling**
the object-level, as illustrated in figure 2. According to control theory, we are not only able to vary parameters of the object level control in real-time, but augment the object-level (cognitive) reasoning process by learned meta-models. Hence, the metacognitive control **idea includes planning, monitoring, authoring,** integration, and evaluation.

The last two steps, integration and evaluation, are implemented by augmenting the data mining life cycle to support a live integration of obtained models.

We call this additional step the (automatic) operationalisation **of learned meta** models. Figure 3 illustrates the Cross Industry Standard Process **for Data Mining**
cycle5 and includes our augmentation. In the modelling phase**, various modelling**
techniques are selected and applied. The modelling phase is finished when one or more models, which appear to be of high quality at least from a data **analysis perspective, have been built. These models then need to be evaluated before**
their deployment. In the evaluation phase **we use the models to review the model**
building process. This evaluation is done by running the system on unseen supervised data or by reinforcement learning experiments. Finally, at **the end of**
the evaluation stage, a decision has to be reached as to whether to **use the data** mining results obtained. Then a new model is deployed and used in the domain or business units.

The CRISP cycle closes with the evaluation of the deployed system in the real application context (domain/business understanding**), whether it performs well,** or not. In fact, this is a kind of metacognitive process conducted by the domain experts. The introspective mechanism represents a new phase between evaluation and (human) domain/business understanding. It automatically optimises the behaviour of the deployed system and provides hints for human understanding by generating transparent metamodels of the system's performance, for example, introspective association rules and decision trees. The cycle now includes the additional step (automatic) operationalisation **before it closes.**
Our aim to integrate the introspective mechanism in order to extend **the data**
mining cycle by a new phase where system introspection is integrated, resulted in a new step of the data mining life cycle, i.e., (automatic) operationalisation**. The** introspective models are directly used in conjunction with the former decision making models for action taking. As a result, the augmentation of the CRISP cycle represents a tractable metacognitive model creation and verification process.

5 See http://www.crisp-dm.org.