Behaviour TP FP FN Recall Precision leaving object 4 0 1 0.8 1 immobile 9 8 0 1 0.52 meeting 6 1 3 0.66 0.85 moving 15 3 2 0.88 0.83 fighting 6 0 0 1 1 activity for 'greeting'.

A particular refinement of CAVIAR's dictionary - the introduction of **a short-term**
activity for abrupt motion - considerably increased LTAR's recognition accuracy. More precisely, compared to our earlier results [2, 3], the introduction of **abrupt motion reduced**
the number of FP regarding moving and meeting**. In the original annotation of CAVIAR,** the short-term activities of people fighting **were sometimes classified as walking or active. In** the first case LTAR incorrectly recognised moving**, because two people were walking while** being 'close' to each other, while in the second case LTAR incorrectly recognised **meeting** (in addition to recognising fighting**), because two people were active while being 'close' to** each other. Labelling the short-term activities of people fighting **as abrupt motion resolved** this issue, because abrupt motion does not initiate moving or **meeting**.

In addition to increasing the recognition accuracy of moving and meeting, the introduction of abrupt motion eliminated FP and FN regarding fighting**. Moreover, the introduction**
of abrupt motion did not increase FP or FN in the recognition of the other long-term activities.

Similar to introducing abrupt motion, we could have enhanced CAVIAR's dictionary by including activities for greeting a person, falling on the floor, etc, and variables explicitly representing that a tracked entity is an object. We did not do this because we are not aware of any short-term activity recognition systems that detect such **activities and explicitly**
represent the aforementioned type of information. In contrast, there are systems that detect abrupt motion - eg, see [8]. We expect that a finer classification of short-term activities and the addition of context variables such as the one mentioned above, will, under certain circumstances, increase the overall activity recognition accuracy, provided that the long-term activity definitions are updated accordingly.

We should like to point out that the issues identified above do not always compromise recognition accuracy. Eg, the lack of explicit information that a tracked entity is an object did not the affect the recognition accuracy of leaving object **in the 28 CAVIAR videos. This** lack of information would have led to FP in the recognition of leaving object **in certain**
conditions, but these conditions did not arise in the CAVIAR videos. Similarly, the lack of consistency in the annotation of activities, and the incompleteness of short-term activity narratives do not always lead to FP or FN. In any case, in the next section we discuss ways of addressing these issues.

Concerning recognition efficiency, we were able to recognise each long-term activity in less than 1 second CPU time, given as input around 1800 temporally sorted short-term activities representing, on average, a CAVIAR video, on an Intel Core i7 920@2.67GHz with 6 GB RAM running Linux Kernel 2.6. Ways to further improve recognition efficiency are presented in the following section.