{nCS1, nCS2, nCS4, nCS5, nCS6, nCS7, nCS10, nCS11, n**CS12**}
for the model (A) in Fig.1.

In the second step, the observation dataset {di} is generated by deleting the covert nodes in C from the patterns {δi}.

The example δ0 results in the observation d0 = δ0 ∩ C =
{nCS1, nCS2, nCS4, nCS5, nCS6, nCS7, nCS11, n**CS12**} if the experimental condition is that C = {n**CS10**}.

The covert node in C may appear multiple times in the communication patterns {δi}. The number of the target observation data to identify is given by Dt =PD−1 i=0 B(di 6= δi).

The function B(s) returns 1 if the statement s is true and 0 otherwise. A few conditions are assumed in the performance evaluation in V for simplicity. At first, the probability fj does not depend on the nodes (fj = 1/|O∪C|). Second, the value of the probability rij is either 0 or 1. The number of the possible communication patterns is bounded (less than or equal to the number of nodes N). Finally, the influence transmission is bidirectional (rjk = rkj ).

## V. Performance Evaluation A. Performance Measure

Precision, recall, and F measure are used as a measure of the performance. In information retrieval (such as search, document classification, and query classification), the precision p is used as evaluation criteria, which is the fraction of the number of relevant data to the number of the all data retrieved by search. The recall r is the fraction of the number of the data retrieved by search to the number of the all relevant data.

The relevant data refers to the data where di 6= δi. They are given by eq.(17) and eq.(18) They are functions of the number of the retrieved data Dr. It can take the value from 1 to D.

The data is retrieved in the order of dσ(1), dσ(2), to dσ(Dr).

$$p(D_{\rm r})=\frac{\sum_{i=1}^{D_{\rm r}}B(d_{\sigma(i)}\neq\delta_{\sigma(i)})}{D_{\rm r}}.$$  $r(D_{\rm r})=\frac{\sum_{i=1}^{D_{\rm r}}B(d_{\sigma(i)}\neq\delta_{\sigma(i)})}{D_{\rm t}}.$
$$(17)$$  $$(18)$$

The F measure F is the harmonic mean of the precision and recall [10]. It is given by eq.(19).

$$F(D_{\rm r})=\frac{1}{\frac{1}{2}(\frac{1}{p(D_{\rm r})}+\frac{1}{r(D_{\rm r})})}=\frac{2p(D_{\rm r})r(D_{\rm r})}{p(D_{\rm r})+r(D_{\rm r})}.\tag{19}$$

The precision, recall, and F measure range from 0 to 1. All the measures take larger values as the performance of retrieval becomes better.

## B. Result

The results of the performance evaluation using the test dataset in IV-B derived from the network models in IV-A are demonstrated.

Let's start with the first class of the network models (real organization) and learn the implication of the method. Fig.3 shows the precision (p), recall (r), and F measure (F) in the trial where the experimental condition is that the node n**CS10** in the model (A) is the target covert node to discover
(C = {nCS10}, |C| = 1, N = |O| **= 106**). The horizontal axis is the rate of the number of the retrieved data (Dr) to the number of the whole data (D). The vertical solid line indicates the position at Dr = Dt. The broken lines indicate the theoretical limit (upper bound) and the random retrieval limit (lower bound). The evaluation is under the condition where the all possible communication patterns are known.

The precision, recall, and F measure are the same value of 0.78 at Dr = Dt. These are much better than those of the random retrieval (F(Dt**) = 0**.04) and close to the theoretical limit. The method fails to discover two suspicious records δi ={nCS10, n**CS11**}, and {nCS10, n**CS12**} when Dr is small.

This indicates that the communication with the nodes having small nodal degree (K(nCS11**) = 1** and K(nCS12**) = 1**)
does not provide much clues for node discovery. On the other hand, the most suspicious observation data dσ(1) includes all the neighbor nodes nCS1, nCS2, nCS4, nCS5, nCS6, nCS7, n**CS11**, and n**CS12**. The method succeeded in discovering most of the suspicious records and the all suspicious nodes. The investigators will decide to collect more detailed information on the communication (and a resulting collaborative activity) of the suspicious neighbor nodes. This will result in identifying, locating, and finally, capturing the covert leader
(C = {n**CS10**}) who is responsible for many terrorism attacks.

Let's move on to the second class of the network model
(mathematical model with adjustable parameters) and study the extensive performance characteristics of the method. Fig.4 shows the F measure F(Dt) as a function of the nodal degree K. Individual plots shows the F measure averaged over the trials where the experimental condition is that a node having a given nodal degree K(ni) = K is the target covert node to discover (N = |O| = 200, |C| = 1). The solid line graphs (a), (b), and (c) are for the model (B), (C), and (D).

The broken lines indicate the theoretical limit and the random retrieval limit. The evaluation is under the condition where the all possible communication patterns are known in Fig.4 through Fig.6. The resulting F measure ranges from 0.7 to 1. It does not depend on the number of groups (or topology of the network model). The performance becomes better as the nodal degree of the target covert nodes increases.

Fig.5 shows the F measure F(Dt) as a function of the number of groups G in the trial where the largest hub is the target covert node to discover (N = |O| = 200, |C| = 1). The horizontal axis is G/N. The number of the nodes is constant. The group contrast parameter is fixed at η(G − **1) = 400**
regardless of the value of G. The broken lines indicate the theoretical limit and the random retrieval limit. The F measure degrades down to 0.7 around G = 0.5N. But the performance still remains much better than that of the random retrieval. The method can be applied for any value of G.

Fig.6 shows the F measure F(Dt) as a function of the number of overt nodes (N = |O|) in the trial where the largest hub is the target covert node to discover (|C| = 1).

The number of the groups is constant (G = 1). The broken lines indicate the theoretical limit and the random retrieval limit. Except the case N **= 50**, the performance remains