Modeling Social Annotation: a Bayesian Approach · 13

| Seed               | # Resources   | # Users   | # Tags   | #Tripples   |
|--------------------|---------------|-----------|----------|-------------|
| Flytecomm          | 3,562         | 34,594    | 14,297   | 2,284,308   |
| Geocoder           | 5,572         | 46,764    | 16,887   | 3,775,832   |
| Wunderground       | 7,176         | 45,852    | 77,056   | 6,327,211   |
| Whitepages         | 6,455         | 12,357    | 64,591   | 2,843,427   |
| Online-Resevationz | 764           | 41,003    | 9,194    | 162,763     |

Table I. The table presents statistics for five data sets for evaluating models' performance. Note that a triple is a resource, user, and tag co-occurrence.

—For each resource, collect all bookmarks (resource-user-tag triples). We wrote a special-purpose page scraper to extract this information from *Delicious*. In principle, we could continue to expand the collection of resources by gathering tags and retrieving more resources tagged with those keywords, but in practice, even after a small traversal, we already obtain millions of triples. In each corpus, each resource has at least one tag in common with the seed. Statistics on these data sets are presented in Table I.

For each corpus, LDA is trained with 80 topics, while the number of topics and interests for ITM is set to 80 and 40 respectively. The topic and interest assignments are randomly initialized, and then both models are trained with the 500 iterations.12 For the last 100 iterations, we use the topic and interest assignments, in each iteration, to compute the distributions of resources over topics, φ. The average of φ in this period is then used as the distributions of resources over topics.

Next, the learned distributions of resources over topics, φ, are used to compute the similarity of resources in each corpus to the seed. The performance of each model is evaluated by manually checking the 100 most similar resources produced by the model. A resource is judged to be similar if it provides an input form that takes semantically the same inputs as the seed and returns semantically the same data. Hence, *flightaware*13 is judged similar to *flytecomm* because both take flight information and return flight status.

Figure 5 shows the number of relevant resources identified within the top x resources returned by LDA and ITM. From the results, we can see that ITM is superior to LDA in three data sets: flytecomm, *geocoder* and *online-reservationz*. However, its performance for *wunderground* and *whitepages* is about the same as that of LDA. Although we have no empirical proof, we hypothesize that weather and directory services are of interest to all users, and are therefore bookmarked by a large variety of users, unlike users interested in tracking flights or booking hotels online. As a result, ITM cannot exploit individual user differences to learn more accurate topics φ in the *wunderground* and *whitepages* cases.

To illustrate utility of ITM, we select examples of topics and interests of the model induced from the *flytecomm* corpus. For purposes of visualization, we first list in descending order the top tags that are highly associated with each *topic*, which are obtained from θz
(aggregated over all interests in the topic z). For each topic, we then enumerate some *interests*, and present a list of top tags for each interest, obtained from θx,z. We manually label topics and interests (in *italics*) according to the meaning of its dominant tags.

Travel & Flights topic: travel, Travel, flights, airfare, airline, flight, airlines, guide, aviation, hotels, deals, 12We discovered that the model converging very quickly. In particular, the model appear to reach the stable state within 300 iterations in all data sets 13http://flightaware.com/live/
ACM Journal Name, Vol. x, No. y, zz 2010.