Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Harvest -- An Open Source Toolkit for Extracting Posts and Post Metadata
  from Web Forums   Automatic extraction of forum posts and metadata is a crucial but challenging
task since forums do not expose their content in a standardized structure.
Content extraction methods, therefore, often need customizations such as
adaptations to page templates and improvements of their extraction code before
they can be deployed to new forums. Most of the current solutions are also
built for the more general case of content extraction from web pages and lack
key features important for understanding forum content such as the
identification of author metadata and information on the thread structure.
  This paper, therefore, presents a method that determines the XPath of forum
posts, eliminating incorrect mergers and splits of the extracted posts that
were common in systems from the previous generation. Based on the individual
posts further metadata such as authors, forum URL and structure are extracted.
We also introduce Harvest, a new open source toolkit that implements the
presented methods and create a gold standard extracted from 52 different Web
forums for evaluating our approach. A comprehensive evaluation reveals that
Harvest clearly outperforms competing systems.

28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
30
The categories of this paper are: q-alg
37
The categories of this paper are: supr-con
21
The categories of this paper are: adap-org
12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
15
The categories of this paper are: nucl-ex
5
The categories of this paper are: gr-qc
31
The categories of this paper are: ao-sci
36
The categories of this paper are: cmp-lg
3
The categories of this paper are: physics
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
26
The categories of this paper are: bayes-an