Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Annotating Derivations: A New Evaluation Strategy and Dataset for
  Algebra Word Problems   We propose a new evaluation for automatic solvers for algebra word problems,
which can identify mistakes that existing evaluations overlook. Our proposal is
to evaluate such solvers using derivations, which reflect how an equation
system was constructed from the word problem. To accomplish this, we develop an
algorithm for checking the equivalence between two derivations, and show how
derivation an- notations can be semi-automatically added to existing datasets.
To make our experiments more comprehensive, we include the derivation
annotation for DRAW-1K, a new dataset containing 1000 general algebra word
problems. In our experiments, we found that the annotated derivations enable a
more accurate evaluation of automatic solvers than previously used metrics. We
release derivation annotations for over 2300 algebra word problems for future
evaluations.

9
The categories of this paper are: nlin
21
The categories of this paper are: adap-org
8
The categories of this paper are: hep-ex
13
The categories of this paper are: quant-ph
35
The categories of this paper are: mtrl-th
12
The categories of this paper are: nucl-th
37
The categories of this paper are: supr-con
29
The categories of this paper are: funct-an
10
The categories of this paper are: q-bio
27
The categories of this paper are: comp-gas
0
The categories of this paper are: hep-ph
32
The categories of this paper are: atom-ph
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
15
The categories of this paper are: nucl-ex
34
The categories of this paper are: chem-ph