Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Evaluating Automatic Difficulty Estimation of Logic Formalization
  Exercises   Teaching logic effectively requires an understanding of the factors which
cause logic students to struggle. Formalization exercises, which require the
student to produce a formula corresponding to the natural language sentence,
are a good candidate for scrutiny since they tap into the students'
understanding of various aspects of logic. We correlate the difficulty of
formalization exercises predicted by a previously proposed difficulty
estimation algorithm with two empirical difficulty measures on the Grade
Grinder corpus, which contains student solutions to FOL exercises. We obtain a
moderate correlation with both measures, suggesting that the said algorithm
indeed taps into important sources of difficulty but leaves a fair amount of
variance uncaptured. We conduct an error analysis, closely examining exercises
which were misclassified, with the aim of identifying additional sources of
difficulty. We identify three additional factors which emerge from the
difficulty analysis, namely predicate complexity, pragmatic factors and
typicality of the exercises, and discuss the implications of automated
difficulty estimation for logic teaching and explainable AI.

36
The categories of this paper are: cmp-lg
18
The categories of this paper are: econ
9
The categories of this paper are: nlin
8
The categories of this paper are: hep-ex
12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
31
The categories of this paper are: ao-sci
32
The categories of this paper are: atom-ph
20
The categories of this paper are: acc-phys
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
35
The categories of this paper are: mtrl-th
13
The categories of this paper are: quant-ph
14
The categories of this paper are: math-ph
33
The categories of this paper are: plasm-ph
37
The categories of this paper are: supr-con