  English proficiency assessments have become a necessary metric for filtering
and selecting prospective candidates for both academia and industry. With the
rise in demand for such assessments, it has become increasingly necessary to
have the automated human-interpretable results to prevent inconsistencies and
ensure meaningful feedback to the second language learners. Feature-based
classical approaches have been more interpretable in understanding what the
scoring model learns. Therefore, in this work, we utilize classical machine
learning models to formulate a speech scoring task as both a classification and
a regression problem, followed by a thorough study to interpret and study the
relation between the linguistic cues and the English proficiency level of the
speaker. First, we extract linguist features under five categories (fluency,
pronunciation, content, grammar and vocabulary, and acoustic) and train models
to grade responses. In comparison, we find that the regression-based models
perform equivalent to or better than the classification approach. Second, we
perform ablation studies to understand the impact of each of the feature and
feature categories on the performance of proficiency grading. Further, to
understand individual feature contributions, we present the importance of top
features on the best performing algorithm for the grading task. Third, we make
use of Partial Dependence Plots and Shapley values to explore feature
importance and conclude that the best performing trained model learns the
underlying rubrics used for grading the dataset used in this study.
