  Systems exhibiting nonlinear dynamics, including but not limited to chaos,
are ubiquitous across Earth Sciences such as Meteorology, Hydrology, Climate
and Ecology, as well as Biology such as neural and cardiac processes. However,
System Identification remains a challenge. In climate and earth systems models,
while governing equations follow from first principles and understanding of key
processes has steadily improved, the largest uncertainties are often caused by
parameterizations such as cloud physics, which in turn have witnessed limited
improvements over the last several decades. Climate scientists have pointed to
Machine Learning enhanced parameter estimation as a possible solution, with
proof-of-concept methodological adaptations being examined on idealized
systems. While climate science has been highlighted as a "Big Data" challenge
owing to the volume and complexity of archived model-simulations and
observations from remote and in-situ sensors, the parameter estimation process
is often relatively a "small data" problem. A crucial question for data
scientists in this context is the relevance of state-of-the-art data-driven
approaches including those based on deep neural networks or kernel-based
processes. Here we consider a chaotic system - two-level Lorenz-96 - used as a
benchmark model in the climate science literature, adopt a methodology based on
Gaussian Processes for parameter estimation and compare the gains in predictive
understanding with a suite of Deep Learning and strawman Linear Regression
methods. Our results show that adaptations of kernel-based Gaussian Processes
can outperform other approaches under small data constraints along with
uncertainty quantification; and needs to be considered as a viable approach in
climate science and earth system modeling.
