  We present algorithmic improvements to the loading operations of certain
reduced data ensembles produced from neutron scattering experiments at Oak
Ridge National Laboratory (ORNL) facilities. Ensembles from multiple
measurements are required to cover a wide range of the phase space of a sample
material of interest. They are stored using the standard NeXus schema on
individual HDF5 files. This makes it a scalability challenge, as the number of
experiments stored increases in a single ensemble file. The present work
follows up on our previous efforts on data management algorithms, to address
identified input output (I/O) bottlenecks in Mantid, an open-source data
analysis framework used across several neutron science facilities around the
world. We reuse an in-memory binary-tree metadata index that resembles data
access patterns, to provide a scalable search and extraction mechanism. In
addition, several memory operations are refactored and optimized for the
current common use cases, ranging most frequently from 10 to 180, and up to 360
separate measurement configurations. Results from this work show consistent
speed ups in wall-clock time on the Mantid LoadMD routine, ranging from 19\% to
23\% on average, on ORNL production computing systems. The latter depends on
the complexity of the targeted instrument-specific data and the system I/O and
compute variability for the shared computational resources available to users
of ORNL's Spallation Neutron Source (SNS) and the High Flux Isotope Reactor
(HFIR) instruments. Nevertheless, we continue to highlight the need for more
research to address reduction challenges as experimental data volumes, user
time and processing costs increase.
