  Nearest Neighbor Search (NNS) has recently drawn a rapid increase of interest
due to its core role in managing high-dimensional vector data in data science
and AI applications. The interest is fueled by the success of neural embedding,
where deep learning models transform unstructured data into semantically
correlated feature vectors for data analysis, e.g., recommend popular items.
Among several categories of methods for fast NNS, similarity graph is one of
the most successful algorithmic trends. Several of the most popular and
top-performing similarity graphs, such as NSG and HNSW, at their core employ
best-first traversal along the underlying graph indices to search near
neighbors. Maximizing the performance of the search is essential for many
tasks, especially at the large-scale and high-recall regime. In this work, we
provide an in-depth examination of the challenges of the state-of-the-art
similarity search algorithms, revealing its challenges in leveraging multi-core
processors to speed up the search efficiency. We also exploit whether
similarity graph search is robust to deviation from maintaining strict order by
allowing multiple walkers to simultaneously advance the search frontier. Based
on our insights, we propose Speed-ANN, a parallel similarity search algorithm
that exploits hidden intra-query parallelism and memory hierarchy that allows
similarity search to take advantage of multiple CPU cores to significantly
accelerate search speed while achieving high accuracy.
  We evaluate Speed-ANN on a wide range of datasets, ranging from million to
billion data points, and show its shorter query latency than NSG and HNSW,
respectively. Besides, with multicore support, we show that our approach offers
faster search latency than highly-optimized GPU implementation and provides
good scalability as the increase of the number of hardware resources (e.g., CPU
cores) and graph sizes.
