  Homomorphic encryption (HE) enables computation on encrypted data, and hence
it has a great potential in privacy-preserving outsourcing of computations to
the cloud. Hardware acceleration of HE is crucial as software implementations
are very slow. In this paper, we present design methodologies for building a
programmable hardware accelerator for speeding up the cloud-side homomorphic
evaluations on encrypted data. First, we propose a divide-and-conquer technique
that enables homomorphic evaluations in a large polynomial ring $R_{Q,2N}$ to
use a hardware accelerator that has been built for the smaller ring $R_{Q,N}$.
The technique makes it possible to use a single hardware accelerator flexibly
for supporting several HE parameter sets. Next, we present several
architectural design methods that we use to realize the flexible and
instruction-set accelerator architecture, which we call `Medha'. At every level
of the implementation hierarchy, we explore possibilities for parallel
processing. Starting from hardware-friendly parallel algorithms for the basic
building blocks, we gradually build heavily parallel RNS polynomial arithmetic
units. Next, many of these parallel units are interconnected elegantly so that
their interconnections require the minimum number of nets, therefore making the
overall architecture placement-friendly on the platform. For Medha, we take a
memory-conservative design approach and get rid of any off-chip memory access
during homomorphic evaluations. Finally, we implement Medha in a Xilinx Alveo
U250 FPGA and measure timing performances of the microcoded homomorphic
addition, multiplication, key-switching, and rescaling for the leveled HE
scheme RNS-HEAAN at 200 MHz clock frequency. For two large parameter sets,
Medha achieves accelerations by up to 68x and 78x times respectively compared
to a highly optimized software implementation Microsoft SEAL running at 2.3
GHz.
