  Homomorphic Encryption (HE) is an emerging encryption scheme that allows
computations to be performed directly on encrypted messages. This property
provides promising applications such as privacy-preserving deep learning and
cloud computing. Prior works have been proposed to enable practical
privacy-preserving applications with architectural-aware optimizations on CPUs,
GPUs and FPGAs. However, there is no systematic optimization for the whole HE
pipeline on Intel GPUs. In this paper, we present the first-ever SYCL-based GPU
backend for Microsoft SEAL APIs. We perform optimizations from instruction
level, algorithmic level and application level to accelerate our HE library
based on the Cheon, Kim, Kimand Song (CKKS) scheme on Intel GPUs. The
performance is validated on two latest Intel GPUs. Experimental results show
that our staged optimizations together with optimizations including low-level
optimizations and kernel fusion accelerate the Number Theoretic Transform
(NTT), a key algorithm for HE, by up to 9.93X compared with the na\"ive GPU
baseline. The roofline analysis confirms that our optimized NTT reaches 79.8%
and85.7% of the peak performance on two GPU devices. Through the highly
optimized NTT and the assembly-level optimization, we obtain 2.32X - 3.05X
acceleration for HE evaluation routines. In addition, our all-together
systematic optimizations improve the performance of encrypted element-wise
polynomial matrix multiplication application by up to 3.10X.
