  Deep learning (DL) models, especially those large-scale and high-performance
ones, can be very costly to train, demanding a great amount of data and
computational resources. Unauthorized reproduction of DL models can lead to
copyright infringement and cause huge economic losses to model owners. Existing
copyright protection techniques are mostly based on watermarking, which embeds
an owner-specified watermark into the model. While being able to provide exact
ownership verification, these techniques are 1) invasive, as they need to
tamper with the training process, which may affect the utility or introduce new
security risks; 2) prone to adaptive attacks that attempt to remove the
watermark; and 3) not robust to the emerging model extraction attacks. Latest
fingerprinting work, though being non-invasive, also falls short when facing
the diverse and ever-growing attack scenarios. In this paper, we propose a
novel testing framework for DL copyright protection: DEEPJUDGE. DEEPJUDGE
quantitatively tests the similarities between two DL models: a victim model and
a suspect model. It leverages a diverse set of testing metrics and test case
generation methods to produce a chain of supporting evidence to help determine
whether a suspect model is a copy of the victim model. Advantages of DEEPJUDGE
include: 1) non-invasive, as it works directly on the model and does not tamper
with the training process; 2) efficient, as it only needs a small set of test
cases and a quick scan of models; 3) flexible, as it can easily incorporate new
metrics or generation methods to obtain more confident judgement; and 4) fairly
robust to model extraction and adaptive attacks. We verify the effectiveness of
DEEPJUDGE under typical copyright infringement scenarios, including model
finetuning, pruning and extraction, via extensive experiments on both image and
speech datasets with a variety of model architectures.
