\section{Introduction\label{sct::intro}}
Symmetric, public-key (asymmetric) and hash-based cryptography constitute a fundamental pillar of modern cryptography.
Symmetric cryptography includes symmetric-key encryption, where a shared secret key is used for both encryption and decryption. Cryptographic hash functions map arbitrarily long strings to strings of a fixed finite length. Currently deployed public-key schemes are
used to establish a common secret key between two remote parties. They are based on factoring large numbers or solving the discrete logarithm problem over a finite group. For more details about modern cryptography the interested reader can consult one of the many excellent references on the topic, e.g.~\cite{Katz:2007:IMC:1206501}.

In contrast to asymmetric schemes based on factoring or solving the discrete logarithm problem and which are completely broken by a quantum adversary via Shor's algorithm~\cite{SJC.26.1484}, symmetric schemes and hash functions are less vulnerable to quantum attacks. The best known quantum attacks against them are based on Grover's quantum search algorithm~\cite{PhysRevLett.79.325}, which offers a quadratic speedup compared to classical brute force searching. Given a search space of size $N$, Grover's algorithm finds, with high probability, an element $x$ for which a certain property such as $f(x)=1$ holds, for some function $f$ we know how to evaluate (assuming such a solution exists). The algorithm evaluates $f$ a total of $\mathcal{O}(\sqrt{N})$ times. It applies a simple operation in between the evaluations of $f$, so the $\mathcal{O}(\sqrt{N})$ evaluations of $f$ account for most of the complexity. In contrast, any classical algorithm that evaluates $f$ in a similar ``black-box'' way requires on the order of $N$ evaluations of $f$ to find such an element.

Any quantum algorithm can be mapped to a quantum circuit, which can be implemented on a quantum computer. The quantum circuit represents what we call the ``logical layer". Such a circuit can always be decomposed in a sequence of ``elementary
gates", such as Clifford gates (CNOT, Hadamard etc.~\cite{NC00}) augmented by a non-Clifford gate such as the T gate.

Running a logical circuit on a full fault-tolerant quantum computer is highly non-trivial. The sequence of logical gates have to be mapped to
sequences of surface code measurement cycles (see e.g.~\cite{PhysRevA.86.032324} for extensive details). By far, the most resource-consuming (in
terms of number of qubits required and time) is the T gate\footnote{Clifford gates are ``cheap", i.e. they require relatively small overhead for implementation in the surface code, but are not universals, hence a non-Clifford gate is required. One such gate is the T gate. There are other possible choices, however all of the non-Clifford gates require special techniques such as magic state distillation~\cite{1367-2630-14-12-123011,PhysRevA.86.052329} and significant overhead (order of magnitudes higher than Clifford gates) to be implemented in the surface code. In fact, to a first order approximation, for the purpose of resource estimation, one can simply ignore the overhead introduced by the Clifford gates and simply focus only on the T gates.}.
In comparison with surface code defects and braiding techniques~\cite{PhysRevA.86.032324}, novel lattice surgery
techniques~\cite{2018arXiv180806709F,1808.02892,1367-2630-14-12-123011} reduce the spatial overhead required for implementing T gates via magic state distillation by approximately a factor of 5, while also modestly improving the running time.

In this paper we first analyze the security of symmetric schemes and hash functions against large-scale fault-tolerant quantum adversaries, using surface code defects and braiding techniques. We take into account the time-space trade-offs with parallelizing quantum search, down to the fault-tolerant layer. Naively, one might hope that $K$ quantum computers (or quantum ``processors'', as we will call them later in the paper) running in parallel reduce the number the circuit depth down to $\mathcal{O}(\sqrt{N})/K$ steps, similar to the classical case of distributing a search space across $K$ classical processors. However quantum searching does not parallelize so well, and the required number of steps
for parallel quantum searching is of the order $\mathcal{O}(\sqrt{N/K})$~\cite{quantph.9711070}. This is a factor of $\sqrt{K}$ larger than $\mathcal{O}(\sqrt{N})/K$ . As shown in~\cite{quantph.9711070}, the optimal way of doing parallel quantum search is to partition the search space into $N/K$ parts, and to perform independent quantum searches on each part.

Secondly, we investigate the security of public-key cryptographic schemes such as RSA and ECC against
quantum attacks, using the latest developments in theory of fault-tolerant quantum error correction, i.e. novel lattice surgery
techniques~\cite{2018arXiv180806709F,1808.02892,1367-2630-14-12-123011}.

The remainder of this paper is organized as follows. In Sec.~\ref{sct::method}, we provide an overview of the methodology used in our analysis. In Sec.~\ref{sct::ciphers} we investigate the security of the AES family of modern symmetric ciphers. In Sec.~\ref{sct::hash} we analyze the security of the SHA family of hash functions. In Sec.~\ref{sct::bitcoin} we investigate the security of Bitcoin's~\cite{satoshi:bitcoin} proof-of-work consensus mechanism. We conclude our investigation of symmetric and hash-based cryptographic schemes in Sec.~\ref{sct::intrinsic_parallel_grover}, where we evaluate the intrinsic cost of running the Grover algorithm with a trivial oracle (i.e., an oracle with a unit cost of 1 for each invocation).

In the subsequent sections  we analyze public-key cryptographic schemes. In Sec.~\ref{sct::rsa} and Sec.~\ref{sct::ecc} we examine the most common public-key establishment schemes, such as RSA and ECC, respectively. In the subsequent sections  we analyze public-key cryptographic schemes. In Sec.~\ref{sct::rsa} and Sec.~\ref{sct::ecc} we examine the most common public-key establishment schemes, such as RSA and ECC, respectively. Finally we summarize our findings and conclude in Sec.~\ref{sct::conclusion}.
\section{Methodology\label{sct::method}}

\subsection{Symmetric cryptography and hash functions\label{sct::symmetric}}
The methodology, sketched in Fig.~\ref{fgr:flowchart_lite} and Fig.~\ref{fgr:full_algorithm}, follows the same lines as the one described in great detail in our earlier paper~\cite{10.1007/978-3-319-69453-5_18}, which we refer the interested reader to for more details.
\begin{figure}[htb]
	\centering
         \includegraphics[width=0.35\textwidth]{figures/flowchart_lite.pdf}
         \caption{Analyzing an attack against a symmetric cryptographic function with a fault-tolerant quantum adversary. Our resource estimation methodology takes into account several of the layers between the high level description of an algorithm and the physical hardware required for its execution. Our approach is modular should assumptions about any of these layers change, and hence it allows one to calculate the impact of improvements in any particular layer.}
         \label{fgr:flowchart_lite}
\end{figure}
\begin{figure}
	\centering
	 \includegraphics[width=0.46\textwidth]{figures/grover_vertical.pdf}
          \caption{Grover searching with an oracle for $f : \{0,1\}^k \rightarrow \{0,1\}^k$. The algorithm makes $\lfloor \frac{\pi}{4} 2^{N/2}\rfloor$ calls to
$G$, the \emph{Grover iteration}, or, if parallelized on $K$ processors, $\lfloor \frac{\pi}{4} 2^{N/(2K)}\rfloor$ calls to $G$. The Grover iteration has two
subroutines. The first, $U_g$, implements the predicate $g : \{0,1\}^k
\rightarrow \{0,1\}$ that maps $x$ to $1$ if and only if $f(x) = y$. Each call to $U_g$ involves two calls to a reversible implementation of $f$ and one call to a comparison circuit that checks whether $f(x) = y$.}
          \label{fgr:full_algorithm}
\end{figure}

We assume a surface-code based fault-tolerant architecture~\cite{PhysRevA.86.032324}, using Reed-Muller distillation schemes~\cite{Fowler:2013aa}. For each scheme we vary the possible physical error rates per gate from $10^{-4}$ to $10^{-7}$. We believe that this range of physical error rates is wide enough to cover both first generation quantum computers as well as more advanced future machines.
In comparison to surface code defects and braiding methods~\cite{PhysRevA.86.032324}, lattice surgery
techniques~\cite{2018arXiv180806709F,1808.02892,1367-2630-14-12-123011} mostly impact the physical footprint of the fault-tolerant layer required to run a specific quantum algorithm, reducing the distillation overhead by approximately a factor of 5. The temporal overhead (i.e. the number of surface code cycles) is reduced less drastically. For this reason, lattice surgery has less significant effects in estimating the security of symmetric schemes or hash functions, reducing the security parameter\footnote{The security parameter is defined as the logarithm base two of the number of fundamental operations (in our case surface code cycles) required to break the scheme.} by at most 1 and decreasing the spatial overhead by at most a factor of 5. Therefore when estimating the security of symmetric and hash-based cryptographic schemes we use surface code defects and braiding techniques.

For each cryptographic primitive, we display four plots, in the following order:
\begin{enumerate}
\item We plot the total number of surface code cycles per CPU (where a CPU is a quantum computer capable of executing a single instance of Grover's quantum search algorithm) as a function of the number of CPUs. We directly tie the quantum security parameter to the total number of surface code cycles (see~\cite{10.1007/978-3-319-69453-5_18} for more details). We also add to the plot the theoretical lower bound achievable by quantum search in the cases of: a) considering the oracle a black box of unit cost (lower line), and b) considering the oracle as composed of ideal quantum gates, each of unit cost (upper line). Note that the difference between b) and a) represents the intrinsic cost of logical overhead (i.e. the overhead introduced by treating the oracle as a logical circuit and not a blackbox), whereas the difference between the upper lines and b) represents the intrinsic cost introduced by the fault-tolerant layer.

\item We plot the total wall-time per CPU (i.e. how long will the whole computation take on a parallel quantum architecture) as a function of the number of CPUs. The horizontal dashed line represents the one-year time line, i.e. the $x$ coordinate of the intersection point between the ``Total time per CPU'' line and the one-year time line provides the number of processors required to break the system within one year (in $\log_2$ units).

\item We plot the total physical footprint (number of qubits) per CPU, as a function of the number of CPUs.
\item Finally we plot the total physical footprint (number of qubits) of all quantum search machines (CPUs) running in parallel.
\end{enumerate}

In the following sections we proceed to analyze symmetric ciphers (AES, Sec.~\ref{sct::ciphers}), hash functions (SHA-256, SHA3-256, Sec.~\ref{sct::hash}, Bitcoin's hash function, Sec.~\ref{sct::bitcoin}), and finally the minimal resources required for running Grover's algorithm with a trivial oracle~\ref{sct::intrinsic_parallel_grover} (e.g. the identity gate) on search spaces of various sizes.

Note that in some ranges of the plots from sections~\ref{sct::ciphers},~\ref{sct::hash},~\ref{sct::intrinsic_parallel_grover} and~\ref{sct::bitcoin} the total physical footprint increases slightly with the number of processors, which may seem counter-intuitive. This happens due to the fact that with more processors the required code distances decrease, and in some instances one can pipeline more magic states factories in parallel into the surface code, which in effect causes an increase in the overall physical footprint. Note that the total time per CPU is monotonically decreasing, as parallelizing distilleries does not increase the wall time. For more details see~\cite{10.1007/978-3-319-69453-5_18}.

\subsection{Public-key cryptography\label{sct::pk}}

Most of the recent progress in quantum cryptanalysis is related to the fault-tolerant layer in Fig.~\ref{fgr:flowchart_lite}. New methods and techniques
based on surface code lattice surgery~\cite{2018arXiv180806709F,1808.02892,1367-2630-14-12-123011} allow a significant decrease of the overall
footprint (number of qubits, or space) taken by the quantum computation, and also a relatively modest decrease in time, in comparison with  methods based on surface code defects and braiding~\cite{PhysRevA.86.032324,Fowler:2013aa}.

We consider the best up-to-date optimized logical quantum circuits for attacking RSA and ECC public-key
schemes~\cite{1706.06752,PhysRevA.52.3457,cuccaro04,Beauregard:2003:CSA:2011517.2011525} then perform a physical footprint resource estimation
analysis using lattice surgery techniques. We remark that the overall time required to run the algorithm depends on the level of parallelization
for the magic state factories\footnote{Every T gate in the circuit must be implemented by a specialized magic state factory, each of which occupies a
significant physical footprint. One can implement more magic states in parallel if one is willing to increase the physical footprint of the computation.}.

For each public-key cryptogrpric scheme, we analyze the space/time tradeoffs and plot the results on a double logarithmic scale. We fit the data using a third degree
polynomial\footnote{A third degree polynomial fits the data very precisely, providing a coefficient of determination $R^2$ greater than 0.997.} and obtain an analytical closed-form formula for the relation between the time and the number of qubits required to attack the scheme, in
the form

\begin{equation}\label{eqn1}
y(x) = \alpha x^3 + \beta x^2 + \gamma x + \delta,
\end{equation}
where $y$ represents logarithm base 2 of the number of qubits and $x$ represents the logarithm base 2 of the time (in seconds). For example,
the quantity
\begin{equation}\label{eqn2}
y\left(\log_2(24\times 3600)\right) \approx y(16.3987)
\end{equation}
represents how many qubits are required to break the scheme in one day (24 hours) for a fixed physical error rate per gate $p_g$, assuming a
surface code cycle time of 200ns. Note that the computation time scales linearly with the surface code cycle time, e.g. a 1000ns surface code cycle
time will result in a computation that is 5 times longer than a $200ns$ surface code cycle time. Therefore, for a specific cryptographic scheme for
which we plotted the space/time tradeoffs using a surface code cycle time of $200ns$ and a fixed physical error rate per gate $p_g$, the number of
qubits required to break a specific scheme in a time $t$ using an alternative surface code cycle time $t_c$ is given by

\begin{equation}\label{eqn3}
y\left(\log_2\left(\frac{200ns}{t_c}t\right)\right),
\end{equation}
where $t$ is expressed in seconds and $t_c$ is expressed in nanoseconds.

We assume a surface code cycle time of 200ns, in conformance with~\cite{PhysRevA.86.032324}. For each scheme we analyze, we compare its security using the more conservative (and realistic in the short term) $p_g=10^{-3}$ and also the more optimistic  $p_g=10^{-5}$. Note that assuming the more optimistic assumption from a quantum computing perspective is the more conservative assumption from a cybersecurity perspective.

Furthermore, in this analysis, we are reporting the full physical footprint, including the memory required for magic state distillation.
Using present-day techniques, the memory required for generating these generic input states accounts for a substantial fraction of the total memory cost and thus we are including these in the total cost estimate and will track the impact of improved methods.

\section{Symmetric ciphers\label{sct::ciphers}}
Below we analyze the security of AES family of symmetric ciphers against large-scale fault-tolerant quantum adversaries. We used the highly optimized logical circuits produced in
\cite{10.1007/978-3-319-29360-8_3}.

\subsection{AES-128}

        \includegraphics[width=0.429\textwidth]{figures/AES-128_cycles.pdf}
      	\captionof{figure}{AES-128 block cipher. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale). The bottom brown line (theoretical lower bound, black box) represents the minimal number of queries required
	by Grover's algorithm, the cost function being the total number of queries to a black-box oracle, each query assumed to have unit cost, and a completely error-free circuit. The purple line (ideal grover, non-black-box) takes into consideration the structure of the oracle, the cost function being the total number of gates in the circuit, each gate having unit cost; the quantum circuit is assumed error-free as well. Both brown and magenta lines are displayed only for comparisons; for both of them, the $y$ axis should be interpreted as number of logical queries (operations, respectively).
The curves above the purple line show the overhead introduced by fault tolerance (in terms of required surface code cycles, each surface code cycle assumed to have unit cost). More optimization at the logical layer will shift the purple line down, whereas more optimization at the fault-tolerant layer will move the upper curves closer to the purple line. Similar remarks to the above hold for the remaining plots in this manuscript.}
      	\label{fgr:aes_128_cycles}

	For example, the plots in Fig.~\ref{fgr:aes_128_cycles} tells us that if we have $2^{50}$ quantum computers running Grover's algorithm in parallel, with no physical errors, then it would take about $2^{63}$ gate calls (where the purple line intersects the vertical line at $50$), where we assume each gate to have unit cost. Still with no errors, a trivial cost for implementing the cryptographic function (oracle) would bring the cost down to about $2^{38}$ oracle calls per quantum computer. Keeping the actual function implementation, but adding the fault-tolerant layer with a physical error rate of $10^{-7}$ (with appropriate assumptions and using state-of-the-art quantum error correction) pushes the cost up to around $2^{76}$ surface code cycles per quantum computer (where now each code cycle is assumed to have unit cost). Similar remarks hold for the remaining plots in this manuscript.
        \includegraphics[width=0.429\textwidth]{figures/AES-128_time.pdf}
      	\captionof{figure}{AES-128 block cipher. Required time per processor, as a function of the  number of processors ($\log_2$ scale). The horizontal dotted line indicates one year. The $x$-axis is deliberately extended to show the necessary number of CPUs for a total time of one year. Thus the figure shows that it would take, with the stated assumptions, over $2^{80}$ parallel quantum searches to break AES-128 in a year. Similar remarks to the above hold for the remaining plots in this manuscript.}
      	\label{fgr:aes_128_time}
        \includegraphics[width=0.429\textwidth]{figures/AES-128_phys.pdf}
	\captionof{figure}{AES-128 block cipher. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:aes_128_phys}
        \includegraphics[width=0.429\textwidth]{figures/AES-128_phys_total.pdf}
	\captionof{figure}{AES-128 block cipher. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:aes_128_phys_total}

\subsection{AES-192}

        \includegraphics[width=0.429\textwidth]{figures/AES-192_cycles.pdf}
      	\captionof{figure}{AES-192 block cipher. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:aes_192_cycles}

        \includegraphics[width=0.429\textwidth]{figures/AES-192_time.pdf}
      	\captionof{figure}{AES-192 block cipher. Required time per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:aes_192_time}

        \includegraphics[width=0.429\textwidth]{figures/AES-192_phys.pdf}
	\captionof{figure}{AES-192 block cipher. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:aes_192_phys}

        \includegraphics[width=0.429\textwidth]{figures/AES-192_phys_total.pdf}
	\captionof{figure}{AES-192 block cipher. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:aes_192_phys_total}

\subsection{AES-256}

        \includegraphics[width=0.429\textwidth]{figures/AES-256_cycles.pdf}
      	\captionof{figure}{AES-256 block cipher. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:aes_256_cycles}

        \includegraphics[width=0.429\textwidth]{figures/AES-256_time.pdf}
      	\captionof{figure}{AES-256 block cipher. Required time per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:aes_256_time}

        \includegraphics[width=0.429\textwidth]{figures/AES-256_phys.pdf}
	\captionof{figure}{AES-256 block cipher. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:aes_256_phys}

        \includegraphics[width=0.429\textwidth]{figures/AES-256_phys_total.pdf}
	\captionof{figure}{AES-256 block cipher. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:aes_256_phys_total}

\section{Hash functions\label{sct::hash}}
In this section we study the effect of parallelized Grover attacks on the SHA-256~\cite{SHA2} snd SHA3-256~\cite{SHA3} family of hash functions. We used the highly optimized logical circuits produced in~\cite{10.1007/978-3-319-69453-5_18}.

\subsection{SHA-256}

        \includegraphics[width=0.429\textwidth]{figures/SHA-256_cycles.pdf}
      	\captionof{figure}{SHA-256 cryptographic hash function. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:sha_256_cycles}
        \includegraphics[width=0.429\textwidth]{figures/SHA-256_time.pdf}
      	\captionof{figure}{SHA-256 cryptographic hash function. Required time per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:sha_256_time}
        \includegraphics[width=0.429\textwidth]{figures/SHA-256_phys.pdf}
	\captionof{figure}{SHA-256 cryptographic hash function. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:sha_256_phys}
        \includegraphics[width=0.429\textwidth]{figures/SHA-256_phys_total.pdf}
	\captionof{figure}{SHA-256 cryptographic hash function. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:sha_256_phys_total}

\subsection{SHA3-256}

        \includegraphics[width=0.429\textwidth]{figures/SHA3-256_cycles.pdf}
      	\captionof{figure}{SHA3-256 cryptographic hash function. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:sha3_256_cycles}
        \includegraphics[width=0.429\textwidth]{figures/SHA3-256_time.pdf}
      	\captionof{figure}{SHA3-256 cryptographic hash function. Required time per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:sha3_256_time}
        \includegraphics[width=0.429\textwidth]{figures/SHA3-256_phys.pdf}
	\captionof{figure}{SHA3-256 cryptographic hash function. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:sha3_256_phys}
        \includegraphics[width=0.429\textwidth]{figures/SHA3-256_phys_total.pdf}
	\captionof{figure}{SHA3-256 cryptographic hash function. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:sha3_256_phys_total}
\section{Bitcoin~\label{sct::bitcoin}}
In this section we analyze the security of Bitcoin's~\cite{satoshi:bitcoin} proof-of-work protocol, which is based on finding a hash\footnote{The hash function being used by the protocol is H($x$) := SHA-256(SHA-256($x$).} pre-image which that starts
with a certain number of zeros. The latter is dynamically adjusted by the protocol so that the problem is on average solved by
the whole network in 10 minutes. Currently, it takes around $2^{75}$ classical hashing operations~\cite{btc_difficulty} for finding a desired hash pre-image successfully via brute-force search with specialized hardware.
        \includegraphics[width=0.429\textwidth]{figures/SHA-256-Bitcoin_cycles.pdf}
      	\captionof{figure}{Bitcoin's cryptographic hash function H($x$) := SHA-256(SHA-256($x$)). Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:sha_256_bitcoin_cycles}
        \includegraphics[width=0.429\textwidth]{figures/SHA-256-Bitcoin_time.pdf}
      	\captionof{figure}{Bitcoin's cryptographic hash function H($x$) := SHA-256(SHA-256($x$)). Required time per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:sha_256_bitcoin_time}
        \includegraphics[width=0.429\textwidth]{figures/SHA-256-Bitcoin_phys.pdf}
	\captionof{figure}{Bitcoin's cryptographic hash function H($x$) := SHA-256(SHA-256($x$)). Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:sha_256_bitcoin_phys}
        \includegraphics[width=0.429\textwidth]{figures/SHA-256-Bitcoin_phys_total.pdf}
	\captionof{figure}{Bitcoin's cryptographic hash function H($x$) := SHA-256(SHA-256($x$)). Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:sha_256_bitcoin_phys_total}

\section{Intrinsic cost of parallelized Grover's algorithm\label{sct::intrinsic_parallel_grover}}

More efficient quantum implementations of AES and SHA imply more efficient cryptanalysis. In this section, we aim to bound how much further optimized implementations of these cryptographic functions could help. We do so by assuming a trivial cost of $1$ for each function evaluation.

\subsection{Searching space of size $2^{56}$}

        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover56bits_cycles.pdf}
      	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{56}$. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_56_cycles}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover56bits_time.pdf}
      	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{56}$. Required time per processor, as a function of the  number of processors ($\log_2$ scale). The dotted horizontal line indicates one year. }
      	\label{fgr:minimal_grover_56_time}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover56bits_phys.pdf}
	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{56}$. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_56_phys}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover56bits_phys_total.pdf}
	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{56}$. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:minimal_grover_56_phys_total}

\subsection{Searching space of size $2^{64}$}

        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover64bits_cycles.pdf}
      	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{64}$. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_64_cycles}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover64bits_time.pdf}
      	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{64}$. Required time per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_64_time}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover64bits_phys.pdf}
	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{64}$. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_64_phys}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover64bits_phys_total.pdf}
	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{64}$. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:minimal_grover_64_phys_total}

\subsection{Searching space of size $2^{128}$}

        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover128bits_cycles.pdf}
      	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{128}$. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_128_cycles}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover128bits_time.pdf}
      	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{128}$. Required time per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_128_time}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover128bits_phys.pdf}
	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{128}$. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_128_phys}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover128bits_phys_total.pdf}
	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{128}$. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:minimal_grover_128_phys_total}

\subsection{Searching space of size $2^{256}$}

        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover256bits_cycles.pdf}
      	\captionof{figure}{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{256}$. Required surface clock cycles per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_256_cycles}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover256bits_time.pdf}
      	\caption{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{256}$. Required time per processor, as a function of the  number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_256_time}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover256bits_phys.pdf}
	\caption{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{256}$. Physical footprint (physical qubits) per processor, as a function of the number of processors ($\log_2$ scale).}
      	\label{fgr:minimal_grover_256_phys}
        \includegraphics[width=0.429\textwidth]{figures/MinimalGrover256bits_phys_total.pdf}
	\caption{Running Grover's algorithm with a trivial oracle, for a searching space of size $2^{256}$. Total physical footprint (physical qubits), as a function of the number of processors ($\log_2$ scale). Note that the qubits are not correlated across processors.}
      	\label{fgr:minimal_grover_256_phys_total}

\section{RSA schemes\label{sct::rsa}}
In the following section we compute the space/time tradeoffs for attacking public-key cryptographic schemes based on factoring large numbers,
namely RSA-1024, RSA-2048, RSA-3072, RSA-4096, RSA-7680 and RSA-15360.
For each scheme, we plot the space/time tradeoff points then fit it with a third degree polynomial, for $p_g=10^{-3}$ and $p_g=10^{-5}$, respectively.

\subsection{RSA-1024}

\includegraphics[width=0.475\textwidth]{figures/10minus3/RSA1024.png}
\captionof{figure}{RSA-1024 space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 3.01\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $3.01\times 10^{11}$, the corresponding number of logical qubits is 2050, and the total number of surface code cycles is $5.86\times 10^{13}$. The quantity $R^2$ represents the coefficient of determination (closer to 1, better the fitting). The classical security parameter is approximately 80 bits.}
\label{fgr:rsa1024a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/RSA1024.png}
\captionof{figure}{RSA-1024 space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 2.14\times 10^6$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $3.01\times 10^{11}$, the corresponding number of logical qubits is 2050, and the total number of surface code cycles is $2.93\times 10^{13}$. The classical security parameter is approximately 80 bits.}
\label{fgr:rsa1024b}

\subsection{RSA-2048}

\includegraphics[width=0.475\textwidth]{figures/10minus3/RSA2048.png}
\captionof{figure}{RSA-2048 space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 1.72\times 10^8$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $2.41\times 10^{12}$, the corresponding number of logical qubits is 4098, and the total number of surface code cycles is $4.69\times 10^{14}$. The classical security parameter is approximately 112 bits.}
\label{fgr:rsa2048a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/RSA2048.png}
\captionof{figure}{RSA-2048 space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 9.78\times 10^6$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $2.41\times 10^{12}$, the corresponding number of logical qubits is 4098, and the total number of surface code cycles is $2.35\times 10^{14}$. The classical security parameter is approximately 112 bits.}
\label{fgr:rsa2048b}

\subsection{RSA-3072}

\includegraphics[width=0.475\textwidth]{figures/10minus3/RSA3072.png}
\captionof{figure}{RSA-3072 space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 6.41\times 10^8$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $8.12\times 10^{12}$, the corresponding number of logical qubits is 6146, and the total number of surface code cycles is $1.58\times 10^{15}$. The classical security parameter is approximately 128 bits.}
\label{fgr:rsa3072a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/RSA3072.png}
\captionof{figure}{RSA-3072 space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 2.55\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $8.12\times 10^{12}$, the corresponding number of logical qubits is 6146, and the total number of surface code cycles is $7.91\times 10^{14}$. The classical security parameter is approximately 128 bits.}
\label{fgr:rsa3072b}

\subsection{RSA-4096}

\includegraphics[width=0.475\textwidth]{figures/10minus3/RSA4096.png}
\captionof{figure}{RSA-4096 space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 1.18\times 10^9$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $1.92\times 10^{13}$, the corresponding number of logical qubits is 8194, and the total number of surface code cycles is $3.75\times 10^{15}$. The classical security parameter is approximatively approximately 156 bits.}
\label{fgr:rsa4096a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/RSA4096.png}
\captionof{figure}{RSA-4096 space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 5.70\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $1.92\times 10^{13}$, the corresponding number of logical qubits is 8194, and the total number of surface code cycles is $1.88\times 10^{15}$. The classical security parameter is approximatively approximately 156 bits.}
\label{fgr:rsa4096b}

\subsection{RSA-7680}

\includegraphics[width=0.475\textwidth]{figures/10minus3/RSA7680.png}
\captionof{figure}{RSA-7680 space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 7.70\times 10^{10}$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $1.27\times 10^{14}$, the corresponding number of logical qubits is 15362, and the total number of surface code cycles is $2.64\times 10^{16}$. The classical security parameter is approximately 192 bits.}
\label{fgr:rsa7680a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/RSA7680.png}
\captionof{figure}{RSA-7680 space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 7.41\times 10^{9}$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $1.27\times 10^{14}$, the corresponding number of logical qubits is 15362, and the total number of surface code cycles is $2.47\times 10^{16}$. The classical security parameter is approximately 192 bits.}
\label{fgr:rsa7680b}

\subsection{RSA-15360}

\includegraphics[width=0.475\textwidth]{figures/10minus3/RSA15360.png}
\captionof{figure}{RSA-15360 space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 4.85\times 10^{12}$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $1.01\times 10^{15}$, the corresponding number of logical qubits is 30722, and the total number of surface code cycles is $2.24\times 10^{17}$. The classical security parameter is approximately 256 bits.}
\label{fgr:rsa15360a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/RSA15360.png}
\captionof{figure}{RSA-15360 space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 7.64\times 10^{10}$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $1.01\times 10^{15}$, the corresponding number of logical qubits is 30722, and the total number of surface code cycles is $1.98\times 10^{17}$. The classical security parameter is approximately 256 bits.}
\label{fgr:rsa15360b}

\section{Elliptic curve schemes\label{sct::ecc}}
In the following section we compute the space/time tradeoffs for attacking public-key cryptographic schemes based on solving the discrete logarithm
problem in finite groups generated over elliptic curves, namely NIST P-160, NIST P-192, NIST P-224, NIST P-256, NIST P-384 and NIST P-521. For
each scheme, we plot the space/time tradeoff points then fit it with a third degree polynomial, for $p_g=10^{-3}$ and $p_g=10^{-5}$, respectively. We
used the logical circuits from~\cite{1706.06752}.

\subsection{NIST P-160}

\includegraphics[width=0.475\textwidth]{figures/10minus3/P160.png}
\captionof{figure}{NIST P-160 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 1.81\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $2.08\times 10^{11}$, the corresponding number of logical qubits is 1466, and the total number of surface code cycles is $4.05\times 10^{13}$. The classical security parameter is 80 bits.}
\label{fgr:p160a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/P160.png}
\captionof{figure}{NIST P-160 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 1.38\times 10^6$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $2.08\times 10^{11}$, the corresponding number of logical qubits is 1466, and the total number of surface code cycles is $2.03\times 10^{13}$. The classical security parameter is 80 bits.}
\label{fgr:p160b}

\subsection{NIST P-192}

\includegraphics[width=0.475\textwidth]{figures/10minus3/P192.png}
\captionof{figure}{NIST P-192 space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 3.37\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $3.71\times 10^{11}$, the corresponding number of logical qubits is 1754, and the total number of surface code cycles is $7.23\times 10^{13}$. The classical security parameter is 96 bits.}
\label{fgr:p192a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/P192.png}
\captionof{figure}{NIST P-192 space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 2.18\times 10^6$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $3.71\times 10^{11}$, the corresponding number of logical qubits is 1754, and the total number of surface code cycles is $3.62\times 10^{13}$. The classical security parameter is 96 bits.}
\label{fgr:p192b}

\subsection{NIST P-224}

\includegraphics[width=0.475\textwidth]{figures/10minus3/P224.png}
\captionof{figure}{NIST P-224 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 4.91\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $5.90\times 10^{11}$, the corresponding number of logical qubits is 2042, and the total number of surface code cycles is $1.15\times 10^{14}$. The classical security parameter is 112 bits.}
\label{fgr:p224a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/P224.png}
\captionof{figure}{NIST P-224 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 3.24\times 10^6$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $5.90\times 10^{11}$, the corresponding number of logical qubits is 2042, and the total number of surface code cycles is $5.75\times 10^{13}$. The classical security parameter is 112 bits.}
\label{fgr:p224b}

\subsection{NIST P-256}

\includegraphics[width=0.475\textwidth]{figures/10minus3/P256.png}
\captionof{figure}{NIST P-256 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 6.77\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $8.82\times 10^{11}$, the corresponding number of logical qubits is 2330, and the total number of surface code cycles is $1.72\times 10^{14}$. The classical security parameter is 128 bits.}
\label{fgr:p256a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/P256.png}
\captionof{figure}{NIST P-256 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 4.64\times 10^6$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $8.82\times 10^{11}$, the corresponding number of logical qubits is 2330, and the total number of surface code cycles is $8.60\times 10^{13}$. The classical security parameter is 128 bits.}
\label{fgr:p256b}

\subsection{NIST P-384}

\includegraphics[width=0.475\textwidth]{figures/10minus3/P384.png}
\captionof{figure}{NIST P-384 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 2.27\times 10^8$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $3.16\times 10^{12}$, the corresponding number of logical qubits is 3484, and the total number of surface code cycles is $6.17\times 10^{14}$. The classical security parameter is 192 bits.}
\label{fgr:p384a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/P384.png}
\captionof{figure}{NIST P-384 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 1.28\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $3.16\times 10^{12}$, the corresponding number of logical qubits is 3484, and the total number of surface code cycles is $3.08\times 10^{14}$. The classical security parameter is 192 bits.}
\label{fgr:p384b}

\subsection{NIST P-521}

\includegraphics[width=0.475\textwidth]{figures/10minus3/P521.png}
\captionof{figure}{NIST P-521 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-3}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 6.06\times 10^8$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $7.98\times 10^{12}$, the corresponding number of logical qubits is 4719, and the total number of surface code cycles is $1.56\times 10^{15}$. The classical security parameter is 256 bits.}
\label{fgr:p521a}

\includegraphics[width=0.475\textwidth]{figures/10minus5/P521.png}
\captionof{figure}{NIST P-521 elliptic curve space/time tradeoffs with physical error rate per gate $p_g=10^{-5}$. The scale is logarithmic (base 2). Approximately $y(16.3987) \approx 2.30\times 10^7$ physical qubits are required to break the scheme in one day (24 hours). The number of T gates in the circuit is $7.98\times 10^{12}$, the corresponding number of logical qubits is 4719, and the total number of surface code cycles is $7.78\times 10^{14}$. The classical security parameter is 256 bits.}
\label{fgr:p521b}

\section{Summary and conclusions}\label{sct::conclusion}
We analyzed the security of several widely used symmetric ciphers and hash functions against parallelized quantum adversaries. We computed the security parameter, wall-time and physical footprint for each cryptographic primitive. Our attack model was based on a brute force searching via a parallelized version of Grover's algorithm, assuming a surface-code fault-tolerant architecture based on defects and braiding techniques.

It is worth noting that throughout we are assuming that brute-force search where we treat the cryptographic function as a black-box is essentially the optimal attack against SHA and AES, which is currently believed to be the case.

Some symmetric key algorithms are susceptible in a model that permits ``superposition attacks''~\cite{quantph.1602.05973}. In most realistic instances, these attacks are not practical, however they do shed light on the limitations of certain security proof methods in a quantum context, and remind us that we shouldn't take for granted that non-trivial attacks on symmetric key cryptography may be possible.
For example, very recently, there have been several cryptanalysis results~\cite{1712.06239} and~\cite{1802.03856} that attempt to reduce breaking some symmetric algorithms to solving a system of non-linear equations.  Solving these non-linear equations is then attacked using a modified version of the quantum linear equation solver algorithm~\cite{PhysRevLett.103.150502}. The results are heavily dependent on the condition number of the non-linear system, which turns to be hard to compute (it is not known for most ciphers and hash functions such as AES or SHA). Provided the condition number is relatively small, then one may get an  advantage compared to brute-force Grover search. However at this time it is not clear whether this is indeed the case, and we do not have large-scale quantum computers to experiment with.

The quantum security parameter (based on our assumptions of using state-of-the-art algorithms and fault-tolerance methods) for symmetric and hash-based cryptographic schemes is summarized in Table~\ref{tbl1}. For more details about space/time tradeoffs achievable via parallelization of Grover's algorithm please see the corresponding Sec.~\ref{sct::ciphers}, Sec.~\ref{sct::hash} and Sec.~\ref{sct::bitcoin}, respectively.
\begin{table}[h!]
\begin{tabular}{ll}
\hline
Name    & qs  \\
\hline
AES-128 & 106 \\
AES-192 & 139 \\
AES-256 & 172 \\
\hline
SHA-256 & 166 \\
SHA3-256	 &167 \\
Bitcoin's PoW & 75\\
\hline
\end{tabular}
\caption{Quantum security parameter ($qs$) for the AES family of ciphers, SHA family of hash functions, and Bitcoin, assuming a conservative physical error rate per gate $p_g=10^{-4}$.}
\label{tbl1}
\end{table}

We also analyzed the security of asymmetric (public-key) cryptography, in particular RSA and ECC, in the light of new improvements in fault-tolerant
quantum error correction based on surface code lattice surgery techniques. We computed the space/time tradeoff required to attack
every scheme, using physical error rates of $10^{-3}$ and $10^{-5}$, respectively. We fitted the data with a third degree polynomial, which resulted in an analytical formula of the number of qubits required to break the
scheme as a function of time.

The total number of physical qubits required to break the RSA schemes in 24 hours, together with the required number of $T$ gates, corresponding number of surface code cycles and corresponding classical security parameter is summarized in Table~\ref{tbl2}. For more details about possible space/time tradeoffs please see the corresponding Section~\ref{sct::rsa} of the manuscript.
\begin{table}[]
\begin{tabular}{lllll}
\hline
Name      & nq                    & Tc                    & scc           & s        \\
\hline
RSA-1024  & $3.01 \times 10^7$    & $3.01 \times 10^{11}$ & $5.86 \times 10^{13}$ & 80\\
RSA-2048  & $1.72 \times 10^8$    & $2.41 \times 10^{12}$ & $4.69 \times 10^{14}$ & 112\\
RSA-3072  & $6.41 \times 10^8$    & $8.12 \times 10^{12}$ & $1.58 \times 10^{15}$ & 128\\
RSA-4096  & $1.18 \times 10^9$    & $1.92 \times 10^{13}$ & $3.75 \times 10^{15}$ & 156\\
RSA-7680  & $7.70 \times 10^{10}$ & $1.27 \times 10^{14}$ & $2.64 \times 10^{16}$ & 192\\
RSA-15360 & $4.85 \times 10^{12}$ & $1.01 \times 10^{15}$ & $2.24 \times 10^{17}$ & 256\\
\hline
\end{tabular}
\caption{The total physical footprint ($nq$) required to break the RSA schemes in 24 hours, together with the required number of $T$ gates ($Tc$), the corresponding number of surface code cycles ($scc$), and the corresponding classical security parameter ($s$).
We assume a very conservative physical error rate per gate $p_g=10^{-3}$, more likely to be achievable by the first generations of fault-tolerant quantum computers.}
\label{tbl2}
\end{table}

The total number of physical qubits required to break the ECC schemes in 24 hours, together with the required number of $T$ gates, corresponding number of surface code cycles and corresponding classical security parameter is summarized in in Table~\ref{tbl3}. For more details about possible space/time tradeoffs please see the corresponding Section~\ref{sct::ecc} of the manuscript. As observed before in~\cite{1706.06752}, breaking RSA schemes demands more quantum resources in comparison with elliptic curve-based schemes, for the same level of classical security.
\begin{table}[]
\begin{tabular}{lllll}
\hline
Name  & nq                 & Tc                    & scc          & s          \\
\hline
P-160 & $1.81 \times 10^7$ & $2.08 \times 10^{11}$ & $4.05 \times 10^{13}$ & 80\\
P-192 & $3.37 \times 10^7$ & $3.71 \times 10^{11}$ & $7.23 \times 10^{13}$ & 96\\
P-224 & $4.91 \times 10^7$ & $5.90 \times 10^{11}$ & $1.15 \times 10^{14}$ & 112\\
P-256 & $6.77 \times 10^7$ & $8.82 \times 10^{11}$ & $1.72 \times 10^{14}$ & 128\\
P-384 & $2.27 \times 10^8$ & $3.16 \times 10^{12}$ & $6.17 \times 10^{14}$ & 192\\
P-521 & $6.06 \times 10^8$ & $7.92 \times 10^{12}$ & $1.56 \times 10^{15}$ & 260\\
\hline
\end{tabular}
\caption{The total physical footprint ($nq$) required to break the ECC schemes in 24 hours, together with the required number of $T$ gates ($Tc$), the corresponding number of surface code cycles ($scc$), and the corresponding classical security parameter ($s$). We assume a very conservative physical error rate per gate $p_g=10^{-3}$, more likely to be achievable by the first generations of fault-tolerant quantum computers.}
\label{tbl3}
\end{table}

Recent developments in the theory of fault-tolerant quantum error correction have great impact on evaluating the effective strength of cryptographic
schemes against quantum attacks, as the fault-tolerant layer of a quantum computation is the most resource-intensive part of running a quantum
algorithm. Therefore, monitoring the advances in the theory of quantum error correction is of crucial importance when estimating the strength (or
weakness) of a cryptographic scheme against a quantum adversary. This work serves as a benchmark against which the impact of future advances can be compared.

\begin{acknowledgments}
Most of this work is based on research supported by the Global Risk Institute for its members.
We also acknowledge support from NSERC and CIFAR. IQC and the Perimeter Institute are supported in part by the
Government of Canada and the Province of Ontario.  Vlad Gheorghiu thanks Austin Fowler for helpful discussions
and clarifications regarding lattice surgery methods.
\end{acknowledgments}

\bibliographystyle{aipnum4-1}


\section{Introduction}
\label{sec:intro}
Past years have witnessed dramatic progress on the application of generative sequential models (also noted as seq2seq learning~\cite{DBLP:conf/nips/SutskeverVL14,DBLP:journals/iclr/BahdanauCB14}) on Neural Response Generation (NRG) fields~\cite{DBLP:journals/iclr/VinyalsL15,DBLP:conf/aaai/SerbanSLCPCB17}.
Seq2seq model has been proved to be capable of directly generating reply given an open domain query~\cite{DBLP:conf/acl/LiGBSGD16,DBLP:conf/aaai/XingWWLHZM17}.
Both the relevant words or phrases are automatically selected, and responses' smoothness and fluency are guaranteed through the end-to-end learning.
Moreover, abundant impressive human-to-machine conversation cases have been presented in many previous studies~\cite{DBLP:conf/aaai/SerbanSBCP16,DBLP:conf/acl/ShangLL15,shao2017generating}.

Despite these promising results, current seq2seq architectures of NRG are still far from steadily generating relevant and coherent replies. One issue identified by many studies is the \emph{Universal Replies}: the model tends to generate short and general replies which contain limited information, such as ``That's great!'', ``I don't know'', etc.~\cite{DBLP:conf/naacl/LiGBGD16,DBLP:journals/corr/LiMJ16,DBLP:conf/coling/MouSYL0J16,DBLP:conf/aaai/XingWWLHZM17}.
Intuitively, this problem was attributed to the vast coverage of common replies in the training set and insufficient guiding knowledge in the models' response generation step~\cite{DBLP:conf/coling/MouSYL0J16,shao2017generating}.
Hence, current efforts mainly focus on introducing external information to the model~\cite{DBLP:conf/coling/MouSYL0J16,DBLP:conf/aaai/XingWWLHZM17}, and encouraging the model to generate diverse responses in the searching space via variational beam search strategies during inference~\cite{shao2017generating,DBLP:conf/naacl/LiGBGD16,DBLP:journals/corr/LiMJ16}.

Nevertheless, most historical analyses over the issue are empirical and lack statistical evidence.
So, in this paper, we conduct an in-depth investigation on the performance of seq2seq models on the NRG task.
In our inspections on the existing dialog corpora, it is shown that those repeatedly appeared replies have two essential traits:
1) Most of them are composed of highly frequented words;
2) They cover a large portion of the dialog corpora that each universal reply stands for the response of various queries.
Above characters of universal replies deviate the NRG from other successful applications of sea2seq model such as translation, and lead the NRG models to treat those common replies as grammatically corrected pattern mistakenly. Therefore common replies are unconsciously preferred by current generative NRG models.
Furthermore, in the conversational corpus, each query corresponds to many responses.
To discuss the influences from the specific distributed corpus,
we decompose the target sequence's probability into two independent parts and analyze the probability leak for each component.

To break down the mentioned characteristics of dialog corpora in the model training step, we propose a ranking-oriented regularization term to prune the scores of those irrelevant replies.
Experimental results reveal that the model with such regularization can produce better results and avoid generating ambiguous responses. Also, case studies show that the issue of generic response is alleviated as these common responses are ranked relatively lower than more appropriate answers.

The main contributions of this paper are concluded as follows: 1) We analyze the loss function of Seq2seq models on NRG task and conclude several critical reasons that the NRG models prefer universal replies; 2) Based on the analysis, a max-margin ranking regularization is presented to help the model converge to informative responses.

\begin{table}[!t]
\setlength\abovedisplayskip{0pt}
\setlength\belowdisplayskip{0pt}
\small
  \caption{Replies and translated version of an example which reveal the different source-target sentence distribution for dialog and translation.}
  \label{tab:case}
  \centering
  \begin{tabular}{ll}
    \toprule
    Query& I would add Metropolis to the list.\\\midrule
    Replies & \underline{I} love \underline{this film} so much.\\
    &Me too, it is a beautiful \underline{film}.\\
    &\underline{This} movie has beautiful background art.\\
    &Fritz is really a good director, \underline{I} like his \underline{film}.\\
    &Is "\underline{Metropolis}" based \underline{on} a book?\\
    &Brigitte cooling off \underline{on} the set of \underline{Metropolis}.\\
    \midrule
    Translate& J'ajouterais\underline{ Metropolis \`{a} la liste}.\\
     &Je voudrais ajouter \underline{Metropolis \`{a} la liste}.\\
    \bottomrule
  \end{tabular}
\end{table}

\section{Analysis of Seq2seq Models for NRG}
Different from significant advances in machine translation~\cite{DBLP:journals/iclr/BahdanauCB14} and abstractive summarization~\cite{rush2015neural,DBLP:conf/conll/NallapatiZSGX16}, it remains challenging to apply seq2seq models on NRG.
One identified issue within current models is that they are inclined to produce common and unrelated replies, even when the quality of data in the training set is significantly improved.
Many responsible factors have been identified, and one primary reason is attributed to the feeble constraint from query to possible answers while the constraints are explicitly specified in translation and summarization tasks.

Specifically, for machine translation, words that appear in the target language should satisfy word level mapping from the source sentence, so the learned word alignment function could ensure the model to generate suitable translated words.
Different from learning the semantic alignments between languages in NMT, in NRG there exists no explicit mapping for the conversation pattern.
By contrast, the replies can be diversified as they only need to satisfy the causality with the given queries.
What's more, the degree of dialog diversity is not explicitly measured and acquired in the sequential model. Instead, it learns a weighted average of these replies. Thus the model is more likely to choose those common replies, and this is also mentioned in~\newcite{DBLP:conf/emnlp/RitterCD11}.

Taking the case in Table~\ref{tab:case} as an example, the topic of this query is about movie, and the replies shown in the table are semantically diversified: the first two replies are related to the responder's opinion toward the movie, and the rest replies are about the director, content, and origin of the movie.  By contrast, the two valid translations
in French are very similar regarding their semantics, which can be attributed to the fixed word-level mapping between query and targets.

\subsection{Problem Decomposition}
The sequence-mapping problem in NRG can be transformed into two independent sub-learning problems, and detailed as follows:
1) \textit{Target word selection}, in which a query is summarized and translated into responses' semantic space, and then a set of target words is selected to represent the meaning;
2) \textit{Word ordering} that is performed to generate a grammatical coherent reply based on the candidate word set~\cite{DBLP:journals/iclr/VinyalsBK15}.
The word selection and ordering of the target sequence are jointly learned by maximizing the instances in the training set, which can also be reflected in the model's loss function with two factored phases:
\begin{equation}\label{equ:bayes}
 \log p(y|x)= \log p(\mathcal{S}(y)|x) +\log p(y|\mathcal{S}(y),x)
\end{equation}
where $x$ stands for the given source query and $y$ is the corresponding response with $n$ words. Besides, $\mathcal{S}(y)=\{w_1,\cdots,w_n|w_i\in y,i\in[1,n]\}$ represents all of the predicted words without sequential order, so $p(\mathcal{S}(y)|x)$ is referred as the probability of the unordered set, or the probability of the \textit{target word selection}.
Meanwhile, $p(y|\mathcal{S}(y),x)$ indicates the probability of \textit{word ordering} given this group of possible words.
Thus, the objective can be rewritten from maximizing the probability of the ground truth response $y$ under query $x$ to maximizing these two joint probabilities simultaneously.

After the above interpretation, we will further discuss the impact of the implicative constriction from two separated probabilities in Eq.~\ref{equ:bayes}, which makes models fail to learn conversational patterns.

\subsection{Target Word Selection Probability}
\label{sec:first_prob}
Assuming we have a set of $\mathcal{K}$ ground truth replies: $\{y_1,\cdots,y_\mathcal{K}\}$ for a given query $x$, the variational upper bound of the \textit{target word selection} probability can be derived using Jensen's Inequality~\cite{Boyd:2004:CO:993483}:
\begin{equation}\label{equ:best_cases}
\begin{split}
\sum_k^\mathcal{K}{\log p(\mathcal{S}(y_k)|x)}
&=  \sum_{w \in \cup_k^\mathcal{K}{\mathcal{S}(y_k)}} \log p(w|x)  \\
&\le \log\sum_{w \in \cup_k^\mathcal{K}{\mathcal{S}(y_k)}} p(w|x)
\end{split}
\end{equation}
where $\cup_k^\mathcal{K}{\mathcal{S}(y_k)}$ denotes all words that appear in the entire response set. Optimizing the first separated segment is proportional to maximizing the conditional probabilities for this union set. So the optimal strategy is to assign probabilities according to the frequency of words in these $\mathcal{K}$ responses.

If we only consider this probability when yielding a response during inference,
the best strategy is to employ more frequently occurring words rather than rare ones,
such as ``background,'' ``art,'' and ``director'' in Table~\ref{tab:case}.
The long-tailed words' distribution in these responses in Fig.~\ref{fig:zipf} further validate this local greedy generation scheme, with just few common words obtained with preferred high probabilities.

Furthermore, if we consider the training process and assume that every response contains a fixed number of  $T$ words,
the probability of each response for a given query $x$ is inversely proportional to $\mathcal{K}$:
\begin{equation}\label{equ:best_case}
\sum_{w \in \cup_k^\mathcal{K} \mathcal{S}(y_k)}  p(w|x)
= \frac{\mathbb{E}(w|x) \times T}{\mathcal{K} \times T}
\propto \frac{1}{\mathcal{K}}
\end{equation}
where $\mathbb{E}(w|x)$ denotes the mean frequency (overall words) to appear in these $\mathcal{K}$ replies,
which is 1.32 for the cases in Table~\ref{tab:case}.
In general, the mean frequency is around $1$, owing to the long-tailed Unigram distribution which is satisfied Zipf's law~\cite{zipf1935psychobiology}.
In other words, the \textit{target word selection probability} is limited by $\mathcal{K}$, so queries with more diverse answers in the training set are more challenging to learn.
Meanwhile, it is difficult to obtain good predictions for lower-informational queries, as they contain more possible responses which are somewhat equivalent to a larger $\mathcal{K}$~\cite{DBLP:conf/coling/LiWWXLZ16}.

Nonetheless, the translation task requires word-level mappings as they are well-aligned in the semantic space, therefore the source and target sentences in the training set are semantically equivalent.
Therefore, translated candidates are confined to $\mathcal{K}\approx 1$.
Thus the upper bound can approximate to the full probability.

\subsection{Word Ordering Probability}
\label{sec:second_prob}

Before the discussion about \textit{word ordering probability}, we present four lemmas
and the proofs of all the lemmas are detailed in the Appendix.
Moreover, all these lemmas are only available for the response generation task except Lemma 1.

\noindent\textbf{Lemma 1} \emph{Supposing that $\mathcal{S}(y)$ is the set of the words composing $y$, $p(\mathcal{S}(y)|y)=1$, $p(\mathcal{S}(y), y) = p(y)$, $p(x, y, \mathcal{S}(y)) = p(x, y)$.}

\noindent\textbf{Lemma 2} \emph{Given a universal reply $y_{ur}$, the transition probability from $y_{ur}$ to any post $x$ is sufficiently small, which can be noted as $p(x|y_{ur}) = \epsilon_1$, where $\epsilon_1 > 0$ and sufficiently small.}

\noindent\textbf{Lemma 3} \emph{
Suppose that $y^{ur}_i$ stands for a universal reply and $y^{o}_j$ is one of the non-universal grammatical replies, meanwhile, $\mathcal{S}(y^{ur}_i) \subseteq \mathcal{S}(y)$ and $\mathcal{S}(y^{o}_j) \subseteq \mathcal{S}(y)$,
then $\sum p(y^{ur}_i | \mathcal{S}(y)) \to 1$, $p(y^{o}_j | \mathcal{S}(y)) = \epsilon_2$, where $\epsilon_2 > 0$ and sufficiently small.}

\noindent\textbf{Lemma 4} \emph{Assuming that each informative query has $\mathcal{K}$ ground-truth replies and the query-response pairs are extracted from a multi-turn conversational corpus, a reply $y$ not belonging to universal replies has $\mathcal{K}$ unique queries. This lemma can be noted as $p(x | y) = \frac{1}{\mathcal{K}}$}.

\noindent On the basis of Lemma 1, the \textit{word ordering probability} could be deducted as:
\begin{equation}
\label{equ:prox}
\begin{aligned}
&log\,p(y|\mathcal{S}(y),x)\\
&=log \frac{p(\mathcal{S}(y)|y)\,p(y)\,p(x|y, \mathcal{S}(y))}{p(\mathcal{S}(y)) \, p(x|\mathcal{S}(y))} \\
&=log1 + log\frac{p(y)}{p(\mathcal{S}(y))} + log\frac{p(x|y, \mathcal{S}(y))}{p(x|\mathcal{S}(y))} \\
&=log \frac{p(y, \mathcal{S}(y))}{p(\mathcal{S}(y))} + log\frac{p(x, y, \mathcal{S}(y))\,p(\mathcal{S}(y))}{p(y, \mathcal{S}(y))\,p(x, \mathcal{S}(y))} \\
&=log\,p(y|\mathcal{S}(y)) + log\frac{p(x, y)\,p(\mathcal{S}(y))}{p(y)\,p(x, \mathcal{S}(y))} \\
&=log\,p(y|\mathcal{S}(y)) + log\frac{p(x | y)}{p(x | \mathcal{S}(y))} \\
&=log\,p(y|\mathcal{S}(y)) + log\frac{p(x | y)}{\sum_i{p(x | y_i)p(y_i | \mathcal{S}(y))}} \\
\end{aligned}
\end{equation}

\begin{figure}
\setlength\abovedisplayskip{0pt}
\setlength\belowdisplayskip{0pt}
\centering
\includegraphics[width=1\linewidth]{./dist.pdf}
  \caption{Replies' Unigram probability in Table~\ref{tab:case}.}\label{fig:zipf}
\end{figure}

\noindent All the possible $y_i$ satisfying $\mathcal{S}(y_i) \subseteq \mathcal{S}(y)$ can be divided into three categories: ground-truth reply $y$, universal replies $y^{ur}$, other replies $y^o$.
After that, we can get the following direct proportion according to the Lemma 2 and Lemma 3,

\begin{equation}
\label{equ:prox_step1}
\begin{aligned}
&\sum_i{p(x | y_i)p(y_i | \mathcal{S}(y))} \\
&=p(x | y)p(y | \mathcal{S}(y)) + \sum_i{p(x | y^{ur}_i)p(y^{ur}_i | \mathcal{S}(y))} \\
& \qquad + \sum_i{p(x | y^{o}_i)p(y^{o}_i | \mathcal{S}(y))} \\
&\propto p(x | y)p(y | \mathcal{S}(y)) + \epsilon_1 + \epsilon_2 \\
\end{aligned}
\end{equation}

On the basis of Eq.~\ref{equ:prox_step1} and Lemma 4, for any reply $y$ not belonging to universal replies, the Eq.~\ref{equ:prox} can be further deducted as:

\begin{equation}
\label{equ:prox_step2}
\begin{aligned}
&log\,p(y|\mathcal{S}(y),x)\\
&\propto log\,p(y|\mathcal{S}(y)) + log\frac{p(x | y)}{p(x | y)p(y | \mathcal{S}(y)) + \epsilon} \\
&\propto log\frac{p(y|\mathcal{S}(y))}{p(y | \mathcal{S}(y)) + \frac{1}{\mathcal{K}}\epsilon} \\
\end{aligned}
\end{equation}
where $\epsilon=\epsilon_1 + \epsilon_2 > 0$, which is also a sufficiently small positive value.
Thus, optimizing the \textit{word ordering probability} for the non-universal replies is somewhat equivalent to maximizing $p(y|\mathcal{S}(y))$.
In fact the term $p(y|\mathcal{S}(y))$ is the language model probability and it is irrelevant with the query $x$~\cite{Maning2009An}.
In the sequential models, it is performed as $\prod_{t} p(y_t|y_{1:t-1},\mathcal{S}(y))$, that the sequences are generated based only on previously outputted words.
This equation indicates that optimizing the  mainly seeks the grammatical competence based on the selected words.

\subsection{Brief Summary}
In conclusion, insufficient constraining of the target words' cross-entropy loss in NRG is the primary reason that hinders seq2seq models from exploring presumable parameters.
Meanwhile, the model is likely to promote responses with more frequent words,
moreover, will prefer sentences with high transition probability of words, according to the distribution of the training set data.
The downside is that current seq2seq models in NRG favor generating universal replies, according to their essential traits mentioned in Section~\ref{sec:intro}.

\section{Max-margin Ranking Regularization}
\label{sec:rank}
As discussed above,
various responses corresponding to the same query appearing in the training data
leads the NRG models to prefer universal replies,
so an intuitive solution is removing the multiple replies and just keeping one-to-one pairs.
Unfortunately, naively removing the multiple replies is detrimental to the diversity of the replies, which is an important concern in NRG task.
As shown in Table~\ref{tab:case}, an ideal chatbot agent could give all these replies rather than just giving responses with limited semantic information. Largely filtering the training dataset makes the model hard to learn and remember this diverse relation. In addition, the model is prospected to build a connection with some keywords such as `film', `background', `director' and `book', instead of the commonly appeared words like `I', `him', `a' and `really'.

Thus, under this assumption, we propose a max-margin ranking loss to emphasize the queries' impact on these important words.
During training, as it becomes a necessity to constrain the learned feature space and reinforce related replies with more discriminative information,
we distinguish the candidate responses as two categories given one input query: positive (i.e., highly related) and negative (i.e., irrelevant) answers.
A training instance is re-constructed as a triplet $(x,y,y^-)$, where a tuple $(x,y)$ is the original query-response pair and noise $y^-$ is uniformly sampled from all of the responses in the training data. Given that, the model's cost function is reconstructed as:
\begin{equation}
\label{equ:rankloss}
\begin{aligned}
&\ell_\theta= \log p(y|x) +\\
&\quad \lambda \max\{0, \log p(y| x)-\log p(y^-| x)-\gamma \}
\end{aligned}
\end{equation}
where $\log p(y|x)$ denotes the cross-entropy loss between the model's prediction and ground truth sequences, and the second part encourages the separability between the irrelevant responses and related replies. Moreover, the hyper-parameter $\lambda$ defines the penalty for the seq2seq loss, and $\gamma$ offers a degree of freedom about the importance of the max-margin between the positive and negative instances.
The model is trained in the same setting as the conventional model when $\lambda=0$.

The gradient of $\ell_\theta$ is computed using the sub-gradient method, as the second term is non-differentiable but convex~\cite{DBLP:conf/ecir/AgarwalC10}. Supposing $\log p(y| x)- \log p(y^-| x) \leq \gamma$, the gradient of the composed loss function can be formalized as:
\begin{equation}\label{equ:grad}
\nabla_\theta \ell_\theta =
   \nabla_\theta{\log p(y|x)},   \\
\end{equation}
If $\log p(y| x)- \log p(y^-| x) > \gamma $, then the gradient should be written as:
\begin{equation}\label{equ:othercase}
\small
\nabla_\theta \ell_\theta= (1+\lambda) \nabla_\theta{\log p(y|x)} -\lambda\nabla_\theta{\log p(y^-|x)}.
\end{equation}
The underlying motivation is based on three considerations:
1) Universal replies are more likely to be sampled from a statistical perspective, so adding a negative term would directly ease the weight of these generic responses, and the ranking regularization can penalize the irrelevant ones; 2) From a verbal perspective, a balanced distribution of generic words is prepared in these negative sentences, which suggests that the focus should be not on these generic words, but on boosting the model to select more informative and expected words; 3) Only differentiable loss can solely be served as the model's optimization goal for the sequence generation model.
Furthermore, the newly proposed loss aims to penalize the more frequent words and irrelevant candidates, rather than repudiating the literal expression included in the negative samples.
Consequently, based on these considerations, we propose to consider this term as a regularization term to constrain the search space of parameters instead of the stand-alone loss function.

\section{Experimental Studies}
\subsection{Experimental Setups}
\textbf{Dataset Description.}
The dataset used in this study contained almost ten million query and response pairs collected from the threads of a popular Chinese social media site: Douban Group Chat\footnote{https://www.douban.com/group/explore}. All the case studies used in this paper were extracted from this dataset, and we translated them into English for better understanding.

\begin{table}[!ht]
\small
  \centering
  \caption{Dataset statistics. For multiple replies, the three values represent the percentages of queries with
  one, two, and more than two responses, respectively.
  For the out of vocabulary (OOV) columns, the number in front of ``/'' denotes the percentage rate of the query, and the other one denotes replies.\label{tab:dataset}}
    \begin{tabular}{lccc}
        \toprule
        &\# train & \# valid & \# test \\ \midrule
        QA Pairs&5,982,868& 315,136& 315,136  \\
        Unique Replies& 4,499,176 & 298,723 &  287,312\\
        Multi Replies(\%)& 70/24/6 &97/2/1 &96/3/1\\
        OOV (\%)& .90/.90 &.92/.93 &.91/.92\\
        Vocab Size & \multicolumn{3}{c}{29241/27859} \\
        \bottomrule
  \end{tabular}
\end{table}

As it is difficult to train recurrent models on overtly long sentences, and because the temporal efficiency of forwarding probability and backward gradient calculation of recurrently connected units scale linearly with the sentence’s length, the maximal lengths of queries and replies were set to 30 and 50, respectively. In all of our experiments, our dataset was split into three sets: the training set, the validation set, and the test set, and detailed statistical characterization is given in Table~\ref{tab:dataset}. Thirty percent of queries had more than one response, and each answer appeared about 1.33 times in the training dataset, this was consistent with our hypothesis in the analysis section.

\vspace{0.7mm}
\noindent\textbf{Baseline Models.}
To validate the performance of the proposed model, the following baseline scenarios were considered:

1) S2SA. We employed the basic seq2seq model with attention model~\cite{DBLP:journals/iclr/BahdanauCB14} at the target output side as our baseline model, because attention is quite important for modeling correlations between the source and target sentences.

2) S2SA + MMI. The best performing model in~\citet{DBLP:conf/naacl/LiGBGD16} with the length norm based on the same S2SA.

3) Ranking-Reg. It denotes the seq2seq model with proposed ranking regularization and attention. Nevertheless, negative samples were uniformly sampled from the corpus, and the process was repeated four times for every positive case. The averaged negative loss was calculated as the probability of universal replies.

4) Ranking-Reg + MMI. We also perform the Maximum Mutual Information (MMI) mechanism during inference procedure.

\begin{table*}[t]
  \centering
  \caption{Summarized results of testing set with metrics: Human Label, ROGUE-1, ROGUE-L, Distinct-1, Distinct-2 and PPL.}
    \begin{tabular}{lccc cc ccc}
        \toprule
        \multirow{2}{*}{Methods}&\multicolumn{3}{c}{Human Label} & \multicolumn{2}{c}{ROUGE} & \multicolumn{2}{c}{Distinct} &  \multirow{2}{*}{PPL}\\
        \cline{2-4}  \cline{5-6} \cline{7-8}
        &0&1 &2  & ROUGE-1 &  ROUGE-L &  1 & 2 &\\\midrule
        S2SA  & 52.46\%&\bf{20.52}\%& 27.02\% & \bf{4.97}\%  &\bf{3.13}\%&  .129 & .285&110.0\\
        S2SA + MMI  & 51.88\%&19.92\%& 28.20\% & 3.96\%  & 2.77\%&  .140 & .312&110.0\\
        Rank-Reg & 48.20\%& 15.38\% &\bf{36.42}\%&3.45\% &  2.55\% & .163  &\bf{.358} &\bf{85.6}\\
        Rank-Reg + MMI & \emph{\underline{47.40\%}}& 18.75\% &33.85\%&3.43\% &  2.63\% & \bf{.167}  &.345 &\bf{85.6}\\
        \bottomrule
  \end{tabular}%
  \label{table:res}
\end{table*}

\begin{figure}[t]
\centering
  \includegraphics[width=0.9\linewidth]{./learn2.pdf}
  \caption{Learning curve for the two models.}\label{fig:learning}
\end{figure}

\vspace{0.7mm}
\noindent\textbf{Evaluation Metrics.}
The quality of response was measured using both numeric metrics and human annotators.
Firstly, \textit{Word Perplexity} (PPL) was used in this work, as it can be used to determine the semantic context of phrase-level utterance~\cite{DBLP:conf/aaai/SerbanSBCP16}.
Secondly, a ``ROGUE'' score, which evaluates the extent of overlapping words between the ground-truth results and the predicted answers, was also used in ~\citet{lin2004rouge}.
Thirdly, we employed ``Distinct-1'' and ``Distinct-2'' to evaluate the number of distinct Unigrams and Bigrams of the generated responses~\cite{DBLP:conf/naacl/LiGBGD16}.

Furthermore, we recruited human annotators to cross verify the quality of generated responses\footnote{Three highly educated labelers were invited.}. We randomly sampled 100 queries and generated 10 replies for each query using different models, with beam size set to 10. The labeled results were categorized into three degree~\cite{DBLP:conf/aaai/XingWWLHZM17,DBLP:conf/coling/MouSYL0J16}:

\noindent\textbf{0}: The response cannot be used as a reply to the message. It is either semantically irrelevant or not fluent (e.g., with grammatical errors or UNK).

\noindent\textbf{1}: The response can be used as a reply to the message, which includes the universal replies such as ``Yes, I see'' , ``Me too'' and ``I don’t know''.

\noindent\textbf{2}: The response is not only relevant and natural, but also informative and interesting.

\vspace{0.7mm}
\noindent\textbf{Training Procedures.}
For all of the models, LSTM was chosen as the recurrent cell, and there were 512 hidden units for both the encoder and decoder~\cite{DBLP:journals/tnn/GreffSKSS17}. Embedding size and batch size were set to $200$ and $20$, respectively.
The Adam algorithm was employed for gradient optimization~\cite{DBLP:journals/corr/KingmaB14}, and the initial learning rate was $1e-4$.
All of the models were implemented in Theano~\cite{2016arXiv160502688short}, and each ran on a stand-alone K40m GPU device for 7 epochs, which took ~7 days; twice longer time was required for models with rank regularization.

The last two models with the rank regularization share the related hyper-parameters. We set $\lambda$ to 0.1 and $\gamma$ to 0.18, according to the model's performance on the validation set.

Fig.~\ref{fig:learning} shows the cross-entropy loss flows vs. the training epoch numbers.
The model with max-margin ranking regularization obviously converges much faster than the Seq2seq model with attention from the very beginning epochs, and it retains this advantage to the end.
This shows that adding the regularization term increases the fitting
by removing these sub-optimal paths.

\subsection{Results and Analysis}

\textbf{Experimental Results.}
The performance of four models on existing metrics is summarized in Table~\ref{table:res}.
The model with the max-margin ranking regularization demonstrates the best performance (best results) on the target loss PPL, and it is much better than the one with primary loss function.
As the MMI method is performing during inference, so losses of models with MMI are identical with the ones without revision.

\begin{figure}[t]
  \centering
  \includegraphics[width=1.\linewidth]{./rank.pdf}
  \caption{Response re-rank capability. Responses generated by the basic model and model with rank loss are linked by arrows, and same topics are typeset using the same color.}\label{fig:rank}
\end{figure}

\begin{figure}[t]
  \centering
  \includegraphics[width=1.\linewidth]{./appendix.pdf}
  \caption{Cases for comparing the S2SA and the model with ranking regularization, and the topics or expressions of the generated replies marked with blue are excluded in the responses generated by SASA.}\label{fig:appendix_cases}
\setlength{\abovedisplayskip}{-5pt}
\setlength{\belowcaptionskip}{-5pt}
\end{figure}

However, the results are inverse regarding the ``ROGUE'' scores, which generated responses by the S2SA model contain most words appearing in the ground truth answers.
Two conditions greatly contribute to these results.
a) Most n-gram overlaps belonging to the hot words, such as ``I'', ``are'', ``that''.
b) The test dataset is also random split from the whole dataset so that a certain proportion of universal replies stand for the ground truth results.
Besides, the overall scores are very low, which show few words shared by any predictions and the ground truth.
Therefore, it is difficult to conclude, except that more results from standard S2SA are consistent with the ground truth results, even they may be not the best choice, and there are many ways to continue a conversation.

The human evaluation is the most important metric, and it is clear from Table~\ref{table:res} that the models with rank regularization beat S2SA with a large margin. It increases the number of meaningful responses by $\approx10$\% and reduces the number of irrelevant cases by $\approx4$\%.
Meanwhile, the largest portion of of the acceptable replies (labeled as ``1'' or ``2'') of SASA is labeled as ``1'', which indicates the model prefer the safe responses.
We attribute the gaps to the promotion of highly related words and reducing of the universal replies.
These results agree with the numeric ``Distinct-1'' and ``Distinct-2'' results, which measure the number of distinct unigrams and bigrams of the generated responses, respectively, and reveal the model's ability to generate diverse responses~\cite{DBLP:conf/naacl/LiGBGD16,DBLP:journals/corr/SerbanLCP15}.
The seq2seq model yields lower levels of unigram and bigram diversity than the rank loss model.

As another comparison, we note that the improvement introduced by MMI is much smaller than that introduced by the ranking regularization, whereas MMI is a widely used mechanism for promoting diverse responses during inference.
This observation indicates that the fundamental reason behind generating tasteless or inappropriate replies is that replying function learned from data lean to these universal replies.
Moreover, the revision during the greedy search cannot solve the underlying problems, and the ranking regularization proposed in this paper is an effective method for affecting the direction of convergence.
Furthermore, it is also demonstrated by the results of performing the MMI as well as the ranking regularization.
It cannot further improve the performance, by contrast, this adjustment reduces the rate of informative and interesting responses.

\noindent\textbf{Ranking Loss for Generic Responses.}
From the generated results, it is found that the seq2seq model with the ranking regularization term prefers some meaningful content when the query contains sufficient amount of information.
We present top eight responses for two queries generated by different models in Fig.~\ref{fig:rank}.
As shown in the first case, the user posting a query initiates a complicated discussion about some locations. It is observed that the basic seq2seq model converges to a typical ``where is your'' pattern of replies when discussing locations, which is an example of a universal reply. As the greedy beam search strategy is utilized during inference, many location-related constraints further promote these relevant universal replies instead of more varied results from different beams. In contrast, some of the responses in the right column captured the ``commercial street'' clues and inferred a possible location ``Joy City shopping mall'' demoting the generic beam’s results. We attributed this to the boosting ability associated with semantically relevant words, as mentioned in Section~\ref{sec:rank}.

The second case is quite different. In this case, the seq2seq model did not perform satisfactorily, even though the subject ``bank'' was correctly extracted into the generated candidates; however, we cannot perceive the results aligned with the same ``not reliable'' topic, and most of them were just chosen from two beams. Inspecting the replies generated by the rank loss model, we found that much more complicated and diverse sentences that discuss ``unreliable'' can be generated, and irrelevant answers that discuss ``bank'' are lower-ranked.
To further investigate the difference brought by the max-margin ranking regularization, we randomly sampled more cases as described in the Fig.~\ref{fig:appendix_cases}. Even though some of these cases were bad cases and contained some grammatical errors, we found more interesting words compared with basic models.

In conclusion, the seq2seq model with rank regularization can not only be used to formulate the conditional language model but also boost related answers to higher ranks than the rest of universal or inappropriate replies.

\section{Related Work}
Conversation dialog has been accumulating in the online communities making the data-driven dialog model possible~\cite{DBLP:conf/emnlp/RitterCD11}. In literature, query-response pairs are modeled by seq2seq model with attention mechanism~\cite{DBLP:conf/nips/SutskeverVL14,DBLP:conf/aaai/SerbanSBCP16,DBLP:journals/iclr/BahdanauCB14}. And the essence of the neural response generation model are designed by maximizing the likelihood of target response given source query in the training procedure. As various response are reasonable to reply to a query, the information included in the query is limited to constrain the model inference, which makes the NRG models prefer universal replies with minimum risk~\cite{shao2017generating,DBLP:conf/coling/MouSYL0J16}.

To address this issue, various works are conducted on bringing more information to influence these models.
~\citet{DBLP:conf/coling/MouSYL0J16} propose to utilize keywords to constrain the topic of responses, and~\citet{DBLP:conf/aaai/XingWWLHZM17} incorporates the replies with topic information.
By contrary, some researcher believed in diverse responses are just buried by the greedy beam-search rules~\cite{DBLP:journals/corr/LiJ16}, so they focus on involving more punishment or randomness in the inference stages. To illustrate, ~\citet{DBLP:conf/naacl/LiGBGD16} constrain the search space using mutual information with the query and ~\citet{shao2017generating} randomly chose candidate words from top beams to consist short phrases.
These existing works mainly focus on the generation strategies during inference, in contrast, the model's architecture and loss function have rarely been explored.

The ranking penalty has also been used by~\citet{DBLP:conf/emnlp/WisemanR16} to employ a word-level margin to promote ground-truth sequences appearing in the beam search results.
The main difference between our method and their policy is that our method enforces a sequence-level margin to guarantee relevant words obtained higher probabilities,
moreover, demote general responses entirely.
In other words, the ability to generate a ground-truth answer is not the ultimate object of NRG,
diverse responses from various perspectives are more similar to human conversations.

\section{Conclusion and Future Work}

In this work, we have theoretically analyzed the reasons for Neural Response Generation (NRG) models
producing generic responses from the statistical perspective,
by investigating the effect of different components on the Seq2Seq loss function.
On this basis, we have proposed a max-margin ranking regularization cooperating with the learning target of Seq2Seq,
so as to help the model converge to the status of producing informative responses,
rather than merely manipulating the decoding procedure to constrain the generation of universal replies.
Furthermore, empirical experiments and analysis on a conversation dataset indicate
that the models utilizing this strategy can notably outperform the current baseline models.

In the future work, we will analyze the model predictions and design more efficient method to avoid learning the universal replies.