\section{Introduction: Cardy's Conjecture}

Except in the rare cases(e.g. \cite{2}) of integrable or otherwise solvable models, it is generally impossible to derive by hand the spectrum of a quantum theory from only the Hamiltonian. Perturbation theory is therefore a standard technique for the professional theorist, but despite its many successes in describing phenomenological aspects of quantum theory, it is plagued by challenges\cite{3}\cite{4}. For the quantum field theorist, there are challenges aplenty; the first and most glaring issue are the short-distance divergences of Feynman diagrams when using the interaction picture to compute observables or scattering amplitudes. To amend this particular problem, one defines a `renormalized' version of their quantum theory, which introduces a mass scale $\mu$ that serves as an upper bound for the momentum-space resolution of the theory - or if one likes, a coarse-graining of spacetime. Since the theory is now manifestly insensitive to arbitrarily short distance events, the operators and parameters of the original theory must be modified in a $\mu$-dependent way to both avoid the divergences and make observable quantities independent of the choice of $\mu$. Since the parameters of the renormalized theory now depend on this scale (we say they ``run" with $\mu$), there is a potential problem with the application of perturbation theory should these parameters ever become large for a given value of $\mu$. This happens in Quantum Electrodynamics (QED): when renormalized at a very high scale the electric charge of the fermions become arbitrarily large. Since this limit is exactly where the renormalized theory meets the original QFT, such theories are generally considered to be unphysical, unless they are embedded into a new theory at some intermediate scale so as to avoid this issue (as is QED in Grand Unified Theories). On the contrary, we have theories like Quantum Chromodynamics (QCD) where the running of the coupling is opposite - the beta function is negative and the coupling becomes very large when renormalized at a low energy scale\cite{5}\cite{6}. QCD renormalized at scales much higher than a few hundred MeV is considered perturbative. However, perturbative QCD has failed to provide a description of the theory's spectrum which is consistent with observations of the long-distance physics\cite{7}. Given that there are many technical challenges with perturbation theory (Haag's theorem\cite{3}, non-convergence of power series expansions\cite{4}, general difficulties with calculating to high order) it should not be surprising that it generally fails to predict the spectrum of QFTs with negative beta functions.\\

Nevertheless, perturbation theory has taught us much about QFTs. The necessary introduction of the renormalized perturbation theory opened the door for the study of the Renormalization Group (RG)\cite{8}, from which we have learned much about non-perturbative physics. One such piece of knowledge is the c-theorem: in 2-dimensional conformal field theories (CFTs), there is a number $c[\mu]$ that enters as a proportionality constant in front of the anomalous divergence of the scale current. It has been established that c decreases monotonically\cite{9} as one flows to the IR ($\mu$ is decreased), regardless of the microscopic details of the theory. This establishes that (at least for 2 dimensional CFTs) a theory renormalized in the UV has more degrees of freedom than one renormalized in the IR. John Cardy conjectured that this is true for all field theories, and proposed a candidate for $c$ in higher dimensional, or non-conformal QFTs\cite{10}. A similar result has been proven in 4 dimensions, called the $a$-theorem\cite{11}. As with the c-theorem, $a$ is a multiplicative constant of the anomalous divergence of the scale current in a CFT. It is not the only anomalous term, but it is the component that survives integration over spacetime of the anomaly. In this context, one often sees the following ``equation" in the literature

\begin{equation}
\int d^D x \braket{T^\mu_\mu}\footnote{Here we are careful to mention that the use of $\mu$ is to indicate a spacetime index, not the scale at which the theory has been renormalized at. When we wish to make RG scale dependence explicit, we will use square brackets, e.g. $c[\mu]$} \sim \text{anomaly}
\end{equation}

In all relevant classical field theories it is the case that the divergence of the scale current is equal to the trace of the stress energy tensor. Cardy's conjecture, simply put, is that the left hand side of this equation is a monotonically increasing function of $\mu$. In two and four dimensions, the $c$ and $a$-theorems respectively, are examples of Cardy's conjecture for theories which exhibit a pattern of spontaneous conformal symmetry breaking. To endow an interacting field theory with conformal invariance, it is generally necessary to couple it to the metric tensor. At least in the case of the proof of the $a$-theorem, this is done through a massless mediator known as the dilaton. The dilaton-matter and dilaton-metric interactions are tuned so that the trace of the total stess-energy tensor vanishes identically. Additionally, the dilaton can be coupled arbitrarily weakly to the matter theory, allowing one to study the trace anomaly perturbatively as a consequence of the RG flow between conformal fixed points\cite{11}. This is possible to do because the IR effective theory of the dilaton and metric tensor is highly constrained by the assumed conformal (or at least Weyl) symmetry, and the anomaly coefficient appears in the four-derivative terms of the effective action. This is effectively how the $a$ and $c$ theorems were established. In three dimensions, any attempt to replicate the proof of the $a$-theorem is doomed, since there are no conformal(Weyl)-invariant terms constructed of the Riemann tensor or its derivatives with which to build an IR effective theory of the dilaton and metric \cite{12}. It is thus often claimed that there is no trace anomaly in three-dimensional quantum field theories. We wish to emphasize now that this is meant to be a statement about CFTs and their pattern of symmetry breaking in three dimensions, not for three-dimensional field theories in general. The present work does \textit{not} consider this pattern of symmetry breaking. We will focus on the physically relevant case of non-conformal theories, and whether conformal/scale symmetry is acquired in the UV is of no consequence to our results. Therefore, we do not need to assume any particular properties of the IR effective actions, and the methods and language used here may be quite different than most literature on Cardy's conjecture and CFTs. The use of the word `anomaly' to describe this phenomena then takes on a different meaning: instead of a violation of some classical conservation law (in general there is no conserved scale current), the trace anomaly serves as an obstruction to using the stress-energy tensor as a generator of scale transformations in the quantum theory.\\

The layout of this paper is as follows. In Section 2 we will make the ``equation" above more clear. In particular, we will argue that what belongs in those brackets (what has been called the divergence of the scale current) is not the trace of the stress-energy tensor, but rather a different tensor $\theta^\mu_\mu$ which only happens to equal $T^\mu_\mu$ when quantum mechanics is turned off. This argument is phrased entirely in the context of RG invariance and makes obvious what role the anomalous scale dimensions of operators should play in the discussion.\\

In section 3, once Cardys conjecture is translated into a statement about the anomalous scaling, we will briefly discuss a criteria for the existence of solutions to the field equations that makes direct use of these anomalous dimensions. It is noticed that for IR strongly coupled theories, the anomalous dimensions generically behave in such a way that allows solutions to become manifest in the field equations of the renormalized theory which are not present in the classical theory - something which the present author has conjectured might happen\cite{13}. The conditions on which such solutions become manifest are proposed.\\

\section{Scale Transformations and RG Invariance}
\subsection{Anomalous Dimension}
When a theory is defined by an arbitrary renormalization scale $\mu$, it no longer obeys the scaling relations expected from classical field theory. For example, consider the correlation function of renormalized fields $\hat{\phi}[\mu]$ with couplings $g_i[\mu]$
\begin{equation}
G^{(3)}(\mu;g_i, x_1,x_2,x_3) = \braket{\hat{\phi}(x_1)\hat{\phi}(x_2)\hat{\phi}(x_3)}
\end{equation}

\noindent It is useful to imagine the result of re-scaling these coordinates by a proportionality constant $\lambda$. From classical physics, we have the so-called `engineering dimension' $\Delta$ of the field defined by
\begin{equation}
\phi(\lambda x) = \lambda^{-\Delta}\phi(x)
\end{equation}

\noindent and naively, one would expect the dimension of $G^{(3)}$ to be $-3\Delta$. This is not the case, as a scale transformation of this form must be accompanied by a change of renormalization scale $\mu \rightarrow \mu^\prime(\lambda)$ as well. The correct result, which is consistent with demanding invariance under change of renormalization scale, is

\begin{equation}
G^{(3)}(\mu;g_i,\lambda x_1,\lambda x_2,\lambda x_3) = \lambda^{-3(\Delta + \gamma(g_i))}G^{(3)}(\mu^\prime;g_i, x_1,x_2,x_3)
\end{equation}

\noindent where $\gamma$ is the anomalous dimension of the field, and on the right hand side of this equation, it is understood that $g_i$ is being renormalized at the scale $\mu^\prime$. In a perturbative theory, $\gamma$ is usually a polynomial function in the couplings that starts at quadratic order. Thus it is small. However, the above equation should be interpereted as an exact statement about the relationship between scale transformations and RG flow, irrespective of the theorists ability to actually calculate such quantities. In a non-perturbative regime, the anomalous dimensions could be potentially $\mathcal{O}(1)$ or greater.\\

The uniqueness of the scale $\mu^\prime$ is also an issue. We suspect that as one increases $\lambda$ and probes the large distance properties of the UV renormalized theory, the corresponding choice of $\mu^\prime$ should decrease, equating to sensitivity of the IR theory. If $\mu^\prime$ is then a monotonically decreasing function of $\lambda$, we should also then suspect that the anomalous dimension does not take the same values twice at different scales, ensuring the uniqueness of the formula. These expectations, as we shall see, are at the heart of Cardy's conjecture and are tied to expectations that RG flows cannot be cyclic\cite{9}.\\

It is also imperative that we discuss the anomalous dimension of composite field operators, like $\hat{\phi}^2$. The anomalous dimension of this operator is not equal to twice that of the fundamental field $\hat{\phi}$. It is easy enough to check that the divergence structure of $\braket{\hat{\phi}^2(x)}$ and $\braket{\hat{\phi}(x)\hat{\phi}(y)}$ are different, and so their renormalized counterparts must subtract divergences differently and therefore scale differently. The exact scaling dimension of operators at separated points will still be only the sum of the scaling dimensions of each operator; this is a consequence of the cluster decomposition property\cite{15}. This rule however does not apply for composite operators in interacting field theories: one might consider this a consequence of the operator product expansion.\\

\subsection{Scale Currents}

In a classical field theory there is a straightforward, easy way to calculate the effects of a scale transformation on solutions to the field equations. One must only identify the right hand side of
\begin{equation}
\phi(x^\mu + \delta x^\mu) = \phi(x^\mu) + \delta\phi(x^\mu)
\end{equation}

\noindent which with the identification $\delta x^\mu = (\lambda-1)x^\mu$ is exactly the scale transformation mentioned previously. With $\lambda$ expanded in a power series around the value of 1, a scale current is obtained by variation of the action in the usual way prescribed by Noether's theorem. The result is

\begin{equation}
j_\mu = x_\nu \theta^{\mu\nu}
\end{equation}

\noindent for some symmetric tensor which is conserved ($\partial_\mu \theta^{\mu\nu} = 0$) if the action is manifestly translation invariant. In the case of gauge theories, it is possible to choose $\delta \phi$ ($\phi$ is meant as a stand-in for any field) in such a way that this quantity is gauge invariant. The divergence of this scale current is therefore the trace
\begin{equation}
\partial_\mu j^\mu = \theta^\mu_\mu
\end{equation}

In a classically scale-invariant theory, this vanishes. However even when the theory is not classically scale invariant, this is a useful quantity to know. By the equations of motion, the spacetime integral of $\theta^\mu_\mu$ can be found to be proportional to any mass terms that are implicit in the Lagrangian. The scale current is therefore a useful tool for analyzing the breaking of scale invariance(or a larger conformal invariance), be it spontaneous or explicit. Given a Lagrangian composed of ``operators" $\mathcal{O}_i(x)$ and their engineering dimensions $\Delta_i$:
\begin{equation}
\mathcal{L} = \sum_i \mathcal{O}_i(x)
\end{equation}

\noindent it is quite easy to compute the divergence of this current in $D=d+1$ dimensions. By definition of the engineering dimensions, it is
\begin{equation}
\theta^\mu_\mu(x) = \sum_i (\Delta_i-D)\mathcal{O}_i(x)
\end{equation}

In many reasonable classical field theories, it happens to be the case that $\theta^{\mu\nu} = T^{\mu\nu}$, the stress energy tensor. As in the case of gravitational physics, $T^{\mu\nu}$ can be computed by variation of the action with respect to the metric tensor. In General Relativity the metric tensor is a dynamical thing so $T^{\mu\nu}$ emerges in its equations of motion, but in non-gravitational field theories the variation with respect to the metric still computes the same quantity. For a static solution to the field equations whose stresses vanish at spatial infinity, the integral of $T^\mu_\mu$ is exactly the energy/rest-mass of the solution\cite{16}, in agreement with the interpretation of $\theta^\mu_\mu$. So long as there is no explicit coordinate dependence in the Lagrangian, these two tensors will be the same, classically.\\

All of this changes once quantum mechanics is turned on. Scale invariance (if it is present) is broken by the RG flow, and can only be effectively restored if the flow terminates at a fixed point\cite{17}. This can happen, but exact scale invariance is broken and $\braket{\hat{\theta}^\mu_\mu}$ is anomalous. It is possible that this `trace anomaly' is largely responsible for the existence of \textit{all} mass scales in nature. In particular, Non-Abelian Yang-Mills theory with no matter fields is classically scale invariant, and the excitations are massless. In the quantum theory however, a mass gap is conjectured\cite{7} and the theory dynamically acquires a scale which was not present classically.\\

Furthermore one should not expect that $\hat{\theta}^{\mu\nu} = \hat{T}^{\mu\nu}$ remains to be generically true once quantum mechanics is involved. One reason is quite simple: variation with respect to the metric is not a quantum mechanical operation. If the theory is not a quantum theory of gravity, there is no reason to suspect that the metric dependence of a quantum action is sensitive to the quantum nature of the matter fields. For example, if
\begin{equation}
S_\text{int} = -\int d^D x \sqrt{-g}\hat{\phi}^4(x)
\end{equation}

\noindent then $\hat{T}^\mu_\mu$ necessarily contains a term $D \hat{\phi}^4(x)$. This is completely insensitive to the fact that the scaling dimension of this operator is dependent on what other operators are included in the Lagrangian as well as the scale $\mu$ that we have renormalized at. The derivation of $\hat{\theta}^\mu_\mu$ naturally leads to the inclusion of a term $-\left(\gamma_{\phi^4}-D\right)\hat{\phi}^4(x)$ which contains both information about the renromalization scale and the rest of the dynamics of the theory (here we have worked in a setting where $\Delta = 0$). If that is not enough to convince one that these tensors are different, consider the principle of RG invariance:

\subsection{RG Invariance and Cardy's Conjecture}

A principle tenant of the renormalization group procedure is the invariance of physical quantities with respect to $\mu$. The foremost example of a physical quantity, which is fundamental to the construction of any theory, is the invariant eigenvalue of the squared momentum operator $\hat{P}^2$. For a stationary state identified with a quanta of the theory, this is just the physical mass $M^2$. Consider a stationary state $\ket{\Psi}$, whose wavefunction vanishes sufficiently fast at spatial infinity. Manifest translation invariance implies, e.g. $\partial_\mu \hat{T}^{\mu i} =0$ or\footnote{This argument was borrowed from \cite{16}}
\begin{equation}
\int d^d x x_i \braket{\Psi|\partial_\mu \hat{T}^{\mu i}|\Psi} = \int d^d x x_i \partial_j\braket{\Psi| \hat{T}^{j i}|\Psi} = -\int d^d x \braket{\Psi|\hat{T}^i_i|\Psi} = 0
\end{equation}

Therefore, since the ``00" component of the stress energy tensor is the Hamiltonian, the spatial integral of $\braket{\Psi| \hat{T}^{\mu}_\mu|\Psi}$ is just the mass of the state, $M_\Psi$. For the vacuum, this is zero. How then could it be that the integrated trace of the stress energy tensor depends on $\mu$, for the vacuum or for other states? We argue of course that it does not, and that the operator relevant to Cardy's conjecture is not the true stress energy tensor, but rather $\hat{\theta}^\mu_\mu$ which generically differs from $\hat{T}^\mu_\mu$ by the $\mu$-dependent scale anomaly.\\

We will now re-state Cardy's conjecture. For a Lagrangian density as in eq. (8), where the terms are now true operators in a quantum theory, the divergence of the scale current is

\begin{equation}
\hat{\theta}^\mu_\mu(x) = \sum_i (\Delta_i + \gamma_i-D)\hat{\mathcal{O}}_i(x)
\end{equation}

\noindent where the RG scale dependence is now implicit in $\gamma_i$'s. Cardy's conjecture is then about the quantity

\begin{equation}
 \Theta[\mu] \equiv\sum_i (\Delta_i + \gamma_i-D)\int d^D x \braket{\hat{\mathcal{O}}_i(x)} = \int d^D x\braket{\hat{T}^\mu_\mu} + \int d^D x \sum_i\gamma_i \braket{\hat{\mathcal{O}}_i}
\end{equation}

Specifically, $\Theta_\text{IR} \leq \Theta_\text{UV}$, i.e. that it is a monotonically increasing function of $\mu$. The first term in the right hand side is a constant: the anomaly is represented as the remainder. Of course there are two dependencies on $\mu$ in the anomaly: that of $\gamma_i$'s and that of the expectation values $\braket{\hat{\mathcal{O}}_i(x)}$. We claim that a model-independent interpretation of Cardy's conjecture is really just a statement about the anomalous dimensions: $\gamma_i[\mu]$ increases monotonically as we flow to the IR. This would in fact affirm our earlier assumption that eq. (4) necessitated a unique choice of $\mu^\prime$. As a reminder, were this not the case we should conclude that RG flows can be cyclic, something which certainly should not describe any physical system. For the remainder of this paper we will work under the assumption that this is the correct interpretation of Cardy's conjecture, and that the conjecture is correct.\\

\section{Finding New States}

Briefly, let's take a detour back to the land of classical field theory. Consider the case of an interacting Klein-Gordon field in 2+1 dimensions, with a $\mathbb{Z}_2$ potential in its spontaneously broken phase (perhaps $V(\phi) = -m^2\phi^2 + g_6 \phi^6 + V_0$). Naively, one might suspect that domain walls are formed, and could be stable. This is not the case. We divide the Hamiltonian into kinetic and potential terms

\begin{equation}
H = K+U
\end{equation}

\noindent where K and U are positive definite. We now imagine scaling a static domain wall of spatial size 1 to a size $\lambda^{-1}$. Since $\Delta_K = 2$ and $\Delta_U = 0$ we have in $d=2$ spatial dimensions:

\begin{equation}
H[\lambda] = K + \lambda^{-2}U
\end{equation}

\noindent which cannot be minimized except at $\lambda = \infty$. Therefore the domain wall is unstable to shrinking indefinitely (this is of course due to the unbalanced tension on the wall). Furthermore, no such extended field configurations exist and are stable in this model. The above classical scaling argument is the core of what is known as Derrick's theorem\cite{1}.\\

\subsection{Large Anomalous Dimensions}

The story is potentially much different in the quantum theory. It is easy to see how different things can be by assuming at first $K$ and $U$ acquire anomalous dimensions $\gamma_K$ and $\gamma_U$. The conditions for a stable state then become the condition for a local minima of $H[\lambda]$ at $\lambda = 1$:
\begin{align}
\gamma_K K + (\gamma_U - 2)U &= 0\\
\gamma_K (\gamma_K -1) K + (\gamma_U - 2)(\gamma_U - 3)U &> 0
\end{align}

Alternatively (by eliminating $U$ for $K$ using eq. (16)), $(2-\gamma_U)/\gamma_K> 0$ and $\gamma_K(2+\gamma_K +\gamma_U) > 0$. Now obviously in a real model the terms that appear in the potential energy will not all have identical scaling dimensions. The argument is clear though; the activation of anomalous scaling of operators that contribute to the energy density of a state can circumvent the conclusions of Derrick's theorem. This of course depends on the relative signs and magnitudes of the anomalous dimensions in question. This is where Cardy's conjecture enters the picture.\\

If the theory is weakly coupled in the IR, $\gamma_K$ and $\gamma_U$ are small, close to zero. As one increases the RG scale, these terms either decrease or stay the same. At some UV scale then, they stand the chance of being large (perhaps before the theory is embedded to become UV complete) but negative. The stated conditions for circumvention of Derrick's theorem are then not satisfied.\\

This theory in 2+1 dimensions happens to have a negative beta function - it is strongly coupled in the IR and weakly coupled in the UV\cite{18}. Suppose then that in the UV where the anomalous dimensions are close to zero (and possibly negative) Derrick's theorem is not circumvented. Cardy's conjecture implies then that as we flow to the IR, $\gamma_K$ and $\gamma_U$ grow to become large as we approach the non-perturbative regime. If the flow does not terminate at a fixed point before this happens, the anomalous dimensions stand a chance of approaching unity (and positive), where the above conditions can almost certainly be satisfied. The meaning of the title of this paper is now revealed: it is precisely the case of an IR strongly-coupled QFT where one expects new massive solitonic states to become manifest in the field equations. That they are not manifest as solutions along the entire RG trajectory should not be surprising: perturbation theory in the UV is by construction only sensitive to a subsection of the total Hilbert space of a QFT\cite{13}.\\

We did not just prove that stable domain walls will exist in the 2+1 dimensional scalar field theory described at the beginning of this section. Rather, we used this simple case to visualize the potential for strongly-coupled theories to host solutions which have no classical analogue. Whether a theory actually hosts such solitonic states must be checked on a case-by-case level, and is subject to details that we will not address here. In upcoming works we will investigate specific cases and demonstrate that signifigant deviations from classical estimates of soliton masses may occur.\\

\subsection{The Masses}

Suppose a theorist makes some exact non-perturbative evaluation of the operator dimensions in the Hamiltonian at some UV scale $\mu$, and uses the beta function for all relevant coupling constants to put bounds on how slowly those dimensions evolve with $\mu$. This theorist then discovers that by our monotinicity arguments, conditions (16) and (17) are met as one flows to the IR, at an RG scale no smaller than $\mu^*$. What then is the mass of the state which emerges there, and how does it depend on the value of $\mu^*$?\\

We don't have an exact answer to this question of course, and necessarily this is a regime where perturbative calculations of anomalous dimensions are likely not very accurate. But if the conditions above can be met, one should be able to put a lower bound on the mass of the lightest solitonic state. Generically we should consider a time-independent state (i.e. a static field configuration) and a Hamiltonian of the form

\begin{align}
H = \sum_i H_i[\mu] + \Omega[\mu]
\end{align}

\noindent where the $H_i$ are derived from the integrated expectation values (with respect to the massive state, not the vacuum) of operators in eq. (8), and $\Omega$ is a free energy. Often one will encounter texts which claim this free energy is a cosmological constant, but this is not correct: while renormalization of vacuum graphs requires the introduction of a field-independent term in the action to regulate divergences, this is an RG-variant term and is compatible with a zero vacuum energy. The true meaning of $\Omega[\mu]$ is that of the free energy associated with the invisible fluctuation modes with momenta greater than $\mu$. Both $\Omega$ and the $H_i$ are RG-variant quantities, but by construction their sum should not be.\\

What matters now is essentially applying stability conditions (16) and (17) to our Hamiltonian (18). The result is not literally (16) and (17), but rather (in $d$ spatial dimensions)

\begin{align}
&\sum_i (\Delta_i+\gamma_i - d)H_i + (\gamma_\Omega -d)\Omega = 0\\
&\sum_i (\Delta_i+\gamma_i - d)(\Delta_i+\gamma_i - d+1)H_i + (\gamma_\Omega -d)(\gamma_\Omega -d+1)\Omega  > 0
\end{align}

Here $\Omega$ is assumed to have no engineering dimension, and only acquires dimension $\gamma_\Omega$ through the anomalous running of the renormalized coupling constants of which it is a function. Assuming that conditions (19) and (20) are met at and below the RG scale $\mu^*$, we now have known relationships between the numerical values of $H_i[\mu^*]$ and $\Omega[\mu^*]$. As is typical with classical extended field configurations, all of these terms should generically differ only by $\mathcal{O}(1)$ coefficients\cite{19}. The anomalous dimensions will generically\footnote{The previous 2+1 dimensional example was a special case. There it seems that these solutions turn on as soon as $\gamma$'s are positive, even if very small. However, in this number of dimensions many models already host solitonic states, e.g. vortices, a consequence of the fact that $\Delta_K = d = 2$. In higher dimensions, where soliton solutions are generically not present, the $\gamma$'s have more work to do and therefore will be $\mathcal{O}(d-\Delta)$ at $\mu^*$} be $\mathcal{O}(d-\Delta)\sim\mathcal{O}(1)$ at $\mu^*$ and since the coefficients of the $H_i$'s and $\Omega$ in eqs. (19) and (20) will be differing in sign, we argue that $H \sim \Omega[\mu^*]$ is as good an estimate for the lower bound of the mass as any. What is the value of $\Omega[\mu^*]$? Naively, it is some integral of the free energy density $\omega(g_i)$, a function of the couplings of the theory renormalized at $\mu^*$. The correct expression should depend on some normalized energy density profile of the state, and since this is at the moment indeterminate, we conjecture the following:

\begin{align}
M \sim \Omega[\mu^*] \gtrsim \frac{\omega[\mu^*]}{(\mu^*)^d}
\end{align}

Since the state only manifests as a solution to our quantum equations of motion when we are insensitive to distance scales less than $1/\mu^*$, the volume of the physical object should be at least $(1/\mu^*)^d$. The natural guess is then eq. (21).

\section{Conclusion and Going Forward}

We have interpreted Cardy's conjecture as a statement about the change of anomalous dimensions under RG flow. The basic requirements that RG flows not be cyclic and that an IR renormalized theory have less degrees of freedom than a UV renormalized theory are realized if some scaling dimensions are larger in the IR than in the UV. This becomes consequential if those dimensions deviate from classical values by $\mathcal{O}(1)$ corrections, and those deviations are positive. This happens when the theory is IR strongly coupled, and it opens the door for a circumvention of Derrick's classical no-go theorem. Should new solutions to the renormalized field equations emerge, we identify them as solitons and have proposed a way to estimate a lower bound on their mass using perturbation theory in the UV. Such a thing is only possible because of monotinicity arguments. Once the solutions are found and characterized, we propose a reorganization of the degrees of freedom at scales at and below $\mu^*$ to reflect the manifestation of these new solutions. One should expand perturbatively around such solutions, rather than the free field theory configurations.

\newpage


\section{Long-range correlated disorder and self-consistent Born approximation}
\subsection{Long-range interacting impurity}
On a square lattice, at each site $\bm{R}$, we introduce an impurity whose strength $W(\bm{R})$ independently takes a random value from a uniform distribution $[-W/2,W/2]$. These impurities are uncorrelated by construction
\begin{equation}
	\mean{W(\bm{R})} = 0,\quad \mean{W(\bm{R})W(\bm{R}')} = \frac{W^2}{12}\delta_{\bm{R},\bm{R}'} = \frac{W^2a^2}{12}\delta(\bm{R}-\bm{R}')
\end{equation}
The lattice constant $a$ appears when we promote the discrete Kronecker to the continuum field description because we place an impurity every distance $a$ in a two dimensional lattice. We now argue that if these impurities interact with electrons via a finite-range interaction, the effective disorder field felt by an electron is effectively long-range correlated. We assume this electron-impurity interaction is given by a Gaussian function so that the net disorder potential is
\begin{equation}
	u(\bm{r}) = \int  \frac{e^{-(\bm{r}-\bm{R})^2/\xi^2}}{\pi \xi^2}W(\bm{R})d^2\bm{R}.
\end{equation}
As a result, $u(\bm{r})$ is spatially correlated by
\begin{equation}\label{correlated}
	\begin{split}
	 \mean{u(\bm{r})u(\bm{r}')} = \frac{W^2a^2}{12\pi^2\xi^4}\int e^{-(\bm{r}-\bm{R})^2/\xi^2}e^{-(\bm{r}'-\bm{R})^2/\xi^2}d^2\bm{R} = \frac{W^2a^2}{24\pi\xi^2}e^{-(\bm{r}-\bm{r}')^2/2\xi^2}
	\end{split}
\end{equation}

\begin{figure}
	\centering
	\includegraphics[width=0.95\textwidth]{SM1.pdf}
	\caption{Self-consistent diagram for the disorder-induced self-energy. The double-line represents the renormalized Green function with the self-energy included. The cross denotes the impurity and the dashed lines denote the impurity-electron interaction. The self-consistent diagram can be thought of as the sum of all non-crossing diagrams with single solid lines representing the bare Green function. After averaging over disorder configurations, each dashed line segment connecting two points of a Green function is substituted by Eq.~\ref{correlated}. \label{fig1}}
\end{figure}

The self-consistent Born approximation sums all the non-crossing diagrams as shown in in Fig.~\ref{fig1}. We also do not consider more-than-two-point correlation functions as in the $T-$matrix approximation.  After averaging over disorder configurations, the theory recovers the translational symmetry so we can compute the self-energy self-consistently as
    \begin{equation}\label{selfconsistent}
\begin{split}
\Sigma &= \frac{W^2a^2}{12}\int \frac{d^2\bm{k}}{(2\pi)^2} \frac{e^{-k^2\xi^2/2}}{\mu-H(m,\alpha,\beta,\gamma;\bm{k})-\Sigma} = \frac{W^2a^2}{12\xi^2}\int \frac{d^2\bm{k}}{(2\pi)^2} \frac{e^{-k^2/2}}{\mu-H(m,\alpha/\xi,\beta/\xi^2,\gamma/\xi^2;\bm{k})-\Sigma}
\end{split}
\end{equation}
The second equality is constructed by rescaling the integration variable $\bm{k}\to \bm{k}\xi$. Compared with the case of uncorrelated disorder, the extra exponential decay comes from the Fourier transform of the finite-range function $O(\bm{r}-\bm{r}')=\mean{W(\bm{r})W(\bm{r}')}$. This term provides a natural cutoff for the integral so we do not need to impose an artificial UV cutoff. For brevity, we define the rescaled quantities $\bar{W}=W/\xi, \bar{\alpha}=\alpha/\xi, \bar{\beta}=\beta/\xi^2$, and $\bar{\gamma}=\gamma/\xi^2$. The integral in Eq.~\ref{selfconsistent} can be performed exactly, yielding an explicit systems of equation
\begin{equation}\label{SCsystem}
\begin{split}
 &\Sigma_0 = \frac{\bar{W}^2}{48\pi \Delta}\left[ F(x_2)\left(\mu-\Sigma_0 + 2\bar{\gamma}x_2\right) - F(x_1)\left(\mu-\Sigma_0 + 2\bar{\gamma}x_1\right)\right],\\
 &\Sigma_z = \frac{\bar{W}^2}{48\pi \Delta}\left[ F(x_2)\left(m+\Sigma_z - 2\bar{\beta}x_2\right) - F(x_1)\left(m+\Sigma_z - 2\bar{\beta}x_1\right)\right]
\end{split}
\end{equation}
Here, we decompose the self-energy matrix as $\Sigma=\Sigma_0\sigma_0+\Sigma_z\sigma_z$. Other terms in Eq.~\ref{SCsystem} are given by
\begin{equation}
\begin{split}
  & F(x) = e^x\left[i\pi \text{sign}[\text{Im}(x)] + \text{Ei}(-x)\right]\\
  &\Delta = (a^2-4bc)^{1/2},\quad x_{1,2} = \frac{a\mp \Delta}{4b},\\
  &a=-\bar{\alpha}^2-2\bar{\gamma}(\mu-\Sigma_0)-2\bar{\beta}(m+\Sigma_z),\quad b=\bar{\gamma}^2-\bar{\beta}^2, \quad c=(\mu-\Sigma_0)^2-(m+\Sigma_z)^2
\end{split}
\end{equation}
The solution of the Eq.~\eqref{SCsystem} is transcendental but can be obtained quickly through multiple numerical iterations. Then, the renormalized mass term and chemical potential are
\begin{equation}
	\bar{\mu} = \mu -\text{Re}(\Sigma_0),\quad \bar{m} = m +\text{Re}(\Sigma_z).
\end{equation}
We emphasize that it is necessary to obtain the full self-consistent solution. If one assume $\Sigma=0$ in the RHS of Eq.~\ref{selfconsistent}, a simple expression for the solution can be found but its accuracy might be limited and more importantly, it is not clear how the imaginary part of the self-energy can be obtained.

\subsection{$L^{-4}$ scaling of the CTI phase}
In this subsection, we show that quantized conductance is reduced by the edge-edge hybridization probability, which is suppressed by $L^{-4}$ in the weak-coupling limit. The longitudinal conductance is given by $G=e^2v_F\rho$ where $v_F$ is the Fermi velocity and $\rho$ is the density of states at the Fermi energy. For a chiral state $\psi(x) = D^{-1/2}e^{ik_Fx}$ with $D$ being the longitudinal dimension for normalization, $v_F=\text{Re}\bra{\psi}i\partial_x\ket{\psi}/m_e$ and $\rho=1/2\pi\hbar v_F$, leading to the quantized conductance $G=e^2/h$. We assume that a small coupling between opposite chiral edges, which effectively acts as a backscattering, mainly reduces the propagating velocity. Specifically, if $\psi(x)\to\psi'(x)\sim \sqrt{1-|\eta|^2}e^{ikx} + \eta e^{-ikx}$ with $\eta$ being the mixing amplitude, then $v_F\to v_F' = v_F(1-2|\eta|^2)$. As a result, the conductance loss $1-G$ is proportional to the hybridization probability.

We now estimate the scaling of the hybridization probability. We assume that the probability of an excitation of energy $E$ to overlap with an eigenstate of energy $E+\Delta E$ is given by a function $F(\Delta E /\Gamma)$ where $\Gamma$ is the self-energy-induced incoherent level broadening. Here, $F(x)$ decreases with increasing $|x|$. Eigenstates within the energy band gap  decay exponentially into the bulk, so the main contribution comes from states near the gap edge where the localization length diverges. Specifically, the probability of an edge mode of energy $E_0$ to appear at a distance $y$ in the bulk is
\begin{equation}\label{scaling}
P(E_0, y) = \int F\left(\frac{E-\bar{E}_0}{\Gamma} \right) e^{-Cy\sqrt{|\bar{m}|-E}}dE \propto y^{-2}F\left(\frac{|\bar{m}|-\bar{E}_0}{\Gamma} \right).
\end{equation}
Here, $\bar{E}_0$ is the disorder-renormalized $E_0$ and $|\bar{m}|$ is the upper edge of bulk gap with $\bar{m}$ being the renormalized mass in the disorder-averaged Hamiltonian. The same argument can be applied to the leakage into the bulk through the lower edge $-|\bar{m}|$ of the bulk band.  The energy level broadening is $\Gamma=\text{Im}\Sigma_0$. The probability of inter-edge overlap then follows analytically  from Eq.~\eqref{scaling} to scale as $L^{-4}$, where $L$ is the sample width or the inter-edge separation. We compare Eq.~\ref{scaling} with the disorder-induced density of states inside the bulk gap
\begin{equation}
	\rho(E_0) = \frac{1}{\pi}\left[ \text{Tr} \int\frac{d^2\bm{k}}{E_0-H_0-\Sigma}  \right] \approx -\frac{1}{\pi} \int\frac{ \text{Tr}\left[(\text{Im}\Sigma_0)\sigma_0 +(\text{Im}\Sigma_z)\sigma_z\right]}{(\bar{m}^2-\bar{E}_0^2) + \alpha^2k^2 +\mathcal{O}(k^4)}   d^2\bm{k} \sim \frac{\text{Im}\Sigma_0}{\bar{m}^2-\bar{E}_0^2}.
\end{equation}
As $\rho(E_0)$ increases, the edge-edge hybridization probability grows concomitantly. This shows the consistence of our Born approximation with the microscopic picture that in the CTI phase, the two edges couple through bulk percolating states inside the bulk gap.

\subsection{TAI suppression by long-range disorder}

Even though an apparent solution for the self-energy is generally not known, some properties can be derived in the large-$\xi$ limit where $\bar{\beta},\bar{\gamma}$ and $\bar{\alpha}^2$ are suppressed by a factor $\xi^{-2}$. We can then simply Eq.~\eqref{SCsystem} into
\begin{equation}
 \begin{split}
  &\Sigma_0 = \frac{\bar{W}^2}{24\pi} \frac{\mu-\Sigma_0}{(\mu-\Sigma_0)^2-(m+\Sigma_3)^2} \\
   &\Sigma_z = \frac{\bar{W}^2}{24\pi} \frac{m+\Sigma_z}{(\mu-\Sigma_0)^2-(m+\Sigma_z)^2}
 \end{split}
\end{equation}
Then it easy to show that if one start at the clean energy gap $\mu=\pm m$ (actually $|\mu|=|m|-\epsilon$ with infinitesimal $\epsilon$ to avoid singularity) then the renormalized value satisfies $\bar{\mu}=\pm\bar{m}$, i.e. the width of the bulk gap is unchanged by the disorder, ruling out the existence of the TAI phase. In addition, one has the symmetry $\Sigma_0\to-\Sigma_0$ for $\mu\to-\mu$, so the the solution is symmetric around $\mu=0$.  This limit is approximately reached when $\xi^2\sim (\gamma,\beta,\alpha^2)/|m|$. However, we now argue the TAI phase is suppressed much sooner than this limit mainly due to the fast growth of the imaginary part of the self-energy.

\begin{figure}
	\centering
	\includegraphics[width=0.95\textwidth]{SM2.pdf}
\caption{Variations of $\text{Im}(\Sigma_0)$ (a), $\bar{m}^2-\bar{\mu}^2$ or the squared gap width (b), and $\Xi$ (c) as a function of the disorder correlation length $\xi$. The parameters are $m=1, \alpha=120,\beta=400,\gamma=200,\bar{W}=300,\mu=5$ (corresponding to Fig.3 in the main text). \label{fig2}}
\end{figure}

As shown in Fig.~\ref{fig2}(a), the imaginary part of the Green function grows rapidly beyond the critical point. Comparing  Figs.~\ref{fig2}(a) and (c), the variation of the $\Xi$-factor used to bound the TAI region is almost concomitant with that of the imaginary part. As a result, the threshold $\Xi=2$ is reached when the renormalized gap is still open and $\bar{m}<0$. For the given parameters, based on the gap-opening condition only, the threshold $\xi$ for a complete TAI loss should be 8; while according to our numerical simulation, this point $\bar{W}=300,\mu=5$ is already AI for $\xi=3$, which agrees better with the threshold $\xi=2.5$ from the self-consistent Born approximation [see the crossing point in Fig.~\ref{fig2}(c)].

\section{Additional numerical data}

\subsection{One-parameter scaling function}
In this subsection, we provide more numerical evidence to support our one-parameter scaling ansatz. In Figs.~\ref{fig3}(a-c), we present the scaling trend across the CTI-AI transition at different chemical potentials $\mu$. Within the error caused by random fluctuations, there is little size dependence for $G<0.3$, supporting that the CTI-AI critical value is universal given a certain set of lattice parameters. In the main text, we already demonstrate the one-parameter scaling function within the CTI phase. Now to fully map out this function, we add two more sets of data near the CTI-AI phase boundary as shown in Fig.~\ref{fig3}(d) and are able fit the collapsed data in a smooth line (dashed line). By taking the log derivative of the fitted line, we compute the scaling exponent $\beta=d\ln(1-G)/\ln L$ in Fig.~\ref{fig3}(e). The scaling exponent approaches $-4$ in the limit $G\to 1$, then graduate increases until the critical value $G\sim 0.3$ and stays zero afterward. For the transverse conductance, one should see the $\beta$ function cutting through zero axis at $G=0.5$ and turning positive afterward. We explain the reason in the main text.

\begin{figure}
	\centering
	\includegraphics[width=0.9\textwidth]{SM3.pdf}
	\caption{(a-c) Finite-size scaling across the CTI-AI transition with respect to the disorder strength $\bar{W}$ at different applied chemical potential $\mu$. (d) Quantization error with respect to the scaled size. The red dashed line is fitted from the numerical data. (e) Scaling exponent derived from the fitted line in (d). The $1-G$ axis in (e) is plotted in the $\ln[x/(1-x)]$ scale. The lattice parameters are similar to Fig.~(1) in the main text.  \label{fig3}}
\end{figure}

\subsection{Other types of disorders}
In this subsection, we consider the full time-reversal invariant Hamiltonian, i.e. two copies of the Hamiltonian we used in the main text, so that $H_{AII}(\bm{k})=h(\bm{k})\oplus -h^*(-\bm{k})$ with the time-reversing operator $i\mathcal{K}\tau_y$ and $\mathcal{K}$ being the complex conjugate. Here $\tau_{x,y,z,0}$ are the Pauli matrices corresponding to the spin degrees of freedom. Two more types of disorders can be studied in this case, i.e. the time-reversal symmetric spin-orbit coupling (SOC) disorder $u(\bm{r})\tau_x\sigma_y$ and the time-reversal breaking magnetic disorder $u(\bm{r})\tau_x\sigma_x$. In the former case, the difference from the onsite potential studied in the main text is that in one spin sector
\begin{equation}
	\Sigma = \sigma_y\left[\frac{W^2a^2}{12}\int \frac{d^2\bm{k}}{(2\pi)^2} \frac{e^{-k^2\xi^2/2}}{\mu-h(\bm{k})-\Sigma}\right]\sigma_y.
\end{equation}
Since $\sigma_y\sigma_z\sigma_y=-\sigma_z$, the sign of the mass correction is inverted. As a result, a clean non-trivial system ($m<0$) will be eventually trivialized when $\bar{m}>0$. This is reflected in Fig.~\ref{fig4}(a). On the other hand, a clean trivial system $(m>0)$ continues to stay trivial. For the case of time-reversal symmetry breaking disorder, the quantization plateau is destroyed at a much weaker strength of disorder compared to the cases of symmetry-preserving disorders [see Fig.~\ref{fig4}(b)], consistent with the fact that the topology of AII systems is protected by the time-reversal symmetry.

\begin{figure}
	\includegraphics[width=0.7\textwidth]{SM4.pdf}
	\caption{Conductance in the presence of SOC (a) and magnetic (b) disorders. The black lines in panel (a) represent the renormalized energy gap with the mass term inverted. The parameters for the clean system are similar to Fig.~2(a) in the main text with $\xi=2$. For the symmetry-breaking disorder, the quantized conductance is lost at $\bar{W}\sim \mathcal{O}(1)$; while in the presence of symmetry-preserving disorder the quantization plateau can extend up to $\bar{W}\sim \mathcal{O}(10^2)$\label{fig4}. }
\end{figure}

\subsection{Pseudo-quantized conductance plateau}
In Fig.~\ref{fig5}, we show that in the CTI phase, due to the slow convergence of the conductance to the quantized value (especially near the CTI-AI transition), it is possible to observe a conductance plateau by tuning the chemical potential but the plateau value is not precisely quantized.

\begin{figure}
	\includegraphics[width=0.5\textwidth]{SM5.pdf}
	\caption{Conductance as a function of the applied chemical potential for different strengths of disorder. The parameters for the clean system are similar to Fig.~2(c) in the main text with $\xi=8$. \label{fig5}. The quantization error of the conductance plateau gets worse as the disorder strength increases, driving the system from TI to CTI phase.}
\end{figure}

\end{document}


\section{Introduction}
Methods of modern spectroscopy make it possible to measure hyperfine splittings in electronic states of atoms and molecules with high accuracy, resulting in a large amount of data for the analysis and interpretation \cite{barzakh2013changes, barzakh2017changes, beiersdorfer2001hyperfine,schmidt2018nuclear,Petrov:13,Mawhorter:2011}. These data are important not only for the nuclear theory but also for the development and testing methods of precise electronic structure calculations. Such methods are required in the field of searching for the effects of spatial and time-reversal symmetry violating effects of fundamental interactions in atomic and molecular systems~\cite{GFreview,SafronovaRev:2018,Skripnikov:14c,Skripnikov:17c}.

To reproduce accurately experimental values of hyperfine splitting energy in heavy atoms with uncertainty of the order of 1\% one has to take into account both relativistic and correlation effects at a very good level of theory. Moreover, one should also take into account nuclear structure contributions (as well as the quantum electrodynamics (QED) corrections~\cite{Ginges:2018}). These are contributions from the distribution of the charge (Breit-Rosental effect) \cite{rosenthal1932isotope, crawford1949mf} and magnetization (Bohr-Weisskopf effect) \cite{bohr1950influence, bohr1951bohr} over the nucleus. For many atomic systems these effects have been extensively studied (for example see Refs.
\cite{maartensson1995magnetic, konovalova2017calculation, gomez2008nuclear, ginges2017ground, dzuba1984va}).

In the point nucleus model the ratio of the hyperfine splittings of two different isotopes is proportional to the ratio of nuclear \emph{g}-factors of the isotopes. However, this is not the case for the real system due to the mentioned effects. Corresponding correction is called a magnetic anomaly:
\begin{equation}
{}^{1}\Delta^{2}=\frac{A_1 \mu_2 I_1}{A_2 \mu_1 I_2}-1,
\end{equation}
where $A_1$ and $A_2$ are hyperfine structure (HFS) constants, $\mu_1$ and $\mu_2$ are nuclear magnetic moments and $I_1$ and $I_2$ are nuclear spins of considered isotopes.

Theoretical value of the anomaly strongly depends on the model of the magnetization distribution. However, as was noted previously (e.g. in  Refs.~\cite{0954-3899-37-11-113101,schmidt2018nuclear,barzakh2012hyperfine}) the ratio of anomalies for different electronic states has a very small model dependence. This feature has been employed here to predict nuclear magnetic moments of short lived ${}^{191}$Tl and ${}^{193}$Tl isotopes.

In the present paper relativistic coupled cluster calculations of the hypefine structure constants and magnetic anomaly in the thallium atom for several electronic states are performed. We show that for this case it is possible to use a simple model of the magnetization distribution in Tl isotopes and fix the parameter of the model from the available experimental data and use it for further predictions. We show that for such electronic structure calculations one can use Gaussian basis sets which implies a direct generalization to similar molecular problems.

\section{Theory}

For both considered stable isotopes of thallium ($^{203}$Tl and $^{205}$Tl) the total nuclear moment is $1/2$. We consider a simple one-particle model in which the nuclear magnetization is due to single spherically distributed valence nucleon (proton) with the nuclear spin equal to $1/2$ and zero orbital moment. The hyperfine interaction in the case of point magnetic dipole moment is given by the following Hamiltonian:
\begin{equation}
\label{hfs1}
h^{\rm HFS}=\frac{1}{c}\frac{\mathbf{\mu}\cdot[\mathbf{r}_{el}\times \bm{\alpha}]}{r_{el}^3},
\end{equation}
where $\bm{\alpha}$ are Dirac matrices and $\mathbf{r}_{el}$ is the electron radius-vector. In the model of the uniformly magnetized ball with radius $R_m$ this interaction inside the nucleus modifies to the following form~\cite{bohr1950influence}:
\begin{equation}
\label{hfs2}
\frac{1}{c}\frac{\mathbf{\mu}\cdot[\mathbf{r}_{el}\times \bm{\alpha}]}{R_m^3}.
\end{equation}
Outside the nucleus the expression for the interaction coincides with that of the point dipole given by Eq. (\ref{hfs1}). In this paper we use the root-mean-square radius $r$ associated with the radius of the ball $R$ by $R=(\frac{5}{3} r)^{1/2}$.

For the hyperfine structure constant one often uses the following parametrization:
\begin{equation}
    A=A_0(1-\delta)(1-\epsilon),
\end{equation}
where $A_0$ is the hyperfine structure constant corresponding to the point nucleus, $\delta$ is the Breit-Rosenthal correction and $\epsilon$ is the Bohr-Weisskopf (BW) correction. The former takes into account the finite charge distribution and is about 10\% for heavy atoms like Tl \cite{shabaev1994hyperfine}. The latter concerns finite magnetization distribution and usually smaller than $\delta$.

In the relativistic correlation calculations of the neutral atoms it is more practical to use the finite charge distribution model from the beginning. In the present paper the Gaussian charge distribution has been employed. Thus, we do not introduce the Breit-Rosenthal correction. In this case the Bohr-Weisskopf correction $\epsilon$ is a function of both nuclear charge and magnetic radii: $\epsilon = \epsilon(r_c, r_m)$.

For hydrogen-like ions one can use the following analytic expression for the hyperfine splitting energy~\cite{shabaev1997ground}:

\begin{equation} \label{eq1}
\begin{split}
\Delta E & = \frac{4}{3}\alpha (\alpha Z)^3
    \frac{\mu}{\mu _N} \frac{m}{m _p}
    \frac{2I+1}{2I}mc^2 \\
 & \times \big (A(\alpha Z)(1-\delta)
(1-\varepsilon)+x_{rad} \big)
\end{split}
\end{equation}
where $\alpha$ is the fine-structure constant, $Z$ is the nuclear charge, $m_p$ is the proton mass, $m$ is the electron mass and $x_{rad}$ is the correction due to QED effects. Besides, the following analytic behavior for the Breit-Rosenthal correction was obtained~\cite{ionesco1960nuclear}:
\begin{equation}
\label{dep}
\delta = b_N \cdot R_N^{2\gamma -1}
\text{   ,   }
\gamma = \sqrt{\kappa^2-(\alpha Z)^2}.
\end{equation}
Here $b_N$ is a constant independent of the nuclear structure, $\kappa$ is the relativistic quantum number.
This expression can be used to test numerical approaches.

Finally, in Ref.~\cite{atoms6030039} the following parametrization of the hyperfine structure constant and Bohr-Weisskopf correction has been used:
\begin{eqnarray}
    \label{d_nuc_dep}
    A=A_0(1-(b_N+b_M d_{nuc}R_N^{2\gamma - 1})),\\
    \epsilon(R_N, d_{nuc})=b_M d_{nuc}R_N^{2\gamma - 1},
\end{eqnarray}
where $b_M$ is the electronic factor independent of the nuclear radius and structure and $d_{nuc}$ is a factor which depends only on the nuclear spin and configuration.

\section{Calculation details}

In the calculations of the hyperfine splittings in the hydrogen-like thallium, the values of the QED corrections from Ref.~\cite{shabaev1997ground} were used. The values of the nuclear charge radii were taken from Ref.~\cite{ANGELI201369}. The nuclear magnetic moments of $^{203}$Tl and $^{205}$Tl isotopes were taken from Ref.~\cite{stone2005table}.

For test calculations of the hydrogen-like Tl three basis sets were used: CVDZ~\cite{Dyall:06,Dyall:98} which consists of $24s, 20p, 14d, 9f$ uncontracted Gaussian functions with a maximum $s$-type exponent parameter equal to $5.8 \cdot 10^7$, GEOM1 which consists of $50s$ functions with exponent parameters forming the geometric progression, where a common ratio is equal to $1.8$ and the largest term is $5 \cdot 10^8$, as well as GEOM2, which differs from the previous one only in that the first term is $5 \cdot 10^9$.

In the calculations of the neutral thallium atoms the QED effects were not taken into account. The main all-electron (81e) correlation calculations were performed within the coupled cluster with single, double and perturbative triple cluster amplitudes, CCSD(T), method \cite{bartlett2007coupled} using the Dirac-Coulomb Hamiltonian.
In the calculation the the AAE4Z basis set~\cite{Dyall:12} augmented with one $h$ and one $i$ functions was used. It includes $35s, 32p, 22d, 16f, 10g, 5h, 2i$. This basis set is called LBas below. For the calculation virtual orbitals were truncated at the energy of 10000 Hartree. Importance of the high energy cutoff for properties dependent on the behaviour of the wave function close to a heavy atom nucleus were analyzed in Ref.~\cite{Skripnikov:17a}. Besides, the basis set correction has been calculated. For this we have considered the extended basis set LBasExt which includes $44s, 40p, 31d, 24f, 15g, 9h, 8i$ basis functions. This correction has been calculated within the CCSD(T) method with frozen $1s..3d$ electrons. Virtual orbitals were truncated at the energy of 150 Hartree.
To test convergence with respect to electron correlation effects we have used the SBas basis set which consists of $30s, 26p, 15d, 9f$ functions and equals to the CVDZ~\cite{Dyall:06,Dyall:98} basis set augmented by diffuse functions.
To calculate the correlation correction $1s-3d$ electrons were excluded from the correlation treatment. In such a way correlation effects up to the level of the coupled cluster with single, double, triple and perturbative quadruple amplitudes, CCSDT(Q) method were considered. Contribution of the Gaunt interaction was calculated within the SBas basis set using the CCSD(T) method. In this calculation all electrons were correlated.

For a number of intermediate all-electron correlation calculations we also used the MBas basis set consisting of $31s, 27p, 18d, 13f, 4g, 1h$ functions and corresponding to the AAETZ basis set~\cite{Dyall:06}.

For the relativistic coupled cluster calculations the {\sc dirac15} \cite{DIRAC15} and {\sc mrcc} codes \cite{MRCC2013,Kallay:1,Kallay:2} were used. The code developed in Ref. \cite{Skripnikov:16b} has been used to compute point dipole magnetic HFS constant integrals. To treat the Gaunt interaction contributions we used the code developed in Ref.~\cite{Maison:2019}. The code to compute the Bohr-Weisskopf correction was developed in the present paper.

\section{Results and discussion}
\subsection{Hydrogen-like thallium \texorpdfstring{$1S_{1/2}$}{1S 1/2} }

Fig. \ref{figure:1} shows calculated dependence of the hyperfine splitting of the ground electronic state of the hydrogen like ${}^{205}$Tl in the point nuclear magnetic dipole moment model  (without QED correction) on $r_c^{2\gamma-1}$ in different basis sets. One can see that this calculated dependence is in a very good agreement with the analytical dependence given by Eq.~(\ref{dep}). Extrapolated value of the hyperfine splitting for the point like nucleus (3.7041 eV) almost coincides with the analytical value obtained within Eq.~(\ref{eq1}) (3.7042 eV) for the GEOM2 basis set.

\begin{figure}[ht]
\begin{center}
\includegraphics[width=1.0\linewidth]{1.eps}
\end{center}
\caption{Calculated dependence of the hyperfine splitting, $\Delta E$, of the ground electronic state of the hydrogen like ${}^{205}$Tl on $r_c^{2\gamma-1}$ in the point  nuclear magnetic dipole moment model in different basis sets (see text). $r_c$ is the nuclear charge radius.}
\label{figure:1}
\end{figure}

Figs. \ref{figure:2} and \ref{figure:3} give calculated dependence of hyperfine splittings of the ground electronic state of the hydrogen like $^{203}$Tl and $^{205}$Tl on magnetic radii. Horizontal lines show the experimental energy splitting with the corresponding uncertainty taken from Ref.~\cite{beiersdorfer2001hyperfine}. From these data it is possible to fix magnetic radii for the used model of the magnetization distribution. For $^{205}$Tl one obtains $r_m / r_c^{exp} = $ 1.109(2) and for $^{203}$Tl $r_m / r_c^{exp} = $ 1.104(2).

Combining theoretical and experimental data, the coefficients $d_{nuc}=1.17$ for ${}^{203}$Tl and $d_{nuc}=1.18$ for ${}^{205}$Tl were obtained for the parametrization given by Eq.~(\ref{d_nuc_dep}).

\begin{figure}[ht]
\begin{center}
\includegraphics[width=1.0\linewidth]{2.eps}
\end{center}
\caption{
Calculated dependence of the hyperfine splitting ($\Delta E$) of the ground electronic state of the hydrogen like ${}^{203}$Tl on the ratio $r_m / r_c^{exp}$ of the magnetic radius $r_m$ and experimental charge radius $r_c^{exp}$. Solid and dashed horizontal lines show the experimental energy splitting value with the corresponding uncertainty from Ref.~\cite{beiersdorfer2001hyperfine}. Vertical lines show the extracted magnetic radius with the corresponding uncertainty.}
\label{figure:2}
\end{figure}

\begin{figure}[ht]
\begin{center}
\includegraphics[width=1.0\linewidth]{3.eps}
\end{center}
\caption{
Calculated dependence of the hyperfine splitting ($\Delta E$) of the ground electronic state of the hydrogen like ${}^{205}$Tl on the ratio $r_m / r_c^{exp}$ of the magnetic radius $r_m$ and experimental charge radius $r_c^{exp}$. Solid and dashed horizontal lines show the experimental energy splitting value with the corresponding uncertainty from Ref.~\cite{beiersdorfer2001hyperfine}. Vertical lines show the extracted magnetic radius with the corresponding uncertainty.}

\label{figure:3}
\end{figure}

\subsection{Neutral thallium
\texorpdfstring{${}^{205}$Tl}{Tl 205} in
\texorpdfstring{$6P_{1/2}$}{6P 1/2} state}

Table \ref{table:2} gives calculated values of the ${}^{205}$Tl hyperfine structure constant for the $6P_{1/2}$ state for a number of magnetic radii. The last column gives values for $r_m/r_c^{exp}=1.11$ which is close to the value obtained from the analysis given above for the hyperfine splitting in the hydrogen like ${}^{205}$Tl.

\begin{table}[ht]
\centering
\begin{tabular}{lrrrr}
\hline
\hline
$r_m/r_c^{exp}$ & 0 & 1 & 1.1 & 1.11 \\
\hline
DHF &~ 18805 &~ 18681 &~ 18660 &~ 18658\\
CCSD & 21965 & 21807 & 21781 & 21778\\
CCSD(T) & 21524 & 21372 & 21347 & 21345\\
\hline
~+Basis corr. & -21 & -21 & -21 & -21 \\
~+CCSDT-CCSD(T) & +73 & +73 & +73 & +73 \\
~+CCSDT(Q)-CCSDT & -5 & -5 & -5 & -5\\
~+Gaunt & -83 & -83 & -83 & -83 \\
\hline
Total & 21488 & 21337 & 21312 & 21309\\
\hline
\hline
\end{tabular}
\caption{Calculated values of the hyperfine structure constant of the $6P_{1/2}$ state of ${}^{205}$Tl (in MHz) at different levels of theory.}
\label{table:2}
\end{table}

The final value for the HFS constant with accounting for the Bohr-Weisskopf correction is 21309 MHz and is in a very good agreement with the experimental value 21310.8 MHz \cite{lurio1956hfs} and previous studies (see Table~\ref{tableCompare}). One can estimate the theoretical uncertainty of the calculated HFS constant to be smaller than 1\%.

\subsection{Neutral thallium
\texorpdfstring{${}^{205}$Tl}{Tl 205} in
\texorpdfstring{$6P_{3/2}$}{6P 3/2} state}

Table \ref{table:5} gives calculated values of the HFS constant for the $6P_{3/2}$ state of the ${}^{205}$Tl atom. One can see that correlation effects dramatically contribute to the constant. This has also been noted in previous studies of this state \cite{konovalova2017calculation, maartensson1995magnetic}. Interestingly that even quadruple cluster amplitudes give non-negligible relative contribution to the HFS constant.

\begin{table}[ht]
\centering
\begin{tabular}{lrrrr}
\hline
\hline
$r_m/r_c^{exp}$ & 0 & 1 & 1.1 & 1.11 \\
\hline
DHF &~ 1415 &~ 1415 &~ 1415 &~ 1415\\
CCSD & 6 & 40 & 46 & 47\\
CCSD(T) & 244 & 273 & 278 & 278\\
\hline
~+Basis corr. & +4 & +4 & +4 & +4\\
~+CCSDT-CCSD(T) & -49 & -49 & -49 & -49 \\
~+CCSDT(Q)-CCSDT & +13 & +13 & +13 & +13 \\
~+Gaunt & +1 & +1 & +1 & +1 \\
\hline
Total & 214 & 243& 248 & 248\\
\hline
\hline
\end{tabular}
\caption{Calculated values of the hyperfine structure constant of the $6P_{3/2}$ state of ${}^{205}$Tl (in MHz) at different levels of theory}
\label{table:5}
\end{table}

Calculated value of the BW correction to the HFS constant of the $6P_{3/2}$ state of ${}^{205}$Tl is $-$16\%. It has an opposite sign with respect to the BW correction to the HFS constant of the $6P_{1/2}$ (see Table \ref{table:6}).

\begin{table}[ht]
\centering
\begin{tabular}{lrrr}
\hline
\hline
$r_m/r_c^{exp}$ & 1 & 1.1 & 1.11 \\
\hline
$6P_{1/2}$  &~ 0.0070 & 0.0082 &~ 0.0083\\
$6P_{3/2}$ & -0.14 & -0.16 & -0.16\\
\hline
\hline
\end{tabular}
\caption{
Calculated values of the Bohr-Weisskopf correction to the hyperfine structure constants of the $6P_{1/2}$ and $6P_{3/2}$ states of ${}^{205}$Tl for different values of the $r_m/r_c^{exp}$ ratio.
}
\label{table:6}
\end{table}

\begin{table}[]
    \centering
\begin{tabular}{lrr}
    \hline
    \hline
    Author, Ref. &~ $6P_{1/2}$ &~ $6P_{3/2}$ \\
    \hline
    Kozlov et al. \cite{kozlov2001parity} & 21663 & 248 \\
    Safronova et al. \cite{safronova2005excitation} & 21390 & 353 \\
    M{\aa}rtensson-Pendrill \cite{maartensson1995magnetic} & 20860 & 256 \\
    This work & 21488 & 214 \\
    \hline
    \hline
\end{tabular}
    \caption{Total values of hyperfine structure constants in the point nuclear magnetic dipole moment model for ${}^{205}$Tl (in MHz) in comparison with previous studies.}
    \label{tableCompare}
\end{table}

Obtained value of the HFS constant is in a reasonable agreement with the experimental value 265 MHz \cite{gould1956hfs}. Theoretical uncertainty of the final value can be estimated as 10\%. Note, however, that it corresponds to about 2\% of the total correlation contribution -- compare the final value with the Dirac-Hartree Fock (DHF) value.

\section{Hyperfine anomaly}

\begin{figure}[ht]
\begin{center}
\includegraphics[width=1.0\linewidth]{4.eps}
\end{center}
\caption{
Calculated dependence of the $6P_{1/2}$ hyperfine anomaly ${}^{205}\Delta^{203}$ on the difference of magnetic radii ${}^{205}r_m-{}^{203}r_m$. Solid and dashed horizontal lines indicate the experimental value and its uncertainty; calculated values are given by circles and vertical lines give the value of the magnetic radii difference and its uncertainty.}
\label{figure:4}
\end{figure}

\begin{figure}[ht]
\begin{center}
\includegraphics[width=1.0\linewidth]{5.eps}
\end{center}
\caption{
Calculated dependence of the $6P_{3/2}$ hyperfine anomaly ${}^{205}\Delta^{203}$ on the difference of magnetic radii ${}^{205}r_m-{}^{203}r_m$. Solid and dashed horizontal lines indicate the experimental value and its uncertainty; calculated values are given by circles and vertical lines give the value of the magnetic radii difference and its uncertainty.}
\label{figure:5}
\end{figure}

\begin{figure}[ht]
\begin{center}
\includegraphics[width=1.0\linewidth]{6.eps}
\end{center}
\caption{
Calculated dependence of the $7S_{1/2}$ hyperfine anomaly ${}^{205}\Delta^{203}$ on the difference of magnetic radii ${}^{205}r_m-{}^{203}r_m$. Solid and dashed horizontal lines indicate the experimental value and its uncertainty; calculated values are given by circles and vertical lines give the value of the magnetic radii difference and its uncertainty.}
\label{figure:6}
\end{figure}

Magnetic moments of $^{203}$Tl and $^{205}$Tl are known with a good accuracy \cite{stone2005table}. Values of HFS constants of the $6P_{1/2}$, $6P_{3/2}$ and $7S_{1/2}$ states have been measured precisely in Refs.~\cite{lurio1956hfs, gould1956hfs, chen2012absolute}. Thus, experimental values of the magnetic anomalies ${}^{205}\Delta^{203}$ for these states are also known with a high precision. Figs. \ref{figure:4}, \ref{figure:5} and \ref{figure:6} show
calculated dependence of the values of anomalies for these states on the difference of magnetic radii ${}^{205}r_m-{}^{203}r_m$. In these calculations charge radii of $^{203}$Tl and $^{205}$Tl were set to experimental values.
Calculations were performed within the CCSD(T) method in the MBas basis set.
In Figs. \ref{figure:4}, \ref{figure:5} and \ref{figure:6} solid and dashed horizontal lines show the experimental value and its uncertainty. By considering the intersection of the calculated (without treatment of the QED effects) dependence (which is approximated by linear functions) with horizontal dashed lines one obtains the difference of magnetic radii ${}^{205}r_m-{}^{203}r_m$ and its uncertainty in the model under consideration.

One can see from Figs. \ref{figure:4} and \ref{figure:5} that the differences extracted from the data for the $6P_{1/2}$ and $6P_{3/2}$ states agree within 10\% which confirms the applicability of the model under consideration. They are also in a good agreement with the difference obtained from the data for the $7S_{1/2}$ state of the neutral Tl as well as from the data for the hydrogen like Tl above --- within the experimental uncertainty for these systems.

\section{Magnetic moments of short lived isotopes}

Magnetic anomalies can be used to determine the value of the nuclear magnetic moment of the short lived isotope (for example see Refs.~\cite{0954-3899-37-11-113101,schmidt2018nuclear,barzakh2012hyperfine}).
For this one should know the nuclear magnetic moment value of the stable isotope as well as HFS constants ($A[a]$ and $A[b]$) for two electronic states ($a$ and $b$) of this isotope and the short lived isotope. Consider isotopes 1 (${}^{205}$Tl, $I=1/2$) and 2 (${}^{193}$Tl, $I=9/2$). The latter is unstable.
From the experimental data \cite{barzakh2012hyperfine, lurio1956hfs, chen2012absolute}  one obtains:
\begin{equation}
    {}^1\theta^2[a,b]=
    \frac{A_1[a]}{A_2[b]}\frac{A_2[a]}{A_1[b]}-1=
    -0.013(7).
\end{equation}
Here $a=7S_{1/2}$, $b=6P_{1/2}$. Calculated value of the ratio of magnetic anomalies for these states is
${}^1k^2[a,b]={}^{1}\Delta^{2}[a]/{}^{1}\Delta^{2}[b]=3.4(2)$. Such ratio depends only slightly on the nuclear magnetization distribution model \cite{0954-3899-37-11-113101,schmidt2018nuclear}. Now one obtains for the nuclear magnetic moment $\mu_2$ of the isotope 2:
\begin{equation}
    \mu_2=\mu_1\cdot\frac{A_2[b]}{A_1[b]}\cdot\frac{I_2}{I_1}
\cdot(1+{}^{1}\Delta^{2}[b])=3.84(3).
\end{equation}

Using the same method and experimental data from Refs.~\cite{barzakh2012hyperfine, lurio1956hfs, chen2012absolute} one obtains the nuclear magnetic moment value of the ${}^{191}$Tl isotope with nuclear spin $I=9/2$: $\mu_{191}=+3.79(2)$.
Obtained values of $\mu_{191}$ and $\mu_{193}$ are in a very good agreement with    Ref.~\cite{barzakh2012hyperfine}: $\mu_{193}=+3.82(3)$ and $\mu_{191}=+3.78(2)$.

\section{Conclusion}

In the present paper the Bohr-Weisskopf effect has been calculated for the $6P_{1/2}$, $6P_{3/2}$ and $7S_{1/2}$ states for several isotopes of the Tl atom. The uniformly magnetized ball model has been tested and used.

It was found that the correlation effects strongly contribute to the HFS constant (they are about 470\% of the final value) as well as to the BW effect for the $6P_{3/2}$ state. BW correction for the $6P_{3/2}$ state was found to be about $-$16\%. Such a significant contribution makes it possible to test models of the nuclear magnetization distribution.

Combining obtained theoretical values of magnetic anomalies and available experimental data nuclear magnetic moments of short-lived ${}^{191}$Tl and ${}^{193}$Tl isotopes were predicted and found to be in a good agreement with the previous study~\cite{barzakh2012hyperfine}.

It was demonstrated that for such calculations the Gaussian basis sets can be used. Thus, the applied method can be extended to the calculation of the BW effect in molecules.

For the calculated value of the HFS constant of the $6P_{1/2}$ state a very good agreement with the experiment and the small theoretical uncertainty has been obtained. A further improvement can be achieved by the treatment of the QED effects contribution.

\section{Acknowledgment}

We are grateful to Prof. M. Kozlov, Prof. I. Mitropolsky and Dr. Yu. Demidov for helpful discussions. Electronic structure calculations were performed at the PIK data center of NRC ``Kurchatov Institute'' -- PNPI.


\section{Introduction}
\label{sec:intro}

Keyword detection is like searching for a needle in a haystack: the detector must listen to continuously streaming audio, ignoring nearly all of it, yet still triggering correctly and instantly. In the last few years, with the advent of voice assistants, keyword spotting has become a common way to initiate a conversation with them (e.g. ``Ok Google", ``Alexa", or ``Hey Siri"). As the assistant use cases spread through a variety of devices, from mobile phones to home appliances and further into the internet-of-things (IoT) --many of them battery powered or with restricted computational capacity, it is important for the keyword spotting system to be both high-quality as well as computationally efficient.

Neural networks are core to the state-of-the-art keyword spotting systems. These solutions, however, are not developed as a single deep neural network (DNN). Instead, they are traditionally comprised of different subsystems, independently trained, and/or manually designed. For example, a typical system is composed by three main components: 1) a signal processing frontend, 2) an acoustic encoder, and 3) a separate decoder. Of those components, it is the last two that make use of DNNs along with a wide variety of decoding implementations. They range from traditional approaches that make use of a Hidden Markov Model (HMM) to characterize acoustic features from a DNN into both ``keyword" and ``background" (i.e. non-keyword speech and noise) classes \cite{Alexa16, Alexa17, AlexaRaw17, AlexaDelayed18, Alexa18}. Simpler derivatives of that approach perform a temporal integration computation that verifies the outputs of the acoustic model are high in the right sequence for the target keyword in order to produce a single detection likelihood score~\cite{Hotwordv1, Agc15, Cascade17, HeySiri17, AlexaCompress16}. Other recent systems make use of CTC-trained DNNs --typically recurrent neural networks (RNNs) \cite{Ctc17}, or even sequence-to-sequence trained models that rely on beam search decoding \cite{Custom17}. This last family of systems is the closest to be considered end-to-end, however they are generally too computationally complex for many embedded applications.

Optimizing independent components, however, creates added complexities and is suboptimal in quality compared to doing it jointly. Deployment also suffers due to the extra complexity, making it harder to optimize resources (e.g. processing power and memory consumption). The system described in this paper addresses those concerns by learning both the encoder and decoder components into a single deep neural network, jointly optimizing to directly produce the detection likelihood score. This system could be trained to subsume the signal processing frontend as well as in \cite{AlexaRaw17, Raw15}, but it is typically computationally costly to replace highly optimized fast Fourier transform implementations with a neural network of equivalent quality. However, it is something we consider exploring in the future. Overall, we find this system provides state-of-the-art quality across a number of audio and speech conditions compared to a traditional, non end-to-end baseline system described in \cite{Cnn15}. Moreover, the proposed system significantly reduces the resource requirements for deployment by cutting computation and size over five times compared to the baseline system.

The rest of the paper is organized as follows. In Section \ref{sec:system} we present the architecture of the keyword spotting system; in particular the two main contributions of this work: the neural network topology, and the end-to-end training methodology. Next, in Section \ref{sec:setup} we describe the experimental setup, and the results of our evaluations in Section \ref{sec:results}, where we compare against the baseline approach of \cite{Cnn15}. Finally, we conclude with a discussion of our findings in Section \ref{sec:conclusion}.

\section{End-to-End system}
\label{sec:system}

This paper proposes a new end-to-end keyword spotting system that by subsuming both the encoding and decoding components into a single neural network can be trained to produce directly an estimation (i.e. score) of the presence of a keyword in streaming audio. The following two sections cover the efficient memoized neural network topology being utilized, as well as the method to train the end-to-end neural network to directly produce the keyword spotting score.

\subsection{Efficient memoized neural network topology}
\label{svdf}

We make use of a type of neural network layer topology called SVDF (single value decomposition filter), originally introduced in~\cite{Svdf15} to approximate a fully-connected layer with a low rank approximation. As proposed in~\cite{Svdf15} and depicted in Equation \ref{eq:basic-eqn-rank-1}, the activation $a$ for each node $m$ in the rank-1 SVDF layer at a given inference step $t$ can be interpreted as performing a mix of selectivity in time ($\mathbf{\alpha}^{(m)}$) with selectivity in the feature space ($\mathbf{\beta}^{(m)}$) over a sequence of input vectors $\mathbf{x}_t = [ \mathbf{X}_{t - T}, \cdots , \mathbf{X}_{t}]$ of size $F$.

\begin{equation}
a^{(m)}_t
=
f\left( \sum_{i=0}^{T - 1} \mathbf{\alpha}^{(m)}_{i} \sum_{j=1}^F \mathbf{\beta}^{(m)}_{j} x_{(t - T + i), j} \right)
\label{eq:basic-eqn-rank-1}
\end{equation}

This is equivalent to performing, on an SVDF layer of $N$ nodes, $N\times T$ 1-D convolutions of the feature filters $\mathbf{\beta}$ (by ``sliding" each of the $N$ filters on the input feature frames, with a stride of $F$), and then filtering each of the $N$ output vectors (of size $T$) with the time filters $\mathbf{\alpha}$.

\begin{figure}
	\centering
	\includegraphics[height=180pt]{SVDF.pdf}
	\caption{A single node (\emph{m}) in the SVDF layer.}
	\label{fig:svdf_node}
\end{figure}

A more general and efficient interpretation, depicted in Figure \ref{fig:svdf_node}, is that the layer is just processing a single input vector $\mathbf{X}_{t}$ at a time. Thus for each node $m$, the input $\mathbf{X}_{t}$ goes through the feature filter $\mathbf{\beta}^{(m)}$, and the resulting scalar output gets concatenated to those $T-1$ computed in previous inference steps. The memory is initialized to zero during training for the first $T$ inferences. Finally the time filter $\mathbf{\alpha}^{(m)}$ is applied to them. This is how stateful networks work, where the layer is able to memorize the past within its state. Different from other stateful approaches \cite{Fsmn15}, and typical recurrent layers, the SVDF does not recur the outputs into the state (memory), nor rewrites the entirety of the state with each inference. Instead, the memory keeps each inference's state isolated from subsequent runs, just pushing new entries and popping old ones based on the memory size $T$ configured for the layer. This also means that by stacking SVDF layers we are extending the receptive field of the network. For example, a DNN with $D$ stacked layers, each with a memory of $T$, means that the DNN is taking into account inputs as old as $\mathbf{X}_{t-D\times (T-1)}$. This approach works very well for streaming execution, like in speech, text, and other sequential processing, where we constantly process new inputs from a large, possibly infinite sequence but do not want to attend to all of it. An implementation is available at \cite{TfliteSvdf}.

This layer topology offers a number of benefits over other approaches. Compared with the convolutions used in \cite{Cnn15}, it allows finer-grained control of the number of parameters and computation, given that the SVDF is composed by several relatively small filters. This is useful when selecting a tradeoff between quality, size and computation. Moreover, because of this characteristic, the SVDF allows creating very small networks that outperform other topologies which operate at larger granularity (e.g. our first stage, always-on network has about 13K parameters~\cite{Cascade17}). The SVDF also pairs very well with linear “bottleneck” layers to significantly reduce the parameter count as in \cite{Bottleneck08, Bottleneck12}, and more recently in \cite{AlexaCompress16}. And because it allows for creating evenly sized deep networks, we can insert them throughout the network as in Figure \ref{fig:topology}. Another benefit is that due to the explicit sizing of the receptive field, it allows for a fine grained control over how much to remember from the past. This has resulted in SVDF outperforming RNN-LSTMs, which do not benefit from, and are potentially hurt by, paying attention to theoretically infinite past. It also avoids having complex logic to reset the state every few seconds as in \cite{Custom17}.

\subsection{Method to train the end-to-end neural network}
\label{training}
The goal of our end-to-end training is to optimize the network to produce the likelihood score, and to do so as precisely as possible. This means to have a high score right at the place where the last bit of the keyword is present in the streaming audio, and not before and particularly not much after (i.e. a ``spiky" behaviour is desirable). This is important since the system is bound to an operating point defined by a threshold (between $0$ and $1$) that is chosen to strike a balance between false-accepts and false-rejects, and a smooth likelihood curve would add variability to the firing point. Moreover, any time in between the true end of the keyword and the point where the score meets the threshold will become latency in the system (e.g. the ``assistant" will be slow to respond) --a common drawback of CTC-trained RNNs \cite{Ctc15} we aim to avoid.

\subsubsection{Label generation}
\label{label_gen}
We generate input sequences composed of pairs \texttt{<$\mathbf{X}_{t}$,$c$>}. Where $\mathbf{X}$ is a 1D tensor corresponding to log-mel filter-bank energies produced by a front-end as in \cite{Hotwordv1, Svdf15, Cnn15}, and $c$ is the class label (one of $\{0, 1\}$). Each tensor $\mathbf{X}$ is first force-aligned from annotated audio utterances, using a large LVCSR system, to break up the components of the keyword \cite{Speech12}. For example, ``ok google" is broken into: ``ou", ``k", ``eI", ``\texttt{<silence>}", ``g", ``u", ``g", ``@", ``l". Then we assign labels of $1$ to all sequence entries, part of a true keyword utterance, that correspond to the last component of the keyword (``l" in our ``Ok google" example). All other entries are assigned a label of $0$, including those that are part of the keyword but that are not its last component. See Figure \ref{fig:labeling}. Additionally, we tweak the label generation by adding a fixed amount of entries with a label of $1$, starting from the first vector $\mathbf{X}_{t}$ corresponding to the final keyword component (e.g. ``l"). This is with the intention of balancing the amount of negative and positive examples, in the same spirit as \cite{Alexa16}. This proved important to make training stable, as otherwise the amount of negative examples overpowered the positive ones.

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{labeling.pdf}
	\caption{Input sequence generation for ``Ok google". }
	\label{fig:labeling}
\end{figure}

\subsubsection{Training recipe}
\label{recipe}
The end-to-end training uses a simple frame-level cross-entropy (CE) loss that for the feature vector $\mathbf{X}_{t}$ is defined by $\lambda_{t}(\mathbf{W}) = - \log y_{c_t}(\mathbf{X}_{t}, \mathbf{W})$, where $\mathbf{W}$ are the parameters of the network, $y_{i}(\mathbf{X}_{t}, \mathbf{W})$ the $i$th output of the final softmax. Our training recipe uses asynchronous stochastic gradient descent (ASGD) to produce a single neural network that can be fed streaming input features and produce a detection score. We propose two variants of this recipe:

\textbf{Encoder+decoder}. A two stage training procedure where we first train an acoustic encoder, as in \cite{Hotwordv1,Svdf15,Cnn15}, and then a decoder from the outputs of the encoder (rather than filterbank energies) and the labels from \ref{label_gen}. We do this in a single DNN by creating a final topology that is composed of the encoder and its pre-trained parameters (including the softmax), followed by the decoder. See Figure \ref{fig:topology}. During the second stage of training the encoder's parameters are frozen, such that only the decoder is trained. This recipe is useful on models that tend to overfit to subsets of the entire training dataset.

\textbf{End-to-end}. In this option, we train the DNN end-to-end directly, with the sequences from \ref{label_gen}. The DNN may be any topology, but we use that of the encoder+decoder, except for the intermediate encoder softmax (now an unnecessary information bottleneck). See Figure \ref{fig:topology}. Similar to the encoder+decoder recipe, we can also initialize the encoder section with a pre-trained model, and use an adaptation rate $[0-1]$ to tune how much the encoder section is being adjusted (e.g. a rate of 0 is equivalent to the encoder+decoder recipe). This end-to-end pipeline, where the entirety of the topology's parameters are adjusted, tends to outperform the encoder+decoder one, particularly in smaller sized models which do not tend to overfit.

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{topology.pdf}
	\caption{End-to-end topology trained to predict the keyword likelihood score. Bottleneck layers reduce parameters and computation. The intermediate softmax is used in encoder+decoder training only.}
	\label{fig:topology}
\end{figure}

\section{Experimental setup}
\label{sec:setup}

In order to determine the effectiveness of our approach, we compare against a known keyword spotting system proposed in \cite{Cnn15}. This section describes the setups used in the results section.

\subsection{Front-end}
\label{frontend}
Both setups use the same front-end, which generates 40-dimensional log-mel filter-bank energies out of 30ms windows of streaming audio, with overlaps of 10ms. The front-end can be queried to produce a sequence of contiguous frames centered around the \emph{current} frame $\mathbf{x}_t = [ \mathbf{X}_{t - C_l}, \cdots , \mathbf{X}_{t}, \cdots, \mathbf{X}_{t + C_r}] $. Older frames are said to form the left context $C_l$, and newer frames form the right context $C_r$. Additionally, the sequences can be requested with a given stride $\sigma$.

\subsection{Baseline model setup}
\label{baseline}
Our baseline system (Baseline\_1850K) is taken from \cite{Cnn15}. It consists of a DNN trained to predict subword targets within the keywords. The input to the DNN consists of a sequence with $C_l=30$ frames of left and $C_r=10$ frames of right context; each with a stride of $\sigma=3$. The topology consists of a 1-D convolutional layer with 92 filters (of shape 8x8 and stride 8x8), followed by 3 fully-connected layers with 512 nodes and a rectified linear unit activation each. A final softmax output predicts 9 subword targets ("k" and "h" share a label for "Ok/Hey Google" detection), obtained from the same forced alignment process described in \ref{label_gen}. This results in the baseline DNN containing 1.7M parameters, and performing 1.8M multiply-accumulate operations per inference (every 30ms of streaming audio). A keyword spotting score between 0 and 1 is computed by first smoothing the posterior values, averaging them over a sliding window of the previous 100 frames with respect to the current $t$; the score is then defined as the largest product of the smoothed posteriors in the sliding window as originally proposed in \cite{Agc15}.

\subsection{End-to-end model setup}
\label{end2end}
The end-to-end system (prefix E2E) uses the DNN topology depicted in Figure \ref{fig:topology}, and all SVDF layers are of $rank=1$. We present results with 3 distinct size configurations (infixes 700K, 318K, and 40K) each representing the approximate number of parameters, and the 2 training recipe variants (suffixes 1stage and 2stage) corresponding to end-to-end and encoder+decoder respectively, as described in \ref{recipe}. The input to all DNNs consists of a sequence with $C_l=1$ frame of left and $C_r=1$ frame of right context; each with a stride of $\sigma=2$. More specifically, the E2E\_700K model uses $N=1280$ nodes in the first 4 SVDF layers, each with a memory $T=8$, with intermediate bottleneck layers each of size $64$; the following 3 SVDF layers have $N=32$ nodes, each with a memory $T=32$. This model performs 350K multiply-accumulate operations per inference (every 20ms of streaming audio). The E2E\_318K model uses $N=576$ nodes in the first 4 SVDF layers, each with a memory $T=8$, with intermediate bottleneck layers each of size $64$; the remainder layers are the same as E2E\_700K. This model performs 159K multiply-accumulate operations per inference. Finally, the E2E\_40K model uses $N=96$ nodes in the first 4 SVDF layers, each with a memory $T=8$, with intermediate bottleneck layers each of size $32$; the remainder layers are the same as the other two models. This model performs 20K multiply-accumulate operations per inference.

\subsection{Dataset}
\label{ssec:data}
The training data for all experiments consists of 1 million anonymized hand-transcribed utterances of the keywords ``Ok Google" and ``Hey Google", with an even distribution. To improve robustness, we create ``multi-style" training data by synthetically distorting the utterances, simulating the effect of background noise and reverberation. 8 distorted utterances are created for each original utterance; noise samples used in this process are extracted from environmental recordings of everyday events, music, and YouTube videos. Results are reported on four sets representative of various environmental conditions: \emph{Clean non-accented} contains 170K non-accented English utterances of the keywords in ``clean" conditions, plus 64K samples without the keywords (1K hours); \emph{Clean accented} has 153K English utterances of the keywords with Australian, British, and Indian accents (also in ``clean" conditions), plus 64K samples without the keywords (1K hours); \emph{High pitched} has 1K high pitched utterances of the keywords, and 64K samples without them (1K hours); \emph{Query logs} contains 110K keyword and 21K non-keyword utterances, collected from anonymized voice search queries. This last set contains background noises from real usage conditions.

\section{Results}
\label{sec:results}

Our goal is to compare the effectiveness of the proposed approach against the baseline system described in \cite{Cnn15}. Inference is floating point, though (unreported) TensorFlow Lite's quantization~\cite{TfliteSvdfHybridQuant} numbers showed no meaningful degradation. We evaluate the false-reject (FR) and false-accept (FA) tradeoff across several end-to-end models of distinct sizes and computational complexities. As can be seen in the Receiver Operating Characteristic (ROC) curves in Figure \ref{fig:roc}, the 2 largest end-to-end models, with 2-stage training, significantly outperform the recognition quality of the much larger and complex Baseline\_1850K system. More specifically, E2E\_318K\_2stage and E2E\_700K\_2stage show up to 60\% relative FR rate reduction over Baseline\_1850K in most test conditions. Moreover, E2E\_318K\_2stage uses only about 26\% of the computations that Baseline\_1850K uses (once normalizing their execution rates over time), but still shows significant improvements. We also explore end-to-end models at a size that, as described in \cite{Cascade17}, is small enough (in both size and computation) to be executed continuously with very little power consumption. These 2 models, E2E\_40K\_1stage and E2E\_40K\_2stage, also explore the capacity of end-to-end training (\emph{1stage}) versus encoder+decoder training (\emph{2stage}). The ROC curves show that \emph{1stage} training outperforms \emph{2stage} training on all conditions, but particularly on both ``clean" environments where it gets fairly close to the performance of the baseline setup. That is a significant achievement considering E2E\_40K\_1stage has 2.3\% the parameters and performs 3.2\% the computations of Baseline\_1850K. Table \ref{tab:perfdiff} compares the recognition quality of all setups by fixing on a very low false-accept rate of 0.1 FA per hour on a dataset containing only negative (i.e. non-keyword) utterances. Thus the table shows the false-reject rates at that operating point. Here we can appreciate similar trends as those described above: the 2 largest end-to-end models outperform the baseline across all datasets, reducing FR rate about 40\% on the clean conditions, and 40\%-20\% on the other 2 sets depending on the model size. This table also shows how \emph{1stage} outperforms \emph{2stage} for small sized models, and presents similar FR rates as Baseline\_1850K on clean conditions.

\begin{table}[h!]
	\begin{center}
		\caption{FR rate over 4 test conditions at 0.1 FAh level.}
		\label{tab:perfdiff}
		\begin{tabular}{l|r|r|r|r}
			\textbf{Models} & Non Acc. & Accented & High P. & Q. Logs \\
			\hline
			Baseline\_1850K & 1.46\% & 2.03\% & 14.41\% & 12.13\%  \\
			E2E\_318K  & 0.87\% & 1.27\% & 11.39\% &  9.99\%  \\
			E2E\_700K  & 0.87\% & 1.22\% &  8.57\% &  8.90\% \\
			E2E\_40K\_2 & 2.80\% & 5.19\% & 26.54\% & 39.22\%  \\
			E2E\_40K\_1 & 1.52\% & 2.09\% & 23.73\% & 35.32\%  \\
		\end{tabular}
	\end{center}
\end{table}

\begin{figure}[htb]
	\begin{minipage}[b]{\linewidth}
		\centering
		\centerline{\includegraphics[height=137pt]{roc_cleanUS.pdf}}

		\centerline{(a) Clean non-accented}\medskip
	\end{minipage}
	\hfill
	\begin{minipage}[b]{\linewidth}
		\centering
		\centerline{\includegraphics[height=137pt]{roc_cleanAccented.pdf}}

		\centerline{(b) Clean accented}\medskip
	\end{minipage}
	%
	\begin{minipage}[b]{\linewidth}
		\centering
		\centerline{\includegraphics[height=137pt]{roc_highpitched.pdf}}

		\centerline{(c) High pitched voices}\medskip
	\end{minipage}
	\hfill
	\begin{minipage}[b]{\linewidth}
		\centering
		\centerline{\includegraphics[height=137pt]{roc_logs.pdf}}

		\centerline{(d) Anonymous query logs}\medskip
	\end{minipage}
	%
	\caption{ROC curves under different conditions.}
	\label{fig:roc}
	%
\end{figure}

\section{Conclusion}
\label{sec:conclusion}
We presented a system for keyword spotting that by combining an efficient topology and two types of end-to-end training can significantly outperform previous approaches, at a much lower cost of size and computation. We specifically show how it beats the performance of a setup taken from \cite{Cnn15} with models over 5 times smaller, and even get close to the same performance with a model over 40 times smaller. Our approach provides further benefits of not requiring anything other than a front-end and a neural network to perform the detection, and thus it is easier to extend to newer keywords and/or fine-tune with new training data. Future work includes exploring other loss-functions, as well as generalizing for multi-channel support.

\clearpage
\bibliographystyle{IEEEbib}