text:
# On a quantum version of Shannon’s conditional entropy

## 1 Introduction

The concept of entropy plays a major role in thermodynamics and statistical mechanics. It serves to describe the behavior of macroscopic systems. The name “entropy” was introduced by Clausius (1865) and derives from $`ϵ\nu \tau \rho o\pi \iota \eta `$ “transformation”. It was von Neumann (1927 ), who generalized the classical expression of Boltzmann and Gibbs for the entropy to quantum mechanics by using the concept of what is now called a density matrix, also introduced quite generally by him in the same year . In the special context of radiation damping the density matrix was discovered independently by L. Landau  and by F. Bloch , again in the same year (see also the citation in ). For a technical overview of the developments up to 1978 and with further historical references see . For recent expositions see . In the theory of dynamical systems entropy and the derived notion of topological entropy also plays an important role, see e.g. the contributions in .

In a seminal article Shannon (1948, ) introduced the concept of entropy into information theory. Roughly speaking a gain in information means a decrease in entropy. Shannon also provided the concept of conditional entropy. It is a measure how entropy is reduced given a preexisting knowledge. To the author’s best knowledge the first construction in quantum mechanics coming close to such a notion is due to E. Lieb  (see also ). It involves tensor product structures and it was called a relative entropy in  (but a conditional entropy in , p. 259). In view of recent developments in quantum computation and quantum coding (see  for a concise account) it is highly desirable to have such a quantity at ones disposal. There is a construction of a non-commutative analogue of Shannon’s conditional entropy by Connes and Størmer  and Connes, Narnhofer and Thirring (for an exposition and a discussion of further developments see e.g. ). More recently attempts have been made to construct a mutual information analogous to Shannon’s conditional entropy in the context of quantum error-correction. In two of these attempts , made independently, yielded the same quantity. The first article exhibits necessary and sufficient conditions for quantum error-correction to be possible in terms of the mutual information like the quantity given there, and a conjecture is made on its connection with quantum channel capacity, explored in more detail . The connection with channel capacity was also analyzed in . In  its connection with entanglement is discussed. In yet another approach  the starting point is one density matrix on a tensor product. The conditioning is then obtained by looking at the two density matrices in the two sub-systems resulting by taking the corresponding partial traces.

In this article we will propose a different candidate for a quantum mechanical conditional entropy $`S(\rho |\sigma )0`$, a function of two density matrices $`\rho `$ and $`\sigma `$ in a same Hilbert space and having the interpretation of the entropy of $`\rho `$ conditioned by the “knowledge” given by $`\sigma `$. For simplicity we will only discuss the finite dimensional case although an extension to the infinite dimensional case seems possible. If we view $`\rho `$ as the analogue of $`X`$ and $`\sigma `$ the analogue of $`Y`$ such that von Neumann’s entropy $`S(\rho )`$ is the analogue of Shannon’s entropy $`H(X)`$, then this conditional entropy shares several but not all properties of Shannon’s conditional entropy $`H(X|Y)`$ (see section 3 for a brief recapitulation of Shannon’s theory). In particular the “knowledge” of $`\sigma `$ reduces the entropy, i.e. the inequality $`S(\rho |\sigma )S(\rho )`$ holds. This corresponds exactly to Shannon’s famous inequality $`H(X|Y)H(X)`$ and was our main motivation for our construction. Also and again in analogy to the classical theory we wanted the conditioning to be given by a quantity on the same footing as the original density matrix, i.e. conditioning should also be given by a density matrix. If as in the classical case $`\sigma `$ contains no information, i.e. if it is a multiple of the identity such that $`S(\sigma )`$ is maximal, then $`S(\rho |\sigma )=S(\rho )`$. In contrast to the classical case $`H(X|X)=0`$, however, the relation $`S(\rho |\rho )=0`$ holds if and only if the non-zero eigenvalues of $`\rho `$ are non-degenerate. In particular $`S(\rho |\rho )=0`$ if $`\rho `$ is pure. We will not elaborate on the question, whether the failure of our $`S(\rho |\sigma )`$ to satisfy all corresponding classical properties, like this last property, is due to a fundamental difference of quantum and classical information theory. In particular we will not provide a more detailed quantum mechanical interpretation of $`S(\rho |\sigma )`$. Also so far we have not analyzed whether it may be used in the context of channel capacity. Rather we will argue that other quantum mechanical versions of conditional entropy, which share more properties with the classical counterpart $`H(X|Y)`$, do not exist.

The article is organized as follows. In section 2 we provide the construction of a quantum version $`S(\rho |\sigma )`$ of the conditional entropy and establish several properties. In section 3 and after a brief review of Shannon’s theory we compare this with Shannon’s conditional entropy. In section 4 we first present a list of desirable properties for a quantum version of conditional entropy given in terms of two density matrices. We then show that even parts of these desiderata can not be fulfilled simultaneously. In particular there is no version involving two density matrices and which reduces to the classical case, when these two density matrices commute. We will provide an alternative in terms of resolutions of the unit matrix in terms of orthogonal projections and which share more properties with the classical case. Briefly we will compare this ansatz with the algebraic constructions given by Connes and Størmer and Connes, Narnhofer and Thirring.

## 2 Construction of a quantum conditional entropy

Let $`\rho `$ be a density matrix on a finite dimensional Hilbert space $``$, i.e $`\rho 0`$ and $`\mathrm{Tr}\rho =1`$, where $`\mathrm{Tr}`$ denotes the canonical trace on $``$. We write $`\rho =_i\rho _iP_i`$ for the spectral representation of $`\rho `$ where the projections $`P_i0`$ are pairwise orthogonal ( i.e. $`P_iP_j=\delta _{ij}P_i,P_i=P_i^{}`$), such that $`\rho _i0`$, $`\rho _i\rho _j`$ for $`ij`$ and $`_iP_i=𝕀`$, where $`𝕀`$ is the identity operator on $``$. Thus $`\mathrm{Tr}\rho =_idimP_i\rho _i=1`$ with $`dimP=\mathrm{Tr}P=dimP`$ for any projection $`P`$. Here and in what follows projection operators are always understood to be orthogonal. With this notational convention the $`P_i`$ are canonically defined in terms of $`\rho `$. Since this fact will be crucial in what follows, let us briefly recall a standard proof. The eigenvalues $`\rho _i`$ (and their degeneracies $`(=dimP_i)`$) are of course uniquely determined by $`\rho `$ as solutions in $`\lambda `$ of the secular equation $`det(\lambda 𝕀\rho )=0`$, a basis independent relation, such that $`det(\lambda 𝕀\rho )=_i(\lambda \rho _i)^{dimP_i}`$. Order the $`\rho _i`$ in such a way that $`1\rho _1>\rho _2>\rho _3>\mathrm{}`$ . Then $`P_1=lim_n\mathrm{}(\rho /\rho _1)^n`$, $`P_2=lim_n\mathrm{}((\rho \rho _1P_1)/\rho _2)^n`$, etc.

The quantum mechanical entropy of $`\rho `$ is given as $`S(\rho )=_idimP_i\rho _i\mathrm{ln}\rho _i`$, which is continuous and concave in $`\rho `$ (for an account of sub-additivity and convexity properties of the entropy and related quantities see e.g. ). Let $`\sigma `$ be another density matrix on the same space $``$ with the spectral representation $`\sigma =_j\sigma _jQ_j`$ again written in a canonical way. We define the conditional entropy by

$`S(\rho |\sigma )`$ $`=`$ $`{\displaystyle \underset{j}{}}dimQ_j\sigma _jF(\rho ,Q_j)`$ (1)
$`=`$ $`{\displaystyle \underset{j}{}}\mathrm{Tr}Q_j\sigma F(\rho ,Q_j)`$
where

$$F(\rho ,Q)=\mathrm{Tr}(Q\rho Q\mathrm{ln}(Q\rho Q))+\mathrm{Tr}(Q\rho Q)\mathrm{ln}\mathrm{Tr}(Q\rho Q)$$
(2)
for any orthogonal projection $`Q`$. Since the $`Q_j`$’s and $`\sigma _j`$’s are well defined in terms of $`\sigma `$ and since trivially $`Q\rho Q0`$, $`S(\rho |\sigma )`$ is well defined. Also as usual in this context $`A\mathrm{ln}A`$ for any non-negative operator $`A`$ is defined in terms of the spectral representation of $`A`$ with the natural convention that $`x\mathrm{ln}x|_{x=0}=0`$. If $`Q\rho Q0`$ then also $`0\mathrm{Tr}Q\rho Q=\mathrm{Tr}Q\rho `$ and then we may write

$$F(\rho ,Q)=\mathrm{Tr}Q\rho S(\rho _Q)$$
(3)
with

$$\rho _Q=\frac{1}{\mathrm{Tr}(Q\rho )}Q\rho Q$$
(4)
being a density matrix. Actually we might use (3) instead of (2) as a definition for $`F(\rho ,Q)`$ with the convention, usually made in similar contexts (see e.g. ), that 0 times something undefined is 0. Relation (3) shows that $`F(\rho ,Q)0`$ for all $`\rho `$ and $`Q`$. Using (3) we may rewrite $`S(\rho |\sigma )`$ as

$$S(\rho |\sigma )=\underset{j:Q_j\rho Q_j0}{}\mathrm{Tr}Q_j\sigma \mathrm{Tr}Q_j\rho S(\rho _{Q_j}).$$
(5)
There is yet another way of writing $`F(\rho ,Q)`$. It uses the relative entropy $`0S_{rel}(A,B)=\mathrm{Tr}A(\mathrm{ln}A\mathrm{ln}B)\mathrm{}`$, which is defined for any $`A0`$ and $`B0`$. The relative entropy is lower semi-continuous in $`A`$ and jointly convex in $`A`$ and $`B`$, see e.g. . Obviously $`S_{rel}(\lambda A,\lambda B)=\lambda S_{rel}(A,B)`$ holds for any $`\lambda >0`$ and we have

$$F(\rho ,Q)=S_{rel}(Q\rho Q,\mathrm{Tr}(Q\rho Q)𝕀)$$
(6)
such that

$$S(\rho |\sigma )=\underset{j}{}\mathrm{Tr}Q_j\sigma S_{rel}(Q_j\rho Q_j,\mathrm{Tr}(Q_j\rho Q_j)𝕀)$$
(7)
It is instructive to compare $`S(\rho |\sigma )`$ with $`S(E_{\underset{¯}{Q}}(\rho ))`$ and which actually motivated our construction of $`S(\rho |\sigma )`$. $`E_{\underset{¯}{Q}}`$ is the linear map on the set of linear operators $`A`$ on $``$ given as $`E_{\underset{¯}{Q}}(A)=_jQ_jAQ_j`$ . The $`Q_j`$’s are as above, i.e a any set $`\underset{¯}{Q}=\{Q_j\}`$ of pairwise orthogonal nonzero projection operators with $`_jQ_j=𝕀`$ and which is called a resolution of the identity. $`E_{\underset{¯}{Q}}`$ is a conditional expectation (see e.g. ) with range being the $``$-algebra consisting of all linear operators which commute with all $`Q_i`$. In particular $`E_{\underset{¯}{Q}}`$ maps density matrices into density matrices. More precisely, let $`=()`$ be the $``$-algebra of all linear operators on $``$, which is (isomorhic to) a full matrix-algebra. Then $`E_{\underset{¯}{Q}}()`$ is a $``$-sub-algebra of $``$ and the direct sum of the $``$-sub-algebras $`Q_jQ_j=(Q_j)`$, which are (isomorphic to) full matrix algebras. Although any finite dimensional $``$-algebra is (isomorphic to) a direct sum of full matrix algebras, not all $``$-sub-algebras of $``$ are of the form $`E_{\underset{¯}{Q}}()`$ for a suitable $`\underset{¯}{Q}`$. As an example consider the algebra generated by $`𝕀`$ alone. It can easily be shown that any $``$-sub-algebra is of this form if and only if it contains a maximal abelian sub-algebra. Also from $`E_{\underset{¯}{Q}}()\underset{¯}{Q}`$ may be recovered. Indeed the $`Q_j`$’s are just the minimal self-adjoint idempotents (i.e. the orthogonal projections) in $`E_{\underset{¯}{Q}}()`$ and which are central. Also on the set of all spectral resolutions of the identity we introduce a partial ordering $``$ by setting $`\underset{¯}{P}\underset{¯}{Q}`$ if to each $`i`$ there is $`j(i)`$ (which is unique) such that $`P_iQ_{j(i)}`$. Note that each $`j`$ is of the form $`j=j(i)`$ for at least one $`i`$. Then in particular all $`P_i`$ commute with all $`Q_j`$. Also $`\underset{¯}{P}\{𝕀\}`$ holds for all $`\underset{¯}{P}`$. It is easy to see that $`\underset{¯}{P}\underset{¯}{Q}`$ if and only if $`E_{\underset{¯}{P}}()E_{\underset{¯}{Q}}()`$. With respect to these orderings $`\underset{¯}{P}`$ or equivalently $`E_{\underset{¯}{P}}()`$ is minimal if and only if each $`P_i`$ is one-dimensional. $`E_{\underset{¯}{P}}()`$ is then commutative with dimension equal to $`dim`$. To sum up, with respect to the partial ordering $``$ there is a unique maximal element but there are many minimal elements in the set of spectral resolutions $`\underset{¯}{P}`$.

Now one has the well known result $`S(E_{\underset{¯}{Q}}(\rho ))S(\rho )`$ (see e.g.  for a direct proof and  for the special case when $`dimQ_j=1`$ for all $`j`$. It is a special case of Uhlmann’s monotonicity theorem , see also ). It means that projective measurements increase entropy and compares with the inequality $`S(\rho |\sigma )S(\rho )`$ to be proven below. Its interpretation is that of a projective measurement described by the family $`\underset{¯}{Q}`$ of projections on a system given by $`\rho `$, but where we never learn of the result of the measurement. In contrast $`S(\rho |\sigma )`$ is interpreted as a set of projective measurements given by the projections $`Q_j`$, each performed with the probability $`dimQ_j\sigma _j`$, and where we learn of each outcome $`F(\rho ,Q_j)`$ separately. The sum in (1) and (5) then reflects the occurrence of a quantum decoherence. In other words one considers the family of density operators $`\rho _{Q_j},Q_j\rho Q_j0`$, takes their von Neumannn entropy and then forms the linear combination with the non-negative coefficients $`\mathrm{Tr}Q_j\sigma \mathrm{Tr}Q_j\rho `$.

By definition we have

$$F(\rho ,Q=𝕀)=S(\rho |\sigma =(1/dim𝕀)𝕀)=S(\rho ).$$
(8)
We consider this property to be necessary for any other sensible definition of a conditional entropy involving two density matrices. It holds for Shannon’s conditional entropy $`H(X|Y)`$ in the form $`H(X|Y)=H(X)`$ when $`Y`$ is the trivial partition (see section 3), which means that there is no gain in information, if $`Y`$ contains no information. We will return to this point in section 3.

Some additional remarks are in order. Since the quantity $`S(\rho |\sigma )`$ is supposed to be a quantum mechanical mechanical analogue of Shannon’s conditional entropy $`H(X,Y)`$, $`\rho `$ corresponds to $`X`$ and $`\sigma `$ to $`Y`$. In analogy to the classical case, where $`X`$ and $`Y`$ may be considered to be stochastic variables living on the same space, here the density matrices $`\rho `$ and $`\sigma `$ also live on the same space. Unfortunately with this correspondence $`S(\rho |\sigma )`$ does not reduce to the classical case when $`\rho `$ and $`\sigma `$ commute (see (33) and its discussion in section 3). As matter of fact, we shall argue in section 4 that a quantum conditional entropy with this property does not exist.

By construction we have the obvious invariance under unitary automorphisms

$$F(\rho ,Q)=F(U\rho U^1,UQU^1),$$
(9)
for any $`U𝒰()`$, the group of unitary operators in $``$. This relation (9) immediately implies

$$S(U\rho U^1|U\sigma U^1)=S(\rho |\sigma )$$
(10)
for all $`U`$. Relation (10) reflects the fact that $`S(\rho |\sigma )`$ is defined intrinsically and is in particular basis independent. Therefore this invariance property should also hold for any alternative, sensible definition of a quantum mechanical conditional entropy defined in terms of two density matrices. We shall comment on the classical analogue to (10) in section 4.

The next observation is also important. It is easy to see that $`F(\rho ,Q)`$ is continuous in $`\rho `$ and $`Q`$ by the same arguments used to prove continuity of $`S(\rho )`$. Therefore $`S(\rho |\sigma )`$ is also continuous in $`\rho `$ for fixed $`\sigma `$. However, $`S(\rho |\sigma )`$ is not continuous in $`\sigma `$ everywhere for all fixed $`\rho `$. It is continuous on the dense open subset where the eigenvalues of $`\sigma `$ are non-degenerate.In fact, it is zero there(see below). So this lack of continuity occurs where $`\sigma `$ has degenerate eigenvalues and is due to the fact that for $`Q=Q^{}+Q^{\prime \prime }`$ being the sum of two projections both $`0`$ and which are orthogonal to each other, i.e. $`Q^{}Q^{\prime \prime }=0`$, in general one has

$$dimQF(\rho ,Q)dimQ^{}F(\rho ,Q^{})+dimQ^{\prime \prime }F(\rho ,Q^{\prime \prime }).$$
(11)
To understand this consider the case when $`dim=2`$. Then $`S(\rho |\sigma )=S(\rho )`$ if $`\sigma =1/2𝕀`$ and $`S(\rho |\sigma )=0`$ otherwise. At the moment we do not know whether this lack of continuity of $`S(\rho |\sigma )`$ in $`\sigma `$ is a desirable feature or not, i.e whether this can be understood quantum mechanically, when we interpret $`S(\rho |\sigma )`$ as the entropy of $`\rho `$ conditioned by $`\sigma `$. Observe that a degeneracy typically occurs when a non-trivial symmetry is present. In other words there is then a non-trivial non-abelian subgroup $`𝒢=𝒢(\sigma )`$ of $`𝒰()`$ such that $`U\sigma U^1=\sigma `$ for all $`U𝒢`$. Note that $`𝒢`$ always contains a subgroup isomorphic to the abelian group $`U(N=dim)`$. In this picture a removal of degeneracies is related to a breakdown of symmetry, a familiar phenomenon in physics.

To proceed further, $`F(\rho ,Q)=0`$ if $`Q\rho Q=0`$, which can happen for $`Q0`$ only if $`\rho `$ has zero as an eigenvalue, i.e. if $`\rho `$ is not strictly positive. Then also $`(𝕀Q)\rho Q=Q\rho (𝕀Q)=0`$. In fact, by Schwarz inequality for any $`\psi ,\psi ^{}`$ we have

$$|<\psi ,Q\rho (𝕀Q)\psi ^{}>|||\rho ^{1/2}Q\psi ||||\rho ^{1/2}(𝕀Q)\psi ^{}||=0.$$

This also shows that $`Q\rho Q=0`$ is equivalent to $`Q\rho =0`$, which in turn by the self-adjointness of $`\rho `$ and $`Q`$ is equivalent to $`\rho Q=0`$. By the trivial identity

$$\rho =Q\rho Q+(𝕀Q)\rho Q+Q\rho (𝕀Q)+(𝕀Q)\rho (𝕀Q),$$
(12)
valid for all $`\rho ,Q`$, we therefore also have $`\rho =(𝕀Q)\rho (𝕀Q)`$ whenever $`Q\rho Q=0`$. Obviously (12) gives $`\mathrm{Tr}\rho =\mathrm{Tr}Q\rho Q+\mathrm{Tr}(𝕀Q)\rho (𝕀Q)`$ such that in particular the inequalities $`0\mathrm{Tr}Q\rho Q1`$ and $`0\mathrm{Tr}(𝕀Q)\rho (𝕀Q)1`$ hold for any $`\rho `$ and $`Q`$. By relation (3) we also have $`F(\rho ,Q)0`$ and hence $`S(\rho |\sigma )0`$ for all $`\rho ,Q`$ and $`\sigma `$. Now $`S(\rho _Q)=0,Q\rho Q0`$ holds if and only if $`\rho _Q`$ is a pure state, i.e. a one-dimensional projection. Also for $`dimQ=1`$ one always has $`Q\rho Q=(\mathrm{Tr}Q\rho Q)Q`$. We collect this observation in

###### Lemma 2.1.

$`F(\rho ,Q)=0`$ if and only if $`Q\rho Q`$ is a multiple of a one-dimensional projection.

This multiple is allowed to be zero. To characterize such $`Q`$’s fulfilling the conditions of the lemma, let $`P(\rho )0`$ be the projection operator onto the subspace corresponding to the non-zero eigenvalues, such that $`P(\rho )\rho =\rho =\rho P(\rho )`$ and in particular $`P(\rho )=𝕀`$ if $`\rho >0`$. Using the spectral representation of $`\rho `$ it is easy to see that $`Q\rho Q`$ is a multiple (possibly zero) of a one-dimensional projection if and only if $`Q`$ may be written as $`Q=Q^{}+Q^{\prime \prime }`$ with $`dimQ^{}1`$ and $`P(\rho )Q^{\prime \prime }=\rho Q^{\prime \prime }=0`$.

More generally consider the case where $`Q\rho Q=(\mathrm{Tr}(Q\rho Q)/dimQ^{})Q^{},Q0`$ holds for a suitable projection operator $`Q^{}`$ such that in particular $`0Q^{}Q`$ and $`Q^{}`$ is unique whenever $`Q\rho Q0`$. Then $`F(\rho ,Q)=(\mathrm{Tr}Q\rho Q)\mathrm{ln}dimQ^{}`$ and $`\rho _Q=(1/dimQ^{})Q^{}`$. This gives the

###### Lemma 2.2.

If all non-zero eigenvalues of $`\sigma `$ are non-degenerate then $`S(\rho |\sigma )=0`$ for all $`\rho `$. More generally if $`Q_j\rho Q_j`$ is a multiple (possibly zero) of some projection operator $`Q_j^{}(Q_j)`$ for all $`j`$ with $`\sigma _j>0`$, then

$$S(\rho |\sigma )=\underset{j}{}\mathrm{Tr}Q_j\rho \mathrm{Tr}Q_j\sigma \mathrm{ln}dimQ_j^{}.$$
(13)
Observe that $`S(\rho |\sigma )=0`$ for all pure states $`\sigma `$ and all $`\rho `$. If $`\rho `$ is pure then $`Q\rho Q`$ is always a multiple of a pure state for all $`Q`$. Therefore $`S(\rho |\sigma )=0`$ also holds for all $`\sigma `$ whenever $`\rho `$ is pure. Also if $`\rho \sigma =0`$ which is equivalent to $`\mathrm{Tr}\rho \sigma =0`$ and which can happen only if neither $`\rho `$ nor $`\sigma `$ is strictly positive, then again $`S(\rho |\sigma )=0`$. Sufficient (but not necessary) for the condition of Lemma 2.2 to hold is that to each $`j`$ with $`\sigma _j>0`$ there is $`i(j)`$ with $`Q_jP_{i(j)}`$. For these $`j`$’s $`Q_j^{}=Q_j,Q_j\rho Q_j=\rho _{i(j)}Q_j`$ and hence $`\mathrm{Tr}Q_j\rho =\rho _{i(j)}dimQ_j`$. This gives in particular

$$S(\rho |\rho )=\underset{i}{}\rho _i^2(dimP_i)^2\mathrm{ln}dimP_i.$$
(14)
Therefore the relation $`S(\rho |\rho )=0`$ holds if and only if all the non-zero eigenvalues of $`\rho `$ are non-degenerate, the if part being a special case of Lemma 2.2.

If in addition to the property $`Q_jP_{i(j)}`$ the density matrix $`\sigma `$ is such that

$$\underset{j:i(j)=i}{}\sigma _j(dimQ_j)^2\mathrm{ln}dimQ_j\rho _i(dimP_i)^2\mathrm{ln}dimP_i$$

holds for all $`i`$, then by (13) and (14) $`S(\rho |\sigma )S(\rho |\rho )`$. Note that this last condition is satisfied if

$$\underset{j:i(j)=i}{}\sigma _jdimQ_j\rho _idimP_i$$

holds since trivially $`dimQ_jdimP_{i(j)}`$.

We return to a discussion of the general properties of $`F(\rho ,Q)`$ and $`S(\rho |\sigma )`$. The first main result of this article shows that $`S(\rho |\sigma )`$ shares an important property with $`S(\rho )`$ (see e.g.  for the classical and the quantum entropy and  for Shannon’s conditional entropy and derived quantities).

###### Theorem 2.1.

$`F(\rho ,Q)`$ and $`S(\rho |\sigma )`$ are both concave in $`\rho `$.

Again we consider this property to be necessary for any sensible definition of a quantum conditional entropy. Like for the entropy $`S(\rho )`$ itself it states that mixing (in $`\rho `$) increases (conditional) entropy. On the other hand the case $`dim=2`$ discussed above shows that in general $`S(\rho |\sigma )`$ for fixed $`\rho `$ is neither convex nor concave in $`\sigma `$. Intuitively it would be desirable to have concavity with respect to $`\sigma `$ since mixing the conditioning should increase conditional entropy.

The proof follows easily from the presentation (6) and (7) and the known convexity property of the relative entropy.

The second main result of this article shows in particular that $`S(\rho |\sigma )`$ satisfies Shannon’s inequality.

###### Theorem 2.2.

The following inequalities hold for all density matrices $`\rho `$ and $`\sigma `$ in a fixed finite dimensional Hilbert space

$$0S(\rho |\sigma )S(\rho ).$$
(15)
If $`\rho >0`$ the last inequality is strict unless $`\sigma =(1/dim𝕀)𝕀`$.

The above comparison of $`S(\rho |\sigma )`$ with $`S(E_{\underset{¯}{Q}}(\rho )`$ suggests another definition of conditional entropy with the conditioning not given in terms of a density matrix $`\sigma `$ but rather only in terms of any resolution $`\underset{¯}{Q}`$ of the identity.

$$S(\rho |\underset{¯}{Q})=\underset{j}{}\frac{dimQ_j}{dim𝕀}F(\rho ,Q_j).$$
(16)
By (17) below we have

$$0S(\rho |\underset{¯}{Q})S(\rho ),$$

where the first inequality is an equality if $`dimQ_j=1`$ for all $`j`$ and the second one an equality if the spectral resolution is trivial, i.e. if $`\underset{¯}{Q}=\{𝕀\}`$. We note that in (16) any sequence of numbers $`\sigma _j^{}0`$ ( labeled in the same way as the $`Q_j`$’s) with $`_j\sigma _j^{}=1`$ and replacing $`dimQ_j/dim𝕀`$ would do equally well. But then we may combine and encode these data $`\underset{¯}{Q}`$ and $`\{\sigma \}`$ in the density matrix $`\sigma =_j\sigma _jQ_j`$ with $`\sigma _j=\sigma _j^{}/dimQ_j`$. If in addition all the $`\sigma _j`$’s are pairwise different, then by our discussion above they and the spectral resolution $`\underset{¯}{Q}`$ may be recovered from $`\sigma `$ and we are back to our construction $`S(\rho |\sigma )`$.

Due to the relation $`1=\mathrm{Tr}\sigma =_jdimQ_j\sigma _j`$ this second theorem is an immediate consequence of the following

###### Lemma 2.3.

For all $`\rho `$ and $`Q`$ the inequality

$$F(\rho ,Q)S(\rho )$$
(17)
holds. If $`\rho >0`$ this inequality is strict unless $`Q=𝕀`$.

Before we turn to a proof we make some remarks. We conjecture that in the general case $`\rho 0`$, the inequality (17) is strict unless $`Q\rho =\rho `$. This would imply that the second inequality in (15) is strict unless $`\sigma \rho =(\mathrm{Tr}\sigma \rho )\rho `$, which means the following. Any $`\sigma `$ with $`\sigma \rho =(\mathrm{Tr}\sigma \rho )\rho `$ is of the form $`\sigma =(\mathrm{Tr}\sigma \rho )P(\rho )+\sigma ^{}`$ with $`(𝕀P(\rho ))\sigma ^{}=\sigma ^{}`$.

Instead of $`F(\rho ,Q)`$ one might be tempted to consider instead the quantity (see (2))

$$\stackrel{~}{F}(\rho ,Q)=\mathrm{Tr}(Q\rho Q\mathrm{ln}Q\rho Q)0$$

and try to prove $`\stackrel{~}{F}(\rho ,Q)S(\rho )`$. Obviously we have $`\stackrel{~}{F}(\rho ,Q)F(\rho ,Q)`$. Consider, however, the case where $`dimQ=1`$ and $`\rho =P,dimP=1`$ (i.e. $`\rho `$ is pure) and with $`P`$ chosen such that $`QPQ=(\mathrm{Tr}QP)Q`$ satisfies $`0<\mathrm{Tr}PQ<1`$. Then $`0=S(\rho =P)<\stackrel{~}{F}(\rho =P,Q)`$. Furthermore one has $`F(\rho ,Q)S(\rho _Q)`$ when $`0<\mathrm{Tr}Q\rho Q(1)`$. But it does not make sense to replace $`F(\rho ,Q)`$ by $`S(\rho _Q)`$ as an alternative, since $`S(\rho _Q)`$ is only defined when $`Q\rho Q0`$. Even if $`Q\rho Q0`$, one does not have $`S(\rho _Q)S(\rho )`$ in general. To see this we will consider an example. For any $`0\psi `$ let $`P_\psi `$ be the 1-dim. projection onto the subspace spanned by $`\psi `$.

###### Example 2.1.

Let $`dim=4`$ with $`\psi _1,\psi _2,\psi _3,\psi _4`$ being an orthonormal basis. Let $`Q=P_{\psi _1}+P_{\psi _2}`$ be the 2-dim. projection onto the sub-space spanned by $`\psi _1`$ and $`\psi _2`$. Choose $`\rho (\varphi _1,\varphi _2)=\rho _1P_{\psi _1^{}}+\rho _2P_{\psi _2^{}},\rho _1+\rho _2=1`$ with

$`\psi _1^{}`$ $`=`$ $`\mathrm{cos}\varphi _1\psi _1+\mathrm{sin}\varphi _1\psi _3,`$
$`\psi _2^{}`$ $`=`$ $`\mathrm{cos}\varphi _2\psi _2+\mathrm{sin}\varphi _2\psi _4,\mathrm{cos}\varphi _10\mathrm{cos}\varphi _2.`$
Then

$$\rho (\varphi _1,\varphi _2)_Q=\frac{\mathrm{cos}^2\varphi _1\rho _1}{\mathrm{cos}^2\varphi _1\rho _1+\mathrm{cos}^2\varphi _2\rho _2}P_{\psi _1}+\frac{\mathrm{cos}^2\varphi _2\rho _2}{\mathrm{cos}^2\varphi _1\rho _1+\mathrm{cos}^2\varphi _2\rho _2}P_{\psi _2}.$$

Assume $`0<\rho _1<1`$ such that $`S(\rho (\varphi _1,\varphi _2))0`$ and choose $`\varphi _1`$ and $`\varphi _2`$ such that $`cos^2\varphi _1\rho _1=\mathrm{cos}^2\varphi _2\rho _2`$. This gives $`\rho (\varphi _1,\varphi _2)_Q=1/2Q`$ with $`S(\rho (\varphi _1,\varphi _2)_Q)=\mathrm{ln}2>S(\rho (\varphi _1,\varphi _2))`$ whenever $`\rho _11/2`$. On the other hand, some easy estimates show that indeed $`F(\rho (\varphi _1,\varphi _2),Q)S(\rho (\varphi _1,\varphi _2))`$ holds for all $`\varphi _1`$ and $`\varphi _2`$.

This example also shows that in general neither $`\rho _Q`$ nor $`S(\rho _Q)`$ for $`Q\rho Q=0`$ may be defined by a limiting procedure. In fact, we may let $`\varphi _1`$ and $`\varphi _2`$ tend to $`\pi /2`$ in such a way that $`\mathrm{cos}^2\varphi _2/\mathrm{cos}^2\varphi _1`$ tends to an arbitrary constant $`0`$ showing that in the limit for $`\rho (\varphi _1,\varphi _2)_Q`$ we may obtain an arbitrary convex combination of $`P_{\psi _1}`$ and $`P_{\psi _2}`$ and hence an arbitrary value between 0 and $`\mathrm{ln}2`$ for the entropy. By the convexity of the relative entropy we also have

$$F(\rho ,Q)+F(\rho ,𝕀Q)S(E_{\{Q,𝕀Q\}}(\rho )).$$

On the other hand, in general $`F(\rho ,Q)+F(\rho ,𝕀Q)`$ is in general not bounded above by $`S(\rho )`$. Indeed, consider the following

###### Example 2.2.

Let the set-up be as in Example 2.1. With repsect to this basis let

$$\rho (\kappa )=\frac{1}{4}\left(\begin{array}{cccc}1& 0& 0& \kappa \\ 0& 1& \kappa & 0\\ 0& \kappa & 1& 0\\ \kappa & 0& 0& 1\end{array}\right)$$

with $`0\kappa 1`$. The two two-fold degenerate eigenvalues are $`1/4(1\pm \kappa )`$. This gives $`F(\rho ,Q)+F(\rho ,𝕀Q)=\mathrm{ln}2`$ whereas $`S(\rho (\kappa ))=\mathrm{ln}21/2((1+\kappa )\mathrm{ln}(1+\kappa )+(1\kappa )\mathrm{ln}(1\kappa ))<\mathrm{ln}2`$, whenever $`0<\kappa `$.

The quantity

$$\mathrm{\Delta }S(\rho )=S(\rho )S(\rho |\rho )\underset{i}{}dimP_i\rho _i\mathrm{ln}(dimP_i\rho _i)$$
(18)
is of special interest. The inequality is a consequence of $`dimP_i\rho _i1`$ and again implies that the right hand side is non-negative and equal to zero if and only if $`\rho =(1/dim𝕀)𝕀`$ such that $`\mathrm{\Delta }S(\rho )>0`$ unless $`\rho =(1/dim𝕀)𝕀`$. In more detail the inequality in (18) may also be written as follows. Let $`S_{cl}(\underset{¯}{p})0`$ be the classical entropy for the probability distribution $`\underset{¯}{p}=(p_1,p_2,\mathrm{}p_n),p_k0,_kp_k=1`$

$$S_{cl}(\underset{¯}{p})=\underset{k=1}{\overset{n}{}}p_k\mathrm{ln}p_k.$$

such that in particular

$$S(\rho )=S_{cl}(\underset{¯}{p}(\rho ))$$
(19)
with

$$\underset{¯}{p}(\rho )=(\underset{dimP_1}{\underset{}{\rho _1,..,\rho _1}},\underset{dimP_2}{\underset{}{\rho _2,..,\rho _2}},\mathrm{}.).$$
(20)
(18) may now be rewritten as

$$0S_{cl}(\underset{¯}{\overset{^}{p}}(\rho ))\mathrm{\Delta }S(\rho )$$
(21)
with

$$\underset{¯}{\overset{^}{p}}(\rho )=(dimP_1\rho _1,dimP_2\rho _2,\mathrm{})$$

and where $`S_{cl}(\underset{¯}{\overset{^}{p}})=0`$ if and only if $`\rho `$ is a pure state. $`\mathrm{\Delta }S(\rho )`$ is easily shown to be continuous in $`\rho `$ and is obviously bounded above by $`\mathrm{ln}dim𝕀=\mathrm{ln}dim=S(\rho =(1/dim𝕀)𝕀)`$. It would be interesting to find its maximum in $`\rho `$ for fixed dimension of $`dim`$. Note also that

$$S(\rho )=S_{cl}(\underset{¯}{p}(\rho ))=S_{cl}(\underset{¯}{\overset{^}{p}}(\rho ))+\underset{i}{}dimP_i\sigma _i\mathrm{ln}dimP_iS_{cl}(\underset{¯}{\overset{^}{p}}(\rho ))$$
(22)
with equality if and only if $`dimP_i=1`$ for all $`i`$ with $`\sigma _i>0`$. We will discuss $`\mathrm{\Delta }S(\rho )`$ below when we compare $`S(\rho |\sigma )`$ with Shannon’s conditional entropy.

We turn to the proof of (17). First recall that $`F(\rho ,Q)`$ is continuous in $`\rho `$ (and $`Q`$). Hence it suffices to consider the case $`\rho >0`$ which implies that $`Q\rho Q0`$ for all $`Q0`$. Since $`F(\rho ,Q)=0`$ for $`Q=0`$ and $`dimQ=1`$ and since $`F(\rho ,𝕀)=S(\rho )`$ it suffices to consider the case $`1<dimQ<dim𝕀`$.

Now $`𝒰()`$ operates transitively and continuously on the Grassmannian of all $`n`$-dimensional subspaces of $``$, $`(1ndim)`$. For each $`n`$ this space is therefore compact and homeomorphic to the set of all projections of dimension $`n`$. Obviously on this set $`𝒰()`$ operates, again continuously, via $`U:PUPU^1`$. By (9)

$$F_n(\rho )=\underset{Q:dimQ=n}{sup}F(\rho ,Q)=\underset{U:U𝒰()}{sup}F(\rho ,UQ_0U^1)=\underset{U:U𝒰()}{sup}F(U\rho U^1,Q_0),$$
(23)
which is finite for each $`n`$. Here $`Q_0`$ is any orthogonal projection with $`dimQ_0=n`$. In particular we may choose $`Q_0`$ such that $`F_n(\rho )=F(\rho ,Q_0)`$. Consider the one-parameter unitary subgroup $`U(t)=\mathrm{exp}(itK)`$, where $`K`$ is an arbitrary self-adjoint operator on $``$. Then we must have $`f_K(t)=F(U(t)\rho U(t),Q_0)F(\rho ,Q_0)=f_K(t=0)`$ for all $`t`$ and all s.a. $`K`$. Now it is well known that for any one parameter family of strictly positive operators $`A(t)`$ which is differentiable in $`t`$ one has

$$\frac{d}{dt}\mathrm{Tr}(A(t)\mathrm{ln}A(t))=\mathrm{Tr}((𝕀+\mathrm{ln}A(t))\frac{d}{dt}A(t)).$$

Recalling the assumption $`\rho >0`$ such that $`Q_0\rho Q_0>0`$ when restricted to the subspace $`Q_0`$, it is easy to see that $`f_K(t)`$ is also differentiable in $`t`$ at $`t=0`$ and

$$\begin{array}{ccc}\frac{d}{dt}f_K(t)|_{t=0}& =& i\mathrm{Tr}((𝕀+\mathrm{ln}Q_0\rho Q_0)Q_0[K,\rho ]Q_0)\\ & & +i(1+\mathrm{ln}\mathrm{Tr}(Q_0\rho Q_0))\mathrm{Tr}Q_0[K,\rho ]Q_0\\ & =& i\mathrm{Tr}([\rho ,\mathrm{ln}\mathrm{Tr}(Q_0\rho Q_0)Q_0Q_0(\mathrm{ln}Q_0\rho Q_0)Q_0]K).\end{array}$$
(24)
By definition of $`Q_0`$ we must have $`d/dtf_K(t=0)=0`$ for all $`K`$. But then (24) implies that $`\rho `$ commutes with $`B=Q_0B=BQ_0`$ given as

$$B=\mathrm{ln}\mathrm{Tr}(Q_0\rho Q_0)Q_0Q_0(\mathrm{ln}Q_0\rho Q_0)Q_0.$$

This in turn implies that $`\rho `$ commutes with $`Q_0`$ itself, which is easy to see. Indeed, use the spectral representation $`Q_0\rho Q_0=_k\rho _k^{}Q_k^{}`$ with $`Q_k^{}Q_0,dimQ_k^{}=1`$ and $`_kQ_k^{}=Q_0`$ to write $`B`$ as

$$B=\underset{k}{}(\mathrm{ln}(\underset{l}{}\rho _l^{})\mathrm{ln}\rho _k^{})Q_k^{}.$$

Now write any $`\psi Q_0`$ as $`\psi =_ka_k\psi _k`$, where $`\psi _k`$ is a unit vector in $`Q_k^{}`$. Set

$$\varphi =\underset{k}{}\frac{a_k}{(\mathrm{ln}(_l\rho _l^{})\mathrm{ln}\rho _k^{})}\psi _kQ_0.$$

$`\varphi `$ is well defined since $`_l\rho _l^{}\rho _k^{}`$ for every $`k`$. This follows from our assumption $`n>1`$, the fact that $`\mathrm{ln}x`$ is strictly monotonic in $`x`$ and that $`\rho _k^{}>0`$ for all $`k`$, since $`Q_0\rho Q_0`$ when restricted to $`Q_0`$ is strictly positive. By construction $`\psi =B\varphi `$ such that $`\rho \psi =\rho B\varphi =B\rho \varphi =Q_0B\rho \varphi Q_0`$. Thus $`\rho `$ leaves $`Q_0`$ invariant and hence commutes with $`Q_0`$, as was claimed. But then we have $`\rho =Q_0\rho Q_0+(𝕀Q_0)\rho (𝕀Q_0)`$ which implies

$$S(\rho )=\mathrm{Tr}(Q_0\rho Q_0\mathrm{ln}Q_0\rho Q_0)\mathrm{Tr}((𝕀Q_0)\rho (𝕀Q_0)\mathrm{ln}(𝕀Q_0)\rho (𝕀Q_0)).$$

This gives

$`S(\rho )`$ $`=F(\rho ,Q_0)`$ $`\mathrm{Tr}((𝕀Q_0)\rho (𝕀Q_0)\mathrm{ln}(𝕀Q_0)\rho (𝕀Q_0))`$ (25)
$`\mathrm{Tr}Q_0\rho Q_0\mathrm{ln}\mathrm{Tr}Q_0\rho Q_0.`$
The two last terms in (25), however, are non-negative. This concludes the proof of the claim (17). To prove the second part of Lemma 2.3, we observe that the last two terms in (25) vanish exactly when $`(𝕀Q_0)\rho (𝕀Q_0)=0`$. But this contradicts the assumption $`\rho >0`$ and $`dimQ_0<dim𝕀`$, the case $`Q=𝕀`$ having been discussed previously. This completes the proof of Lemma 2.3.

## 3 Comparison with the classical case

In this section we provide a comparison with the classical theory of Shannon (see  and for expositions e.g. ). For the convenience of the reader and in order to establish notation we recall the basic facts. Let $`\{\mathrm{\Omega },\mu \}`$ be a probability space. Furthermore let $`X=\{X_\alpha \}`$ and $`Y=\{Y_\beta \}`$ be any two partitions (up to measure zero) of $`\mathrm{\Omega }`$ into disjoint subsets of non-zero measure. For simplicity we will assume these partitions to be finite, i.e. we choose the indices $`\alpha `$ and $`\beta `$ to be in the range $`1\alpha n,1\beta m`$. Set $`\underset{¯}{p}(X)=\{p_\alpha \}`$ with $`p_\alpha =\mu (X_\alpha )>0`$ and $`\underset{¯}{p}(Y)=\{q_\beta \}`$ with $`q_\beta =\mu (Y_\beta )>0`$ such that $`_\alpha p_\alpha =1`$ and $`_\beta q_\beta =1`$. Here and in what follows $`\alpha `$ is an index referring to $`X`$ and $`\beta `$ to $`Y`$. Then $`H(X)=_\alpha p_\alpha \mathrm{ln}p_\alpha 0`$ and similarly $`H(Y)=_\beta q_\beta \mathrm{ln}q_\beta 0`$ is Shannon’s entropy. Actually Shannon used $`\mathrm{log}_2`$ instead of $`\mathrm{ln}`$ adapting to the situation where information is coded in bits, but this is not relevant for our purpose. Since $`H(X)=S_{cl}(\underset{¯}{p}(X))`$ this concept of information theory relates to the concept of entropy in classical statistical mechanics. Shannon’s conditional entropy is now given as follows. Let

$$p_{\alpha |\beta }=\frac{\mu (X_\alpha Y_\beta )}{\mu (Y_\beta )},q_{\beta |\alpha }=\frac{\mu (Y_\beta X_\alpha )}{\mu (X_\alpha )}$$

be conditional probabilities associated to $`X`$ and $`Y`$ (i.e. $`p_{\alpha |\beta }`$ is the probability that $`X_\alpha `$ will happen, given that $`Y_\beta `$ has happened). Obviously

$$p_{\alpha |\beta }q_\beta =q_{\beta |\alpha }p_\alpha (=\mu (A_\alpha B_\beta ))$$
(26)
for all $`\alpha ,\beta `$, which is called Bayes rule for $`p_{\alpha |\beta }`$ and $`q_{\beta |\alpha }`$. Let $`\underset{¯}{p}_\beta =(p_{1|\beta },p_{2|\beta },..,p_{n|\beta })`$ and $`\underset{¯}{q}_\alpha =(q_{1|\alpha },q_{2|\alpha },\mathrm{}.q_{m|\alpha })`$, such that

$$\underset{¯}{p}=\underset{\beta =1}{\overset{m}{}}q_\beta \underset{¯}{p}_\beta ,\underset{¯}{q}=\underset{\alpha =1}{\overset{n}{}}p_\alpha \underset{¯}{q}_\alpha .$$
(27)
Shannon’s conditional entropy is now defined as

$$H(X|Y)=\underset{\beta =1}{\overset{m}{}}q_\beta S_{cl}(\underset{¯}{p}_\beta )$$
(28)
and it satisfies

$$0H(X|Y)H(X).$$
(29)
We observe that the second inequality, called Shannon’s inequality, is a consequence of the concavity of the function $`\underset{¯}{p}S_{cl}(\underset{¯}{p})`$ and (27). It states that on average information on $`X`$ is gained if $`Y`$ is known. Also $`0H(X,Y)=H(Y)+H(X|Y)`$ is symmetric in $`X`$ and $`Y`$ and satisfies

$$H(Y)H(X,Y)H(X)+H(Y).$$
(30)
Actually $`H(X,Y)=H(XY)`$, where $``$ denotes the join of two partitions. The inequalities in (29) and (30) turn into equalities if the following conditions hold. $`X`$ and $`Y`$ are said to be independent if $`p_{\alpha |\beta }=p_\alpha `$ holds for all $`\alpha `$ and $`\beta `$. This means that $`\underset{¯}{p}_\beta `$ is actually independent of $`\beta `$ and equals $`\underset{¯}{p}`$ and $`\underset{¯}{q}_\alpha `$ is independent of $`\alpha `$ and equals $`\underset{¯}{q}`$. In particular $`S_{cl}(\underset{¯}{p}_\beta )=S_{cl}(\underset{¯}{p})`$ holds for all $`\beta `$ and $`S_{cl}(\underset{¯}{q}_\alpha )=S_{cl}(\underset{¯}{q})`$ for all $`\alpha `$. The second inequality in (29) and the second inequality in (30) (which are equivalent) are now equalities if and only if $`X`$ and $`Y`$ are independent. It follows from the fact that $`S_{cl}(\underset{¯}{p})`$ is strictly concave in $`\underset{¯}{p}`$. Secondly $`X`$ is called a consequence of $`Y`$ if to each $`\alpha `$ there is $`\beta (\alpha )`$ such that $`p_{\alpha |\beta (\alpha )}=1`$. So this means that $`p_{\alpha |\beta }=0`$ for all $`\beta \beta (\alpha )`$ and hence $`S_{cl}(\underset{¯}{p}_\beta )=0`$ for all $`\beta `$. Therefore the first inequality in (29) and equivalently the first inequality in (30) are equalities if and only if $`X`$ is a consequence of $`Y`$. In particular

$$H(X|X)=0,$$
(31)
i.e. $`H(X,X)=H(X)`$.

With this brief review of Shannon’s theory we turn to a comparison with our quantum mechanical construction. Obviously (29) corresponds to (15) when we let $`X`$ correspond to $`\rho `$ and $`Y`$ to $`\sigma `$. Note, however, the difference between (31) and (14). Moreover for the quantity $`S(\rho ,\sigma )=S(\sigma )+S(\rho |\sigma )`$ we have the inequalities

$$S(\sigma )S(\rho ,\sigma )S(\rho )+S(\sigma ),$$
(32)
which correspond to (30). $`S(\rho ,\sigma )`$ is in general not symmetric in $`\rho `$ and $`\sigma `$ . To see this consider commuting $`\rho `$ and $`\sigma `$. Then we have

$$S(\rho |\sigma )=\underset{j,i}{}dimQ_j\sigma _j\rho _idim(P_iQ_j)\mathrm{ln}\frac{\rho _i}{\mathrm{Tr}(\rho Q_j)}.$$
(33)
We remark that if $`\mathrm{Tr}(\rho Q_j)=0`$ for a fixed $`j`$ then $`\mathrm{Tr}(P_iQ_j)=0`$ for all $`i`$. Also (14) is a special case of (33). (33) shows that even in the commutative case $`S(\rho ,\sigma )`$ is not symmetric in $`\rho `$ and $`\sigma `$. So this implies that in the commutative case $`S(\rho |\sigma )`$ does not reduce to $`H(X|Y)`$ for any choice of $`X=X(\rho )`$ and $`Y=Y(\sigma )`$ with $`H(X)=S(\rho )`$ and $`H(Y)=S(\sigma )`$. This lack of symmetry of $`S(\rho ,\sigma )`$ is in contrast to the symmetry of its classical counterpart $`H(X,Y)`$, which has an important interpretation. The relation $`H(X,Y)=H(Y,X)`$ is equivalent to $`H(Y)+H(X|Y)=H(X)+H(Y|X)`$, a consequence of Bayes rule. But this means that on average the information on $`Y`$ plus the information on $`X`$ given $`Y`$ is equal to the information on $`X`$ plus the information on $`Y`$ given $`X`$. It would be interesting to see whether this failure of symmetry for $`S(\rho ,\sigma )`$ has a sensible interpretation in the context of the familiar Alice and Bob set-up in quantum information theory, see e.g. .

Finally consider

$$0S(\rho ||\sigma )=S(\rho )+S(\sigma )S(\rho ,\sigma )=S(\rho )S(\rho |\sigma )S(\rho )$$
(34)
which corresponds to

$$0I(X||Y)=H(X)+H(Y)H(X,Y)=H(X)H(X|Y).$$

On average $`0I(X||Y)H(X)`$ gives the information gain on $`X`$ when knowing $`Y`$. Thus if there is no information content at all in $`Y`$, i.e. if $`Y`$ is the trivial partition $`\{\mathrm{\Omega }\}`$, then there is no information gain in $`X`$

$$I(X||Y=\{\mathrm{\Omega }\})=0.$$
(35)
Thus (35) corresponds to (8) when rewritten as $`S(\rho ||\sigma =(1/dim𝕀)𝕀)=0`$. Therefore we also interpret the quantum mechanical analogue $`S(\rho ||\sigma )`$ as a quantum information gain for $`\rho `$ given $`\sigma `$ and which by (34) can be at most $`S(\rho )`$. In particular the gain is maximal for all $`\rho `$, if all non-zero eigenvalues of $`\sigma `$ are non-degenerate. The gain is also maximal if $`\rho \sigma =0`$, since then $`S(\rho |\sigma )=0`$, see Lemma 2.2 and the remark thereafter.

Finally $`\mathrm{\Delta }S(\rho )`$ (see (18)) corresponds to $`I(X||X)`$ and describes the situation where $`\rho `$ is conditioned on itself, $`\sigma =\rho `$. Then by (21) there is non-zero information gain unless $`\rho `$ is pure (and then a gain is not necessary). In contrast to the classical situation, $`I(X||X)=H(X)`$, which gives complete information gain when $`X`$ is conditioned on itself, there is complete information gain in the quantum case, $`\mathrm{\Delta }S(\rho )=S(\rho )`$, if and only if all non-zero eigenvalues of $`\sigma `$ are non-degenerate.

## 4 Attempts of alternative constructions

We conclude by addressing the natural question whether there is a quantity $`S^\mathrm{?}(\rho |\sigma )`$ which shares more properties with Shannon’s conditional entropy than the $`S(\rho |\sigma )`$ we have given. More precisely and by the arguments given in the preceding sections it would be desirable for $`S^\mathrm{?}(\rho |\sigma )`$ to have (most of) the following properties

1. Invariance under the group $`𝒰()`$: $`S^\mathrm{?}(U\rho U^1|U\sigma U^1)=S^\mathrm{?}(\rho |\sigma )`$ for all $`U𝒰()`$ (compare (10)).
2. Bounds: $`0S^\mathrm{?}(\rho |\sigma )S(\rho )`$ for all $`\rho `$ and $`\sigma `$ with $`S^\mathrm{?}(\rho |\rho )=0`$ and
$`S^\mathrm{?}(\rho |\sigma =(1/dim𝕀)𝕀)=S(\rho )`$.
3. Classical equivalence with Shannon’s conditional entropy.
4. Symmetry: $`S^\mathrm{?}(\rho ,\sigma )=S(\sigma )+S^\mathrm{?}(\rho |\sigma )`$ is symmetric in $`\rho `$ and $`\sigma `$.
5. Continuity of $`S^\mathrm{?}(\rho |\sigma )`$ in $`\rho `$ and in $`\sigma `$.
6. Concavity of $`S^\mathrm{?}(\rho |\sigma )`$ in $`\rho `$ and $`\sigma `$.

Note that $`S(\rho |\sigma )`$ fulfills condition 1, condition 2 apart from the property $`S(\rho |\rho )=0`$, condition 5 up to a set of measure zero and condition 6 only with respect to $`\rho `$.

Both the equality requirements of condition 2 can never be satisfied simultaneously. Indeed, with the choice $`\rho =\sigma =1/dim𝕀`$ we should have both $`S(1/dim𝕀|1/dim𝕀)=0`$ and $`S(1/dim𝕀|1/dim𝕀)=\mathrm{ln}dim`$. Also the condition $`S(\rho |\rho )=0`$ combined with $`S(\rho |\sigma )0`$ is incompatible with concavity of $`S(\rho |\sigma )`$ in $`\rho `$ (condition 6). In fact, let $`\rho =\lambda \rho _1+(1\lambda )\rho _2,\mathrm{\hspace{0.17em}0}<\lambda <1`$. But this gives $`0=S(\rho |\rho )\lambda S(\rho _1|\rho )+(1\lambda )S(\rho _2|\rho )`$. Hence $`S(\rho _1|\rho )=0`$ for all $`\rho _1`$ for which there is $`\lambda >0`$ with $`\lambda \rho _1<\rho `$. This condition is fulfilled for all $`\rho _1`$, whenever $`\rho >0`$ (I owe these observations to H. Narnhofer).

Next let us look at the condition 3, by which we mean the situation where $`\rho `$ and $`\sigma `$ commute such that $`S(\rho )=H(X),S(\sigma )=H(Y)`$ and $`S^\mathrm{?}(\rho |\sigma )=H(X|Y)`$ holds for suitable $`X=X(\rho )`$ and $`Y=Y(\sigma )`$. Also the dependence of $`X(\rho )`$ and $`Y(\sigma )`$ on $`\rho `$ and $`\sigma `$ respectively should be non-trivial w.r.t. their eigenvalues. In particular condition 3 means that the symmetry condition 4 must hold at least when $`\rho `$ and $`\sigma `$ commute. In view of the destruction of quantum coherence when measurements are performed and due to the occurrence of the sum by which $`S^\mathrm{?}(\rho ,\sigma )`$ is defined, it is unclear to the author whether the symmetry condition 4 also should hold for non-commuting $`\rho `$ and $`\sigma `$ (see below , however, a construction of conditional entropy in terms of spectral resolutions of the identity below). It is natural to make the assumption on $`X(\rho )`$, that $`\mu (X_k)=p_k(\rho )`$, see (19). Then it may be shown that the continuity condition and the classical equivalence condition are not compatible. The concavity condition in $`\sigma `$ is at least intuitively desirable since taking convex combinations decreases conditioning, i.e. increases uncertainty, and hence should increase conditional entropy.

We would also like to point out another difference between the classical and the quantum case in the way we have presented it so far. In the classical case the conditioning $`Y`$ is trivial when $`Y=\{\mathrm{\Omega }\}`$, which means no information content and for which we have $`H(Y)=0`$. Within the context of density matrices the only sensible candidate for a trivial conditioning is $`\sigma =1/dim𝕀`$, since this is the density matrix with no information content. Its von Neumann entropy, however, is maximal. Recall that we used this quantum notion of trivial conditioning in our discussion of the inequality $`S(\rho |\sigma )S(\rho )`$ (see also the discussion following (35)). We note that several authors consider von Neumann’s entropy not to be a good generalization of classical entropy (see e.g. , page 141). In fact, in classical theory finer partitions give rise to higher uncertainty and hence to larger classical entropy. This was the reason for the algebraic approach of Connes and Størmer and of Connes, Narnhofer and Thirring, in which a classical finer partitioning corresponds to a larger algebra. In particular the larger the algebra, the larger the entropy and similarly the larger the conditioning algebra the larger the conditional entropy.

We claim, however, that there is a way to reconcile this with von Neumann’s entropy. Indeed, given a quantum system in the state $`\rho `$, the measurements one can perform without disturbing $`\rho `$ are given by the observables (i.e. the self-adjoint operators) in $`𝒜(\rho )`$, which by definition is the $``$-sub-algebra of $``$ consisting of all elements in $``$ which commute with $`\rho `$. In particular $`𝒜(\rho =1/dim𝕀)=`$. In this sense again larger uncertainties correspond to larger algebras. In other words, the larger the entropy the more measurements on can perform without disturbing the system in the given state $`\rho `$. To be more precise, we introduce a partial ordering $``$ on the set of all density matrices (which differs from the one introduced by Uhlmann, see e.g. ). By definition $`\rho \sigma `$ ($`\sigma `$ is more mixed than $`\rho `$), if and only if a) $`\underset{¯}{P}\underset{¯}{Q}`$ and b) $`\mathrm{Tr}\rho Q_j=\mathrm{Tr}\sigma Q_j=\sigma _j\mathrm{Tr}Q_j`$ holds for all $`j`$. It is easy to see that $`\rho \sigma `$ and $`\sigma \tau `$ implies $`\rho \tau `$ and that $`\rho 1/dim𝕀`$ and $`\rho \rho `$ holds for all $`\rho `$. So whenever $`\rho \sigma `$ then condition a) implies $`𝒜(\rho )𝒜(\sigma )`$ and a) and b) combined imply $`S(\rho )S(\sigma )`$ by the concavity of the von Neumann entropy. Note, however, that the correspondence between $`\rho `$ and $`𝒜(\rho )`$ is not one-to-one. In fact, $`𝒜(\rho )`$ only depends on the spectral resolution of the identity $`\underset{¯}{P}=\underset{¯}{P}(\rho )`$ associated to $`\rho `$ and not on the eigenvalues $`\rho _i`$ of $`\rho `$. Indeed, one has $`𝒜(\rho )=E_{\underset{¯}{P}(\rho )}()`$, as one may easily verify.

Returning to our discussion of conditions 1-6, there is a way out, however, if one considers spectral resolutions of the identity $`\underset{¯}{P}`$ instead of density matrices. It works as follows. First observe that the actual choice of the probability space $`\{\mathrm{\Omega },\mu \}`$ for Shannon’s theory is irrelevant. What is relevant are the the sets of non-negative numbers $`\underset{¯}{p}=\{p_\alpha \},\underset{¯}{q}=\{q_\beta \},`$ $`\underset{¯}{p}\underset{¯}{q}=\{p_{\alpha |\beta }\}`$ and $`\underset{¯}{q}\underset{¯}{p}=\{q_{\beta |\alpha }\}`$ subject to the following conditions of which the last one is Bayes rule

$$\underset{\alpha }{}p_\alpha =\underset{\beta }{}q_\beta =1,\underset{\beta }{}p_{\alpha |\beta }q_\beta =p_\alpha ,\underset{\alpha }{}q_{\beta |\alpha }p_\alpha =q_\beta ,p_{\alpha |\beta }q_\beta =q_{\beta |\alpha }p_\alpha .$$
(36)
Note that then

$`{\displaystyle \underset{\alpha }{}}p_{\alpha |\beta }`$ $`=`$ $`{\displaystyle \frac{1}{q_\beta }}{\displaystyle \underset{\alpha }{}}q_{\beta |\alpha }p_\alpha =1`$
$`{\displaystyle \underset{\beta }{}}q_{\beta |\alpha }`$ $`=`$ $`{\displaystyle \frac{1}{p_\alpha }}{\displaystyle \underset{\beta }{}}p_{\alpha |\beta }q_\beta =1.`$
We consider these conditions (36), which mean independence of a particular realization of partitions $`X`$ and $`Y`$ on a probability space, the classical analogue of the relation (10). Setting $`p_{\alpha ,\beta }=p_{\alpha |\beta }q_\beta `$ and $`q_{\beta ,\alpha }=q_{\beta |\alpha }p_\alpha `$, Bayes rule gives $`p_{\alpha ,\beta }=q_{\beta ,\alpha }`$. We will therefore write $`H(X|Y)=H(\underset{¯}{p}|\underset{¯}{q})`$ by a slight abuse of notation since all the data $`\underset{¯}{p},\underset{¯}{q},\underset{¯}{p}\underset{¯}{q}`$ and $`\underset{¯}{q}\underset{¯}{q}`$ in (36) are necessary for a specification of $`H(X|Y)`$. But given these data it makes sense to say that $`\underset{¯}{p}`$ is a consequence of $`\underset{¯}{q}`$ or that $`\underset{¯}{p}`$ and $`\underset{¯}{q}`$ are independent.

Now let $`\tau =1/dim\mathrm{Tr}`$ denote the normalized trace, i.e. $`\tau (𝕀)=1`$. For any two spectral resolutions $`\underset{¯}{P}`$ and $`\underset{¯}{Q}`$ let $`p_i=\tau (P_i),q_j=\tau (Q_j),p_{i|j}=\tau (P_iQ_j)/\tau (Q_j),q_{j|i}=\tau (Q_jP_i)/\tau (P_i)`$. Note that by definition all $`P_i`$ and all $`Q_j`$ are non-zero projections. The conditions (36) are obviously satisfied. We then set $`H(\underset{¯}{P})=S_{cl}(\underset{¯}{p}),H(\underset{¯}{Q})=S_{cl}(\underset{¯}{q})`$ ,such that $`H(\underset{¯}{Q}=\{𝕀\})=\mathrm{ln}dim`$ and finally $`H(\underset{¯}{P}|\underset{¯}{Q})=H(\underset{¯}{p}|\underset{¯}{q})`$, such that $`0H(\underset{¯}{P}|\underset{¯}{Q})H(\underset{¯}{P})`$ as desired. Note that now $`H(\underset{¯}{P}|\underset{¯}{Q})`$ is completely specified by $`\underset{¯}{P}`$ and $`\underset{¯}{Q}`$. Also $`H(\underset{¯}{P},\underset{¯}{Q})=H(\underset{¯}{Q})+H(\underset{¯}{P}|\underset{¯}{Q})`$ is symmetric in $`\underset{¯}{P}`$ and $`\underset{¯}{Q}`$.

It is easy to see that $`\underset{¯}{p}`$ is a consequence of $`\underset{¯}{q}`$ if and only if $`\underset{¯}{Q}\underset{¯}{P}`$ such that $`H(\underset{¯}{P}|\underset{¯}{Q})=0`$ if and only if $`\underset{¯}{Q}\underset{¯}{P}`$. Similarly $`\underset{¯}{p}`$ and $`\underset{¯}{q}`$ are independent if and only if $`\underset{¯}{P}=\{𝕀\}`$ or $`\underset{¯}{Q}=\{𝕀\}`$. Therefore, whenever $`H(\underset{¯}{P})0`$, $`H(\underset{¯}{P}|\underset{¯}{Q})=H(\underset{¯}{P})`$ if and only if $`\underset{¯}{Q}=\{𝕀\}`$, which in this context is the trivial conditioning and for which the entropy is zero in contrast to our construction in terms of density matrices. Finally we set $`AdU\underset{¯}{Q}=\{UQ_iU^1\}`$ for any $`\underset{¯}{Q}`$ and any unitary $`U`$. Then obviously $`H(AdU\underset{¯}{P}|AdU\underset{¯}{Q})=H(\underset{¯}{P}|\underset{¯}{Q})`$ (compare condition 1).

Since the $`P_i`$’s and the $`Q_j`$’s need not commute, this construction is a non-commutative version of Shannon’s conditional entropy in (commutative) classical probability theory. Thus a classical partition $`X`$ is replaced by a spectral resolution of the identity $`\underset{¯}{P}`$, which in turn corresponds to the $``$-algebra $`E_{\underset{¯}{P}}()`$ and which is abelian if and only if each $`P_i`$ is one-dimensional. The choice $`\underset{¯}{Q}=\{𝕀\}`$ giving maximal entropy $`H(\underset{¯}{Q})`$ and maximal conditional entropy $`H(\underset{¯}{P}|\underset{¯}{Q})`$ corresponds to the maximal algebra $`E_{\underset{¯}{Q}=\{𝕀\}}()=`$. Our construction of $`H(\underset{¯}{P}|\underset{¯}{Q})`$ differs from the construction in .

We might have defined the conditional entropy of two density matrices $`\rho `$ and $`\sigma `$ by $`H(\underset{¯}{P}(\rho )|\underset{¯}{Q}(\sigma ))`$. Conditions 1,2 and 4 are then satisfied but not condition 5 and condition 3, since the dependence on the eigenvalues of $`\rho `$ and $`\sigma `$ drops out. We conjecture that condition 6 is also not satisfied.

Acknowledgements: The author would like to thank M. Karowski, H. Narnhofer, M.A. Nielsen, M. Schmidt and E. Størmer for helpful remarks.