

![0_image_0.png](0_image_0.png)

| Model   | Test      | Average   | SAT    |
|---------|-----------|-----------|--------|
| error   | kJf (x)kF |           |        |
| CAE     | 1.14      | 0.73 10−4 | 86.36% |
| DAE-g   | 1.18      | 0.86 10−4 | 17.77% |

![0_image_1.png](0_image_1.png)

CAE 1.14 0.73 10−4 86.36% DAE-g 1.18 0.86 10−4 17.77% RBM-binary 1.30 2.50 10−4 78.59%
DAE-b 1.57 7.87 10−4 68.19%
AE+wd 1.68 5.00 10−4 12.97%
AE 1.78 17.5 10−4 49.90%

CAE 47.86 2.40 10−5 85,65%
DAE-b 49.03 4.85 10−5 80,66%
DAE-g 54.81 4.94 10−5 19,90%
AE+wd 55.03 34.9 10−5 23,04%
AE 55.47 44.9 10−5 22,57%
Finally, we evaluate the classification performance of our algorithm with a linear classifier. The preprocessing steps are applied convolutionnaly with a stride equal to one to get nhid feature maps of size 25 × 25. Features are sumpooled together over the quadrants of the feature maps. By applying this coarse dimensionality reduction technique, we obtain features of dimension 4nhid. We fed a linear L2-regularized SVM with these features and reported the test classification accuracy in TAB XX. L2 regularizer was chosen using a 5-fold cross validation.

## 5.2 Closer Examination Of The Contraction

To better understand the feature extractor produced by each algorithm, in terms of their contractive properties, we used the following analytical tools:
What happens locally: looking at the singular values of the Jacobian. A high dimensional Jacobian contains directional information: the amount of contraction is generally not the same in all directions. This can be examined by performing a singular value decomposition of Jf . We computed the average singular value spectrum of the Jacobian over the validation set for the above models. Results are shown in Figure 2 and will be discussed in section 5.3.

What happens further away: contraction curves. The Frobenius norm of the Jacobian at some point x measures the contraction of the mapping locally at that point. Intuitively the contraction induced by the proposed penalty term can be measured beyond the immediate training examples, by the ratio of the distances between two points in their original (input) space and their distance once mapped in the feature space. We call this measure contraction ratio. In the limit where the variation in the input space is infinitesimal, this