text: The goal of this paper is to compare surface-based and volumetric 3D object shape representations, as well as viewer-centered and object-centered reference frames for single-view 3D shape prediction. We propose a new algorithm for predicting depth maps from multiple viewpoints, with a single depth or RGB image as input. By modifying the network and the way models are evaluated, we can directly compare the merits of voxels vs. surfaces and viewer-centered vs. object-centered for familiar vs. unfamiliar objects, as predicted from RGB or depth images. Among our findings, we show that surface-based methods outperform voxel representations for objects from novel classes and produce higher resolution outputs. We also find that using viewer-centered coordinates is advantageous for novel objects, while object-centered representations are better for more familiar objects. Interestingly, the coordinate frame significantly affects the shape representation learned, with object-centered placing more importance on implicitly recognizing the object category and viewer-centered producing shape representations with less dependence on category recognition.
  @cite_29: How do people recognize an object in different orientations? One theory is that the visual system describes the object relative to a reference frame centered on the object, resulting in a representation that is invariant across orientations. Chronometric data show that this is true only when an object can be identified uniquely by the arrangement of its parts along a single dimension. When an object can only be distinguished by an arrangement of its parts along more than one dimension, people mentally rotate it to a familiar orientation. This finding suggests that the human visual reference frame is tied to egocentric coordinates.
  @cite_11: Although most recent research on visual object recognition suggests that shape is represented in a viewer-centered reference frame, Tarr and Pinker (Psychological Science, 1990) propose that object-centered representations exist, but can code the spatial relations among the parts of an object along only one dimension. This hypothesis was formulated on the basis of research with novel abstract shapes. We tested and confirmed the generality of this hypothesis for the recognition of realistic line drawings of familiar objects. The results also allow us to rule out previous hypotheses concerning the shift from orientation dependence to orientation independence in the recognition of repeatedly viewed drawings.
summary: In experiments on 2D symbols, Tarr and Pinker @cite_29 found that human perception is largely tied to viewer-centered coordinates; this was confirmed by McMullen and Farah @cite_11 for line drawings, who also found that object-centered coordinates seem to play more of a role for familiar exemplars. Note that in the human vision literature, viewer-centered'' usually means that the object shape is represented as a set of images in the viewer's coordinate frame, and object-centered'' usually means a volumetric shape is represented in the object's coordinate frame. In our work, we consider both the shape representation (volumetric or surface) and coordinate frame (viewer or object) as separate design choices. We do not claim our computational approach has any similarity to human visual processing, but it is interesting to see that in our experiments with 3D objects, we also find a preference for object-centered coordinates for familiar exemplars (i.e., novel view of known object) and for viewer-centered coordinates in other cases.