Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Single Image 3D Interpreter Network   Understanding 3D object structure from a single image is an important but
difficult task in computer vision, mostly due to the lack of 3D object
annotations in real images. Previous work tackles this problem by either
solving an optimization task given 2D keypoint positions, or training on
synthetic data with ground truth 3D information. In this work, we propose 3D
INterpreter Network (3D-INN), an end-to-end framework which sequentially
estimates 2D keypoint heatmaps and 3D object structure, trained on both real
2D-annotated images and synthetic 3D data. This is made possible mainly by two
technical innovations. First, we propose a Projection Layer, which projects
estimated 3D structure to 2D space, so that 3D-INN can be trained to predict 3D
structural parameters supervised by 2D annotations on real images. Second,
heatmaps of keypoints serve as an intermediate representation connecting real
and synthetic data, enabling 3D-INN to benefit from the variation and abundance
of synthetic 3D objects, without suffering from the difference between the
statistics of real and synthesized images due to imperfect rendering. The
network achieves state-of-the-art performance on both 2D keypoint estimation
and 3D structure recovery. We also show that the recovered 3D information can
be used in other vision applications, such as 3D rendering and image retrieval.

29
The categories of this paper are: funct-an
0
The categories of this paper are: hep-ph
25
The categories of this paper are: patt-sol
4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
7
The categories of this paper are: hep-th
36
The categories of this paper are: cmp-lg
10
The categories of this paper are: q-bio
26
The categories of this paper are: bayes-an
37
The categories of this paper are: supr-con
11
The categories of this paper are: hep-lat
14
The categories of this paper are: math-ph
35
The categories of this paper are: mtrl-th
22
The categories of this paper are: chao-dyn
33
The categories of this paper are: plasm-ph