text: Light field imaging involves capturing both angular and spatial distribution of light; it enables new capabilities, such as post-capture digital refocusing, camera aperture adjustment, perspective shift, and depth estimation. Micro-lens array (MLA)-based light field cameras provide a cost-effective approach to light field imaging. There are two main limitations of MLA-based light field cameras: low spatial resolution and narrow baseline. While low spatial resolution limits the general purpose use and applicability of light field cameras, narrow baseline limits the depth estimation range and accuracy. In this paper, we present a hybrid stereo imaging system that includes a light field camera and a regular camera. The hybrid system addresses both spatial resolution and narrow baseline issues of the MLA-based light field cameras while preserving light field imaging capabilities.
  @cite_5: We propose a novel light field super-resolution framework based on hybrid imaging system, which combines two different imaging mechanisms: conventional imaging and current art-of-the-state imaging — light field imaging. We take advantage of conventional imaging in spatial resolution to make up light field and reconstruct a higher quality light field. In our method, we classify the points of the 3D scene: First, for highlight and occlusion, dictionary learning based interpolation is utilized, Second, for other areas, an improved patch matching algorithm is applied. As shown in experimental results, compared with four methods, which include the art-of-the-state algorithms, our approach is effective.
  @cite_38: Recently, light field cameras have drawn much attraction for their innovative performance in photographic and scientific applications. However, narrow baselines and constrained spatial resolution of current light field cameras impose restrictions on their usability. Therefore, we design a hybrid imaging system containing a light field camera and a high-resolution digital single lens reflex camera, and these two kinds of cameras share the same optical path with a beam splitter so as to achieve the reconstruction of high-resolution light fields. The high-resolution 4D light fields are reconstructed with a phase-based perspective variation strategy. First, we apply complex steerable pyramid decomposition on the high-resolution image from the digital single lens reflex camera. Then, we perform phase-based perspective-shift processing with the disparity value, which is extracted from the upsampled light field depth map, to create high-resolution synthetic light field images. High-resolution digital refocused images and high-resolution depth maps can be generated in this way. Furthermore, controlling the magnitude of the perspective shift enables us to change the depth of field rendering in the digital refocused images. We show several experimental results to demonstrate the effectiveness of our approach.
  @cite_8: Current light field (LF) cameras provide low spatial resolution and limited depth-of-field (DOF) control when compared to traditional digital SLR (DSLR) cameras. We show that a hybrid imaging system consisting of a standard LF camera and a high-resolution standard camera enables (a) achieve high-resolution digital refocusing, (b) better DOF control than LF cameras, and (c) render graceful high-resolution viewpoint variations, all of which were previously unachievable. We propose a simple patch-based algorithm to super-resolve the low-resolution views of the light field using the high-resolution patches captured using a high-resolution SLR camera. The algorithm does not require the LF camera and the DSLR to be co-located or for any calibration information regarding the two imaging systems. We build an example prototype using a Lytro camera (380×380 pixel spatial resolution) and a 18 megapixel (MP) Canon DSLR camera to generate a light field with 11 MP resolution (9× super-resolution) and about 1 over 9 th of the DOF of the Lytro camera. We show several experimental results on challenging scenes containing occlusions, specularities and complex non-lambertian materials, demonstrating the effectiveness of our approach.
summary: All single-sensor light field imaging systems are fundamentally limited by the spatial-angular resolution trade-off, and the above-mentioned restoration methods have performance limitations in addition to the computational costs. Another approach for improving spatial resolution is to use a hybrid two camera system, including a light field camera and a high-resolution camera, and merge the images to improve spatial resolution @cite_8 @cite_5 @cite_38 . Dictionary-learning based techniques are adopted @cite_8 @cite_5 in this problem as well: High-resolution image patches from the regular camera are extracted and stored as a high-resolution patch dictionary. These high-resolution patches are downsampled; and from the downsampled patches, low-resolution features are extracted to form a low-resolution patch dictionary. During super-resolution reconstruction, a low-resolution image patch is enhanced through determining (based on feature matching) and using the corresponding high-resolution patches in the dictionary. In @cite_38 , high-resolution image is decomposed with complex steerable pyramid filters; the depth map from the light field is upsampled using joint bilateral upsampling; perspective shift amounts are estimated from the upsampled depth map, and these shift amounts are used to modify the phase of the decomposed high-resolution image; with the modified phases, pyramid reconstruction is applied to obtain high-resolution light field.