text: Many telepresence robots are equipped with a forward-facing camera for video communication and a downward-facing camera for navigation. In this paper, we propose to stitch videos from the FF-camera with a wide-angle lens and the DF-camera with a fisheye lens for telepresence robots. We aim at providing more compact and efficient visual feedback for the user interface of telepresence robots with user-friendly interactive experiences. To this end, we present a multi-homography-based video stitching method which stitches videos from a wide-angle camera and a fisheye camera. The method consists of video image alignment, seam cutting, and image blending. We directly align the wide-angle video image and the fisheye video image based on the multi-homography alignment without calibration, distortion correction, and unwarping procedures. Thus, we can obtain a stitched video with shape preservation in the non-overlapping regions and alignment in the overlapping area for telepresence. To alleviate ghosting effects caused by moving objects and or moving cameras during telepresence robot driving, an optimal seam is found for aligned video composition, and the optimal seam will be updated in subsequent frames, considering spatial and temporal coherence. The final stitched video is created by image blending based on the optimal seam. We conducted a user study to demonstrate the effectiveness of our method and the superiority of telepresence robots with a stitched video as visual feedback.
  @cite_43: 
  @cite_27: Recent advances in unmanned aerial vehicle (UAV) technologies have made it possible to deploy an aerial video surveillance system to provide an unprecedented aerial perspective for ground monitoring in real time. Multiple UAVs would be required to cover a large target area, and it is difficult for users to visualize the overall situation if they were to receive multiple disjoint video streams. To address this problem, we designed and implemented SkyStitch, a multiple-UAV video surveillance system that provides a single and panoramic video stream to its users by stitching together multiple aerial video streams. SkyStitch addresses two key design challenges: (i) the high computational cost of stitching and (ii) the difficulty of ensuring good stitching quality under dynamic conditions. To improve the speed and quality of video stitching, we incorporate several practical techniques like distributed feature extraction to reduce workload at the ground station, the use of hints from the flight controller to improve stitching efficiency and a Kalman filter-based state estimation model to mitigate jerkiness. Our results show that SkyStitch can achieve a stitching rate that is 4 times faster than existing state-of-the-art methods and also improve perceptual stitching quality. We also show that SkyStitch can be easily implemented using commercial off-the-shelf hardware.
  @cite_31: We conducted high-frame-rate (HFR) video mosaicing for real-time synthesis of a panoramic image by implementing an improved feature-based video mosaicing algorithm on a field-programmable gate array (FPGA)-based high-speed vision platform. In the implementation of the mosaicing algorithm, feature point extraction was accelerated by implementing a parallel processing circuit module for Harris corner detection in the FPGA on the high-speed vision platform. Feature point correspondence matching can be executed for hundreds of selected feature points in the current frame by searching those in the previous frame in their neighbor ranges, assuming that frame-to-frame image displacement becomes considerably smaller in HFR vision. The system we developed can mosaic 512×512 images at 500 fps as a single synthesized image in real time by stitching the images based on their estimated frame-to-frame changes in displacement and orientation. The results of an experiment conducted, in which an outdoor scene was captured using a hand-held camera-head that was quickly moved by hand, verify the performance of our system.
  @cite_16: User generated videos with mobile phone cameras are becoming more and more ubiquitous allowing people to share live content with remote parties, possibly in real-time. However with limited mobile phone capabilities, these videos are usually of small resolution, resulting in a small field of view for acceptable quality. Fortunately with the proliferation of video capture-enabled mobile phones, there is high chance that one or more persons will be shooting the same scene from different views. In this demonstration, we are showing an end-to-end system which receives video streams coming from different mobile phones, time synchronizes the streams and produces a single composite mosaic video, and all of this is done in real-time. The proposed system operates without coordination between users. The system has been tested under various capturing conditions such as indoor, outdoor, day and night conditions.
  @cite_13: This paper presents a parallax-robust video stitching technique for timely synchronized surveillance video. An efficient two-stage video stitching procedure is proposed in this paper to build wide Field-of-View (FOV) videos for surveillance applications. In the stitching model calculation stage, we develop a layered warping algorithm to align the background scenes, which is location-dependent and turned out to be more robust to parallax than the traditional global projective warping methods. On the selective seam updating stage, we propose a change-detection based optimal seam selection approach to avert ghosting and artifacts caused by moving foregrounds. Experimental results demonstrate that our procedure can efficiently stitch multi-view videos into a wide FOV video output without ghosting and noticeable seams.
summary: Besides, some work was designed for real-time processing or time-critical applications. For video surveillance applications, He and Yu @cite_13 employed a background modeling algorithm and a change-detection-based optimal seam selection approach to stitch videos captured by fixed cameras. A Multi-UAV-based video surveillance system, SkyStitch @cite_27 , was designed and implemented for real-time aerial surveillance, employing flight information (e.g., the UAV attitude and GPS location) got from the flight controller as assistance. @cite_31 introduced a real-time video stitching method by implementing and improving a feature-based algorithm on a field-programable gate array (FPGA). Apart from hardware acceleration, EI- @cite_16 developed a real-time method to stitch independent videos streamed by different mobile phones, while @cite_43 stitched several live videos into a @math field of view and spread the stitched video based on GPU.