text: In multiview video systems, multiple cameras generally acquire the same scene from different perspectives, such that users have the possibility to select their preferred viewpoint. This results in large amounts of highly redundant data, which needs to be properly handled during encoding and transmission over resource-constrained channels. In this work, we study coding and transmission strategies in multicamera systems, where correlated sources send data through a bottleneck channel to a central server, which eventually transmits views to different interactive users. We propose a dynamic correlation-aware packet scheduling optimization under delay, bandwidth, and interactivity constraints. The optimization relies both on a novel rate-distortion model, which captures the importance of each view in the 3D scene reconstruction, and on an objective function that optimizes resources based on a client navigation model. The latter takes into account the distortion experienced by interactive clients as well as the distortion variations that might be observed by clients during multiview navigation. We solve the scheduling problem with a novel trellis-based solution, which permits to formally decompose the multivariate optimization problem thereby significantly reducing the computation complexity. Simulation results show the gain of the proposed algorithm compared to baseline scheduling policies. More in details, we show the gain offered by our dynamic scheduling policy compared to static camera allocation strategies and to schemes with constant coding strategies. Finally, we show that the best scheduling policy consistently adapts to the most likely user navigation path and that it minimizes distortion variations that can be very disturbing for users in traditional navigation systems.
  @cite_26: Multiview video is well known to involve interactions with audience and offer better view experience than conventional single-view video. The interactive nature of multiview video has made the service very delay-sensitive and thus imposes great challenges for providing multiview streaming service in Internet. Currently, almost none of existing works have addressed the delay issue in multiview streaming. To ensure the quality of service and offer preferable view experience for users, we propose a novel streaming framework to provide delay-guaranteed service for interactive multiview video. The basic tradeoff between consumed server bandwidth and the required delay is carefully studied. We leverage the features of MVC to keep the server bandwidth costs low while guarantee the delay constraint. In addition, we introduce a neighbor-assisted view switching scheme: peer's neighbors are involved in its switching process and their upload bandwidth resources are effectively utilized to reduce the bandwidth costs on server. Simulation results show that our proposed framework meet the delay-guaranteed requirement with restrained server bandwidth costs while maintaining scalability and resilience to users dynamics.
  @cite_8: As a recently emerging service, multiview video provides a new viewing experience with a high degree of freedom. However, due to the huge data amounts transferred, multiview video's delivery remains a daunting challenge. In this paper, we propose a multiview video-streaming system based on IP multicast. It can support a large number of users while still maintaining a high degree of interactivity and low bandwidth consumption. Based on a careful user study, we have developed two schemes: one is for automatic delivery and the other for on-demand delivery. In automatic delivery, a server periodically multicasts special effect snapshots at a certain time interval. In on-demand delivery, the server delivers the snapshots based on distribution of user requests. We conducted extensive experiments and user-experience studies to evaluate the proposed system's performance, and found that it provides satisfying multiview video service for users on a large scale.
  @cite_28: We investigate coding tools for interactive multiview streaming (IMVS), where clients interactively request desired views for successive video frames, and in response the server sends the appropriate pre-compressed video data to the clients. Solution based on using only I-frames to support view switching would incur high transmission cost, while for that based on using only P-frames to encode every possible traversal, although it can minimize transmission cost, prohibitive server's storage may be required. Therefore, efficient solutions for IMVS need to consider the trade-off between transmission and storage cost. In this paper, we study the potential use of distributed source coding (DSC) in IMVS. Specifically, we propose two DSC constructions that could achieve good transmission-storage trade-offs. Central to these constructions is a method that can efficiently encode the least significant bits (LSB) of a frame to be decoded, leading to competitive storage and transmission requirements. Experiment results demonstrate these constructions compare favorably to existing tools, and could be valuable for interactive multiview streaming.
  @cite_21: The traffic of multi-view video is several times larger than traditional multimedia, which brings much more increment in the bandwidth requirement. To overcome this problem, Simulcast and MVC (Multi-view Video Coding) can be used to decrease the traffic for multi-view video. However, although Simulcast removes redundant information within each view, it still contains a large amount of inter-view redundant information. MVC further removes the correlation between the views, but all the views should be encoded and transmitted. So the bit-rates are still too high in these schemes. In this paper, in order to reduce the bit-rate for multi-view video transmission, we analyze the user's motion which is classified into random access model and successive motion model. We find that in the successive motion model only part of frames are possible to be displayed in a period of time. According to the analytical result, we propose a user dependent scheme to decrease the bit-rate of multi-view video transmission. The proposed scheme only encodes and transmits those frames that are possible to be displayed according to the user's motion. Further, in order to support the proposed scheme, we also propose a prediction structure as a substitute of the prediction structure in MVC. Performance evaluation proves that this proposed scheme reduces the average bit-rate for the transmission of multi-view video.
  @cite_32: This paper presents a rate-distortion (RD) optimized interactive streaming method for multiview video pre-compressed by H.264 Joint Multiview Video Model (JMVM). In the proposed method, multiple encodings are first used to facilitate the flexible server-client interaction. Second, a RD-optimized scheduling strategy is provided to guarantee the optimal view-dependent delivery of multiview video. In the RD-optimized scheduling strategy, a distortion model is proposed to estimate the expected end-to-end distortion by accounting for both coding and packet-loss-induced distortions, as well as rendering-induced distortion. With the end-to-end distortion model, the server can select the optimal encoding combination for transmission. Experimental results demonstrate that the proposed method can achieve a significant end-to-end RD performance improvement over the selective streaming methods with simulcast coding or scalable multiview coding. In addition, it has better error-resilience performance to combat with packet-losses over the Internet protocol (IP) networks.
  @cite_3: 
  @cite_6: 
  @cite_24: 
  @cite_19: We present a novel client-driven multiview video streaming system that allows a user to watch 3D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his her head position. The user's head position is tracked and predicted into the future to select the views that best match the user's current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be prefetched in order to account for delays due to network transport and stream switching. The system allocates more bandwidth to the selected views in order to render the current viewing angle. Highly compressed, lower quality versions of some other views are also prefetched for concealment if the current user viewpoint differs from the predicted viewpoint. An objective measure based on the abruptness of the head movements and delays in the system is introduced to determine the number of additional lower quality views to be prefetched. The proposed system makes use of multiview coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compression efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions.
  @cite_0: We propose to stream multi-view video over a multi-tree peer- to-peer (P2P) network using the NUEPMuT protocol. Each view of the multi-view video is streamed over an independent P2P streaming tree and each peer only contributes upload capacity in a single tree, in order to limit the adverse effects of ungraceful peer departures. Additionally, we introduce a quick join procedure to reduce the start-up delay for the first data packet after a join request. Continuity index and decoded video quality performance for simulcast and MVC encoding in a large topology under different settings are reported, in addition to the improvements achieved by the quick join procedure.
  @cite_5: In multiview video services, multiple cameras acquire the same scene from different perspectives, which results in correlated video streams. This generates large amounts of highly redundant data, which need to be properly handled during encoding and transmission of the multi-view data. In this work, we study coding and transmission strategies in multicamera sets, where correlated sources need to be sent to a central server through a bottleneck channel, and eventually delivered to interactive clients. We propose a dynamic correlation-aware packet scheduling optimization under delay, bandwidth, and interactivity constraints. A novel trellis-based solution permits to formally decompose the multivariate optimization problem, thereby significantly reducing the computation complexity. Simulation results show the gain of the proposed algorithm compared to baseline scheduling policies.
  @cite_31: 
summary: Some prior studies address the problem of providing to users interactivity in selecting views, while saving on transmitted bandwidth and view-switching delay @cite_19 @cite_24 @cite_3 @cite_31 @cite_0 @cite_26 . The work in @cite_6 is mainly focused on coding views with a minimum level of redundancy in order to simplify the view switching, and the works in @cite_32 @cite_21 optimize the selection of views to be encoded and transmitted based on the user interest. The authors in @cite_0 @cite_8 investigate the transmission of multiview video coded streams on P2P networks and IP multicast, respectively. These works mainly focus on the coding optimization proposed as an a priori defined solution to provide interactive access to the different views @cite_28 . In our work, we rather dynamically optimize the coding modes and the scheduling of video frames for interactive multiview navigation. We extend our preliminary work in @cite_5 to include users' interactivity in the scheduling optimization. This allows the system to dynamically adapt the transmission of the coded frames to various system's dynamics and to outperform the above transmission policies of a priori encoded frames.