  What can we learn about a scene by watching it for months or years? A video
recorded over a long timespan will depict interesting phenomena at multiple
timescales, but identifying and viewing them presents a challenge. The video is
too long to watch in full, and some occurrences are too slow to experience in
real-time, such as glacial retreat. Timelapse videography is a common approach
to summarizing long videos and visualizing slow timescales. However, a
timelapse is limited to a single chosen temporal frequency, and often appears
flickery due to aliasing and temporal discontinuities between frames. In this
paper, we propose Video Temporal Pyramids, a technique that addresses these
limitations and expands the possibilities for visualizing the passage of time.
Inspired by spatial image pyramids from computer vision, we developed an
algorithm that builds video pyramids in the temporal domain. Each level of a
Video Temporal Pyramid visualizes a different timescale; for instance, videos
from the monthly timescale are usually good for visualizing seasonal changes,
while videos from the one-minute timescale are best for visualizing sunrise or
the movement of clouds across the sky. To help explore the different pyramid
levels, we also propose a Video Spectrogram to visualize the amount of activity
across the entire pyramid, providing a holistic overview of the scene dynamics
and the ability to explore and discover phenomena across time and timescales.
To demonstrate our approach, we have built Video Temporal Pyramids from ten
outdoor scenes, each containing months or years of data. We compare Video
Temporal Pyramid layers to naive timelapse and find that our pyramids enable
alias-free viewing of longer-term changes. We also demonstrate that the Video
Spectrogram facilitates exploration and discovery of phenomena across pyramid
levels, by enabling both overview and detail-focused perspectives.
