  Photonic time-varying systems have attracted significant attention owing to
their rich physics and potential opportunities for new and enhanced
functionalities. In this context, the duality of space and time in wave physics
has been particularly fruitful to uncover interesting physical effects in the
temporal domain, such as reflection/refraction at temporal interfaces and
momentum-bandgaps in time crystals. However, the characteristics of the
temporal/frequency dimension, particularly its relation to causality and energy
conservation ($\hbar \omega$ is energy, whereas $\hbar \boldsymbol{k}$ is
momentum), create challenges and constraints that are unique to time-varying
systems and are not present in their spatially varying counterparts. Here, we
overview two key physical aspects of time-varying photonics that have only
received marginal attention so far, namely temporal dispersion and external
power requirements, and explore their implications. We discuss how temporal
dispersion, an inherent property of any causal material, makes the fields
evolve continuously at sharp temporal interfaces and may limit the strength of
fast temporal modulations and of various resulting effects. Furthermore, we
show that changing the refractive index in time always involves large amounts
of energy. We derive power requirements to observe a time-crystal response in
one of the most popular material platforms in time-varying photonics, i.e.,
transparent conducting oxides, and we argue that these effects are almost
always obscured by less exotic nonlinear phenomena. These observations and
findings shed light on the physics and constraints of time-varying photonics,
and may guide the design and implementation of future time-modulated photonic
systems.
