  At the pinnacle of computational imaging is the co-optimization of camera and
algorithm. This, however, is not the only form of computational imaging. In
problems such as imaging through adverse weather, the bigger challenge is how
to accurately simulate the forward degradation process so that we can
synthesize data to train reconstruction models and/or integrating the forward
model as part of the reconstruction algorithm. This article introduces the
concept of computational image formation (CIF). Compared to the standard
inverse problems where the goal is to recover the latent image $x$ from the
observation $y = G(x)$, CIF shifts the focus to designing an approximate
mapping $H$ such that $H \approx G$ while giving a good image reconstruction
result. The word "computational" highlights the fact that the image formation
is now replaced by a numerical simulator. While matching the mother nature
remains an important goal, CIF pays even greater attention on strategically
choosing an $H$ so that the reconstruction performance is maximized.
  The goal of this article is to conceptualize the idea of CIF by elaborating
on its meaning and implications. The first part of the article is a discussion
on the four attributes of a CIF simulator: accurate enough to mimic $G$, fast
enough to be integrated as part of the reconstruction, provides a well-posed
inverse problem when plugged into the reconstruction, and differentiable to
allow backpropagation. The second part of the article is a detailed case study
based on imaging through atmospheric turbulence. A plethora of simulators, old
and new ones, are discussed. The third part of the article is a collection of
other examples that fall into the category of CIF, including imaging through
bad weather, dynamic vision sensors, and differentiable optics. Finally,
thoughts about the future direction and recommendations to the community are
shared.
