  Diphoton searches at high invariant mass are an integral part of the
experimental high energy frontier. Using the analyses of the 750\,GeV diphoton
resonance as a case study, we examine the methodology currently employed by the
experimental analyses in estimating the dominant standard model backgrounds. We
assess the dependence of the significance associated with the excess on the
background modeling. In particular, we show that close to the high energy tails
of the distributions, estimates of the jet faking backgrounds relying on
functional extrapolations or Monte Carlo estimates of the challenging
photon-jet contributions introduce a large uncertainty. Analyses with loose
photon cuts, low photon $p_T$ cuts and those susceptible to high photon
rapidity regions are especially affected. Given that diphoton-based searches
beyond 1\,TeV are highly motivated as discovery modes, these considerations are
relevant and applicable in the future.
  We first consider a physics-driven deformation of the shape of the diphoton
faking photon-jet spectrum by next-to-leading order effects combined with a
rapidity and transverse momentum dependent fake rate. We show that the
resulting local significance of the excess is reduced due to such a
deformation. Then using a simple, but more general, ansatz to modify the
contribution of the jet faking backgrounds at high invariant masses but keeping
the inclusive and differential sample purities within experimental uncertainty
estimates, we demonstrate that the originally reported local 750\,GeV excess
significances could have been overestimated by more than one standard
deviation. We furthermore cross-check our analysis by comparing fit results
based on the smaller 2015 and the larger 2016 LHC datasets. Finally we employ
our methodology on the complete 13\,TeV LHC dataset to asses the systematics
involved in the current diphoton searches beyond the TeV region.
