  Nonlinearity mitigation using digital signal processing has been shown to
increase the achievable data rates of optical fiber transmission links. One
especially effective technique is digital back propagation (DBP), an algorithm
capable of simultaneously compensating for linear and nonlinear channel
distortions. The most significant barrier to implementing this technique,
however, is its high computational complexity. In recent years, there have been
several proposed alternatives to DBP with reduced computational complexity,
although such techniques have not demonstrated performance benefits
commensurate with the complexity of implementation. In order to fully
characterize the computational requirements of DBP, there is a need to model
the algorithm behavior when constrained to the logic used in a digital coherent
receiver. Such a model allows for the analysis of any signal recovery algorithm
in terms of true hardware complexity which, crucially, includes the bit-depth
of the multiplication operation. With a limited bit depth, there is
quantization noise, introduced with each arithmetic operation, and it can no
longer be assumed that the conventional DBP algorithm will outperform its low
complexity alternatives. In this work, DBP and a single nonlinear step DBP
implementation, the \textit{Enhanced Split Step Fourier} method (ESSFM), were
compared with linear equalization using a generic software model of fixed point
hardware. The requirements of bit depth and fast Fourier transform (FFT) size
are discussed to examine the optimal operating regimes for these two schemes of
digital nonlinearity compensation. For a 1000 km transmission system, it was
found that (assuming an optimized FFT size), in terms of SNR, the ESSFM
algorithm outperformed the conventional DBP for all hardware resolutions up to
13 bits.
