  Block RAMs (BRAMs) are the storage houses of FPGAs, providing extensive
on-chip memory bandwidth to the compute units implemented using Logic Blocks
(LBs) and Digital Signal Processing (DSP) slices. We propose modifying BRAMs to
convert them to CoMeFa (Compute-In-Memory Blocks for FPGAs) RAMs. These RAMs
provide highly-parallel compute-in-memory by combining computation and storage
capabilities in one block. CoMeFa RAMs utilize the true dual port nature of
FPGA BRAMs and contain multiple programmable single-bit bit-serial processing
elements. CoMeFa RAMs can be used to compute in any precision, which is
extremely important for evolving applications like Deep Learning. Adding CoMeFa
RAMs to FPGAs significantly increases their compute density. We explore and
propose two architectures of these RAMs: CoMeFa-D (optimized for delay) and
CoMeFa-A (optimized for area). Compared to existing proposals, CoMeFa RAMs do
not require changing the underlying SRAM technology like simultaneously
activating multiple rows on the same port, and are practical to implement.
CoMeFa RAMs are versatile blocks that find applications in numerous diverse
parallel applications like Deep Learning, signal processing, databases, etc. By
augmenting an Intel Arria-10-like FPGA with CoMeFa-D (CoMeFa-A) RAMs at the
cost of 3.8% (1.2%) area, and with algorithmic improvements and efficient
mapping, we observe a geomean speedup of 2.5x (1.8x), across several
representative benchmarks. Replacing all or some BRAMs with CoMeFa RAMs in
FPGAs can make them better accelerators of modern compute-intensive workloads.
