  It is generally observed that the fraction of live lines in shared last-level
caches (SLLC) is very small for chip multiprocessors (CMPs). This can be
tackled using promotion-based replacement policies like re-reference interval
prediction (RRIP) instead of LRU, dead-block predictors, or reuse-based cache
allocation schemes. In GPU systems, similar LLC issues are alleviated using
various cache bypassing techniques. These issues are worsened in heterogeneous
CPU-GPU systems because the two processors have different data access patterns
and frequencies. GPUs generally work on streaming data, but have many more
threads accessing memory as compared to CPUs. As such, most traditional cache
replacement and allocation policies prove ineffective due to the higher number
of cache accesses in GPU applications, resulting in higher allocation for GPU
cache lines, despite their minimal reuse. In this work, we implement the Reuse
Cache approach for heterogeneous CPU-GPU systems. The reuse cache is a
decoupled tag/data SLLC which is designed to only store the data that is being
accessed more than once. This design is based on the observation that most of
the cache lines in the LLC are stored but do not get reused before being
replaced. We find that the reuse cache achieves within 0.5% of the IPC gains of
a statically partitioned LLC, while decreasing the area cost of the LLC by an
average of 40%.
