  Massive multi-threading in GPU imposes tremendous pressure on memory
subsystems. Due to rapid growth in thread-level parallelism of GPU and slowly
improved peak memory bandwidth, the memory becomes a bottleneck of GPU's
performance and energy efficiency. In this work, we propose an integrated
architectural scheme to optimize the memory accesses and therefore boost the
performance and energy efficiency of GPU. Firstly, we propose a thread batch
enabled memory partitioning (TEMP) to improve GPU memory access parallelism. In
particular, TEMP groups multiple thread blocks that share the same set of pages
into a thread batch and applies a page coloring mechanism to bound each stream
multiprocessor (SM) to the dedicated memory banks. After that, TEMP dispatches
the thread batch to an SM to ensure high-parallel memory-access streaming from
the different thread blocks. Secondly, a thread batch-aware scheduling (TBAS)
scheme is introduced to improve the GPU memory access locality and to reduce
the contention on memory controllers and interconnection networks. Experimental
results show that the integration of TEMP and TBAS can achieve up to 10.3%
performance improvement and 11.3% DRAM energy reduction across diverse GPU
applications. We also evaluate the performance interference of the mixed
CPU+GPU workloads when they are run on a heterogeneous system that employs our
proposed schemes. Our results show that a simple solution can effectively
ensure the efficient execution of both GPU and CPU applications.
