IEEE Consumer Electronics Magazine - September/October 2019 - 48

Data Transfer in Multicore CE

Figure 1. Results of nn bypassing LLC.

applications make up a mix of workload. When
the IPC ratio is greater than 1, it indicates that
the application performance is improved. Otherwise, if the IPC ratio is less than 1, this means
that the application performance is degraded.
When the LLC is only allocated to the CPUs,
the IPC of the CPU application is significantly
improved. For the GPU program nn, bypassing
the LLC and directly accessing the memory significantly reduces its performance.
We choose a different GPU application backprop and evaluate the performance impact of
bypass of this benchmark with other CPU applications. We observe that the CPU applications'
performance is greatly improved when running
with backprop. However, compared with nn, in
most cases, the performance of backprop is less
impacted.
The bypass technique improves the performance of the CPU applications, but it has positive or negative effects on the performance of
the GPU applications. To improve overall performance, simply limiting the GPU's access to the
LLC is not able to achieve the best result.

Bypass-Based Shared Cache Management
We propose an enhanced bypass-based shared
cache management strategy that takes into
account the characteristics of different programs
and dynamically monitors each program's performance metrics during program execution (see
Figure 2). It dynamically decides whether to limit
LLC accesses initiated by the GPU cores based on
some indicator values of the application's runtime behavior. When the number of concurrent
threads of a GPU core is large enough and the GPU
application is not sensitive to the LLC, the policy
selectively bypasses the LLC to access the memory directly. For cache-sensitive CPU programs,

Figure 2. Dynamic flowchart.

when the GPU application directly accesses the
main memory, such CPU programs access to
larger cache space to reduce their cache miss rate.
Cache sensitivity indicates the extent to
which application performance can benefit from
increased cache capacity. We dynamically monitor a program's cache sensitivity by collecting
the memory access behavior information of the
CPU applications and the GPU applications
periodically.
To estimate the impact of the bypass on the
performance of GPU applications more accurately, two GPU cores (sp00 and sp01) are
selected to allow core sp00 to access LLC and
core sp01 to access memory. We count the number of execution instructions (sp00_inst and
sp01_inst) for sp00 and sp01 in a time interval,
respectively, and based on a large number
of experiments, we set a threshold called
Threshold_warp to be 40. If the ratio of sp00_inst
and sp01_inst is greater than this threshold, the
GPU application is determined to be sensitive to
cache, otherwise, it is insensitive.
In our approach, when the CPU and GPU applications send requests to the LLC, we need to first
decide the type of message source. Then,

IEEE Consumer Electronics Magazine

IEEE Consumer Electronics Magazine - September/October 2019

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - September/October 2019