IEEE Consumer Electronics Magazine - May/June 2022 - 80

Special Section on ISVLSI-2020
Figure 1. Conceptual view of an NoC based multicore processor. A packet in NoC is divided into multiple
smaller units called flits. A control packet consists of a single head flit (H), whereas a data packet consists of a
head flit followed by multiple body flits and ended by a tail flit (H, B0, B1, T).
keeps increasing.1 Such devices rely on networkon-chip
(NoC) for an efficient, reliable, and scalable
communication infrastructure. Conceptual
illustration of an NoC-based multicore processor
used in such devices is shown in Figure 1.
Some of the most popular multicore-based
consumer electronics like smartphones, tablets,
personal computers, autonomous vehicles, etc.,
have up to 32 processing cores.2 With the ongoing
increase in data-driven applications that run
on these devices, future consumer electronics
may have even higher number of processing
cores. When applications run, they expect minimum
data access latency, where on-chip caches
play a major role. However, due to the increasing
core counts, limited on-chip area, and the associated
cost, most of these devices have only two
levels of on-chip caches, as shown in Figure 1. In
case of an L1 cache miss, the requested block is
brought from the next level of memory (L2
cache). Similarly, during an L2 cache miss, the
requested block is brought from the off-chip
memory. The time spent in bringing a requested
block into the L1 cache is called miss penalty. As
the entire communication between L1, L2, and
off-chip memory is done through the underlying
NoC, it plays a significant role in the miss penalty.
Hence, NoC in traditional multicore processors
is found to be responsible for 60%-75%
of miss penalty and, thus, has a significant say
on performance.3
Due to limited capacity, servicing L1 cache
miss beyond a point requires evicting a valid
block. If an evicted L1 cache block is clean, it is
discarded, whereas if the block is dirty (modified),
it is sent for write-back. The evicted, dirty
L1 cache block enters NoC (as a packet) through
80
the local router, which connects a core to the
underlying NoC (refer Figure 1). In-transit packets
are stored in router buffers during their routing
and arbitration decisions. Hence, the packet
carrying the dirty block traverses multiple
routers to finally reach its destination L2 cache
bank for write-back. In applications that exhibit
decent temporal locality of reference, if a
recently evicted L1 cache block is re-referenced,
it needs to be brought back from the L2 cache
bank. In the worst case, when the re-referenced
block is not present in the L2 cache bank (L2
miss), it is brought from off-chip memory. If the
recently evicted block could have stayed in L1
cache, a little longer until its re-reference, these
kinds of miss penalty could be avoided. Such
unfortunate miss penalties delay applications
and affect system performance. Increasing the
size of L1 cache may temporarily work but is not
feasible due to its impact on L1 cache hit time,
on-chip area, and power constraints of consumer
electronics.
Due to very high L1 cache hit rate of around
90%-95%, L1 cache misses are sporadic. Hence,
an average packet injection rate is only around
5%, and the NoC resources remain severely
underutilized.4 With a low packet injection rate,
buffers in NoC routers are underutilized. Since
buffers are both area and power-hungry, promising
alternatives like bufferless and minimally
buffered NoC routers are proposed.5 There are
also solutions like run-time power gating with
look-ahead routing that optimizes power consumption
of routers.6 However, these alternatives
fail to provide the necessary performance
bandwidth during on-chip network congestion.
Buffered NoC routers provide lower latency and
IEEE Consumer Electronics Magazine

IEEE Consumer Electronics Magazine - May/June 2022

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May/June 2022