IEEE Circuits and Systems Magazine - Q2 2020 - 41

Single-pass training leads to low accuracy. To improve this, iterative training
might be one efficient solution. However, a lack of controllability of training
iterations in HD classification may result in slow training or divergence.
CompHD is suitable for low-power IoT devices to achieve
higher efficiency with a comparable accuracy.
5) Adaptive Efficient Training for HD Computing
Single-pass training leads to low accuracy. To improve
this, iterative training might be one efficient solution.
However, a lack of controllability of training iterations
in HD classification may result in slow training or divergence. To solve this training issue, [54] proposes a retraining approach, AdaptHD.
The basic idea is illustrated as follows: 1). Conduct
the initial training by using binary hypervectors to generate the non-binary class hypervectors. 2). Retrain the
class hypervectors by looking at the similarity of each
trained class hypervectors (C) with the training hypervector (H ). Update the model using Eq. (18) if the
current training -hypervector leads to a misclassification error. Otherwise there is no change. For example,
there is a mismatch if H i is supposed to belong to C correct
but is classified as C wrong, where C correct and C wrong denote different class hypervectors and H i represents the
ith training hypervector. 3). After convergence, which
means the last three iterations of retraining show less
than 0.1% accuracy change, then binarize the final
trained model for inference.
	

'

C wrong = C wrong - aH i,
(18)
C correct = C correct + aH i .

Cjd

Cjd-D

Cjd

Insights are gained by their results: 1). Small a needs
more iterations to get the near best accuracy. The smooth
curve indicates small a is better for fine-tuning. 2). Large
a gets to the near best accuracy much faster, but its high
fluctuation may lead to divergence. Based on these two
findings, AdaptHD uses large a first to get the near best
accuracy faster, then changes to smaller a for fine-tuning
until convergence. This is similar to adjusting the step
size in the normalized least mean square (LMS) algorithm
[55]. AdaptHD offers three types of adaptive methods:
■■ Iteration-dependent AdaptHD. The change of value
a depends on iterations. In the beginning, a starts
with a large a max . The learning rate a changes
based on the average error rate in the previous b
iterations. If error rate decreases, indicating convergence, then use smaller a; otherwise, increase a.
■■ Data-dependent AdaptHD. The value a differs in a
certain iteration for all data points, and it changes
depending on the similarity of the data point with
the class hypervectors. Large distance uses large
a to reduce the difference.
■■ Hybrid AdaptHD. Combining the two models, hybrid
AdaptHD can achieve high accuracy as iterationdependent AdaptHD and fast speedup as datadependent AdaptHD.
The evaluation shows that, compared to the existing
HD algorithm, their hybrid AdaptHD can achieve 6.9#
speedup and 6.3# energy-efficiency improvement.

Cj 1

hd-D

hd

hd

h1

First Segment

S-th Segment

First Segment

S-th Segment

Cjd

Cjd-D

Cjd

Cj 1

hd

hd-D

hd

h1

-1

+1

+1

-1

-1

+1

+1

-1

PS

P1
Compressed
Model C′

Cjd′

Cj ′1
(a)

PS

P1
Compressed
Query Q′

hD′

h1′
(b)

Figure 12. CompHD for (a) an HD model and (b) a query data [51].

SECOND QUARTER 2020 		

IEEE CIRCUITS AND SYSTEMS MAGAZINE	

41
IEEE Circuits and Systems Magazine - Q2 2020

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2020
