Instrumentation & Measurement Magazine 26-2 - 21

Fig. 1. The architecture of Dynamic vision transformer using hybrid window attention (DHT).
maintains the resolution of the tensor, it performs self-attention
locally by dividing patch markers into non-overlapping
windows, and then calculates local self-attention only within
each window [14].
Normalization Methods
Normalization techniques have been widely used in computer
vision tasks [21]. BN [22] solves the internal covariate
bias problem by including normalization as part of the model
architecture and performing normalization for each training
small batch. The LN [23] is very effective in stabilizing the
hidden state dynamics in recurrent networks, reducing the
training time of feed forward neural networks. IN demonstrates
that a small change in the stylized architecture leads to
significant quality improvements in the generated images [24].
lN outperforms the above normalizers in transformers and
has dominated a variety of transformer-based models [21].
However, we found that the ViT models with LN could not effectively
encode local information in an image. To resolve this
issue, we propose a new normalizer named DTN, which can
April 2023
capture both the global information and the local features of
defects.
Proposed Approach
Network Overview
The highlight of our model is the use of hybrid window attention
and DTN normalization method to build a dynamic visual
transformer. As shown in Fig. 1, a 4×4 non-overlapping convolutional
embedding is first performed on the input image
of shape H×W×3, followed by a normalization layer to obtain
a patch embedding of H/4×W/4×C. The aim is to construct
a hierarchical pyramid of features, with a backbone consisting
of four stages in progressively increasing steps. Between
the two stages, we downsample the feature map using a patch
merging operation that doubles the number of channels while
halving the space size. After four stages which contain a fixed
size number of blocks, the feature maps are pooled equally to
the fully-connected layer, and finally the industrial defect image
classification results are obtained. In particular, in the DHT
IEEE Instrumentation & Measurement Magazine
21

Instrumentation & Measurement Magazine 26-2

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 26-2

Instrumentation & Measurement Magazine 26-2 - Cover1
Instrumentation & Measurement Magazine 26-2 - Cover2
Instrumentation & Measurement Magazine 26-2 - 1
Instrumentation & Measurement Magazine 26-2 - 2
Instrumentation & Measurement Magazine 26-2 - 3
Instrumentation & Measurement Magazine 26-2 - 4
Instrumentation & Measurement Magazine 26-2 - 5
Instrumentation & Measurement Magazine 26-2 - 6
Instrumentation & Measurement Magazine 26-2 - 7
Instrumentation & Measurement Magazine 26-2 - 8
Instrumentation & Measurement Magazine 26-2 - 9
Instrumentation & Measurement Magazine 26-2 - 10
Instrumentation & Measurement Magazine 26-2 - 11
Instrumentation & Measurement Magazine 26-2 - 12
Instrumentation & Measurement Magazine 26-2 - 13
Instrumentation & Measurement Magazine 26-2 - 14
Instrumentation & Measurement Magazine 26-2 - 15
Instrumentation & Measurement Magazine 26-2 - 16
Instrumentation & Measurement Magazine 26-2 - 17
Instrumentation & Measurement Magazine 26-2 - 18
Instrumentation & Measurement Magazine 26-2 - 19
Instrumentation & Measurement Magazine 26-2 - 20
Instrumentation & Measurement Magazine 26-2 - 21
Instrumentation & Measurement Magazine 26-2 - 22
Instrumentation & Measurement Magazine 26-2 - 23
Instrumentation & Measurement Magazine 26-2 - 24
Instrumentation & Measurement Magazine 26-2 - 25
Instrumentation & Measurement Magazine 26-2 - 26
Instrumentation & Measurement Magazine 26-2 - 27
Instrumentation & Measurement Magazine 26-2 - 28
Instrumentation & Measurement Magazine 26-2 - 29
Instrumentation & Measurement Magazine 26-2 - 30
Instrumentation & Measurement Magazine 26-2 - 31
Instrumentation & Measurement Magazine 26-2 - 32
Instrumentation & Measurement Magazine 26-2 - 33
Instrumentation & Measurement Magazine 26-2 - 34
Instrumentation & Measurement Magazine 26-2 - 35
Instrumentation & Measurement Magazine 26-2 - 36
Instrumentation & Measurement Magazine 26-2 - 37
Instrumentation & Measurement Magazine 26-2 - 38
Instrumentation & Measurement Magazine 26-2 - 39
Instrumentation & Measurement Magazine 26-2 - 40
Instrumentation & Measurement Magazine 26-2 - 41
Instrumentation & Measurement Magazine 26-2 - 42
Instrumentation & Measurement Magazine 26-2 - 43
Instrumentation & Measurement Magazine 26-2 - 44
Instrumentation & Measurement Magazine 26-2 - 45
Instrumentation & Measurement Magazine 26-2 - 46
Instrumentation & Measurement Magazine 26-2 - 47
Instrumentation & Measurement Magazine 26-2 - 48
Instrumentation & Measurement Magazine 26-2 - 49
Instrumentation & Measurement Magazine 26-2 - 50
Instrumentation & Measurement Magazine 26-2 - 51
Instrumentation & Measurement Magazine 26-2 - 52
Instrumentation & Measurement Magazine 26-2 - 53
Instrumentation & Measurement Magazine 26-2 - 54
Instrumentation & Measurement Magazine 26-2 - 55
Instrumentation & Measurement Magazine 26-2 - 56
Instrumentation & Measurement Magazine 26-2 - 57
Instrumentation & Measurement Magazine 26-2 - 58
Instrumentation & Measurement Magazine 26-2 - 59
Instrumentation & Measurement Magazine 26-2 - 60
Instrumentation & Measurement Magazine 26-2 - 61
Instrumentation & Measurement Magazine 26-2 - 62
Instrumentation & Measurement Magazine 26-2 - 63
Instrumentation & Measurement Magazine 26-2 - Cover3
Instrumentation & Measurement Magazine 26-2 - Cover4
https://www.nxtbook.com/allen/iamm/26-6
https://www.nxtbook.com/allen/iamm/26-5
https://www.nxtbook.com/allen/iamm/26-4
https://www.nxtbook.com/allen/iamm/26-3
https://www.nxtbook.com/allen/iamm/26-2
https://www.nxtbook.com/allen/iamm/26-1
https://www.nxtbook.com/allen/iamm/25-9
https://www.nxtbook.com/allen/iamm/25-8
https://www.nxtbook.com/allen/iamm/25-7
https://www.nxtbook.com/allen/iamm/25-6
https://www.nxtbook.com/allen/iamm/25-5
https://www.nxtbook.com/allen/iamm/25-4
https://www.nxtbook.com/allen/iamm/25-3
https://www.nxtbook.com/allen/iamm/instrumentation-measurement-magazine-25-2
https://www.nxtbook.com/allen/iamm/25-1
https://www.nxtbook.com/allen/iamm/24-9
https://www.nxtbook.com/allen/iamm/24-7
https://www.nxtbook.com/allen/iamm/24-8
https://www.nxtbook.com/allen/iamm/24-6
https://www.nxtbook.com/allen/iamm/24-5
https://www.nxtbook.com/allen/iamm/24-4
https://www.nxtbook.com/allen/iamm/24-3
https://www.nxtbook.com/allen/iamm/24-2
https://www.nxtbook.com/allen/iamm/24-1
https://www.nxtbook.com/allen/iamm/23-9
https://www.nxtbook.com/allen/iamm/23-8
https://www.nxtbook.com/allen/iamm/23-6
https://www.nxtbook.com/allen/iamm/23-5
https://www.nxtbook.com/allen/iamm/23-2
https://www.nxtbook.com/allen/iamm/23-3
https://www.nxtbook.com/allen/iamm/23-4
https://www.nxtbookmedia.com