IEEE Technology and Society Magazine - September 2023 - 92

Feature
images. At the same time, the LSTM can use these
features to model the sequence of actions being performed.
Training the network on a large dataset of
labeled fisherman actions makes it possible to learn
a real-time model that can accurately recognize and
classify different actions a fisherman performs. Convolutional
LSTM (ConvLSTM) [12] is a type of RNN
that is used to model temporal sequences. It combines
the strengths of CNNs and LSTM networks,
allowing it to process spatial and temporal information
effectively. Figure 3 presents the structure of
our fisherman action recognition model. Our model
has been trained to recognize four distinct actions:
fish collection, baiting, no work, and fighting. The
fish collection involves the fisherman gathering and
cleaning fish from the net, typically for more than
16 hours on DWF vessels. This activity is considered
strenuous work within the DWF context. On the
other hand, baiting work is typically performed for
approximately 6 hours on DWF vessels and involves
the fisherman preparing the fishing net. It is vital to
capture abnormal behavior, mainly fighting, within
DWF vessels. Additionally, the fourth action recognized
by our model is " no work, " where the fisherman
may take a break within the designated work
area. The accurate recognition of this action is crucial
for labor exploitation analysis, as it allows us to
identify rest periods and subtract them from the total
work hours. By implementing a fisherman action
recognition method, we can effectively analyze
labor exploitation by identifying instances where the
fisherman may abstain from work. This enables us to
calculate work hours and address potential discrepancies
accurately.
The convolutional layer is used to extract spatial
features from the input data, while the LSTM layer is
used to capture temporal dependencies in the data.
Combining these two layers allows the ConvLSTM
to effectively model complex patterns in sequential
data, such as time-series or video data. ConvLSTMs
have been used in fisherman video prediction and
anomaly detection.
Statistical assessment method
Labor exploitation is complicated in the DWF
industry because of poor video quality. The statistical
assessment method uses two computer vision
technology for finding labor exploitation: 1) you
only look once (YOLO)v7 [13] customized object
detection and 2) fisherman action recognition. The
statistical assessment method is depicted in Figure 4.
CCTV video is given to the YOLOv7 object detection
module, which detects the number of workers
in particular locations minute by minute. The fisherman
action recognition method also takes CCTV
video and finds the fisherman's action minutes by
the minute. We can combine the fisherman action
data and YOLOv7 object detection module data to
compute 24-hour statistical worktime data.
Developed a statistical assessment method
applying the Kolmogorov--Smirnov (KS) [14], [15]
test to validate whether the MFVAS data distributions
match the data distributions from the YOLOv7
object detection and fisherman action recognition
data. Statistical assessment methods will generate an
alert message on a particular date if there are discrepancies.
The KS test is a nonparametric test of the
null hypothesis that two empirical distributions are
drawn from the same distribution. It is based on the
maximum difference between the practical cumulative
distribution functions, asymptotic relative efficiency,
and critical values.
Figure 3. Fisherman action recognition model.
92
IEEE Technology and Society Magazine
IEEE Technology and Society Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2023
