IEEE Robotics & Automation Magazine - December 2021 - 110

Problem-Solving Approach and Students'
Performances
Figure 5 shows the actual distribution of students' positive
and negative performances (whether the exercises were completed
or not) across the three problem-solving behaviors.
The " mathematical/planning " and " tinkering with refining "
behaviors have a large portion of success in both exercises A
and B, while " tinkering with significantly high changes "
shows the worst results, especially in exercise A.
To analyze the relationship between these behaviors
and teams' success in the two exercises, the Pearson correlation
coefficient (PCC) between the clusters presented in
Tables 2 and 3, and the final results obtained by students'
teams were computed. In exercise A, two clusters showed a
statistically significant negative correlation: the number of
trials (PCC = -0.48; p value < 0.001) and percentage of
sequences of cluster 4 (PCC = -0.39; p value < 0.01). High
values of these features, which are typical for " tinkering
with significantly high changes " teams, indicate a higher
probability of negative performance. In Exercise B, two
clusters showed a statistically significant negative correlation:
the percentage of sequences of cluster 2 (PCC =
-0.62; p value < 0.001), which is typical for " tinkering with
30
25
20
15
10
5
refining " groups that did not use the loop block, and the
percentage of sequences of cluster 3 (PCC = -0.29; p value
< 0.05), which is typical for " tinkering with significantly
high changes. "
Success Prediction With ML Algorithms
The ability to predict the success or lack thereof for students'
teams was tested using four ML algorithms: the LR, SVM,
KNN, and RF classifiers [Figure 1(b)]. These are four supervised
ML techniques belonging to three different approaches
[9]: information- (RF), similarity- (KNN), and error-based
learning (LR and SVM).
On average, SVM can reach higher accuracies in prediction
tasks [9]-[11]. In terms of speed, LR models are faster at making
predictions than KNN. Regarding the transparency of
knowledge (the interpretability of the results), decision tree
models are very easily interpretable compared to SVM and RF.
Moreover, even if the features selected in this article are continuous,
and information-based models do not perform very well
with this kind of feature [9], the authors wanted to compare
various approaches to find the best ML algorithm for the educational
data set. Moreover, they also aimed to compare the
performances of the supervised and mixed approaches to
Mathematical/Planning
Tinkering With Significantly High Changes
Tinkering With Refining
Cluster 1
60
50
40
30
20
10
Cluster 2 Cluster 3 Cluster 4 Cluster 5
(a)
Mathematical/Planning
Tinkering With Significantly High Changes
Tinkering With Refining
Cluster 6 Cluster 7 Cluster 8 Trials
Cluster
1
Cluster
2
Cluster
3
Cluster
4
Cluster
5
Cluster
6
Cluster
7
(b)
Figure 5. The distribution of the emerging problem-solving behaviors in exercises (a) A and (b) B as well as the positive and negative
performances across the three styles in exercises. (Continued)
110 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2021
Cluster
8
Cluster
9
Cluster
10
Cluster
11
Trials
Number of Programming Sequences
Number of Programming Sequences
IEEE Robotics & Automation Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2021
