JUNE 2020 * IEEE ROBOTICS & AUTOMATION MAGAZINE * 73 Loss 0.05 2 0.5 0 0.05 0.05 0.05 0.5 Loss (h) 0.5 Loss (e) 0.5 Loss (b) 50 2 2 2 5 5 5 20 20 20 128 64 8 0.005 256 512 0.5 0.4 0.3 0.2 0.1 0 0.005 ReLU 0.005 ELU 100 0.05 0.05 0.05 0.5 Loss (i) 0.5 Loss (f) 0.5 Loss (c) 150 Trial (a) 2 2 2 5 5 5 20 20 20 200 32 16 0.005 64 128 0.005 0.1 0.08 0.06 0.04 0.02 0 1 0.005 2 3 4 5 0.05 0.05 0.05 250 0.5 Loss (j) 0.5 Loss (g) 0.5 Loss (d) 2 2 2 5 5 5 20 20 20 300 Figure 8. The hyperparameter optimization for the 3D surface. (a) The loss against the trial number. (b)-(j) The hyperparameter dependence on the loss, for each of the parameters in Table 2. The optimal network is to the left of the scatter plot. 0.1 0.08 0.06 0.04 0.02 0 0.005 128 64 8 0.005 256 512 1 0.005 2 Optimal 3 4 5 0.005 Convolutional Layers Dense Units L2 Norm Convolutional Filters Activation Function Dropout Dense Layers L1 Norm Batch Size