10.2 10.4 RS REA 10 9.6 9.8 9.4 9.2 9 8.8 10 20 30 40 50 60 70 80 90 100 Number of Samples FIGURE 9 Performance comparison of NAS algorithms on NASBench-201 for CIFAR10 classification. BANANAS-PE BANANAS-PAPE NPENAS-NP NPENAS-SSRL NPENAS-SSCCL Table III. The performance of BANANAS using the positionaware path-based encoding exceeds that of using path-based encoding, proving that the proposed position-aware path-based encoding is an efficient and effective encoding scheme. In addition, the performance of NPENAS-NP improves from 5.86% [6] to 5.83% after filtering out isomorphic graphs using the proposed position-aware path-based encoding. Table III also shows that NPENAS methods using the proposed self-supervised pretrained neural predictors (NPENAS-SSRL and NPENAS-SSCCL) have only a slight performance drop when the search budget is reduced from 150 to 90, but still outperform the unsupervised arch2vec that uses a large search budget of 400. The impact of the search budget on NPENAS-SSRL and NPENAS-SSCCL methods are shown in Table IV. The performance of NPENAS-SSRL continues to improve as the search budget increases, while NPENAS-SSCCL achieves its best performance using only 80 evaluated neural architectures. These results are consistent with those in Section IV-C and again show 31.5 31 30.5 29.5 28.5 30 29 28 27 10 20 30 40 50 60 70 80 90 100 Number of Samples FIGURE 10 Performance comparison of NAS algorithms on NASBench-201 for CIFAR-100 classification. 27.5 26.5 RS REA BANANAS-PE BANANAS-PAPE NPENAS-NP NPENAS-SSRL NPENAS-SSCCL 56.9 56.4 55.9 55.4 54.9 53.9 54.4 53.4 10 20 30 40 50 60 70 80 90 100 Number of Samples FIGURE 11 Performance comparison of NAS algorithms on NASBench-201 for ImageNet-16-120 classification. RS REA BANANAS-PE BANANAS-PAPE NPENAS-NP NPENAS-SSRL NPENAS-SSCCL TABLE V Performance comparison of NAS algorithms on NASBench-201. METHODS RS [50] REA [25] BANANAS-PE [3] BANANAS-AE [3] BANANAS-PAPE [3] NPENAS-NP [6] SEARCH BUDGET 100 100 100 100 100 100 NPENAS-NP-FIXED [6] 50† NPENAS-SSRL NPENAS-SSRL-FIXED NPENAS-SSCCL NPENAS-SSCCL-FIXED 100 50† 100 50† 8.98 ± 0.24 9.06 ± 0.31 8.95 ± 0.14 8.93 ± 0.16 8.95 ± 0.13 8.95 ± 0.15 8.94 ± 0.11 8.92 ± 0.11 8.94 ± 0.09 8.93 ± 0.10 TEST ERR (%) AVG CIFAR-10 TEST ERR (%) AVG CIFAR-100 TEST ERR (%) AVG IMAGENET-16-120 9.27 ± 0.32 28.55 ± 0.86 27.20 ± 0.89 27.41 ± 1.03 26.77 ± 0.67 26.86 ± 0.67 26.70 ± 0.64 26.72 ± 0.57 26.55 ± 0.34 26.57 ± 0.30 26.50 ± 0.12 26.58 ± 0.33 54.65 ± 0.80 53.89 ± 0.71 53.85 ± 0.64 53.67 ± 0.34 53.71 ± 0.46 53.90 ± 0.59 53.84 ± 0.57 53.75 ± 0.51 53.63 ± 0.38 53.80 ± 0.36 53.66 ± 0.36 †The neural predictor is trained with 50 evaluated neural architectures, while other algorithms use 100 neural architectures for evaluation. AUGUST 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 47 Testing Error of Best Neural Net Testing Error of Best Neural Net Testing Error of Best Neural Net