experimental condition. As shown in Figure 4, even though the PER DQN has no advantage in the early stage of training, its performance gradually surpasses the traditional DQN as the process continues. This phenomenon also occurs when comparisons are made to the other models. In Figure 5, based on the lidar point cloud input, the tests indicate that the proposed PER DQN is 70 better than the traditional DQN in overtaking and lane changing. Specifically, in Figure 5(a), the PER DQN is at the same level as the traditional DQN in terms of speed, which is also reflected in Table 2. In Figure 6, although the performance of the PER DQN and DQN are close in terms of speed with multimodal inputs, the proposed PER DQN is better than the 65 60 55 0 246 81012 Step (a) ×105 40 35 30 25 20 15 10 5 0 246 81012 Step (b) ×105 120 100 80 60 20 40 0 246 81012 Step (c) ×105 DQN MRA DQN PER DQN PMRA DQN Figure 5. A performance comparison of the proposed PMRA DQN and other baselines using the lidar point cloud input only. (a) The AS. (b) The AO. (c) The AL. 50 55 0 246 81012 Step (a) ×105 45 40 35 30 25 20 15 10 0 246 81012 Step (b) ×105 140 120 100 80 60 40 0 246 81012 Step (c) ×105 DQN MRA DQN PER DQN PMRA DQN Figure 6. A performance comparison of the proposed PMRA DQN and other baselines using image and lidar point cloud input. (a) The AS. (b) The AO. (c) The AL. DECEMBER 2021 * IEEE ROBOTICS & AUTOMATION MAGAZINE * 29 75 70 65 60 Number of Lane Changes Number of Overtaken Vehicles Speed (km/h) Number of Lane Changes Number of Overtaken Vehicles Speed (km/h)