Worst Perception Scenario Search via Recurrent Neural Controller and K-Reciprocal Re-Ranking

Achieving excellent generalization on perceiving real traffic scenarios with diversity is the long-term goal for building robust autonomous driving systems. A recent theoretical study shows that the generalization on the worst-group of test samples is far more difficult than others. Therefore, we propose to discover potential shortness of certain perception module by analyzing its worst-scenario performance. However, with the benchmark datasets growing huge and tremendous, exhaustive searching for the worst perception scenario (WPS) seems to be time consuming and unnecessary. To address this, we present an automatic searching scheme empowered by reinforcement learning. In this case, worst scenario mining is formulated as the discrete search on the Visual Operation Design Domain (ODD), namely scenario representation, by optimizing LSTM-RNN controller with the worst-performance reward. Moreover, a time-efficient K-reciprocal re-ranking technique is utilized to match the predicted scenario parameters with existing test data. The proposed method has been validated by finding the most challenging scenarios for various vehicle detectors on KITTI, BDD100k and our own benchmark set EVB. Furthermore, searching performances w.r.t different Visual ODDs are investigated and it is found that visual representations through generative adversarial network contribute to a better performance.