A Bayesian Optimization Approach via Iteratively Least Squares for High-Dimensional Black-Box Systems’ Design Space Exploration

The design space exploration in black-box systems, which lack complete mechanistic models, has wide applications in scientific and industrial research. Bayesian optimization(BO) can obtain high-quality solutions with limited evaluations, which is particularly suitable for expensive black-box optimization problems. However, existing research indicates that BO methods face challenges in high-dimensional black-box optimization problems. A BO algorithm based on iterative least squares(ILS) dimensionality reduction is proposed to reduce the impact of the dimensionality in high-dimensional black-box function optimization with low-dimensional effective subspace. The algorithm first performs dimensionality reduction by a limited dataset, iteratively conducting least squares fitting, and obtains important parameters through weight analysis of the combination with the smallest error. Subsequently, it applies Bayesian optimization only for the important parameters, while other parameters are set to the optimal values obtained from the primal dataset. Experimental results demonstrate that, compared to directly USE BO with MACE acquisition function, an advanced BO method, this method performs better in high-dimensional black-box functions with the same number of evaluations.