A Subspace Gradient Descent Method for High-Dimensional Simulation-based Optimization

High-dimensional simulation-based optimization problems are common and important in practice. Most of these problems have two key characteristics: (1) the objective function is unimodal, and (2) the factors influencing the objective function are far fewer than the number of dimensions. Gradient descent (GD) methods are well-suited for unimodal problems, but in high-dimensional cases, gradient estimation requires a large number of simulations, making it difficult to apply. This paper proposes a Subspace gradient descent (SubGD) method. First, a subspace is determined using historical data, ensuring that the gradient projection in this subspace closely aligns with the actual gradient. Then, the gradient projection in this subspace is estimated using the finite difference method (FDM). Finally, update the parameters by the estimated gradient projection in this subspace. Compared to the traditional GD methods directly using FDM for gradient estimation, SubGD significantly reduces the number of simulations. In our experiments on a 200-dimensional optimization problem and a 124-dimensional processor software parameter optimization problem, our method outperformed genetic algorithms, traditional GD methods, and Bayesian optimization methods.