Intelligent structural optimization of composite mini-channel heat exchangers using reinforcement learning algorithms

Structural optimization of compact heat exchangers involves high-dimensional design spaces where thermal, hydraulic, and structural constraints are tightly coupled. This paper develops a reinforcement learning (RL) framework that couples three tabular algorithms (Sarsa, Q-learning, Dyna-Q) and one deep RL algorithm (Proximal Policy Optimization, PPO) with a physics-informed environment based on segmented LMTD methodology and ASME structural verification, simultaneously optimizing the cross-sectional channel geometries and spatial layouts of a composite mini-channel heat exchanger. The model-based Dyna-Q algorithm reduces the training episodes required for convergence by 75% relative to Sarsa and maintains the lowest post-convergence reward fluctuation. PPO achieves a peak ECO of 329.17 at γ = 0.5, exceeding the genetic algorithm (GA) baseline by 14.4%, but suffers a 32.5% degradation through catastrophic forgetting at γ = 0.9, confirming the robustness advantage of tabular methods in low-dimensional discrete optimization. The RL-optimized design attains a volumetric power density of 223 MW/m³, which is a 32.7% improvement over the GA baseline. This work provides a methodology for intelligent design of mini-channel heat exchangers with application potential in aerospace thermal management systems.