User association as a stochastic game for enhanced performance in heterogeneous networks

In heterogeneous networks, users are usually confronted with multiple covering base stations (BSs) that differ in the respects of transmit power, bandwidth resources, and so forth, which makes the user association problem more challenging. In this paper, we consider this problem by emphasizing the long-term effect of the user association policy against the dynamic wireless environment for each individual user. In particular, we exploit the stochastic game model to characterize users' non-cooperative behaviors that they compete for the limited resources at BSs for better services, where the reward function for users is defined as their infinite-horizon discounted sum rate. Such a formulation has the advantage to track the users' performance in the long run with respect to the channel state variations. The Nash equilibrium of the game is obtained from users' best-reply playing, which is formulated as a Markov decision process with the value iteration algorithm providing the solution. Furthermore, we specially analyze the two-BS scenario and derive the threshold-based results for the association policy. The simulation results demonstrate that, compared with the counterparts, our proposal achieves higher system sum rate with relatively lower frequency of handovers, and improves the fairness in terms of transmission rate among users.