Joint Throughput Maximization and Energy Management for Ultralow Power Ambient Backscatter Communication in WBANs by Distributed Deep Reinforcement Learning

In this article, we present an ultralow power ambient backscatter communications (AmBC) framework for wireless body area networks (WBANs). In these WBANs, the AmBC sensor nodes are energy harvesting (EH) powered and are able to transmit their own collected physiological information by backscattering the signals emitted by the Wi-Fi access point (AP). To achieve maximum throughput performance with sustainable operation for these ultralow power AmBC sensor nodes in WBANs, we propose a three-phase AmBC transmission model and formulate a joint throughput maximization and energy management (JTMEM) problem. The first-order Markov process channel model and random data collection are also developed in the system model to represent practical health monitoring application environments. Since the full real-time channel state information (CSI) is difficult to obtain at the beginning of each time slot, the optimal working mode selection (WMS) policy is not available for this problem. To overcome this issue, we propose a deep reinforcement learning (DRL)-based algorithm, which can use historical CSI to learn the potential channel correlation to infer the channel changing and decide the appropriate working mode for each AmBC sensor node. Moreover, we establish a distributed DRL structure to overcome huge action space issues and make the proposed algorithm flexible and scalable. Finally, extensive numerical simulation results demonstrate that our proposed algorithm can approach the average throughput performance and rewards of the ergodic policy, which has full real-time CSI in different scenarios, and represents favorable energy management performance after convergence.