Dynamic Caching and Rate Control System for Short Videos using Reinforcement Learning

In this paper, we present a novel strategy for short video caching in cache queues within short video apps. Our aim is to minimize the number of user jams and reduce user bandwidth wastage. To achieve this goal, we propose a dynamic caching and rate control system based on reinforcement learning. Our system is designed to cache fewer video blocks during periods of good bandwidth to avoid wasting network resources. Conversely, during periods of poor bandwidth, our system ensures that there are more video blocks in the cache and reduces the playback rate of the next short video. We also aim to maintain a high short video cache ratio to reduce the probability of user lag. To achieve these objectives, we introduce a delay-based bandwidth prediction algorithm in WebRTC to predict the network bandwidth. We also design a block-based short video caching policy. Finally, we used the Actor-Critic algorithm to decide three actions: whether to cache video blocks in this time slot, which video block to cache, and the playback rate of the next time slot. Compared to traditional algorithms that aim to fill the cache as much as possible and TikTok-like caching algorithms, our system significantly reduces the number of video jams and network bandwidth waste, thereby enhancing the user's viewing experience and improving the efficiency of network resource utilization.