Cross-Domain Iterative Detection for OTFS Transmission With Frequency Domain Equalization

Orthogonal time frequency space (OTFS) modulation has received significant attention recently due to its superior performance compared to conventional multicarrier waveforms. However, symbol detection with OTFS is significantly more involved and typically operates on large signal blocks with intersymbol interference (ISI) in the delay-Doppler (DD) domain. In this paper, we investigate the performance of OTFS within the cross-domain iterative detection (CDID) framework. Specifically, three distinct CDID algorithms are presented and investigated, which estimate/detect the information symbols iteratively across the frequency and DD domains via passing either the a posteriori or extrinsic information using a full-sized or single-tap linear minimum mean square error (LMMSE) estimator. Building upon this framework, we study the average mean square error (MSE) for the considered CDID algorithms, where both the bias evolution and the state (variance) evolution are investigated. Particularly, we show that the proposed CDIDs can provide unbiased estimation under certain channel conditions. Furthermore, a fixed point exists in the state evolution when the estimation is unbiased, indicating that the algorithm’s convergence is guaranteed. More importantly, we reveal that passing the a posteriori information is more beneficial when the underlying channel has negligible Doppler spread while passing the extrinsic information is more suitable for non-negligible Doppler spread cases, where the frequency domain channel matrix lacks diagonal dominance. Our numerical results confirm our analytical findings and unveil the near-optimal error performance achieved by the proposed design.