Boris KARANOV  (Eindhoven University of Technology)

Autoencoders in Optical Communications – From Modulation Format Optimization to Blind EqualizationEnd-to-end deep learning for communication over dispersive nonlinear channels

Abstract
Deep learning, allowing the approximation of any nonlinear function, finds an increasing application in the digital signal processing modules of communication systems. Often, a specific transmitter or receiver function, such as coding, modulation or equalization, is optimized using deep learning. Moreover, deep learning and neural networks allow to design a complete communication system by carrying out the optimization in a single process spanning from the transmitter input to the receiver output. Such systems, implemented as a single deep neural network, have the potential to achieve the optimal end-to-end performance and recently gained popularity in communication scenarios, where the optimum pair of transmitter and receiver or optimum processing modules are not known or prohibitive because of complexity. In low-cost optical fiber systems based on intensity modulation and direct detection (IM/DD), the joint effects of chromatic dispersion and square-law photodiode detection render the communication channel nonlinear with memory. Such systems are particularly suitable for deep learning-based signal processing due to absence of optimal algorithms as well as complexity constraints. We discuss how end-to-end deep learning can be implemented in the low-cost fiber system and compare their performance and complexity with classical designs.