Onofrio Semeraro - LISN - Reinforcement learning strategies for shear flows

Abstract: Flow control has attracted numerous research efforts in the last decades. Common approaches range from applications of optimal control to more sophisticated methodologies found in applied control theory. From a more fundamental viewpoint, among the challenges that these applications pose, the high dimensionality of the discretized operators governing the dynamics of fluid flows is one of the most studied. A common path to circumvent this challenge is to reproduce the essential dynamics by means of reduced order models. Recently, in alternative to the design of controllers based on reduced order models, the community is moving towards data-driven applications where a first principle model is not necessarily at hand; an example is provided by Reinforcement Learning (RL), which encompasses a large variety of algorithms and strategies. Interestingly, these tools do not require any a-priori knowledge of the equations governing the system to be controlled and solely rely on the local measurements of the flow, based on which a policy is learnt from the interaction of the agent with the environment.

Despite successes, the first documented applications of RL for control in fluid dynamics often result in highly non-intuitive control policies, also when ``cheaper'' optimal solutions are available; an example is provided by our results obtained for the control of the Kuramoto-Sivashinsky (KS) equation, where Deep Deterministic Policy Gradient is used for the control of chaotic regime: the dynamics is controlled by rather complex policies although simpler, optimal control laws for this case are known. These results will be illustrated together with an overview on possible challenges and limitations that this application poses.