## A05 - Probing scales in equilibrated systems by optimal nonequilibrium forcing

**Head(s):** Prof. Dr. Carsten Hartmann (BTU Cottbus-Senftenberg), Prof. Dr. Susanna Röblitz (ZIB and University of Bergen), Prof. Dr. Christof Schütte (FU Berlin), PD Dr. Marcus Weber (ZIB)

**Project member(s):** Dr. Ilja Klebanov, Lara Neureither, Lorenz Richter, Alexander Sikorski, Dr. Martin Weiser, Dr. Wei Zhang

**Participating institution(s):** FU Berlin, ZIB

### Project Summary

This project is devoted to the analysis and simulation of rare statistical fluctuations in multiscale random dynamical systems beyond equilibrium that are driven by external forcing. Stochastic control theory is a key methodology in the project, specifically, we exploit an intimate duality between the cumulant generating functions of certain path functionals and entropy minimisation where the latter is interpreted within the framework of stochastic control theory. As a result we obtain a variational principle that determines a probability measure that we use to obtain precise importance sampling estimates of the quantities under consideration.

For multiscale system, the corresponding optimal controls can be highly oscillatory and thus the primary focus of this project so far has been to develop a thorough theoretical understanding of the variational problem using averaging, homogenisation and model reduction techniques. For large-scale systems having multiple time scales, especially without clear scale separation (such as large biomolecules with multivalent bonds), solving optimal control problems numerically can be notoriously difficult, which underpins the need for suitable data-driven model reduction and stochastic approximation techniques. These will be explored in the second funding period.

One of the key ideas for the second funding period of the project is to exploit the intimate connection of the variational formulation with Bayes’ Theorem and to develop algorithms that scale polynomially with the system dimension. Specifically, we plan to use this connection in both directions and (i) employ non-parametric inference techniques to approximate potentially high-dimensional optimal importance sampling distributions as Bayesian posteriors by minimizing either the corresponding cross-entropy or the relative entropy (a.k.a. Kullback-Leibler divergence), and (ii) use the connection with importance sampling to design efficient numerical strategies for data assimilation of multiscale diffusions.

A second idea is to study iterative methods for solving the variational problem in connection with model reduction techniques. Specifically, we plan to revisit approximate policy iteration and iterative numerical schemes for (decoupled) forward-backward stochastic differential equations. Here the key insight is that the semilinear dynamic programming equation associated with the aforementioned optimal control problem has a rather specific structure, which makes them amenable to data-driven model reduction methods that do not rely on scale separation, such as conditional expectation or Markov state modelling. The aim is to derive computable error bounds for situations, in which the high-dimensional optimal controls are approximated by a numerical discretisation of a reduced-order model.

We, furthermore, plan to study realistic molecular systems, aiming at the (in-silico) design of multivalent drug-like molecules with a specific multiscale dissociation profile. Solving either the corresponding data assimilation problems or the associated stochastic control problem cannot be done without using tailored sampling techniques, and the idea here is to reformulate available enhanced sampling algorithms, such as metadynamics or the adaptive biasing force method, using our optimal control framework. Not only will this lead to optimised molecular dynamics sampling algorithms, but, conversely, it will also allow us to incorporate prior information that is obtained from, e.g., a metadynamics simulation or a Markov state model into the construction of adaptive importance sampling schemes.

### Project publications

Nüsken, N. and Richter, L.
(2020)
*Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using
neural networks: perspectives from the theory of controlled diffusions and measures on path space.*
SFB 1114 Preprint in arXiv
.
pp. 1-40.
(Submitted)

Hartmann, C. and Neureither, L. and Sharma, U.
(2019)
*Coarse-graining of non-reversible stochastic differential equations: quantitative results and connections to averaging.*
SIAM J. Math. Anal.
.
pp. 1-39.
(Submitted)

Kebiri, O. and Neureither, L. and Hartmann, C.
(2019)
*Adaptive importance sampling with forward-backward stochastic differential equations.*
In:
Stochastic Dynamics Out of Equilibrium. IHPStochDyn 2017.
Springer Proceedings in Mathematics & Statistics, 282
.
Springer, pp. 265-281.
ISBN 978-3-030-15095-2

Neureither, L. and Hartmann, C.
(2019)
*Time scales and exponential trends to equilibrium: Gaussian model problems.*
In:
Stochastic Dynamics Out of Equilibrium. IHPStochDyn 2017.
Springer Proceedings in Mathematics & Statistics, 282
.
Springer, pp. 391-410.
ISBN 978-3-030-15095-2

Hartmann, C. and Kebiri, O. and Neureither, L. and Richter, L.
(2019)
*Variational approach to rare event simulation using least-squares regression.*
Chaos, 29
(6).
063107.
ISSN 1054-1500 (print); 1089-7682 (online)

Hartmann, C. and Schütte, Ch. and Zhang, W.
(2019)
*Jarzysnki equality, fluctuation theorems and variance reduction: Mathematical analysis and numerical algorithms.*
J. Stat. Phys., 175
(6).
pp. 1214-1261.
ISSN 0022-4715; ESSN: 1572-9613

Donati, L. and Heida, M. and Weber, M. and Keller, B.
(2018)
*Estimation of the infinitesimal generator by square-root approximation.*
Journal of Physics: Condensed Matter, 30
(42).
p. 425201.
ISSN 0953-8984, ESSN: 1361-648X

Kebiri, O. and Neureither, L. and Hartmann, C.
(2018)
*Singularly perturbed forward-backward stochastic differential equations: application to the optimal control of bilinear systems.*
Computation, 6
(3).
p. 41.
ISSN 2079-3197 (online)

Hartmann, C. and Schütte, Ch. and Weber, M. and Zhang, W.
(2018)
*Importance sampling in path space for diffusion processes with
slow-fast variables.*
Probab. Theory Rel. Fields, 170
(1-2).
pp. 177-228.
ISSN 0178-8051 (print) 1432-2064 (online)

Weber, M.
(2018)
*Implications of PCCA+ in Molecular Simulation.*
Computation, 6
(1).
ISSN 2079-3197 (online)

Hartmann, C. and Richter, L. and Schütte, Ch. and Zhang, W.
(2017)
*Variational characterization of free energy: theory and algorithms.*
Entropy (Special Issue), 19
(11).
pp. 1-27.
ISSN 1099-4300

Quer, J. and Lie, H.
(2017)
*Some connections between importance sampling and enhanced sampling methods in molecular dynamics.*
Journal of Chemical Physics
.
pp. 1-19.
ISSN 0021-9606

Delle Site, L. and Ciccotti, G. and Hartmann, C.
(2017)
*Partitioning a macroscopic system into independent subsystems.*
Journal of Statistical Mechanics: Theory and Experiment, 2017
.
pp. 1-13.

Quer, J. and Donati, L. and Keller, B.G. and Weber, M.
(2017)
*An automatic adaptive importance sampling algorithm for molecular dynamics in reaction coordinates.*
SIAM J. Sci. Comput.
.
pp. 1-19.
ISSN 1064-8275 (print)
(In Press)

Weber, M. and Fackeldey, K. and Schütte, Ch.
(2017)
*Set-free Markov state model building.*
Journal of Chemical Physics, 146
(12).
p. 124133.

Koltai, P. and Ciccotti, G. and Schütte, Ch.
(2016)
*On metastability and Markov state models for non-stationary molecular dynamics.*
Journal of Chemical Physics, 145
(17).
p. 174103.

Quer, J. and Weber, M.
(2016)
*Estimating exit rate for rare event dynamical systems by extrapolation.*
ZIB-Report
.
pp. 1-19.
ISSN 2192-7782 (online)

Banisch, Ralf and Hartmann, C.
(2016)
*A sparse Markov chain approximation of LQ-type stochastic control problems.*
Math. Control Relat. F., 6
(3).
pp. 363-389.
ISSN 1064-8275

Hartmann, C. and Schütte, Ch. and Zhang, W.
(2016)
*Model reduction algorithms for optimal control and importance sampling of diffusions.*
Nonlinearity, 29
(8).
pp. 2298-2326.
ISSN 0951-7715

Zhang, W. and Hartmann, C. and Schütte, Ch.
(2016)
*Effective dynamics along given reaction coordinates, and reaction rate theory.*
Faraday discussions, 195
.
pp. 365-394.
ISSN 1359-6640

Bittracher, Andreas and Hartmann, C. and Junge, O. and Koltai, Péter
(2015)
*Pseudo generators for under-resolved molecular dynamics.*
The European Physical Journal Special Topics, 224
(12).
pp. 2463-2490.
ISSN 1951-6355

Hartmann, C. and Latorre, J.C. and Pavliotis, G. A. and Zhang, W.
(2014)
*Optimal control of multiscale systems using reduced-order models.*
J. Computational Dynamics, 1
(2).
pp. 279-306.
ISSN 2158-2505

Lie, Han Cheng and Schütte, Ch. and Hartmann, C.
(2014)
*Martingale-based gradient descent algorithm for estimating free energy values of diffusions.*
SIAM J. Sci. Comput.
.
ISSN 1064-8275
(Submitted)