Optimal feedback control of dynamical systems via value-function approximation
A self-learning approach for optimal feedback gains for finite-horizon nonlinear continuous time control systems is proposed and analysed. It relies on parameter dependent approximations to the optimal value function obtained from a family of universal approximators. The cost functional for the trai...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Académie des sciences
2023-07-01
|
| Series: | Comptes Rendus. Mécanique |
| Subjects: | |
| Online Access: | https://comptes-rendus.academie-sciences.fr/mecanique/articles/10.5802/crmeca.199/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825206033807572992 |
|---|---|
| author | Kunisch, Karl Walter, Daniel |
| author_facet | Kunisch, Karl Walter, Daniel |
| author_sort | Kunisch, Karl |
| collection | DOAJ |
| description | A self-learning approach for optimal feedback gains for finite-horizon nonlinear continuous time control systems is proposed and analysed. It relies on parameter dependent approximations to the optimal value function obtained from a family of universal approximators. The cost functional for the training of an approximate optimal feedback law incorporates two main features. First, it contains the average over the objective functional values of the parametrized feedback control for an ensemble of initial values. Second, it is adapted to exploit the relationship between the maximum principle and dynamic programming. Based on universal approximation properties, existence, convergence and first order optimality conditions for optimal neural network feedback controllers are proved. |
| format | Article |
| id | doaj-art-c490b99b8edd48879a03e966ad1b19a4 |
| institution | Kabale University |
| issn | 1873-7234 |
| language | English |
| publishDate | 2023-07-01 |
| publisher | Académie des sciences |
| record_format | Article |
| series | Comptes Rendus. Mécanique |
| spelling | doaj-art-c490b99b8edd48879a03e966ad1b19a42025-02-07T13:46:20ZengAcadémie des sciencesComptes Rendus. Mécanique1873-72342023-07-01351S153557110.5802/crmeca.19910.5802/crmeca.199Optimal feedback control of dynamical systems via value-function approximationKunisch, Karl0Walter, Daniel1Johann Radon Institute for Computational and Applied Mathematics (RICAM), Austrian Academy of Sciences, Altenberger Straße 69, 4040 Linz, Austria; University of Graz, Institute of Mathematics and Scientific Computing, Heinrichstr. 36, A-8010 Graz, AustriaInstitut für Mathematik, Humboldt-Universität zu Berlin, Rudower Chaussee 25, 10117 Berlin, GermanyA self-learning approach for optimal feedback gains for finite-horizon nonlinear continuous time control systems is proposed and analysed. It relies on parameter dependent approximations to the optimal value function obtained from a family of universal approximators. The cost functional for the training of an approximate optimal feedback law incorporates two main features. First, it contains the average over the objective functional values of the parametrized feedback control for an ensemble of initial values. Second, it is adapted to exploit the relationship between the maximum principle and dynamic programming. Based on universal approximation properties, existence, convergence and first order optimality conditions for optimal neural network feedback controllers are proved.https://comptes-rendus.academie-sciences.fr/mecanique/articles/10.5802/crmeca.199/optimal feedback controlneural networksHamilton–Jacobi–Bellman equationself-learningreinforcement learning |
| spellingShingle | Kunisch, Karl Walter, Daniel Optimal feedback control of dynamical systems via value-function approximation Comptes Rendus. Mécanique optimal feedback control neural networks Hamilton–Jacobi–Bellman equation self-learning reinforcement learning |
| title | Optimal feedback control of dynamical systems via value-function approximation |
| title_full | Optimal feedback control of dynamical systems via value-function approximation |
| title_fullStr | Optimal feedback control of dynamical systems via value-function approximation |
| title_full_unstemmed | Optimal feedback control of dynamical systems via value-function approximation |
| title_short | Optimal feedback control of dynamical systems via value-function approximation |
| title_sort | optimal feedback control of dynamical systems via value function approximation |
| topic | optimal feedback control neural networks Hamilton–Jacobi–Bellman equation self-learning reinforcement learning |
| url | https://comptes-rendus.academie-sciences.fr/mecanique/articles/10.5802/crmeca.199/ |
| work_keys_str_mv | AT kunischkarl optimalfeedbackcontrolofdynamicalsystemsviavaluefunctionapproximation AT walterdaniel optimalfeedbackcontrolofdynamicalsystemsviavaluefunctionapproximation |