The backtrack Hölder gradient method with application to min-max and min-min problems
We present a new algorithm to solve min-max or min-min problems out of the convex world. We use rigidity assumptions, ubiquitous in learning, making our method – the backtrack Hölder algorithm applicable to many optimization problems. Our approach takes advantage of hidden regularity properties and...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Université de Montpellier
2023-12-01
|
| Series: | Open Journal of Mathematical Optimization |
| Subjects: | |
| Online Access: | https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.24/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825205039334948864 |
|---|---|
| author | Bolte, Jérôme Glaudin, Lilian Pauwels, Edouard Serrurier, Mathieu |
| author_facet | Bolte, Jérôme Glaudin, Lilian Pauwels, Edouard Serrurier, Mathieu |
| author_sort | Bolte, Jérôme |
| collection | DOAJ |
| description | We present a new algorithm to solve min-max or min-min problems out of the convex world. We use rigidity assumptions, ubiquitous in learning, making our method – the backtrack Hölder algorithm applicable to many optimization problems. Our approach takes advantage of hidden regularity properties and allows us, in particular, to devise a simple algorithm of ridge type. An original feature of our method is to come with automatic step size adaptation which departs from the usual overly cautious backtracking methods. In a general framework, we provide convergence theoretical guarantees and rates. We apply our findings on simple Generative Adversarial Network (GAN) problems obtaining promising numerical results. It is worthwhile mentioning that a byproduct of our approach is a simple recipe for general Hölderian backtracking optimization. |
| format | Article |
| id | doaj-art-df63be0048a74d26bfd07d753f3a7ce9 |
| institution | Kabale University |
| issn | 2777-5860 |
| language | English |
| publishDate | 2023-12-01 |
| publisher | Université de Montpellier |
| record_format | Article |
| series | Open Journal of Mathematical Optimization |
| spelling | doaj-art-df63be0048a74d26bfd07d753f3a7ce92025-02-07T14:02:57ZengUniversité de MontpellierOpen Journal of Mathematical Optimization2777-58602023-12-01411710.5802/ojmo.2410.5802/ojmo.24The backtrack Hölder gradient method with application to min-max and min-min problemsBolte, Jérôme0Glaudin, Lilian1Pauwels, Edouard2Serrurier, Mathieu3Toulouse School of Economics, Université Toulouse Capitole, Toulouse, FranceANITI, University of Toulouse, Toulouse FranceToulouse School of Economics, Institut Universitaire de France, Toulouse, France.Université Paul-Sabatier, IRIT Toulouse, FranceWe present a new algorithm to solve min-max or min-min problems out of the convex world. We use rigidity assumptions, ubiquitous in learning, making our method – the backtrack Hölder algorithm applicable to many optimization problems. Our approach takes advantage of hidden regularity properties and allows us, in particular, to devise a simple algorithm of ridge type. An original feature of our method is to come with automatic step size adaptation which departs from the usual overly cautious backtracking methods. In a general framework, we provide convergence theoretical guarantees and rates. We apply our findings on simple Generative Adversarial Network (GAN) problems obtaining promising numerical results. It is worthwhile mentioning that a byproduct of our approach is a simple recipe for general Hölderian backtracking optimization.https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.24/Hölder gradientbacktracking line searchmin-max optimizationridge methodsemi-algebraic optimization |
| spellingShingle | Bolte, Jérôme Glaudin, Lilian Pauwels, Edouard Serrurier, Mathieu The backtrack Hölder gradient method with application to min-max and min-min problems Open Journal of Mathematical Optimization Hölder gradient backtracking line search min-max optimization ridge method semi-algebraic optimization |
| title | The backtrack Hölder gradient method with application to min-max and min-min problems |
| title_full | The backtrack Hölder gradient method with application to min-max and min-min problems |
| title_fullStr | The backtrack Hölder gradient method with application to min-max and min-min problems |
| title_full_unstemmed | The backtrack Hölder gradient method with application to min-max and min-min problems |
| title_short | The backtrack Hölder gradient method with application to min-max and min-min problems |
| title_sort | backtrack holder gradient method with application to min max and min min problems |
| topic | Hölder gradient backtracking line search min-max optimization ridge method semi-algebraic optimization |
| url | https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.24/ |
| work_keys_str_mv | AT boltejerome thebacktrackholdergradientmethodwithapplicationtominmaxandminminproblems AT glaudinlilian thebacktrackholdergradientmethodwithapplicationtominmaxandminminproblems AT pauwelsedouard thebacktrackholdergradientmethodwithapplicationtominmaxandminminproblems AT serruriermathieu thebacktrackholdergradientmethodwithapplicationtominmaxandminminproblems AT boltejerome backtrackholdergradientmethodwithapplicationtominmaxandminminproblems AT glaudinlilian backtrackholdergradientmethodwithapplicationtominmaxandminminproblems AT pauwelsedouard backtrackholdergradientmethodwithapplicationtominmaxandminminproblems AT serruriermathieu backtrackholdergradientmethodwithapplicationtominmaxandminminproblems |