Estimating Shape Parameters of Piecewise Linear-Quadratic Problems
Piecewise Linear-Quadratic (PLQ) penalties are widely used to develop models in statistical inference, signal processing, and machine learning. Common examples of PLQ penalties include least squares, Huber, Vapnik, 1-norm, and their asymmetric generalizations. Properties of these estimators depend o...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Université de Montpellier
2021-09-01
|
| Series: | Open Journal of Mathematical Optimization |
| Online Access: | https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.10/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825204958681628672 |
|---|---|
| author | Zheng, Peng Ramamurthy, Karthikeyan Natesan Aravkin, Aleksandr Y. |
| author_facet | Zheng, Peng Ramamurthy, Karthikeyan Natesan Aravkin, Aleksandr Y. |
| author_sort | Zheng, Peng |
| collection | DOAJ |
| description | Piecewise Linear-Quadratic (PLQ) penalties are widely used to develop models in statistical inference, signal processing, and machine learning. Common examples of PLQ penalties include least squares, Huber, Vapnik, 1-norm, and their asymmetric generalizations. Properties of these estimators depend on the choice of penalty and its shape parameters, such as degree of asymmetry for the quantile loss, and transition point between linear and quadratic pieces for the Huber function.In this paper, we develop a statistical framework that can help the modeler to automatically tune shape parameters once the shape of the penalty has been chosen. The choice of the parameter is informed by the basic notion that each PLQ penalty should correspond to a true statistical density. The normalization constant inherent in this requirement helps to inform the optimization over shape parameters, giving a joint optimization problem over these as well as primary parameters of interest. A second contribution is to consider optimization methods for these joint problems. We show that basic first-order methods can be immediately brought to bear, and design specialized extensions of interior point (IP) methods for PLQ problems that can quickly and efficiently solve the joint problem. Synthetic problems and larger-scale practical examples illustrate the utility of the approach. Code for the new IP method is implemented using the Julia language (https://github.com/UW-AMO/shape-parameters/tree/ojmo). |
| format | Article |
| id | doaj-art-4600d2aa6ab4462d96ccbb93333d306f |
| institution | Kabale University |
| issn | 2777-5860 |
| language | English |
| publishDate | 2021-09-01 |
| publisher | Université de Montpellier |
| record_format | Article |
| series | Open Journal of Mathematical Optimization |
| spelling | doaj-art-4600d2aa6ab4462d96ccbb93333d306f2025-02-07T14:02:30ZengUniversité de MontpellierOpen Journal of Mathematical Optimization2777-58602021-09-01211810.5802/ojmo.1010.5802/ojmo.10Estimating Shape Parameters of Piecewise Linear-Quadratic ProblemsZheng, Peng0Ramamurthy, Karthikeyan Natesan1Aravkin, Aleksandr Y.2Department of Health Metrics Sciences, University of Washington Seattle, WA, USAIBM Thomas J. Watson Research Center Yorktown Heights, NY, USADepartment of Applied Mathematics, University of Washington Seattle, WA, USAPiecewise Linear-Quadratic (PLQ) penalties are widely used to develop models in statistical inference, signal processing, and machine learning. Common examples of PLQ penalties include least squares, Huber, Vapnik, 1-norm, and their asymmetric generalizations. Properties of these estimators depend on the choice of penalty and its shape parameters, such as degree of asymmetry for the quantile loss, and transition point between linear and quadratic pieces for the Huber function.In this paper, we develop a statistical framework that can help the modeler to automatically tune shape parameters once the shape of the penalty has been chosen. The choice of the parameter is informed by the basic notion that each PLQ penalty should correspond to a true statistical density. The normalization constant inherent in this requirement helps to inform the optimization over shape parameters, giving a joint optimization problem over these as well as primary parameters of interest. A second contribution is to consider optimization methods for these joint problems. We show that basic first-order methods can be immediately brought to bear, and design specialized extensions of interior point (IP) methods for PLQ problems that can quickly and efficiently solve the joint problem. Synthetic problems and larger-scale practical examples illustrate the utility of the approach. Code for the new IP method is implemented using the Julia language (https://github.com/UW-AMO/shape-parameters/tree/ojmo).https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.10/ |
| spellingShingle | Zheng, Peng Ramamurthy, Karthikeyan Natesan Aravkin, Aleksandr Y. Estimating Shape Parameters of Piecewise Linear-Quadratic Problems Open Journal of Mathematical Optimization |
| title | Estimating Shape Parameters of Piecewise Linear-Quadratic Problems |
| title_full | Estimating Shape Parameters of Piecewise Linear-Quadratic Problems |
| title_fullStr | Estimating Shape Parameters of Piecewise Linear-Quadratic Problems |
| title_full_unstemmed | Estimating Shape Parameters of Piecewise Linear-Quadratic Problems |
| title_short | Estimating Shape Parameters of Piecewise Linear-Quadratic Problems |
| title_sort | estimating shape parameters of piecewise linear quadratic problems |
| url | https://ojmo.centre-mersenne.org/articles/10.5802/ojmo.10/ |
| work_keys_str_mv | AT zhengpeng estimatingshapeparametersofpiecewiselinearquadraticproblems AT ramamurthykarthikeyannatesan estimatingshapeparametersofpiecewiselinearquadraticproblems AT aravkinaleksandry estimatingshapeparametersofpiecewiselinearquadraticproblems |