Constrained nonlinear least squares: an exact penalty approach with projected structured quasi-Newton updates

Reviewer: Benjamin L. Schwartz

This paper gives theory, implementation, and testing of an algorithm for nonlinear constrained least squares problems. The algorithm is adapted from an exact penalty method due to Coleman and Conn [1–3]. The word “exact” has a technical meaning relating to linear independence of the constraints at the isolated stationary solution point. The adaptation includes a more formal structuring of the formulation to include both equality and inequality constraints. Solution methods require setting convergence bounds (“epsilons”) for several functions under several conditions, such as global or local. Step size and direction are revised at each iteration, and the switch from global to local solution procedure is made based on the epsilon for that iteration. The bounds are all chosen empirically. While the bounds selected by the authors work well for a large sample set of problems, the paper offers no theory to guide the choices. Some theoretical questions could create difficulties, such as testing whether a stationary point is a true minimum. The implementation does not address this point or some other similar issues. In the sample problems, these omissions are not important, but a user of the code should be aware of the possible difficulties. On the other hand, the code explicitly accommodates other pathological conditions, such as a vacuous solution set for Z, one of the intermediate computation products used in the algorithm. From the sample problems, I conclude that for least squares problems, this approach is competitive with the general methods studies by Hock and Schittkowski [4].