Stiff Models and Gradient Methods with the Exponential Relaxation

Professor Igor G. Chernorutskiy
Saint Petersburg State Polytechnical University
Russia
E-mail: igcher@spbstu.ru

Abstract: 1. For a class of matrix gradient methods a new concept of the relaxation function is suggested. This concept allows to evaluate the effectiveness of each gradient optimization procedure, and to synthesize new methods for special classes of ill conditioned (stiff) non-convex optimization problems. According to the suggested formula , it is possible to build relevant search procedures for any given relaxation function.
2. The theorem about the relaxation conditions of each matrix gradient method is proven. Based on the concept of the relaxation functions it is given the geometric interpretation of relaxation properties of gradient methods. According to this interpretation it is possible to build a relaxation area, and to evaluate the speed of the objective function values decreasing.
3. The analysis of classical matrix gradient schemes such as simple gradient method, Newton's methods, Marquardt method is given. It is shown that the relaxation function and its geometric interpretation gives almost full information about the properties and capabilities of relevant gradient optimization methods.
4. A new class of matrix gradient methods with the exponential relaxation function (ERF) is suggested. It is shown that ERF-method summarizes the classical gradient methods including Newton methods, and Marquardt method. In contrast to these methods, ERF-methods have the relaxation functions, entirely located in the relaxation area, which significantly increases the computational efficiency of gradient methods.
5. The ERF-methods convergence for a wide class of non-convex objective functions is established.

Brief Biography of the Speaker: Dr. Chernorutskiy currently is a Professor of Saint-Petersburg State Polytechnical University ( SPbSPU). Degrees ( SPbSPU ): Professor, 1990; Doctor of Technical Science, 1987; Associate Professor, 1982; Ph.D., 1978; M.S., 1970.
Professor Chernorutskiy is the Chair of Information & Control Systems Division of Computer Science and Engineering School (CSES).
Research Interests
Applied Software Engineering, Optimization Tools, Real - Time Systems Modeling and Simulation, Parameter Estimation, and Adaptive Optimization, Decision Support Systems, Artificial Intelligence and Expert Systems.