OPTIMIZATION SEMINAR
A New Stochastic Procedure for Finding Roots of Systems of Unknown Functions
Burt Simon
University of Colorado at Denver
Department of Mathematics
Tuesday, Oct. 14, 1997, 12:00 noon
CU-Denver Bldg., Room 626
Abstract
A new method is presented for tackling problems traditionally in the
domain of the Robbins-Monro algorithm and other forms of stochastic
approximation. The goal is to find a simultaneous solution to
f_i(x)=0,i=1,2,...,m, where x is in a set B in R^d, and F(x) =
(f_1(x),f_2(x),...,f_m(x))' is estimated by a noisy measurement such
as a simulation whose d parameters are set to x =
(x_1,x_2,...,x_d). The most important applications are the standard
root finding problem, where typically m=d=1, and optimization problems
where F is the gradient of the objective function. The basic
stochastic approximation algorithms are ``Markovian'' in the sense
that the n-th iterate, X_n, depends only on X_(n-1) and the result of
the (n-1)-st simulation. The new method is non Markovian, using all
available data from the first n-1 simulations to choose X_n. Roughly
speaking, with the new method you `always simulate where you think the
root is'. We provide conditions for X_n --> x^*, a.s and compare the
new method numerically with Robbins-Monro.