Mathematics Colloquia and Seminars
Applications of Stein's Identity to Non-Convex Optimization and SamplingMathematics of Data and Decisions
|Speaker:||Krishnakumar Balasubramanian, UC Davis|
|Start time:||Tue, Jan 22 2019, 4:10PM|
Optimization and sampling are arguably the two computational backbones of Frequentist and Bayesian statisticsrespectively. In this talk the use of Stein's identity for performing stochastic Zeroth-Order (ZO) non-convex optimization and sampling will be discussed. Specifically, we consider the case when the function (or density) under consideration is not available to us analytically, rather we are able to only obtain noisy evaluations. Using Stein's identity, techniques for estimating gradient and Hessian of the function (or the potential functions of a density) will be introduced. Based on this, the following algorithms and the corresponding theoretical results will be discussed: (i) ZO conditional gradient method, (ii) ZO stochastic gradient method in high-dimensions (iii) ZO Newton's method for escaping saddle points (iv) ZO Euler and Ozaki discretized Monte Carlo sampling.