Abstract: Instrumental variable regression (IVar) is a popular method to handle endogeneity in regression. The first part of this talk will be about gradient-based methods for IVaR. Our approach is motivated by viewing IVaR as a conditional stochastic optimization problem and deriving a stochastic gradient-style algorithm. Importantly, under the availability of a two-sample oracle, our procedure avoids explicitly modeling and estimating the relationship between covariate and the instrumental variables, demonstrating the benefit of the proposed viewpoint. We establish rates of conference for this derived algorithm under various settings. The second part of this talk will be about the in-context learning (ICL) capability of transformers in handling endogeneity in linear regression. We show that transformers are capable of efficiently implementing gradient-based IVaR algorithms and derive precise existence guarantees. We further show that the trained transformers (i.e., global optimum of ICL loss) achieve small excess loss. Through these two threads, the talk will highlight the advantages of algorithmic thinking for a classical statistics/econometrics problem.