Hongseok Namkoong

Adaptivity can significantly improve efficiency of experimentation, but it is challenging to implement even at large online platforms with mature experimentation systems. As a result, many real-world experiments are deliberately implemented with large batches and a handful of opportunities to update the sampling allocation as a way to reduce operational costs of experimentation.
In this work, we focus on adaptive experiments with limited adaptivity (short horizons T < 10). Bandit algorithms focusing on long-horizon settings are tailored to provide regret guarantees for each specific case, and we find they often underperform static A/B tests on practical problem instances with batched feedback, non-stationarity, multiple objectives and constraints, and personalization.
In response, we develop a mathematical programming framework for developing adaptive experimentation algorithms. Instead of the problem-specific research paradigm (akin to an optimization solver developed for a particular linear program), we ask the modeler to write down a flexible optimization formulation and use modern machine learning systems to (heuristically) solve for adaptive designs. Since a naive formulation of the adaptive experimentation problem as a dynamic program is intractable, we propose a batched view of the experimentation process. We model the uncertainty around batch-level sufficient statistics necessary to make allocation decisions, instead of attempting to model unit-level outcomes whose distributions are commonly unknown and leads to intractable dynamic programs with combinatorial action spaces.
Sequential Gaussian approximations is the main intellectual vehicle powering our mathematical programming framework. CLT-based normal approximations are universal in statistical inference, and a sequential variant we prove provides a simple optimization formulation that lends itself to modern computational tools. Through extensive empirical evaluation, we observe that even a preliminary and heuristic solution approach can provide major robustness benefits. Unlike bespoke methods (e.g., Thompson sampling variants), our mathematical programming framework provides consistent gains over static randomized control trials and exhibits robust performance across problem instances.

AI models are omni-present yet extrapolate in unexpected ways, posing a significant barrier to robust and fair systems. Building AI systems that can articulate their own uncertainty has been a longstanding challenge in ML, such probabilistic reasoning capability is key to bounding downside risk (e.g., delegating to human experts) and continually improving system performance by gathering data to resolve uncertainty. Despite recent advances in large language models, uncertainty quantification remains a challenge, with methods attempting to leverage these deep neural networks—such as Bayesian neural networks—frequently facing scalability limitations.
This work takes an important conceptual step towards building large-scale AI systems that can reason about uncertainty through natural language. We revisit De Finetti’s view of uncertainty coming from missing observations rather than latent parameters, which allows us to pose learning to do statistical inference as a prediction problem involving masked inputs. This formal connection between autoregressive generation with probabilistic reasoning allows pre-trained sequence models to express their epistemic uncertainty on underlying concepts, and refine their beliefs as they gather more information.
Our findings open a promising avenue for addressing uncertainty in complex, data-rich settings in a scalable way. We are excited by how this work leverages a timeless insight to inform a timely topic: guiding the next generation of AI systems.
1. As internet data depletes, the pace of progress in LLM capabilities has been widely observed to slow down (even in public media). This suggests that the limited paradigm of pre-training on passively scraped web data has reached its full potential. To move forward, the authors believe that the next generation of AI systems must be able to understand tasks on which they suffer high uncertainty, and actively gather data in order to continually improve their performance.
2. Since scalable uncertainty quantification poses a key intellectual bottleneck, we resolve this by going back to De Finetti’s insight developed in the 1920s. We believe the connection between Bayesian inference and autoregressive generation provides the groundwork for building LLMs with probabilistic reasoning capabilities.

Taken together, our work showcases how principled scientific insights have the potential to shape the design of even the largest scale AI systems.