Papers

See also my Google scholar page.

  1. 2026
    Causal Inference
    A Sensitivity Framework for Assessing External Validity
    Ariel Boyarsky, Naoki Egami, and Hongseok Namkoong
  2. 2026
    Benchmarks
    MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability
    Julian Poeltl, Thomson Yen, Patrick Shea, Shubham Anil Varma, Dhruv Rastogi, Zihao Chen, Mengshi Zhang, and Hongseok Namkoong
    Site Code Blog Data
  3. 2026
    Continual Learning Benchmarks
    GameBoyWorlds: A Testbed for Self-Improvement in Embodied Video Games
    Dhananjay Ashok, Adam Shen, Aslan Huo Feng, Chinmay Khanna, Jun Rui Huang, Raghav Sarmukaddam, Surendira Balaji Natarajan, Xiaotong Cui, Xincan Zhang, Thomson Yen, Hongseok Namkoong, Jonathan May, and Jesse Thomason
    Site
  4. 2026
    Trustworthy AI
    Rare Event Estimation via Iterative Unalignment
    Hanming Yang, Daksh Mittal, Jing Dong, and Hongseok Namkoong
    Code
  5. 2026
    Benchmarks
    MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance
    Thomson Yen, Julian Poeltl, Harshith Srinivas Gear, Yilin Meng, Joshua Fan, Adam Shen, Yili Liu, Ali Bauyrzhan, Patrick Shea, Siri Du, Haoyang Liu, Daniel Guetta, and Hongseok Namkoong
    Advances in Neural Information Processing Systems (NeurIPS) 2026
    Site Code Data
  6. 2026
    Continual Learning Benchmarks
    LatentGym: A Testbed for Cross-Task Experiential Learning with Controllable Latent Structure
    Daksh Mittal, Tommaso Castellani, Thomson Yen, Naimeng Ye, Fangyu Wu, Minghui Chen, Tiffany Cai, Emmanouil Koukoumidis, William Zeng, and Hongseok Namkoong
    Code Blog Data
  7. 2026
    Benchmarks
    Empirical Rigor Through Benchmarking in Operations Research
    Jing Dong, Daksh Mittal, and Hongseok Namkoong
    Site Code
  8. 2026
    Trustworthy AI
    Local Sensitivity Under Transport Restrictions
    Hongseok Namkoong
  9. 2026
    Causal Inference
    When Representative Samples Produce Worse Outcomes: Scale-up Decisions and Testing in Small-Budget RCTs
    Hannah Li, Hongseok Namkoong, and Isaac Scheinfeld
  10. 2026
    Causal Inference
    Empirical Likelihood for Nonsmooth Functionals
    Hongseok Namkoong
    Major revision in Operations Research.
  11. 2026
    Continual Learning Adaptive Experimentation
    A Broader View of Thompson Sampling
    Yanlin Qu, Hongseok Namkoong, and Assaf Zeevi
    International Conference on Machine Learning (ICML) 2026
  12. 2026
    Trustworthy AI Benchmarks
    Rethinking Distribution Shifts: Empirical Analysis and Modeling for Tabular Data
    Tianyu Wang, Jiashuo Liu, Peng Cui, and Hongseok Namkoong
    Management Science. Conference version appeared in Neural Information Processing Systems (NeurIPS) 2023.
    Site Slides Video
  13. 2026
    Continual Learning Adaptive Experimentation
    Learning-To-Measure: In-context Active Feature Acquisition
    Yuta Kobayashi, Zilin Jing, Jiayu Yao, Hongseok Namkoong, and Shalmali Joshi
    International Conference on Machine Learning (ICML) 2026
  14. 2026
    Benchmarks
    FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning
    Liang Hu, Jianpeng Jiao, Jiashuo Liu, Yanle Ren, Zhoufutu Wen, Kaiyuan Zhang, Xuanliang Zhang, Xiang Gao, Tianci He, Fei Hu, Yali Liao, Zaiyuan Wang, Chenghao Yang, Qianyu Yang, Mingren Yin, Zhiyuan Zeng, Ge Zhang, Xinyi Zhang, Xiying Zhao, Zhenwei Zhu, Hongseok Namkoong, Wenhao Huang, and Yuwen Tang
    International Conference on Learning Representations (ICLR) 2026
    Site
  15. 2026
    Adaptive Experimentation Deployed in Practice
    Distilled Thompson Sampling: Practical and Efficient Thompson Sampling via Imitation Learning
    Hongseok Namkoong, Samuel Daulton, and Eytan Bakshy
    Transactions on Machine Learning Research. Selected for an oral presentation at the Neural Information Processing Systems (NeurIPS) 2020 Offline Reinforcement Learning Workshop.
    Code
  16. 2026
    Trustworthy AI
    Evaluating Model Performance Under Worst-case Subpopulations
    Mike Li, Daksh Mittal, Hongseok Namkoong, and Shangzhou Xia
    Mathematics of Operations Research. Short version appeared in Neural Information Processing Systems (NeurIPS) 2021.
  17. 2025
    Queueing
    Data-driven Stochastic Modeling using Autoregressive Sequence Models
    Daksh Mittal, Shunri Zheng, Jing Dong, and Hongseok Namkoong
  18. 2025
    Trustworthy AI Optimization
    DRO: A Python Library for Distributionally Robust Optimization in Machine Learning
    Jiashuo Liu, Tianyu Wang, Henry Lam, Hongseok Namkoong, and Jose Blanchet
    Revise and resubmit at Journal of Machine Learning Research.
    Code
  19. 2025
    Continual Learning Benchmarks Adaptive Experimentation
    Benchmarking In-context Experiential Learning Through Repeated Product Recommendations
    Gilbert Yang, Yaqin Chen, Thomson Yen, and Hongseok Namkoong
    Site Code
  20. 2025
    Continual Learning Benchmarks
    SynthTools: A Framework for Scaling Synthetic Tools for Agent Development
    Tommaso Castellani, Naimeng Ye, Daksh Mittal, Thomson Yen, and Hongseok Namkoong
  21. 2025
    Causal Inference
    A Sensitivity Approach to Causal Inference Under Limited Overlap
    Yuanzhe Ma and Hongseok Namkoong
  22. 2025
    Continual Learning Adaptive Experimentation
    Adaptive Elicitation of Latent Information Using Natural Language
    Jimmy Wang, Thomas Zollo, Richard Zemel, and Hongseok Namkoong
    International Conference on Machine Learning (ICML) 2025. Best Paper Award at International Conference on Learning Representations (ICLR) 2025 Workshop on Quantify Uncertainty and Hallucination in Foundation Models.
    Code
  23. 2025
    Adaptive Experimentation Optimization
    Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework
    Thomson Yen, Andrew Wei Tung Siah, Haozhe Chen, Tianyi Peng, Daniel Guetta, and Hongseok Namkoong
    Advances in Neural Information Processing Systems (NeurIPS) 2025
    Code
  24. 2025
    Trustworthy AI
    LLM Generated Persona is a Promise with a Catch
    Ang Li, Haozhe Chen, Hongseok Namkoong, and Tianyi Peng
    Advances in Neural Information Processing Systems (NeurIPS) 2025
    Data
  25. 2025
    Continual Learning Adaptive Experimentation
    Architectural and Inferential Inductive Biases For Exchangeable Sequence Modeling
    Daksh Mittal, Ang Li, Tzu-Ching Yen, Daniel Guetta, and Hongseok Namkoong
    Advances in Neural Information Processing Systems (NeurIPS) 2025
  26. 2025
    Continual Learning Adaptive Experimentation
    Adaptive Measurement for Posterior Variance Minimization
    Daksh Mittal, Yuanzhe Ma, Shalmali Joshi, and Hongseok Namkoong
    Under second-round review in Operations Research; conference version appeared in Neural Information Processing Systems (NeurIPS) 2024.
  27. 2025
    Continual Learning Adaptive Experimentation
    Contextual Thompson Sampling via Generation of Missing Data
    Kelly Zhang, Tiffany Cai, Hongseok Namkoong, and Daniel Russo
    Advances in Neural Information Processing Systems (NeurIPS) 2025
  28. 2025
    Trustworthy AI Causal Inference
    AI as an intervention: improving clinical outcomes relies on a causal approach to AI development and validation
    Shalmali Joshi, Inigo Urteaga, Wouter A C van Amsterdam, George Hripcsak, Pierre Elias, Benjamin Recht, Noemie Elhadad, James Fackler, Mark P Sendak, Jenna Wiens, Kaivalya Deshpande, Yoav Wald, Madalina Fiterau, Zachary Lipton, Daniel Malinsky, Madhur Nayan, Hongseok Namkoong, Soojin Park, Julia E Vogt, and Rajesh Ranganath
    Journal of the American Medical Informatics Association
  29. 2025
    Continual Learning Benchmarks
    PersonalLLM: Tailoring LLMs to Individual Preferences
    Thomas Zollo, Andrew Siah, Naimeng Ye, Ang Li, and Hongseok Namkoong
    International Conference on Learning Representations (ICLR) 2025
    Code
  30. 2025
    Trustworthy AI Deployed in Practice
    Diagnosing Model Performance Under Distribution Shift
    Tiffany Cai, Hongseok Namkoong, and Steve Yadlowsky
    Operations Research. Conference version appeared at Symposium on Foundations of Responsible Computing (FORC) 2023.
    Code Slides Video
  31. 2025
    Trustworthy AI
    Minimax Optimal Estimation of Stability Under Distribution Shift
    Hongseok Namkoong, Yuanzhe Ma, and Peter W. Glynn
    Operations Research
    Code
  32. 2025
    Trustworthy AI
    From Models to Systems: A Comprehensive Framework for AI System Fairness in Compositional Recommender Systems
    Brian Hsu, Cyrus DiCiccio, Natesh S. Pillai, and Hongseok Namkoong
    Algorithmic Fairness Through the Lens of Metrics and Evaluation, Proceedings of Machine Learning Research (PMLR), 2025
  33. 2024
    Continual Learning Trustworthy AI
    LLM Embeddings Improve Test-time Adaptation to Tabular Y|X-Shifts
    Yibo Zeng, Jiashuo Liu, Henry Lam, and Hongseok Namkoong
    Code
  34. 2024
    Queueing Benchmarks
    QGym: Scalable Simulation and Benchmarking of Queuing Network Controllers
    Haozhe Chen, Ang Li, Ethan Che, Tianyi Peng, Jing Dong, and Hongseok Namkoong
    Advances in Neural Information Processing Systems (NeurIPS) 2024 Datasets and Benchmarks Track
    Code
  35. 2024
    Queueing Optimization
    Differentiable Discrete Event Simulation for Queuing Network Control
    Ethan Che, Jing Dong, and Hongseok Namkoong
    Major revision in Operations Research.
    Code
  36. 2024
    Adaptive Experimentation Optimization
    A Unified Planning Framework for Short-Horizon Adaptive Experimentation
    Ethan Che, Daniel Jiang, Hongseok Namkoong, and Jimmy Wang
    Under second-round review in Management Science; selected for oral presentations at the Econometric Society conference on Interdisciplinary Frontiers: Economics and Artificial Intelligence/Machine Learning and Conference on Digital Experimentation.
    Code Slides
  37. 2024
    Adaptive Experimentation Benchmarks
    AExGym: Benchmarks and Environments for Adaptive Experimentation
    Jimmy Wang, Ethan Che, Daniel Jiang, and Hongseok Namkoong
    Code
  38. 2024
    Continual Learning Adaptive Experimentation
    Exchangeable Sequence Models Quantify Uncertainty Over Latent Concepts
    Naimeng Ye and Hongseok Namkoong
    Under second-round review in Operations Research.
    Slides
  39. 2024
    Causal Inference Optimization
    Constrained Learning for Causal Inference and Semiparametric Statistics
    Tiffany Cai, Yuri Fonseca, Kaiwen Hou, and Hongseok Namkoong
  40. 2024
    Queueing
    Design and Scheduling of an AI-based Queueing System
    Jiung Lee, Hongseok Namkoong, and Yibo Zeng
    Selected for presentation at Manufacturing and Service Operations Management (MSOM) Special Interest Group (SIG) Day 2025.
  41. 2024
    Continual Learning Adaptive Experimentation
    Active Exploration via Autoregressive Generation of Missing Data
    Tiffany Cai, Hongseok Namkoong, Daniel Russo, and Kelly Zhang
    Selected for presentation at the Econometric Society conference on Interdisciplinary Frontiers: Economics and Artificial Intelligence/Machine Learning.
    Slides
  42. 2023
    Adaptive Experimentation Optimization Deployed in Practice
    Adaptive Experimentation at Scale: A Computational Framework for Flexible Batches
    Ethan Che and Hongseok Namkoong
    Under third-round review in Operations Research.
    Site Slides Video
  43. 2023
    Causal Inference
    Modeling Interference via Experiment Rollout
    Ari Boyarsky, Hongseok Namkoong, and Jean Pouget-Abadie
    ACM Conference on Economics and Computation (ACM EC) 2023
    Slides
  44. 2022
    Causal Inference Trustworthy AI Deployed in Practice
    Assessing External Validity to Subpopulation Shift via Worst-case Treatment Effects
    Sookyo Jeong and Hongseok Namkoong
    Under third-round review in Management Science; conference version appeared in Conference on Learning Theory (COLT) 2020.
    Code Slides Video
  45. 2022
    Trustworthy AI Deployed in Practice
    Model Soups: Averaging Weights of Multiple Fine-tuned Models Improves Accuracy Without Increasing Inference Time
    Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt
    International Conference on Machine Learning (ICML) 2022
    Code Video
  46. 2022
    Trustworthy AI Deployed in Practice
    Robust Fine-tuning of Zero-shot Models
    Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo-Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt
    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Best Paper Finalist.
    Code Video
  47. 2022
    Causal Inference Trustworthy AI
    Bounds on the Conditional and Average Treatment Effect with Unobserved Confounding Factors
    Steve Yadlowsky, Hongseok Namkoong, Sanjay Basu, John Duchi, and Lu Tian
    Annals of Statistics
    Slides Video
  48. 2022
    Trustworthy AI Optimization
    Distributionally Robust Losses Against Mixture Covariate Shifts
    John C. Duchi, Tatsunori Hashimoto, and Hongseok Namkoong
    Operations Research
    Code Video
  49. 2021
    Trustworthy AI Optimization
    Learning Models with Uniform Performance via Distributionally Robust Optimization
    John C. Duchi and Hongseok Namkoong
    Annals of Statistics
    Code Slides Video
  50. 2021
    Trustworthy AI Optimization
    Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach
    John C. Duchi, Peter W. Glynn, and Hongseok Namkoong
    Mathematics of Operations Research. Applied Probability Society (APS) Best Student Paper Prize, and featured in Mathematics of Operations Research's 50th anniversary selection.
    Video
  51. 2020
    Causal Inference Trustworthy AI
    Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confounding
    Hongseok Namkoong, Ramtin Keramati, Steve Yadlowsky, and Emma Brunskill
    Advances in Neural Information Processing Systems (NeurIPS) 2020
    Code
  52. 2019
    Trustworthy AI Optimization
    Variance-based regularization with convex objectives
    John C. Duchi and Hongseok Namkoong
    Journal of Machine Learning Research. Conference version won Neural Information Processing Systems (NeurIPS) 2017 Best Paper Award.
    Code Slides Video
  53. 2018
    Trustworthy AI Optimization
    Certifiable Distributional Robustness with Principled Adversarial Training
    Aman Sinha, Hongseok Namkoong, and John Duchi
    International Conference on Learning Representations (ICLR) 2018. Selected for a full oral presentation.
    Code Video
  54. 2018
    Trustworthy AI
    Generalizing to Unseen Domains via Adversarial Data Augmentation
    Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John C. Duchi, Vittorio Murino, and Silvio Savarese
    Advances in Neural Information Processing Systems (NeurIPS) 2018
    Code
  55. 2018
    Trustworthy AI Deployed in Practice
    Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation
    Mathew O'Kelly, Aman Sinha, Hongseok Namkoong, John Duchi, and Russ Tedrake
    Advances in Neural Information Processing Systems (NeurIPS) 2018
  56. 2018
    Trustworthy AI Optimization
    Fairness Without Demographics in Repeated Loss Minimization
    Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang
    International Conference on Machine Learning (ICML) 2018. Best Paper Runner-up Award.
    Code
  57. 2017
    Optimization
    Adaptive sampling probabilities for non-smooth optimization
    Hongseok Namkoong, Aman Sinha, Steve Yadlowsky, and John C. Duchi
    International Conference on Machine Learning (ICML) 2017
    Code
  58. 2016
    Trustworthy AI Optimization
    Stochastic Gradient Methods for Distributionally Robust Optimization with f-divergences
    Hongseok Namkoong and John C. Duchi
    Advances in Neural Information Processing Systems (NeurIPS) 2016
    Slides