Efficient Exploration in Bayesian Optimization – Optimism and Beyond by Andreas Krause - 2021
Details
Title : Efficient Exploration in Bayesian Optimization – Optimism and Beyond by Andreas Krause Author(s): Google TechTalks Link(s) : https://www.youtube.com/watch?v=p_PK1CuEuAE
Rough Notes
Running example: Tuning the Swiss Free Electron Laser, femto-second duration x-ray device. It has many parameters that need to be adapted to application specific requirements.
Experimentation (learning/exploration) vs. Optimization (control/exploitation).
Some metrics:
- Cumulative regret.
- Simple regret.
- Inference regret.
A lot of Bayesian Optimization relies on having well-calibrated uncertainty intervals on the unknown function.
Safe Optimization: Noisy reward and noisy constraints. Involves a model over both the unknown function and constaints. E.g. SafeOpt ICML 2015, Goal-directed Safe Exploration NeurIPS 2019, SafeLineBO ICML 2019.
Drawback of optimistic exploration: It favours uncertainty. Issues arise in heteroscedaticity, e.g. UCB cannot distinguish between informative and noisy actions - fails to distinguish epistemic from aleatoric uncertainty.
Beyond optimism: Information Directed Sampling. Information-directed exploration.
Meta-learning priors for sequential decision making: PAC-Bayesian Meta Learning ICML 2021, Meta-learning reliable priors in function space NeurIPS 2021 - these works looks similar to the work from Fillipone's group.