Efficient Exploration in Bayesian Optimization – Optimism and Beyond by Andreas Krause - 2021

Title : Efficient Exploration in Bayesian Optimization – Optimism and Beyond by Andreas Krause Author(s): Google TechTalks Link(s) : https://www.youtube.com/watch?v=p_PK1CuEuAE

Running example: Tuning the Swiss Free Electron Laser, femto-second duration x-ray device. It has many parameters that need to be adapted to application specific requirements.

Experimentation (learning/exploration) vs. Optimization (control/exploitation).

Some metrics:

• Cumulative regret.
• Simple regret.
• Inference regret.

A lot of Bayesian Optimization relies on having well-calibrated uncertainty intervals on the unknown function.

Safe Optimization: Noisy reward and noisy constraints. Involves a model over both the unknown function and constaints. E.g. SafeOpt ICML 2015, Goal-directed Safe Exploration NeurIPS 2019, SafeLineBO ICML 2019.

Drawback of optimistic exploration: It favours uncertainty. Issues arise in heteroscedaticity, e.g. UCB cannot distinguish between informative and noisy actions - fails to distinguish epistemic from aleatoric uncertainty.

Beyond optimism: Information Directed Sampling. Information-directed exploration.

Meta-learning priors for sequential decision making: PAC-Bayesian Meta Learning ICML 2021, Meta-learning reliable priors in function space NeurIPS 2021 - these works looks similar to the work from Fillipone's group.