Abstract

Experimental design is a key challenge in science and engineering. Classical experimental design associates experiments with states that can freely be chosen. We consider the rich case where the structure of a given Markov Decision Process constrains the choice of experiments. Additionally, many real-world experiments have safety constraints. Certain states or actions may be dangerous or simply off-limits. At the same time, we often want to learn about some unknown quantity in the unsafe region. This problem is relevant in many real-world settings, from robotic inspection of industrial sites to nuclear fusion. We propose a safe state-action density constraint set that guarantees safe policies. Additionally, we propose a new reweighted experimental design objective that leverages correlation between the safe and unsafe regions. This allows us to estimate an unknown quantity in the unsafe region from observations in the safe region. Finally, we provide both theoretical and experimental analysis of the convergence.

Website Template.