Reinforcement learning for database optimization

The performance of database systems depends on various tuning choices, including join order, system parameter settings, and physical design. For many of these tuning problems, reliable cost models are not available. This makes it hard to estimate the quality of tuning choices before trying them out.

We are exploring approaches to solve such optimization problems via reinforcement learning. Tuning choices become actions in the reinforcement learning framework, and rewards are obtained by evaluating tuning choices on data samples. Instead of applying reinforcement learning on top of classical systems, we have designed novel system architectures that cater to the requirements of reinforcement learning frameworks. In particular, these architectures make fast iterations with different configuration choices efficient by avoiding redundant work.

We have applied these principles in the context of join ordering (SkinnerDB), parallel data processing (SkinnerMT), attribute ordering in worst-case optimal joins (ADOPT), and physical design tuning (UDO). In all cases, our approaches have resulted in dramatic improvements over prior work, in terms of optimization time and solution quality.