Reinforcement Learning for the Stochastic Optimal Control of a Hybrid Residential Energy System

We consider a hybrid residential heating system equipped with a photovoltaicthermal
production unit. A battery and hot water tank serves as buffer to
balance out fluctuations in production and demand. A connection to the
energy grid enables pursing energy or to generate revenue from the sale of
excess production. Additionally, a heat pump provides efficient local heat
supply.
Controlling this type of system can be quite complex due to the connection of
various components and uncertainty in future electricity price as well as the
local electricity and heat production, which is effected by the solar irradiance
and outdoor temperature.

The main objective is to minimize the cost of purchasing electricity from the
grid, net of revenue from selling electricity back to the grid, while taking
into account the operational constraints of the heating system. The resulting
stochastic optimal control problem is treated as finite horizon Markov decision
process for a multi-dimensional controlled state process.
Classical backward recursion techniques for the value function suffer from the
curse of dimensionality. Therefore, we apply reinforcement learning methods
in order to solve the underlying high-dimensional control problem. In order
to validate and compare results, a smaller simplified model will be solved for
which both approaches are applicable.

This is a joint work with Ralf Wunderlich.