Optimización experimental con presupuesto finito combinando heurísticas Bayesianas en un POMDP

Resumo

Improving decision making from the observed results after experimentation is a usual task in many applications, from the research lab scale to the industrial one. However, conducting experiments often takes a non-negligible cost. Consequently, an excessive exploration is harmful. Bayesian optimisation is a widely-used technique in this context, due to its low computational cost and because it provides good exploration-exploitation trade-offs. However, this technique does not explicitly account for the actual cost of the experiment, nor whether a limited budget (economic, number of experiments, time, etc.) exists. The problem of decision making under uncertainty and finite budget is a Partially-Observable Markov Decision Process (POMDP). This work addresses the experimental optimisation problem by combining well-known Bayesian heuristics in a POMDP framework solvable via dynamic programming, where a scenario tree is built from the available system/process knowledge (with uncertainty) at each stage. Such a knowledge is modelled as a Gaussian process which is updated with each new observation. The developed algorithm has been tested successfully to optimise the setpoints in a continuous stirred tank reactor that must produce a certain number of batches.