Evaluating Predictive Uncertainty Challenge

The goal of this challenge is to evaluate probabilistic methods for regression and for classification problems. A number of regression classification tasks are proposed. Training data (input-output pairs) are given, and the contestants are asked to predict the outputs associated to a set of validation and test inputs. These predictions are probabilistic and take the form of predictive distributions. The performance of the competing algorithms will be evaluated both with traditional losses that only take into account “point predictions” and with losses that evaluate the quality of the probabilistic predictions.