| A tool that I've found myself reaching for more and more often is Gaussian Process Regression [1] [2] * It allows you to model essentially arbitrary functions. The main model assumption is your choice of kernel, which defines the local correlation between nearby points. * You can draw samples from the distribution of all possible functions that fit your data. * You can quantify which regions of the function you have more or less certainty about. * Imagine this situation: you want to discover the functional relationship between the inputs and outputs of a long-running process. You can test any input you want, but it's not practical to exhaustively grid-search the input space. A Gaussian Process model can tell you which inputs to test next so as to gain the most information, which makes it perfect for optimising complex simulations. Used in this way, it's one means of implementing "Bayesian Optimisation" [3] [1] https://en.wikipedia.org/wiki/Gaussian_process [2] http://scikit-learn.org/stable/modules/generated/sklearn.gau... [3] https://en.wikipedia.org/wiki/Bayesian_optimization |