[aeytan]

Were you trying to do optimization using a binary outcome? I am not familiar with many packages that do this out of the box. Your implementation looks like a good start for the binary case, but you will get better results by computing expected improvement on the probability of winning, (i.e., Phi(f(x)), rather than f(x), where Phi() is the standard normal CDF). For more on why, see http://proceedings.mlr.press/v28/tesch13.pdf and https://arxiv.org/pdf/2110.09361.pdf. AFAIK this should be as simple as defining a simple MCAcquisitionObjective (https://botorch.org/docs/objectives) that passes samples through a torch.distributions.normal CDF. You can then just pass that objective into qNoisyExpectedImprovement. Feel free to open a botorch GH issue if you try this out or need help!

I would also recommend using many more raw_samples and random_restarts in opimize_acqf(). 512 and 20, respectively, are good defaults. The current values are much too small to be able to effectively optimize the acquisition functions.

[thomasahle]

Yes, I'm using a binary outcome, since that's what I get from playing a game. To get probabilities I'd have to play a lot of games with the same settings/features/point and take the mean, but it seems that defeats the point of Bayesian optimization finding the best point to evaluate for each iteration.

The SPSA method seems to work quite well with binary outcomes. This is what I was trying to beat. Unfortunately I was never able to converge faster than SPSA (or even close to that) even increasing the number of samples. There is a pretty long thread of us trying to make it work here: https://www.talkchess.com/forum3/viewtopic.php?f=7&t=71650&h...

I got some feedback form the botorch team back then: https://github.com/pytorch/botorch/issues/347#:~:text=thomas...