SigOpt run improvement

In the “trials vs best value” visualization I see a lot of points with the same value. Does this mean that SigOpt is not finding any improvement for those runs?

What that particular graph is showing is the best seen value, or trace, so far for all the completed iterations. This means that the graph will only go up when a new and better run has been found.

The fact that there can be multiple runs in between improvements might seem a bit counter intuitive. However, this is a product of Bayesian optimization relaying on both exploration and exploitation to find a good optimum.

To be the most efficient at finding a good optimum the optimization algorithms both need to explorer the search space, while also exploiting the optimums that we already know.

So whenever we are not seeing a better run for a given iteration, so does not mean that the iteration has been wasted, this just means that we have learned that a given run is not a better optimum - this information is just as valuable for the algorithm.

1 Like