Review CMA-ES implementation

[btjanaka]

• pyribs version: 0.3.1
• Python version: N/A
• Operating System: N/A

Description

It seems we may have some errors in the CMA-ES implementation.

For instance, when calculating covariance matrix update (https://github.com/icaros-usc/pyribs/blob/master/ribs/emitters/opt/_cma_es.py#L264), we don't multiply by the sum of the weights, but it looks like they do this in pycma (https://github.com/CMA-ES/pycma/blob/master/cma/purecma.py#L417). This does not matter since the weights sum to 1 anyway, but this would be an issue when implementing active CMA-ES (#85).

As such, we should go thru the CMA-ES implementation and check that it matches the purecma implementation in pycma.

[tehqin]

This value is always one. But we should add this when we implement active-CMA-ES.

[btjanaka]

Updated to enhancement. I think this would be something good to do if we want to tune the CMA-ES implementation further, but for now, it seems to work fine for folks.