Review: SyntheticSlateBanditDataset (w/ new feature: calc_ground_truth_policy_value)

[aiueola]

new feature

Implemented calc_ground_truth_policy_value() function in SyntheticSlateBanditDataset.
Calculation procedure is given as follows.

1. receive evaluation_policy_logit (n_rounds, n_unique_action) and context (n_rounds, dim_context).
2. enumerate combinatorial slate actions.
3. for each slate action list, calculate (combinatorial) pscore and expected reward.
4. sum up expected reward weighted by pscore.

Note that we use expected reward instead of sampled reward to replicate click models.
https://github.com/aiueola/zr-obp/blob/bf6f6c8ca4ec76c84ef660263870369eb568f831/obp/dataset/synthetic_slate.py#L655

refactor

• Renamed from calc_item_position_pscore() to _calc_pscore_given_action_list(), as it does not directly calculate marginal pscore. It rather calculates combinatorial pscore given action list inside.
  https://github.com/aiueola/zr-obp/blob/bf6f6c8ca4ec76c84ef660263870369eb568f831/obp/dataset/synthetic_slate.py#L319
• Renamed from behavior_policy_logit_i_ arg in _calc_pscore_given_action_list() to policy_logit_i_ as we use this function when calling calc_ground_truth_policy_value() for evaluation policy.
  https://github.com/aiueola/zr-obp/blob/bf6f6c8ca4ec76c84ef660263870369eb568f831/obp/dataset/synthetic_slate.py#L320
• Removed unused arg tau in obtain_batch_bandit_feedback().
  https://github.com/aiueola/zr-obp/blob/d52501696038def2189ae1a61abb106fdd8db688/obp/dataset/synthetic_slate.py#L474
• Changed from standard_exponential and cascade_exponential reward structure to standard_decay and cascade_decay, respectively. In addition, introduce decay_function, which is either exponential or inverse.
  https://github.com/aiueola/zr-obp/blob/bf6f6c8ca4ec76c84ef660263870369eb568f831/obp/dataset/synthetic_slate.py#L50
  https://github.com/aiueola/zr-obp/blob/bf6f6c8ca4ec76c84ef660263870369eb568f831/obp/dataset/synthetic_slate.py#L1241

tests

• Fixed and added corresponding tests.

others

• Minor fix on typos and docstrings.

[usaito]

@aiueola

• These lines can be outside of the above for i in range(n_rounds):, right?
  https://github.com/aiueola/zr-obp/blob/f3dc59c19a7f4298662ae8e201e6cec743d83e8f/obp/dataset/synthetic_slate.py#L690-L693

• enumerated_slate_actions has to be np.ndarray, but here it is still a list. You should put enumerated_slate_actions=np.array(enumerated_slate_actions) between lines 713 and 714.
  https://github.com/aiueola/zr-obp/blob/f3dc59c19a7f4298662ae8e201e6cec743d83e8f/obp/dataset/synthetic_slate.py#L718

• "distance must be 1-dimensional ndarray"
  https://github.com/aiueola/zr-obp/blob/bf6f6c8ca4ec76c84ef660263870369eb568f831/obp/dataset/synthetic_slate.py#L1265

[usaito]

@aiueola

• Regarding the above comment, I think it is good to add the base_reward_function=None case in the same test function
  https://github.com/aiueola/zr-obp/blob/afd6577e64a0b6ad14a6273ce24ae80bce604236/tests/dataset/test_synthetic_slate.py#L1368

• How about adding the comparisons with varying eta? For example, for both "pbm" and "cascade", a larger value of eta should lead to a lower ground-truth policy value given the same eval_policy right?
  https://github.com/aiueola/zr-obp/blob/afd6577e64a0b6ad14a6273ce24ae80bce604236/tests/dataset/test_synthetic_slate.py#L1441-L1442