Online trial generation
This video gives a good impression of how the online trial generation algorithm works. On the right, you see the choice screen as presented to a participant who makes decisions, on the left you see the estimate of the participant's indifference curve. Note that the settings dictated that the first 4 trials were "burn trials" and that the piecewise linear function is fitted from trial 5 onwards.
In the majority of studies, you will create a trial list at the beginning of the experiment, keeping everything counterbalanced and randomized, and have the participant go through all of the predefined trials. One issue with this is that you do not know in advance which trials will be most sensitive to your experimental manipulation. The intuition is that psychological background states (e.g. mood, fatigue) tend to have only a small impact on choice preferences, so that they are most likely to be detected when a decision-maker is approximately indifferent between the two presented choice options. Thus, you want to present choices of which you know the choice probability is around 50% (the ensemble of these choices form the indifference curve).
More generally, finding the indifference curve is of great computational interest: the more precise your knowledge of the participant's indifference curve, the more precise your estimates of the parameters that control the value function (e.g., a weight on reward or a discounting factor). Simulations show that the posterior variance for value function parameters decrease most as choices approach the "true" indifference curve more - in other words: the closer to indifference, the more efficient the trial for fitting the model parameters. Note that this does not apply to a choice temperature parameter, which takes indecisiveness/noise into account. When you include a choice temperature in your model, the most efficient trials are slightly above or below the indifference curve - depending on the participant's level of noise. For present (and past) purposes, the noise is not our parameter of interest (and even so, we had no trouble fitting it to the data afterwards anyway).
So, how do we get the decision-maker's indifference curve? The answer is to algorithmically update its estimate with every choice that is made - this is why the trial generation is "online". A model is fitted after every trial, and the features (cost, reward) of the upcoming decision to be presented are determined with this updated model estimate. This iterative procedure is entirely Bayesian in its conception.
The problem with this "online" approach to presenting trials, though, is that you have to commit to a value model in advance. For example, it may turn out detrimental to select an exponential decay model if it turns out that a substantial part of your subjects' choice behavior does not reflect an exponential value function at all. Moreover, by committing to a model in advance, you defeat the purpose of model comparison afterwards, because your trials were generated to optimize the fitting of a predefined model. Here, we have circumvented this problem with a special function that is implemented in the OTG toolbox.
We have created an online trial generation algorithm that does not require to commit to a model of a certain predetermined shape (e.g. convex, concave - see Figure (a), center and bottom). Instead, the fitted indifference curve can approximate any shape after a sufficient number of trials. It does so by dividing the sampling space for the costly option into 5 bins. It then simply fits a linear model with a slope and an intercept to the trials from each bin, with the constraint that the slopes from each section may not be discontinuous with its neighbors. This is guaranteed by defining the value of a given bin's intercept to be equal to the previous bin's uncostly reward level at the upper bound of the cost spectrum, except for the first bin, where the intercept is a free parameter. Thus, there are only 6 parameters to fit in total (b0 and k1-5 in Figure (a), top below).
Algorithmically, the generation of choice option happens as follows. After every trial, a map of the indifference space is computed given the updated estimates of the 6 parameters (Figure (b), step 1). Brighter colors in the top panel correspond to [reward, cost] combinations for which choice probability is closer to 50%. From this map, a probability density function is derived, under which a cost level is randomly drawn (Figure (b), step 2). Then, the small reward that is expected to yield an indifferent choice for this particular cost level is identified (Figure (b), step 3). The selected cost level and small reward are finally presented to the participant at the next choice trial (Figure (b), step 4).
There two notable reasons why online trial generation can be a very valuable addition to your experiment:
-
You want rapid phenotyping. Some participants, typically patients, do not have much time to be tested, and you want your impression of their choice behavior to be as good as possible, as quickly as possible. Note that Choice Calibration comes down to the same thing: before your main experiment, you want to quickly and accurately calibrate the individual preferences of your participant.
-
You want to measure fluctuations in choice behavior. Preferences are not necessarily stable, and variability in choice behavior may be precisely what you are interested in. Past studies have taken the following forms:
- Test-retest studies: after a choice calibration, the participant does another task (e.g., they run a race and get fatigued) and then performs a second battery of economic choice trials. The preference shift in the second session with respect to the first session can be ascribed to mental fatigue.
- Slow drift: participants do an experiment where their mood is manipulated (based on Vinckier et al., 2018) by trial-by-trial feedbacks. In addition, participants rate their current mood and make one economic decision following each (positive or negative) feedback. Slow drifts in choice preferences were correlated to self-rated mood.
Note that calculating the area under the indifference curve (shaded area in Figure (a), top) is a convenient way of assessing momentary willingness to accept costs in order to receive larger rewards. In the first example above, the runners had lower AUCs after their race, and in the second example the participants' AUC was significantly predicted by mood.
The key function in the OTG toolbox for online trial generation is BEC_OnlineTrialGeneration_VBA. This function is made to present one choice at the time, so you want to put it inside a loop (see the simulation demo below). The function is highly automated so that you only need to specify minimal information if the default settings are good enough for you (and typically, they will be). You do not need to provide a triallist structure that dictates the sequence of choice types (although you can, of course) - if you leave it to the algorithm, it will ensure that the order of the specified cost types (say, types 1,2,3, and 4 for delay, risk, physical effort, and mental effort) is randomized within each set of 4 choices, whilst guaranteeing that no 2 subsequent choices are of the same cost type.
Have a look at the %% Get necessary input section on top of the script to see how the inputs to the algorithm are organized, and which ones you can provide yourself if you wish.
There is an equivalent of this function called BEC_OnlineTrialGeneration that is also Bayesian but does not use VBA. This version was developed for deployment online and in smartphone apps. Model inversion is made simpler by implementing a Gauss-Newton algorithm, without the need of the VBA toolbox. However, this model fitting algorithm sometimes does not converge, in which case it applies a heuristic to generate the next trial. In exp_settings.OTG, all settings that have "[Gauss-Newton]" commented behind it, pertain only to this version of the function.
We suggest you always use the VBA version of the online trial generation function if your experiment is run in a MATLAB environment.
In both functions, most of the settings for online trial generation are the same. Have a look at the choice trial generation section on the Getting Started page to see which settings must be adjusted for your experimental needs, and which are better left untouched./ The output of both online trial generation functions is also the same. Behaviour of the participant is stored in AllData.trialinfo and results from the algorithm can be found in AllData.OTG_posterior.(cost):
- muPhi: the most recent estimate of the 6 parameters (the first parameter is the intercept, parameters 2-6 are slopes 1-5 of cost bins 1-5). Note that these values are the log-transformed! Take exp(muPhi) to get the 'true', positively-constrained parameter estimates.
- all_muPhi: the history of all parameter estimates for the given cost type
- P_indiff: the estimate of the indifference grid that is used for sampling the upcoming cost.
Simulations are made easy by leaving the Psychtoolbox window empty when looping through the trial generation function.
Keeping all of the default settings in BEC_Settings, you can quickly simulate 100 trials of the online trial generation procedure as follows:
AllData = struct; for trial = 1:100 AllData = BEC_OnlineTrialGeneration_VBA(AllData); end
Of course, you can exert a bit more control over the simulations by adjusting the first lines of the functions (see AllData.sim), where you can choose a cost type and a model to simulate. You can also adjust the specifics of the algorithm by editing the simulation settings structure OTG_settings - this is the equivalent of the exp_settings.OTG subfield from BEC_Settings (with defaults for experimentation).
The function BEC_Visualize_OTG.m makes an image of the generated trials, the choices that were made, the current estimate of the indifference grid, and the sampling of the upcoming cost. this function is useful when running simulations but can also be applied post-hoc to acquired experimental datasets, you just have to enter AllData and the last trial number as inputs.
In this visualization of the simulation, you can see on the top left the current fitted indifference curve, the "true" indifference curve used for simulations, and the prior indifference curve that was entered here (the population average from a previous study). The colored background expresses the closeness-to-indifference in the sampling grid. The top right figure shows the recent history of presented choices. The bottom left figure shows a probability density function derived from the indifference grid, generated for sampling the cost of the upcoming trial