AbstractsPsychology

Finding effective ways to improve subjective probability predictions through model learning

by Xin (Edward) Wei




Institution: University of Technology, Sydney
Department:
Year: 2014
Keywords: Decision making.
Record ID: 1054984
Full text PDF: http://hdl.handle.net/10453/28054


Abstract

Predicting probabilities of marketing events and choices is a primary activity in marketing. Prediction can be from the outcome of a formal model or one’s knowledge but the two sources often conflict. Although overwhelming evidence has demonstrated that models often outperform people in terms of accuracy, there is little doubt that decisions are made mostly by people based on their own knowledge. Past research suggests that models and intuition should work together for better outcomes but it is unclear how this may be accomplished other than by using “plain vanilla” style Decision Support Systems (DSSs). This researcher adopted an alternative approach and believes that the key to solving this problem is to improve people’s own knowledge. To do so, people need to gain a substantive understanding of a reliable model to improve predictions. Four generalisable model-learning approaches based on concepts from learning theories in psychology and cognitive science were developed and tested in an experiment to ascertain which approach was more effective in helping learners develop an understanding of the model’s parameters and to improve their consequent predictions. The experiment was supported by an online Intelligent Support System (ITS) with both learning approaches and a target model built in. This target learning model is a consumer choice model of airline flights. The system evaluates predictions, estimates learner models, and classifies answers. Moreover, it provides real-time feedback matching the design of each learning approach. According to the results, the most effective approach for both model learning and prediction improvement is a learning approach generating outcome feedback with correct answers after each experimental design controlled training task. This finding disagrees with a common view of multiple cue probability learning (MCPL). Having regard for effectiveness, the above learning approach is followed by an approach showing feedback with a comparison of estimated learner model and target model outcomes on all parameters. Both approaches outperformed the approach where learners performed self-regulated learning in a DSS which is actually the status quo of decision support nowadays. Another approach tested was to learn a model for a consumer class from the similarities of classes. This approach achieved slow improvement but can be further refined. In conclusion, this research opens a new path for prediction improvement by combining a learning approach, and methods and technology for experimental design, ITS and DSS.