When robustness and validity of models – as well as the “as if” assumption put forward in the defense of unrealistic updating mechanisms, and the strong learning assumptions – are questioned, a natural answer is to test them in experiments. Compared to the growth of the literature on “learning” theories, the body of experimental work to test proposed learning mechanisms is still relatively small.
One reason may be that most theories are not designed for testing. Instead, they contribute to refinements of theoretical answers to theoretical questions, based mainly on introspection and casual empiricism since the unstated goal of most such analyses has been to predict behavior entirely by theory.[32]
Hence, when testing a specific learning mechanism, a number of problems arise. One main problem is that the mechanism cannot be tested per se, that is, it is always tested in connection with a model or game that involves various additional assumptions.[33] Furthermore, most models contain little or no information about the conditions under which the proposed mechanism is assumed to be a good approximation of human learning behavior. Hence, a theory that would specify the conditions for and limits of the proposed learning mechanisms is missing, so that experimenters must pursue a course of trial-and-error-meta-learning in order to find out which mechanism describes human learning best in which setting (e.g., game).[34]
Since every model or game is precisely defined by numerous restrictions and assumptions that critically influence the agents’ actions and the model’s possible equilibria, the learning process itself is likely to be strongly influenced by these restrictions and assumptions. – Moreover, as CRAWFORD (1995a, 3) has argued in his recent review of experimental studies of strategic interaction, none of the leading theoretical frameworks for analyzing games adequately identifies the principles that govern behavior by itself.[35] Thus, within this methodology the quest of finding a general learning mechanism – i.e., a theory that would enrich our understanding of learning in economics under a wider range of assumptions, and in other than perfect environments – may be questioned.
A traditional way of finding more general evidence for a given mechanism is to adjust a model to, or calibrate a model with experimental data,[36] and then to test it in different other situations. Several attempts to fit models with experimental data have been made. CHEUNG & FRIEDMAN (1994) fitted experimental data to a modified version of fictitious play with some success and found a better fit than for stimulus-response type models. ROTH & EREV (1995; see also EREV & ROTH, forthcoming) have proposed a stimulus-response (or reinforcement) model who’s simple learning mechanism fits aggregate level data of several experiments relatively well (especially the intermediate term behavior).
RAPOPORT ET AL. (1995) found that the adaptive dynamics of a modified version of the Roth-Erev model can be relatively accurate at the individual and aggregate levels in fitting the experimental data of a coordination (i.e., market entry) game.[37] VAN HUYCK, BATTALIO & RANKIN (1996) compared several learning models and report that exponential fictitious play fits their data best, whereas NAGEL & TANG (1997) find that some simple reinforcement models outperform the standard Nash equilibrium model and the quantal response model on experimental data in a centipede game.
The results of a study by EREV & RAPOPORT (1997) also favor a reinforcement model, but show that this model cannot account for the observed effects of variations in the content of feedback information. [38]
32 See CRAWFORD (1995a, 2).
33This problem of testing joint hypotheses has a long tradition in empirical economics.
34 On the problem of evaluating learning models see also RAPOPORT ET AL. (1995, 35) who emphasize the lack of alternative models that may be compared.
35 CRAWFORD (1995a) examines experimental evidence with respect to traditional noncooperative game theory, evolutionary game theory, and adaptive learning models.
36 See ARTHUR (1994, pp. 135) for a calibrated learning algorithm for automata.
37 In DANIEL, SEALE & RAPOPORT (1996) and RAPOPORT, SEALE & WINTER (1997) an alternative reinforcement learning model to Roth/Erev was proposed and tested for individual level behavior.
38 For more models that are designed to fit experimental data see for example CRAWFORD (1995b), ZAUNER (1994), and MCKELVEY & PALFREY (1992).
Prof. Tilman Slembeck
Next: Comment
Summary: Index