Contingency, Contiguity, and Causality in Conditioning: Applying Information Theory and Weber's Law to the Assignment of Credit Problem.
Contingency is a critical concept for theories of associative learning and the assignment of credit problem in reinforcement learning. Measuring and manipulating it has, however, been problematic. The information-theoretic definition of contingency—normalized mutual information—makes it a readily co...
| Publicado en: | Psychological Review Vol. 126; no. 5; pp. 761 - 774 |
|---|---|
| Autores principales: | , , |
| Formato: | Artículo |
| Publicado: |
American Psychological Association
Oct2019
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=138919874&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 138919874 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 0033295X PYV jtl: Psychological Review issn: 0033295X maglogo: N pubinfo: dt: Oct2019 vid: 126 iid: 5 pid: 34 pub: American Psychological Association artinfo: ui: 138919874 10.1037/rev0000163 ppf: 761 ppct: 13 formats: tig: atl: Contingency, Contiguity, and Causality in Conditioning: Applying Information Theory and Weber's Law to the Assignment of Credit Problem. aug: au: Gallistel, C. R. Craig, Andrew R. Shahan, Timothy A. affil: Department of Psychology, Rutgers University Department of Psychology, Utah State University, and Department of Pediatrics, SUNY Upstate Medical University Department of Psychology, Utah State University su: Information theory Weber-Fechner law Assignment problems (Programming) Reinforcement learning Credit laws sug: subj: Information theory Other Activities Related to Credit Intermediation Weber-Fechner law Assignment problems (Programming) Reinforcement learning Credit laws keyword: cue competition operant conditioning Pavlovian conditioning reinforcement learning time scale invariance cue competition operant conditioning Pavlovian conditioning reinforcement learning time scale invariance ab: Contingency is a critical concept for theories of associative learning and the assignment of credit problem in reinforcement learning. Measuring and manipulating it has, however, been problematic. The information-theoretic definition of contingency—normalized mutual information—makes it a readily computed property of the relation between reinforcing events, the stimuli that predict them and the responses that produce them. When necessary, the dynamic range of the required temporal representation divided by the Weber fraction gives a psychologically realistic plug-in estimates of the entropies. There is no measurable prospective contingency between a peck and reinforcement when pigeons peck on a variable interval schedule of reinforcement. There is, however, a perfect retrospective contingency between reinforcement and the immediately preceding peck. Degrading the retrospective contingency by gratis reinforcement reveals a critical value (.25), below which performance declines rapidly. Contingency is time scale invariant, whereas the perception of proximate causality depends—we assume—on there being a short, fixed psychologically negligible critical interval between cause and effect. Increasing the interval between a response and reinforcement that it triggers degrades the retrograde contingency, leading to a decline in performance that restores it to at or above its critical value. Thus, there is no critical interval in the retrospective effect of reinforcement. We conclude with a short review of the broad explanatory scope of information-theoretic contingencies when regarded as causal variables in conditioning. We suggest that the computation of contingencies may supplant the computation of the sum of all future rewards in models of reinforcement learning. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|