Conceptual short-term memory: a new tool for understanding perception, cognition, and consciousness

Henry Shevlin, Research Associate, Leverhulme Centre for the Future of Intelligence at The University of Cambridge

The notion of memory, as used in ordin­ary lan­guage, may seem to have little to do with per­cep­tion or con­scious exper­i­ence. While per­cep­tion informs us about the world as it is now, memory almo­­st by defi­­nition tells us about the past. Similarly, where­as con­scious exper­i­ence seems like an ongo­ing, occur­rent phe­nomen­on, it’s nat­ur­al to think of memory as being more like an inert store of inform­a­tion, access­ible when we need it but cap­able of lying dormant for years at a time.

However, in con­tem­por­ary cog­nit­ive sci­ence, memory is taken to include almost any psy­cho­lo­gic­al pro­cess that func­tions to store or main­tain inform­a­tion, even if only for very brief dur­a­tions (see also James, 1890). In this broad­er sense of the term, con­nec­tions between memory, per­cep­tion, and con­scious­ness are appar­ent. After all, some mech­an­ism for the short-term reten­tion of inform­a­tion will be required for almost any per­cep­tu­al or cog­nit­ive pro­cess, such as recog­ni­tion or infer­ence, to take place: as one group of psy­cho­lo­gists put it, “stor­age, in the sense of intern­al rep­res­ent­a­tion, is a pre­requis­ite for pro­cessing” (Halford, Phillips, & Wilson, 2001). Assuming, then, as many the­or­ists do, that per­cep­tion con­sists at least partly in the pro­cessing of sens­ory inform­a­tion, short-term memory is likely to have an import­ant role to play in a sci­entif­ic the­ory of per­cep­tion and per­cep­tu­al exper­i­ence.

In this lat­ter sense of memory, two major forms of short-term store have been widely dis­cussed in rela­tion to per­cep­tion and con­scious­ness. The first of these is the vari­ous forms of sens­ory memory, and in par­tic­u­lar icon­ic memory. Iconic memory was first described by George Sperling, who in 1960 demon­strated that large amounts of visu­ally presen­ted inform­a­tion were retained for brief inter­vals, far more than sub­jects were able to actu­ally util­ize for beha­viour dur­ing the short win­dow in which they were avail­able (Figure 1). This phe­nomen­on, dubbed par­tial report superi­or­ity, was brought to the atten­tion of philo­soph­ers of mind via the work of Fred Dretske (1981) and Ned Block (1995, 2007). Dretske sug­ges­ted that the rich but incom­pletely access­ible nature of inform­a­tion presen­ted in Sperling’s paradigm was a mark­er of per­cep­tu­al rather than cog­nit­ive pro­cesses. Block sim­il­arly argued that sens­ory memory might be closely tied to per­cep­tion, and fur­ther, sug­ges­ted that such sens­ory forms of memory could serve as the basis for rich phe­nom­en­al con­scious­ness that ‘over­flowed’ the capa­city for cog­nit­ive access.

A second form of short-term term that has been widely dis­cussed by both psy­cho­lo­gists and philo­soph­ers is work­ing memory. Very roughly, work­ing memory is a short-term inform­a­tion­al store that is more robust than sens­ory memory but also more lim­ited in capa­city. Unlike inform­a­tion in sens­ory memory, which must be cog­nit­ively accessed in order to be deployed for vol­un­tary action, inform­a­tion in work­ing memory is imme­di­ately poised for use in such beha­viour, and is closely linked to notions such as cog­ni­tion and cog­nit­ive access. For reas­ons such as these, Dretske seemed inclined to treat this kind of capacity-limited pro­cess as closely tied or even identic­al to thought, a sug­ges­tion fol­lowed by Block.[1] Psychologists such as Nelson Cowan (2001: 91) and Alan Baddeley (2003: 836) take encod­ing in work­ing memory to be a cri­terion of con­scious­ness, while glob­al work­space the­or­ists such as Stanislas Dehaene (2014: 63) have regarded work­ing memory as intim­ately con­nec­ted – if not identic­al – with glob­al broad­cast.[2]

The fore­go­ing sum­mary is over-simplistic, but hope­fully serves to motiv­ate the claim that sci­entif­ic work on short-term memory mech­an­isms may have import­ant roles to play in under­stand­ing both the rela­tion between per­cep­tion and cog­ni­tion and con­scious exper­i­ence. With this idea in mind, I’ll now dis­cuss some recent evid­ence for a third import­ant short-term memory mech­an­ism, namely Molly Potter’s pro­posed Conceptual Short-Term Memory. This is a form of short-term memory that serves to encode not merely the sens­ory prop­er­ties of objects (like sens­ory memory), but also higher-level semant­ic inform­a­tion such as cat­egor­ic­al iden­tity. Unlike sens­ory memory, it seems some­what res­ist­ant to inter­fer­ence by the present­a­tion of new sens­ory inform­a­tion; where­as icon­ic memory can be effaced by the present­a­tion of new visu­al inform­a­tion, CSTM seems some­what robust. In these respects, it is sim­il­ar to work­ing memory. Unlike work­ing memory, how­ever, it seems to have both a high capa­city and a brief dur­a­tion; inform­a­tion in CSTM that is not rap­idly accessed by work­ing memory is lost after a second or two (for a more detailed dis­cus­sion, see Potter 2012).

Evidence for CSTM comes from a range of paradigms, only two of which I dis­cuss here (inter­ested read­ers may wish to con­sult Potter, Staub, & O’Connor, 2004; Grill-Spector and Kanwisher, 2005; and Luck, Vogel, & Shapiro, 1996). The first par­tic­u­larly impress­ive demon­stra­tion is a 2014 exper­i­ment examin­ing sub­jects’ abil­ity to identi­fy the pres­ence of a giv­en semant­ic tar­get (such as “wed­ding” or “pic­nic”) in a series of rap­idly presen­ted images (see Figure 2).

A num­ber of fea­tures of this exper­i­ment are worth emphas­iz­ing. First, sub­jects in some tri­als were cued to identi­fy the pres­ence of a tar­get only after present­a­tion of the images, sug­gest­ing that their per­form­ance did indeed rely on memory rather than merely, for example, effect­ive search strategies. Second, a rel­at­ively large num­ber of images were dis­played in quick suc­ces­sion, either 6 or 12, in both cases lar­ger than the nor­mal capa­city of work­ing memory. Subjects’ per­form­ance in the 12-item tri­als was not drastic­ally worse than in the 6‑item tri­als, sug­gest­ing that they were not rely­ing on nor­mal capacity-limited work­ing memory alone. Third, because the images were dis­played one after anoth­er in the same loc­a­tion in quick suc­ces­sion, it seems unlikely that they were rely­ing on sens­ory memory alone; as noted earli­er, sens­ory memory is vul­ner­able to over­writ­ing effects. Finally, the fact that sub­jects were able to identi­fy not merely the pres­ence of cer­tain visu­al fea­tures but the pres­ence or absence of spe­cif­ic semant­ic tar­gets sug­gests that they were not merely encod­ing low-level sens­ory inform­a­tion about the images, but also their spe­cif­ic cat­egor­ic­al iden­tit­ies, again telling against the idea that sub­jects’ per­form­ance relied on sens­ory memory alone.

Another rel­ev­ant exper­i­ment for the CSTM hypo­thes­is is that of Belke et al. (2008). In this exper­i­ment, sub­jects were presen­ted with a single array of either 4 or 8 items, and asked wheth­er a giv­en cat­egory of pic­ture (such as a motor­bike) was present. In some tri­als in which the tar­get was absent, a semantic­ally related dis­tract­or (such as a motor­bike hel­met) was present instead. The sur­pris­ing res­ult of this exper­i­ment, which involved an eye-tracking cam­era, was that sub­jects reli­ably fix­ated upon either tar­gets or semantic­ally related dis­tract­ors with their ini­tial eye move­ments, and were just as likely to do wheth­er the arrays con­tained 4 or 8 items, and even when assigned a cog­nit­ive load task before­hand (see Figure 3).

Again, these res­ults argu­ably point to the exist­ence of some fur­ther memory mech­an­ism bey­ond sens­ory memory and work­ing memory: if sub­jects were rely­ing on work­ing memory to dir­ect their eye move­ments, then one would expect such move­ments to be sub­ject to inter­fer­ence from the cog­nit­ive load, where­as the hypo­thes­is that sub­jects were rely­ing on exclus­ively sens­ory mech­an­isms runs into the prob­lem that such mech­an­isms do not seem to be sens­it­ive to high-level semant­ic prop­er­ties of stim­uli such as their spe­cif­ic cat­egory iden­tity, where­as in this tri­al, sub­jects’ eye move­ments were sens­it­ive to just such semant­ic prop­er­ties of the items in the array.[3]

Interpretation of exper­i­ments such as these is a tricky busi­ness, of course (for a more thor­ough dis­cus­sion, see Shevlin 2017). However, let us pro­ceed on the assump­tion that the CSTM hypo­thes­is is at least worth tak­ing ser­i­ously, and that there may be some high-capacity semant­ic buf­fer in addi­tion to more widely accep­ted mech­an­isms such as icon­ic memory and work­ing memory. What rel­ev­ance might this have for debates in philo­sophy and cog­nit­ive sci­ence? I will now briefly men­tion three such top­ics. Again, I will be over­sim­pli­fy­ing some­what, but my goal will be to out­line some areas where the CSTM hypo­thes­is might be of interest.

The first such debate con­cerns the nature of the con­tents of per­cep­tion. Do we merely see col­ours, shapes, and so on, or do we per­ceive high-level kinds such as tables, cats, and Donald Trump (Siegel, 2010)? Taking our cue from the data on CSTM, we might sug­gest that this ques­tion can be reframed in terms of which forms of short-term memory are genu­inely per­cep­tu­al. If we take there to be good grounds for con­fin­ing per­cep­tu­al rep­res­ent­a­tion to the kinds of rep­res­ent­a­tions in sens­ory memory, then we might be inclined to take an aus­tere view of the con­tents of exper­i­ence. By con­trast, if the kind of pro­cessing involved in encod­ing in CSTM is taken to be a form of late-stage per­cep­tion, then we might have evid­ence for the pres­ence of high-level per­cep­tu­al con­tent. It might reas­on­ably be objec­ted that this move is merely ‘kick­ing the can down the road’ to ques­tions about the perception-cognition bound­ary, and does not by itself resolve the debate about the con­tents of per­cep­tion. However, more pos­it­ively, this might provide a way of ground­ing largely phe­nomen­o­lo­gic­al debates in the more con­crete frame­works of memory research.

A second key debate where CSTM may play a role con­cerns the pres­ence of top-down effects on per­cep­tion. A copi­ous amount of exper­i­ment­al data (dat­ing back to early work by psy­cho­lo­gists such as Perky, 1910, but pro­lif­er­at­ing espe­cially in the last two dec­ades) has been pro­duced in sup­port of the idea that there are indeed ‘top-down’ effects on per­cep­tion, which in turn has been taken to sug­gest that our thoughts, beliefs, and desires can sig­ni­fic­antly affect how the world appears to us. Such claims have been force­fully chal­lenged by the likes of Firestone and Scholl (2015), who have sug­ges­ted that the rel­ev­ant effects can often be explained in terms of, for example, post­per­cep­tu­al judg­ment rather than per­cep­tion prop­er.

However, the CSTM hypo­thes­is may again offer a third com­prom­ise pos­i­tion. By dis­tin­guish­ing core per­cep­tu­al pro­cesses (namely those that rely on sens­ory buf­fers such as icon­ic memory) from the kind of later cat­egor­ic­al pro­cessing per­formed by CSTM, there may be oth­er pos­i­tions avail­able in the inter­pret­a­tion of alleged cases of top-down effects on per­cep­tion. For example, Firestone and Scholl claim that many such res­ults fail to prop­erly dis­tin­guish per­cep­tion from judg­ment, sug­gest­ing that, in many cases, exper­i­ment­al­ists’ res­ults can be inter­preted purely in terms of strictly cog­nit­ive effects rather than as involving effects on per­cep­tu­al exper­i­ence. However, if CSTM is a dis­tinct psy­cho­lo­gic­al pro­cess oper­at­ive between core per­cep­tu­al pro­cesses and later cent­ral cog­nit­ive pro­cesses, then appeals to things such as ‘per­cep­tu­al judg­ments’ may be bet­ter foun­ded than Firestone and Scholl seem to think. This would allow us to claim that at least some putat­ive cases of top-down effects went bey­ond mere post­per­cep­tu­al judg­ments while also respect­ing the hypo­thes­is that early vis­ion is encap­su­lated; see Pylyshyn, 1999).

A final debate in which CSTM may be of interest is the ques­tion of wheth­er per­cep­tu­al exper­i­ence is rich­er than (or ‘over­flows’) what is cog­nit­ively accessed. As noted earli­er, Ned Block has argued that inform­a­tion in sens­ory forms of memory may be con­scious even if it is not accessed – or even access­ible – to work­ing memory (Block, 2007). This would explain phe­nom­ena such as the appar­ent ‘rich­ness’ of exper­i­ence; thus if we ima­gine stand­ing in Times Square, sur­roun­ded by chaos and noise, it is phe­nomen­o­lo­gic­ally tempt­ing to think we can only focus on and access a tiny frac­tion of our ongo­ing exper­i­ences at any one moment. A com­mon chal­lenge to this kind of claim is that it threatens to divorce con­scious­ness from per­son­al level cog­nit­ive pro­cessing, leav­ing us open to extreme pos­sib­il­it­ies such as the ‘pan­psych­ic dis­aster’ of per­petu­ally inac­cess­ible con­scious exper­i­ence in very early pro­cessing areas such as the LGN (Prinz, 2007). Again, CSTM may offer a com­prom­ise pos­i­tion. As noted earli­er, the capa­city of CSTM does indeed seem to over­flow the sparse resources of work­ing memory. However, it also seems rely on per­son­al level pro­cessing, such as an individual’s store of learned cat­egor­ies. Thus one new pos­i­tion, for example, might claim that inform­a­tion must at least reach the stage of CSTM to be con­scious, thus allow­ing that per­cep­tu­al exper­i­ence may indeed over­flow work­ing memory while also rul­ing it out in early sens­ory areas.

These are all bold sug­ges­tions in need of extens­ive cla­ri­fic­a­tion and argu­ment, but it is my hope that I have at least demon­strated to the read­er how CSTM may be a hypo­thes­is of interest not merely to psy­cho­lo­gists of memory, but also those inter­ested in broad­er issues of men­tal archi­tec­ture and con­scious­ness. And while I should also stress that CSTM remains a work­ing hypo­thes­is in the psy­cho­logy of memory, it is one that I think is worth explor­ing on grounds of both sci­entif­ic and philo­soph­ic­al interest.

 

REFERENCES:

Baddeley, A.D. (2003). Working memory: Looking back and look­ing for­ward.Nature Reviews Neuroscience, 4(10), 829–839.

Belke, E., Humphreys, G., Watson, D., Meyer, A. and Telling, A., (2008). Top-down effects ofse­mant­ic know­ledge in visu­al search are mod­u­lated by cog­nit­ive but not per­cep­tu­al load. Perception & Psychophysics, 70 8, 1444 – 1458.

Bergström, F., & Eriksson, J. (2014). Maintenance of non-consciously presen­ted inform­a­tion engages the pre­front­al cor­tex. Frontiers in Human Neuroscience 8:938.

Block, N. (2007). Consciousness, Accessibility, and the Mesh Between Psychology and Neuroscience, Behavioral and Brain Sciences 30, pp. 481–499.

Cowan, N., (2001). The magic­al num­ber 4 in short-term memory: A recon­sid­er­a­tion of men­tal stor­age capa­city. Behavioral and Brain Sciences 241, 87–114.

Dehaene, S. (2014). Consciousness and the Brain: Deciphering How the Brain Codes Our Thoughts. Viking Press, 2014.

Dretske, F. (1981). Knowledge and the Flow of Information. MIT Press.

Firestone, C., & Scholl, B.J. (2015). Cognition does not affect per­cep­tion: Evaluating the evid­ence for ‘top-down’ effects. Behavioral and Brain Sciences:1–77.

Grill-Spector, K., & Kanwisher, N. (2005). Visual Recognition. Psychological Science, 16(2), 152–160.

Halford, G. S., Phillips, S., & Wilson, W. H. (2001). Processing capa­city lim­its are not explained by stor­age lim­its. Behavioral and Brain Sciences 24 (1), 123–124.

James, W. (1890). The Principles of Psychology. Dover Publications.

Luck, S. J., Vogel, E. K., & Shapiro, K. L. (1996). Word mean­ings can be accessed but not repor­ted dur­ing the atten­tion­al blink. Nature, 383(6601), 616–618.

Ma, W. J., Husain, M., & Bays, P. M. (2014). Changing con­cepts of work­ing memory. Nature Neuroscience, 17(3), 347–356.

Potter, M. C. (2012). Conceptual Short Term Memory in Perception and Thought. Frontiers in Psychology, 3:113.

Potter, M. C., Staub, A., & O’Connor, D. H. (2004). Pictorial and con­cep­tu­al rep­res­ent­a­tion of glimpsed pic­tures. Journal of Experimental Psychology: Human Perception and Performance, 30, 478–489.

Prinz, J. (2007). Accessed, access­ible, and inac­cess­ible: Where to draw the phe­nom­en­al line. Behavioral and Brain Sciences, 305–6.

Pylyshyn, Z. (1999). Is vis­ion con­tinu­ous with cog­ni­tion?: The case for cog­nit­ive impen­et­rab­il­ity of visu­al per­cep­tion. Behavioral and Brain Sciences, 22(03).

Shevlin, H. (2017). Conceptual Short-Term Memory: A Missing Part of the Mind? Journal of Consciousness Studies, 24, No. 7–8.

Siegel, S. (2010). The Contents of Visual Experience. Oxford.

Sperling, G. (1960). The Information Available in Brief Visual Presentations, Psychological Monographs: General and Applied 74, pp. 1–29.

 

[1] Note that Dretske does not use the term work­ing memory in this con­text, but clearly has some such pro­cess in mind, as made clear by his ref­er­ence to capacity-limited mech­an­isms for extract­ing inform­a­tion.

[2] A com­plic­at­ing factor in dis­cus­sion of work­ing memory comes from the recent emer­gence of vari­able resource mod­els of work­ing memory (Ma et al., 2014) and the dis­cov­ery that some forms of work­ing memory may be able to oper­ate uncon­sciously (see, e.g., Bergström & Eriksson, 2014).

[3] Given that the arrays remained vis­ible to sub­jects through­out the exper­i­ment, one might won­der why this exper­i­ment has rel­ev­ance for our under­stand­ing of memory. However, as noted earli­er, I take it that any short-term pro­cessing of inform­a­tion pre­sumes some kind of under­ly­ing tem­por­ary encod­ing mech­an­ism.