Describing Inner Experience?
Proponent Meets Skeptic
Russell T. Hurlburt
Eric Schwitzgebel
In Press
The MIT Press
Anticipated Summer 2007
Prepublication version
Do not quote without permission of the authors.
Part Three
Reflections
Chapter Ten
Eric’s Reflections
Eric Schwitzgebel
Melanie makes a number of interesting claims in these interviews – claims which, if true, reveal much about one person’s stream of conscious experience. But the question is, are her claims true? What license do we have to believe them? In my mind, this is the first and most central question that must be answered.
Let’s grant this from the outset: Melanie is a sincere and conscientious subject, Russ a careful and even-handed interviewer. What they deliver is probably about as good as can reasonably be expected from open interviews about sampled experiences. If we reject it, we reject the method in general – and in its wake surely also a plethora of related but less careful approaches. We then either resign in defeat or face the difficult task of specifying some better way to garner reports about spontaneously generated emotion, imagery, and the like. If, on the other hand, we are justified in accepting what Melanie says about her experience, then perhaps, by repetitions of this method, we can make some headway in the vexed field of consciousness studies. In the merit or failure of these interviews, we can glimpse a possible future of the discipline.
My position is this. We should tentatively accept the most basic claims Melanie makes about her experiences, pending further evidence. However, we should view the details she provides, even plausible details confidently asserted, with a high degree of skepticism. So, for example, in Beep 5.1, I think we should tentatively accept – as more likely to be true than not – that Melanie had visual imagery of an intersection and also a feeling or recognition of anxiety sometime roughly around the moment of the beep. We should, I suggest, accept this tentatively, barring countervailing evidence. (Such evidence is not available in this case but could include such things as later recantations or physiological or facial measures suggestive of a different emotional state.) However, even without specific countervailing evidence, I think we should be very wary of the details. I don’t think we should accept, even tentatively, what Melanie says about the specifics of the image, about the level of detail in the image, about whether she was actually feeling anxiety at the moment of the beep (as opposed to just “knowing” that she was anxious without an anxious feeling), about what this knowledge or feeling of anxiety was like, about whether she is right to deny the presence of other experiences at the moment, etc. We should, I think, withhold judgment about the accuracy or inaccuracy of such assertions, absent further physiological or behavioral evidence of some sort for or against them. The details of Melanie’s reports may be true. But, without some further corroboration, we should not cite them as serious support for particular philosophic or scientific theses about the nature of experience – for example, in defense of a particular account of imagery or emotion. They are, at best, merely suggestive.
I regard this as a moderate view, and the course I would chart for consciousness studies in light of it is a cautious and pluralistic one – neither a wholesale rejection of Russ’s experience sampling, nor the elevation of it over previous approaches. The field is for now, I think, in the unenviable position of possessing a stable of suggestive but unreliable (or at least unproven) methods, to none of which we can harness full scientific confidence.
1. We Have Not
Established the Validity of Russ’s Interview Method
Russ rejects my cautious pluralism because he believes his approach to the study of conscious (or “inner”) experience gives substantially more faithful access to experience than does any other contemporary scientific approach. He believes, if I understand him correctly, that we should largely disregard the accounts of experience given by other contemporary scientific methods because he thinks they don’t adequately manage the methodological problems DES is designed to avoid. Of course, as we mentioned in the opening chapters, many philosophers and psychologists over the centuries have claimed they possessed singularly trustworthy methods of studying consciousness. The contradictory results arising from this diversity of methods show that many such claims must be false. The burden of proof is squarely on Russ to show that his method, unlike the others, does in fact merit our trust. In my view, Russ has not shown this.
Russ has emphasized the advantages of obtaining an arbitrary, brief sample of experience, reflected on immediately after it occurs. He has emphasized the advantages of not forcing that experience into a preconceived structure and of restraining the interviewee from making general claims or claims about causation. He has shown in the interview chapters that he is capable of soliciting reports without palpable bias. Melanie, for her part, makes interesting assertions about her experience, assertions that are not obviously self-contradictory and don’t crumble into an uninterpretable mess when she is asked to elaborate. This is all good. But it still falls a considerable distance short of showing that we should, as a general matter, accept the deliverances of Russ’s method. We need, in addition, some sort of external corroboration. That is, we need to find evidence not grounded solely in interviews of this sort that sheds light on the accuracy or inaccuracy of Melanie’s reports. And this book, of course, presents nothing of the sort. It records an exploration, not a verification.
A measurement technique may require external corroboration at the outset without remaining forever hostage to, and judged inferior to, the sources of evidence that first help establish its validity. A scientist intends to create an extraordinarily precise thermometer, let’s say. She has good theoretical reason to anticipate outstanding accuracy. Yet she will not accept its deliverances immediately. She compares its measurements with the measurements of other, cruder, thermometers she already trusts to some extent. If it’s too far off, she has cause for concern. She puts the thermometer in a situation where she would expect, theoretically, a very slight rise in temperature – a rise perhaps unmeasurable by earlier thermometers – and hopes her new device registers it. If her device passes enough such tests, she may go back and use it to correct or displace her older thermometers or to revise some of the theories she initially used in testing it. Corroboration doesn’t imply subservience. It is no objection to the demand for corroboration that the method in question will likely prove superior to the prior methods (and theories grounded in those methods) to be used in corroborating it. In science, few methods command trust without independent corroboration, at least at first.
Consequently,
even if we had excellent reason to think Russ’s method superior to all prior
methods, prudence dictates that we compare its results to the results of those
other methods (as the measurements of the new thermometer were compared to the measurements
of old thermometers) and that we check its results against what can be
theoretically predicted or retrodicted (as the thermometer was checked to see
if it recorded the predicted slight rise in temperature). Direct verification
of the first sort is beyond the scope of this book: We employ no independent
means of measuring Melanie’s experience. Russ does offer some corroboration of
the second sort in Chapter Two when he discusses features of Fran’s behavior
that seem to support her unusual introspective reports. (I find the case of
Robert less compelling, for reasons described in
Perhaps we wouldn’t need such external support if Russ’s method had no flaws, left no sizable space for error to enter, was indisputably massively superior to all the preceding methods that have produced uneven (but sometimes interesting) results. However, as I’m sure Russ would agree, his method is not as resplendent as that. Even if, in the end, we decide it is better than all preceding methods, granting exemption from the general requirement of external corroboration is extreme.
Russ will dispute with me the extent to which his method leaves room for distortions due to experimenter bias and situational pressures, even with a skilled interviewer (more on this later). However, I think he cannot reasonably dispute that his method (like many others) leaves considerable room for errors of memory and communication and for distortions due to the preconceptions and reconstructions of the subject. Even if there is no significant memory issue in the minute or so after the beep, when the subject is first reflecting on her sampled experience (which is very optimistic), the interview itself is conducted up to 24 hours later. The interviews touch on many details the subject did not explicitly record, or possibly even reflect on, immediately after the beep. Surely, there is substantial room for error here.
The interviewer exhorts the subject to set aside preconceptions, to be fully receptive to her experience regardless of how surprising it may be, not to confabulate or reconstruct on the basis of theory, to express uncertainty where it seems appropriate, to be absolutely frank. But of course exhortation alone, though it may be helpful, can’t guarantee that the subject actually attains all these desirable (?) goals. Nor can we be assured that the appearance of frankness, of open-mindedness, of atheoreticity, indicates their actual presence. Indeed, I doubt it’s humanly possible to attain some of these goals even approximately. What would it be to be completely open-minded, atheoretical, unreconstructive in one’s memory and reports? Would that be mere infancy? Don’t we need pre-existing frameworks, categories, theories, causal maps to remember, even to perceive, anything at all – to have anything other than unreportable, immemorable, “blooming, buzzing confusion” (as William James [1890/1981] puts it)? Melanie’s biases and preconceptions can’t but inform her reports. Risks to her accuracy ensue, which may be impossible to disentangle from the benefits.
Let’s contrast Russ’s method with the archetypal method of introspective psychology as practiced by Titchener and others a century ago. The latter method generally involved setting up a controlled situation with precisely measurable stimuli (color plates, in constant lighting conditions, for example, viewed at a constant distance and angle). Practiced introspective observers reported on their experience as it occurred or immediately afterwards, and in cases of uncertainty, or for verification, the stimuli could be repeated. This method has some of the same virtues as Russ’s method, including that it targets specific, brief episodes after only a short (or no) delay. Like Russ, Titchener and other introspective psychologists exhorted their observers to set aside their presuppositions. Also like Russ, they generally attempted to reduce or disarm their own expectations. Experiences weren’t sampled arbitrarily from everyday life, however, and Titchener’s observers were surely affected by the experimental set-up, by the expectation of experiences of a certain sort (e.g., visual experiences of varying hue), and by the potentially distracting or distorting knowledge that they would shortly be reporting on those experiences. On the other hand, conditions were better controlled and the observers’ reports more easily allowed for certain sorts of verification (e.g., checks for consistency with what’s theoretically predicted; see Titchener 1901-1905; Schwitzgebel, 2004, 2005). Most importantly, perhaps, the memory demands in Titchener’s studies were not nearly as great as in Russ’s. Titchener generally asked his observers only to report one aspect of their experience, very swiftly. He didn’t ask them to reflect on the experience as a whole. Thus, observers didn’t require several minutes to generate their reports, as Russ’s subjects often do. (Nor were Titchener’s observers interrupted by the task of turning off a beeper, retrieving pen and paper, etc.) They could focus on making an instant judgment about a single thing. And of course, Titchener’s observers generated their final reports on the spot, not after an interview the next day. Titchener also emphasized his preference for trained observers, with considerable introspective experience. Maybe trained observers have more theoretical commitments and bias than observers who enter untrained – but it’s not clear that this is so. Titchener stressed that even untrained observers are prone to preconceptions and theories about their experiences and often leap to generalizations quickly after one or a few trials (Titchener, 1899, 1901-1905, 1912).
Or consider the armchair phenomenological investigations of contemporary philosophers such as Charles Siewert (1998, 2006) and Terry Horgan (Horgan & Tienson, 2002; Horgan, Tienson, & Graham, 2003). Siewert (forthcoming-a, forthcoming-b) is particularly explicit about his method, which he calls “plain phenomenology.” He urges phenomenologists to reflect repeatedly and patiently on both their ordinary lived experience and on particular types of invoked experience. He asks them to take special care in drawing conceptual distinctions, to bear in mind the theoretical implications, and to consider a variety of related and nearby cases before reaching their final judgment. For example, Siewert (1998, ch. 8) invites the reader to reflect on the difference, if any, between the experience of imagining an “M” tilted on its side and the experience of imagining the Greek letter sigma. Are these imagery experiences the same or different, and in what respects? Must there be a difference in imagined shape for a difference in imagery experience? To what extent does it seem that such imagery experiences vary with, and depend on, one’s intentions and concepts? Maybe Siewert’s approach risks, more than Russ’s approach, importing the theories of observers invested in particular answers. On the other hand, careful theoretical reflection and the consideration of nearby contrasting cases may also help forestall confusion. The method runs comparatively greater risks than does DES of unrepresentative selection, and of potentially severe and undetectable distortion of the experience by the act of introspectively reflecting on it as it occurs. But on the other hand, wise selection may help us better appreciate subtle contrasts and discern issues of theoretical import. Furthermore, concurrent introspective reflection reduces or eliminates problems of memory inherent in reflecting on experiences already past, and it may allow us to slow down and focus better on detail.
It’s by no means clear a priori whether Titchener’s introspective methodology and contemporary philosophical armchair phenomenological reflection, which Russ rebuke, contain more potential for error than Russ’s own methodology. Each has its apparent strengths and shortcomings. Maybe Russ’s method will find compelling external corroboration that warrants its elevation over other introspective approaches, but absent such corroboration, I see no reason to regard DES as vastly superior to other methods, with their flawed and divergent results.
2. Should We Credit Melanie’s
Reports at All?
We might, then, put every introspective method, Russ’s, Titchener’s, Siewert’s, and all others, on an equal footing: prone to obvious sources of error, inconsistent in their results, relatively uncorroborated, unworthy of scientific credence. Proper scientific caution, we might think, demands that we discard everything Melanie says, pending positive and robust evidence that we’re on firm ground.
The problem with this approach is that no swift and decisive corroboration or disconfirmation is in the cards for any method of studying experience. Insisting on firm ground thus means abandoning the theoretical exploration of consciousness. Of course, we should at least try, more than we do, to find external corroborations of subjective descriptions of experience and to illuminate the conditions under which such descriptions are credible. But the results of any such attempts will inevitably be controversial and difficult to interpret for some time to come. As I mentioned in discussing Russ’s disagreement with Flavell about how to interpret his children’s denial of the experience of thinking (Ch. 3.2), there’s just too much room to posit whatever experience best supports our theory or conforms to our favorite method.
I see no reason to think the task Russ sets Melanie is absolutely impossible. People must have at least some inkling of what’s going on in their own present and immediately past conscious experience. That inkling is, I think, surprisingly poor and unstable (as I’ve argued in other work), but it would be a radical skepticism indeed to suppose that we have no clue whatsoever about the ongoing flow of our experience. Asking people about their present or immediately past experience is not entirely pointless. Suppose someone judges himself to have just been (consciously) thinking about his plans for Saturday. Suppose also the usual sources of error in judgment are minimal, as far as we can tell. It seems churlish not to give him at least tentative credit.
Russ’s method builds on that fundamental credibility. Although the interview isn’t conducted until hours later, basic features of the experience are explicitly noted within a short time. I see no reason to think that such basic features couldn’t, in general, be accurately recalled in the later interview, especially with a notepad as a cue. Russ allows the subject to approach the task in her own terms, solicits a report without overt pressure toward any particular outcome, discourages mere hypothesizing. It again seems churlish, a mere stance, to give her no credit whatsoever, absent some specific reason for skepticism. Minimally, let’s say, it seems in most cases more likely than not that the basic topics of thought or reflection that Melanie reports (e.g., her chair in 1.1, scuba diving in 4.1, her appointment in 5.1) were indeed in consciousness somewhere around the time of the beep. Maybe also (more questionably perhaps) those topics were present in roughly the modes she describes (inner speech or hearing in 1.1 [but for cautions about this case in particular, see Russ’s comments in the next chapter], bodily imagery or feeling in 4.1, visual imagery in 5.1).
Given the uncertain state of consciousness studies and the lack of any well-established general methods, to endorse a blanket skepticism about about such matters exhibits a misguided and crippling purism. However, I do think a blanket skepticism may be in order regarding the details of Melanie’s reports, unless we find further corroboration of them – corroboration either of those reports in particular or of the validity of Russ’s experience sampling method in general. I’ll develop this idea in Sections Four and beyond. But first I’ll describe an experiment of my own. Perhaps this experiment – an awkward experiment, I confess – can in an imperfect way illustrate some of the untapped potential in experience sampling.
3. Adapting Russ’s
Methodology to Explore the Richness of Experience
The
experiment addresses the “richness” of experience – the extent to which we have
constant experience in a variety of modalities. According to the rich
view, we have constant visual experience (at least when our eyes are open,
maybe also when they’re closed), constant tactile experience (for example of
our clothes against skin), constant auditory experience, maybe constant
emotional experience, conscious thought, imagery, etc. – all simultaneously
(see
What
I called the “refrigerator light phenomenon” in
I divided 21 subjects (11 philosophy graduate students and 10 non-philosophers) into five roughly balanced groups (for more methodological detail, as well as considerable self-critique, see Schwitzgebel, forthcoming-c). With one group, I did something like Russ’s DES procedure (less expertly, I’m sure), but with a few modifications: First, I avoided the phrase “inner experience,” which I worried might be interpreted as emphasizing imagery, thoughts, emotions, and the like over (“outer”?) sensory experience. Second, I spent some time explaining what’s meant by “consciousness” or “experience” or “phenomenology,” citing examples of conscious processes (vivid emotions, focal sensory experiences, inner speech) and non-conscious ones (subliminal perception, immune system response). I invited discussion of this topic. Third, in discussing the first sample, once the participant was done reporting the most salient aspects of her experience, I explicitly asked whether there were also other aspects of her experience, giving examples like feelings of hunger or tiredness; visual, auditory, tactile, or olfactory experiences; emotions; visual imagery; conscious thoughts; etc., repeating this question with arbitrary examples of potential experiences until the subject denied recalling anything more. Fourth, generally in discussing the first sample, and always on the first interview day, I mentioned the debate between the rich and the thin view, citing arguments on either side and expressing neutrality on the question. However, I did not particularly emphasize that issue. I encouraged theoretical and methodological discussion on a variety of topics, generally recommending cautious restraint in such matters. Broad and open theoretical discussion was encouraged throughout four days of sampling and interview.
Participants in the other four conditions, unlike those in the first condition, were told explicitly that the purpose of the research was to explore the richness or thinness of experience. They were given an explanation of the debate and some arguments and intuitive examples on both sides of the question, and they were asked for their own initial impressions. Like the first group, they were mostly beeped over four days and invited to reflect on the theoretical and methodological issues pertinent to their reports. Each was asked about one aspect of sensory experience. One group was asked simply to report if they were having visual experience at the time of the beep, and if so what that experience was. Another group was similarly asked about experience in the far right visual field, another group about tactile experience, another about tactile experience in the left foot. They collected their samples with these types of experiences in mind, instructed to make a first judgment of “yes, I had such an experience” or “no, I didn’t” (or “maybe” or “sort of”) as quickly as possible after each beep. Participants were repeatedly assured that it was fine if they had no experiences of the sort in question – that that would be nice evidence for the thin view – and conversely that it would be fine, and good evidence for the rich view, if they found such experiences in every single sample. Participants who leaned toward one view were periodically reminded of the viability of the other view. Occasionally, a participant who claimed to have had an experience of the sort under study was pressed about whether there really was such an experience, or whether she was just reporting some external object in the visual or tactile environment. Conversely, participants who claimed to be having no experience of the sort under study were occasionally pressed about whether they really thought there was no experience, as opposed to merely vague or secondary or peripheral experience.
Every participant in the full experience condition (resembling DES) and the two visual experience conditions reported some sort of visual experience in most samples – even those initially inclined toward a thin view, with no obvious difference or trend between the three conditions. A majority (8 out of 13) reported visual experience in every single sample. In contrast, participants in the full experience and full tactile experience conditions reported tactile experience in only about half to three-quarters of all samples (depending on how liberally one interprets “tactile” – e.g., whether pain and proprioceptive experiences count). Somewhat lower rates of experience were reported in the far right visual field and the tactile left foot conditions – though even the “thinnest” participant in the tactile left foot condition confidently reported tactile left foot experience in 2 of her 19 samples. How exactly to interpret these results is a complicated and uncertain matter that I can only partially explore here. However, if we credit the participants’ reports, overall they seem more supportive of a moderate view than either a rich or a thin view.
Russ will surely say that in conducting the interviews as I did, I allowed my own biases and presuppositions to inform the results. I acutely feel the merits of this objection. Here are a few not entirely sufficient responses to that concern: (1.) At the end of my time with each participant, I asked her to guess whether I personally leaned toward the rich or the thin view. Subjects were divided on this question, generally saying they felt I was even-handed. (2.) The asymmetry of response between the visual and tactile conditions suggests that situational pressures creating a general bias toward reporting sensory experience can’t fully explain the results. (3.) Explicitly discussing the theoretical possibilities and explaining some of the appeal of both sides of an issue may actually be preferable to allowing such issues to pass undiscussed, since it may serve as something of a check on participants’ initial suppositions. Let me also add that since the experiment (except in the DES-like, full experience condition) centers around a single yes/no/maybe question, the memory demands are considerably less severe than in the standard DES format. There’s no several minutes of recalling and describing the experience, no great likelihood of forgetting the key piece of data between the after-beep scribble and the next-day interview.
In our discussion of Beep 1.1, Melanie explicitly denies having visual experience at the moment of the beep, and in general she denies having any more than 2-3 types of experience at any one moment. She doesn’t usually include visual sensory experience in her reports. In these respects, she’s typical of Russ’s subjects and different from mine. This difference could, of course, be due entirely to design flaws in my experiment or to my deficiencies as an interviewer, or it could point to an inexcusable instability in beep-and-interview methods.
Another
possibility is that the difference turns on linguistic or theoretical issues. Russ
uses a variety of terms and phrases to talk about what he’s trying to get at,
all somewhat interchangeably, including “inner experience” (with its hint of
favoring “inner” processes over sensory ones), “attention,” and “awareness”
(see
Russ: So is there anything else going on at this particular moment? You’re seeing the white parchmenty paper…
Melanie: Mm hm.
Russ: And does that seem to be in your awareness, or is it….
Melanie: No it’s not. I’m not aware of how my body is positioned or of what I’m holding. It’s very much just in my head.
Russ: You’re paying much more attention to your thought process here, about “isn’t it strange…?” “isn’t it funny?” You’re obviously seeing the parchment, because that’s what started this process, but it’s not in your awareness.
Melanie: Yes, exactly.
By using “attention”
and “awareness” interchangeably here to
mean “experience,” Russ implicitly suggests that something outside attention is
outside experience, in direct contravention of the rich view. Melanie might
thus be forgiven for interpreting Russ’s questions as about what she is
attentionally focused on or centrally aware of, rather than about a wider
panoply of peripheral experience that may or may not exist. Though Russ’s first
use of the word “attention” in the recorded dialogues is only after Melanie’s
first denial of experience above, its use here conveys an implicit assumption
that may already more subtly have been communicated to Melanie – or even
explicitly communicated in Russ’s initial interview with her, in which he gave
instructions about how to use the beeper. The difficulty may be further
compounded by what I regard as Russ’s frequent blurring of the epistemic
(having to do with knowledge) and phenomenal (having to do with the stream of
experience) senses of “awareness” (see my discussion in
Since Russ avoids general theoretical and terminological discussions, it’s difficult to know exactly how his subjects understand him on such matters. By the time I have the opportunity to raise the question in my own way, Melanie has already committed herself to denying visual experience. (If you think it problematic that Melanie would both deny visual experience and not object to Russ’s statement that she was “obviously seeing,” you probably lean toward the rich view.) After her exchange with me, Melanie’s denial of visual experience becomes more qualified – not absolute, but relative, expressed in words like “primarily” and “much more.” Such a relative claim is of course consistent with the rich view, which generally assumes a focal center of experience and a periphery that is in some way vaguer or less vivid.
Three
more factors may further support Russ’s subjects’ tendency to disregard
peripheral aspects of their experience, if any exist. (1.) Subjects will
naturally tend, after the beep, to focus first on what was central in their
experience. By the time they start to think about whether they were also having
(say) visual experience – if they ever think about that – it may be several
minutes after the beep, and their memory may have faded too much for accurate
recall. (2.) Subjects will have collected six to eight samples per interview
session. Given the details of Russ’s questions, if there is any hope of getting
through a substantial portion of those samples, peripheral aspects of
experience must be excluded. (3.) Russ himself explicitly declares a lack of
interest in peripheral aspects of experience, which he thinks probably can’t be
reported accurately (see
I regard the experiment described in this section as preliminary and exploratory. I don’t entirely trust my own subjects’ reports, and I certainly don’t mean to suggest that the reader should accept that in fact people do have visual experience most of the time and tactile experience about half the time. The results require interpretation and are at best only suggestive. (Schwitzgebel forthcoming-c discusses a variety of concerns about the data, suggesting ways in which they can be reconciled with thinner or, especially, richer views of experience.) However, the debate about the richness of experience has thus far been conducted largely impressionistically, or in terms of questionable general theories of consciousness (e.g., James, 1890/1981; Jaynes, 1976; Dennett, 1991; Searle, 1992; Siewert, 1998). A version of the beep-and-interview method gives us the opportunity to explore the question in a different and maybe better way – a way beyond even what Russ himself envisions. If future researchers discover still other means of exploring this question, and if the results of the various researches appear to converge, then perhaps we will have some solid basis for a scientific opinion.
4. Memory in Introspective and
Eyewitness Testimony
Let me return now to my case for skepticism
about the details of Melanie’s reports. I propose a blanket skepticism about
all but the grossest features of her reported experience. I simply don’t trust Melanie
accurately to remember the details.
Actually, I don’t really trust Melanie’s
descriptions of the grosser features of her experience either, though I’m
willing tentatively to accept them. So let me put the point a bit differently. When
Melanie reports the details of an experience – for example when she describes
the details of an image or attempts to specify her degree of self-consciousness
in feeling an emotion – her reports may accurately reflect the content
of the original experience; but I think it just as likely that the imputed
details are erroneous inventions, arising from her theories and preconceptions
about experience, from situational pressures, from accidents of language, from
distorted and unrepresentative reconstructions formed either shortly after the
beep or during the course of the interview, etc. I’m willing to accept that in
most cases Melanie preserves some rudimentary memory of her experience as it transpired
shortly before the beep, but how that trace is articulated and described in the
course of the interview, the specifics in which it’s dressed, seems to me very
likely to depend as much on factors only tenuously associated, or unassociated,
or even negatively associated, with accuracy, as on genuinely remembered
particulars.
I could be wrong about this. If Russ or
others are able consistently to corroborate reports like Melanie’s down to a
fine level of detail, then we may be justified in accepting all or most of what
Melanie says. Of course, as the field stands now, even the most basic aspects
of DES reports remain uncorroborated and will require considerable effort and
ingenuity to corroborate. Judgments about how far to believe Melanie can only be
speculative.
The task Russ and I set to Melanie invites
comparison to the task of reporting the details of an outwardly witnessed event.
Although Melanie isn’t literally an “eyewitness” of her experience – we see things,
of course, not our experiences of those things – her task bears an
important resemblance to the task of an eyewitness asked to report some
specific event, such as a crime. Like an eyewitness, Melanie is expected to
report details of specific, unique events that she was (presumably!) in good
position to record, as the result of a relatively swift and unmediated process
beginning with those events. A comparison between Melanie’s reports and
eyewitness testimony is inviting because although the literature on the
accuracy of reports of conscious experience is spotty and controversial, the
accuracy of eyewitness testimony has been extensively examined, with some
relatively robust findings. Chief among them: Eyewitness reports are prone to
what most people find to be a surprising degree of error.
Two passages from Elizabeth Loftus’s classic
book on the topic (Loftus, 1979) give the flavor:
Two female students entered a train station, one of them leaving her large bag on a bench while both walked away to check the train schedules. While they were gone, a male student lurked over to the bag, reached in, and pretended to pull out an object and stuff it under his coat. He then walked away quickly. When the women returned, the older one noticed that her bag had been tampered with, and began to cry, “Oh my God, my tape recorder is missing!” She went on to lament that her boss had loaned it to her for a special reason, that it was very expensive, and so on. The two women began to talk to the real eyewitnesses who were in the vicinity. Most were extremely cooperative in offering sympathy and whatever details could be recalled. The older woman asked these witnesses for their telephone numbers “in case I need it for insurance purposes.” Most people gladly gave their number.
One week later an “insurance agent” called the eyewitnesses as part of a routine investigation of the theft. All were asked for whatever details they could remember, and finally, they were asked, “Did you see the tape recorder?” Although there was in fact no tape recorder, over half the eyewitnesses “remembered” seeing it, and nearly all of those could describe it in reasonably good detail. Their descriptions were quite different from one another: some said it was gray and others said black; some said it was in a case, others said it was not; some said it had an antenna, others claimed it did not. Their descriptions indicated a rather vivid “memory” for a tape recorder that was never seen (Loftus 1979, p. 61-62).
And:
… subjects viewed a film of a traffic accident and then answered questions about the accident. Some subjects were asked, “About how fast were the cars going when they smashed into each other?” whereas others were asked, “About how fast were the cars going when they hit each other?” The former question elicited a much higher estimate of speed. One week later the subjects returned and, without viewing the film again, they answered a series of questions about the accident. The critical question was, “Did you see any broken glass?” There was no broken glass in the accident, but because broken glass usually results from accidents occurring at high speed, it seemed likely that the subjects who had been asked the question with the word “smashed” might more often say yes to this critical question. And that is what we found (Loftus, 1979, p. 77-78).
Distortive influences on eyewitness
testimony include information or suggestions built into the wording of
questions, the expectations or theories of the witness, the expectations of the
interviewer, stress, the solidification of guesses or conjectures into
confident assertions as they are repeated over time (“confidence inflation”),
and the confusion of what is imagined with what is remembered, to name a few
(see, for example, Loftus, 1979; Narby, Cutler, & Penrod, 1996; Wells &
Loftus, 2003).
It’s not
surprising, of course, that eyewitness testimony is subject to distortion. We
don’t need a raft of journal articles to tell us that. What is striking,
however, and repeatedly confirmed, is the extent of the distortion. Most
people just don’t expect witnesses to be as badly mistaken or as easily
influenced as they often are. They don’t expect a majority of
eyewitnesses to invent, and describe in detail, a tape recorder they have never
seen. They don’t expect subtle differences in the phrasing of questions to have
the profound effects they often have on witnesses’ reports.
Since the
earliest days of eyewitness research, instructors have been fond of classroom
demonstrations of eyewitness inaccuracy. Münsterberg (1908/1927) describes one
typical classroom demonstration: A shouting match breaks out between two
students; one draws a weapon; the professor intervenes. Immediately afterward,
the professor tells the class that the episode was staged and asks them for a
written account of the events (in the case Münsterberg describes, some students
recounted the events only later). Inevitably in such demonstrations, the
students’ reports are rife with error. Münsterberg writes:
Words were
put into the mouths of men who had been silent spectators during the whole short
episode; actions were attributed to the chief participants of which not the
slightest trace existed; and essential parts of the tragi-comedy were
completely eliminated from the memory of a number of witnesses (1908/1927, p.
50-51).
The students
witnessing such demonstrations are generally quite surprised at the results,
shocked that they and their peers could diverge so widely in their descriptions
of the perpetrators’ height, race, hair color, and clothing, in their
characterization of key events, in almost every feature of the evaporated scene.
This surprise is a crucial pedagogical tool in undermining students’ misplaced
faith in the accuracy of eyewitness testimony (Charlton, 1999; Gee & Dyck,
2000).
Psychologists
have also more formally tested the degree to which people tend to overestimate
eyewitness accuracy. Researchers have, for example, asked undergraduates and
ordinary citizens to read through descriptions of eyewitness testimony
experiments and then to predict the outcome of the experiments. Subjects in
such studies often severely overestimate the accuracy of other subjects’
eyewitness performance (for reviews see Leippe, 1995; Devenport, Penrod, & Cutler,
1997). In another series of studies, Wells and colleagues (e.g., Wells,
Lindsay, &
Indeed,
generally speaking, people seem to be fairly poor at distinguishing accurate
from inaccurate eyewitness testimony, except in extreme cases, such as when a
witness is blatantly self-contradictory or explicitly avows uncertainty
(Leippe, Manion, & Romanczyk, 1992; Leippe, 1995; Devenport, Penrod, & Cutler,
1997). Our poor judgment on this front may spring from a variety of factors. For
example, psychological research suggests that people tend especially to believe
confident eyewitnesses, but that confidence correlates only weakly with
accuracy, or correlates well only in special conditions (e.g., Wells, Lindsay, &
Ferguson, 1979; Wells & Murray, 1984; Bothwell, Deffenbacher, & Brigham,
1987; Sporer, Penrod, Read, & Cutler, 1995; Kassin, Tubb, Hosch, & Memon,
2001; Brewer & Burke, 2002; Weber & Brewer, 2004).
Back to Melanie.
Are we to think her better than a typical eyewitness? What we asked her to
observe was in some sense closer to her than any outward event – but is that
sort of proximity an advantage? In vision, certainly, one can get too close. Things
nearby and essential may nonetheless be only poorly seen and rarely reflected
on – such as one’s eyeglasses. I may talk more coherently about, and reach more
accurate judgments about, the road I’m driving on than the steering wheel I use
to drive on it. (I know the road curves 90 degrees; but can I say how far I
need to rotate the steering wheel to make that turn?) Likewise, even if sensory
phenomenology is in some sense essential to sensory judgment, we may know it
only very poorly. As I pointed out in Chapter Three (along with a variety of
other reasons to doubt the accuracy of our introspective judgments), we
normally observe, attend to, think about, and describe outward events,
not inner ones.
The task we
set Melanie was an alien one – one that strikes many subjects at first as
strange and difficult. Though Melanie gained some practice over the course of
the interviews, it seems unlikely to me that her comfort with the task in the
end should justifiably exceed the ordinary eyewitness’s comfort in reporting
nearby outward events. Immediately after each beep, Melanie knew that she would
need to remember and report the experience in question, but at least some
criminal eyewitnesses (not to mention subjects in eyewitness experiments) are
in a similar position, realizing either immediately after a crime occurs, or
even as it is occurring, that they should remember details for later report. Russ
and I gave Melanie some feedback about her reports, but that feedback consisted
mainly in exhortations to be open-minded, to resist generalizations, and to
attend closely to the beeped moment, coupled with Russ’s general willingness to
accept confidently-asserted declarations about specific episodes and my varying
degrees of theory-laden skepticism. Although our feedback may have had some
limited value, I certainly risked affecting Melanie with my theories and Russ
may have encouraged a kind of blasé confidence by his readiness to accept
confident reports, almost regardless of content. The events Melanie reported
were mostly fleeting – momentary images, passing thoughts. Opportunities
abounded for theory-laden reconstruction, for unintentional confabulation. No
external check or second witness existed to keep Melanie careful and modest.
Here’s a
further point of difference between eyewitness testimony and introspective
report: Normally, when someone witnesses a robbery or car accident, she’ll have
some sort of schema or sense of the world into which they fit. Such events may
be surprising in a certain way, perhaps undermining our expectations and
stereotypes, but they rarely impugn our sense of the possible. In
experience sampling, however, our most basic conceptualizations are often
undermined: We simply must be wrong in much of what we believe about our stream
of experience – if for no other reason than that the massive diversity of
opinion about basic structural features of human experience considerably
exceeds the likely diversity in the experiences themselves (
Consequently,
in introspecting we must frequently encounter events that fit our concepts poorly.
Such events, especially if they’re fleeting and we’re unpracticed in reflecting
upon them, may be difficult to report accurately and particularly susceptible
to theory-based reconstructive distortion. In the relatively rare cases when
externally witnessed events challenge our sense of the possible – for example,
when the final position of the cars doesn’t seem to make sense given the
trajectories we seem to recall – our memories, theory-laden and
reconstruction-based as they are, appear to be undermined. I’m judging here
only from personal experience: I know of no research directly on that issue in
the eyewitness literature. However, a slew of studies spawned by Schacter,
Cooper, and Delaney (1990) suggests that memory is poorer for line drawings of
“impossible” objects than for (novel) possible ones. On a more introspective
note, Gopnik (1993a-b) argues persuasively that children’s memory for past
false beliefs is severely hampered when children accept a theory that allows no
room for false belief in general. Likewise, surely, what is alien (a cricket
match) will generally be harder to remember and report than what is comfortable
and familiar (a baseball game). The merely unusual may vanish in
reconstruction, or it may be better remembered because striking and salient;
but events so foreign to our ordinary conceptions that we lack easy schemata or
categories for them – events, if Russ and I are right, that we are quite likely
to encounter in introspection – should, it seems, be hard to retain.
People asked
to imagine events often confuse those events with events actually
experienced (for a review of the extensive literature on this see Johnson,
Hashtroudi, & Lindsay, 1993; for connections with eyewitness testimony see
Lindsay, 1994). I’ve spent half an hour looking for my keys. Suddenly, I
picture them on the kitchen ledge. But am I having a genuine memory of
having seen them there, or does the image of them on the ledge seem familiar
only because I imagined the keys there before, earlier in my search? After the
crime, I imagine the perpetrator with a moustache; later I’m confused about
whether I actually saw him that way, or only imagined it. What about events in
the stream of experience? Once again, I’m forced to conjecture: I know of no
research that looks directly at whether we can conflate “inner experiences” we
actually had with those only imagined later. However, it seems likely the rate
of conflation would be comparatively high. If Melanie has a visual image of a
shed at the moment of Beep 1.3, then reconstructs that image shortly afterward
in taking note of that experience, then reconstructs it again when she is
reviewing her notes prior to the interview, then again finally (as she admits)
during the interview itself, she runs a considerable risk, I think, of
misattributing features of one image to the other. If the information available
to me as I entertain that image of my keys on the ledge marks only poorly
whether the source of that representation is a past imagination or a past
perception, mustn’t it even more poorly mark one past imagination from another?
The
literature on eyewitness testimony calls into question the very project of this
book as Russ and to a lesser extent I conceive it. The reader is invited, as I
was invited, to listen to Melanie and reach his own more or less intuitive
judgment about how believable she is. But if people tend greatly to
overestimate the trustworthiness of eyewitness reports, and if we have only
mediocre skills in discerning accurate from inaccurate eyewitness testimony,
and if our standards for assessing accuracy arise principally from our
experience with what seems to me the comparatively easy and familiar
matter of reporting on outward events and judging the accuracy of such reports,
then maybe we’ve been invited into a trap. Melanie’s testimony may well be
considerably less accurate, and we may be considerably poorer judges of where
it is accurate, than most of us are initially inclined to think.
The point
here is not that Russ’s method introduces some special source of distortion
into Melanie’s testimony. The point is that we should be wary of trusting our
intuitive judgments about how accurately she is reporting. Melanie’s
preconceptions, Russ’s and my subtly (or not so subtly) communicated
expectations, Melanie’s potential confusion of the remembered with the merely
imagined, the changeable and elusive nature of the events to be described, the
universal human investment in being right in what one has said in the past –
all these and their kin have larger effects than most of us naively expect. Melanie’s
testimony may seem trustworthy and yet be surprisingly full of error. Given
the novelty of the task and the methodology, we can only speculate how far such
error may go, but my sense is that it likely penetrates quite far. If an
eyewitness can invent a tape recorder, replete with convincing detail, then
much more easily, I think, can Melanie invent a feeling of lightness in her
chest, or confuse inner speech with unsymbolized thinking, or be mistaken about
the degree of detail in a visual image.
5. Pressures of the Interview Situation
and Experimenter Expectations
A large and compelling body of evidence in social psychology (reviewed, for example, in Ross & Nisbett, 1991) has demonstrated that subtle features of a situation can have a striking impact on behavior. An oft-cited example (from Isen & Levin, 1972) is the following. People who had used a phone booth in a suburban shopping plaza saw another person (a confederate of the experimenter) spill a folder of papers in their path. The situation had been arranged in advance so that some callers had found a dime in the phone’s coin return slot immediately prior to witnessing the mishap and others had not. Among those who had not found a dime only one of 25 helped to gather the papers. Among those who had found a dime, fourteen of sixteen helped. Apparently, what we might have thought to be principally determined by durable character traits – how considerate and helpful someone is – can be largely decided by a minor feature of the situation. Hundreds of experiments, using a variety of methods and venues, show similar results.
It’s well known that such subtle situational pressures can greatly compromise a psychological study, through their effects on both experimenters and subjects. Expectations conveyed by or to experimenters, in particular (which we might think of as part of the social situation surrounding the experiment), can have a surprisingly large influence on the outcome of research (e.g., R. Rosenthal, 1976). In one famous study (R. Rosenthal & Fode, 1963), undergraduates acted as experimenters, running supposedly “bright” or “dull” rats in a maze (the rats were actually from the same population). The “bright” rats performed considerably better than the “dull” rats, and continued to improve over the course of the experiment. Presumably, they were better encouraged, better treated, and given the benefit of the doubt in multiple difficult-to-track ways. Similar effects have been found, disturbingly, with children in the classroom, even with only minimal experimenter contact (R. Rosenthal & Jacobson, 1968/1992).
Closer to the present topic of study, Intons-Peterson (1983), using advanced undergraduates, has shown substantial experimenter expectation effects on subjects’ reports of their imagery experiences and on imagery-related tasks, even when many of the most overt sources of potential experimenter influence are eliminated. For example, undergraduate experimenters gave subjects a mental rotation task, requiring the subjects to quickly judge whether a visually presented outline of a hand was a left hand or a right hand (as seen from the back). Hands were presented at different angles of rotation (always from the back), and prior to each presentation the subject either received a “perceptual prime” (a left or right hand in canonical upright position, presented for comparison with the target hand) or was asked to visually imagine such a comparison hand. All presentations and time-recordings were done by computer. When experimenters expected better performance in the perceptual prime condition than in the imagination condition, the computer recorded performance times in accord with that expectation. Conversely, when the experiment was conducted by experimenters with the opposite expectation, the opposite result was found. Outside observers brought in to check for subtle sources of experimenter influence (e.g., in voice modulations and facial expressions) had difficulty discerning any such differences between the two groups; Intons-Peterson did, however, find substantial differences in the experimenters’ pauses while reading the instructions.
Situational and experimenter-effect influences tend to be stronger than most people (or at least most Westerners) expect (Ross & Nisbett, 1991; Choi, Nisbett, & Norenzayan, 1999). Thus, we must be cautious in relying on our intuitive sense about the extent to which Russ’s and my expectations, and the pressures of the interview situation, may have influenced Melanie’s reports. Here, as with eyewitness testimony, an untutored sense of Melanie’s believability may lead us astray: She may be considerably more swayed by us than the reader would naively expect. In fact, Melanie later wrote: “I struggled during the first set of samples, and, I will admit, for most of the experiment with a desire to gloss over what I was really experiencing and try to say what I thought was expected of me” (personal communication, September 2004). This later statement (if it is to be believed) supports my impression that Melanie felt potentially distortive pressure from what she took to be our expectations.
The open structure of Russ’s interviews allows plenty of
opportunity for experimenter expectations to some into play, especially if such
subtle factors as the length of a pause are considered relevant. So I don’t
think we’re safe inferring from the lack of palpable bias on Russ’s part
that his expectations had only negligible distortive effect. I myself, of
course, made much less effort to hide my biases, and in one case at least I’m
inclined to think they had a discernible effect: in Melanie’s move from reporting
very detailed imagery with almost no indeterminacies to her reporting more
indeterminate imagery (see Box 5.11).
One situational pressure that may be easily missed is the
pressure on Melanie to provide some kind of fairly specific description of her
experience. She has worn a device for exactly that purpose; to confess
ignorance would be a defeat; other subjects apparently can do this; two
professors await her report with interest. Intuitively, one might think it
nonetheless quite open to Melanie – especially given Russ’s and my verbal
endorsements of caution – regularly to say she doesn’t recall very well, for
her to provide only a very rough sketch and then stop, to open the door to uncertainty.
Such restraint would probably better reflect her (and most of our) actual
capacities. And if the general picture that Ross and Nisbett draw is right, such
situational pressures toward specific and confident reports may be
substantially more compelling than they seem to untutored intuition. Furthermore,
Russ’s persistence in asking for details, while in many ways laudable, may
amplify this pressure (for example, in Beep 4.1, where Melanie struggles to
describe the experience of craving to go scuba diving; see Box 7.3).
Inaccuracies of memory may thus conspire with subtle
situational pressures – pressures both to conform to our (perceived)
expectations and to confidently produce details of some sort or other – to
create substantial inaccuracies in Melanie’s reports. And the vaguer the
memory, the more ineffable and elusive the targeted experience, the more room
for such factors to operate. If the task is intrinsically very difficult – if
we’re simply not capable of accurately reporting that kind of detail –
confabulation, or simply taking one’s best stab, without much sensitivity to
whether confidence is justified, may be practically forced.
Let me mention also that situational pressures doubtless affect the interviewer, as well as the interviewee. In particular, I’d like to emphasize one pressure that I think may run pretty deep in the DES situation: the pressure to accept what the subject says, especially when she’s reporting confidently on a moment of experience conscientiously sampled and carefully scrutinized in the interview. For the interviewer to remain unsatisfied in such a condition undermines the apparent basis of the activity. The subject has been asked to describe her experience and no flaws have been found in her report. What more could the interviewer want? If the interviewer consistently remains skeptical, the subject may legitimately wonder if she has been lured into a winless task. I’m quite familiar, from numerous informal interviews, with the awkward tension that arises when I ask someone her opinion about some aspect of her experience and then express a disinclination to believe the resulting statement. It feels much more natural and comfortable to come on board, agree, be collaborative rather than judgmental.
In his 1990 book, Russ explicitly states that the subject and interviewer should try to reach agreement. Indeed, Russ had his subjects examine and criticize the interviewer’s final descriptions of their experiences. Russ’s highly collaborative method no doubt vividly conveys a respect for the subject and a concern for deeply scrutinizing what the subject might antecedently have thought to be irrelevant details. The interview may benefit enormously from conveying these impressions. However, it may also become difficult for the interviewer to achieve the distance and detachment necessary to view the subjects’ reports in a sufficiently skeptical light.
6. Further Concerns Particular to
Reporting Conscious Experience, and “Bracketing Preconceptions”
I’ve recommended a general skepticism about the details of Melanie’s reports. In light of these concerns, should we still, at least tentatively, accept the gross features of her reports, as I suggested in Section 2? Should we accept that Melanie experienced, at or around the time of Beep 1.1, a thought in inner speech or hearing about the peculiarity of having to plan the inheritance of her new chair, as well as an experience, perhaps visual (“rosy-yellow”?), of the humorousness of that thought – never mind details about the pacing and vocal characteristics of the speech, the location and exact tint of the glow? Should we accept that Melanie experienced, at Beep 5.1, a visual image of some sort, of an intersection, and some kind of feeling or knowledge of anxiety? Such gross features seem much more likely than the details to have been reflected upon and written down immediately after the beep, and thus to have been accurately ascertained and preserved in memory, relatively unchanged, from the moment of experience to the moment of report. One might also suspect that gross features would be less subject to revision or confabulation under situational pressures than a nuance or detail.
But how likely are we to get it right about the gross features of our conscious experience in the first moment of introspective reflection? Let’s set aside questions of long-term memory for the moment and consider short-term memory or even concurrent introspection. Returning to the eyewitness analogy: How likely is Melanie to have “seen” things correctly in the first place? Except in unusual circumstances of visual illusion or magic shows, we’re generally unlikely to misperceive the gross features of nearby events witnessed in good conditions. We might misperceive the thief’s hair color in the sun, but we wouldn’t misperceive his blue getaway car as red or see him as driving off thataway (south) when he’s actually driving the opposite direction (north) – much as we might, surprisingly often, misremember such matters later. An eyewitness who immediately (within a few seconds) explicitly notes such easily perceptible features of the event, and who keeps her notes for consultation, is considerably less likely to misremember those features later than one who waits a few minutes or hours. If Melanie’s immediate knowledge of the gross features of her own experience is as good as an eyewitness’s knowledge of large, nearby events, we might likewise be justified in accepting the first thing or two she notes in each sample.
However, I would argue that our introspective and immediately retrospective knowledge of our own experience is generally not as good as our knowledge of the most easily perceptible outward events in our vicinity. This is the reversal of Cartesianism that I advocated in Chapter Three. Preconception, expectation, lack of practice, weak linguistic and conceptual tools, the instability and skittishness of experience (combined, perhaps, with its complexity), conspire to produce introspective judgments that are often grossly false, even regarding the most basic features of current or immediately past experience. As I suggested in Chapter Three, there’s little reason to think we get it right, even in the most careful reflection, about such things as the basic structural features of our imagery (regarding, for example, how detailed it is in the periphery and whether it arrives instantly or is built up a piece at a time as we think about different aspects of the imagined scene) and emotional experience (regarding, for example, whether it is experienced as entirely bodily or whether there’s some non-bodily cognitive component) and the clarity of peripheral vision (as I argue in Box 4.18). (For further development of these points, see also Schwitzgebel, 2002a, forthcoming-b, and in preparation.) Philosophers, psychologists, and ordinary folk persistently disagree about such matters, and it seems indolent utopianism to suppose that everyone is simply right about their own experiences and wrong about everyone else’s – especially given the lack of evidence for cognitive differences between people corresponding to their different experiential reports.
We
can even neglect and invent whole modalities of experience, as I’ve argued in
the case of echolocation (the ability to hear the location and properties of
silent objects through attunement to how they reflect and alter environmental
sounds). Many people – even, historically, many blind people who’ve actively
used echolocation in navigating around walls and obstacles (and also, famously,
Nagel, 1974) – deny any auditory experience of or capacity for echolocation;
yet most can be brought to change their minds with a few introspective
experiments. (Close your eyes and say “shhhh...” while a friend moves her hand
around in front of your face; you can hear where her hand is; see
Schwitzgebel & Gordon, 2000, for details.) Likewise, there’s a lively controversy
about whether there’s a distinctive experience of thinking over and above the
experience of imagery and inner speech (for a brief review, see Schwitzgebel,
in preparation; see also Russ’s discussion of “unsymbolized thinking” in
I won’t argue these points farther here, but I draw the following conclusion: Even at the first instant of reflection about her experience, Melanie might be quite badly mistaken about it. Introspection is more difficult than ordinary perception. Convincing or reminding ourselves of this difficulty is crucial in our evaluation of Melanie’s accuracy. Thus, I think we must add to the concerns discussed in the previous two sections another major source of error, one that undermines even the first and most basic aspects of Melanie’s reports: the intrinsic difficulty of the observation. Although Russ has done good work in trying to reduce certain sources of error (as discussed in Chapter Two), the fundamental difficulty of the observation remains.
We
cannot, of course, given our current state of knowledge about experience in
general and Melanie in particular, prove gross error in any of Melanie’s
reports. However, let me list some of the relatively gross claims about which
I’m most suspicious: that Melanie literally visually experienced a “rosy-yellow
glow” in Beep 1.1 (see Box 4.7); that she was as consistently and robustly
self-conscious as she claims on Days 5 and 6 (Beeps 5.1, 6.1, 6.2, esp. Boxes
8.9, 9.3); that her imagery was as detailed as she says in Beeps 1.3, 2.1, 2.2,
and 5.1 (see esp. Boxes 4.18, 5.4); that she literally imagined individual
overlapping echoes of “nice long time” in Beep 6.4. In some sense, such matters
are details. They’re not the sort of thing an untrained reporter would probably
first notice about her experience – and maybe, indeed, Melanie did not reflect
on such matters swiftly after the beep (without her notes [see
Why, you might ask, am I more skeptical of these particular reports – the ones described at the beginning of the previous paragraph – than others? I have no reasons internal to the interviews. I can detect no telltale signs of error in Melanie’s patterns of speech, for example, or any special hesitation, uncertainty, or inconsistency on Melanie’s part in making these reports (though I confess that I may have a tin ear for such things). My reasons are entirely external: Melanie’s reports here poorly match my pre-existing impressions about what’s common in experience, based on my understanding of my own experience and my reading of the psychological and philosophical literature. Now Russ will surely object here that in so evaluating Melanie’s reports, I’ve failed to “bracket preconceptions,” and thus am not giving his method a fair shake in its own terms. Russ and I have been through this dialogue multiple times (e.g., Box 9.9). Let me add, here, a few thoughts to it.
First, I acknowledge the appeal of “bracketing preconceptions” for the purpose of conducting a friendly, relatively unbiased interview. Surely an interviewer can err through too lively a commitment to seeing the subject a particular way; perhaps indeed this is the error most to be avoided. But it’s one thing to bracket preconceptions (insofar as possible or desirable; see Section 1 above) as part of an interview technique, and quite a different thing to discard all prior (non-DES) evidence about experience in one’s later evaluation of that interview. I don’t know whether Russ really means to recommend the latter course; but sometimes it seems to me he comes across that way, for example in his reference to earlier methods as “failures” and in his tendency to disregard previous literature in interpreting his results. This point is central to understanding the role Russ envisions for DES vis-a-vis other methods. So let me re-emphasize here that one could only justifiably take the extreme position of disregarding all prior evidence in one’s evaluation of Melanie’s assertions if it were somehow already established that Russ’s experience sampling method was so superior to all other sources of evidence as to automatically trump anything contrary. As I argued in Section 1, I don’t think that has been established. Unless we’ve decided to accept DES as our sole guide to the truth about conscious experience, it makes no sense entirely to forgo our previous inclinations – whether the fruit of other methods or general plausibility arguments – in reaching our final judgments about how far to believe Melanie’s reports.
Second,
although I said in the first section of this concluding essay that we have no
means of comparing Russ’s picture of Melanie’s experience with the picture that
other methods would have produced, that may not be quite true. No direct
comparison between methods is possible here, but maybe we can make indirect
comparisons. If Russ’s picture of Melanie generally comports with what we’d
expect based on prior methods, that provides a kind of support for it; if not,
that may raise concerns. It seems to me that the above-cited claims comport
worse with my sense of prior research, and my own experience, than other of Melanie’s
claims. In particular, all those claims strike me as relatively unusual.
I think I am, then, justified in being somewhat more suspicious of them (see
Third, I acknowledge that my own sense of plausibility and likelihood differs from others’. This is problematic. I claim no unusual introspective expertise. I’ve read widely on consciousness and reflected somewhat on my own experience, but no more than others who disagree with me about various substantive issues. The phenomena are elusive, the literatures complex, contradictory, and confused. So I can’t say that I feel myself to be on any especially solid ground when I am inclined to accept one piece of Melanie’s testimony more than another based on prior impressions.
Reflecting in this way, I begin to feel near total darkness about experience. Can I really make any good judgments about the better and worse in Melanie’s reports? When I consider my own poor antecedent knowledge about conscious experience, my self-assurance begins to fail. I was inclined to mistrust Melanie’s reports of detailed imagery because I have a general impression that visual imagery is sketchier, less determinate in its details, especially when quickly generated, than many people suppose. But on the basis of what have I arrived at this opinion? I’ve already said that there is no single, dependable method for studying consciousness. Maybe, then, I believe what I do about imagery because there’s a consensus among researchers applying a variety of methods, individually weak but jointly persuasive? No, there is no such consensus. I must admit by my own lights, then, that I could easily be quite wrong in my opinions about imagery. Indeed, it was my genuine dissatisfaction with my own (and the field’s) condition on such matters that led me to Russ in the first place, looking for something better, or at least something additional. So maybe Melanie is quite right about her imagery (her rosy-yellow glow, her self-consciousness, her echoes), and I am wrong.
Conversely,
however, maybe in other cases I should mistrust Melanie more than I do. When
she reports a feeling of conviction in 6.1, or a lightness in her chest in 6.2,
or imagery (of any sort at all) in Beeps 1.3, 2.1, 2.2, and 5.1, I feel no
particular suspicion. But if the state of the field, and my own epistemic
state, is as much a mess as I think – and if I am right in insisting that gross
introspective errors are generally quite possible – perhaps I’m too easily
taken in by what seems to me the plausibility of these reports. Does conviction
really carry some sort of positive phenomenology, a “feeling” of conviction? Or
is conviction just a state of reaching a definite judgment, perhaps accompanied
by general arousal or specific imagery, but without any distinctive
experiential element of its own? Is “lightness in the chest” a misleading
metaphor (but one that, for some reason, ensnares me more than “seeing rosy-yellow”
[
In his talk of “bracketing preconceptions” and of the need to reject “armchair” speculations and earlier failed methods, Russ conveys doubt about the value of people’s – including my – prior (non-DES) sense of what’s credible or relatively less credible in Melanie’s reports. Perhaps I should join Russ here and mistrust myself. However, I can’t afterward bring myself to the next move Russ recommends: trusting his interview techniques instead. Melanie’s internal consistency, her evident conscientiousness, her happy confidence alone, I’ve argued, can’t justify our credence, even if Russ has succeeded in producing an admirably neutral context for reporting. But now it seems only a short step to radical uncertainty about Melanie’s reports. I have no idea where to doubt and where to believe, so I am left only doubting. And worse: Since I have no reason to think myself any better an introspector than Melanie, my own introspective judgments come under skeptical threat as well. In any reflection I could very easily be wrong, and my prior sense of plausibility is too ill-founded to be of much help.
But utter uncertainty
(about anything sufficiently broad: the external world, other minds, the
future) is philosophical madness, isn’t it? And few philosophers even among the
radical skeptics have dared entirely discard our knowledge of our own ongoing
conscious experience. How could we be totally in the dark about that? I
just experienced some “auditory imagery” or “inner speech” or “inner hearing”
(if I try to be too precise here, I might lose hold of it): I heard or spoke,
silently, the sentence I was about to type. I can’t seem to bring myself
sincerely to doubt that claim or to assign it any but the very smallest
probability of being false, despite all the reflections that have led me here. And
if Melanie seems to be reporting something similar in her own experience –
well, there we have a beginning!
So maybe it’s only modesty and caution I should recommend, and not utter skeptical uncertainty. For what is nearer to hand and riper for discovery than our own experience? Yet even the meekest and most tentative reflections about experience are bound soon to conflict with what others have said, so widespread and fundamental are the disagreements in consciousness studies.
At the most general level, I suppose I haven’t moved far from where I began before meeting Russ – tempted by radical skepticism, suspicious of every method, doubtful about the future of the field. At the same time, this temptation, suspicion, and doubt, this kind of half-convinced pessimism, is not a deep conviction that introspective science must fail. In fact, in the long run I feel hopeful that we will make some sort of progress, simply by virtue of applying our good minds to it hard enough and long enough in enough different ways. And Russ has convinced me that beep-and-interview methods deserve as large a role, for now, as anything else.
Chapter Eleven
Russ’s Reflections
Russ Hurlburt
We have traversed a crooked path
over the course of this book, following randomly selected concrete instances of
Melanie’s experience into whatever thickets they happened to lead. Now it’s
time to straighten things out, for me to say what I thought happened here and
why the path was worth the effort. I do so in two parts. In section 1, I
discuss my own observations. In section 2, I reply to Eric’s observations from
Chapter Ten.
Before I do that, I wish to emphasize how much I respect Eric’s participation in this project. Despite his skepticism, he was willing first to try out DES for himself; then to recognize the conflict of serving as his own subject and to agree that we should find a more neutral subject (ultimately Melanie); and then to participate in the making public of this interchange that took place in an arena where I was far more experienced than was he. That is the heart of good science: as much as possible to subject one’s own views to the scrutiny of reasonable but not-necessarily-like-minded others. In passing, let me say that over the years I have made similar would-you-like-to-participate-in-DES offers to many other philosophers and psychologists, nearly always with the same result: their retreats make Roadrunner look like he is dragging an anchor.
I think Eric and I did a good job of avoiding a “My theory is better than your theory” interchange. Instead, we have brought our quite different views into a candid collaboration / confrontation where both of us expected to be altered and would have been happy to be proven partly or entirely wrong.
Also at the outset I emphasize that I agree with Eric that skepticism about introspective reports is highly desirable; that the base rate of successful introspections is small; and that Melanie’s sincerity and conscientiousness and my carefulness and even-handedness does not in any way guarantee that my conclusions about Melanie’s experience are correct. It is the size and extent of the skepticism, not its desirability, that is at issue here.
1. Russ’ Views
1.1. About Melanie
We discussed 17 samples with Melanie and on that slim basis learned quite a bit, I think, about her. We discovered that she engaged in an active self-monitoring of her own actions: observing her mouth closing while speaking in Beep 1.3, observing being bent over at sink in Beep 2.4, observing the fogginess of her experience in Beep 3.2, observing her forgetfulness of the parking brake in Beep 3.3, observing that her eyes were looking straight ahead while talking in Beep 6.1, observing the bodily aspects of feeling happy in Beep 6.2, observing her brow furrowed in concentrating and the positioning of her feet in Beep 6.3.
We discovered that she paid thematic attention to the sensory aspects of her environment: the green color of the TV screen in Beep 1.4, the coldness in her toes in Beep 2.3, the coldness and gooiness of the toothpaste in Beep 2.4, the bodily bobbing up and down in her imagination in Beep 4.1. These awarenesses are not merely paying attention to the objects in her environment, but paying particular attention to the sensory aspects of those objects.
We discovered that she had detailed visual images: of the soldier on a dirt road in Beep 2.1, of Stukas in Beep 2.2, of a shopping-list pad in Beep 2.3, of the Bicycle card joker in Beep 4.2, of an intersection with apartment buildings in Beep 5.1; and we discovered that she created those details even in the absence of the correct knowledge of what those details should look like: the Stukas were F-18s in Beep 2.2, the joker was incorrectly imagined in Beep 4.2.
We discovered that she had feelings, sometimes expressed bodily: of sadness/dread pressing on her chest in Beep 2.2, of yearning in Beep 4.1, of conviction that she was correct in Beep 6.1, of happiness (a lightweight feeling in her lungs) in Beep 6.2, of concentrating in Beep 6.3. But we also discovered that sometimes her feelings were apparently ongoing in her body but are not directly experienced: of being exasperated but not experiencing it directly in Beep 3.3, of concern and resentment not being directly experienced in Beep 4.2, of anxiety about being late not being experienced in her body but being thought about in Beep 5.1.
Melanie, by her own retrospective report, was surprised by some aspect of all these characteristics. She had apparently no knowledge at all of the fact that she was as absorbed by the sensory aspects as she was; she knew she had visual images but was surprised by the incorrectness of their detail; she was unaware of the emotional processes ongoing outside of her awareness.
1.2. How Far Does Russ believe Melanie?
I believe, pretty much as does Eric, that there is reason to accept at least in broad strokes the veridicality of Melanie’s reports. Certainly there is reason to quibble about some things: as we have seen, her reports on the first sampling day or so might reflect more her presuppositions than her actual experience; her images may be incorrect in some of their details because the interview took place the next day; and so on. But none of these quibbles is enough to overturn the overall accuracy of the observations. If we were particularly concerned about the first few sampling days, we could discard those beeps and sample with Melanie for a few more days. If we were particularly concerned about the forgetting or confabulating of image details, we could give Melanie a tape recorder and ask her to dictate the image details immediately following the beep rather than rely on written notes. Thus I believe that Melanie’s accounts are pretty darn good; we could incrementally but not dramatically improve on them if we wished.
I have sampled with several hundreds of subjects at the same or greater level of detail and skepticism as we applied with Melanie. I am convinced that the general statements that we made about Melanie (that she engaged in active self-monitoring of her own actions, that she paid thematic attention to the sensory aspects of her environment, etc.) do not apply to all subjects or even to most subjects. I do not wish to claim that we discovered Melanie’s essential uniqueness, but I do believe that, for example, most of my subjects do not engage in the kind of active self-monitoring that Melanie did. Whether observed differences reflect actual phenomenological differences or merely expressional differences is of fundamental importance. I assure the reader that for 30 years I have interrogated subjects in what most would say is excruciating detail on this particular issue, and during that time I held no particular position on the desirability of one outcome or the other. Those observations have forced me to conclude that people’s experience actually differs from one person to the next – that these differences are not merely differences in reporting style. I would have been just as happy if the universe had turned out otherwise, but it didn’t.
1.2.1. Raw vs. Exposed Reports. In evaluating
Melanie’s accuracy, we need to make a distinction between what I call “raw
reports” and “exposed reports.” A raw report is what a subject unaidedly
reports about her inner experience; an exposed report is the result of the DES
expositional interview, the result of clarifying to the extent possible the
subject’s inner experience. As we have seen, I think that Melanie’s raw reports
contained much that was believable and much that was not to be believed
(particularly early in her participation in this process). Melanie was, I
think, a typical subject in this regard. I was skeptical of her early raw
reports, for example, of her inner thought voice of Beep 1.1 (see
I
think our exposed reports of Melanie’s experience, the understanding of her
experience that Melanie and I (and to some extent Eric) shared at the end of
each interview about her experience, contained very much that was believable
and very little that was not to be believed. Thus, for example, I do believe
Melanie’s reports about the detailed nature of the images of the soldier on the
road (Beep 2.1) and that the Stukas really looked like F-18s (Beep 2.2). I am
perfectly willing to accept that a few of the details in those images may have
been confabulated or otherwise mistaken – Melanie and we are not
infallible – but I see no reason to believe that Melanie confabulated
all or most of the details. The Stukas-as-F-18s is a good example (see
I accept the fact that the exposed reports so obtained may not be a complete account of Melanie’s momentary experience; see the discussion below in section 2.2. However, in general I agree with Eric’s limited approval: I do think that “what [our interviews of Melanie] deliver is probably about as good as can reasonably be expected from open interviews about sampled experiences.”
1.2.2. Faux Generalization. When Melanie uses the terms “all the time” and “whenever,” (for example, in Beep 1.1: “It’s my inner thought voice, so it’s the one I recognize and hear all of the time whenever I’m thinking”), she shows that she is making what I have called a faux generalization (see Box 5.17). Her statement has the appearance of a truly inductive generalization, as if she had observed a series of instances of thought voices, noted that they all have the same characteristics, and reported that generalization. But it is highly unlikely that her statement is actually the result of such a truly inductive process. That statement is much more likely the result of the cognitive heuristics such as availability, recency, salience, accessibility that Kahneman and Tversky (and others) have described.
DES has shown that such self-characterizations are often not true and occasionally dramatically not true. I have had seemingly normal graduate students say they experience frequent images, but sampling produced none. I have had other seemingly normal graduate students say they have no visual images, but sampling produced many. Now it is certainly possible that some faux generalizations are true – Melanie may well have the kind of inner voice she described – but by and large they cannot be trusted. That’s why part of the DES strategy is to discourage faux generalizations, to encourage subjects to suspend their belief in their own self-characterizations, to focus on the actually occurring instants on which true generalizations can be built.
Subjects typically understand this quite readily. If I say something like, “Well, your self-characterization might or might not be true; let’s try not to be influenced by it one way or the other and see what emerges in the samples,” most subjects are not offended and recognize that value of such an approach. As a result, most of the time, the expression of faux generalizations gradually disappears during sampling. One might argue that I punish the expression, so the expression disappears while the belief lingers on. I don’t think that’s true; most subjects would convincingly say that it’s not true. Melanie was a quite typical subject in this regard. I believe she came to see that her faux generalizations interfered with her ability to observe her experience accurately and that she gradually developed the skill of suspending them. As a result, her raw reports became more accurate, and it became easier for us to filter out remaining inaccuracies and do a better job of creating accurate exposed reports.
1.3. Inner Speech
It is useful to comment on Melanie's lack of inner speech because many theorists hold that all thinking is inner speech and that inner speech should therefore be ubiquitous across all DES subjects. Baars, for example, claims that “human beings talk to themselves every moment of the waking day” (Baars 2003, p. 106).
It is usually very easy for DES subjects to report inner speech and very easy for investigators to recognize it. Subjects who have frequent inner speech (and there are many such subjects) make for generally the easiest, least ambiguous sampling studies. Inner speech often involves full sentences, naturally inflected, with the same kind of pauses, stutters, voice pitch and rate, emotional tone, and so on as external speech.
Melanie was not like that. First of all, she had no clear cut examples of inner speech. She reported inner speech twice out of the four samples on the first sampling day, but those reports are dismissible, I think. It is often the case that DES subjects, like Melanie, frequently report talking to themselves on the first sampling day and rarely make such reports later in sampling. I take that to be the result of the subjects’ initial presupposition that thinking is inner speech. The questioning of the first day is designed to bracket all presuppositions including that one. If subjects later report no inner speech, then I attribute the early reports to the presupposition and the later lack of report to the successful bracketing of that presupposition.
The closest Melanie got to inner speech was sample 3.1, where she was thinking “peri-, peri-,” to herself, the first part of the word “periodontist” that she was trying to remember. But questioning revealed that she wasn’t really sure whether she was saying “peri-” or experiencing it in some other way. I take no position on whether this “peri-, peri-” experience was or was not (vaguely or faintly experienced) inner speech; I think trying to force such a conclusion is a mistake (see section 2.2 below). Certainly Melanie did not have the kind of clear and frequent inner speaking phenomenon that is common among many DES subjects.
At sample 3.3, Melanie experienced her own voice saying the first part of the sentence, “Why can’t I remember about the parking brake.” Melanie, like most DES subjects, apparently made the discrimination between inner speech and inner hearing confidently and experienced this sample to be inner hearing. Inner speech, the more common phenomenon across subjects, is experienced to be “going away,” “produced by,” “under the control of” the subject, “just like speaking aloud except no sound.” Inner hearing, by contrast, is the experience of a sound “coming toward,” “experienced by” rather than produced by, “listened to” rather than spoken, “just like listening to a CD.” The typical subject is not confused between inner speech and inner hearing any more than they are confused between speaking aloud and hearing a tape recording of themselves speaking.