Conversation with Ralf-Peter Behrendt

Hi,

Very late to this - found whilst googling thinking about times past. Ralf was a friend of mine, I knew him from his time in Sheffield.

His death was very much a shock, though I have a suspicion of what may have happened. He was a very intelligent man, great fun to talk to, but also very troubled.

I miss him still.

Adrian Hill

Thanks for this post

Does anyone know Ralfs birthday??

okokok

I met Ralf in person. I spent a lot of time with Ralf. We watched Mighty Bush. We would sit on his carpet and watch it. We loved it, we were like little kids.

A patient reminded me of Ralf today. Episodic memory. Ralf will always be the most intelligent person I have ever met. I heard him though. Loud and clear.

I miss him so much now.

Hey Ralf,

I've been working on my latest revisions of the paper, in many places

starting from scratch. Most of the arguments are the same, I'm just

unpacking them much more thoroughly.

There is one detail that I'm starting to see differently, and I'd love

to hear your take on it. It's about the same referee question I sent

you recently, about how does the hippocampal simulation become

experience? I have, heretofore, been agnostic about that question,

and just left the simulation within the hippocampus.

However, that answer seems theoretically troubling to me. I had built

the hippocampus argument on the model of a 'pyramid of reports', i.e.

each module sends its report to the next module, which does its

computation, and then sends an output report on to the module after

that. In other words, information can only be known in the brain, if

it is shared. In this model, there must be a transmission. The

encode of memory cannot just exist, 'unseen', or it could not be

subjective experience.

Of course subjective experience is not 'seen', but I do think that the

metaphor of the hippocampus as the TV news function of the brain is

useful anyway. My current explanation in the paper, the one that

makes the most sense to me at this point, is that subjective

experience is not the memory encode itself, rather that it is the

process of that encode being 'broadcast' back out to the rest of the

brain. Again, the rest of the brain is not 'watching' the news.

Rather, the brain is activated by the memory broadcast, in the same

way that the visual cortex doesn't watch the outside world; rather, it

is activated by it. Same thing here: as the broadcast signal

activates the neocortex (presumably at layer 6), as a top-down signal

to help bias the next moment's predictions, that activation creates

experience throughout the brain. That experience has already been

unified and simulated in the hippocampus, but it is now made 'real' by

being shared with the rest of the brain.

One piece of evidence that I find compelling in this direction: the

fact that patients with hemispatial neglect exhibit the same neglect

when remembering scenes that they had first committed to memory,

pre-morbidity. If they imagine themselves at one side of St. Mark's

square, they can only describe items on their right, but if they

imagine themselves at the other end, looking back, they describe the

opposite half. What this suggests to me is: the encode is intact, CA3

is intact, but the recalled memory is only 'known' to the rest of the

brain, if those parts of the brain can be re-activated by the

broadcast of the recall. Hence, subjective experience of the recall

fails, because of neocortical damage. Which suggests: subjective

experience 'happens' at the neocortex, even though it's formed in the

hippocampus.

Now, that's not the only possible answer, I'm sure, but right now,

it's the one that makes the most sense to me. As you know, I've been

bugged with some variant of this question for many months now, and

this is the first formulation that really feels elegant and

explanatory. This doesn't change the overall theory, as I see it, but

it does make specific predictions that I was not previously ready to

commit to.

I've often seen you state that consciousness is the patterns in CA3,

but I have a hard time with that theoretically (especially since the

downstream CA1 seems to be where events are placed into chronological

order, and embedded into theta cycles). But it's also hard to imagine

that the entire brain 'believes' that the new memory encode is

reality, unless the entire brain (or most of it) receives that

broadcast transmission first. And it's hard to explain a rich

experience that only exists in the hippocampus, as if the hippocampus

could somehow experience its own creation. I think of CA3 as the

auto-associator and decoder of previous memory encodes, and CA1 as the encoder of the new memory (as prepared for it by CA3, DG and the EC, and prior to them, PA and PHC, and prior to them, everywhere else in

the brain). The parts of the brain which are capable of experience

seem like they should be the very sense cortices that process initial

stimuli. This closes the loop, and it explains why subjective

experience is happening in the background, even during 'selfless' flow

(i.e. neocortical) states.

I'd love to hear your thoughts.

best,

matt faw

Hi Ralf,

I think I see what you mean. Even though 'subjective experience' is

important from a human-scale perspective, it is merely a epiphenomenon

of the patterns that underly it, because it is the patterns that

actually run the show. The patterns aren't asking themselves: "what

color was that?" The patterns don't ask anything, because they are

just patterns. They reflect natural processes, just at an extremely

high level of complexity. Those patterns happen to be members of the

'ecosystem' or 'weather' of the mind; they're part of a more complex

set of patterns, but there is no identity in there, no will, no actor,

no perceiver.

-----------

BTW, when I sent you the reviewer's criticism, I should have included

the following paragraph, because it's more to the point, as far as the

actual locus/mechanism of subjective experience (SE). It's a question

that I'm sure is answerable, but I have no way of being able to make

anything more than a wild guess. I'd love to hear your thoughts on

it.

"Why is the HF so important for SE?

The authors have not actually answered a key question - why does the

HF have this special property of SE, and not other regions? What they

have done is picked out some functional properties of the HF that map

quite well onto important features of SE, but this is merely

correlational. This does not, therefore, answer the fundamental

question of how the (according to the authors) unconscious processing

occurring across the rest of the brain is somehow transformed into SE

within the HF. I do not understand the rationale for this. One could

accept the authors arguments about the functional role of the HF in

binding together many different processing streams, and creating a

"movie" simulation etc., but could then generate many different

theories about the actual neural correlate of SE on the basis of this,

other than their own argument that it is in the HF. For instance, it

could be that the HF processing is also unconscious, and that the

"movie" created within the HF is then passed to other areas (such as

the posterior cingulate cortex or medial prefrontal cortex, or a

distributed network of high-level regions) which use this information

to generate SE. Or one could argue that SE is an emergent property of

a complex, dynamic distributed system, such that the HF contributes to

SE only to the extent that it contributes unique aspects of

processing. By this account (which I personally find the most

plausible), the HF is essential for certain kinds of very important

SE, such as SE of an extended, spatially coherent world as opposed to

disjointed visual objects, and such as episodic memory and

imagination, but it is not the route of all SE. Extensive

clarification is needed."

Well, I think we're in agreement, Ralf. I believe in zero magic.

Everything is organic. And even organic is not magic as 'life' but is

the consequence of self-organization.

My dream next documentary is precisely about this, using the snowflake

metaphor to show how organization from the big bang through stars and planets and organic molecules and life and us is inevitable; it's the

way that pattern formation and chaos harmonize with each other in the

particular key that makes up our universe.

I believe there are only a handful of fundamental geometric tendencies

(which probably fold up into even simpler symmetries) that initially

set in motion all the amazing organization that has arisen from them.

Everything, at every level, is self-organization. No magic.

best,

matt

I guess we do differ there, because I don't understand your reasoning.

As I understand it, your "appetitive behaviors" paper was all about

how the 'situation' is the complex stimulus for conditioning. Isn't

that 'situation' the same thing as consciousness? Doesn't the paper

say that the situation is a piece of information that represents the

outside world, and that it is precisely the pattern matching between

today and tomorrow's situation, that allows today's situation to have

an effect?

Would you say that episodic memory is epiphenomenal? Doesn't the

memory of yesterday effect the perception and behavior of today?

Isn't consciousness the same thing as episodic memory?

I would agree with this aspect: HM can be explained by saying that he

doesn't need the feedback of consciousness, in order to continue

interacting with the world. But that's not the same thing as saying

that it wouldn't have an effect, if he had it. A person with

blindsight may be able to 'see' somewhat via the dorsal visual stream,

but as long as they believe they are blind, they will act accordingly

(unless somebody throws something at their head).

Even dreams, which seem like the most epiphenomenal aspect of human

cognitive life, still potentially impact people's beliefs, and change

their behavior.

So yes, I agree that consciousness is information. And I agree that

it is information that is decodable by the rest of the brain, but not

(so far) by human society (it is not words or images, but patterns

that represent those things). But information is not epiphenomenal;

or it would not inform.

scratching my head,

matt

Hi Ralf,

I believe mathematics is the best possible way to describe the entire universe (particularly geometry). My metaphor for the universe is a snowflake, one symmetry built fractally upon another. The deepest symmetries all formed in the first second of the universe, and are so stable that they exist to today: the relationship between energy and space, between mass and energy. The hadron era of protons naturally gave way to the lepton era with electrons, symmetry upon symmetry. Every particle can be described geometrically; two protons repel because of like charge, but that could also be described as a mathematical relationship that has a very hard time bending past a certain curvature.

So yes, I'm sure that the brain is ultimately described by mathematics, and that on a pure enough level, it is foolish to talk about causative powers, because causation is just a dynamic geometrical unfolding of the original symmetries.

Although the phenomena near the center of the snowflake are extremely stable and predictable (an electron never fades), they give rise to fractal patterns at the edges that are increasingly noisy and chaotic. Geometrically, it was inevitable that protein chains would eventually form, that they would develop the ability to copy themselves, and then that they would eventually copy themselves into dominance over the world. But the tendency for things to fall naturally into patterns is balanced with the tendency for things to fall apart. Yang balanced by yin. Without yin, we would have no death or mutation, and hence, no evolution. In fact, without chaos, we would still have an entirely homogenous universe, full of increasingly rarified energy and no structure. The universe needed noise for energy to clump and accrete, just as an actual snowflake needs a dust grain, in order to form.

So yes, on a cosmological level (all the way to brain dynamics) it is all math. However, I'm not a mathematician. I'm a filmmaker, and I need to choose the language of dreams. I need to personify, to use metaphor, to build arguments in ways that can ease my audience past confusion into clarity.

On that level, I see no point in talking epiphenomena. Especially when I'm already trying to demystify consciousness to information, trying to pry people from their intuitions. Since we all believe that we are conscious, then we need to deal with that belief. Belief is a phenomena that consciousness explains, and it gives the theory all its explanatory power. Anosognosia, schizophrenia, etc. are mis-matches between belief and reality. The ability to recall a memory matters, even if there is no actual 'I' to do the remembering, even if it is all auto-associative.

So I have no problem with an argument that eventually ends at mathematics, because I imagine that they all do. Eventually. But I think my argument will just be confused by going there, especially as I have no way of describing the math. All I have are metaphors, and a skin-deep understanding of the anatomy. I think that my arguments have to arise from what's available to me. I think that if I focus on a human-scale argument, I will be much more likely to communicate this theory without confusion.

That's why I'm seeking a describable mechanism for memory to give rise to belief. Not because I think there is a self in there to remember or believe, but because I need to start from where my readers already think, and tease it apart gently, build a bridge of understanding for them, take them from there to here. It's true with the movie, and I'm increasingly convinced it's true with the paper, as well. I've had referees fall back on dualism, and contradict themselves without the least self-consciousness, because they're not the ones who have to build an argument; they can just fire missiles from within their own current understanding, and reject my argument without really thinking it through. Almost everything that the reviewers complained about were already in the paper, for each version. The answers were there in the various drafts, but I didn't drag the referees to the answers, force them to drink, and so they ignored them. Those are the people I have to reach, and I have to do it by making everything as plain and simple as possible.

best,

matt

Thanks, Ralf, for your quick reply. I am very grateful for your

ongoing friendship and conversation, so please, no need to apologize.

I completely agree with what you wrote. Clearly, there is no homunculus.

On this draft, I'm trying to simplify and spell out absolutely

everything, leave nothing unsaid. Most of the times our reviewers

said some answer was missing, I had actually covered it, but

apparently unsuccessfully. So I'm trying to break everything down

into small little steps, like a Joy of Cooking recipe, so there's no

gap through which understanding can fall.

On this particular issue, I can see a reasonable path of explanation,

but actually finding the right words to convey that explanation is a

whole other issue. It's like the HM thing; it doesn't seem that

difficult or controversial to me, but it's damnably hard to get across

(even to my own dad). Oh well, it's a good challenge.

best,

matt

Hi Ralf,

I hope you're well.

Sad to say, my paper just got rejected from a third journal.

Thankfully, the responses are much kinder this time than the first

two, due I think, to all the revisions I've made. I'm now trying to

figure out how best to revise and address these new criticisms.

Almost all the reviewers' comments have been addressed within the

paper already, so it's clear most of them are about me spelling things

out more clearly. However, there are a couple of questions they have

that are still stumpers.

In particular is an issue we've talked about before: yes, the

experience 'movie' is bound together in the hippocampus, but that

still leaves open the question of whether that movie is then

'experienced' by other brain parts. I've mentioned the metaphor of

subjective experience as the news of the brain, that each module sends

its report to the EC to be bound in the HF, and then broadcast back to

the rest of the brain as multimodal memory, a news report of the big

picture from a moment before. This is the solution that makes the

most sense to me, but it's out there on the periphery of the my

understanding, and I can't assert anything with confidence.

Following is the reviewer's comment. I'd love to hear what you think.

best,

matt faw

"Where is subjective experience actually located?

I am left feeling extremely unclear as to exactly what this theory is

suggesting with regard to the neural locus of SE. Most of the time the

authors appear to be suggesting that the HF is entirely responsible

for generating SE, but at other times they appear to flatly contradict

this view, and suggest that the HF actually contains indices which do

not themselves generate SE, but are used to activate the various

cortical representations from which SE arises e.g. p.9-10

"Technically, a memory's perceptual content is not embedded within the

encode, but is instead indexed, like hyperlinks, to sensory cortices

which were originally responsible for sending their report to the HF

for memory...the HF, which then follows the encode's indexing

instructions, to enlist the requisite sensory cortices in supplying

the data necessary to re-construct the episode. The more vivid the

memory, the more the sensory cortices are involved in its

reconstruction.". These are very different theories, which produce

very different predictions, and the current manuscript does not make

it clear as to which the authors are actually suggesting."

Hi Bob, yes I agree.

A word on "Pain doesn’t exist context free." I think there's

something to be learned by the recent experiments where the subjects

would rather shock themselves than be left alone with their thoughts.

I think that the body sends pain signals to the brain all the time, as

information as to where to stretch, where to be careful about, etc.

Pain demands to be listened to, but it is also part of the brain's

overall balancing scheme that there are inhibitors to block pain, for

a certain time period, so the organism can escape from harrowing

circumstances, and wait to feel pain when it is back in safe

territory. I'm speculating that most of the discomfort that was felt

by the subjects was the pain that they usually had distractions to

cover. No distractions means having to face what's already there,

which of course just makes the thoughts that must worse.

However, this ability to inhibit pain can be over-used, and I think

people, be default, do so, we tune out from bodily pain. I think this

is taught by parents to their children, so that kids learn fear of

injury beyond what is necessary, and they generalize it to pain. Pain

is bad. It is something to be avoided, to be "killed'.

But pain is just a sense, like any other. And in extreme cases,

people have inhibited other senses, including vision. So it is no

surprise that we do what we can to minimize the pain in our

simulations. Once the simulator habit includes "exclude pain" then it

creates a reality simulation that intentionally restricts a sizable

portion of its own input data. Only when the delta signal is over

some threshold do we even notice pain in the body. So old injuries

can hide for years, not because the signal is not being generated, but

because they are too familiar to consider. All of a sudden they

'flare up', but if we had been paying attention to pain all along,

surely they would have made themselves known. We have the same

attitude in geopolitics; the general public only responds when we're

freaked out by changes. Up until then, we're numb. But if we were

paying attention all along, we'd use an ounce of cure, and not have to

wait for the freak-out.

The parable of the frog in hot water is everywhere, but nowhere do I

see it more displayed, than in the way people retreat from pain, as

they age. People's bodies, as they get older, turn into caricatures

of their peculiarities when they were younger. If my back is a little

bit hunched in my 30s, you sure know I'm going to be defined by my

hunch in my 70s. But what is it that keeps people forced in these

postures? I propose that it is only pain that stops them, and if they

learned to listen, and to stretch, and to really take care of the

parts that hurt, then we'd all age a lot more gracefully. That's my

experience, anyway.

So in that sense, I think that pain is constant. The body is always

offering up pain as feedback, to help the brain make decisions. I

experience the immediate moment and respond to it with input from all

my stimulations, including pain. But since I (the planning self of

the default network) have excluded pain from my experience, I do not

believe it to be one of my motivators, and I create stories that make

other salient characters (like other people) to become the sources of

whatever pain and discomfort I feel. Pain is out-sourced, made alien,

and seen as an invader, whenever it flares up.

matt faw

Hey y'all,

Bob's new blog post also resonates with this distinction:

http://goo.gl/Tq7SNR

Bob may certainly have a different take, but part of what I get from this is:

As an organism, I have some control over what makes it into my

simulation, and I learn to block out some of the noxious stimuli, in

particular: pain. The 'observer' (as I hear it) is taking refuge

within the simulation, a relatively pain-free construction that he can

analyze and manipulate. From that safe distance, the observer can

work on his philosophy, and handle or accept pain better. But he

cannot heal, because healing involves paying attention to the source

of pain as raw stimulus, rather than as a concept (i.e. pain must be

felt, rather than just thought about). The 'participant', on the

other hand, seems caught up in all the drama, but is at least living

in the moment, including the discomfort of the moment. Pain is the

sense that demands: "pay attention here", but the simulation allows

the observers to do the opposite, and block it out.

matt

Hello gentlemen,

I don't know if you've yet seen this TED talk, but I highly recommend it. I think he is very much talking about the division between the neocortical expression of self vs. the hippocampal interpretation of that. This is related to the Martin Conway material I've been pushing on both of you, about how a self is created in memory, that is parallel but separate from the self which reflects neocortical activity. The first-person experience of self is always hippocampal, because that's where my experience becomes available to my default system. Whereas the 3rd person experience of me is essentially of my neocortical self, what I do in the moment (but of course they experience a neocortical self that is informed by the hippocampal story).

http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory?language=en

I hope you enjoy it!

matt faw

And here is a short article that I think is very

worth reading, on mise en place, or how our perception of our environment changes how we think and act. For example, a playful fantastical environment, in which some things seem to contradict our notions of what is possible, is actually the ideal learning environment, precisely because it liberates children from the tyranny of the known.

This may seem somewhat tangential to the question of consciousness, but I think place only happens at the consciousness level; this is where everything comes together on the "spatial scaffolding" to use Lynn Nadel's term. The gestalt is the subjective mise en place.

matt

Interesting Ralf.

And, I have to say, this is where I think story fits in again. The "way we ought to be" is exactly story, whereas the "way we are tempted to be" is really who we are, it's our biology.

As I see it, the dissonance is really between who I am, vs. the story I have of myself.

Following up on my post yesterday: just found this review article on motivated forgetting. The gist is that the right dlPFC and vlPFC both activate to inhibit the memory, the entorhinal cortex also activates, as if this is where the lateral PFC is acting upon, and the hippocampus de-activates. It's as if the lateral PFC is "choking off" the data stream at the EC, to reduce likelihood of hippocampal encoding.

matt faw

Yes, to feel proud about ourselves, but also I think to smooth other cognitive dissonances. Finding rationales for ourselves, but also for our loved ones (unless they are opposed to us), and for our beloved religious and political parties and figures.

And often, I think it's just to sort out complicated events and feelings. I don't have as much late-night rumination as I once did, but it used to be robust when I was a teenager-20s, and include extensive replay and rehash of the embarrassing moments, much more than the positive ones. It was salience that brought back the day's memories, much more than pride, and the process rarely made me felt better about myself.

I'd be interested in unpacking the term "superego" more, in conversation with Bob. Although I do not subscribe to the Freudian world-view, I think that there is potential illumination there, for other lines of inquiry.

matt

One thing this discussion raises for me is the difference between my childhood memories and those of my siblings. In particular, one brother tends to heighten the circumstances of the memory, if it makes a better story. So the event in which I remember having a run-in with 5 neighborhood thugs, he remembers there being 10. And in later tellings of the story, it's become 15, and then 20. (Of course, yes, it's possible that I'm the one whose memory has shifted). In fact, it was thinking about this brother that led me to the hippocampus. I called up my dad (who is my science advisor on the doc, and had been giving me my informal education in consciousness), and commented on how similar memory recall and imagination seemed to be, like they were related processes. And he told me what he knew about the whole new Hassabis / Maguire / Schacter line of research. I went to my computer and googled "hippocampus" and "simulation", and everything started to fall in place (unfortunately, this was February 2013, a couple months before Ralf's paper was published, so I didn't connect to him until later). This is part of why I see confabulation (i.e. the wider phenomenon, not the diagnosis) as being such an important part of experience, because the experience is always constructed with the biases built in. The decisions of what goes into the experience and what is left out (or is immediately forgotten thereafter) are made by algorithms / defense mechanisms that are entirely unrepresented in consciousness. Memory is constructed with purpose and meaningfulness to create continuity and confirmation of self- and world-story. And experience, being memory, is built the same way. If I let my desk get messy, then my brain will start canceling out the mess, so I'm no longer perceiving it. If my girlfriend is going to come over, suddenly I have a different set of motivators, and the mess on my desk becomes a lot more obvious to me. I find repression / motivated forgetting / self-blindness an equally interesting phenomenon to embellishment, because they seem like the two gate-keepers to experience, deleting, enhancing, or even completely reconstructing signals / content. And of course, one of the embellishments that I see as important is language thought. In particular, once the original salient memory is formed, it's the "mulling it over afterward" action which seems to make the memory labile, include the narration of hindsight, and reshape the episode for eventual consolidation. matt faw

Hi Bob,

Maybe it's a semantic issue, but it seems to me that confabulation as a phenomenon is much wider than confabulation as a diagnosis.

Yes, confabulation as I formally know it, is something to fill in the blanks for patients with frontal lobe amnesia.

But the very fact that these patients regularly do this behavior says a lot, I think, about the human need to make sense of things, put things in some kind of order (that reflects their beliefs).

Why is it that we know everything at age 18 (especially if we've just found religion or nationalism) but in our 40s, we know less and less? It is because the 18 year old has confabulated reality, has created a level of certainty that the more experienced version of him cannot rationally support. I see this with my (somewhat troubled) nephew, who is trying on different absolutist victim, martyr and hero stories, each which satisfies some longing in him, at various times.

Even you saying, that we "perceive what we believe" seems to be describing a form of confabulation, a constructed reality that specifically avoids causing us anxiety by not challenging our previous views.

matt faw

Hi Ralf,

I would agree that confabulation often is self-serving, but my speculation is that confabulation is equally distributed between what we wish and what we fear. If my girlfriend starts with "I have something serious to talk about", I can hear my little theory-generator sparking all kinds of worst-case-scenario stories to anticipate her subject.

Another example: there are some internet memes running around Facebook, etc. that show "baby yoga" and "Roma acrobats training babies" which show adults doing all kinds of absurd and dangerous-looking things with babies, in the name of making them limber. The videos themselves never show kids actually getting hurt, or even acting scared, but the comments (especially from parents) are like they are watching the work of the devil. Every parent comments with extreme revulsion, and utmost certainty that they have seen child abuse and damage on the video, even though it is not to be found. That is because they have confabulated that damage, in order to justify their own fears. They "know" that the yoga is bad, precisely because their inner voice yells out its freak-out so loudly.

I think that homophobia has been justified similarly; since there is some part of me which reacts with revulsion, therefore I confabulate a moral dimension to explain my disgust.

matt faw

Hi Ralf,

Your explanation of what happens when someone joins a new religion makes a lot of sense to me. There does seem to be what you call an "infantile relatedness", which is even reflected in the religious language of "God's children", the "flock", etc. And it is, indeed, a radical shift, to a more child-like worldview in which there is an absolute truth, a certainty, a way to avoid suffering and death, etc. It is very much like having the chance to cling, once again, to one's mother's leg, and to surrender complexity for reassuring simplicity.

matt faw

Hi Bob and everyone,

A few thoughts:

1. I've recently gone through my interview with Steven Macknick. He focuses on the art of magic, to help explain the neuroscience of attention. He uses the word "confabulation" to explain our visual perception of the world, that what we perceive is dramatically different than the noisy incomplete data that hits the retina. This is pre-language story-telling, inner movie-making, in which we direct stimulus into a film that conforms to our expectations. One could also call this "pattern completion", but I think confabulation is a usefully alarming term for it, because subjectively we only see our perception as "what's real" instead of realizing how much of it we're making up or filling-in.

2. I've always been fascinated by the "new recruit" phenomenon of religions, in which the person who just converted does indeed seem to experience a short-term dramatic shift in the sense of self, one's 'mission, and how one fits into the world. Especially considering how little change happens from self-declared resolutions, like those at New Year's, it is pretty amazing to see how much change can happen (at least for a while) in the lives of the recently converted. Unless the religious shift accompanies a new-found sobriety or whatever, it appears as if the new belief system is responsible for most of the changes. But I don't think it's the religion, because it seems to happen to new recruits to (almost?) all religions. Instead, it seems to be a shift in story. In fact, it may be a lot like what happens when we travel out of the country: our right hemispheres seem to be called into action to explain all of the new stimuli, and we get out of our left brained stereotypical processes.

Now #1 is not necessarily about character, but it is about the drama. Because the movie confabulation may just be: e.g. I find the world ugly (or beautiful). If I am leaning towards seeing ugliness, then I will emphasize that aspect of reality, in my simulation. And I will be offended, just by looking at the world. That is drama, regardless of whether other people are involved.

#2 often involves a shift in #1: world perception, plus in other-people perception, event perception, etc. Many events become translated by a new paradigm. The victim may become the hero and the former perpetrator may decide that he was really a victim.

What I think is unlikely to change in #2 is the overall importance of self, within the model. Even if the previous view was: I'm an asshole, and I make people miserable, then I was still somehow an important asshole. If the new persona is now important, even if in the need to save others, to recruit them to the "right and just cause", then the original persona was probably equally important, just in a different paradigm.

Here's confabulation upon a different axis:

3. Another interview I conducted in Tucson this year was with Jeffery Martin, who studies self-designated "enlightened" people. The diminution of the sense of self is one of the hallmarks of this population, across all kinds of religious and spiritual backgrounds. So in their case, the story of self seems to become a lot less important, at least according to self-reports. However, their backgrounds do tend to explain how they experience "enlightenment" with Christians and Buddhists each recognizing elements of their own belief system within the process. So even though self (which I see as a constellation of habits, a sub-set of which are beliefs) is explicitly less important, the belief system actually becomes more solidified. They begin to see their own beliefs (and even their random thoughts) as perfect, as somehow divinely inspired. So in many ways their delusion is the greatest, because no one can ever convince them to consider a different point of view. Plus, when different "enlightened" people get together, especially those from different backgrounds, they do not recognize each other as such, and assume the others are deluded or fakers. They each know what it's like to be enlightened (at least from the inside), so obviously this other person is not it. It's also important to know that the self-proclaimed "enlightened" people universally feel radically different, after their enlightenment experience, but most of the people in their lives do not notice a difference in their behavior.

Just some thoughts.

matt faw

Bob and Ralf, I just came across this excellent article about the "hero narrative" and confabulation of the self-image. Very interesting, and very fitting for our discussion.

http://aeon.co/magazine/psychology/how-a-hero-narrative-can-transform-the-self/

Dear Ralf,

I'm curious if you know Martin Conway's work. Other than your papers, his were the closest in parallel to my thesis. I relied on Thomas Metzinger's model of self and experience as representations, but Conway's writing covers a lot of the same ground, and is specifically about the relationship between self and memory. He also wrote a lot about memory and narrative, the idea that I was trying to get across in our conversation with Bob. I only discovered his writing at the very end of my research phase, so he didn't influence the paper as much, but I think he will be very helpful for the documentary.

Conway, M. A., & Pleydell-Pearce, C. W. (2000). The construction of autobiographical memories in the self-memory system. Psychological review, 107(2), 261. http://homepage.psy.utexas.edu/HomePage/Class/Psy394U/Bower/11%20Soc%20Cog%20Personality/XX%20Constr%20Life%20Stories/Autobio%20Mem-Conway.pdf Conway, M.A. (2005). Memory and the self. Journal of Memory and Language, 53(4), 594-628. http://homepages.abdn.ac.uk/k.allan/pages/dept/webfiles/4thyear/conway%202005%20jml.pdf

Hi Ralf,

When I was in Tucson recently for the Consciousness conference, I found out that hippocampus specialist Lynn Nadel was teaching at U of AZ, so I contacted him for an interview.

By coincidence, he had somehow found a copy of some draft of my paper online, and read it before the interview. Not surprisingly, the idea of putting experience in the hippocampus was novel to him, and he had the same thalamocortical assumptions as everyone else. But during the interview, we ended up basically talking about the theory, and he ended up agreeing with all the technical points.

One such point was my metaphor of there being a "pyramid of reports" from the senses, through the sensory cortices to the MTL, with the hippocampus on the top of the pyramid, receiving reports from all over the cortex. In fact, Nadel had just published a paper with his vision specialist wife, Mary Peterson, in which they described the visual scene only coming together at the hippocampus.

http://media.wix.com/ugd/b6bc32_851529a9dd4c4c31ad13d3fdc1cc3aa8.pdf

Of course, the hippocampus is only on the top of the perceptual pyramid, but is at different places in regards to associational and simulational work. I'm just now editing the interview with Demis Hassabis, which is all about simulation in the hippocampus.

Speaking of interviews, I just contacted Martin Conway, whose work I cited a good deal in my paper. He's in London, so I have to find an opportunity to fly over and interview you both. Any thoughts as to what time of year would be good to come over? Maybe there is a good consciousness or memory conference in the UK that I could also attend? How far are you from London?

best,

matt faw

I was just working on editing interviews for my documentary, and I was going through my interview last year with hippocampal simulation pioneer Demis Hassabis. He had this to say about the retrosplenial cortex: he says that the output of the hippocampus is probably translated by the retrosplenial cortex so that the hippocampal allocentric view can be represented as an egocentric view, for the sake of the parietal cortex (which works in egocentric views only).

Hi Bob, I've thought of the retrosplenial cortex as part of the default network, and its activity is correlated with the intrinsic hippocampal simulations, like mental navigation, theory of mind, mind-wandering, etc. The dense connections with the hippocampus includes CA1, parasubiculum, and post-subiculum, in other words, regions that are relatively late in the hippocampal processing, suggesting that it is receiving output from the hippocampus, and perhaps modulating it. Vogt and Laureys (2005) pointed to the retrosplenial cortex, as well as other default network structures precuneus and posterior cingulate cortex, as part of the neural net that supports the contents of consciousness. http://www.coma.ulg.ac.be/papers/varia/vogt_PBR_vol150_205_217.pdf I'm sure Ralf will have more to say on the subject. In another paper Ralf wrote last year (which I haven't shared with Bob), he explores the interaction between hippocampal output and certain parts of the default network (largely the vmPFC). My understanding is that the default network uses the new memory as a complex conditioned stimulus, drives emotional and behavioral response to it, helps plan for the future, and adds context to the new memory. best, matt faw

Hi Bob, thanks for your reply.

I think that the brain works on multiple levels, simultaneously. One level is defined by society and language, and includes the "I" as a unified agent, a character, as you might say, working within a confabulated milieu, that is constructed to interface, as well as possible, with the consensus reality around us. On the other level, we're talking about brain modules, algorithms, modulatory neurochemicals, and neuronal democracy, i.e. interpretation of stimulus by population vote.

Personally, I think that the latter level is the more accurate one, and can help illustrate causes of pathology, etc. But clearly, the other level of processing includes all that we consider to be "conscious". On the brain level, self is defined by habit, but on the conscious level, self is defined by belief (belief being a semantic, non-procedural form of habit). These two "selves" may be radically different from each other.

Interestingly, I just got finished reading a blog talking about this, about how confabulation and delusion may manifest themselves very differently through language than through behavior.

http://imperfectcognitions.blogspot.co.uk/2014/06/are-delusions-acceptances.html?m=1

The blog's example is that sufferer's of Cotard's syndrome believe themselves to be dead, and will tell you so. However, they may continue to behave much as they always have, so you'd never know that belief exists.

Likewise, my understanding of patients with anosognosia for hemiplegia: they will not act as if they are fully functional, unless you ask them to move. When you ask them to move, they engage the social self, the self of belief, which views reality from the point of view of the hippocampus. In the hippocampus, the body is a figment, a concept, and so it can move easily, just as it would in a dream. However, in the neocortex, the body response has been severed by the lesion, and so the organic self, i.e. the body/brain, does not try to defy its own paralysis.

In fact, delusion seems to be precisely about the schism between the self of belief and the organic self. The further afield the belief sense roams from the actual responses and needs of the body/brain, the more deluded it is. The simulated self, although it is generally adaptive, can become cocooned by the rigidity of belief, until it is no longer able to function in the realm of the material or social worlds.

Or that self can limp along, as does your character "Eddy", interfacing as best as possible, but unable to witness the inner biochemical milieu which constrains the confabulated one. From within consciousness, which is the confabulated now, only the confabulated self exists, but all of that, of course, is represented on a neurological level by massively distributed and parallel processes.

Personally, I think neither view of self and reality is sufficient to explain pathology, nor even normal function. We have to be able to describe brain events from a neurological POV, but describe human events from a social-/self-model POV. Both make us who we are and influence our behavior, so both need to be considered.

best,

matt faw

Dear Ralf and Bob,

Very happy to hear both of your responses. I'm not surprised that we all have different views on this; precisely why it's nice to hash out these ideas.

By coincidence, I was just yesterday listening to a science podcast in which they were talking about the importance of labels (i.e. language tags for what we experience). One striking example they gave: Russians have separate words for "blue" and "light blue" and thereby, they are able to distinguish shades of the two with much greater accuracy than people who speak languages that only have the one word for "blue". In other words, the ability to perceive is shaped by language.

In communication, the dualism of words forces fake dualism upon the world. We treat "smart" and "dumb" or "good" and "bad" as polar opposites, instead of relative ranges of multiple overlapping spectra. This is probably most pernicious when it comes to race-related words like "black" and "white".

I think language plays an important role in what Bob calls the "characterological drama". If we internalize labels like "rebel", "good girl", "lazy", racist, sexist or homophobic epithets, etc., then our drama can be shaped by our understanding of what those words mean.

But the territory that I think language plays the most important role is in memory. Martin Conway wrote a lot about this, about how memories get boiled down from being the full multimodal movie to becoming something more like a narrative: "my summer in France", or "my hellish commute to work today". The remaining episodic movie clips, audio and stills become illustration for the story. Nadel and Moscovitch's multiple trace theory suggests something along these lines, too: the hippocampus is needed to replay the episodic movie, but the narrative has been recorded in the neocortex, as an organizing principle for the memory.

In my life, I recognize the time lying in bed before falling asleep, as one time in which my brain is hungry to make sense of memories from the day. It plays back events, seeking meaning in them, and that meaning is encoded as language. I.e. "he did this to me" or "I really messed up" or "she was just being mean". These thoughts become encoded with the memory, and inform future recall of it. If I self-justify (e.g. "I was being 'righteous' not 'angry'"), then I will re-classify the emotional context of the memory, affecting not only what I remember, but how and when I remember it.

Michael Gazzaniga, in his 2011 "Who's In Charge?" and elsewhere, talks about "the interpreter", a left-hemisphere language module that constantly feeds a narration of events to help the organism make sense of experience, as it happens. Of course, since I believe that "experience" is actually a brand new episodic memory, then I think the "interpreter" is precisely helping to consolidate memories. After all, language makes an excellent mnemonic, and is much more compact and efficient (albeit less accurate) than a movie. It is much more efficient to say "my boss was a total jerk to me today" and have a few small hyperlinked memory movies attached to that story, rather than to have a full movie of the whole workday.

By the way, Bob, when I'm talking about a 'representational workspace', I am referring to the hippocampus. Specifically, I'm talking about the work by Demis Hassabis, Eleanor Maguire, Dan Schacter, et al, who have been imaging the hippocampus, in relation to the default network, in the staging of mental rehearsal and other interior scenarios, including social theorizing. In each of various simulations, this network is representing other people, objects, one's own body in physical rehearsal, etc., in order to pre-visualize how things will play out.

We can see how language plays a role in this inner representation when we read a menu, and invoke sights, sounds and smells in our head. Or when a friend tells us a story, and we involuntarily see some of the details in our own mind, even though we didn't experience them directly. So how different is it, when I tell myself a story, and re-arrange my own memory details, in order to support that story?

best,

matt faw

Ralf, following is a hypothesis that's been buzzing around my head for awhile. Bob and Dad have already read the following, from my dad and my paper, about language thought and the hippocampus. Still, I think there's plenty here to unwrap and discuss.

best to you all,

matt faw

Language Thought:

In a brain in which data moves very quickly between modules, it is a mystery that we are also reliant upon inter-brain data transmission which is represented in a form as slow and imprecise as language. Surely other forms of intra-modular communication must be more efficient than language thought.

HST: We think it likely that language thought is one form of representing pre-memory cognitive processes, for use in the simulation. One aspect of this is specifically to simplify memory processes, by giving the episode a narration [e.g. ‘he did this to me’ (Gazzaniga, 2011)]. This allows the memory to be lean in terms of content, and to be easily understandable upon recall. It’s also a potential source of error, as language implies a level of certainty that is often not warranted in a world of complex causality and incomplete evidence.

Language thought is probably also a tool of intrinsic simulation, like weighing the pros and cons of a decision. Various behavioral maps [e.g. in the ventromedial prefrontal cortex (Behrendt, 2013b)] suggest different choices or outcomes, which are then sequentially formed into language using the left hemisphere’s language encoder, and then passed into the HF to buffer in memory. This allows multiple options to be weighed simultaneously. This is what we know as ‘conscious deliberation’, but not because there is a consciousness doing the deliberation. Rather, pre-memory structures use the HF as a problem-solving workspace in which to manipulate representations, and the HF activity consequently gives rise to subjective experience / memory.

Thank you, Bob. I wonder if the end of your e-mail points toward a bridge between your book and a hippocampal model of subjective experience.

In particular, in my paper we claim that the neocortex has its own set of representations that are specific to how it processes immediate interaction with the world. However, none of those representations are available to experience (i.e. memory). What is available is the post-facto reports of neocortex activity, sent to the hippocampus for simulation into the experience, for possible storage.

This implies (at least) two levels of mapping. External reality (especially viz. vision) cannot really be mapped until birth, because of the lack of reality to interact with (mostly proprioceptive, motor, and some audio mapping during gestation). So upon birth there begins to be a neocortical mapping of external stimuli and their meanings, that helps the baby recognize objects and people.

The hippocampus, however, takes some time to come online, and so its mapping comes later. In the MTL, most visual objects are mapped to the perirhinal area (via concept cells), which supplies a mature hippocampus meaningful objects to be added to the spatial simulation (which has been mapped through the parahippocampal cortex). These two mappings come together in the entorhinal cortex to form a scene with objects, and are brought into the hippocampus for final simulation that includes self- and other-mapping as well as location and objects. Each step in the MTL is about activating mapping to bring meaningfulness into the scene, including reports from the amygdala, vmPFC, hypothalamus etc. Ralf's excellent "Appetitive Behaviors" paper from late last year dealt with a good deal of that.

The neocortical mapping drives immediate interaction with the world, and so this is the personality that other people see, and may be very different than the personality that is represented back to the person, from within the hippocampus. In fact, there can be a great divorce between the actions and feelings in immediate behavior vs. the introspected actions and feelings that one creates in memory. For example, someone can appear and behave as if they are full of anger, without being aware (from within experience/memory) that they are feeling that anger at all.

The belief we have about ourselves may be formed from experience, but of course, it also shapes experience of ourselves. Not only in the immediate moment, but also in how we remember our past. Repression of old memories, like the times when I acted like a jerk, is the norm, I think. We are very good at maintaining the memories that support our self-image, and equally good at forgetting those memories which don't.

IMO, the hippocampal simulation is a good deal of where the play of reality takes place. It is at least where the play is remembered, which shapes future drama. It is also where the mapped 'other' is brought into the simulation, where attempted mind-reading is added to experience. You may remember my hypothesis about Capgras Syndrome, in which the mapping for "mom" breaks down, and therefore she is simulated as a stranger-who-just-happens-to-look-like-mom, an impostor.

In my paper, I dealt only superficially with the causal nature of "habits of simulation", tendencies to construct the world in a way that satisfies some self-image or biases about reality. I dealt with it superficially because it felt like a whole other can of worms, but I think that it's some of the most important take-away from this theory. We need to know that we are creating and biasing reality moment-by-moment, because otherwise reality continues to seem self-evident. And we need to realize how much we confabulate our experience of self, because otherwise we won't wake up to our responsibility to behave well in society.

best,

matt faw

I was reading Bob's blog just now, about nature and nurture, and that brought me back to the question of mapping.

The way I look at it is: when the blastula in the womb begins to subdivide, there is no differentiation between cells, not to mention structure of the body. But gestation is a process not only of division but of cells identifying themselves, strictly by the nature of their neighbors. What neighborhood you are in decides whether you become a heart cell or a neuron. This is autopoetic mapping, organization from within.

As the body and brain continue to mature in the womb, "nature" is still connecting wires, but it's starting to involve "nurture" (i.e. experience). As the fetus kicks, it is testing muscular pathways, proprioceptive response, etc. This is a Hebbian mapping of the body upon itself, through internal experimentation.

The first 6 months outside the womb reflect the body still getting to know itself, learning its muscular feedback, starting with mouth and fingers and eyes. This is when vision begins to get mapped (albeit onto a cortex predisposed toward human sight). What are these blobs? What do they signify? What are the rules by which they operate? If the infant is deprived of some aspects of vision (like color or vertical lines), then its brain doesn't learn how to process them.

And then, as you point out, the emotional mapping starts (probably in the womb, if there is too high a level of cortisol, etc., but also) postpartum, in engagement (or lack thereof) with the parents, siblings, etc. Emotional mapping is not only about temperament, but about survival: strategies of navigating this particular family and environment. If mom freaks out whenever baby cries, baby may learn alternate ways of expressing herself.

I think that's part of why those first few years are so powerful in the shaping of personality, because the major emotional strategies and habits have been deeply mapped onto the organism, and they shape the lens by which the person interprets and interacts with the world, thereafter.

Hi Bob,

Thanks for your divergence!

Ralf could probably address this better than I, but I see various forms of mapping at play here. One is via concept cells, primarily in the perirhinal area, but also in the EC and hippocampus. These are indexing place holders which bind various object features into one concept, whether various viewed angles of the same object, or various ways of considering Jennifer Aniston, for example: her image, voice, name as spoken, name as written.

Another form of mapping is the way that memories index to each other; the previous experience informs how I simulate the current one. That mapping actually happens within the hippocampus, attractor dynamics in field CA3 eliciting related memories. Again, Ralf can tell this a lot better.

There are maps of actual geographic locations that help us navigate 3D environments. These are probably accessed through the parahippocampal place area, before being fed through the EC grid cells to the hippocampal place cells for binding with object information. The PHC place area translates the parietal lobe's egocentric "where" dorsal visual stream and translates it to an allocentric hippocampal representation, and as you mentioned, Bob, the retrosplenial cortex seems to translate the allocentric hippocampal output back into an egocentric view for outputting back to the parietal lobe.

Ralf wrote quite a bit about behavioral maps in the ventral-medial PFC, in which the current "situation" (i.e. hippocampal output) elicits motivational arousal and behavior. I think of this kind of mapping as complex algorithmic if/then mapping. The amygdala also affects immediate behavior, through concept cell mapping of apparent threats.

I have also heard speculation that the vmPFC contains Theory of Mind mapping of other people, although I'm not sure Ralf would agree with that. Somewhere in the default network, at least, there seems to be personality mappings that allow us to make predictions about other people, especially people we know well.

The body is mapped in the parietal lobe (somatosensory), the motor cortex (as procedural memory), the cerebellum, and (in my paper, at least) also in the hippocampus, as a concept. The neocortical maps all seem to interact together to guide intentional movement, and the cerebellum probably compensates for that movement, adjusting the rest of the body to keep balance and coordination. In my paper, the body is represented in memory predictively as a concept, which is why so little of the vast amounts of information that the brain receives about the body actually makes it into experience. As Michael Graziano said it (in my interview with him in April): the body seems subjectively to move by magic, because most of the interaction of muscle and sinew is left out of the simulation.

And then self is mapped through all of these, as a chain of memories, as concept cells, as body schema, as behavioral repertoire. Self is not easily reducible to any brain part, process or concept, since it really reflects processing of the whole brain and body.

H.M. certainly would respond like you said: he'd forget that he had met any of the researchers, but would be inclined to move towards the ones who were nice to him and away from those who were not. This shows clearly that episodic memory is not the only kind of memory we have and use (although without episodic memory, new semantic and procedural memories take a lot longer to become knowledge/habit). H.M. had a good deal of self-mapping, and would recognize his wife. But there was also a good deal of what we normally think of as self, that was probably missing with H.M. The Damasio quote I used about Daniel was the most clear-cut description I've heard of what's missing. But I've also heard from Lynn Nadel and others that the AOCBHD patients are missing important elements of spatiotemporal processing and of self-awareness, introspection and personal narrative.

best,

matt faw

In direct response to Ralf's post: I think this very question (the anticipatory image) is what was tested with DB in the Klein, Loftus and Kilhstrom paper in 2002:

http://guilfordjournals.com/doi/abs/10.1521/soco.20.5.353.21125

Plus all of the Maguire and Hassabis, Dan Schacter, etc. experiments in the same vein.

The problem with the simple argument is that everyone I've had this discussion with is so sure that consciousness is necessary for current interaction with the world, that they won't even consider otherwise. Even my use of the term "subjective experience" still ignites the same argument, but with less certainty than when I use "consciousness".

I think that part of that problem is that "conscious" also can mean "responsive" (as in, not in a coma), and since H.M. was responsive, he was assumed to also be "conscious". Of course, plants and protozoans are also responsive, but presumably have no experience, which is really the distinction we're making.

I think that Gerald Edelman would've arrived (or inspired someone to arrive) at the hippocampus, a long time ago, if not for H.M., DB, KC, etc. And it is damnably hard to imagine what their experience must be like, lacking memory.

Dear Ralf, in particular,

Bob and my dad have both seen this newer section from our paper, but I don't think I had such an extensive H.M. rebuttal in the draft that Ralf has seen.

Whenever dad and I explained our theory to people in the field (e.g. during our poster presentation in Tucson this April), the first question was always about H.M.

Following is my response about H.M. and similar patients. I'm curious to hear what you think, whether my response is sufficient, and whether are other arguments that should be brought into this question.

best,

matt faw

What does this say about H.M. (and other patients like him)? Henry Molaison, commonly known as H.M., is the most famous patient with adult-onset complete bilateral hippocampal damage (AOCBHD). Most of his MTL was removed surgically, to save him from the overwhelming epilepsy that was focused there (Scoville & Milner, 1957). Patients like H.M. are unable to form detailed or coherent intrinsic simulations, like mental navigation, future projection, imagination and social rehearsal (Spreng, Mar & Kim, 2009). These patients are afflicted with anterograde amnesia, the inability to form new episodic memories, and retrograde amnesia, the inability to replay previous episodic memories (Kartsounis, Rudge, & Stevens, 1995).

In the model we are proposing, such patients do not have SE, which may sound implausible, given SE’s usual definition. But this theory’s definition of (what those of us with working hippocampi call) ‘subjective experience’ is a brand new episodic memory, an after-the-fact representation of the perceptual, cognitive, affective and motor processes that came before it. That is what H.M. was missing.

We with working hippocampi have the same approximate pre-memory representations and processes as H.M. None of us, including H.M., act directly in response to the hippocampal SE simulation; we act according to representations in the neocortex and other pre-memory structures (see figure 1, below). As we can see in the case of ‘Driving Mind’ (section 2.1), the sensory and motor cortices, guided by the task-positive network, can drive the car, at the same time that the MTL is being used by the default network to construct intrinsic simulations (e.g. daydreams). This shows that SE, although it usually feeds back to the rest of the brain one unified story of “what just happened”, is not necessary for complex interaction with the world. Only upon introspection or recall [both which are forms of memory search (Edelman, 1989)], do we act specifically according to our hippocampal simulation (see section 2.13 on Anosognosia).

What is different for those of us with working hippocampi, is that episodic memory is being generated (almost) all the time, and so we believe that episodic representation to be the experience, the ‘I’, the actual self-in-the-world. H.M. could interact with the world as we can, but was never generating the movie of memory, and so did not suffer from the illusion that the rest of us do. Those of us with intact MTLs are fooled by memory in a way that H.M. could not be (see section 2.10 on Confabulation).

Antonio Damasio, based on his work with AOCBHD patient Daniel, said that Daniel does not have “an elaborate sense of self … at a point in individual historical time, richly aware of the lived past and of the anticipated future” (1999, p.16). What is missing in Daniel is the sense of self and context that is provided by memory.

Unfortunately, it is impossible to guess what H.M.’s non-memory experience is like, since our own normal pre-memory experience is not available to our subjective view. Nor can we ask patients with AOCBHD whether their perceptions have changed, post-morbidity because, missing their MTLSN, they can no longer evoke the vivid memories of previous experience to compare to their current way of seeing the world.

But we predict there are probably subtle testable differences, [e.g. egocentric vs. allocentric views of the world, or differences in timing or ability for certain associational, metaphorical, spatial and representational tasks], which may help distinguish our SE from AOCBHD patients’ non-memory experience.

Dear Ralf,

Thank you for your e-mail and good words. I'm happy to do what I can to popularize the idea, as my documentary will need some consensus behind the idea, by the time it's released. I don't want it to seem too out on the fringe. Do you know how good it was to find your 2013 paper? The theory always made total sense to me, but it wasn't until I found your paper that I felt like I wasn't alone, like it was grounded in reality.

As you say, part of gaining traction for the idea includes appealing to those who are philosophically aligned with us, which is why I was so eager to include Bob Berezin in this conversation. He has a whole framework of self and experience as representation, which is very much in line with what you and I have been talking about. His model did not attempt to bring in a neural correlate, but it dovetails perfectly with the hippocampal theory.

That's also part of why I've been interested in Thomas Metzinger's work for awhile, because he writes about self and consciousness as representations. I finally got to interview him this spring, and at his solicitation, I have sent him both of our papers, but I have not heard back from him about them.

I think there are certain consciousness luminaries in particular who will also be in alignment with the theory. Michael Graziano is developing a radically parallel theory (consciousness as information), but he's looking at the temporal-parietal junction as the NCC, because it's so connected to intrinsic simulations. I think also Susan Blackmore, the Churchlands, Bob Kentridge, Demis Hassabis, Itzhak Fried, Martin Conway, and Jesse Prinz, among others, may be friendly to the theory. I'd like to include Daniel Dennett and Bernard Baars on that list, but they each have their own theories to defend.

Lynn Nadel, who I interviewed in Tucson, had actually stumbled across an early draft of my paper, and brought it with him to the interview. He had the usual immediate response of: "hippocampus can't be the seat of consciousness because of H.M." that everyone says, but as we talked, he ended up confirming major parts of the theory, and acknowledged that our theory was more likely than he had originally assumed. I've also sent him your paper and my updated one, but I haven't heard back from him about them yet.

As for the experimentation question, I agree that Alzheimer's patients, schizophrenics and hippocampal-damaged patients are all difficult subjects, because it is very difficult to parse out the various effects of the injury. Russ Hurlburt, who I interviewed about his pager-studies of inner experience, said that any amount of brain damage makes it very difficult to explore subjectivity, because the ability of the subject to report accurately becomes so distorted.

That's part of why I think that timing experiments, like the ones Libet conducted, are probably the best way in. If we can disentangle "the neocortical ability to respond" from "consciousness", then we illustrate that they are separate things, and that the latter follows after the former. And because they can be conducted on non-pathological subjects, we can be sure we're talking about normal consciousness.

Also, Chadwick, Mullally and Maguire's "boundary extension" experiment reveals a promising possibility. The hippocampally damaged subjects were able to complete the task more accurately than the normal subjects, because the normal subjects' hippocampi fooled them, by automatically expanding the boundary of image they were supposed to copy from immediate memory. The AOCBHD patients drew from their neocortical representations, and were thus more accurate in terms of boundary than the normals. This kind of experiment that shows us being fooled by the simulation in a way that AOCBHD patients are not, seems like very strong support.

best,

matt faw

Dear Ralf,

Thank you for your response. Good stuff to think about.

As you know, my paper focuses heavily on what you point out in your last sentence: trying "to demonstrate the explanatory and integrative power of the paradigm". That's what my section 2 is all about.

Personally, I haven't yet heard of any other explanation for consciousness that fits such a broad constellation of pathologies and subjective mysteries. Plus, there is so much new hippocampal research which points to it as a simulator of experiences. So I do feel like there is plenty of indirect evidence already in place.

Plus, there is so much of the ordinary visual scene that only comes together in the MTL, that it seems impossible for the 3D spatiotemporal experience to arise, prior to it. I am just now going over my interview with Lynn Nadel from Tucson this year, and his new paper is very clear that the visual scene only comes together at the hippocampus, and he gives the mechanics of EC grid cells and hippocampal spatial scaffolding to show how the PHC/dorsal stream creates the allocentric map that is adorned with objects from the perirhinal area/ventral stream.

I think that is one potential realm for experimentation, looking at elements of subjective experience that only come together at the hippocampus, and show how those elements are missing from patients with what I call AOCBHD (adult-onset complete bilateral hippocampal damage). Nadel, in his interview, confirmed that patients with AOCBHD do have strange deficits when it comes to space and time, but unfortunately he didn't elaborate on that. Both sequence and spatial manipulation may be areas for experimentation.

Another such element of normal subjective experience is boundary extension, which Eleanor Maguire's lab identified as hippocampal. Without that, supposedly vision would be streaks of saccades, without a sense of a stable allocentric world, little stitches of images and blur that don't quite come together. One possible experiment to come from that would be to have some kind of meaningful visual stimulus that is too widely spread out to view in one glance, like a large projected pattern. The subject, whose head might be constrained, would have to scan his entire peripheral vision, in order to figure out what the pattern is. That pattern would have to be "stitched together" mentally into one visual representation, since the whole could never be seen at once. Now, I don't know if the visual cortex can do that without the hippocampus, but assuming no, that might be one way to demonstrate that the hippocampus is needed for this simple trick of normal consciousness.

What you wrote about theta rhythms makes me wonder: is there a measurable functional or subjective blip every ~140ms? In our paper, my dad wrote that the theta brings in more entorhinal cortex data, when it's at its peak, and brings in more CA3 auto-associations, during the theta down-cycle. That kind of oscillation could conceivably leave some kind of fingerprint, although it's possible all that has been averaged out, by the time the final memory is released from the hippocampus.

I've also been thinking about how we track objects that are outside of our view at the moment. This would be a component that likely could only take place in the MTL, because it involves a representation that does not appear directly in the sensory fields, but only appears as a concept, out of view. Thus tests could be made for patients with AOCBHD, to see if their tracking ability is impaired.

Unfortunately, with this group of patients, it might be hard to divorce any results from their whole memory loss aspect. It's also hard to predict how they should experience the world, since I lack access to my own pre-memory processes.

One group of patients that I think may be very telling are patients with anosognosia, because they are otherwise quite mentally intact. My hypothesis is that at least some forms of anosognosia represent a divorce between the neocortical and hippocampal body-representations. If so, that divorce seems like a natural fault line for experimentation.

Also patients with alien hand syndrome or phantom limb also seem to be revealing similar fault lines. In fact, my understanding is that the hippocampal representation of phantom limb takes 18 months to update, as evidenced by how long it takes for the amputee's dreams to include the experience of the missing limb. That's another natural fault line.

If cold water in the ear temporarily revives the anosognosic hemiplegic from being stuck in his hippocampal representation (as per Ramachandran), maybe it could do the same for someone with DID, Capgras syndrome or even schizophrenia?

Oh, I just remembered an experiment that I heard about but couldn't find the source paper for. I think it provides robust support for our theory. It goes like this: subjects are asked to push a button, as soon as possible, when a buzzer goes off. This would be a neocortical response, immediate reaction. Then the subjects are asked to delay their response for 100ms before responding, but otherwise still respond as soon as possible. However, the experimenter found that the subjects delayed by an additional 500ms or so, without being subjectively aware of the additional lag. This of course is because they introspected to find the 100ms delay, which left them responding to the hippocampal representation of the buzzer, rather than the neocortical one. I think this would be an excellent experiment for our theory, because it entirely relies on normal subjects, and is precisely about the delta between the neocortical and hippocampal representations. If you extend this experiment to epileptic patients at Itzhak Fried's single-neuron measurement lab, then we should be able to simultaneously show when the buzzer (or similar trigger) stimulus reached the MTL, demonstrating that the ~100ms subjective delay is related to that stimulus.

Timing issues in general are I think a great area for experimentation, like Libet's work.

The more I think about it, there are probably a ton of experiments, none or few of which would be smoking guns, but all of which would help support the theory. If the theory fits a wide-pattern of observable phenomena, as I claimed in my section 2, then each case probably offers some testable hypotheses.

best,

matt faw

Dear Ralf, Bob and Dad,

As I mentioned to each of you, I was hoping to get a conversation started that included all of us, and which hashes out some of the more ambiguous areas of our respective theories. I know that Bob has read Ralf's paper, and I have sent Bob's blog and a link to his book to Ralf, so y'all know some about each other. My dad is very busy preparing to move, but I'm hoping he can at least enjoy the conversation and join in here and there.

The first big topic was suggested by Robert Reid who found our poster presentation at Tucson compelling, and who has since read our and Ralf's papers. He asks: "Have you thought of your own confirming brain experiment – or is one even possible? (Is self-reporting on subjectivity reliable?)"

Well, I don't have a confirming brain experiment, although I had a couple ideas for possible directions.

One is a take-off on Libet's timing experiments; if our subjective experience of will follows the neocortical activation of action, then a patient with adult-onset complete bilateral hippocampal damage (AOCBHD) may not have the same delay. However, they also may not be able to follow the instructions, either, because of memory loss, so that might be a confounding issue. More likely would be Libet's other timing experiments, in which he stimulated the somatosensory cortex, and witnessed a half-second delay in the patient's report of the sensation. I think that delay is likely to vanish, in patients with AOCBHD.

Another possible experiment could utilize the 'boundary extension' function of the hippocampus, as detected by Eleanor Maguire's lab. This is the function that allows a greater scope of the scene to be 'perceived' than what the eyes' immediate saccades can allow. This gives vision its sense of stability and breadth. Patients with AOCBHD can reproduce pictures more accurately than those of us with working hippocampi, because our brains automatically add more boundary than exists in the original. I'm not sure exactly what the experiment would be, but that seems like an area that is ripe for testing.

It might also be possible to use trans-cranial stimulation to inhibit the entorhinal cortex (hub to the hippocampus), and see what happens to subjective experience. Or perhaps Itzhak Fried's UCLA lab (in which epileptics are wired up with deep-brain single-cell electrodes, often in their hippocampus) can try certain timing experiments (his timing experiments thus far have corresponded very closely with Libet's findings). Or he may even be able to stimulate the hippocampus, and observe results.

Anyone have thoughts as to what would be the smoking gun experiment?

Thanks, and I look forward to hearing y'all's ideas!

best,

Matt Faw

Hi Ralf, I hope you're well. I've begun to strike up some new conversations with far-flung thinkers who are telling a very similar story to yours and mine. One is an American psychiatrist, Bob Berezin, who writes a blog on the Psychology Today site. His personal version of that blog is at: http://www.robertberezin.com He and I are beginning an e-mail conversation that reminds me a lot of your and my correspondence. And it made me wonder if you'd like to be cc'd on that correspondence, if you'd like to be part of a discussion of three people who each have come to very similar conclusions, but also who have some different opinions on how things play out? I sent him your paper from May of last year, and he really liked it. Speaking for myself, I have a ton of thoughts and questions that pass through my head about these issues, and I mostly keep them to myself, because there's no one really to talk to about it, in my day-to-day life. It'd be nice to hash some things out with a couple of friendlies! Speaking of no one to talk to, I just finished a 6 min. video explainer to encapsulate the idea for my friends and family. It's not part of the documentary, just a communication tool, to get the rough gist across. I hope you enjoy it: https://vimeo.com/98785998 Take care, and I hope to talk more to you soon! matt faw p.s. Bob's book is available at: http://www.amazon.com/Psychotherapy-Character-Consciousness-Theater-Brain/dp/160494918X/ref=la_B00D4C7K36_1_1?s=books&ie=UTF8&qid=1404514605&sr=1-1

Hey Ralf,

I don't recall where I heard this metaphor, but I think it's a great one for the documentary:

The hippocampus is like the TV station of the brain. It receives reports from embedded reporters throughout the whole brain, and it turns them into the "news", a broadcasted master report which lets all the various modules know what's going on. Like all broadcast news, it's a vastly simplified view of reality, but because it is the report which is most widely shared, it's also what is believed.

I'm curious what you think.

matt faw

Hi Ralf, When I was in Tucson recently for the Consciousness conference, I found out that hippocampus specialist Lynn Nadel was teaching at U of AZ, so I contacted him for an interview. By coincidence, he had somehow found a copy of some draft of my paper online, and read it before the interview. Not surprisingly, the idea of putting experience in the hippocampus was novel to him, and he had the same thalamocortical assumptions as everyone else. But during the interview, we ended up basically talking about the theory, and he ended up agreeing with all the technical points. One such point was my metaphor of there being a "pyramid of reports" from the senses, through the sensory cortices to the MTL, with the hippocampus on the top of the pyramid, receiving reports from all over the cortex. In fact, Nadel had just published a paper with his vision specialist wife, Mary Peterson, in which they described the visual scene only coming together at the hippocampus. http://media.wix.com/ugd/b6bc32_851529a9dd4c4c31ad13d3fdc1cc3aa8.pdf Of course, the hippocampus is only on the top of the perceptual pyramid, but is at different places in regards to associational and simulational work. I'm just now editing the interview with Demis Hassabis, which is all about simulation in the hippocampus. Speaking of interviews, I just contacted Martin Conway, whose work I cited a good deal in my paper. He's in London, so I have to find an opportunity to fly over and interview you both. Any thoughts as to what time of year would be good to come over? Maybe there is a good consciousness or memory conference in the UK that I could also attend? How far are you from London? best, matt faw

Dear Ralf,

Following is a hypothesis that's been buzzing around my head for awhile. Wondering what you think about this:

Language Thought: In a brain in which data moves very quickly between modules, it is a mystery that we are also reliant upon inter-brain data transmission which is represented in a form as slow and imprecise as language. Surely other forms of intra-modular communication must be more efficient than language thought. HST: We think it likely that language thought is one form of representing pre-memory cognitive processes, for use in the simulation. One aspect of this is specifically to simplify memory processes, by giving the episode a narration [e.g. ‘he did this to me’ (Gazzaniga, 2011)]. This allows the memory to be lean in terms of content, and to be easily understandable upon recall. It’s also a potential source of error, as language implies a level of certainty that is often not warranted in a world of complex causality and incomplete evidence. Language thought is probably also a tool of intrinsic simulation, like weighing the pros and cons of a decision. Various behavioral maps [e.g. in the ventromedial prefrontal cortex (Behrendt, 2013b)] suggest different choices or outcomes, which are then sequentially formed into language using the left hemisphere’s language encoder, and then passed into the HF to buffer in memory. This allows multiple options to be weighed simultaneously. This is what we know as ‘conscious deliberation’, but not because there is a consciousness doing the deliberation. Rather, pre-memory structures use the HF as a problem-solving workspace in which to manipulate representations, and the HF activity consequently gives rise to subjective experience / memory.

I think this very question (the anticipatory image) is what was tested with DB in the Klein, Loftus and Kilhstrom paper in 2002:

http://guilfordjournals.com/doi/abs/10.1521/soco.20.5.353.21125 Plus all of the Maguire and Hassabis, Dan Schacter, etc. experiments in the same vein. The problem with the simple argument is that everyone I've had this discussion with is so sure that consciousness is necessary for current interaction with the world, that they won't even consider otherwise. Even my use of the term "subjective experience" still ignites the same argument, but with less certainty than when I use "consciousness". I think that part of that problem is that "conscious" also can mean "responsive" (as in, not in a coma), and since H.M. was responsive, he was assumed to also be "conscious". Of course, plants and protozoans are also responsive, but presumably have no experience, which is really the distinction we're making. I think that Gerald Edelman would've arrived (or inspired someone to arrive) at the hippocampus, a long time ago, if not for H.M., DB, KC, etc. And it is damnably hard to imagine what their experience must be like, lacking memory.

Hi Ralf, good to hear from you.

You wrote: "from what you wrote i got the impression that part of you does believe after all that conscious experience of the world is somehow a function of the neocortex; you seem to not completely agree with my suggestion that conscious experience of the world around us is indistinguishable from episodic memory; memory is not just the past it is also the present; memory captures the present and can be recalled as the past"

What I believe is as follows: in the present moment, the functioning of the sensory, motor and task-positive cortices is not directly dependent on the hippocampal simulation, or else HM would have not been able to function. The new moment memory is (in those of us with working hippocampi) being generated in the background of the current task, but it is only relevant in the way it top-down informs the various neuronal populations of 'what just happened'. During deep involvement in a world-oriented task, neocortical perceptual processing mostly ignores the hippocampal simulation, because it would be too slow to be the main input.

All of the actual task-positive work is being done via 'unconscious' (i.e. pre-memory) processes working on bottom-up data flow from the senses (as informed by top-down guidance from the MTL simulation as well as all the various higher processing neocortical areas). In fact, all processing in the brain is essentially unconscious, because 'conscious' only means "included in the memory representation". It is never the process which is conscious, only the after-the-fact representation of or report from, the process.

Since the memory is all that is left of the moment, we believe the memory to be true, and so any mis-representations from the task get enshrined in our view of reality.

In much of life, however (especially social life), we are not just deeply focused on a task, but are continually shifting back and forth from engaging with our TP network, and engaging with intrinsic simulations in the MTL. For example, "how do I look right now?", "is she getting bored?", or "what was that look on his face just now?" are all forms of hippocampal simulations, of the self, of her, or recall of the immediate past. All introspection is retrospection, and this is when we are most involved in the hippocampal simulation, when it forms the most immediate sense of reality and of self.

This is born out by an experiment that I've seen described, but have not yet found online. The researcher (I think a Dr. Johnson) first asked the subjects to hit a button as soon as a cue appeared. They were able to respond in a little over 100ms. The researcher then asked the subjects to delay ~100ms, after seeing the cue, before hitting the button. However, their response times in the second experiment jumped to ~800ms. My interpretation of that is that in the first situation, the subjects were responding directly to neocortical representations of the cue, and thus, were able to respond quickly. They were responding in the way that H.M. can respond. However, given the second condition, the subjects had to introspect upon when they noticed the cue, in order to add a delay. But because they were introspecting a stimulus that could not be predicted, they were not 'conscious' of the cue until ~500ms, which is the time necessary for the predictive memory simulation to be retroactively edited, to add in the cue. When the subject references the neocortical representation of the cue, the response is much faster than when he references the hippocampal representation.

So yes, I agree, our introspective view of reality is always of the hippocampal simulation, even in the present moment. This world that I'm experiencing is the hippocampal simulation, which is built out of reports from the neocortex, but which does not take place within it. This sense of self is entirely hippocampal. All experience itself is memory, the brain reflecting back to itself the reports from other modules, those reports which are bound together in the MTL.

But I think a good deal of the time, our processes are relatively independent of that new memory, and are based upon pre-memory representations. These are all considered to be "unconscious" but they are not "dumb" or "automatic" (or no more so than any brain process). These processes are fast, and are equipped to deal with partial information, fragmented bottom-up representations, and are not reliant on the memory simulation to sense, decide and act.

These neocortical processes are NEVER directly available within conscious experience (i.e. memory). All that is available is a report of those processes, in the form of a thought, image, intuition, fringe sense, etc., sent from the processing module to the MTL, specifically for the purpose of being bound into the full 'consciousness' simulation.

As I wrote in the paper: there is no such thing as a 'conscious process'. Just like dreams of me-performing-processes seems real, so too does the memory of me-performing processes. But in each case, the process happens unconsciously, and is only represented after the fact by the memory simulation. And/or, during intrinsic hippocampal simulations, the unconscious brain uses the hippocampus as a workspace, and manipulates representations within it. But the sense that 'I' am doing the manipulation is just an illusion, as 'I' am just another cobbled-together representation.

Hi Ralf,

Thank you for your reply.

I agree that a good deal of life probably happens without the person ever responding to the new memory simulation being created in the hippocampus. Driving mind is the most obvious example, since it happens during a dangerous activity, but I'm sure that there many stretches of daily life where inner and outer life are similarly divorced, from seconds to tens of minutes, if not more. Probably a great deal of assembly line factory work is done 'unconsciously'.

However, that's a tricky assertion to make in a paper, because I don't know where I'd get a citation for it.

I agree; it seems necessary that someone with complete bilateral hippocampal damage MUST have a different subjective experience, whether or not we're right about consciousness being an output of the hippocampus, just because the hippocampus (and previous memories) bring so much to the visual scene. However, finding some such statement in the literature is proving very difficult. Mostly the statements are along the lines of "other than memory, all cognitive function seems intact" or "his personality didn't seem to change much". Our most skeptical referee claimed that he had had plenty of experience with such patients, and they clearly had "subjective experience". I don't know what that referee was referring to, and so I have a hard time imagining how to convince him otherwise.

My current argument in the paper is: when it comes to immediate interaction with the world, our neocortex is not dealing with the hippocampal representation; rather, it's working from it's own representations, and then sending evidence of 'what happened' to the MTL for binding into memory, which is of course, what we remember as the interaction, even though the real (neocortex) interaction happened prior to the memory. It's only when we introspect or recall that we are specifically engaging the neocortex with the memory encode. That's my argument, but it's filled with assertion, and no reference.

In social life, I think we move quickly back and forth between the two representations, engaging in the conversation with the neocortex and then using the hippocampal simulation to ask one's self: how am I seen? did I say the right thing? what was that look on her face? how's he going to respond? etc.

In the case of anosognosia for hemiplegia (I'm guessing), the patient's neocortex is "aware" that one half the body is non-responsive, but the pathways which should get that information to the MTL are off-line, and so it never makes it to memory. When the doctor asks the patient to move, and then asks him if he did move, he is asking the patient to introspect upon a brand new memory, and that memory was created predictively, with the movement built in. However, it is only while consulting the MTL representation of self-and-body that the patient would try to move, because the rest of the time, it's abundantly clear that they can't move. Behavior arises from neocortex alone, but belief is based upon memory (flavored, of course, with semantic assumptive bias). That's the most reasonable explanation I can think of, but I certainly can't test it without access to such patients.

Same of course with hippocampal-damaged patients. If every speculation in a paper needs to be sourced, but no one has yet asked the questions experimentally, it makes it hard to get references.

As for HM, I read just the other day that small bits of his hippocampus were left in place, but they were isolated from their connections to other brain regions, so they were functionally moot.

I'm curious how you deal with HM viz. your "appetitive behaviors" paper. If the vmPFC is conditioned to the situation, as presented by the MTL, then what happens when there is no MTL? It seems like something must change, because the information flow has been cut.

Maybe that answer is in the last couple sentences of your most recent e-mail, about "it is bound to reduce the adaptability of behaviors...", etc. At the least, that gives a fair list of testable hypotheses, of probable subtle changes to look for, when testing this patient population. I hope we can get to the point where someone with access to these patients takes this theory seriously enough to test for these changes, because I haven't found any evidence that these tests have yet taken place. (And as I am learning about academic papers, nothing is considered worth listening to, unless someone else in a lab has said it first).

best,

matt faw

Hi Ralf,

Thanks for the distinctions viz. task-positive and -negative modes.

I'm glad to hear your mom is well. Both my parents came to Tucson, so it was a good chance to catch up, and have them meet my girlfriend. My folks live on the east coast, 3000 miles from me, so I don't see them as often as I'd like.

When at Tucson, my dad and I presented the theory as a poster, and received some interest. I hope you don't mind I'm sending some people your paper, as well as ours, to give them a more robust treatment.

While presenting the poster, the most common first response was about H.M. and other patients like him. My argument is that it seems implausible that H.M. not have 'subjective experience' only given the old definition of the term, but if we just see subjective experience as the memory that follows the neocortex's interaction with the world, and forget the notion that it is the "inner light" or intelligence of the brain, then it no longer need seem impossible. I use "driving mind" as an example of how the neocortex can have complex interaction with the outside world, while the hippocampus is being recruited in a daydream, recall or social rehearsal.

However, I know this issue is sticky for a lot of people (and was part one referees' objection), so I've been trying to find some literature, somewhere, in which some researcher said: "something's missing" (other than memory). Whether it's some spatial misunderstanding, or inability to distinguish temporality of sequence, some kind of fragmentation of the scene, inability to introspect, something. All I've really found is that these patients don't confabulate (which I think fits the theory) Damasio's statement that his patient didn't have "extended consciousness" and a couple of vague references to difficulty with spatial perception. Do you have any insight into this? Any good research you've found, which you think helps solidify the case?

Thanks and take care!

matt faw

p.s. I did find the autopsy report on patient D.B. who had lesions exclusively in field CA1, bilaterally. He suffered no retrograde amnesia at all, just complete anterograde.

http://www.jneurosci.org/content/6/10/2950.full.pdf

Hey Ralf,

You're right, I didn't mean "thinking style" per se, as much as distinct process.

For example, maybe task-positive is more about "what's now?", whereas task-negative is more about "what's next?" TP about immediate action, TN about planning the next move. Or TP about reacting to the world, whereas TN defines a social self, the organism representing itself, interacting with others. vmPFC about planning action, dlPFC about putting the plan into action (or inhibiting it).

I know these are overly broad generalizations, but it's helpful for me to get an overall grasp on how things fit together.

I hope you had a good time visiting Germany (assuming you're back in the UK now). I hope your mother is well.

best,

matt faw

Hi Ralf, I hope you and your family are well!

I've just finished up on an intense week in Tucson, interviewing Daniel Dennett and Bernard Baars, among others. Truthfully, the most interesting interviews of the week included Michael Graziano, who is preaching a radically parallel theory, except he's focusing on the TPJ, instead of the hippocampus. Also, through my interview with Alison Gopnik, and talking about the hippocampus, she recommended I talk to U of AZ professor Lynn Nadel, who of course has been publishing hippocampus research since the 70s. That interview was pure gold, as he confirmed many of the steps along the way to seeing the hippocampus as the center of subjective experience!

Oh yes, and I interviewed Thomas Metzinger as well, a week after Tucson, since he happened to be in California. As you saw, I referenced him in the paper, more than anyone else but you, because he was talking about experience and self as representation, rather than process.

And now, I'm back to the paper, and trying to resolve whatever was missing in the previous drafts, and seeking a great deal more research, and finally ending up with this surprising discovery today:

I did not realize that the task-positive network did not include the primary perceptual and motor cortices, so when I read a paper today (co-authored by Raichle and Fox among others) that said they were not part of the TP network, I was flummoxed.

But, on retrospect, it makes sense. "Driving mind" makes more sense if the perceptual and motor cortices keep on chugging along, even when a robust daydream is brewing in the default and simulation networks. It makes sense that these networks should be divorced from the cognitive ones (positive/negative). Pre-motor is considered task-positive, but motor is not, which divorces the decision from the doing.

This all suggests to me that the positive/negative anti-correlation is really about thinking styles. DLPFC is more about what's now? whereas VMPFC is more about what's next? DL is more about the immediate moment, responding to environment, whereas VM seems to be more about the future, using the Hippocampus to figure out a plan for the near or far future.

I'm curious to hear what you think about all of the above.

sincerely,

matt faw

Hey Ralf,

I hope you're well.

I just wanted to share an excellent article: "The Construction of Autobiographical Memories in the Self-Memory System" by Conway and Pleydell-Pearce (2000). I haven't found anything else quite like it. http://homepage.psy.utexas.edu/HomePage/Class/Psy394U/Bower/11%20Soc%20Cog%20Personality/XX%20Constr%20Life%20Stories/Autobio%20Mem-Conway.pdf It describes why we remember what we remember, why we forget, how the self fits into all of it, etc. I'm sure it would be very helpful for any work on the Self.

best,

matt faw

Thanks for the good words, Ralf!

I appreciate, too, your input to help make the paper better.

I assume you're going back to Potsdam to see your mother. I wish you well on that journey.

I'll let you know how the conference went.

best,

matt

Hi Ralf! Great to hear from you!

I'm doing well; focusing on prepping for the Tucson consciousness conference (which starts in 10 days). I have a nice collection of interviews lined up, including Dennett and Bernie Baars. I'm also prepping my first conference poster, to explain the hippocampal simulation theory.

Sad to say, my paper was rejected by the Journal of Consciousness Studies. Evidently, as a novice writer, I fell into pitfalls I didn't know existed. We got attacked on one side for our anatomical explanation not being comprehensive enough, and from the other side for our philosophical argument. Perhaps I should just focus on the philosophical part, and say: if you want an anatomical argument, go read Behrendt's paper! 🤣

I probably won't be able to work on the paper until after Tucson, but I'm committed to making it better. I'd rather be working on the movie, but I think the future of the doc is intimately connected to the paper, so I'm not giving up on it. Hopefully with the poster, I can engage in discussion with the critics, rather than just get their comments, and I can have a better idea of what needs fixing and addressing.

I hope you are doing well and enjoying spring!

best,

matt faw

Dear Ralf,

Thank you for for your swift response on repression. What you wrote about the superego makes me think about a phenomenon I've noticed in myself, and have heard described elsewhere. It's the sense of a watching presence, that I used to feel all the time, and brought shame to me, even when I was alone. I knew perfectly well that no one was there, and theorized that it was an internalization of my parents' presence, that they had trained it into me, so I would behave as if they were watching, even when they weren't.

In fact, I realized at some point, that when I was experiencing strangers as being judgmental towards me, that I was really projecting this watching presence out to them, making them into the cause of my shame, instead of realizing it was a self-reflective loop. I have since wondered if that loop function is part of the default-network-driven simulation, a simulation of other people that arose from fear of social approbation.

I can see why you say that the sense of self is intimately tied to this sense of being observed. I've often wondered if the reason why we spend so much time in default mode, is because we're so often projecting out to some 3rd party point of view, and trying to witness ourselves.

In ecstatic dance, this is the element I try to surrender: any attempt at viewing myself from outside. "Dance as if no one is watching". Including: as if I am not watching. Dance without a sense of story, right or wrong, shame or glory. It's an ongoing practice.

best,

matt faw

Thank you for your reply, Ralf. Interesting what you say about repression; I was just having a similar discussion with my dad. I had noticed that the right posterior ventrolateral PFC is implicated in inhibition of motor action, and was wondering whether that was connected with a case of hysterical paralysis that he had told me about. It also made me wonder about the repression of thoughts and memories (especially since the left vlPFC is implicated in the cognitive control of memories). Do you have any insight into the mechanism of repression? I'd love to hear it. It would go a long way toward understanding something like dissociative identity disorder. best, matt faw

Hi Ralf,

Great to hear from you. I completely understand that you have a very busy schedule, and I always appreciate whenever you have time to correspond.

Yes, I completely agree with you that the phenomenal self is just part of the simulation. In the first (rejected) version of the paper I wrote, I had a whole section about the difference between the small-s self (the apparent, illusory self) and the capital-S Self (the collective processes of the entire organism). But that paper was too sprawling for the reviewers, so the paper you read was simplified, and left out that distinction.

The distinction between the two is very important, I think, because the subjective self is conditioned to represent itself as a singular moral entity. It is taught that any brain and body functions that fall outside of its constructed self-image (like desire or anger) are wrong, dirty, sinful, or even separate from the self (e.g. an "inner demon"). I do not think the current dominant societal view of consciousness is the Cartesian theater, but is instead the Abrahamic religious view of a unitary moral self, trying to steer a base and immoral body through life.

One of the take-aways I want to present in the last 5 minutes of the documentary is the perspective that by believing in a unitary subjective self, we reject our own brain functioning, by trying to ignore, marginalize, or shout-down the processes in our heads that we do not recognize as ourselves. I think we'd be a lot less at war with ourselves, and consequently, with each other, if we learned to love all the parts of our heads and bodies, and let them make peace among themselves. How to do that would be a whole other documentary, but I want to at least point at what I think is a very important truth of neuroscience: the brain is all in there to help us, and we do ourselves a disservice, when we carve our own processes up into good and bad camps.

Another concept I wrote about in the earlier paper, but have since removed, is one that I've been wanting to bring up with you. It's the idea of seeing language as shaping our memories.

Specifically, I think that human brains have learned to simplify storage and recall of memories, by translating parts of the memory into language/narrative structure. Since the "multimodal movie" is data-rich (to represent the various stimuli of the original episode), it takes more effort to recall and compare those memories with previous ones. However, with a story structure, the movie can quickly be boiled down into something like a power-point presentation: mostly narrative, with hyperlinked still images, and snippets of audio and video for playback. Our recall ends up being an illustrated story, which we recount with great confidence, because of the presence of the most vivid multimodal clips.

I think this language function is likely added by Gazzaniga's interpreter module, which narrates what happened, in order to streamline the memory consolidation. Rumination while in bed, late at night, of the various events of the day, is likely one part of this process, as is the stream of thoughts that accompanies ongoing experience.

Of course, the story that is told, which is largely influenced by what we fear or wish to be true, is another easy source of error in the encoding and replaying of memories. Narration takes on habits, just like simulation does, including rationales and work-arounds. In fact, it is probably partly the way that we are taught about ourselves, through language, that helps build up our narrative habits, including our stories about self. This leads to the small-s self being what Dennett calls "the center of narrative gravity".

Thanks again for your reply, and I look forward to hearing from you next!

best,

matt faw

Dear Ralf,

Yes, I agree. The subjective self is an illusion, a construction for memory. But what about the capital-S Self: the body and its brain? Cannot the body+brain be fooled by the simulation? If a vision module receives hippocampal output, which helps it predict the next moment more efficiently, cannot we say that that module "believes" the report that it receives from the hippocampus, and can be fooled by it? If I meet someone new who reminds me of someone else I don't like, I end up simulating them with distaste built in. Am I not fooled into a false reality of that other person? If I have a strong self-image, then I craft my memories to cater to that image. Am I not fooling myself with my simulations? Isn't the sufferer of Capgras syndrome fooled by the fact that his mother is simulated without strong emotional content? Isn't the clueless drunk guy at the bar fooling himself when he believes that: "She was so into me!"? Aren't Fox News viewers fooling themselves when they intentionally avoid letting contradictory information into their memories? Aren't we all fooling ourselves when we treat pain as something to be ignored or tuned out from? I don't see any contradiction in saying that the subjective self, the self of belief and memory, is a fiction, but that the organism itself can be fooled by that fiction. That seems to describe the exact world we live in. It doesn't even make sense to me to say that the "self is just an illusion" without there being some entity that is fooled by that illusion. Still scratching my head, matt faw

Thanks Ralf, for your e-mail.

I wonder if you can help me bridge the gap in my understanding. I have told you many reasons why I think the simulation, by defining reality to the rest of the brain, has powerful causal effects on human behavior. Absolutely agreed that the simulation is ultimately nothing but patterns in the hippocampus. But the whole explanatory power in this model, as I wrote in the paper, and throughout this conversation, is that people are fooled into thinking that the simulation is reality.

I understand your assertion, but not your reasoning behind it. You've heard all my reasons; can you please explain yours, so I can understand better?

Thanks!

matt faw

Dear Ralf,

Thanks for your reply. I'm so glad to hear that this discussion is a positive experience for you; I very much value the conversation! I've been living with the (rough outlines of) a hippocampal simulation theory for over a year now, and never had anyone to talk to about the subtleties of it. Even after I convinced my dad that the theory was possible, he never embraced it enough to really consider the ramifications. So it's very refreshing to talk theory with you, gives me a much-needed avenue of contemplation.

You operate in a conceptual space that is beyond me, because I am so new to the neuroanatomy. All I have are 'philosophical' arguments, and the results of the research I did for the paper. But it's very important for me to ground this in science; thank you for offering that viewpoint to me.

As far as consciousness and behavior: I do not believe that there is such a thing as a "conscious process". I think that's an illusion, created by the viewpoint from within the simulation. All brain processes are equally "unconscious" (albeit intelligent), but only a tiny bit are represented, somewhat, in memory.

However, that isn't the same as saying that the simulation doesn't have causal effects. The very fact that there are illusions based upon the viewpoint from within the simulation is such an effect. In fact, I think all its effects are based on such illusions.

When preparing for the documentary, the first book I read after "Consciousness Explained" was Douglas Hofstadter's "I am a Strange Loop". He talks about the self as being an illusion, created by the continual looping of self-reference within experience. That is how I see the illusions of self and reality as being causal, by their looping, self-referential quality. By their consistency, they create a very compelling reality that is extremely difficult to see past.

Personally, I only became aware of reality-as-simulation through exploration on psychedelics (mostly pot and occasional mushrooms). I could see how my nervous system constructed the world differently, which revealed that it had all been a construct, all along. And then I began to see how I had been constructing myself in the simulation, and have been trying to weaken that self-story, ever since. This is what I previously referred to as my "spiritual" path, even though it follows no tradition: ongoing surrender of stories that arise about myself and the world, so that my organism can experience beyond my previous (simulated) boundaries.

It was through ecstatic dance that I realized that I was not the doer of the brain. By getting on the dance floor and "getting out of the way" of the dance, I could allow my body to take over and do what it wanted to do, a practice which has had truly dramatic positive effects on my health over the last thirteen years. By surrendering my attempts to imagine what I looked like from someone else's perspective, by surrendering self-judgment and even letting go of keeping track of what my body was doing, I could offer my body more and more freedom to move joyfully, which led me from being a sufferer of a chronic bad back in my 20s to being a break-dancer in my 40s. My body, I found out, felt totally neglected, and wanted to live large.

I also began to see that one of the important qualities of the simulation is what is left out, what is repressed. For example, I finally realized that I had been avoiding looking in mirrors, my whole life. And I began to allow pain and discomfort in, as necessary signals from my body.

So that's (one reason) why I feel like the simulation is causal, because my own life experience has changed so dramatically since I started to doubt the simulation. I am still living "in the simulation" of course, and still living "as a self", but in a different, less-concrete way. The illusions are much less compelling.

Thanks again for such an engaging, stimulating conversation!

best,

matt faw

Dear Ralf,

Thank you for your primer on attactor state dynamics. This helps me a great deal in understanding the auto-associative processes of CA3.

Forgive me, however, if I still have resistance to seeing the simulation as a "one-way process".

What I hear you describing is the function of the "hardware" of the system. What I propose is that the simulation itself can act as software, that it can be a causal agent, even though it is only a story, a series of patterns.

In society, we can witness many many stories that work as causal agents, even though they're essentially just ideas. Patriotism, marriage, religion, justice, art, values, money, rules, societal norms, etc., are all stories that have powerful causal effect on behavior.

Likewise, even though the self and experience are only stories, are only patterns of neural encoding, they still shape behavior by defining reality. The anosognosic hemiplegic behaves according to the simulated memory of self-movement. The dissociative identity patient behaves according to whatever memory stream is activated. The gay man who is closeted from himself behaves accordingly to his simulation of himself as straight, rather than how the rest of his brain would have him behave. I think that memory would serve no purpose, if it did not represent actual things in the world, including the self. Memory would only be like the accretion of rings in a tree, a record of what happened that is meaningless to the tree itself. The CA3 attractor patterns must encode within them facets of personal history, or we could not recall those facets, could not use them in our future decision-making. The brain is the hardware, and does all the work. But I think the simulation ends up becoming software, an ersatz reality, including a self. This, despite the fact that it is only a story, a chain of memories that is entirely dependent on the attractor dynamics that underlie the hardware process. It doesn't make decisions, but it continually informs them.

Thank you again for your continued conversation and for hearing my thoughts on the matter.

With much respect,

matt faw

Hi Ralf, thanks for your reply.

I think I'm finally understanding the difference in how we're talking about this.

When I wrote that the rest of the brain "believes the simulation", I did not mean this literally, like the way some people "believe Fox News". Because there are no homunculi in the brain, and there is no screen.

Because I am not a neuroscientist, and do not have access to the all the subtle mechanics of the brain, I use the colloquial language to describe the brain processes. But it is clearly the processes that run the ship.

On my computer, I interface with the GUI, which is the people-friendly appearance of the programming. That is the part that I can understand and manipulate, but clearly it is not what gets things done; it is the underlying code (and the physical processes of the computer) which make everything possible.

Likewise, "consciousness" is the memory-friendly appearance. Within it, memories interact with each other, in this experience-like way. However, every memory itself, and all of its interactions, are clearly just processes of the underlying structures.

But I'm also wary of portraying the hippocampus as a "holodeck", a place of entertainment that has no bearing on the actual function of the ship. Nor as the "captain's log", just a record of what happened. Instead, I see it as an associative engine, a predictive engine, a workspace in which other brain modules are able to create more complex views of reality, to behave beyond simple conditioned algorithms. To me, the hippocampus seems like it's intimately involved in ongoing processing of the now, and seems to powerfully inform behavior and beliefs. And that's what I hear from your "appetitive behaviors" paper, as well.

In order to influence the other modules of the brain, and to elicit data from them, it must share its output somehow ( albeit clearly not as a TV show for an audience). That mechanism is what I'm curious about.

best,

matt faw