Conversation with Ralf-Peter Behrendt

okokok

I met Ralf in person. I spent a lot of time with Ralf. We watched Mighty Bush. We would sit on his carpet and watch it. We loved it, we were like little kids.

A patient reminded me of Ralf today. Episodic memory. Ralf will always be the most intelligent person I have ever met. I heard him though. Loud and clear.

I miss him so much now.

Hey Ralf,

I've been working on my latest revisions of the paper, in many places

starting from scratch. Most of the arguments are the same, I'm just

unpacking them much more thoroughly.

There is one detail that I'm starting to see differently, and I'd love

to hear your take on it. It's about the same referee question I sent

you recently, about how does the hippocampal simulation become

experience? I have, heretofore, been agnostic about that question,

and just left the simulation within the hippocampus.

However, that answer seems theoretically troubling to me. I had built

the hippocampus argument on the model of a 'pyramid of reports', i.e.

each module sends its report to the next module, which does its

computation, and then sends an output report on to the module after

that. In other words, information can only be known in the brain, if

it is shared. In this model, there must be a transmission. The

encode of memory cannot just exist, 'unseen', or it could not be

subjective experience.

Of course subjective experience is not 'seen', but I do think that the

metaphor of the hippocampus as the TV news function of the brain is

useful anyway. My current explanation in the paper, the one that

makes the most sense to me at this point, is that subjective

experience is not the memory encode itself, rather that it is the

process of that encode being 'broadcast' back out to the rest of the

brain. Again, the rest of the brain is not 'watching' the news.

Rather, the brain is activated by the memory broadcast, in the same

way that the visual cortex doesn't watch the outside world; rather, it

is activated by it. Same thing here: as the broadcast signal

activates the neocortex (presumably at layer 6), as a top-down signal

to help bias the next moment's predictions, that activation creates

experience throughout the brain. That experience has already been

unified and simulated in the hippocampus, but it is now made 'real' by

being shared with the rest of the brain.

One piece of evidence that I find compelling in this direction: the

fact that patients with hemispatial neglect exhibit the same neglect

when remembering scenes that they had first committed to memory,

pre-morbidity. If they imagine themselves at one side of St. Mark's

square, they can only describe items on their right, but if they

imagine themselves at the other end, looking back, they describe the

opposite half. What this suggests to me is: the encode is intact, CA3

is intact, but the recalled memory is only 'known' to the rest of the

brain, if those parts of the brain can be re-activated by the

broadcast of the recall. Hence, subjective experience of the recall

fails, because of neocortical damage. Which suggests: subjective

experience 'happens' at the neocortex, even though it's formed in the

hippocampus.

Now, that's not the only possible answer, I'm sure, but right now,

it's the one that makes the most sense to me. As you know, I've been

bugged with some variant of this question for many months now, and

this is the first formulation that really feels elegant and

explanatory. This doesn't change the overall theory, as I see it, but

it does make specific predictions that I was not previously ready to

commit to.

I've often seen you state that consciousness is the patterns in CA3,

but I have a hard time with that theoretically (especially since the

downstream CA1 seems to be where events are placed into chronological

order, and embedded into theta cycles). But it's also hard to imagine

that the entire brain 'believes' that the new memory encode is

reality, unless the entire brain (or most of it) receives that

broadcast transmission first. And it's hard to explain a rich

experience that only exists in the hippocampus, as if the hippocampus

could somehow experience its own creation. I think of CA3 as the

auto-associator and decoder of previous memory encodes, and CA1 as the encoder of the new memory (as prepared for it by CA3, DG and the EC, and prior to them, PA and PHC, and prior to them, everywhere else in

the brain). The parts of the brain which are capable of experience

seem like they should be the very sense cortices that process initial

stimuli. This closes the loop, and it explains why subjective

experience is happening in the background, even during 'selfless' flow

(i.e. neocortical) states.

I'd love to hear your thoughts.

best,

matt faw

Hi Ralf,

I think I see what you mean. Even though 'subjective experience' is

important from a human-scale perspective, it is merely a epiphenomenon

of the patterns that underly it, because it is the patterns that

actually run the show. The patterns aren't asking themselves: "what

color was that?" The patterns don't ask anything, because they are

just patterns. They reflect natural processes, just at an extremely

high level of complexity. Those patterns happen to be members of the

'ecosystem' or 'weather' of the mind; they're part of a more complex

set of patterns, but there is no identity in there, no will, no actor,

no perceiver.

-----------

BTW, when I sent you the reviewer's criticism, I should have included

the following paragraph, because it's more to the point, as far as the

actual locus/mechanism of subjective experience (SE). It's a question

that I'm sure is answerable, but I have no way of being able to make

anything more than a wild guess. I'd love to hear your thoughts on

it.

"Why is the HF so important for SE?

The authors have not actually answered a key question - why does the

HF have this special property of SE, and not other regions? What they

have done is picked out some functional properties of the HF that map

quite well onto important features of SE, but this is merely

correlational. This does not, therefore, answer the fundamental

question of how the (according to the authors) unconscious processing

occurring across the rest of the brain is somehow transformed into SE

within the HF. I do not understand the rationale for this. One could

accept the authors arguments about the functional role of the HF in

binding together many different processing streams, and creating a

"movie" simulation etc., but could then generate many different

theories about the actual neural correlate of SE on the basis of this,

other than their own argument that it is in the HF. For instance, it

could be that the HF processing is also unconscious, and that the

"movie" created within the HF is then passed to other areas (such as

the posterior cingulate cortex or medial prefrontal cortex, or a

distributed network of high-level regions) which use this information

to generate SE. Or one could argue that SE is an emergent property of

a complex, dynamic distributed system, such that the HF contributes to

SE only to the extent that it contributes unique aspects of

processing. By this account (which I personally find the most

plausible), the HF is essential for certain kinds of very important

SE, such as SE of an extended, spatially coherent world as opposed to

disjointed visual objects, and such as episodic memory and

imagination, but it is not the route of all SE. Extensive

clarification is needed."

Well, I think we're in agreement, Ralf. I believe in zero magic.

Everything is organic. And even organic is not magic as 'life' but is

the consequence of self-organization.

My dream next documentary is precisely about this, using the snowflake

metaphor to show how organization from the big bang through stars and planets and organic molecules and life and us is inevitable; it's the

way that pattern formation and chaos harmonize with each other in the

particular key that makes up our universe.

I believe there are only a handful of fundamental geometric tendencies

(which probably fold up into even simpler symmetries) that initially

set in motion all the amazing organization that has arisen from them.

Everything, at every level, is self-organization. No magic.

best,

matt

I guess we do differ there, because I don't understand your reasoning.

As I understand it, your "appetitive behaviors" paper was all about

how the 'situation' is the complex stimulus for conditioning. Isn't

that 'situation' the same thing as consciousness? Doesn't the paper

say that the situation is a piece of information that represents the

outside world, and that it is precisely the pattern matching between

today and tomorrow's situation, that allows today's situation to have

an effect?

Would you say that episodic memory is epiphenomenal? Doesn't the

memory of yesterday effect the perception and behavior of today?

Isn't consciousness the same thing as episodic memory?

I would agree with this aspect: HM can be explained by saying that he

doesn't need the feedback of consciousness, in order to continue

interacting with the world. But that's not the same thing as saying

that it wouldn't have an effect, if he had it. A person with

blindsight may be able to 'see' somewhat via the dorsal visual stream,

but as long as they believe they are blind, they will act accordingly

(unless somebody throws something at their head).

Even dreams, which seem like the most epiphenomenal aspect of human

cognitive life, still potentially impact people's beliefs, and change

their behavior.

So yes, I agree that consciousness is information. And I agree that

it is information that is decodable by the rest of the brain, but not

(so far) by human society (it is not words or images, but patterns

that represent those things). But information is not epiphenomenal;

or it would not inform.

scratching my head,

matt

Hi Ralf,

I believe mathematics is the best possible way to describe the entire universe (particularly geometry). My metaphor for the universe is a snowflake, one symmetry built fractally upon another. The deepest symmetries all formed in the first second of the universe, and are so stable that they exist to today: the relationship between energy and space, between mass and energy. The hadron era of protons naturally gave way to the lepton era with electrons, symmetry upon symmetry. Every particle can be described geometrically; two protons repel because of like charge, but that could also be described as a mathematical relationship that has a very hard time bending past a certain curvature.

So yes, I'm sure that the brain is ultimately described by mathematics, and that on a pure enough level, it is foolish to talk about causative powers, because causation is just a dynamic geometrical unfolding of the original symmetries.

Although the phenomena near the center of the snowflake are extremely stable and predictable (an electron never fades), they give rise to fractal patterns at the edges that are increasingly noisy and chaotic. Geometrically, it was inevitable that protein chains would eventually form, that they would develop the ability to copy themselves, and then that they would eventually copy themselves into dominance over the world. But the tendency for things to fall naturally into patterns is balanced with the tendency for things to fall apart. Yang balanced by yin. Without yin, we would have no death or mutation, and hence, no evolution. In fact, without chaos, we would still have an entirely homogenous universe, full of increasingly rarified energy and no structure. The universe needed noise for energy to clump and accrete, just as an actual snowflake needs a dust grain, in order to form.

So yes, on a cosmological level (all the way to brain dynamics) it is all math. However, I'm not a mathematician. I'm a filmmaker, and I need to choose the language of dreams. I need to personify, to use metaphor, to build arguments in ways that can ease my audience past confusion into clarity.

On that level, I see no point in talking epiphenomena. Especially when I'm already trying to demystify consciousness to information, trying to pry people from their intuitions. Since we all believe that we are conscious, then we need to deal with that belief. Belief is a phenomena that consciousness explains, and it gives the theory all its explanatory power. Anosognosia, schizophrenia, etc. are mis-matches between belief and reality. The ability to recall a memory matters, even if there is no actual 'I' to do the remembering, even if it is all auto-associative.

So I have no problem with an argument that eventually ends at mathematics, because I imagine that they all do. Eventually. But I think my argument will just be confused by going there, especially as I have no way of describing the math. All I have are metaphors, and a skin-deep understanding of the anatomy. I think that my arguments have to arise from what's available to me. I think that if I focus on a human-scale argument, I will be much more likely to communicate this theory without confusion.

That's why I'm seeking a describable mechanism for memory to give rise to belief. Not because I think there is a self in there to remember or believe, but because I need to start from where my readers already think, and tease it apart gently, build a bridge of understanding for them, take them from there to here. It's true with the movie, and I'm increasingly convinced it's true with the paper, as well. I've had referees fall back on dualism, and contradict themselves without the least self-consciousness, because they're not the ones who have to build an argument; they can just fire missiles from within their own current understanding, and reject my argument without really thinking it through. Almost everything that the reviewers complained about were already in the paper, for each version. The answers were there in the various drafts, but I didn't drag the referees to the answers, force them to drink, and so they ignored them. Those are the people I have to reach, and I have to do it by making everything as plain and simple as possible.

best,

matt

Thanks, Ralf, for your quick reply. I am very grateful for your

ongoing friendship and conversation, so please, no need to apologize.

I completely agree with what you wrote. Clearly, there is no homunculus.

On this draft, I'm trying to simplify and spell out absolutely

everything, leave nothing unsaid. Most of the times our reviewers

said some answer was missing, I had actually covered it, but

apparently unsuccessfully. So I'm trying to break everything down

into small little steps, like a Joy of Cooking recipe, so there's no

gap through which understanding can fall.

On this particular issue, I can see a reasonable path of explanation,

but actually finding the right words to convey that explanation is a

whole other issue. It's like the HM thing; it doesn't seem that

difficult or controversial to me, but it's damnably hard to get across

(even to my own dad). Oh well, it's a good challenge.

best,

matt

Hi Ralf,

I hope you're well.

Sad to say, my paper just got rejected from a third journal.

Thankfully, the responses are much kinder this time than the first

two, due I think, to all the revisions I've made. I'm now trying to

figure out how best to revise and address these new criticisms.

Almost all the reviewers' comments have been addressed within the

paper already, so it's clear most of them are about me spelling things

out more clearly. However, there are a couple of questions they have

that are still stumpers.

In particular is an issue we've talked about before: yes, the

experience 'movie' is bound together in the hippocampus, but that

still leaves open the question of whether that movie is then

'experienced' by other brain parts. I've mentioned the metaphor of

subjective experience as the news of the brain, that each module sends

its report to the EC to be bound in the HF, and then broadcast back to

the rest of the brain as multimodal memory, a news report of the big

picture from a moment before. This is the solution that makes the

most sense to me, but it's out there on the periphery of the my

understanding, and I can't assert anything with confidence.

Following is the reviewer's comment. I'd love to hear what you think.

best,

matt faw

"Where is subjective experience actually located?

I am left feeling extremely unclear as to exactly what this theory is

suggesting with regard to the neural locus of SE. Most of the time the

authors appear to be suggesting that the HF is entirely responsible

for generating SE, but at other times they appear to flatly contradict

this view, and suggest that the HF actually contains indices which do

not themselves generate SE, but are used to activate the various

cortical representations from which SE arises e.g. p.9-10

"Technically, a memory's perceptual content is not embedded within the

encode, but is instead indexed, like hyperlinks, to sensory cortices

which were originally responsible for sending their report to the HF

for memory...the HF, which then follows the encode's indexing

instructions, to enlist the requisite sensory cortices in supplying

the data necessary to re-construct the episode. The more vivid the

memory, the more the sensory cortices are involved in its

reconstruction.". These are very different theories, which produce

very different predictions, and the current manuscript does not make

it clear as to which the authors are actually suggesting."

Hi Bob, yes I agree.

A word on "Pain doesn’t exist context free." I think there's

something to be learned by the recent experiments where the subjects

would rather shock themselves than be left alone with their thoughts.

I think that the body sends pain signals to the brain all the time, as

information as to where to stretch, where to be careful about, etc.

Pain demands to be listened to, but it is also part of the brain's

overall balancing scheme that there are inhibitors to block pain, for

a certain time period, so the organism can escape from harrowing

circumstances, and wait to feel pain when it is back in safe

territory. I'm speculating that most of the discomfort that was felt

by the subjects was the pain that they usually had distractions to

cover. No distractions means having to face what's already there,

which of course just makes the thoughts that must worse.

However, this ability to inhibit pain can be over-used, and I think

people, be default, do so, we tune out from bodily pain. I think this

is taught by parents to their children, so that kids learn fear of

injury beyond what is necessary, and they generalize it to pain. Pain

is bad. It is something to be avoided, to be "killed'.

But pain is just a sense, like any other. And in extreme cases,

people have inhibited other senses, including vision. So it is no

surprise that we do what we can to minimize the pain in our

simulations. Once the simulator habit includes "exclude pain" then it

creates a reality simulation that intentionally restricts a sizable

portion of its own input data. Only when the delta signal is over

some threshold do we even notice pain in the body. So old injuries

can hide for years, not because the signal is not being generated, but

because they are too familiar to consider. All of a sudden they

'flare up', but if we had been paying attention to pain all along,

surely they would have made themselves known. We have the same

attitude in geopolitics; the general public only responds when we're

freaked out by changes. Up until then, we're numb. But if we were

paying attention all along, we'd use an ounce of cure, and not have to

wait for the freak-out.

The parable of the frog in hot water is everywhere, but nowhere do I

see it more displayed, than in the way people retreat from pain, as

they age. People's bodies, as they get older, turn into caricatures

of their peculiarities when they were younger. If my back is a little

bit hunched in my 30s, you sure know I'm going to be defined by my

hunch in my 70s. But what is it that keeps people forced in these

postures? I propose that it is only pain that stops them, and if they

learned to listen, and to stretch, and to really take care of the

parts that hurt, then we'd all age a lot more gracefully. That's my

experience, anyway.

So in that sense, I think that pain is constant. The body is always

offering up pain as feedback, to help the brain make decisions. I

experience the immediate moment and respond to it with input from all

my stimulations, including pain. But since I (the planning self of

the default network) have excluded pain from my experience, I do not

believe it to be one of my motivators, and I create stories that make

other salient characters (like other people) to become the sources of

whatever pain and discomfort I feel. Pain is out-sourced, made alien,

and seen as an invader, whenever it flares up.

matt faw

Hey y'all,

Bob's new blog post also resonates with this distinction:

http://goo.gl/Tq7SNR

Bob may certainly have a different take, but part of what I get from this is:

As an organism, I have some control over what makes it into my

simulation, and I learn to block out some of the noxious stimuli, in

particular: pain. The 'observer' (as I hear it) is taking refuge

within the simulation, a relatively pain-free construction that he can

analyze and manipulate. From that safe distance, the observer can

work on his philosophy, and handle or accept pain better. But he

cannot heal, because healing involves paying attention to the source

of pain as raw stimulus, rather than as a concept (i.e. pain must be

felt, rather than just thought about). The 'participant', on the

other hand, seems caught up in all the drama, but is at least living

in the moment, including the discomfort of the moment. Pain is the

sense that demands: "pay attention here", but the simulation allows

the observers to do the opposite, and block it out.

matt

Hello gentlemen,

I don't know if you've yet seen this TED talk, but I highly recommend it. I think he is very much talking about the division between the neocortical expression of self vs. the hippocampal interpretation of that. This is related to the Martin Conway material I've been pushing on both of you, about how a self is created in memory, that is parallel but separate from the self which reflects neocortical activity. The first-person experience of self is always hippocampal, because that's where my experience becomes available to my default system. Whereas the 3rd person experience of me is essentially of my neocortical self, what I do in the moment (but of course they experience a neocortical self that is informed by the hippocampal story).

http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory?language=en

I hope you enjoy it!

matt faw

And here is a short article that I think is very

worth reading, on mise en place, or how our perception of our environment changes how we think and act. For example, a playful fantastical environment, in which some things seem to contradict our notions of what is possible, is actually the ideal learning environment, precisely because it liberates children from the tyranny of the known.

This may seem somewhat tangential to the question of consciousness, but I think place only happens at the consciousness level; this is where everything comes together on the "spatial scaffolding" to use Lynn Nadel's term. The gestalt is the subjective mise en place.

matt

Interesting Ralf.

And, I have to say, this is where I think story fits in again. The "way we ought to be" is exactly story, whereas the "way we are tempted to be" is really who we are, it's our biology.

As I see it, the dissonance is really between who I am, vs. the story I have of myself.

Following up on my post yesterday: just found this review article on motivated forgetting. The gist is that the right dlPFC and vlPFC both activate to inhibit the memory, the entorhinal cortex also activates, as if this is where the lateral PFC is acting upon, and the hippocampus de-activates. It's as if the lateral PFC is "choking off" the data stream at the EC, to reduce likelihood of hippocampal encoding.

matt faw

Yes, to feel proud about ourselves, but also I think to smooth other cognitive dissonances. Finding rationales for ourselves, but also for our loved ones (unless they are opposed to us), and for our beloved religious and political parties and figures.

And often, I think it's just to sort out complicated events and feelings. I don't have as much late-night rumination as I once did, but it used to be robust when I was a teenager-20s, and include extensive replay and rehash of the embarrassing moments, much more than the positive ones. It was salience that brought back the day's memories, much more than pride, and the process rarely made me felt better about myself.

I'd be interested in unpacking the term "superego" more, in conversation with Bob. Although I do not subscribe to the Freudian world-view, I think that there is potential illumination there, for other lines of inquiry.

matt

One thing this discussion raises for me is the difference between my childhood memories and those of my siblings. In particular, one brother tends to heighten the circumstances of the memory, if it makes a better story. So the event in which I remember having a run-in with 5 neighborhood thugs, he remembers there being 10. And in later tellings of the story, it's become 15, and then 20. (Of course, yes, it's possible that I'm the one whose memory has shifted). In fact, it was thinking about this brother that led me to the hippocampus. I called up my dad (who is my science advisor on the doc, and had been giving me my informal education in consciousness), and commented on how similar memory recall and imagination seemed to be, like they were related processes. And he told me what he knew about the whole new Hassabis / Maguire / Schacter line of research. I went to my computer and googled "hippocampus" and "simulation", and everything started to fall in place (unfortunately, this was February 2013, a couple months before Ralf's paper was published, so I didn't connect to him until later). This is part of why I see confabulation (i.e. the wider phenomenon, not the diagnosis) as being such an important part of experience, because the experience is always constructed with the biases built in. The decisions of what goes into the experience and what is left out (or is immediately forgotten thereafter) are made by algorithms / defense mechanisms that are entirely unrepresented in consciousness. Memory is constructed with purpose and meaningfulness to create continuity and confirmation of self- and world-story. And experience, being memory, is built the same way. If I let my desk get messy, then my brain will start canceling out the mess, so I'm no longer perceiving it. If my girlfriend is going to come over, suddenly I have a different set of motivators, and the mess on my desk becomes a lot more obvious to me. I find repression / motivated forgetting / self-blindness an equally interesting phenomenon to embellishment, because they seem like the two gate-keepers to experience, deleting, enhancing, or even completely reconstructing signals / content. And of course, one of the embellishments that I see as important is language thought. In particular, once the original salient memory is formed, it's the "mulling it over afterward" action which seems to make the memory labile, include the narration of hindsight, and reshape the episode for eventual consolidation. matt faw

Hi Bob,

Maybe it's a semantic issue, but it seems to me that confabulation as a phenomenon is much wider than confabulation as a diagnosis.

Yes, confabulation as I formally know it, is something to fill in the blanks for patients with frontal lobe amnesia.

But the very fact that these patients regularly do this behavior says a lot, I think, about the human need to make sense of things, put things in some kind of order (that reflects their beliefs).

Why is it that we know everything at age 18 (especially if we've just found religion or nationalism) but in our 40s, we know less and less? It is because the 18 year old has confabulated reality, has created a level of certainty that the more experienced version of him cannot rationally support. I see this with my (somewhat troubled) nephew, who is trying on different absolutist victim, martyr and hero stories, each which satisfies some longing in him, at various times.

Even you saying, that we "perceive what we believe" seems to be describing a form of confabulation, a constructed reality that specifically avoids causing us anxiety by not challenging our previous views.

matt faw

Hi Ralf,

I would agree that confabulation often is self-serving, but my speculation is that confabulation is equally distributed between what we wish and what we fear. If my girlfriend starts with "I have something serious to talk about", I can hear my little theory-generator sparking all kinds of worst-case-scenario stories to anticipate her subject.

Another example: there are some internet memes running around Facebook, etc. that show "baby yoga" and "Roma acrobats training babies" which show adults doing all kinds of absurd and dangerous-looking things with babies, in the name of making them limber. The videos themselves never show kids actually getting hurt, or even acting scared, but the comments (especially from parents) are like they are watching the work of the devil. Every parent comments with extreme revulsion, and utmost certainty that they have seen child abuse and damage on the video, even though it is not to be found. That is because they have confabulated that damage, in order to justify their own fears. They "know" that the yoga is bad, precisely because their inner voice yells out its freak-out so loudly.

I think that homophobia has been justified similarly; since there is some part of me which reacts with revulsion, therefore I confabulate a moral dimension to explain my disgust.

matt faw

Hi Ralf,

Your explanation of what happens when someone joins a new religion makes a lot of sense to me. There does seem to be what you call an "infantile relatedness", which is even reflected in the religious language of "God's children", the "flock", etc. And it is, indeed, a radical shift, to a more child-like worldview in which there is an absolute truth, a certainty, a way to avoid suffering and death, etc. It is very much like having the chance to cling, once again, to one's mother's leg, and to surrender complexity for reassuring simplicity.

matt faw