Conversation with Ralf-Peter Behrendt

Hi Ralf,

Thank you for your reply.

I agree that a good deal of life probably happens without the person ever responding to the new memory simulation being created in the hippocampus. Driving mind is the most obvious example, since it happens during a dangerous activity, but I'm sure that there many stretches of daily life where inner and outer life are similarly divorced, from seconds to tens of minutes, if not more. Probably a great deal of assembly line factory work is done 'unconsciously'.

However, that's a tricky assertion to make in a paper, because I don't know where I'd get a citation for it.

I agree; it seems necessary that someone with complete bilateral hippocampal damage MUST have a different subjective experience, whether or not we're right about consciousness being an output of the hippocampus, just because the hippocampus (and previous memories) bring so much to the visual scene. However, finding some such statement in the literature is proving very difficult. Mostly the statements are along the lines of "other than memory, all cognitive function seems intact" or "his personality didn't seem to change much". Our most skeptical referee claimed that he had had plenty of experience with such patients, and they clearly had "subjective experience". I don't know what that referee was referring to, and so I have a hard time imagining how to convince him otherwise.

My current argument in the paper is: when it comes to immediate interaction with the world, our neocortex is not dealing with the hippocampal representation; rather, it's working from it's own representations, and then sending evidence of 'what happened' to the MTL for binding into memory, which is of course, what we remember as the interaction, even though the real (neocortex) interaction happened prior to the memory. It's only when we introspect or recall that we are specifically engaging the neocortex with the memory encode. That's my argument, but it's filled with assertion, and no reference.

In social life, I think we move quickly back and forth between the two representations, engaging in the conversation with the neocortex and then using the hippocampal simulation to ask one's self: how am I seen? did I say the right thing? what was that look on her face? how's he going to respond? etc.

In the case of anosognosia for hemiplegia (I'm guessing), the patient's neocortex is "aware" that one half the body is non-responsive, but the pathways which should get that information to the MTL are off-line, and so it never makes it to memory. When the doctor asks the patient to move, and then asks him if he did move, he is asking the patient to introspect upon a brand new memory, and that memory was created predictively, with the movement built in. However, it is only while consulting the MTL representation of self-and-body that the patient would try to move, because the rest of the time, it's abundantly clear that they can't move. Behavior arises from neocortex alone, but belief is based upon memory (flavored, of course, with semantic assumptive bias). That's the most reasonable explanation I can think of, but I certainly can't test it without access to such patients.

Same of course with hippocampal-damaged patients. If every speculation in a paper needs to be sourced, but no one has yet asked the questions experimentally, it makes it hard to get references.

As for HM, I read just the other day that small bits of his hippocampus were left in place, but they were isolated from their connections to other brain regions, so they were functionally moot.

I'm curious how you deal with HM viz. your "appetitive behaviors" paper. If the vmPFC is conditioned to the situation, as presented by the MTL, then what happens when there is no MTL? It seems like something must change, because the information flow has been cut.

Maybe that answer is in the last couple sentences of your most recent e-mail, about "it is bound to reduce the adaptability of behaviors...", etc. At the least, that gives a fair list of testable hypotheses, of probable subtle changes to look for, when testing this patient population. I hope we can get to the point where someone with access to these patients takes this theory seriously enough to test for these changes, because I haven't found any evidence that these tests have yet taken place. (And as I am learning about academic papers, nothing is considered worth listening to, unless someone else in a lab has said it first).

best,

matt faw

Hi Ralf,

Thanks for the distinctions viz. task-positive and -negative modes.

I'm glad to hear your mom is well. Both my parents came to Tucson, so it was a good chance to catch up, and have them meet my girlfriend. My folks live on the east coast, 3000 miles from me, so I don't see them as often as I'd like.

When at Tucson, my dad and I presented the theory as a poster, and received some interest. I hope you don't mind I'm sending some people your paper, as well as ours, to give them a more robust treatment.

While presenting the poster, the most common first response was about H.M. and other patients like him. My argument is that it seems implausible that H.M. not have 'subjective experience' only given the old definition of the term, but if we just see subjective experience as the memory that follows the neocortex's interaction with the world, and forget the notion that it is the "inner light" or intelligence of the brain, then it no longer need seem impossible. I use "driving mind" as an example of how the neocortex can have complex interaction with the outside world, while the hippocampus is being recruited in a daydream, recall or social rehearsal.

However, I know this issue is sticky for a lot of people (and was part one referees' objection), so I've been trying to find some literature, somewhere, in which some researcher said: "something's missing" (other than memory). Whether it's some spatial misunderstanding, or inability to distinguish temporality of sequence, some kind of fragmentation of the scene, inability to introspect, something. All I've really found is that these patients don't confabulate (which I think fits the theory) Damasio's statement that his patient didn't have "extended consciousness" and a couple of vague references to difficulty with spatial perception. Do you have any insight into this? Any good research you've found, which you think helps solidify the case?

Thanks and take care!

matt faw

p.s. I did find the autopsy report on patient D.B. who had lesions exclusively in field CA1, bilaterally. He suffered no retrograde amnesia at all, just complete anterograde.

http://www.jneurosci.org/content/6/10/2950.full.pdf

Hey Ralf,

You're right, I didn't mean "thinking style" per se, as much as distinct process.

For example, maybe task-positive is more about "what's now?", whereas task-negative is more about "what's next?" TP about immediate action, TN about planning the next move. Or TP about reacting to the world, whereas TN defines a social self, the organism representing itself, interacting with others. vmPFC about planning action, dlPFC about putting the plan into action (or inhibiting it).

I know these are overly broad generalizations, but it's helpful for me to get an overall grasp on how things fit together.

I hope you had a good time visiting Germany (assuming you're back in the UK now). I hope your mother is well.

best,

matt faw

Hi Ralf, I hope you and your family are well!

I've just finished up on an intense week in Tucson, interviewing Daniel Dennett and Bernard Baars, among others. Truthfully, the most interesting interviews of the week included Michael Graziano, who is preaching a radically parallel theory, except he's focusing on the TPJ, instead of the hippocampus. Also, through my interview with Alison Gopnik, and talking about the hippocampus, she recommended I talk to U of AZ professor Lynn Nadel, who of course has been publishing hippocampus research since the 70s. That interview was pure gold, as he confirmed many of the steps along the way to seeing the hippocampus as the center of subjective experience!

Oh yes, and I interviewed Thomas Metzinger as well, a week after Tucson, since he happened to be in California. As you saw, I referenced him in the paper, more than anyone else but you, because he was talking about experience and self as representation, rather than process.

And now, I'm back to the paper, and trying to resolve whatever was missing in the previous drafts, and seeking a great deal more research, and finally ending up with this surprising discovery today:

I did not realize that the task-positive network did not include the primary perceptual and motor cortices, so when I read a paper today (co-authored by Raichle and Fox among others) that said they were not part of the TP network, I was flummoxed.

But, on retrospect, it makes sense. "Driving mind" makes more sense if the perceptual and motor cortices keep on chugging along, even when a robust daydream is brewing in the default and simulation networks. It makes sense that these networks should be divorced from the cognitive ones (positive/negative). Pre-motor is considered task-positive, but motor is not, which divorces the decision from the doing.

This all suggests to me that the positive/negative anti-correlation is really about thinking styles. DLPFC is more about what's now? whereas VMPFC is more about what's next? DL is more about the immediate moment, responding to environment, whereas VM seems to be more about the future, using the Hippocampus to figure out a plan for the near or far future.

I'm curious to hear what you think about all of the above.

sincerely,

matt faw

Hey Ralf,

I hope you're well.

I just wanted to share an excellent article: "The Construction of Autobiographical Memories in the Self-Memory System" by Conway and Pleydell-Pearce (2000). I haven't found anything else quite like it. http://homepage.psy.utexas.edu/HomePage/Class/Psy394U/Bower/11%20Soc%20Cog%20Personality/XX%20Constr%20Life%20Stories/Autobio%20Mem-Conway.pdf It describes why we remember what we remember, why we forget, how the self fits into all of it, etc. I'm sure it would be very helpful for any work on the Self.

best,

matt faw

Thanks for the good words, Ralf!

I appreciate, too, your input to help make the paper better.

I assume you're going back to Potsdam to see your mother. I wish you well on that journey.

I'll let you know how the conference went.

best,

matt

Hi Ralf! Great to hear from you!

I'm doing well; focusing on prepping for the Tucson consciousness conference (which starts in 10 days). I have a nice collection of interviews lined up, including Dennett and Bernie Baars. I'm also prepping my first conference poster, to explain the hippocampal simulation theory.

Sad to say, my paper was rejected by the Journal of Consciousness Studies. Evidently, as a novice writer, I fell into pitfalls I didn't know existed. We got attacked on one side for our anatomical explanation not being comprehensive enough, and from the other side for our philosophical argument. Perhaps I should just focus on the philosophical part, and say: if you want an anatomical argument, go read Behrendt's paper! 🤣

I probably won't be able to work on the paper until after Tucson, but I'm committed to making it better. I'd rather be working on the movie, but I think the future of the doc is intimately connected to the paper, so I'm not giving up on it. Hopefully with the poster, I can engage in discussion with the critics, rather than just get their comments, and I can have a better idea of what needs fixing and addressing.

I hope you are doing well and enjoying spring!

best,

matt faw

Dear Ralf,

Thank you for for your swift response on repression. What you wrote about the superego makes me think about a phenomenon I've noticed in myself, and have heard described elsewhere. It's the sense of a watching presence, that I used to feel all the time, and brought shame to me, even when I was alone. I knew perfectly well that no one was there, and theorized that it was an internalization of my parents' presence, that they had trained it into me, so I would behave as if they were watching, even when they weren't.

In fact, I realized at some point, that when I was experiencing strangers as being judgmental towards me, that I was really projecting this watching presence out to them, making them into the cause of my shame, instead of realizing it was a self-reflective loop. I have since wondered if that loop function is part of the default-network-driven simulation, a simulation of other people that arose from fear of social approbation.

I can see why you say that the sense of self is intimately tied to this sense of being observed. I've often wondered if the reason why we spend so much time in default mode, is because we're so often projecting out to some 3rd party point of view, and trying to witness ourselves.

In ecstatic dance, this is the element I try to surrender: any attempt at viewing myself from outside. "Dance as if no one is watching". Including: as if I am not watching. Dance without a sense of story, right or wrong, shame or glory. It's an ongoing practice.

best,

matt faw

Thank you for your reply, Ralf. Interesting what you say about repression; I was just having a similar discussion with my dad. I had noticed that the right posterior ventrolateral PFC is implicated in inhibition of motor action, and was wondering whether that was connected with a case of hysterical paralysis that he had told me about. It also made me wonder about the repression of thoughts and memories (especially since the left vlPFC is implicated in the cognitive control of memories). Do you have any insight into the mechanism of repression? I'd love to hear it. It would go a long way toward understanding something like dissociative identity disorder. best, matt faw

Hi Ralf,

Great to hear from you. I completely understand that you have a very busy schedule, and I always appreciate whenever you have time to correspond.

Yes, I completely agree with you that the phenomenal self is just part of the simulation. In the first (rejected) version of the paper I wrote, I had a whole section about the difference between the small-s self (the apparent, illusory self) and the capital-S Self (the collective processes of the entire organism). But that paper was too sprawling for the reviewers, so the paper you read was simplified, and left out that distinction.

The distinction between the two is very important, I think, because the subjective self is conditioned to represent itself as a singular moral entity. It is taught that any brain and body functions that fall outside of its constructed self-image (like desire or anger) are wrong, dirty, sinful, or even separate from the self (e.g. an "inner demon"). I do not think the current dominant societal view of consciousness is the Cartesian theater, but is instead the Abrahamic religious view of a unitary moral self, trying to steer a base and immoral body through life.

One of the take-aways I want to present in the last 5 minutes of the documentary is the perspective that by believing in a unitary subjective self, we reject our own brain functioning, by trying to ignore, marginalize, or shout-down the processes in our heads that we do not recognize as ourselves. I think we'd be a lot less at war with ourselves, and consequently, with each other, if we learned to love all the parts of our heads and bodies, and let them make peace among themselves. How to do that would be a whole other documentary, but I want to at least point at what I think is a very important truth of neuroscience: the brain is all in there to help us, and we do ourselves a disservice, when we carve our own processes up into good and bad camps.

Another concept I wrote about in the earlier paper, but have since removed, is one that I've been wanting to bring up with you. It's the idea of seeing language as shaping our memories.

Specifically, I think that human brains have learned to simplify storage and recall of memories, by translating parts of the memory into language/narrative structure. Since the "multimodal movie" is data-rich (to represent the various stimuli of the original episode), it takes more effort to recall and compare those memories with previous ones. However, with a story structure, the movie can quickly be boiled down into something like a power-point presentation: mostly narrative, with hyperlinked still images, and snippets of audio and video for playback. Our recall ends up being an illustrated story, which we recount with great confidence, because of the presence of the most vivid multimodal clips.

I think this language function is likely added by Gazzaniga's interpreter module, which narrates what happened, in order to streamline the memory consolidation. Rumination while in bed, late at night, of the various events of the day, is likely one part of this process, as is the stream of thoughts that accompanies ongoing experience.

Of course, the story that is told, which is largely influenced by what we fear or wish to be true, is another easy source of error in the encoding and replaying of memories. Narration takes on habits, just like simulation does, including rationales and work-arounds. In fact, it is probably partly the way that we are taught about ourselves, through language, that helps build up our narrative habits, including our stories about self. This leads to the small-s self being what Dennett calls "the center of narrative gravity".

Thanks again for your reply, and I look forward to hearing from you next!

best,

matt faw

Dear Ralf,

Yes, I agree. The subjective self is an illusion, a construction for memory. But what about the capital-S Self: the body and its brain? Cannot the body+brain be fooled by the simulation? If a vision module receives hippocampal output, which helps it predict the next moment more efficiently, cannot we say that that module "believes" the report that it receives from the hippocampus, and can be fooled by it? If I meet someone new who reminds me of someone else I don't like, I end up simulating them with distaste built in. Am I not fooled into a false reality of that other person? If I have a strong self-image, then I craft my memories to cater to that image. Am I not fooling myself with my simulations? Isn't the sufferer of Capgras syndrome fooled by the fact that his mother is simulated without strong emotional content? Isn't the clueless drunk guy at the bar fooling himself when he believes that: "She was so into me!"? Aren't Fox News viewers fooling themselves when they intentionally avoid letting contradictory information into their memories? Aren't we all fooling ourselves when we treat pain as something to be ignored or tuned out from? I don't see any contradiction in saying that the subjective self, the self of belief and memory, is a fiction, but that the organism itself can be fooled by that fiction. That seems to describe the exact world we live in. It doesn't even make sense to me to say that the "self is just an illusion" without there being some entity that is fooled by that illusion. Still scratching my head, matt faw

Thanks Ralf, for your e-mail.

I wonder if you can help me bridge the gap in my understanding. I have told you many reasons why I think the simulation, by defining reality to the rest of the brain, has powerful causal effects on human behavior. Absolutely agreed that the simulation is ultimately nothing but patterns in the hippocampus. But the whole explanatory power in this model, as I wrote in the paper, and throughout this conversation, is that people are fooled into thinking that the simulation is reality.

I understand your assertion, but not your reasoning behind it. You've heard all my reasons; can you please explain yours, so I can understand better?

Thanks!

matt faw

Dear Ralf,

Thanks for your reply. I'm so glad to hear that this discussion is a positive experience for you; I very much value the conversation! I've been living with the (rough outlines of) a hippocampal simulation theory for over a year now, and never had anyone to talk to about the subtleties of it. Even after I convinced my dad that the theory was possible, he never embraced it enough to really consider the ramifications. So it's very refreshing to talk theory with you, gives me a much-needed avenue of contemplation.

You operate in a conceptual space that is beyond me, because I am so new to the neuroanatomy. All I have are 'philosophical' arguments, and the results of the research I did for the paper. But it's very important for me to ground this in science; thank you for offering that viewpoint to me.

As far as consciousness and behavior: I do not believe that there is such a thing as a "conscious process". I think that's an illusion, created by the viewpoint from within the simulation. All brain processes are equally "unconscious" (albeit intelligent), but only a tiny bit are represented, somewhat, in memory.

However, that isn't the same as saying that the simulation doesn't have causal effects. The very fact that there are illusions based upon the viewpoint from within the simulation is such an effect. In fact, I think all its effects are based on such illusions.

When preparing for the documentary, the first book I read after "Consciousness Explained" was Douglas Hofstadter's "I am a Strange Loop". He talks about the self as being an illusion, created by the continual looping of self-reference within experience. That is how I see the illusions of self and reality as being causal, by their looping, self-referential quality. By their consistency, they create a very compelling reality that is extremely difficult to see past.

Personally, I only became aware of reality-as-simulation through exploration on psychedelics (mostly pot and occasional mushrooms). I could see how my nervous system constructed the world differently, which revealed that it had all been a construct, all along. And then I began to see how I had been constructing myself in the simulation, and have been trying to weaken that self-story, ever since. This is what I previously referred to as my "spiritual" path, even though it follows no tradition: ongoing surrender of stories that arise about myself and the world, so that my organism can experience beyond my previous (simulated) boundaries.

It was through ecstatic dance that I realized that I was not the doer of the brain. By getting on the dance floor and "getting out of the way" of the dance, I could allow my body to take over and do what it wanted to do, a practice which has had truly dramatic positive effects on my health over the last thirteen years. By surrendering my attempts to imagine what I looked like from someone else's perspective, by surrendering self-judgment and even letting go of keeping track of what my body was doing, I could offer my body more and more freedom to move joyfully, which led me from being a sufferer of a chronic bad back in my 20s to being a break-dancer in my 40s. My body, I found out, felt totally neglected, and wanted to live large.

I also began to see that one of the important qualities of the simulation is what is left out, what is repressed. For example, I finally realized that I had been avoiding looking in mirrors, my whole life. And I began to allow pain and discomfort in, as necessary signals from my body.

So that's (one reason) why I feel like the simulation is causal, because my own life experience has changed so dramatically since I started to doubt the simulation. I am still living "in the simulation" of course, and still living "as a self", but in a different, less-concrete way. The illusions are much less compelling.

Thanks again for such an engaging, stimulating conversation!

best,

matt faw

Dear Ralf,

Thank you for your primer on attactor state dynamics. This helps me a great deal in understanding the auto-associative processes of CA3.

Forgive me, however, if I still have resistance to seeing the simulation as a "one-way process".

What I hear you describing is the function of the "hardware" of the system. What I propose is that the simulation itself can act as software, that it can be a causal agent, even though it is only a story, a series of patterns.

In society, we can witness many many stories that work as causal agents, even though they're essentially just ideas. Patriotism, marriage, religion, justice, art, values, money, rules, societal norms, etc., are all stories that have powerful causal effect on behavior.

Likewise, even though the self and experience are only stories, are only patterns of neural encoding, they still shape behavior by defining reality. The anosognosic hemiplegic behaves according to the simulated memory of self-movement. The dissociative identity patient behaves according to whatever memory stream is activated. The gay man who is closeted from himself behaves accordingly to his simulation of himself as straight, rather than how the rest of his brain would have him behave. I think that memory would serve no purpose, if it did not represent actual things in the world, including the self. Memory would only be like the accretion of rings in a tree, a record of what happened that is meaningless to the tree itself. The CA3 attractor patterns must encode within them facets of personal history, or we could not recall those facets, could not use them in our future decision-making. The brain is the hardware, and does all the work. But I think the simulation ends up becoming software, an ersatz reality, including a self. This, despite the fact that it is only a story, a chain of memories that is entirely dependent on the attractor dynamics that underlie the hardware process. It doesn't make decisions, but it continually informs them.

Thank you again for your continued conversation and for hearing my thoughts on the matter.

With much respect,

matt faw

Hi Ralf, thanks for your reply.

I think I'm finally understanding the difference in how we're talking about this.

When I wrote that the rest of the brain "believes the simulation", I did not mean this literally, like the way some people "believe Fox News". Because there are no homunculi in the brain, and there is no screen.

Because I am not a neuroscientist, and do not have access to the all the subtle mechanics of the brain, I use the colloquial language to describe the brain processes. But it is clearly the processes that run the ship.

On my computer, I interface with the GUI, which is the people-friendly appearance of the programming. That is the part that I can understand and manipulate, but clearly it is not what gets things done; it is the underlying code (and the physical processes of the computer) which make everything possible.

Likewise, "consciousness" is the memory-friendly appearance. Within it, memories interact with each other, in this experience-like way. However, every memory itself, and all of its interactions, are clearly just processes of the underlying structures.

But I'm also wary of portraying the hippocampus as a "holodeck", a place of entertainment that has no bearing on the actual function of the ship. Nor as the "captain's log", just a record of what happened. Instead, I see it as an associative engine, a predictive engine, a workspace in which other brain modules are able to create more complex views of reality, to behave beyond simple conditioned algorithms. To me, the hippocampus seems like it's intimately involved in ongoing processing of the now, and seems to powerfully inform behavior and beliefs. And that's what I hear from your "appetitive behaviors" paper, as well.

In order to influence the other modules of the brain, and to elicit data from them, it must share its output somehow ( albeit clearly not as a TV show for an audience). That mechanism is what I'm curious about.

best,

matt faw

Dear Ralf,

Good to hear from you. Thank you for your reply.

Forgive me if I'm still not getting it. Let me at least say how things appear from my (admittedly naive) position.

I think of belief as straddling the line between memory and habit. Every episodic memory encode can be thought of as a belief, that "this is what happened". And people will react as if you contradicted their belief, when you contradict their memory (at least the emotional ones).

The episodic memory informs the semantic memory, and then the belief becomes "unconscious" because it no longer needs the hippocampus. It could also be said to start being habit at this point, because the association is "automatic".

Habit I think of as an algorithm: if A + B + C stimuli cross some threshold activation, then X outcome, whether X is a thought, an emotional activation and/or a behavior. If I understand your "appetitive behaviors" paper, the vmPFC "behavioral map" works as such an algorithm. Of course, there are competing algorithms with mutually inhibitive outputs, so the formulas get complicated.

In the case of semantic ('unconscious') belief, the mechanisms may be largely concept cells, mapping associations. So that whenever I see a picture of "Halle Berry" or "Bill Clinton", the appropriate concept cells are activated, which co-activate semantic, emotional, and perhaps episodic associations.

Now story-based beliefs are another thing, and I think they tend to fall more in the territory of 'conscious' associations. For example, ask a religious bigot their feelings on homosexuality, and they will bring up a story-based answer about some verse from the scriptures. However, their physiological response to the idea of homosexuality has almost nothing to do with the story of scripture; scripture is a self-story told to explain to themselves why they have such intense feelings against the idea. The actual feelings probably are conditioned by years of being raised in a society in which homosexuality is treated as a perversion, in which the person has witnessed or experienced horrific teasing and bullying. There may even be some Freudian self-loathing going on, fear that some part of the unconscious self is gay, which makes the response even more violent. But none of these really have anything to do with the stated belief, the quoted scripture.

That said, these story-based beliefs, even if they are postdictive rationales given to explain self-phenomena, are still forms of habit. The algorithm is: if X (someone questions my feelings about homosexuality), then Y (the scriptural story comes up to explain it).

The most deeply held (and least conscious) belief, I think, is that the simulation is real. This is the real me, this is my body, these events are what happened, this construction is the real reality. This belief is at the heart of the pathologies of subjectivity. Weaken this belief, as Buddhist meditators supposedly do, and the simulation becomes less dogmatic, less concrete. Reality is understood as a construct, and beliefs/habits are revealed as conditioning. This allows for slower, deeper responses, in which more task-positive stimuli is allowed in, to inform the simulation, and a wider range of algorithms are allowed to participate in the decision-making. This is the opposite of "having one's buttons pushed".

As far as the hippocampus is concerned, I understand that the CA3 field is auto-associative, which I understand to mean that no other brain module is needed, to elicit related memories.

However, CA3, as I understand it, then feeds its associative data to CA1, which has already received the input from the EC on a faster route. CA1 then generates a simulation that is informed by 1. the task-positive network's flow from the EC, 2. its own predictive functions, and 3. the associations from the CA3 field.

Then the feed from the CA1 simulation informs not only the vmPFC and multimodal areas like the IPL, but also returns re-entrantly to all of the major neocortical modules, as "top-down" information, to help speed up predictive processing. For example, "object identification" gives way to the less-processing-heavy "change detection"

In your "appetitive behaviors" paper you talk about the output from the hippocampus then being fed to the vmPFC, where the "situation" is the conditioning stimulus. It seems to me that the "situation" and the "simulation" are the same thing, so in this way, the simulation directly informs behavior and emotion.

Plus, all of the intrinsic simulations (prospection, mental rehearsal, theory of mind, etc.) are constructed in the hippocampus, and help inform possible future actions.

It is in all these loops in the brain in which it seems as if the simulation becomes known elsewhere. The hippocampus is informed, and then it informs. No neuron is sitting in a theater, watching the show, but many different modules receive at least part of the report from the hippocampus, and react accordingly.

This is my understanding, and it says very clearly to me that the simulation is not epiphenomenal. It is not itself a cognitive structure, but it IS a packet of cognitive information. The hippocampus, as I understand it, is an associative engine that connects many different modules, helps line them all up temporally, brings their content into alignment, and then prepares for the possible long-term storage of the associations, primarily during sleep.

The very fact that our bodies have to reduce muscle tone during REM sleep, so we don't act out our dreams, says to me that the hippocampal simulation must have direct influence on our behavior.

So that's my understanding of the relationship between association, simulation, memory, habit, belief and behavior. I see a ton of evidence that the simulation influences other parts of our brain, including behavior. Again, maybe I'm missing something, but I don't know what that something is.

Thanks again for your continued conversation!

best,

matt faw

Wow, Ralf, that's an amazing statement. I'm still trying to wrap my brain around it.

Certainly pathologies like schizophrenia or anosognosia show that the simulation can become severely detached from consensus reality, can be like a dream.

The one thing that I can't get past, though, is belief. The simulation shapes belief so powerfully, and subjective reality is so compelling, that I have a hard time thinking about that as epiphenomenal.

Maybe some of that subjectivity is in the pre-memory processes, but much of it is not.

For example, people neglect their bodies, because they are not well-represented in the simulation. But I'm confident that the body is very well represented in the brain. We have a sitting society, precisely because we tune out of our bodies, and we imagine ourselves as being something other than the body. We tune out from pain and discomfort, even though they are the body's own signals, designed precisely to get us to pay attention. These are serious consequences of the simulation.

Plus, we know that the brain can manage multiple interpretations of stimuli at once, can handle extremely fuzzy processing and multiple perspectives. But most of that is gone by the time we get to the hippocampus, which basically represents one perspective for memory. And that is what is believed.

All of the pathologies of subjectivity that I wrote about in our paper are consequences of believing the simulation. Racism, homophobia, sexism, etc. are all consequences of believing the simulation. People often live scripted lives, lives that are more informed by story, than by experience, because the simulation is so well trained in childhood.

Belief in the self has powerful ramifications. Almost everyone believes that the aesthetic qualities of the world are actually out in the world, rather than in the simulation. I'd venture to say that most people think their opinions on the world are basically the "right" ones, despite the sheer statistical unlikelihood of that. And people get very angry and defensive about their ideas, tastes, and values, even though these are non-corporeal things, that cannot be threatened. We believe in entities like "reputation" or "relationship" as if they were concrete, singular things, as opposed to just descriptors for our projections and feelings. We even believe that our simulations of other people are real, despite how suspect that is.

Plus, we can look at H.M. and other complete bilateral hippocampal patients, and see that although they can still function, they have stunted lives, lives with no imagination or possibility. They are no longer doers, and can only be patients, recipients of others' welfare.

It's very hard to see all that as epiphenomenal. To me, this says that our brains are fooled, whenever we are awake, by the simulation, and by the long chain of memories.

Maybe I'm misunderstanding you, but I have a hard time seeing the simulation as being completely "orthogonal" to brain function, since it shapes our behavior so powerfully.

respectfully,

matt faw

Thanks, Ralf!

I agree with everything you wrote: clearly, the sense of self is part of the simulation. And personal history is a chain (or collection of chains, given their state-dependent nature) of memories.

But the question I have is not literally "who" is watching, but how does the simulation become known to the rest of the brain? And I am agnostic, thus far, on this point. I'm just very curious about it. Clearly, memory influences behavior, so somehow, the rest of the brain must get wind of the simulation. The following is a collection of musings, not an argument one way or the other.

In the following article, Armin Schnider asserts that the posterior medial OFC serves to reality-check a memory, to see if it is part of the "now" vs. previous history. He shows that at 200-300ms after the onset of a stimulus, the pmOFC enters an era of activity, which tags the current experience as "now".

In patients with lesions to the pmOFC, there is confabulation and reality confusion, i.e. the inability to tell the age and relevancy of memories viz. current experience. They may be living a previous reality in their lives.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3677127/

I can't help but wonder if this pmOFC activation follows from the 'novelty detection' activation in the hippocampus at ~100ms. If so, then the pmOFC does not necessarily need to evaluate the scene, just the novelty signal.

Your paper on the vmPFC and "situationally appropriate behavior" also suggests that the vmPFC is somehow "witness" to the simulation, in order to associate the situation with the behavior maps. Of course, this is not the same as a homunculus, watching a screen. It may be closer to a sophisticated calculator, in which qualia and related memories serve as the variables, and the different behavioral maps may serve as if/then algorithms.

Priming reveals that the neocortex holds on to alternate or vague interpretations of stimuli, at least for awhile, in echoic or other local short term memories. This means that the simulation does not immediately replace alternate interpretations in the neocortex, even though Edelman and others say that the highly reciprocal pathways between the EC and the neocortex show that some quality of the memory is being fed back to various levels of the neocortex, to aid in predictive functions.

Multiple draft phenomena suggests that there is a comparative function between the HF's prediction, and the neocortex conclusions. Now, that might just be a function of the MTL simulator network being fed information from the neocortex which confirms or disconfirms its predictions, and adjusting the simulation, without a manager. Or, there might be a separate "manager" which compares the two.

The fact that we can easily inhibit memories, particularly the ones which reflect ourselves in a light that we don't want to believe, suggests a manager. However, maybe that's just one feature of an auto-associative CA3 field, that it enhances the memorability of events that reinforce the self-model, and inhibits the memorability of events that contradict it.

There is something specifically about belief that feels much more like an interplay between memories and cognitive processes, than just an auto-association between memories. Semantic memory is essentially belief (sometimes dressed up as "knowledge"), and seems to reflect a conversation between the hippocampus and the various structures which hold semantic indexes and concept cells.

Anosognosia seems to suggest that the simulation is dream-like, and when it is missing input, just continues on its dream-like way, predicting a future that never comes true, but which never gets disproven, either.

Hemispatial neglect seems to suggest two things, however. The fact that the current situation only represents one half of forward space supports the dream-like aspect of the simulation, that pathological input into the hippocampus just creates a subjectively normal-seeming world, despite all the weirdness that accompanies it. However, the fact that recall of pre-morbid memories follows the same pattern of neglect suggests something different. Since the damage is not to the hippocampus, it seems that pre-morbid memories should come back as full simulations, with no neglect. But they do not; they reflect the damage to the parietal lobe. This suggests that the recall of memories needs to be played out upon the parietal lobe, in order to send to the hippocampus enough information for a full simulation. When the damaged side does not respond to the memory encode, the recall simulation is built without it.

Likewise the fact that visualization, active memory recall, rehearsal and imagining all activate the neocortex, and may even play out at the retina and muscular level. These intrinsic simulations absolutely seem to begin at the hippocampus, but then draw in information from the neocortex (and beyond), as needed. This seems very much like the "player piano" model.

I wonder if there is any difference (in terms of involvement of the neocortex) between replaying the moment that literally just passed, versus replaying a memory from a few minutes or days ago.

So I don't know where that leaves us. I see the potential that all memory is just auto-associative, and that's who we are: a string of memories. But I also seem some plausible suggestion for a memory/simulation manager, and even more evidence that (at least the intrinsic) simulations do "play upon" the neocortex, to elicit appropriate qualia feedback. And even the fact that our simulations so rarely drift too far away from the reality presented by the neocortex suggests that healthy memory is the product of an ongoing conversation between the MTL simulator network and the neocortex. How does that conversation take place; I don't know.

best,

matt faw

Thank you, Ralf.

There is an interesting philosophical problem in that: almost Cartesian Theater-like.

If the hippocampus played its simulation out on the neocortex, then there is no question of "who" (i.e. what structure) experiences the memory output - (almost) the entire brain does.

However, if the entire simulation exists in the hippocampus, then it suggests that there may be a brain part that "witnesses" the new memory. Or is the hippocampus itself somehow experiencing its own simulation? Or is that the wrong way of looking at it; since the simulation = experience, is it enough to have been generated (and then buffered, or later recalled) in the hippocampus, so each memory builds on the previous one?

And, are the two experience generators (neocortex vs. hippocampus) working largely independently, and only checking in with each other periodically to compare notes? H.M. and similar patients experience without a hippocampus, which means that there is a kind of experience happening, separate from the simulation. Maybe it's less unified, less specific temporally or spatially, but H.M. could interact with the world consistently, as long as his attention was caught.

This latter view seems to fit with my hypothesis about hemineglect and anosognosia: the hippocampal simulation, unable to be updated by the neocortex experience, just moves forward with its own internal logic, its own story-in-progress, almost like a dream.

Which is part of why I find so fascinating the datum that ice cold water, poured into the anosognosic hemiplegic patient's right ear, was able to temporarily rouse the patient from her delusion (until eventually she fades back into the string-of-memories, and forgets that she previously was aware of her paralysis). All of which suggests that it is possible to experience in either realm: simulation or neocortex (flow states, like "the zone" suggest this as well). Perhaps there will be techniques developed to allow us to switch, at will, between these experiences, depending upon what the situation calls for (and maybe that's what repetition training, like learning to play the piano, already does). Maybe that's what mindfulness is?

Of course, there's the question whether a pure neocortex experience can actually be remembered, or if only the hippocampal experience is available later. When the anosognosic forgets the epiphany she had after the ice water in the ear trick, it suggests that since that epiphany does not fit with the stream of memories in the hippocampus, it is discarded (or sectioned off, similar perhaps to the division of memories within dissociative identity disorder).

best,

matt faw

Thanks, Ralf! I found the following article, about the connections between the CA1 field and the IPL.

http://www.jneurosci.org/content/21/16/6283.full.pdf+html Which illustrates a robust output from CA1 to the IPL, and a smaller feed in the other direction. Interesting how the article shows that the IPL also receives robust input from cerebellum and superior colliculus. This suggests to me that the IPL may help compare or coordinate egocentric with vestibular with allocentric, to help orient the body in the world. One of the crucial questions, I think, is how much of an experience is contained wholly in the hippocampus, vs. how much of it is represented by an index/mapping in the hippocampus, but is played out in other parts of the brain. This is particularly true with intrinsic simulations, like daydreaming or recall. We have our "player piano" metaphor of the memory playing upon the simulation engine of the HF, which then plays the sensory cortices to re-construct memories or generate imaginary experiences, but I don't know how accurate it is, or what the mechanism is for doing so. It's not necessarily something that needs to be resolved within the scope of our paper, but it does feel like the one of the next necessary questions. I think if we are right about the hippocampus being the neural correlate of subjectivity, then it suggests a good deal about what functions are played by other related structures. I think your theory opens up the door to an enormous amount of related questions and testable hypotheses, which I hope will usher in a new era of consciousness studies. The question no longer has to be: what is consciousness? But rather: how does the simulation relate to the rest of the brain, and how can we minimize our own confabulation and delusion, from within the simulation? All of which is beyond the scope of the paper or the movie, but it will be very exciting to see what science does with this model. Thanks again for your correspondence! best, matt faw