Understanding Shitsukan, or realistic qualities of the world, in visual experiences requires more than simple models or language-based descriptions. This chapter introduces a brain decoding approach centered on mid-level visual features represented in both the brain and deep neural networks (DNNs). By translating brain activity into DNN features and using generative models to reconstruct detailed visual images, we achieve high-fidelity decoding of subjective experiences, including mental imagery and visual illusions. Our approach extends beyond individual subjects, demonstrating inter-individual and inter-site generalization of these brain-derived patterns. Despite these advances, challenges persist in ensuring accurate hierarchical correspondence between brain and DNN features and mitigating spurious reconstructions. We propose a new research direction, the Neuroverse, which aims to externalize not only sensory representations but also the brain’s internal model of the world. This framework opens new avenues for exploring how Shitsukan and other complex perceptual experiences are constructed and represented in the brain.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Decoding Shitsukan in The Brain

  • Yukiyasu Kamitani,
  • Misato Tanaka,
  • Tomoyasu Horikawa

摘要

Understanding Shitsukan, or realistic qualities of the world, in visual experiences requires more than simple models or language-based descriptions. This chapter introduces a brain decoding approach centered on mid-level visual features represented in both the brain and deep neural networks (DNNs). By translating brain activity into DNN features and using generative models to reconstruct detailed visual images, we achieve high-fidelity decoding of subjective experiences, including mental imagery and visual illusions. Our approach extends beyond individual subjects, demonstrating inter-individual and inter-site generalization of these brain-derived patterns. Despite these advances, challenges persist in ensuring accurate hierarchical correspondence between brain and DNN features and mitigating spurious reconstructions. We propose a new research direction, the Neuroverse, which aims to externalize not only sensory representations but also the brain’s internal model of the world. This framework opens new avenues for exploring how Shitsukan and other complex perceptual experiences are constructed and represented in the brain.