Research Interests
Towards Cognitive-Affective Symbiosis
Moving beyond surface-level signal processing to build embodied, psychologically grounded intelligence.
How can we build psychologically grounded, fine-grained, multimodal, efficient, evolving, and full-duplex conversational AI systems that better understand and respond to human emotions?
Paradigm Shift Analysis Gap Analysis
Current large language models excel at text generation but face critical bottlenecks in social cognition, memory retention, and interaction dynamics.
Social Agency
Powerful semantic engines remain fundamentally solipsistic. Limited Theory of Mind capabilities can result in superficial empathy.
Systems intrinsically motivated to align with human mental states, moving from understanding to resonating through psychological grounding.
Multimodal Synchrony
Fine-grained behavioral cues such as micro-expressions are often lost in text-dominant latent spaces.
Efficient adapters that preserve low-level acoustic and visual features, enabling real-time affective synchronization.
Temporal Dynamics
Systems suffer from temporal amnesia about affective history and are constrained by wait-to-speak turn taking.
Full-duplex, interruptible interaction backed by evolving long-term affective memory.
Research Horizons Strategic Trajectory
Foundation & Efficiency
- Develop data infrastructure for high-fidelity, fine-grained multimodal emotion analysis.
- Design cost-effective multimodal adapters that align language models with non-verbal signals.
- Enable efficient test-time adaptation for personalized user alignment.
Cognitive Kernel
- Build long-term affective memory with dynamic update and retrieval mechanisms.
- Integrate Theory of Mind to infer implicit mental states and intent.
- Create a “psychological mirror” that reflects and validates user emotions.
Embodiment & Complexity
- Realize full-duplex interaction that handles interruptions, backchannels, and multi-party dynamics.
- Bridge generative AI and robotics for controllable, parametric expression.
- Achieve cognitive-affective symbiosis in complex real-world settings.
迈向认知-情感共生系统
超越表层的信号处理，构建具身的、具有心理学基础的智能系统。
如何构建具有心理学基础、细粒度、多模态、高效率、可演进且全双工的对话式 AI 系统，使其更好地理解并回应人类情感？
范式转换分析 差距分析
当前大语言模型擅长文本生成，但在社会认知、长期记忆和实时交互动态方面仍面临关键瓶颈。
社会代理
模型虽是强大的语义引擎，却仍偏向“唯我式”推理；受限的心智理论能力容易产生表层共情。
让系统具有对齐人类心理状态的内在动机，通过心理学基础从“理解”走向“共鸣”。
多模态同步
微表情等细粒度行为线索常在以文本为主导的潜空间中丢失。
设计高效适配器，保留底层声学与视觉特征，实现实时情感同步。
时间动态
系统对情感历史存在“时间失忆”，且受限于等待-发言式轮流对话。
构建由持续演进的长期情感记忆支持、可打断的全双工交互。
研究视界 战略路径
基石与效率
- 建立用于高保真、细粒度多模态情感分析的数据基础设施。
- 设计低成本多模态适配器，将语言模型与非语言信号对齐。
- 实现高效测试时适应，达成个性化用户对齐。
认知核心
- 构建支持动态更新与检索的长期情感记忆。
- 整合心智理论，推断隐含心理状态与意图。
- 打造能够反映并确认用户情绪的“心理镜像”。
具身与复杂性
- 实现能够处理打断、反馈词和多人动态场景的全双工交互。
- 连接生成式 AI 与机器人技术，实现可控的参数化表情。
- 在复杂现实场景中实现认知-情感共生。