Publications

Selected Publications (2014–2026)

An asterisk (*) marks the corresponding author. Underlined names indicate current or former graduate students supervised by Yunlong Liu.

In addition to the publications below, several papers are under review. These papers are led by supervised students as first authors, with Liu as corresponding author. Topics include deep reinforcement learning theory (representation learning, reward design, and imitation learning) and applications in autonomous navigation and obstacle avoidance, medical decision-making, and traffic signal control. Ongoing work focuses on navigation, obstacle avoidance, and landing for autonomous systems—including autonomous vehicles, quadruped robots, and UAVs—in complex and extreme environments.

[29] Liang Dayang, Lai Jinyang, Liu Yunlong*. Intrinsic Dynamic-Driven Representation Learning for Generalization in Visual Reinforcement Learning. Under review.

[28] Zhang Qiwei, Lin Bin, Liu Yunlong*. A data augmentation-based deep reinforcement learning approach for high-risk states avoidance in sepsis treatment. Applied Intelligence, 56(16), Article 496, 2026. (Chinese Academy of Sciences (CAS) Q3). doi:10.1007/s10489-026-07393-w.

[27] Liang Dayang, Liu Ruihang, Wan Lipeng, Liu Yunlong*, An Bo. Task-Aware Exploration via a Predictive Bisimulation Metric. Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:73336–73355, 2026. (Top conference in machine learning; CCF A).

[26] Jiang Zejie, Lai Jinyang, Liu Yunlong*. BEV-OSP: Obstacle State Prediction in Bird’s-Eye View to Enable Obstacle Avoidance and Navigation in Dynamic Environments. IEEE Robotics and Automation Letters, 11(7):7924–7931, 2026. (Leading robotics journal; CAS Q2). doi:10.1109/LRA.2026.3692074.

[25] Lai Jinyang, Jiang Zejie, Liu Yunlong*. DARE to Navigate: Spatiotemporal Dynamic Augmented LiDAR Encoder for Obstacle Detection and Avoidance in Crowded Environments. IEEE Robotics and Automation Letters, 11(2):1930–1937, February 2026. (Leading robotics journal; CAS Q2). doi:10.1109/LRA.2025.3645654.

[24] Liang Dayang, Liu Yunlong*. Learning Representations via Dynamics-based Behavioral Similarity for Deep Reinforcement Learning. Neural Networks, 197:108468, 2026. (CAS Q2; Top journal).

[23] Zhang Yaru, Chen Kaizhou, Liu Yunlong*. Causal Representation Learning in Offline Visual Reinforcement Learning. Knowledge-Based Systems, 320:113565, 2025. (CAS Q1; Top journal).

[22] Zhang Qiwei, Lin Bin, Liu Yunlong*. A Sequential Coupled Adversarial Learning Method for Sepsis Trajectory Generation. Computer Engineering, 2026, 52(2).

[21] Liang Dayang, Chen Qihang, Liu Yunlong*. Sequential Action-Induced Invariant Representation for Reinforcement Learning. Neural Networks, 179, Article 106579, 2024. (CAS Q1; Top journal; CCF B).

[20] Yuan Linghui, Lu Xiaowei, Liu Yunlong*. Learning task-relevant representations via rewards and real actions for reinforcement learning. Knowledge-Based Systems, 294, Article 111788, 2024. doi:10.1016/j.knosys.2024.111788. (CAS Q1; Top journal).

[19] Liang Dayang, Zhang Yaru, Liu Yunlong*. Episodic Reinforcement Learning with Expanded State-reward Space. AAMAS 2024, May 2024. (CCF B; full/oral paper).

[18] Chen Qihang, Zhang Qiwei, Liu Yunlong*. Balancing Exploration and Exploitation in Episodic Reinforcement Learning. Expert Systems with Applications, 2023. (CAS Q1; Top journal).

[17] Liang Dayang, Deng Huiyi, Liu Yunlong*. The Treatment of Sepsis: An Episodic Memory-assisted Deep Reinforcement Learning Approach. Applied Intelligence, 2022. (CAS Q2).

[16] Xu Aimin, Yuan Linghui, Liu Yunlong*. Sequential Decision Making with “Sequential Information” in Deep Reinforcement Learning. PRICAI 2022. (CCF C).

[15] Liu Yunlong*, Zheng Jianyang, Chang Fangfang. Learning and planning in partially observable environments without prior domain knowledge. International Journal of Approximate Reasoning, 142:147–160, 2022. (CAS Q2; CCF B journal).

[14] Chang Fangfang, Chen Qihang, Liu Yunlong*. Model-free Deep Reinforcement Learning in Partially Observable Environments with Future Information. Journal of Nanjing University (Natural Sciences), 2022, 58(5):796–804. doi:10.13232/j.cnki.jnju.2022.05.006.

[13] Chen Qihang, Liang Dayang, Liu Yunlong*. Hard Negative Sample Mining for Contrastive Representation in Reinforcement Learning. The 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2022), May 2022. (CCF C).

[12] Liang Dayang, Chen Qihang, Liu Yunlong*. Gated multi-attention representation in reinforcement learning. Knowledge-Based Systems, 2021, 233:107535. (CAS Q1; Top journal).

[11] Yu Danning, Ni Kun, Liu Yunlong*. A POMDP Value Iteration Algorithm Based on Recurrent Convolutional Neural Networks. Computer Engineering, 47(2):90–94, 2021.

[10] Ni Kun, Liu Yunlong*, Yu Danning. A Model-based Deep Reinforcement Learning Algorithm Based on a Memory Exploration Strategy. Microelectronics & Computer, 38(4):23–28, 2021.

[9] Yu Danning, Ni Kun, Liu Yunlong*. Deep Q-Network with Predictive State Models in Partially Observable Domains. Mathematical Problems in Engineering, 2020:1–9, 2020. (SCI).

[8] Liu Yunlong*, Zheng Jianyang. Online learning and planning in partially observable domains without prior knowledge. ICML 2019 Workshop on Generative Modeling and Model-Based Reasoning for Robotics and AI, 2019. (The ICML main conference is CCF A; this paper appeared in a workshop).

[7] Ni Kun, Yu Danning, Liu Yunlong*. Attention-Based Deep Q-Network in Complex Systems. International Conference on Neural Information Processing (ICONIP 2019), 2019:323–332. (CCF C).

[6] Zheng Jianyang, Zhu Hexing, Chang Fangfang, Liu Yunlong*. An improved relief feature selection algorithm based on Monte-Carlo tree search. Systems Science and Control Engineering, 7(1):304–310.

[5] Huang Chunqing, An Yisheng, Zhou Sun, Hong Zhezheng, Liu Yunlong*. Basis selection in spectral learning of predictive state representations. Neurocomputing, 310:183–189, 2018. (CAS Q2).

[4] Liu Yunlong*, Zeng Yifeng, Zhu Hexing, Tang Yun. Making and Improving Predictions of Interest Using an MDP Model. Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017), 1610–1612, 2017. (CCF B).

[3] Liu Yunlong*, Zhu Hexing, Zeng Yifeng, Dai Zongxiong. Learning Predictive State Representations via Monte-Carlo Tree Search. The 25th International Joint Conference on Artificial Intelligence (IJCAI 2016), 3192–3198, 2016. (CCF A; full/oral paper).

[2] Liu Yunlong*, Tang Yun, Zeng Yifeng. Predictive State Representations with State Space Partitioning. Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), 1259–1266, 2015. (CCF B; full/oral paper).

[1] Liu Yunlong*, Yang Zijiang, Ji Guoli. Solving Partially Observable Problems with Inaccurate PSR Models. Information Sciences, 283:142–152, 2014. (CCF B journal; CAS Q2).