About
I am a Ph.D. student at Harbin Institute of Technology, affiliated with the Web Intelligence Research Group, under the mentorship of Professors Yi Guan and Jingchi Jiang. Concurrently, I’m a visiting scholar in Autonomous Agents Research Group in the School of Informatics at the University of Edinburgh, advised by Professor Stefano V. Albrecht. I am primarily engaged in research on causal reinforcement learning, out-of-distribution generalization, and medical informatics, involving applications of intelligent control technologies and industrial implementation in fields such as smart healthcare and smart agriculture.
My long-term research goal is to use RL to develop an autonomous intelligence system for the real world. I believe that good RL agents need to model how the world works first [3] [4] and have strong generalization and robustness to deal with new challenges proposed by environments second [1] [2] [5] [7].
In modeling how the world works, we have done two research:
- The world model based on cascading theory [3] [4], which models the physiological domino effect in environment dynamics.
- Hidden-parameter block causal graph dynamics (Hip-BCGDs), which models environment dynamics with causal prompting from pre-trained models. With the help of Hip-BCGDs, we proposed a novel model-based offline RL framework -- Causal Prompting Reinforcement Learning (CPRL) [7] , which is suitable for highly suboptimal and diverse offline datasets. CPRL is validated in simulation-based glucose-insulin systems and real-world offline datasets from Dnurse APP.
- Active RL: A data-efficient and robust RL agent that can use real-time collected data to generalize in online testing quickly. We proposed a meta-RL framework -- Active Reinforcement Learning with Personalized Embeddings (ARLPE) [1] [5].
- Autonomous intelligence: A hierarchical RL framework for self-supervised learning skills and reusing learned skills. We propose a Causal Coupled Mechanism (CCM) [2] to train a single policy to reuse skills and reach multiple goals instead of training policies for each task separately. CCM is validated in synthetic systems and a real-world biological regulatory system.
Published Papers
|
[1] ARLPE: A Meta Reinforcement Learning Framework for Glucose Regulation in Type 1 Diabetics
Xuehui Yu, Yi Guan, Lian Yan, Shulang Li, Xuelian Fu, Jingchi Jiang*.
Expert Systems With Applications, IF: 8.665.
Keywords: Artificial pancreas, automated insulin treatment, diabetes, meta reinforcement learning, active learning.
How to accomplish fast adaptation in the meta-testing period with limited interaction data? How to address the data distribution mismatch problem? Here are the tricks! :)
Open-source Code / Paper
|
|
[2] Causal Coupled Mechanisms: A Control Method with Cooperation and Competition for Complex System
Xuehui Yu, Jingchi Jiang, Xinmiao Yu, Yi Guan*,Xue Li
The (BIBM) 2022 IEEE International Conference on Bioinformatics and Biomedicine.
Keywords: complex system control, causal reasoning, hierarchical reinforcement learning.
We propose a novel hierarchical reinforcement learning framework for complex biological system control, which can self-supervised learn skills and reuse learned skills.
Paper
|
|
[3] PercolationDF: A percolation-based medical diagnosis framework
Jingchi Jiang, Xuehui Yu, Lin Y, Yi Guan*, et al.
Mathematical Biosciences and Engineering, 2022, 19(6): 5832-5849.
Keywords: complex networks; knowledge representation; medical diagnosis; percolation theory
Paper
Partner: Xingyi People's Hospital
|
|
[4] DECAF: An Interpretable Deep Cascading Framework for ICU Mortality Prediction
Jingchi Jiang, Xuehui Yu, Boran Wang, Linjiang Ma, Yi Guan.
Artificial Intelligence in Medicine (2022): 102437.
Paper
|
|
[5] Contextual Policy Transfer in Meta-Reinforcement Learning via Active Learning
Jingchi Jiang, Lian Yan, Xuehui Yu and Yi Guan
19th International Conference on Web Information Systems and Applications.
Paper
|
|
[6] Unified Fine-Grained Biomedical Entity Recognition as a Combination of Boundary Detection and Sequence Generation
Xue Li, Yang Yang, Mingchen Ye, Yi Guan, Xuehui Yu, and Jingchi Jiang
Paper
The (BIBM) 2022 IEEE International Conference on Bioinformatics and Biomedicine.
|
Unpublished Papers
|
[7] Causal Prompting Model-based Offline Reinforcement Learning
Xuehui Yu, Yi Guan, Rujia Shen, Chen Tang and Jingchi Jiang*.
International Conference on Autonomous Agents and Multiagent Systems 2023. Under Review.
Keywords: Reinforcement learning, Model-based offline reinforcement learning, Causal, Prompt.
How to complete the online deployment of offline RL agents in low-resource scenarios? A brave attempt in a medical large-scale online system!
A medical benchmark is being built! A decision model can be trained in either supervised learning or offline reinforcement learning in the benchmark.
Partner: Beijing Dnurse Technology Company
|
Education
|
|
Harbin Institute of Technology - (2019-)
I began my doctoral studies directly following my undergraduate degree, thanks to the postgraduate recommendation scheme. I am currently pursuing a PhD at the Faculty of Computing at Harbin Institute of Technology.
|
|
Harbin Engineering University - (2015-2019)
I earned my bachelor’s degree in Internet of Things Engineering from the College of Computer Science and Technology, Harbin Engineering University in 2019. I was honoured the Outstanding Graduates in 2019.
|
Additional Information
|
