Biography

I am a 3rd-year Ph.D. Student at College of Computer Science and Technology, Zhejiang University, China, working on Federated Learning under the supervision of Prof. Shuiguang Deng. Currently, I mainly focus on 1) Federated Fine-tuning of Large Language Models, 2) Personalized Federated Learning and 3) Trustworthy Federated Learning. So far, I have published 12 papers on ICML, SIGKDD, AAAI, ICSOC, IEEE ICWS, IEEE TSC, KBS, etc.

Currently, I am working as a research intern at DAMO Academy, Alibaba Group, focusing on Federated Fine-tuning of Large Language Models. Prior to that, I worked as a research intern in 2012 Lab, Huawei Technologies Ltd. Co. from Mar. 2022 to May 2023, focusing on Preliminary Research of 6G architecture.

Research Topics

Currently, I am focusing on several research topics in Federated Learning, including:

• Federated Fine-tuning of Large Language Models
  Designing federated fine-tuning techniques for billion-sized large language models (LLMs), leveraging the vast quantities of data continuously generated at end devices to enhance the responsibility of LLMs to tasks described in natural language.
• Personalized Federated Learning
  Designing the personalized FL framework to provide adaptability to the statistical heterogeneity of data among distributed clients, to provide high model accuracy regardless of the degree of non-IIDness.
• Trustworthy Federated Learning
  • The defenses to backdoor attacks in FL through anomaly detection techniques.
  • Building fully decentralized FL systems on the basis of blockchain, providing distributed trustworthy for FL among peer participants.

Selected Publications

A full list of publications is available at Google Scholar.

• Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes
  Zhen Qin, Daoyuan Chen, Bingchen Qian, Bolin Ding, Yaliang Li, Shuiguang Deng.
  Accepted by ICML 2024 (CCF A)

• MACE: A Multi-pattern Accommodated and Efficient Anomaly Detection Method in the Frequency Domain
  Feiyi Chen, Yingying zhang, Zhen Qin, Lunting Fan, Renhe Jiang, Yuxuan Liang, Qingsong Wen, Shuiguang Deng.
  Accepted by ICDE 2024 (CCF A)

• BlockDFL: A Blockchain-based Fully Decentralized Peer-to-Peer Federated Learning Framework
  Zhen Qin, Xueqiang Yan, Mengchu Zhou, Shuiguang Deng*.
  WWW 2024 (CCF A)

• LARA: A Light and Anti-overfitting Retraining Approach for Unsupervised Anomaly Detection
  Feiyi Chen, Zhen Qin, Yingying Zhang, Shuiguang Deng, Yi Xiao, Guansong Pang, Qingsong Wen.
  WWW 2024 (CCF A)

• Resisting Backdoor Attacks in Federated Learning via Bidirectional Elections and Individual Perspective
  Zhen Qin, Feiyi Chen, Chen Zhi, Xueqiang Yan, Shuiguang Deng*.
  AAAI 2024 (CCF A)

• FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity
  Zhen Qin, Shuiguang Deng, Mingyu Zhao, Xueqiang Yan,
  SIGKDD 2023 (CCF A)

• 6G Data Plane: A Novel Architecture Enabling Data Collaboration with Arbitrary Topology
  Zhen Qin, Shuiguang Deng, Xueqiang Yan, Lu Lu, Mingyu Zhao, Yan Xi, Jianjun Wu, Tao Sun, Nanxiang Shi.
  Mobile Networks and Applications, 2022 (CCF C, JCR Q2)

• ST-EUA: Spatio-temporal Edge User Allocation with Task Decomposition.
  Guobing Zou, Ya Liu, Zhen Qin, Jin Chen, Zhiwei Xu, Yanglan Gan*, Bofeng Zhang, Qiang He*.
  IEEE Transactions on Services Computing, 2022 (CCF A, JCR Q1)

• Towards the Optimality of Service Instance Selection in Mobile Edge Computing
  Guobing Zou, Zhen Qin, Shuiguang Deng, Kuan-Ching Li, Yanglan Gan*, Bofeng Zhang*.
  Knowledge-Based Systems, 2021 (CCF C, JCR Q1)

• DeepWSC: Clustering Web Services via Integrating Service Composability into Deep Semantic Features
  Guobing Zou, Zhen Qin, Qiang He, Pengwei Wang, Bofeng Zhang, Yanglan Gan*.
  IEEE Transactions on Services Computing, 2020 (CCF A, JCR Q1)

• TD-EUA: Task-decomposable Edge User Allocation with QoE Optimization.
  Guobing Zou, Ya Liu, Zhen Qin*, Jin Chen, Zhiwei Xu, Yanglan Gan, Bofeng Zhang, Qiang He*.
  International Conference on Service Oriented Computing (ICSOC) 2020 (CCF B)

• DeepWSC: A Novel Framework with Deep Neural Network for Web Service Clustering.
  Guobing Zou, Zhen Qin, Qiang He, Pengwei Wang, Bofeng Zhang, Yanglan Gan*.
  IEEE International Conference on Web Services (IEEE ICWS) 2019 (CCF B)

Experience

• Research Intern at Damo Academy, Alibaba Group (2023.06 - Now)
  • Federated Fine-tuning of LLMs: Exploring the possibilities of tuning large models based on federated learning, mainly addressing communication overhead and memory cost issues.
  • System Development: Exploring memory-efficient fine-tuning techniques for LLMs which are suitable for cross-device FL, and integrating them into FederatedScope.
• Research Intern at 2012 Lab, Huawei Technologies, Ltd. Co. (2022.03 - 2023.04)
  • Network Architecture: Investigating existing network architectures; and design 6G network architecture according to the visions of 6G communications.
  • Collaboration Architecture: Designing collaboration frameworks for decentralized FL.
  • White Paper: Participating in the writing of two white papers for 6G ANA, i.e., i) 6G Data Service Framework and Enabling Technologies, and ii) Wireless Network Data Dictionary

Selected Honor & Awards

• Outstanding Graduate Student of Zhejiang University in 2023
• Outstanding Graduate Student of Zhejiang University in 2022
• Excellent Graduate of Shanghai
• National Scholarship for Graduate Students in 2020 (in Shanghai University)
• National Scholarship for Graduate Students in 2019 (in Shanghai University)
• Second Prize of China Post-graduate Mathematical Contest in Modeling (<14.5\%)