Zhuosheng Zhang

Tenure-Track Assistant Professor
School of Cyber Science and Engineering
Shanghai Jiao Tong University
Email: zhangzs@sjtu.edu.cn
Office: SEIEE Building 3-220,
800 Dongchuan Road, Shanghai.

Profile

Zhuosheng Zhang is a tenure-track assistant professor at Shanghai Jiao Tong University. He received his Ph.D. degree and his M.S. degree from Shanghai Jiao Tong University in 2023 and 2020, advised by Prof. Hai Zhao. He was an intern at Amazon Web Services, Microsoft Research Redmond, Langboat Tech, NICT (Japan), and IBM. He serves as a PC member for ARR, ICML, NeurIPS, ICLR, ACL, AAAI, etc. He served as an area chair for LREC-Coling 2024, ICLR 2023 TinyPapers and a co-chair at CCL 2022.

His primary research interests include natural language processing, large language models, and autonomous agents. He has published over 50 papers in top-tier conferences and journals, including TPAMI, ICLR, ACL, AAAI, EMNLP, TNNLS, TASLP, and COLING. He has won 1st place in various language understanding and reasoning leaderboards, such as SQuAD2.0, MuTual, RACE, ShARC, and CMRC. He was awarded as an Academic Star at Shanghai Jiao Tong University and was selected as one of the Global Top 100 Chinese Rising Stars in Artificial Intelligence. He won the Baidu Scholarship and WAIC YunFan Award: Rising Star.

Tutorials

  • LREC-COLING 2024: From Multimodal LLM to Human-level AI: Modality, Instruction, Reasoning, Efficiency and Beyond
    Hao Fei, Yuan Yao, Zhuosheng Zhang, Fuxiao Liu, Ao Zhang and Tat-Seng Chua.
  • IJCNLP-AACL 2023: Learning WHO Saying WHAT to WHOM in Multi-Party Conversations
    Jia-Chen Gu, Zhuosheng Zhang, and Zhen-Hua Ling.
  • IJCAI 2021: Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
    Zhuosheng Zhang and Hai Zhao. [slides]

Talks

  • 2023/11: Keynote "Autonomous Language Agents" at CJNLP 2023.
  • 2023/09: Talk "Autonomous Language Agents" at MNNLP 2023. [slides]
  • 2023/03: Talk "Chain-of-Thought Reasoning In Language Models" at WestlakeNLP, FudanNLP, and Bytedance. [slides]
  • 2022/12: Talk "Automatic Chain of Thought Prompting in Large Language Models" at Amazon AWS. [slides]
  • 2022/07: Talk "Large-scale Multi-task Pre-training" at Tencent AI Lab. [slides]
  • 2022/06: Talk "Large-scale Multi-task Pre-training" at Microsoft Research. [slides]
  • 2021/11: Talk "Mengzi Lightweight Pre-trained Models" at Big Model Meetup, with Dr. Ming Zhou. [slides]
  • 2021/11: Talk "Machine Reading Comprehension: The Paradigm of Pre-trained Models" at MLNLP 2021. [slides]
  • 2021/08: Tutorial "Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond" at IJCAI 2021, with Prof. Hai Zhao. [slides]
  • 2021/07: Talk "Machine Reading Comprehension and Dialogue Systems" at Huawei Shanghai Institute. [slides]
  • 2020/10: Talk "My Way to Reading Comprehension: Self-cognition and Persistence" at CCL 2020 Student Workshop. [slides]
  • 2020/05: Talk "Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond" at UofA NLP seminar on MRC. [slides]
  • 2017/10: Talk "Fine-grained Embedding for Reading Comprehension" at CMRC 2017 workshop in CCL 2017. [slides]

Selected Publications

Discover the full list | google scholar | semantic scholar | dblp.
[LLMs & Agents]
  • Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents
    Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, Zhiwei He, Yiming Wang, Mark Gerstein, Rui Wang, Gongshen Liu, Hai Zhao.
    arXiv, 2023
    [PDF] [Abstract] [Bib]
  • You Only Look at Screens: Multimodal Chain-of-Action Agents
    Zhuosheng Zhang, Aston Zhang.
    arXiv, 2023
    "Perform a task on smart phones? Train an agent using screenshots."
    [PDF] [Abstract] [Bib] [slides]
  • Multimodal Chain-of-Thought Reasoning in Language Models
    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, Alex Smola.
    arXiv, 2023
    "Imagine learning a textbook with no figures: Multimodal-CoT surpasses humans on ScienceQA."
    Featured in Dive into Deep Learning (Adopted at 400 universities from 60 countries)
    [Top Trending Research on paperswithcode] [Idea Inspiration] [PDF] [Abstract] [Bib] [slides]
  • Automatic Chain of Thought Prompting in Large Language Models
    Zhuosheng Zhang, Aston Zhang, Mu Li, Alex Smola.
    ICLR, 2023
    "Let's think not just step by step, but also one by one."
    Featured in Dive into Deep Learning (Adopted at 400 universities from 60 countries)
    [PDF] [Abstract] [Bib] [bilibili] [slides]
  • On Element-aware Automatic Summarization: Expert-writing Test Set and Chain-of-Thought Method
    Yiming Wang, Zhuosheng Zhang, Rui Wang.
    ACL, 2023
    "You really need higher-quality reference summaries to evaluate LLMs!"
    [PDF] [Abstract] [Bib]
  • Exploring Human-Like Translation Strategy with Large Language Models
    Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, Xing Wang.
    TACL, 2023
    Delves into LLMs' potential for mimicking human translation strategies.
    [PDF] [Abstract] [Bib]
  • Self-Prompting Large Language Models for Open-Domain QA
    Junlong Li, Zhuosheng Zhang, Hai Zhao.
    arXiv, 2023
    "Free from training data and external knowledge corpus for ODQA."
    [PDF] [Abstract] [Bib]
  • Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
    Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, Diyi Yang
    EMNLP, 2023
    "Benchmarking ChatGPT on 20 popular NLP datasets covering 7 representative task categories."
    [PDF] [Abstract] [Bib]
[Augmented Language Models]
  • Learning Better Masking for Better Language Model Pre-training
    Dongjie Yang, Zhuosheng Zhang*, Hai Zhao*.
    ACL, 2023
    [PDF] [Abstract] [Bib]
  • Universal Multimodal Representation for Language Understanding
    Zhuosheng Zhang, Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Zuchao Li, Hai Zhao
    TPAMI, 2023
    "Let's retrieve images to overcome the lack of large-scale bilingual pairs."
    [PDF] [Abstract] [Bib]
  • SG-Net: Syntax Guided Transformer for Language Representation
    Zhuosheng Zhang, Yuwei Wu, Junru Zhou, Sufeng Duan, Hai Zhao, Rui Wang.
    TPAMI, 2022
    [PDF] [Abstract] [Bib]
  • Task Compass: Scaling Multi-task Pre-training with Task Prefix
    Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
    EMNLP (Findings), 2022
    Rank 1st on the HellaSwag commonsense reasoning leaderboard & The first to achieve human parity.
    [PDF] [Abstract] [Bib] [Slides]
  • Structural Pre-training for Dialogue Comprehension
    Zhuosheng Zhang, Hai Zhao.
    ACL, 2021
    [PDF] [Abstract] [Bib]
  • Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
    Zhuosheng Zhang, Hanqing Zhang, Keming Chen, Yuhang Guo, Jingyun Hua, Yulong Wang, Ming Zhou.
    arXiv, 2021
    [PDF] [Abstract] [Bib] [Slides]
[LLM Applications]
  • Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehension
    Xinbei Ma, Zhuosheng Zhang*, Hai Zhao*.
    TASLP, 2023
    [PDF] [Abstract] [Bib]
  • Decker: Double Check with Heterogeneous Knowledge for Commonsense Fact Verification
    Anni Zou, Zhuosheng Zhang, Hai Zhao.
    ACL (Findings), 2023
    [PDF] [Abstract] [Bib]
  • Retrospective Reader for Machine Reading Comprehension
    Zhuosheng Zhang, Junjie Yang, Hai Zhao.
    AAAI, 2021
    Rank 1st on the SQuAD2.0 leaderboard.
    * Highly Influential: featured as Most Influential AAAI 2021 Paper (top 5) in Paper Digest
    . [PDF] [Abstract] [Bib]
  • Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
    Zhuosheng Zhang, Hai Zhao, Rui Wang.
    arXiv, 2020
    [PDF] [Abstract] [Bib] [Slides]
  • Semantics-aware BERT for Natural Language Understanding
    Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, Xiang Zhou.
    AAAI, 2020
    * Highly Influential: featured in the Google Scholar Metrics 2022, citations rank: 19/1591 (top 1.2%) in AAAI 2020.
    [PDF] [Abstract] [Bib]
  • Modeling Multi-turn Conversation with Deep Utterance Aggregation
    Zhuosheng Zhang, Jiangtong Li, Pengfei Zhu, Hai Zhao and Gongshen Liu.
    COLING, 2018
    * Highly Influential: featured in the Google Scholar 2020 h5-index list, top 1.2% (4/331) in COLING 2018.
    [PDF] [Abstract] [Bib]

Shared Tasks

[May 2022] HellaSwag Leaderboard on Commonsense Reasoning
[January 2021] ShARC Leaderboard on Conversational Question Answering
[September 2020] MuTual Leaderboard on Dialogue Reasoning Challenge
[July 2019] SQuAD2.0 Leaderboard on Machine Reading Comprehension
  • The best models for both single and ensemble settings among all submissions (2020.01).
  • The first to surpass human benchmark on both EM and F1 scores with a single model (from 2019.07-09).
  • The first time to exceed 90% F1 score with ensemble models.
    [Leaderboard] [Paper] [Report]
[March 2019] RACE Leaderboard on Machine Reading Comprehension
[April 2019] SNLI Leaderboard on Language Inference [March 2019] GLUE Leaderboard on Language Understanding
  • The 3rd best among all submissions.
  • The best among all academic submissions.
    [Leaderboard] [Paper]
[August 2017] Chinese Machine Reading Comprehension (CCL-CMRC 2017)

Awards & Honors

  • 2023: WAIC YunFan Award, Rising Star, World Artificial Intelligence Conference.

  • 2023: Shanghai Outstanding Doctoral Graduate.

  • 2022: Academic Stars of Graduate Students (10 recipients), Shanghai Jiao Tong University.

  • 2021: Global Top 100 Chinese Rising Stars in Artificial Intelligence (Top 10 recommended), Baidu Research.

  • 2021: Baidu Scholarship (Top-10, worldwide), Baidu.

  • 2020: National Scholarship of China, Ministry of Education of the P.R. China.

  • 2019: Yang Yuanqing Education Fund, The foundation of Class 1988 in CS @ Shanghai Jiao Tong University.

  • 2018: Academic Stars of Graduate Students (The only master student awardee), Shanghai Jiao Tong University.

  • 2016: National Figures Nomination of College Students (20 total recipients), Ministry of Education of the P.R. China.

  • 2015: CCF Elite Collegiate Award, China Computer Federation.

Teaching

  • Guest Lecturer, NIS8021: Frontier Technology in Natural Language Processing”
    Graduate, Shanghai Jiao Tong University, Winter 2022.
  • Teaching Assistant, F03356: Natural Language Understanding
    Graduate, Shanghai Jiao Tong University, Spring 2021.
  • Teaching Assistant, F03356: Natural Language Understanding
    Graduate, Shanghai Jiao Tong University, Spring 2019.
  • Teaching Assistant, F03356: Natural Language Understanding
    Graduate, Shanghai Jiao Tong University, Spring 2018.

Academic Service

  • Organization:
    • Co-chair of CCL Student Seminar, 2022
    • President of IBM Tech Club at Wuhan University, 2014-2015.
  • Area Chair:
    • LREC-Coling 2024
    • ICLR 2023 TinyPapers
  • Program Committee Member:
    • ML/AI conferences: ICLR, ICML, NeurIPS, AAAI, IJCAI, etc.
    • CL/NLP conferences: ARR, ACL, EMNLP, COLING, NAACL, AACL, NLPCC, CCL, etc.

  • Journal Reviewer: Artificial Intelligence, IEEE/ACM TASLP, IEEE TNNLS, IEEE TETCI, IEEE Communications Magazine, ACM TALLIP, ACM TOIS, TMLR, Neurocomputing, Multimedia Systems, Neural Computing and Applications, Expert Systems With Applications.

Student Advising

  • Siru Ouyang (Undergraduate Student at SJTU → PhD Student at UIUC)
  • Jialin Chen (Undergraduate Student at SJTU → PhD Student at Yale University)
  • Yuchen He (Undergraduate Student at SJTU → MS Student at SJTU)
  • Junlong Li (Undergraduate Student at SJTU → MS Student at SJTU)
  • Longxiang Liu (Undergraduate Student at SJTU → MS Student at ICT/CAS)
  • Yuwei Wu (Undergraduate Student at SJTU → MS Student at CMU)