Zhuosheng Zhang

Ph.D. Candidate
Department of Computer Science and Engineering
Shanghai Jiao Tong University

Email: zhangzs@sjtu.edu.cn
Office: SEIEE Building 3-220

Profile

I am a Ph.D. candidate in the Department of Computer Science and Engineering at Shanghai Jiao Tong University, advised by Prof. Hai Zhao. Before that, I received M.S. degree from Shanghai Jiao Tong University in 2020, and B.S. degree from Wuhan University in 2016.

My primary research interests are natural language processing and machine learning, with a long-term goal of building foundation models to assist humans in an effective, interpretable, and robust way (Survey). In pursuit of this goal, I develop principled methodologies of powering the deep neural networks with massive linguistic, commonsense, and multimodal knowledge, in support of real-world application scenarios such as question answering and multi-turn dialogue.

News

Experience

Education

Tutorial

  • IJCAI-21 Tutorial: Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
    Zhuosheng Zhang and Hai Zhao. [slides]

Invited Talks

  • 2022/07: Talk "Large-scale Multi-task Pre-training" at Tencent AI Lab. [slides]
  • 2022/06: Talk "Large-scale Multi-task Pre-training" at Microsoft Research. [slides]
  • 2021/11: Talk "Mengzi Lightweight Pre-trained Models" at Big Model Meetup, with Prof. Ming Zhou. [slides]
  • 2021/11: Talk "Machine Reading Comprehension: The Paradigm of Pre-trained Models" at MLNLP 2021. [slides]
  • 2021/07: Talk "Machine Reading Comprehension and Dialogue Systems" at Huawei Shanghai Institute. [slides]
  • 2020/10: Talk "My Way to Reading Comprehension: Self-cognition and Persistence" at CCL 2020 Student Workshop. [slides]
  • 2020/05: Talk "Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond" at UofA NLP seminar on MRC. [slides]
  • 2017/10: Talk "Fine-grained Embedding for Reading Comprehension" at CMRC 2017 workshop in CCL 2017. [slides]

Selected Publications

Discover the Full list | google scholar | semantic scholar | dblp.
(#: equal contribution; Open resources are available in GitHub ★ )
[Preprints]
  • Automatic Chain of Thought Prompting in Large Language Models
    Zhuosheng Zhang, Aston Zhang, Mu Li, Alex Smola.
    "Let's think not just step by step, but also one by one."
    [PDF] [Abstract] [Bib]
  • Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
    Zhuosheng Zhang, Hai Zhao, Rui Wang.
    [PDF] [Abstract] [Bib] [Slides]
  • Advances in Multi-turn Dialogue Comprehension: A Survey
    Zhuosheng Zhang, Hai Zhao.
    [PDF] [Abstract] [Bib]
  • Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
    Zhuosheng Zhang, Hanqing Zhang, Keming Chen, Yuhang Guo, Jingyun Hua, Yulong Wang, Ming Zhou.
    [PDF] [Abstract] [Bib] [Slides]
Conference Papers
  • Instance Regularization for Discriminative Language Model Pre-training
    Zhuosheng Zhang, Hai Zhao and Ming Zhou
    The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022)
    [PDF] [Abstract] [Bib]
  • Task Compass: Scaling Multi-task Pre-training with Task Prefix
    Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu and Michael Zeng
    Findings of The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022)
    Rank 1st on the HellaSwag commonsense reasoning leaderboard & The first to achieve human parity.
    [PDF] [Abstract] [Bib] [Slides]
  • Tracing Origins: Coreference-aware Machine Reading Comprehension
    Baorong Huang#, Zhuosheng Zhang#, Hai Zhao.
    The 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022)
    [PDF] [Abstract] [Bib]
  • Smoothing Dialogue States for Open Conversational Machine Reading
    Zhuosheng Zhang#, Siru Ouyang#, Hai Zhao, Masao Utiyama and Eiichiro Sumita.
    The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021)
    [PDF] [Abstract] [Bib]
  • Structural Pre-training for Dialogue Comprehension
    Zhuosheng Zhang, Hai Zhao.
    The 59th Annual Meeting of the Association for Computational Linguistics (ACL 2021)
    [PDF] [Abstract] [Bib]
  • Dialogue Graph Modeling for Conversational Machine Reading
    Siru Ouyang#, Zhuosheng Zhang#, Hai Zhao.
    Findings of The 59th Annual Meeting of the Association for Computational Linguistics (ACL 2021)
    Rank 1st on the ShARC leaderboard.
    [PDF] [Abstract] [Bib]
  • Retrospective Reader for Machine Reading Comprehension
    Zhuosheng Zhang, Junjie Yang, Hai Zhao.
    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021)
    Rank 1st on the SQuAD2.0 leaderboard.
    [PDF] [Abstract] [Bib]
  • Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue
    Longxiang Liu#, Zhuosheng Zhang#, Hai Zhao, Xi Zhou, Xiang Zhou.
    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021)
    Rank 1st on the MuTual leaderboard.
    [PDF] [Abstract] [Bib]
  • Neural Machine Translation with Universal Visual Representation
    Zhuosheng Zhang, Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Zuchao Li, Hai Zhao.
    International Conference on Learning Representations (ICLR 2020)
    Spotlight Oral, acceptance rate: 4.16%.
    [PDF] [Abstract] [Bib] [Video]
  • Semantics-aware BERT for Natural Language Understanding
    Zhuosheng Zhang#, Yuwei Wu#, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, Xiang Zhou.
    The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), 34(5): 9628-9635, 2020.
    * Highly Influential: appeared in the Google Scholar Metrics 2022, citations rank: 19/1591 (top 1.2%) in AAAI 2020.
    [PDF] [Abstract] [Bib]
  • Syntax-Guided Machine Reading Comprehension
    Zhuosheng Zhang#, Yuwei Wu#, Junru Zhou, Sufeng Duan, Hai Zhao, Rui Wang.
    The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020)
    The first single model to surpass human benchmark on the SQuAD2.0 leaderboard.
    [PDF] [Abstract] [Bib]
  • Open Vocabulary Learning for Neural Chinese Pinyin IME
    Zhuosheng Zhang, Yafang Huang, Hai Zhao.
    The 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019)
    [PDF] [Abstract] [Bib]
  • One-shot Learning for Question-Answering in Gaokao History Challenge
    Zhuosheng Zhang and Hai Zhao.
    The 27th International Conference on Computational Linguistics (COLING 2018)
    [PDF] [Abstract] [Bib]
  • Modeling Multi-turn Conversation with Deep Utterance Aggregation
    Zhuosheng Zhang#, Jiangtong Li#, Pengfei Zhu, Hai Zhao and Gongshen Liu.
    The 27th International Conference on Computational Linguistics (COLING 2018)
    Appeared in the Google Scholar 2020 h5-index list, top 1.2% (4/331) in COLING 2018.
    [PDF] [Abstract] [Bib]
  • Subword-augmented Embedding for Cloze Reading Comprehension
    Zhuosheng Zhang#, Yafang Huang# and Hai Zhao.
    The 27th International Conference on Computational Linguistics (COLING 2018)
    [PDF] [Abstract] [Bib]
    Rank 1st on the CCL-CMRC 2017 shared task (single model).
Journal Articles

Shared Tasks

[May 2022] HellaSwag Leaderboard on Commonsense Reasoning
[January 2021] ShARC Leaderboard on Conversational Question Answering
[September 2020] Mutual Leaderboard on Dialogue Reasoning Challenge
[July 2019] SQuAD2.0 Leaderboard on Machine Reading Comprehension
[March 2019] RACE Leaderboard on Machine Reading Comprehension
[April 2019] SNLI Leaderboard on Language Inference [March 2019] GLUE Leaderboard on Language Understanding [August 2017] Chinese Machine Reading Comprehension (CCL-CMRC 2017)

Awards & Honors

Professional Service

Student Advising

(* I am co-advising undergraduate students at SJTU with Prof. Hai Zhao.)

Miscellaneous