Zhuosheng Zhang

Ph.D. Candidate
Department of Computer Science and Engineering
Shanghai Jiao Tong University

Email: zhangzs@sjtu.edu.cn
Office: SEIEE Building 3-220

Profile

I am a Ph.D. candidate in the Department of Computer Science and Engineering at Shanghai Jiao Tong University, advised by Prof. Hai Zhao. Before that, I received M.S. degree from Shanghai Jiao Tong University in 2020, and B.S. degree from Wuhan University in 2016.

My primary research interests are natural language processing and machine learning, with the long-term goal of building intelligent machine learning systems with the human-level language comprehension ability to assist humans in an effective, interpretable, and robust way (Survey). In pursuit of this goal, I develop principled methodologies of powering the deep neural networks with massive linguistic, commonsense, and multimodal knowledge, in support of real-world application scenarios such as question answering and multi-turn dialogue.

News

Experience

Education

Tutorial

  • IJCAI-21 Tutorial: Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
    Zhuosheng Zhang and Hai Zhao. [slides]

Invited Talks

  • 2021/11: Talk "Mengzi Lightweight Pre-trained Models" at Big Model Meetup, with Prof. Ming Zhou.
  • 2021/11: Talk "Machine Reading Comprehension: The Paradigm of Pre-trained Models" at MLNLP 2021. [slides]
  • 2021/07: Talk "Machine Reading Comprehension and Dialogue Systems" at Huawei Shanghai Institute.
  • 2020/10: Talk "My Way to Reading Comprehension: Self-cognition and Persistence" at CCL 2020 Student Workshop. [slides]
  • 2020/05: Talk "Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond" at UofA NLP seminar on MRC. [slides]
  • 2017/10: Talk "Fine-grained Embedding for Reading Comprehension" at CMRC 2017 workshop in CCL 2017. [slides]

Selected Publications

Discover the Full list | google scholar | semantic scholar | dblp.
(#: equal contribution; Open resources are available in GitHub ★ )
[Preprints]
  • Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
    Zhuosheng Zhang, Hai Zhao, Rui Wang.
    [PDF] [Abstract] [Bib] [Slides]
  • Advances in Multi-turn Dialogue Comprehension: A Survey
    Zhuosheng Zhang, Hai Zhao.
    [PDF] [Abstract] [Bib]
  • Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
    Zhuosheng Zhang, Hanqing Zhang, Keming Chen, Yuhang Guo, Jingyun Hua, Yulong Wang, Ming Zhou.
    [PDF] [Abstract] [Bib]
[2022]
  • Tracing Origins: Coreference-aware Machine Reading Comprehension
    Baorong Huang#, Zhuosheng Zhang#, Hai Zhao.
    The 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022)
    [PDF] [Abstract] [Bib]
  • Structural Characterization for Dialogue Disentanglement
    Xinbei Ma, Zhuosheng Zhang, Hai Zhao.
    The 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022)
    [PDF] [Abstract] [Bib]
    Rank 1st on the Ubuntu IRC dialogue disentanglement benchmark.
  • Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval
    BoHong Wu, Zhuosheng Zhang, Jinyuan Wang, Hai Zhao.
    The 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022)
    [PDF] [Abstract] [Bib]
  • Distinguishing Non-natural from Natural Adversarial Samples for More Robust Pre-trained Language Model
    Jiayi Wang, Rongzhou Bao, Zhuosheng Zhang, Hai Zhao.
    The 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022, Findings)
    [PDF] [Abstract] [Bib]
  • Which Apple Keeps Which Doctor Away? Colorful Word Representations with Visual Oracles
    Zhuosheng Zhang, Haojie Yu, Hai Zhao, Masao Utiyama.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)
    [PDF] [Abstract] [Bib]
  • Syntax-aware Multi-spans Generation for Reading Comprehension
    Zhuosheng Zhang, Yiqing Zhang, Hai Zhao.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)
    [PDF] [Abstract] [Bib]
  • DUMA: Reading Comprehension with Transposition Thinking
    Pengfei Zhu, Zhuosheng Zhang, Hai Zhao, Xiaoguang Li.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)
    [PDF] [Abstract] [Bib]
  • Reference Knowledgeable Network for Machine Reading Comprehension
    Yilin Zhao, Zhuosheng Zhang, Hai Zhao.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)
    [PDF] [Abstract] [Bib]
[2021]
  • Smoothing Dialogue States for Open Conversational Machine Reading
    Zhuosheng Zhang#, Siru Ouyang#, Hai Zhao, Masao Utiyama and Eiichiro Sumita.
    The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021)
    [PDF] [Abstract] [Bib]
  • Span Fine-tuning for Pre-trained Language Models
    Rongzhou Bao, Zhuosheng Zhang, Hai Zhao.
    The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021, Findings)
    [PDF] [Abstract] [Bib]
  • Structural Pre-training for Dialogue Comprehension
    Zhuosheng Zhang, Hai Zhao.
    The 59th Annual Meeting of the Association for Computational Linguistics (ACL 2021)
    [PDF] [Abstract] [Bib]
  • Dialogue Graph Modeling for Conversational Machine Reading
    Siru Ouyang#, Zhuosheng Zhang#, Hai Zhao.
    The 59th Annual Meeting of the Association for Computational Linguistics (ACL 2021, Findings)
    Rank 1st on the ShARC leaderboard.
    [PDF] [Abstract] [Bib]
  • SG-Net: Syntax Guided Transformer for Language Representation
    Zhuosheng Zhang, Yuwei Wu, Junru Zhou, Sufeng Duan, Hai Zhao, Rui Wang.
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
    [PDF] [Abstract] [Bib]
  • Text Compression-aided Transformer Encoding
    Zuchao Li, Zhuosheng Zhang, Hai Zhao, Rui Wang, Kehai Chen, Masao Utiyama, and Eiichiro Sumita.
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
    [PDF] [Abstract] [Bib]
  • Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge
    Zhuosheng Zhang, Junlong Li, Hai Zhao.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)
    [PDF] [Abstract] [Bib]
  • Retrospective Reader for Machine Reading Comprehension
    Zhuosheng Zhang, Junjie Yang, Hai Zhao.
    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021)
    Rank 1st on the SQuAD2.0 leaderboard.
    [PDF] [Abstract] [Bib]
  • Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue
    Longxiang Liu#, Zhuosheng Zhang#, Hai Zhao, Xi Zhou, Xiang Zhou.
    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021)
    Rank 1st on the MuTual leaderboard.
    [PDF] [Abstract] [Bib]
[2020]
  • Neural Machine Translation with Universal Visual Representation
    Zhuosheng Zhang, Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Zuchao Li, Hai Zhao.
    International Conference on Learning Representations (ICLR 2020)
    Spotlight Oral, acceptance rate: 4.16%.
    [PDF] [Abstract] [Bib] [Video]
  • Semantics-aware BERT for Natural Language Understanding
    Zhuosheng Zhang#, Yuwei Wu#, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, Xiang Zhou.
    The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020)
    Rank 1st on the SNLI leaderboard.
    [PDF] [Abstract] [Bib]
  • Syntax-Guided Machine Reading Comprehension
    Zhuosheng Zhang#, Yuwei Wu#, Junru Zhou, Sufeng Duan, Hai Zhao, Rui Wang.
    The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020)
    The first single model to surpass human benchmark on the SQuAD2.0 leaderboard.
    [PDF] [Abstract] [Bib]
[2019] [2018]

Shared Tasks

[January 2021] ShARC Leaderboard on Conversational Question Answering
[September 2020] Mutual Leaderboard on Dialogue Reasoning Challenge
[July 2019] SQuAD2.0 Leaderboard on Machine Reading Comprehension
[March 2019] RACE Leaderboard on Machine Reading Comprehension
[April 2019] SNLI Leaderboard on Language Inference [March 2019] GLUE Leaderboard on Language Understanding [August 2017] Chinese Machine Reading Comprehension (CCL-CMRC 2017)

Awards & Honors

Professional Service