Zhuosheng Zhang

Ph.D. Candidate
Department of Computer Science and Engineering
Shanghai Jiao Tong University

Email: zhangzs@sjtu.edu.cn
Office: SEIEE Building 3-220

Profile

I am a Ph.D. candidate in the Department of Computer Science and Engineering at Shanghai Jiao Tong University, advised by Prof. Hai Zhao. Before that, I received M.S. degree from Shanghai Jiao Tong University in 2020, and B.S. degree from Wuhan University in 2016.

My primary research interests are natural language processing and machine learning, with the long-term goal of building foundation models with the human-level language comprehension ability to assist humans in an effective, interpretable, and robust way (Survey). In pursuit of this goal, I develop principled methodologies of powering the deep neural networks with massive linguistic, commonsense, and multimodal knowledge, in support of real-world application scenarios such as question answering and multi-turn dialogue.

News

Experience

Education

Tutorial

  • IJCAI-21 Tutorial: Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
    Zhuosheng Zhang and Hai Zhao. [slides]

Invited Talks

  • 2022/07: Talk "Large-scale Multi-task Pre-training" at Tencent AI Lab. [slides]
  • 2022/06: Talk "Large-scale Multi-task Pre-training" at Microsoft Research. [slides]
  • 2021/11: Talk "Mengzi Lightweight Pre-trained Models" at Big Model Meetup, with Prof. Ming Zhou. [slides]
  • 2021/11: Talk "Machine Reading Comprehension: The Paradigm of Pre-trained Models" at MLNLP 2021. [slides]
  • 2021/07: Talk "Machine Reading Comprehension and Dialogue Systems" at Huawei Shanghai Institute. [slides]
  • 2020/10: Talk "My Way to Reading Comprehension: Self-cognition and Persistence" at CCL 2020 Student Workshop. [slides]
  • 2020/05: Talk "Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond" at UofA NLP seminar on MRC. [slides]
  • 2017/10: Talk "Fine-grained Embedding for Reading Comprehension" at CMRC 2017 workshop in CCL 2017. [slides]

Selected Publications

Discover the full list: google scholar | semantic scholar | dblp.
(#: equal contribution; Open resources are available in GitHub ★ )
[Preprints]
  • Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
    Zhuosheng Zhang, Hai Zhao, Rui Wang.
    [PDF] [Abstract] [Bib] [Slides]
  • Advances in Multi-turn Dialogue Comprehension: A Survey
    Zhuosheng Zhang, Hai Zhao.
    [PDF] [Abstract] [Bib]
  • Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
    Zhuosheng Zhang, Hanqing Zhang, Keming Chen, Yuhang Guo, Jingyun Hua, Yulong Wang, Ming Zhou.
    Rank 1st on the CLUE leaderboard.
    [PDF] [Abstract] [Bib]
[Refereed Papers]
  • SG-Net: Syntax Guided Transformer for Language Representation
    Zhuosheng Zhang, Yuwei Wu, Junru Zhou, Sufeng Duan, Hai Zhao, Rui Wang.
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(6):3285-3299, 2022.
    [PDF] [Abstract] [Bib]
  • Text Compression-aided Transformer Encoding
    Zuchao Li, Zhuosheng Zhang, Hai Zhao, Rui Wang, Kehai Chen, Masao Utiyama, and Eiichiro Sumita.
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(7):3840-3857, 2022.
    [PDF] [Abstract] [Bib]
  • Tracing Origins: Coreference-aware Machine Reading Comprehension
    Baorong Huang#, Zhuosheng Zhang#, Hai Zhao.
    The 60th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1281-1292, 2022.
    [PDF] [Abstract] [Bib]
  • Which Apple Keeps Which Doctor Away? Colorful Word Representations with Visual Oracles
    Zhuosheng Zhang, Haojie Yu, Hai Zhao, Masao Utiyama.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 30:49-59, 2022.
    [PDF] [Abstract] [Bib]
  • Syntax-aware Multi-spans Generation for Reading Comprehension
    Zhuosheng Zhang, Yiqing Zhang, Hai Zhao.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 30:260-268, 2022.
    [PDF] [Abstract] [Bib]
  • Structural Pre-training for Dialogue Comprehension
    Zhuosheng Zhang, Hai Zhao.
    The 59th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 5134-5145, 2021.
    [PDF] [Abstract] [Bib]
  • Smoothing Dialogue States for Open Conversational Machine Reading
    Zhuosheng Zhang#, Siru Ouyang#, Hai Zhao, Masao Utiyama and Eiichiro Sumita.
    The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3685-3696, 2021.
    [PDF] [Abstract] [Bib]
  • Retrospective Reader for Machine Reading Comprehension
    Zhuosheng Zhang, Junjie Yang, Hai Zhao.
    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI), 35(16): 14506-14514, 2021.
    Rank 1st on the SQuAD2.0 leaderboard.
    [PDF] [Abstract] [Bib]
  • Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue
    Longxiang Liu#, Zhuosheng Zhang#, Hai Zhao, Xi Zhou, Xiang Zhou.
    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI), 35(15): 13406-13414, 2021.
    Rank 1st on the MuTual leaderboard.
    [PDF] [Abstract] [Bib]
  • Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge
    Zhuosheng Zhang, Junlong Li, Hai Zhao.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 29:1161-1173, 2021.
    [PDF] [Abstract] [Bib]
  • Open Named Entity Modeling from Embedding Distribution
    Ying Luo#, Hai Zhao, Zhuosheng Zhang#, Bingjie Tang.
    IEEE Transactions on Knowledge and Data Engineering (TKDE), 2021
    [PDF] [Abstract] [Bib]
  • Neural Machine Translation with Universal Visual Representation
    Zhuosheng Zhang, Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Zuchao Li, Hai Zhao.
    International Conference on Learning Representations (ICLR), 2020.
    Spotlight Oral, acceptance rate: 4.16%.
    [PDF] [Abstract] [Bib] [Video]
  • Semantics-aware BERT for Natural Language Understanding
    Zhuosheng Zhang#, Yuwei Wu#, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, Xiang Zhou.
    The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), 34(5): 9628-9635, 2020.
    * Highly Influential: appeared in the Google Scholar Metrics 2022, citations rank: 19/1591 (top 1.2%) in AAAI 2020.
    [PDF] [Abstract] [Bib]
  • Syntax-Guided Machine Reading Comprehension
    Zhuosheng Zhang#, Yuwei Wu#, Junru Zhou, Sufeng Duan, Hai Zhao, Rui Wang.
    The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), 34(5): 9636-9643, 2020.
    The first single model to surpass human benchmark on the SQuAD2.0 leaderboard.
    [PDF] [Abstract] [Bib]
  • Open Vocabulary Learning for Neural Chinese Pinyin IME
    Zhuosheng Zhang, Yafang Huang, Hai Zhao.
    The 57th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1584-1594, 2019.
    [PDF] [Abstract] [Bib]
  • Effective Subword Segmentation for Text Comprehension
    Zhuosheng Zhang, Hai Zhao, Kangwei Ling, Jiangtong Li, Zuchao Li, Shexia He, Guohong Fu.
    IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 27(11):1664-1674, 2019.
    [PDF] [Abstract] [Bib]
  • Modeling Multi-turn Conversation with Deep Utterance Aggregation
    Zhuosheng Zhang#, Jiangtong Li#, Pengfei Zhu, Hai Zhao and Gongshen Liu.
    The 27th International Conference on Computational Linguistics (COLING), pp. 3740-3752, 2018.
    * Highly Influential: appeared in the Google Scholar Metrics 2020, citations rank: 4/331 (top 1.2%) in COLING 2018.
    [PDF] [Abstract] [Bib]
  • One-shot Learning for Question-Answering in Gaokao History Challenge
    Zhuosheng Zhang and Hai Zhao.
    The 27th International Conference on Computational Linguistics (COLING), pp. 449-461, 2018.
    [PDF] [Abstract] [Bib]
  • Subword-augmented Embedding for Cloze Reading Comprehension
    Zhuosheng Zhang#, Yafang Huang# and Hai Zhao.
    The 27th International Conference on Computational Linguistics (COLING), pp. 1802-1814, 2018.
    [PDF] [Abstract] [Bib]
    Rank 1st on the CCL-CMRC 2017 shared task (single model).

Shared Tasks

[May 2022] HellaSwag Leaderboard on Commonsense Reasoning
[January 2021] ShARC Leaderboard on Conversational Question Answering
[September 2020] Mutual Leaderboard on Dialogue Reasoning Challenge
[July 2019] SQuAD2.0 Leaderboard on Machine Reading Comprehension
[March 2019] RACE Leaderboard on Machine Reading Comprehension
[April 2019] SNLI Leaderboard on Language Inference [March 2019] GLUE Leaderboard on Language Understanding [August 2017] Chinese Machine Reading Comprehension (CCL-CMRC 2017)

Awards & Honors

Professional Service

  • Journal Reviewer: ACM Trans. on ALLIP (TALLIP), Neurocomputing, Multimedia Systems, Transactions on Machine Learning Research (TMLR), Neural Computing and Applications, Expert Systems With Applications.

  • Teaching Assistant for "Natural language understanding", Shanghai Jiao Tong University, Spring 2018, Spring 2019, and Spring 2021.

    Student Advising

    (* I am co-advising undergraduate students at SJTU with Prof. Hai Zhao.)
    • Siru Ouyang (Undergraduate Student, 2020 → PhD Student at UCLA, 2021)
    • Jialin Chen (Undergraduate Student, 2020 → PhD Student at Yale University, 2021)
    • Yuchen He (Undergraduate Student, 2021 → MS Student at SJTU, 2021)
    • Junlong Li (Undergraduate Student, 2020 → MS Student at SJTU, 2021)
    • Longxiang Liu (Undergraduate Student, 2020 → MS Student at ICT/CAS, 2021)
    • Yuwei Wu (Undergraduate Student, 2019 → MS Student at CMU, 2021)

    Miscellaneous

    • Polishing work takes much time and energy. Though the process is painful, jumping out of comfort zone can make one stronger, once by once. As things around become so impatient and utilitarian, how long have we not settled down to focus? Maybe sit down, drink a coffee, pick up passion, and broaden the scope.
      -- Reflection, 2021/09

    • Now, a few words on looking for things. When you go looking for something specific, your chances of finding it are very bad. Because of all the things in the world, you're only looking for one of them. When you go looking for anything at all, your chances of finding it are very good. Because of all the things in the world, you're sure to find some of them.
      -- The Zero Effect

    • Never give up the faith. Pass on the torch, and keep the light burning.
      念念不忘,必有回响。

      -- The Grandmaster《一代宗师》