Li "Harry" Zhang  张力

About Me

I am a third-year PhD student researcher focusing on Natural Language Processing, mentored by Prof. Chris Callison-Burch at the University of Pennsylvania. I was an undergraduate at the University of Michigan in 2018, previously mentored by Prof. Rada Mihalcea and Prof. Dragomir Radev. I'm also a drummer, musician, producer, and video creator.

My area of focus is on procedures, like recipes or instructions. I teach machines to understand and reason about them.

CV   bilibili              


Work Experience

Research InternIBM Research
2019, 2021

Teaching AssistantUM-Penn
2016 - 2020

Summer Analyst in TechnologyGoldman Sachs


University of Pennsylvaniaupenn logo
Ph.D.; GPA: 3.92/4.00; In progress

University of Michiganumich logo
B.S.E.; Class of 2018; GPA: 3.82/4.00

Shenzhen Middle SchoolSMS logo
High School Diploma; Class of 2015; GPA: 4.23/4.30


Reviewer of COLING 2022
Reviewer of ARR since October 2021
Reviewer of LREC 2022
Session chair of AACL-IJCNLP 2020
Co-organizer of CLUNCH in 2020, Penn's NLP seminar series
Reviewer of COLING 2020
Reviewer of Computer Speech and Language 2018


Reasoning about Procedures with Natural Language Processing

A procedure consists of a goal and some steps. To "record music" you need to "buy suitable equipments"; such is a goal-step relation. But before you "buy suitable equipments", you should "plan out your budget"; such is a step-step temporal relation. We use wikiHow to build datasets to train and test models that can reason about such relations. Our models show strong zero- and few-shot results on other tasks, especially on intent detection in dialogs. We also extend this task to multimodal settings.

[6] Reasoning about Goals, Steps, and Temporal Ordering with WikiHow; Li Zhang*Equal contribution, Qing Lyu*Equal contribution and Chris Callison-Burch; in EMNLP 2020; presented at Workshop on Enormous Language Models at ICLR 2021.Paper BibTeX Repo

[16] Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models; (441 authors in total) ... Li Zhang*Equal contribution, Qing Lyu*Equal contribution and Chris Callison-Burch; in arXiv.Paper

[7] Intent Detection with WikiHow; Li Zhang, Qing Lyu and Chris Callison-Burch; in AACL-IJCNLP 2020.Paper BibTeX Repo

[9] Visual Goal-Step Inference using wikiHow; Yue Yang, Artemis Panagopoulou, Qing Lyu, Li Zhang, Mark Yatskar and Chris Callison-Burch; In EMNLP 2021; presented at the 2nd Workshop on Advances in Language and Vision Research at NAACL 2021.Paper BibTeX

One of the holy-grails of procedural knowledge is for models to construct complete, high-quality procedures from scratch. We push the limit of our models and explore this difficult task in 18 languages, using techniques in both information retrieval and generation by language models. The task turns out extremely challenging, but not impossible.

[8] Goal-Oriented Script Construction; Qing Lyu*Equal contribution, Li Zhang*Equal contribution and Chris Callison-Burch; in INLG 2021.Paper BibTeX Repo

While all our previous work has treated procedures as a flat structure, procedural events are inherently hierarchical. To "become a great musician", you need to "practice", which in turn requires "making effective plans". We constructed an event hierarchy from wikiHow and showed that it efficiently aided users perform daily tasks and improved performance of downstream tasks such as video retrieval.

[10] Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data; Shuyan Zhou*Equal contribution, Li Zhang*Equal contribution, Yue Yang, Qing Lyu, Pengcheng Yin, Chris Callison-Burch and Graham Neubig; in ACL 2022.Paper BibTeX Demo

To see how our research on procedures can be applied, I co-led University of Pennsylvania's effort to participate in the Alexa Prize TaskBot challenge, where we build a dialog system to help users do household tasks.

[14] QuakerBot: A Household Dialog System Powered by Large Language Models; Artemis Panagopoulou, Manni Arora, Li Zhang, (9 more), Chris Callison-Burch, Mark Yatskar; To be published.Paper BibTeX

I wrote a tutorial on reasoning about procedures.

[15] Reasoning about Procedures with Natural Language Processing: A Tutorial; Li Zhang; in arXiv.Paper BibTeX


I play, record and produce music regularly. I enjoy genres ranging from progressive metal to pop to jazz. I'm passionate about rearranging and recreating soundtracks from video games and animes (ACG). I take pride in my 10k-subscriber channel on Bilibili. My music can be found on all major streaming platforms and YouTube.
To view in HD, hover on a video and click on 去bilibili观看.