Li "Harry" Zhang  张力

About Me

I am a third-year PhD student researcher focusing on Natural Language Processing, mentored by Prof. Chris Callison-Burch at the University of Pennsylvania. I was an undergraduate at the University of Michigan in 2018, previously mentored by Prof. Rada Mihalcea and Prof. Dragomir Radev. I'm also a drummer, musician, producer, and video creator.

CV   bilibili              

University of Pennsylvania

Shenzhen, China



I play, record and produce music regularly. I enjoy genres ranging from progressive metal to pop to jazz. I'm passionate about rearranging and recreating soundtracks from video games and animes (ACG). I take pride in my 10k-subscriber channel on Bilibili. My music can be found on all major streaming platforms and YouTube.

ACG music rearranged and remade as metal or rock

ACG music rearranged and remade as jazz, fusion or funk

Metal music covers

Drum covers

Guitar and bass covers

Research Highlights

Reasoning about Procedures
Nov 2019 - Present
Semantics Natural Language Understanding Reasoning
Events are at the core of natural language, capturing "what happens" instead of just "what is written down". Procedural events, specifically, are human activities that can be seen as a series of steps to accomplish a certain goal. For example, recipes, manuals to assemble furniture, how-to guides are all about procedrual events. These human-centered events involve idiosyncratic challenges and fascinating benefits.

Our first effort is to extract the procedural events. For example, to "record music" you need to "buy suitable equipments"; such is a goal-step relation. But before you "buy suitable equipments", you should "plan out your budget"; such is a step-step temporal relation. We use wikiHow to build datasets to train and test models that can reason about such relations. Our models show strong zero- and few-shot results on other tasks, especially on intent detection in dialogs.[6][7]

The inference of goal-step relations does not have to be confined to texts. Images and videos can also represent procedural events, with potentially richer information. We extend our task to a multimodal setting, showing the potential to generalize from the visuals in wikiHow to other domains.[9]

One of the holy-grails of procedural knowledge is for models to construct complete, high-quality procedures from scratch. We push the limit of our models and explore this difficult task in 18 languages, using techniques in both information retrieval and generation by language models. The task turns out extremely challenging, but not impossible.[8]

While all our previous work has treated procedures as a flat structure, procedural events are inherently hierarchical. To "become a great musician", you need to "practice", which in turn requires "making effective plans". We constructed an event hierarchy from wikiHow and showed that it efficiently aided users perform daily tasks and improved performance of downstream tasks such as video retrieval.[10]

To show the power of our datasets and models, I co-led University of Pennsylvania's effort to participate in the Alexa Prize TaskBot challenge, where we build a dialog system to help users do household tasks.

I wrote a tutorial on reasoning about procedures.


[12] Is "my favorite new movie" my favorite movie? Probing the Understanding of Recursive Noun Phrases
Qing Lyu, Hua Zheng, Daoxin Li, Li Zhang, Marianna Apidianaki and Chris Callison-Burch

Paper BibTeX Repo In NAACL 2022.

[11] Label Definitions Improve Semantic Role Labeling
Li Zhang, Ishan Jindal, Yunyao Li
BibTeX Repo In NAACL 2022.

[10] Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data
Shuyan Zhou*Equal contribution, Li Zhang*Equal contribution, Yue Yang, Qing Lyu, Pengcheng Yin, Chris Callison-Burch and Graham Neubig

Paper BibTeX Demo In ACL 2022.

[9] Visual Goal-Step Inference using wikiHow
Yue Yang, Artemis Panagopoulou, Qing Lyu, Li Zhang, Mark Yatskar and Chris Callison-Burch

Paper BibTeX In EMNLP 2021; presented at the 2nd Workshop on Advances in Language and Vision Research at NAACL 2021.

[8] Goal-Oriented Script Construction
Qing Lyu*Equal contribution, Li Zhang*Equal contribution and Chris Callison-Burch

Paper BibTeX Repo In INLG 2021.

[7] Intent Detection with WikiHow
Li Zhang, Qing Lyu and Chris Callison-Burch

Paper BibTeX Repo  In AACL-IJCNLP 2020.

[6] Reasoning about Goals, Steps, and Temporal Ordering with WikiHow
Li Zhang*Equal contribution, Qing Lyu*Equal contribution and Chris Callison-Burch

Paper BibTeX Repo  In EMNLP 2020; presented at Workshop on Enormous Language Models at ICLR 2021; a part of the Beyond the Imitation Game Benchmark.

Work Experience

Research Intern @ IBM ResearchIBM Research
2019, 2021

I did NLP research and software development on semantic role labeling, and previously, text simplification.

Teaching AssistantUM-Penn
2016 - 2020

At Penn, I instructed CIS 530: Computational Linguistics (Winter, Fall 2020). At Michigan, I instructed EECS 595: Natural Language Processing (Fall 2018) and EECS 280: Programming and Introductory Data Structures (Winter, Fall 2016).

Summer Analyst in Technology @ Goldman SachsGoldman Sachs

I performed software engineering, data analytics and machine learning.


University of Pennsylvaniaupenn logo
Ph.D. Computer and Information Science; GPA: 3.92/4.00; In progress

University of Michiganumich logo
B.S.E. Computer Science; Class of 2018; GPA: 3.82/4.00

Shenzhen Middle SchoolSMS logo
High School Diploma; Class of 2015; GPA: 4.23/4.30


Reviewer of ACL Rolling Review (ARR) since October 2021
Reviewer of LREC 2022
Session chair of AACL-IJCNLP 2020
Co-organizer of CLUNCH in 2020, Penn's NLP seminar series
Reviewer of COLING 2020
Reviewer of Computer Speech and Language 2018