Avatar

Li "Harry" Zhang  张力


About Me

I am a third-year PhD student researcher focusing on Natural Language Processing, mentored by Prof. Chris Callison-Burch at the University of Pennsylvania. I graduated from the University of Michigan in 2018, previously mentored by Prof. Rada Mihalcea and Prof. Dragomir Radev. I'm also a multi-instrumental musician, music producer, and video creator.

CV   bilibili              

University of Pennsylvania

Shenzhen, China

zharry@seas.upenn.edu


Music

I play, record and produce music regularly. I enjoy genres ranging from progressive metal to pop to jazz. I'm passionate about rearranging and recreating soundtracks from video games and animes (ACG). My music can be found on all major streaming platforms and my video channels on Bilibili and YouTube.


ACG music rearranged and remade as metal or rock


ACG music rearranged and remade as jazz, fusion or funk


Band cover of metal songs


Drum covers


Guitar and bass covers


Pool

I used to play pool competitively. I was in the university team and played in intercollegiate tournaments regularly.



Research Highlights

Learning Procedural Knowledge with wikihow
Nov 2019 - Present
Semantics Dialog Commonsense reasoning
Events are at the core of natural language, capturing "what is happening" instead of just "what is written down". Procedural events, specifically, are human activities that can be seen as a series of steps to accomplish a certain goal. For example, recipes, manuals to assemble furniture, how-to guides are all about procedrual events. These human-centered events involve idiosyncratic challenges and fascinating benefits.

Our first effort is to extract the procedural events. For example, to "record music" you need to "buy suitable equipments"; such is a goal-step relation. But before you "buy suitable equipments", you should "plan out your budget"; such is a step-step temporal relation. We use wikiHow to build datasets to train and test models that can reason about such relations. Our models show strong zero- and few-shot results on other tasks, especially on intent detection in dialogs.[6][7]

The inference of goal-step relations does not have to be confined to texts. Images and videos can also represent procedural events, with potentially richer information. We extend our task to a multimodal setting, showing the potential to generalize from the visuals in wikiHow to other domains.[9]

One of the holy-grails of procedural knowledge is for models to construct complete, high-quality procedures from scratch. We push the limit of our models and explore this difficult task in 18 languages, using techniques in both information retrieval and generation by language models. The task turns out extremely challenging, but not impossible.[8]

While all our previous work has treated procedures as a flat structure, procedural events are inherently hierarchical. To "become a great musician", you need to "practice", which in turn requires "making effective plans". We constructed an event hierarchy from wikiHow and showed that it efficiently aided users perform daily tasks and improved performance of downstream tasks such as video retrieval.

To show the power of our datasets and models, I co-lead University of Pennsylvania's effort to participate in the Alexa Prize TaskBot challenge, where we build a dialog system to help users do household tasks.

Publications

[9] Visual Goal-Step Inference using wikiHow
Yue Yang, Artemis Panagopoulou, Qing Lyu, Li Zhang, Mark Yatskar and Chris Callison-Burch

Paper BibTeX In EMNLP 2021; presented at the 2nd Workshop on Advances in Language and Vision Research at NAACL 2021.

[8] Goal-Oriented Script Construction
Qing Lyu*Equal contribution, Li Zhang*Equal contribution and Chris Callison-Burch

Paper BibTeX Repo In INLG 2021.

[7] Intent Detection with WikiHow
Li Zhang, Qing Lyu and Chris Callison-Burch

Paper BibTeX Repo  In AACL-IJCNLP 2020.

[6] Reasoning about Goals, Steps, and Temporal Ordering with WikiHow
Li Zhang*Equal contribution, Qing Lyu*Equal contribution and Chris Callison-Burch

Paper BibTeX Repo  In EMNLP 2020; presented at Workshop on Enormous Language Models at ICLR 2021.



Work Experience

Research Intern @ IBM ResearchIBM Research
2019, 2021

I did NLP research and software development on semantic role labeling, and previously, text simplification.


Teaching AssistantUM-Penn
2016 - 2020

At Penn, I instructed CIS 530: Computational Linguistics (Winter, Fall 2020). At Michigan, I instructed EECS 595: Natural Language Processing (Fall 2018) and EECS 280: Programming and Introductory Data Structures (Winter, Fall 2016).


Summer Analyst in Technology @ Goldman SachsGoldman Sachs
2017

I performed software engineering, data analytics and machine learning.


Education

University of Pennsylvaniaupenn logo
Philadephia, PA, USA
Ph.D. Computer and Information Science; In progress

GPA: 3.92/4.00


University of Michiganumich logo
Ann Arbor, MI, USA
B.S.E. Computer Science; Class of 2018

GPA: 3.82/4.00 summa cum laude


Shenzhen Middle SchoolSMS logo
Shenzhen, China
High School Diploma; Class of 2015

GPA: 4.23/4.30


Service

Reviewer of ARR November 2021
Session chair of AACL-IJCNLP 2020
Co-organizer of CLUNCH in 2020, Penn's NLP seminar series
Reviewer of COLING 2020
Reviewer of Computer Speech and Language 2018