GenKIE: Robust Generative Multimodal Document Key Information Extraction

Panfeng Cao¹, Ye Wang², Qiang Zhang³, Zaiqiao Meng⁴

¹University of Michigan
²National University of Defense Technology
³Zhejiang University
⁴University of Glasgow

Datasets and Pretrained Model

We give an example of processed SROIE dataset with the question prompt to demonstrate how to use GenIE. The dataset is available at this link. Our pretrained GenIE is available at this link.

Environments

pip install -r requirements.txt

Examples

bash run_scripts/run_example.sh

Acknowledgement

Our code is based on the work of OFA. We added additional multimodal feature embedding in the encoder as described in the paper.