TensorFlow implementation of the paper "CUTIE: Learning to Understand Documents with Convolutional Universal Text Information Extractor." Xiaohui Zhao Paper Link
CUTIE 是用于“票据文档” 2D 关键信息提取/命名实体识别/槽位填充 算法。 使用CUTIE前，需先使用OCR算法对“票据文档” 中的文字执行检测和识别，而后将格式化的文本输入入CUTIE网络，具体流程可参照论文。
CUTIE can be considered as one type of 2-Dimensional Key Information Extraction, 2-D NER (Named Entity Recognition) or a 2-Dimensional 2D Slot Filling algorithm. Before training / inference with CUTIE, prepare your structured texts in your scanned document images with any type of OCR algorithm. Refer to the CUTIE paper for details about the procedure.
Result evaluated on 4,484 receipt documents, including taxi receipts, meals entertainment receipts, and hotel receipts, with 9 different key information classes. (AP / softAP)
| Method | #Params | Taxi | Hotel |
|---|---|---|---|
| CloudScan | - | 82.0 / - | 60.0 / - |
| BERT | 110M | 88.1 / - | 71.7 / - |
| CUTIE | 14M | 94.0 / 97.3 | 74.6 / 87.0 |
pip install -r requirements.txt
- Generate your own dictionary with main_build_dict.py / main_data_tokenizer.py
- Train your model with main_train_json.py
CUTIE achieves best performance with rows/cols well configured. For more insights, refer to statistics in the file (others/TrainingStatistic.xlsx).
For information about the input example, refer to issue discussion.
- Apply any OCR tool that help you detecting and recognizing words in the scanned document image.
- Label image OCR results with key information class as the .json file in the invoice_data folder. (thanks to @4kssoft)