MMD-Paper

Summary of papers on multi-modal fields (including document domain multi-modal), updated and maintained regularly.

文档布局分析（文档/自然场景）

Title	Conf	Code	Layer	Recom
Towards end-to-end unified scene text detection and layout analysis	CVPR	Code	2022	4
自然场景布局、文档布局分析、端到端、聚类、双路径Transformer
Post-ocr paragraph recognition by graph convolutional networks	WACV	Code	2022	4
图卷积、自然场景文档布局
Vision Grid Transformer for Document Layout Analysis	ICCV	Code	2023
多模态、新数据集
Efficient few-shot learning for pixel-precise handwritten document layout analysis	WACV	—	2023
古迹手写文档、语义分割、少样本学习
Multi-scale Cell-based Layout Representation for Document Understanding	WACV	Code	2023
多尺度单元、数据增强
M6Doc_ A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis	CVPR	Code	2023
新数据集：多类型、多结构、多语言、多注释

With OCR VDU 【依赖OCR的文档理解】

Title	Conf	Code	Layer	Recom
Layoutlm: Pre-training of text and layout for document image understanding	KDD	Code	2020	2
第一篇文档领域多模态论文
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding	ACL	Code	2021	2
多模态
DocFormer: End-to-End Transformer for Document Understanding	ICCV	非官方	2021	2
多模态、新颖的自注意力层
SelfDoc: Self-Supervised Document Representation Learning	CVPR	Code	2021	2
多模态、新颖的跨模态注意力机制
Layoutlmv3: Pre-training for document ai with unified text and image masking	ACM MM	Code	2022	4
多模态、预训练任务、
DocFormerv2: Local Features for Document Understanding	—	—	2023	4
多模态、新颖的预训练设置、预训练任务、
Unifying Vision, Text, and Layout for Universal Document Processing	CVPR	Code	2023	5
统一结构、统一任务、大模型

WithOut OCR VDU 【不依赖OCR的文档理解】

Title	Conf	Code	Layer	Recom
OCR-free Document Understanding Transformer	ECCV	Code	2022	4
第一篇无OCR的VDU、端到端、光学字符识别
End-to-end Document Recognition and Understanding with Dessurt	ECCV	Code	2022	3
更多微调结果、端到端、手写识别
Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentration	ICCV	—	2023	5
局部解码、端到端、手写识别、光学字符识别

image segmentation 【图像分割】

Title	Conf	Code	Layer	Recom
GroupViT: Semantic Segmentation Emerges from Text Supervision	CVPR	Code	2022	3
多模态、无像素级标注、第一个探索从单独的文本监督到语义分割任务的零样本迁移
Masked-attention Mask Transformer for Universal Image Segmentation	CVPR	Code	2022	4
通用分割（包括语义、实例、全景）、关注局部特征、多尺度
Generalized Decoding for Pixel, Image, and Language	CVPR	Code	2023	4
多模态、通用分割、指定分割、基于Mask2former、（以及后续一系列新的应用）