分类:Extracting Scientific Figures with Distantly Supervised Neural Networks

N. Siegel, Nicholas Lourie, R. Power, Waleed Ammar. Extracting Scientific Figures with Distantly Supervised Neural Networks. Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries

Abstract

Non-textual components such as charts, diagrams and tables provide key information in many scientific documents, but the lack of large labeled datasets has impeded the development of data-driven methods for scientific figure extraction. In this paper, we induce high-quality training labels for the task of figure extraction in a large number of scientific documents, with no human intervention. To accomplish this we leverage the auxiliary data provided in two large web collections of scientific documents (arXiv and PubMed) to locate figures and their associated captions in the rasterized PDF. We share the resulting dataset of over 5.5 million induced labels---4,000 times larger than the previous largest figure extraction dataset---with an average precision of 96.8%, to enable the development of modern data-driven methods for this task. We use this dataset to train a deep neural network for end-to-end figure detection, yielding a model that can be more easily extended to new domains compared to previous work. The model was successfully deployed in Semantic Scholar,\footnote\urlhttps://www.semanticscholar.org/ a large-scale academic search engine, and used to extract figures in 13 million scientific documents.\footnoteA demo of our system is available at \urlhttp://labs.semanticscholar.org/deepfigures/,and our dataset of induced labels can be downloaded at \urlhttps://s3-us-west-2.amazonaws.com/ai2-s2-research-public/deepfigures/jcdl-deepfigures-labels.tar.gz. Code to run our system locally can be found at \urlhttps://github.com/allenai/deepfigures-open.Collapse

总结和评论

这篇文章构建了一个科学论文中的插图的数据库，包含图和图的说明文字。在这个数据库的基础上，文章还训练了一个图片识别模型，并且把这个模型用于基于图片的检索。实际上，也可以尝试用这个模型来做一个图片对比，用于例如剽窃论文一稿多投等的识别。

实际上，类似地，在中，我们也需要第一整理一个这样的数据库，训练一个识别方程（从方程到名字）的模型，然后用于对论文内容更深刻的解读——例如，是否实际上基于同一个理论同一个方程。

概念地图