Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Linking Entities Across Images and Text
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.
2015 (English)In: Proceedings of the 19th Conference on Computational Language Learning, Association for Computational Linguistics, 2015, p. 185-193Conference paper, Published paper (Refereed)
Abstract [en]

This paper describes a set of methods to link entities across images and text. Asa corpus, we used a data set of images, where each image is commented by a short caption and where the regions in the images are manually segmented and labeled with a category. We extracted the entity mentions from the captions and we computed a semantic similarity between the mentions and the region labels. We also measured the statistical associations between these mentions and the labels and we combined them with the semantic similarity to produce mappings in the form of pairs consisting of a region label and a caption entity. In a second step, we used the syntactic relationships between the mentions and the spatial relationships between the regions to rerank the lists of candidate mappings. To evaluate our methods, we annotated a test set of 200 images, where we manually linked the image regions to their corresponding mentions in the captions. Eventually, we could match objects in pictures to their correct mentions for nearly 89 percent of the segments, when such a matching exists.

Place, publisher, year, edition, pages
Association for Computational Linguistics, 2015. p. 185-193
National Category
Information Systems
Research subject
Computer and Systems Sciences
Identifiers
URN: urn:nbn:se:su:diva-144927ISBN: 978-1-941643-77-8 (print)OAI: oai:DiVA.org:su-144927DiVA, id: diva2:1117585
Conference
CoNLL 2015, Beijing, China, July 30-31, 2015
Available from: 2017-06-29 Created: 2017-06-29 Last updated: 2018-10-08Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Free full text

Search in DiVA

By author/editor
Weegar, Rebecka
By organisation
Department of Computer and Systems Sciences
Information Systems

Search outside of DiVA

GoogleGoogle Scholar

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 3 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf