Large scale datasets for Image and Video Captioning in Italian
The application of Attention-based Deep Neural architectures to the automatic captioning of images and videos is enabling the development of increasingly performing systems. Unfortunately, while image processing is language independent, this does not hold for caption generation. Training such archit...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Accademia University Press
2019-12-01
|
| Series: | IJCoL |
| Online Access: | https://journals.openedition.org/ijcol/478 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850165680209395712 |
|---|---|
| author | Scaiella Antonio Danilo Croce Roberto Basili |
| author_facet | Scaiella Antonio Danilo Croce Roberto Basili |
| author_sort | Scaiella Antonio |
| collection | DOAJ |
| description | The application of Attention-based Deep Neural architectures to the automatic captioning of images and videos is enabling the development of increasingly performing systems. Unfortunately, while image processing is language independent, this does not hold for caption generation. Training such architectures requires the availability of (possibly large-scale) language specific resources, which are not available for many languages, such as Italian.In this paper, we present MSCOCO-it e MSR-VTT-it, two large-scale resources for image and video captioning. They have been derived by applying automatic machine translation to existing resources. Even though this approach is naive and exposed to the gathering of noisy information (depending on the quality of the automatic translator), we experimentally show that robust deep learning is enabled, rather tolerant with respect to such noise. In particular, we improve the state-of-the-art results with respect to image captioning in Italian. Moreover, in the paper we discuss the training of a system that, at the best of our knowledge, is the first video captioning system in Italian. |
| format | Article |
| id | doaj-art-fcaeeb6f334643b79730cbdd486105bb |
| institution | OA Journals |
| issn | 2499-4553 |
| language | English |
| publishDate | 2019-12-01 |
| publisher | Accademia University Press |
| record_format | Article |
| series | IJCoL |
| spelling | doaj-art-fcaeeb6f334643b79730cbdd486105bb2025-08-20T02:21:41ZengAccademia University PressIJCoL2499-45532019-12-0152496010.4000/ijcol.478Large scale datasets for Image and Video Captioning in ItalianScaiella AntonioDanilo CroceRoberto BasiliThe application of Attention-based Deep Neural architectures to the automatic captioning of images and videos is enabling the development of increasingly performing systems. Unfortunately, while image processing is language independent, this does not hold for caption generation. Training such architectures requires the availability of (possibly large-scale) language specific resources, which are not available for many languages, such as Italian.In this paper, we present MSCOCO-it e MSR-VTT-it, two large-scale resources for image and video captioning. They have been derived by applying automatic machine translation to existing resources. Even though this approach is naive and exposed to the gathering of noisy information (depending on the quality of the automatic translator), we experimentally show that robust deep learning is enabled, rather tolerant with respect to such noise. In particular, we improve the state-of-the-art results with respect to image captioning in Italian. Moreover, in the paper we discuss the training of a system that, at the best of our knowledge, is the first video captioning system in Italian.https://journals.openedition.org/ijcol/478 |
| spellingShingle | Scaiella Antonio Danilo Croce Roberto Basili Large scale datasets for Image and Video Captioning in Italian IJCoL |
| title | Large scale datasets for Image and Video Captioning in Italian |
| title_full | Large scale datasets for Image and Video Captioning in Italian |
| title_fullStr | Large scale datasets for Image and Video Captioning in Italian |
| title_full_unstemmed | Large scale datasets for Image and Video Captioning in Italian |
| title_short | Large scale datasets for Image and Video Captioning in Italian |
| title_sort | large scale datasets for image and video captioning in italian |
| url | https://journals.openedition.org/ijcol/478 |
| work_keys_str_mv | AT scaiellaantonio largescaledatasetsforimageandvideocaptioninginitalian AT danilocroce largescaledatasetsforimageandvideocaptioninginitalian AT robertobasili largescaledatasetsforimageandvideocaptioninginitalian |