A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers

Mahsa Shafaei; Christos Smailis; Ioannis A. Kakadiaris; Thamar Solorio

doi:10.26615/978-954-452-072-4_146

A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers

Mahsa Shafaei, Christos Smailis, Ioannis A. Kakadiaris, Thamar Solorio

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Scopus citations

Abstract

In this work, we explore different approaches to combine modalities for the problem of automated age-suitability rating of movie trailers. First, we introduce a new dataset containing videos of movie trailers in English downloaded from IMDB and YouTube, along with their corresponding age-suitability rating labels. Secondly, we propose a multi-modal deep learning pipeline addressing the movie trailer age suitability rating problem. This is the first attempt to combine video, audio, and speech information for this problem, and our experimental results show that multi-modal approaches significantly outperform the best mono and bimodal models in this task.

Original language	English (US)
Title of host publication	International Conference Recent Advances in Natural Language Processing, RANLP 2021
Subtitle of host publication	Deep Learning for Natural Language Processing Methods and Applications - Proceedings
Editors	Galia Angelova, Maria Kunilovskaya, Ruslan Mitkov, Ivelina Nikolova-Koleva
Publisher	Incoma Ltd
Pages	1297-1307
Number of pages	11
ISBN (Electronic)	9789544520724
DOIs	https://doi.org/10.26615/978-954-452-072-4_146
State	Published - 2021
Event	International Conference on Recent Advances in Natural Language Processing: Deep Learning for Natural Language Processing Methods and Applications, RANLP 2021 - Virtual, Online Duration: Sep 1 2021 → Sep 3 2021

Publication series

Name	International Conference Recent Advances in Natural Language Processing, RANLP
ISSN (Print)	1313-8502

Conference

Conference	International Conference on Recent Advances in Natural Language Processing: Deep Learning for Natural Language Processing Methods and Applications, RANLP 2021
City	Virtual, Online
Period	9/1/21 → 9/3/21

ASJC Scopus subject areas

Software
Computer Science Applications
Artificial Intelligence
Electrical and Electronic Engineering

Access to Document

10.26615/978-954-452-072-4_146

Cite this

Shafaei, M., Smailis, C., Kakadiaris, I. A., & Solorio, T. (2021). A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers. In G. Angelova, M. Kunilovskaya, R. Mitkov, & I. Nikolova-Koleva (Eds.), International Conference Recent Advances in Natural Language Processing, RANLP 2021: Deep Learning for Natural Language Processing Methods and Applications - Proceedings (pp. 1297-1307). (International Conference Recent Advances in Natural Language Processing, RANLP). Incoma Ltd. https://doi.org/10.26615/978-954-452-072-4_146

A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers. / Shafaei, Mahsa; Smailis, Christos; Kakadiaris, Ioannis A. et al.
International Conference Recent Advances in Natural Language Processing, RANLP 2021: Deep Learning for Natural Language Processing Methods and Applications - Proceedings. ed. / Galia Angelova; Maria Kunilovskaya; Ruslan Mitkov; Ivelina Nikolova-Koleva. Incoma Ltd, 2021. p. 1297-1307 (International Conference Recent Advances in Natural Language Processing, RANLP).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Shafaei, M, Smailis, C, Kakadiaris, IA & Solorio, T 2021, A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers. in G Angelova, M Kunilovskaya, R Mitkov & I Nikolova-Koleva (eds), International Conference Recent Advances in Natural Language Processing, RANLP 2021: Deep Learning for Natural Language Processing Methods and Applications - Proceedings. International Conference Recent Advances in Natural Language Processing, RANLP, Incoma Ltd, pp. 1297-1307, International Conference on Recent Advances in Natural Language Processing: Deep Learning for Natural Language Processing Methods and Applications, RANLP 2021, Virtual, Online, 9/1/21. https://doi.org/10.26615/978-954-452-072-4_146

Shafaei M, Smailis C, Kakadiaris IA, Solorio T. A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers. In Angelova G, Kunilovskaya M, Mitkov R, Nikolova-Koleva I, editors, International Conference Recent Advances in Natural Language Processing, RANLP 2021: Deep Learning for Natural Language Processing Methods and Applications - Proceedings. Incoma Ltd. 2021. p. 1297-1307. (International Conference Recent Advances in Natural Language Processing, RANLP). doi: 10.26615/978-954-452-072-4_146

Shafaei, Mahsa ; Smailis, Christos ; Kakadiaris, Ioannis A. et al. / A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers. International Conference Recent Advances in Natural Language Processing, RANLP 2021: Deep Learning for Natural Language Processing Methods and Applications - Proceedings. editor / Galia Angelova ; Maria Kunilovskaya ; Ruslan Mitkov ; Ivelina Nikolova-Koleva. Incoma Ltd, 2021. pp. 1297-1307 (International Conference Recent Advances in Natural Language Processing, RANLP).

@inproceedings{651411a64341401aa65d7d9ca7751d04,

title = "A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers",

abstract = "In this work, we explore different approaches to combine modalities for the problem of automated age-suitability rating of movie trailers. First, we introduce a new dataset containing videos of movie trailers in English downloaded from IMDB and YouTube, along with their corresponding age-suitability rating labels. Secondly, we propose a multi-modal deep learning pipeline addressing the movie trailer age suitability rating problem. This is the first attempt to combine video, audio, and speech information for this problem, and our experimental results show that multi-modal approaches significantly outperform the best mono and bimodal models in this task.",

author = "Mahsa Shafaei and Christos Smailis and Kakadiaris, {Ioannis A.} and Thamar Solorio",

note = "Funding Information: This work was partially funded by the National Science Foundation under award 2036368. Publisher Copyright: {\textcopyright} 2021 Incoma Ltd. All rights reserved.; International Conference on Recent Advances in Natural Language Processing: Deep Learning for Natural Language Processing Methods and Applications, RANLP 2021 ; Conference date: 01-09-2021 Through 03-09-2021",

year = "2021",

doi = "10.26615/978-954-452-072-4_146",

language = "English (US)",

series = "International Conference Recent Advances in Natural Language Processing, RANLP",

publisher = "Incoma Ltd",

pages = "1297--1307",

editor = "Galia Angelova and Maria Kunilovskaya and Ruslan Mitkov and Ivelina Nikolova-Koleva",

booktitle = "International Conference Recent Advances in Natural Language Processing, RANLP 2021",

}

TY - GEN

T1 - A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers

AU - Shafaei, Mahsa

AU - Smailis, Christos

AU - Kakadiaris, Ioannis A.

AU - Solorio, Thamar

PY - 2021

Y1 - 2021

N2 - In this work, we explore different approaches to combine modalities for the problem of automated age-suitability rating of movie trailers. First, we introduce a new dataset containing videos of movie trailers in English downloaded from IMDB and YouTube, along with their corresponding age-suitability rating labels. Secondly, we propose a multi-modal deep learning pipeline addressing the movie trailer age suitability rating problem. This is the first attempt to combine video, audio, and speech information for this problem, and our experimental results show that multi-modal approaches significantly outperform the best mono and bimodal models in this task.

AB - In this work, we explore different approaches to combine modalities for the problem of automated age-suitability rating of movie trailers. First, we introduce a new dataset containing videos of movie trailers in English downloaded from IMDB and YouTube, along with their corresponding age-suitability rating labels. Secondly, we propose a multi-modal deep learning pipeline addressing the movie trailer age suitability rating problem. This is the first attempt to combine video, audio, and speech information for this problem, and our experimental results show that multi-modal approaches significantly outperform the best mono and bimodal models in this task.

UR - http://www.scopus.com/inward/record.url?scp=85123581934&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85123581934&partnerID=8YFLogxK

U2 - 10.26615/978-954-452-072-4_146

DO - 10.26615/978-954-452-072-4_146

M3 - Conference contribution

AN - SCOPUS:85123581934

T3 - International Conference Recent Advances in Natural Language Processing, RANLP

SP - 1297

EP - 1307

BT - International Conference Recent Advances in Natural Language Processing, RANLP 2021

A2 - Angelova, Galia

A2 - Kunilovskaya, Maria

A2 - Mitkov, Ruslan

A2 - Nikolova-Koleva, Ivelina

PB - Incoma Ltd

T2 - International Conference on Recent Advances in Natural Language Processing: Deep Learning for Natural Language Processing Methods and Applications, RANLP 2021

Y2 - 1 September 2021 through 3 September 2021

ER -

A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this