RO  EN
IMI/Publicaţii/CSJM/Ediţii/CSJM v.31, n.1 (91), 2023/

Image Description Generator using Residual Neural Network and Long Short-Term Memory

Authors: Mahesh Kumar Morampudi, Nagamani Gonthina, Nuthanakanti Bhaskar, V. Dinesh Reddy
Keywords: Image, description generator, ResNet, LSTM.

Abstract

Human beings can describe scenarios and objects in a picture through vision easily whereas performing the same task with a computer is a complicated one. Generating captions for the objects of an image helps everyone to understand the scenario of the image in a better way. Instinctively describing the content of an image requires the apprehension of computer vision as well as natural language processing. This task has gained huge popularity in the field of technology and there is a lot of research work being carried out. Recent works have been successful in identifying objects in the image but are facing many challenges in generating captions to the given image accurately by understanding the scenario. To address this challenge, we propose a model to generate the caption for an image. Residual Neural Network (ResNet) is used to extract the features from an image. These features are converted into a vector of size 2048. The caption generation for the image is obtained with Long Short-Term Memory (LSTM). The proposed model is experimented on the Flickr8K dataset and obtained an accuracy of 88.4\%. The experimental results indicate that our model produces appropriate captions compared to the state of art models.

Mahesh Kumar Morampudi
ORCID: https://orcid.org/0000-0002-6888-4637
Department of Computer Science and Engineering
SRM University AP
Amaravathi
Andhra Pradesh
India
E-mail:

Nagamani Gonthina
ORCID: https://orcid.org/0000-0001-9559-8030
Department of Computer Science and Engineering
Institute of Aeronautical Engineering
Hyderabad
Telangana
India
E-mail:

Nuthanakanti Bhaskar
ORCID: https://orcid.org/0000-0001-9852-1004
Department of Computer Science and Engineering
CMR Technical Campus
Hyderabad

V. Dinesh Reddy
ORCID: https://orcid.org/0000-0003-3945-6171
Department of Computer Science and Engineering
SRM University AP
Amaravathi
Andhra Pradesh
India
E-mail:

DOI

https://doi.org/10.56415/csjm.v31.01

Fulltext

Adobe PDF document0.58 Mb