"Intelligent and Adaptive Web Data Extraction System Using Convolutiona" by Sudhir Kumar Patnaik, C. Narendra Babu et al.

Article Title

Intelligent and Adaptive Web Data Extraction System Using Convolutional and Long Short-Term Memory Deep Learning Networks

Authors

Sudhir Kumar Patnaik, Department of Computer Science and Engineering, M. S. Ramaiah University of Applied Sciences, Bangalore 560054, India; Gibraltar India Solutions LLP, Bangalore 560103, India
C. Narendra Babu, Department of Computer Science and Engineering, M. S. Ramaiah University of Applied Sciences, Bangalore 560054, India
Mukul Bhave, Gibraltar India Solutions LLP, Bangalore 560103, India

Keywords

adaptive web scraping, deep learning, Long Short-Term Memory (LSTM), Web data extraction, You only look once (Yolo)

Abstract

Data are crucial to the growth of e-commerce in today’s world of highly demanding hyper-personalized consumer experiences, which are collected using advanced web scraping technologies. However, core data extraction engines fail because they cannot adapt to the dynamic changes in website content. This study investigates an intelligent and adaptive web data extraction system with convolutional and Long Short-Term Memory (LSTM) networks to enable automated web page detection using the You only look once (Yolo) algorithm and Tesseract LSTM to extract product details, which are detected as images from web pages. This state-of-the-art system does not need a core data extraction engine, and thus can adapt to dynamic changes in website layout. Experiments conducted on real-world retail cases demonstrate an image detection (precision) and character extraction accuracy (precision) of 97% and 99%, respectively. In addition, a mean average precision of 74%, with an input dataset of 45 objects or images, is obtained.

Publisher

Tsinghua University Press

Recommended Citation

Patnaik, Sudhir Kumar; Babu, C. Narendra; and Bhave, Mukul (2021) "Intelligent and Adaptive Web Data Extraction System Using Convolutional and Long Short-Term Memory Deep Learning Networks," Big Data Mining and Analytics: Vol. 4: Iss. 4, Article 6.
DOI: https://doi.org/10.26599/BDMA.2021.9020012
Available at: https://dc.tsinghuajournals.com/big-data-mining-and-analytics/vol4/iss4/6

Download

Request a Copy

Included in

Computer Engineering Commons, Computer Sciences Commons, Data Science Commons

COinS