Regularised feed forward neural networks for streamed data classification problems

Loading...
Thumbnail Image

Authors

Ellis, Mathys
Bosman, Anna Sergeevna
Engelbrecht, Andries P.

Journal Title

Journal ISSN

Volume Title

Publisher

Elsevier

Abstract

Streamed data classification problems (SDCPs) require classifiers to not just find the optimal decision boundaries that describe the relationships within a data stream, but also to adapt to changes in the decision boundaries in real-time. The requirement is due to concept drift, i.e., incorrect classifications caused by decision boundaries changing over time. Changes include disappearing, appearing or shifting decision boundaries. This article proposes an online learning approach for feed forward neural networks (FFNNs) that meets the requirements of SDCPs. The approach uses regularisation to dynamically optimise the architecture, and quantum particle swarm optimisation (QPSO) to dynamically adjust the weights. The learning approach is applied to a FFNN, which uses rectified linear activation functions, to form a novel SDCP classifier. The classifier is empirically investigated on several SDCPs. Both weight decay (WD) and weight elimination (WE) are investigated as regularisers. Empirical results show that using QPSO with no regularisation causes the classifier to completely saturate. However, using QPSO with regularisation makes the classifier efficient at dynamically adapting both its architecture and weights as decision boundaries change. Furthermore, the results favour WE over WD as a regulariser for QPSO.

Description

DATA AVAILABILITY : Data will be made available on request.
SUPPLEMENTARY MATERIAL : MMC S1. The supplementary material contains empirical results, performance graphs, illustrations, pseudo code and equations.

Keywords

Streamed data classification problem (SDCP), Quantum particle swarm optimisation (QPSO), Feed forward neural network (FFNN), Data streams, Classification problems, Regularisation, Concept drift, SDG-09: Industry, innovation and infrastructure

Sustainable Development Goals

SDG-09: Industry, innovation and infrastructure

Citation

Ellis, M., Bosman, A.S. & Engelbrecht, A.P. 2024, 'Regularised feed forward neural networks for streamed data classification problems', Engineering Applications of Artificial Intelligence, vol. 133, art. 108555, pp. 1-22, doi : 10.1016/j.engappai.2024.108555.