Regularised feed forward neural networks for streamed data classification problems
Loading...
Date
Authors
Ellis, Mathys
Bosman, Anna Sergeevna
Engelbrecht, Andries P.
Journal Title
Journal ISSN
Volume Title
Publisher
Elsevier
Abstract
Streamed data classification problems (SDCPs) require classifiers to not just find the optimal decision boundaries that describe the relationships within a data stream, but also to adapt to changes in the decision boundaries in real-time. The requirement is due to concept drift, i.e., incorrect classifications caused by decision boundaries changing over time. Changes include disappearing, appearing or shifting decision boundaries. This article proposes an online learning approach for feed forward neural networks (FFNNs) that meets the requirements of SDCPs. The approach uses regularisation to dynamically optimise the architecture, and quantum particle swarm optimisation (QPSO) to dynamically adjust the weights. The learning approach is applied to a FFNN, which uses rectified linear activation functions, to form a novel SDCP classifier. The classifier is empirically investigated on several SDCPs. Both weight decay (WD) and weight elimination (WE) are investigated as regularisers. Empirical results show that using QPSO with no regularisation causes the classifier to completely saturate. However, using QPSO with regularisation makes the classifier efficient at dynamically adapting both its architecture and weights as decision boundaries change. Furthermore, the results favour WE over WD as a regulariser for QPSO.
Description
DATA AVAILABILITY : Data will be made available on request.
SUPPLEMENTARY MATERIAL : MMC S1. The supplementary material contains empirical results, performance graphs, illustrations, pseudo code and equations.
SUPPLEMENTARY MATERIAL : MMC S1. The supplementary material contains empirical results, performance graphs, illustrations, pseudo code and equations.
Keywords
Streamed data classification problem (SDCP), Quantum particle swarm optimisation (QPSO), Feed forward neural network (FFNN), Data streams, Classification problems, Regularisation, Concept drift, SDG-09: Industry, innovation and infrastructure
Sustainable Development Goals
SDG-09: Industry, innovation and infrastructure
Citation
Ellis, M., Bosman, A.S. & Engelbrecht, A.P. 2024, 'Regularised feed forward neural networks for streamed data classification problems', Engineering Applications of Artificial Intelligence, vol. 133, art. 108555, pp. 1-22, doi : 10.1016/j.engappai.2024.108555.