Supervised level-wise pretraining for sequential data classification

Ienco Dino, Interdonato Roberto, Gaetano Raffaele. 2020. Supervised level-wise pretraining for sequential data classification. In : Neural information processing, Part V. Yang Haiqin (ed.), Pasupa Kitsuchart (ed.), Leung Andrew Chi-Sing (ed.), Kwok James T. (ed.), Chan Jonathan H. (ed.). Cham : Springer, 449-457. (Communications in Computer and Information Science, 1333) ISBN 978-3-030-63822-1 International Conference on Neural Information Processing (ICONIP 2020). 27, Thaïlande, 18 Novembre 2020/22 Novembre 2020.

Url - éditeur : https://link.springer.com/chapter/10.1007%2F978-3-030-63823-8_52

Résumé : Recurrent Neural Networks (RNNs) can be seriously impacted by the initial parameters assignment, which may result in poor generalization performances on new unseen data. With the objective to tackle this crucial issue, in the context of RNN based classification, we propose a new supervised layer-wise pretraining strategy to initialize network parameters. The proposed approach leverages a data-aware strategy that sets up a taxonomy of classification problems automatically derived by the model behavior. To the best of our knowledge, despite the great interest in RNN-based classification, this is the first data-aware strategy dealing with the initialization of such models. The proposed strategy has been tested on five benchmarks coming from three different domains, i.e., Text Classification, Speech Recognition and Remote Sensing. Results underline the benefit of our approach and point out that data-aware strategies positively support the initialization of Recurrent Neural Network based classification models.

Mots-clés libres : Deep Learning

Auteurs et affiliations

• Ienco Dino, IRSTEA (FRA)
• Interdonato Roberto, CIRAD-ES-UMR TETIS (FRA) ORCID: 0000-0002-0536-6277
• Gaetano Raffaele, CIRAD-ES-UMR TETIS (FRA)

Source : Cirad-Agritrop (https://agritrop.cirad.fr/597790/)

Métriques

[ Page générée et mise en cache le 2024-06-24 ]