Exploring multiobjective training in multiclass classification

Multinomial logistic loss and L-2 regularization are often conflicting objectives as more robust regularization leads to restrained multinomial parameters. For many practical problems, leveraging the best of both worlds would be invaluable for better decision-making processes. This research proposes a novel framework to obtain representative and diverse L-2-regularized multinomial models, based on valuable tradeoffs between prediction error and model complexity. The framework relies upon the Non-Inferior Set Estimation (NISE) method - a deterministic multiobjective solver. NISE automatically implements hyperparameter tuning in a multiobjective context. Given the diverse set of efficient learning models, model selection and aggregation of the multiple models in an ensemble framework promote high performance in multiclass classification. Additionally, NISE uses the weighted sum method as scalarization, thus being able to deal with the learning formulation directly. Its deterministic nature and the convexity of the learning problem confer scalability to the proposal. The experiments show competitive performance in various setups, taking a broad set of multiclass classification methods as contenders. (C) 2021 Elsevier B.V. All rights reserved. (AU)