Beyond algorithm selection: meta-learning for data and algorithm analysis and understanding

Abstract

The area of Meta-learning (MtL) leverages onknowledge from problems for which successful Machine Learning (ML) solutions are known to support automated algorithm selection for new problems. But far more meta-knowledge can be extracted by relating data properties to algorithmic performance, a topic which remains under-explored compared to the usage of MtL for automated algorithm selection. For instance, one may reveal the competences and limitations of different ML algorithms and highlight data quality issues that are worth investigating. Building on the previous experience of the researcher during her Young Research Project phase 1 which involved the study, proposal and usage of data complexity measures for characterizing the hardness level of classification and regression problems, this project will go one step further and employ such measures for supporting algorithm and data understanding in a MtL perspective. By deepening such understanding, we expect to contribute on improving the comprehensibility and reliability in the usage of ML models. We also expect to generate contributions in three areas which can directly benefit from data and algorithm understanding: data pre-processing, ensemble learning and transfer learning. The idea is to guide the solution of the previous tasks using meta-knowledge extracted about the dichotomous relationship between data properties and algorithmic performance. (AU)