Feature Selection

Feature selection, also known as variable or descriptor selection, is the process of finding a subset of features to use with a given task and learner. Finding the optimal set of features can improve predictive performance, reduce noise in data, and make models easier to interpret. There are multiple ways to perform feature selection, including with filters and wrappers. Filters calculate a value for each feature which can then be used to select them, for example selecting features where the feature value is greater than some value Y. Wrappers fit models on selected feature subsets and evaluate their performance to then pick the optimal subset – sequential forward selection is a well-known example. This chapter introduces mlr3fselect and mlr3filters for feature selection and filter methods. The design of these packages is similar to mlr3tuning so the interface should feel familiar to readers who read Chapter 4. Filters are first discussed including feature importance and embedded methods special cases. Then wrappers are discussed, including automating the process with nested resampling.