John Williams answered .
2025-07-13 14:05:15
This is a bug in the search space for hyperparameter optimization in fitcensemble, which will cause it to evaluate some unnecessary points. During the optimization, whenever 'Method' is LogitBoost or GentleBoost, 'SplitCriterion' is always internally set to 'mse'. But the search space is defined in a way that doesn't acknowledge that, so it unnecessarily passes SplitCrierion values of 'gdi' and 'deviance' during the optimization.
So all points that you see with LogitBoost/gdi and LogitBoost/deviance are really just LogitBoost/mse.
Oddly, you are not allowed to explicitly specify 'SplitCriterion','mse' with fitcensemble. Instead, when LogitBoost or GentleBoost are used, you need to omit the SplitCriterion argument entirely.
The optimization results are still valid. You just need to adjust what arguments you pass to fitcensemble at the end. The simple fix in your case is to delete line 8 of your original code sample, which specifies the SplitCriterion. Here's the result:
load carsmall
X = table(Acceleration,Cylinders,Displacement,Horsepower,Mfg,Model_Year,Weight,MPG);
X.Cylinders(X.Cylinders < 8) = 0; % Create two classes in the Cylinders variable
t = templateTree( 'MaxNumSplits', 30,...
'MinLeafSize', 10);
classificationEnsemble = fitcensemble(X,'Cylinders',...
'Method', 'LogitBoost', ...
'NumLearningCycles',12,...
'Learners',t,...
'KFold',7,...
'LearnRate',0.1);
Not satisfied with the answer ?? ASK NOW