Question
I am currently trying to run a kfold cross validation on a decision tree with a custom classification loss function, as described here. However, I don't understand how the C and S matrices, which are passed to the loss function, are helpful. 1. Under the link it says "C is an n-by-K logical matrix with rows indicating which class the corresponding observation belongs." So this is not predicted and hence a repetition of the input data? 2. The S matrix. "S is an n-by-K numeric matrix of classification scores". Why can I not simply use the predicted classifications instead of the scores? To be more specific: I create a classification decision tree. Next, I use crossval to get a partitionedModel. Then, I calculate the validation accuracy by using kfoldLoss. Now, instead of using the in built 'classiferror' function, I would like to use my own classification loss function, e.g. matthews correlation coefficient. % create set of cross-validated classification model(s) from a classification model partitionedModel = crossval(trainedClassifier.ClassificationTree, 'KFold', 10); % Loss, by default the fraction of misclassified data, is a scalar and averaged over all folds validationAccuracy = 1 - kfoldLoss(partitionedModel, 'LossFun', 'classiferror'); Any help is greatly appreciated.
Expert Answer
John Michell
PhD Expert
Answered Aug 20, 2026
In case anybody else is looking for a solution. I used the crossval function to wrap the training of the decision tree. This way the implementation of other loss functions is straightforward.
function [trainedClassifier, qualityMeasures] = trainDTwCrossVal(data, predictorNames, MaxNumSplits) % cross validation numberOfFolds=5; cp = cvpartition(data.typeBehavior,'k',numberOfFolds); % creates a random partition for a stratified k-fold cross-validation vals=crossval(@trainDT2, data, 'partition', cp); % Loss estimate using cross validation function testval = trainDT2(trainingData, testingData) % nested function to train one DT with trainingData and test with testingData
Testval are quality measures of the prediction, derived from the confusion matrix, calculated inside the nested function to train the decision tree.
% C=[TP FP
% FN TN]
TP=C(1,1); FP=C(1,2); FN=C(2,1); TN=C(2,2);
% Matthews correlation coefficient, worst value = -1, best value = 1
if ( (TP+FP)*(TP+FN)*(TN+FP)*(TN+FN) ) == 0
MCC = 0; % set MCC to zero, if the denominator is zero
else
MCC = (TP*TN - FP*FN) / ...
sqrt( (TP+FP)*(TP+FN)*(TN+FP)*(TN+FN) );
end
accuracy=(TP+TN)/(TP+TN+FP+FN); % accuracy, worst value = 0, best value = 1
F1score=2*TP/(2*TP+FP+FN); % F1 score, worst value = 0, best value = 1
testval=[accuracy F1score MCC];
100% Run Guarantee
3-Hour Fast-Track Delivery
Need a Custom Version or Complete Simulation for This Problem?
Our 500+ PhD engineers build, debug, and optimize working MATLAB scripts and Simulink (.slx) models tailored to your exact assignment rubrics with zero plagiarism.
Tested on MATLAB R2024b / R2026a
Turnitin 0% Plagiarism Report
Free 7-Day Revisions Guarantee
Have a different question? Ask here
Related classification Questions & Solutions
Browse All →
Explore similar technical troubleshooting questions and verified MATLAB solutions:
Ready-to-Run MATLAB & Simulink Projects
Browse All Projects →