Aaronne asked . 2022-06-14

What exactly the ROC curve can tell us or can be inferred?

I wrote some codes to run a linear discriminant analysis based classification:

%%Construct a LDA classifier with selected features and ground truth information
LDAClassifierObject                     = ClassificationDiscriminant.fit(featureSelcted, groundTruthGroup, 'DiscrimType', 'linear');
LDAClassifierResubError                 = resubLoss(LDAClassifierObject);

Thus, I can get

Resubstitution Error of LDA (Training Error): 1.7391e-01
Resubstitution Accuracy of LDA: 82.61%
Confusion Matrix of LDA:
    14     3
     1     5

Then I run a ROC analysis for the LDA classifier:

% Predict resubstitution response of LDA classifier
[LDALabel, LDAScore]                     = resubPredict(LDAClassifierObject);

% Fit probabilities for scores (the groundTruthGroup contains lables either 'Good' or 'Bad')
[FPR, TPR, Thr, AUC, OPTROCPT]       = perfcurve(groundTruthGroup(:,1), LDAScore(:,1), 'Good');

I have got:

OPTROCPT =

      0.1250    0.8667

Therefore, we can get:

Accuracy of LDA after ROC analysis: 86.91%
Confusion Matrix of LDA after ROC analysis:
    13     1
     2     7

cvLDAClassifier                    = crossval(LDAClassifierObject, 'leaveout', 'on');

%%Get the performance of the classifier
LDAClassifierPerformace                 = classperf(groundTruthGroup, resubPredict(LDAClassifierObject));

AI, Data Science, and Statistics , Statistics and Machine Learning Toolbox , Classification , Disc