Got it

Performance Metrics for Classification Problems

Latest reply: Dec 20, 2021 23:26:30 475 14 9 0 0

        Hi everyone I hope you are doing well, to we are going to talk about classification metrics used in Classification problems.

As you know a classification problem is a problem where the output (the so-called y-hat) is discrete: y^:ℝ→(1,2,3...k).

For example, we may need to set-up  a Machine Learning Algorithm that predicts if a woman is likely to have breast Cancer or Not. Indeed, our data set may have features like Family History, Gene Mutation, Increased breast density...

        Alert technical knowledge !!!:Machine is a Multidisciplinary , some machine learning  problems require  some other domain experience in Health Care, sports, House businesses…

Is it not interesting ?At this point two options are available: either you work with someone who is an expert in the domain you’re working (He will help you in deciding for example which feature is more important, the one that’s less and other stuffs that are out of the scope of my present text) or you try to acquire the knowledge by yourself.

Whatever you choose to do, it’s better in my opinion to work with a domain expert because you won’t lose much time and will gain sufficient insight for your problem.

        Let’s go back to performance metrics of classification problem I hope I didn’t lose anyone .

        Suppose you’re a data scientist. You’re required to build a Machine Learning model that predicts if a person giving his age and physical score (features) is likely to succeed the test (1) or Not(0) which are the labels.

The data looks like this:

dataset

        After performing Exploratory Data Analysis, you get to the conclusion that the Logistic Regression Algorithm is bet suited for this data set. You then build your model and train it.

Now it’s time to measure how good you model will perform on unseen data before launching it to the production.

The first thing to do is to plot a confusion matrix. It is a table that show the summary results of our classification problem.

confusionmatrix

Source : HCIA AI Version 3.0

 

TP: True Positive represent the number of people who succeed the test and that where correctly classified by the model.

TN: True Negative represent the number of people who didn’t succeed the test and that were correctly classified by the model.

FN: False Negative represent the number of people who succeed the test and that didn’t succeed according to the model. Number of incorrectly classified Negative cases

FP: False Positive  represent the number of people who didn’t succeed the test and that did succeed according to the model. Number of incorrectly classified Positive cases

P=TP+FN; N=TN+FP

Many metrics can be used:

metrics

Source : HCIA AI Version 3.0

            Accuracy or Recognition rate answer to the question how often does my model is correct ?

Note: that if we’re dealing with an imbalanced data set (like 900 samples for one category and 100 samples for the other Category), accuracy is not enough we need other metrics.

That’s where Precision and Recall come in.

        Precision, Positive Predicted Values answer the question Out of the total Positive predicted values, how many did the model predicted correctly?

        Recall, Sensitivity, True Positive rate answer the question Out of the total Positive actual values, how many did the model predicted correctly?

Note: Recall is much used when dealing with health care problems because we prefer a data incorrectly classified positive than negative. Such a patient will go for more advanced exams to get the confirmation. In other words, we want to reduce the FN number.

        Fβindicates how much we care about precision and Recall. It helps to have precision and recall in just one number. We don’t use the mean because if one these metrics is 0 the mean wouldn’t have shown that.

 

In conclusion, we presented  some of the most commonly used metrics for classification problems. In another text I will talk about ROC,AUC and also TPR,FPR.I hope you enjoyed reading this text.

I will appreciate that you put down below your observations and remarks so that I can do better next time.

Note : I tried to make the above mentioned informations as accurate as possible, but I’m exposed to mistakes, so kindly let me know if you disagree with me on certain points.

We are a Team (Together Everyone Achieves More).

 


  • x
  • convention:

wissal
MVE Created May 11, 2021 07:41:32

Interesting learning!
View more
  • x
  • convention:

csk99
csk99 Created May 12, 2021 04:59:01 (0) (0)
Thanks for your feedback  
user_4411447
user_4411447 Created Dec 20, 2021 23:26:19 (0) (0)
oh  
Unicef
MVE Created May 11, 2021 10:04:50

GOOD SHARE
View more
  • x
  • convention:

csk99
csk99 Created May 12, 2021 04:59:26 (0) (0)
Thanks for your feedback  
LilStylz237
Moderator Created May 11, 2021 14:13:23

Thanks, it is very interesting
View more
  • x
  • convention:

csk99
Created May 12, 2021 05:01:51

Thanks for your feedback @LilStylz237
View more
  • x
  • convention:

user_4001805
Created May 12, 2021 07:56:58

Thanks for sharing
View more
  • x
  • convention:

csk99
csk99 Created May 13, 2021 05:23:01 (0) (0)
Thanks for your feedback  
Irina
Admin Created May 17, 2021 13:07:23

Thank you for the article! I enjoyed reading it!
View more
  • x
  • convention:

csk99
csk99 Created May 18, 2021 10:40:07 (0) (0)
My pleasure  
Caroline_Herrera
Created Sep 23, 2021 13:00:54

Very good share
View more
  • x
  • convention:

user_4358465
Created Dec 18, 2021 16:27:06

Very helpful content, thank you for sharing!
View more
  • x
  • convention:

user_4411447
Created Dec 20, 2021 23:26:30

nice
View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.