Logistic Regression
Logistic Regression Output interpretation

Introduction

This is for you if you are looking for interpretation of p-value,coefficient estimates,odds ratio,logit score and how to find the final probability from logit score in logistic regression in R.
Let’s begin !!

For Best Course on Data Science Developed by Data Scientist ,please follow the below link to avail discount

https://www.udemy.com/machine-learning-using-r/?couponCode=DISFOR123

Importing libraries,Reading Data & Looking at Data

Importing the required libraries.MASS is used for importing birthwt dataset

library(MASS)

####  Storing the data set named "birthwt" into DataFrame
DataFrame <- birthwt

####  To read about the dataset use following command by uncommenting
#### help("birthwt")

####  Check first 3 rows
head(DataFrame,3)




##    low age lwt race smoke ptl ht ui ftv  bwt
## 85   0  19 182    2     0   0  0  1   0 2523
## 86   0  33 155    3     0   0  0  0   3 2551
## 87   0  20 105    1     1   0  0  0   1 2557

Machine Learning and Data Science best online courses

Model fitting & Model Summary

Now we will fit the logistic regression model using only two continuous variables as independent variables i.e age and lwt.

####  Fitting the model
LogisticModel<- glm(low ~ age+lwt, data = DataFrame,family=binomial (link="logit"))

#### Let's check the summary of the model
summary(LogisticModel)
## 
## Call:
## glm(formula = low ~ age + lwt, family = binomial(link = "logit"), 
##     data = DataFrame)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.1352  -0.9088  -0.7480   1.3392   2.0595  
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)  
## (Intercept)  1.748773   0.997097   1.754   0.0795 .
## age         -0.039788   0.032287  -1.232   0.2178  
## lwt         -0.012775   0.006211  -2.057   0.0397 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 234.67  on 188  degrees of freedom
## Residual deviance: 227.12  on 186  degrees of freedom
## AIC: 233.12
## 
## Number of Fisher Scoring iterations: 4

 

Basic Maths of Logistic Regression

We must know odds-ratio and logit score in order to understand logistic regression.

What is Odds Ratio

It represents the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring in the absence of that exposure.

Formula for Odds ratio

The mathematical formula for odds ratio is given by:

Odds=probability of success(p)/ probability of failure
=probability of (target variable=1)/probability of (target variable=0)
=p/(1-p)

Formula for logit 

The logit score can defined as follows:

logit(p) = log(p/(1-p)) = b0 + b1*x1 + … + bk*xk

Probability Calculation

Let’s follow the steps as below to find the probability of getting “low=1” (i.e probability of getting success).

NOTE: Do not confuse p-value with probability.They are different things

  • Intercept Coefficients interpretation (b0, b1 and b2)

1.  Intercept Coefficient(b0)=1.748773
2.  lwt coefficient(b1) =-0.012775
Interpretation: The increase in logit score per unit increase in weight(lwt)
is -0.012775
age coefficient(b2) =-0.039788




Interpretation: The increase in logit score per unit increase in age
is -0.039788

  • p-value interpretation

3.  p-value for lwt variable=0.0397
Interpretation: According to z-test,p-value is 0.0397 which is comparatively low
which implies its unlikely that there is “no relation” between lwt and target variable i.e low.Star next to p-value in the summary shows that lwt is significant variable in predicting low variable.




4. p-value for age=0.2178
Interpretation: According to z-test,p-value is 0.2178 which is comparatively high which                                implies its unlikely that there is “any relation” between age and target variable i.e low.

  • Logit score Calculation

5. Let’s consider a random person with age =25 and lwt=55.Now let’s find the logit score for this person
b0 + b1*x1 + b2*x2= 1.748773-0.039788*25-0.012775*55=0.05144(approx).

6. So logit score for this observation=0.05144

  • Odds ratio calculation

7. Now let’s find the probability that birthwt <2.5 kg(i.e low=1).See the help page on birthwt data set (type ?birthwt in the console)

8. Odds value=exp(0.05144) =1.052786

  • Probability Calculation

9. probability(p) = odds value / odds value + 1
     p=1.052786/2.052786=0.513(approx.)

p=0.513

Interpretation

0.513 or 51.3% is the probability of birth weight less than 2.5 kg when the  mother age =25 and mother’s weight(in pounds)=55

Machine Learning and Data Science best online courses