Logistic Regression
Logistic Regression Output interpretation

Introduction

This is for you if you are looking for interpretation of p-value,coefficient estimates,odds ratio,logit score and how to find the final probability from logit score in logistic regression in R.
Let’s begin !!

Importing libraries,Reading Data & Looking at Data

Importing the required libraries.MASS is used for importing birthwt dataset

library(MASS)

####  Storing the data set named "birthwt" into DataFrame
DataFrame <- birthwt

####  To read about the dataset use following command by uncommenting
#### help("birthwt")

####  Check first 3 rows
head(DataFrame,3)




##    low age lwt race smoke ptl ht ui ftv  bwt
## 85   0  19 182    2     0   0  0  1   0 2523
## 86   0  33 155    3     0   0  0  0   3 2551
## 87   0  20 105    1     1   0  0  0   1 2557

Machine Learning and Data Science best online courses

Model fitting & Model Summary

Now we will fit the logistic regression model using only two continuous variables as independent variables i.e age and lwt.

####  Fitting the model
LogisticModel<- glm(low ~ age+lwt, data = DataFrame,family=binomial (link="logit"))

#### Let's check the summary of the model
summary(LogisticModel)
## 
## Call:
## glm(formula = low ~ age + lwt, family = binomial(link = "logit"), 
##     data = DataFrame)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -1.1352  -0.9088  -0.7480   1.3392   2.0595  
## 
## Coefficients:
##              Estimate Std. Error z value Pr(>|z|)  
## (Intercept)  1.748773   0.997097   1.754   0.0795 .
## age         -0.039788   0.032287  -1.232   0.2178  
## lwt         -0.012775   0.006211  -2.057   0.0397 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 234.67  on 188  degrees of freedom
## Residual deviance: 227.12  on 186  degrees of freedom
## AIC: 233.12
## 
## Number of Fisher Scoring iterations: 4

 

Basic Maths of Logistic Regression

We must know odds-ratio and logit score in order to understand logistic regression.

What is Odds Ratio

It represents the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring in the absence of that exposure.

Formula for Odds ratio

The mathematical formula for odds ratio is given by:

Odds=probability of success(p)/ probability of failure
=probability of (target variable=1)/probability of (target variable=0)
=p/(1-p)

Formula for logit 

The logit score can defined as follows:

logit(p) = log(p/(1-p)) = b0 + b1*x1 + … + bk*xk

Probability Calculation

Let’s follow the steps as below to find the probability of getting “low=1” (i.e probability of getting success).

NOTE: Do not confuse p-value with probability.They are different things

  • Intercept Coefficients interpretation (b0, b1 and b2)

1.  Intercept Coefficient(b0)=1.748773
2.  lwt coefficient(b1) =-0.012775
Interpretation: The increase in logit score per unit increase in weight(lwt)
is -0.012775
age coefficient(b2) =-0.039788




Interpretation: The increase in logit score per unit increase in age
is -0.039788

  • p-value interpretation

3.  p-value for lwt variable=0.0397
Interpretation: According to z-test,p-value is 0.0397 which is comparatively low
which implies its unlikely that there is “no relation” between lwt and target variable i.e low.Star next to p-value in the summary shows that lwt is significant variable in predicting low variable.




4. p-value for age=0.2178
Interpretation: According to z-test,p-value is 0.2178 which is comparatively high which                                implies its unlikely that there is “any relation” between age and target variable i.e low.

  • Logit score Calculation

5. Let’s consider a random person with age =25 and lwt=55.Now let’s find the logit score for this person
b0 + b1*x1 + b2*x2= 1.748773-0.039788*25-0.012775*55=0.05144(approx).

6. So logit score for this observation=0.05144

  • Odds ratio calculation

7. Now let’s find the probability that birthwt <2.5 kg(i.e low=1).See the help page on birthwt data set (type ?birthwt in the console)

8. Odds value=exp(0.05144) =1.052786

  • Probability Calculation

9. probability(p) = odds value / odds value + 1
     p=1.052786/2.052786=0.513(approx.)

p=0.513

Interpretation

0.513 or 51.3% is the probability of birth weight less than 2.5 kg when the  mother age =25 and mother’s weight(in pounds)=55

Machine Learning and Data Science best online courses