Simple Linear Regression Model

This article considers the relationship between two variables in two ways: (1) by using regression analysis and (2) by computing the correlation coefficient. By using the regression model, we can evaluate the magnitude of change in one variable due to a certain change in another variable. For example, an economist can estimate the amount of change in food expenditure due to a certain change in the income of a household by using the regression model.

A sociologist may want to estimate the increase in the crime rate due to a particular increase in the unemployment rate. Besides answering these questions, a regression model also helps predict the value of one variable for a given value of another variable. For example, by using the regression line, we can predict the (approximate) food expenditure of a household with a given income. The correlation coefficient, on the other hand, simply tells us how strongly two variables are related.

It does not provide any information about the size of the change in one variable as a result of a certain change in the other variable. Let us return to the example of an economist investigating the relationship between food expenditure and income. What factors or variables does a household consider when deciding how much money it should spend on food every week or every month? Certainly, income of the household is one factor. However, many other variables also affect food expenditure.

For instance, the assets owned by the household, the size of the household, the preferences and tastes of household members, and any special dietary needs of household members are some of the variables that influence a household’s decision about food expenditure. These variables are called independent or explanatory variables because they all vary independently, and they explain the variation in food expenditures among different households. In other words, these variables explain why different households spend different amounts of money on food.

Food expenditure is called the dependent variable because it depends on the independent variables. Studying the effect of two or more independent variables on a dependent variable using regression analysis is called multiple regressions. However, if we choose only one (usually the most important) independent variable and study the effect of that single variable on a dependent variable, it is called a simple regression. Thus, a simple regression includes only two variables: one independent and one dependent. Note that whether it is a simple or a multiple regression analysis, it always includes one and only one dependent variable.

It is the number of independent variables that changes in simple and multiple regressions. The relationship between 2 variables in a regression analysis is expressed by a mathematical equation called a regression equation or model. A regression equation, when plotted, may assume one of many possible shapes, including a straight line. A regression equation that gives a straight-line relationship between two variables is called a linear regression model; otherwise, the model is called a nonlinear regression model.