A model called **logistic regression** is introduced in Week 3 of *Machine Learning* course taught by Andrew Ng. This post tries to explain how to obtain **logistic regression** model from *linear regression* model which has been explained in Week 1 and Week 2 of the course. Therefore, I recommend all readers for studying Week 1 and Week 2 before reading this post.

Week 1 of the course shows that *linear regression* model has the capability to solve classification problem with prediction model (*hypothesis*) as follows:

with is called parameter model and is a *test example* whose we would like to predict.

The question arises as follows:

How can we derive

logistic regressionmodel fromlinear regressionmodel?

As we already know that $h_\theta(x)$ in **logistic regression** has a range of values between $0$ and $1$; on the other hand, $h_\theta(x)$ in *linear regression* has a range of values between $-\infty$ and $\infty$. Therefore, $h_\theta(x)$ which belongs to **logistic regression** needs to be converted into $-\infty < h_\theta(x) < \infty$. Moreover, after the conversion is done, the new $h_\theta(x)$ shall be substituted into Equation \eqref{eq:linear-regression}.

Let us start by converting $h_\theta(x)$ which belongs to **logistic regression** and then substitute the new $h_\theta(x)$ into Equation \eqref{eq:linear-regression}.

$\frac{h_\theta(x)}{1-h_\theta(x)}$ in Equation \eqref{eq:odds-ratio} is called **odds ratio** and $\log \left( \frac{h_\theta(x)}{1-h_\theta(x)} \right)$ in Equation \eqref{eq:logit} is named as **logit** function (Raschka & Mirjalili, 2017).

Now that the new $h_\theta(x)$ has the range of values between $-\infty < h_\theta(x) < \infty$, this new $ h_\theta(x) $ can be substituted into Equation \eqref{eq:linear-regression} as follows:

Accordingly, **logistic regression** model that we have derived is

**Logistic regression** model in Equation \eqref{eq:sigmoid} is popularly also called **sigmoid function**. Finally, we have successfully derived **logistic regression** model (Equation \eqref{eq:sigmoid}) from *linear regression* model (Equation \eqref{eq:linear-regression}).

### References

Raschka, S. and Mirjalili, V. (2017) *Python Machine Learning Second Edition*. Packt Publishing Ltd. Page 59-60.