# 逻辑回归

###### 代价函数（Cost Function）

function）。

hθ与Cost之间的函数关系如下图所示：

y=1：

• hθ -> 1，则Cost = 0；
• hθ -> 0，则Cost -> ∞。

y=0：

• hθ -> 0，则Cost = 0；
• hθ -> 1，则Cost -> ∞。
###### 过拟合问题（The Problem of Overfitting）

Bias）；第二个采用二次多项式的线性回归模型来拟合数据集，其效果恰好，因此我们将这种情况称为“Just
Right”；第三个采用四次多项式的线性回归模型来拟合数据集，其虽然对数据集拟合的非常好，但其曲线忽上忽下难以针对新数据进行预测，因此我们将这种情况称为过拟合（Overfitting）或高方差（
high variance）。

Question:
Consider the medical diagnosis problem of classifying tumors as
malignant or begin. If a hypothesis hθ(x) has overfit the
training set, it means that:
A. It makes accurate predictions for examples in the training set and
generalizes well to make accurate predictions on new, previously unseen
examples.
B. It does not make accurate predictions for examples in the training
set, but it does generalize well to make accurate predictions on new,
previously unseen example.
C. It makes accurate predictions for examples in the training set, but
it does not generalize well to make accurate predictions on new,
previously unseen examples.
D. It does not make accurate predictions for examples in the training
set and does not generalize well to make accurate predictions on new,
previously unseen examples.

1. 减少特征变量的个数：
• 人工选择特征变量
• 使用模型选择算法，自动选择特征变量
2. 正则化：保留所有特征变量，但减小参数θj的值
###### Cost Function

We cannot use the same cost function that we use for linear regression
because the Logistic Function will cause the output to be wavy, causing
many local optima. In other words, it will not be a convex function.

Instead, our cost function for logistic regression looks like:

When y = 1, we get the following plot for J vs hθ:

Similarly, when y = 0, we get the following plot for J vs hθ:

If our correct answer ‘y’ is 0, then the cost function will be 0 if our
hypothesis function also outputs 0. If our hypothesis approaches 1, then
the cost function will approach infinity.

If our correct answer ‘y’ is 1, then the cost function will be 0 if our
hypothesis function outputs 1. If our hypothesis approaches 0, then the
cost function will approach infinity.

Note that writing the cost function in this way guarantees that J is
convex for logistic regression.

###### The Problem of Overfitting

Consider the problem of predicting y from x ∈ R. The leftmost figure
below shows the result of fitting a y = θ01x to
a dataset. We see that the data doesn’t really lie on straight line, and
so the fit is not very good.

Underfitting, or high bias, is when the form of our hypothesis function
h maps poorly to the trend of the data. It is usually caused by a
function that is too simple or uses too few features. At the other
extreme, overfitting, or high variance, is caused by a hypothesis
function that fits the available data but does not generalize well to
predict new data. It is usually caused by a complicated function that
creates a lot of unnecessary curves and angles unrelated to the data.

This terminology is applied to both linear and logistic regression.
There are two main options to address the issue of overfitting:

1. Reduce the number of features:
• Manually select which features to keep.
• Use a model selection algorithm (studied later in the course).
2. Regularization
• Keep all the features, but reduce the magnitude of parameters
θj.
• Regularization works well when we have a lot of slightly useful
features.
###### 梯度下降算法

function）详解和logistic回归详解：梯度下降训练方法这两篇文章。谢谢！

###### 代价函数（Cost Function）

• θ2x22 +
θ3x33 +
θ4x44，则会出现对下图数据集过拟合的情况。

= θ0 + θ1x1 +
θ2x22

Parameter）。因此，我们将这种方法称为正则化。

= θ0

###### Simplified Cost Function and Gradient Descent

We can compress our cost function’s two conditional cases into one case:

Notice that when y is equal to 1, then the second term log⁡ will be zero
and will not affect the result. If y is equal to 0, then the first term
−ylog⁡ will be zero and will not affect the result.

We can fully write out our entire cost function as follows:

A vectorized implementation is:

Remember that the general form of gradient descent is:

We can work out the derivative part using calculus to get:

Notice that this algorithm is identical to the one we used in linear
regression. We still have to simultaneously update all values in theta.

A vectorized implementation is:

###### Cost Function

If we have overfitting from our hypothesis function, we can reduce the
weight that some of the terms in our function carry by increasing their
cost.

Say we wanted to make the following function more quadratic:

We’ll want to eliminate the influence of θ3x3 and
θ4x4 . Without actually getting rid of these
features or changing the form of our hypothesis, we can instead modify
our cost function:

We’ve added two extra terms at the end to inflate the cost of
θ3 and θ4. Now, in order for the cost function to
get close to zero, we will have to reduce the values of θ3
and θ4 to near zero. This will in turn greatly reduce the
values of θ3x3 and θ4x4 in
our hypothesis function. As a result, we see that the new hypothesis
(depicted by the pink curve) looks like a quadratic function but fits
the data better due to the extra small terms θ3x3
and θ4x4.

We could also regularize all of our theta parameters in a single
summation as:

The λ, or lambda, is the regularization parameter. It determines how
much the costs of our theta parameters are inflated.

Using the above cost function with the extra summation, we can smooth
the output of our hypothesis function to reduce overfitting. If lambda
is chosen to be too large, it may smooth out the function too much and
cause underfitting.