Linear regression-hypothesis
\[h_{\theta}(x)=\theta_{0} + \theta_{1}x\J(\theta)=\frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2
\]
Linear Regression Gradient descent
\[\theta_j:= \theta_j-\alpha\frac{\partial}{\partial\theta_j}J(\theta)\Linear Regression:h_{\theta}(x)=\theta_0+\theta_1x\j=0=>\theta_j:=\theta_j-\alpha\frac{1}{m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})\j=1=>\theta_j:=\theta_j-\alpha\frac{1}{m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})x^{(i)}\Multiple Features:h_{\theta}(x)=\theta_0+\theta_1x_1+\theta_2x_2+...\theta_nx_n\引入x_0 = 1\=>h_{\theta}(x) = \theta^TX\X = [x_0,x_1,x_2,x_3,...,x_n]其中x_0=1\for(j=0,1,...,n)\\theta_j:=\theta_j -\alpha\frac{1}{m}\sum_{i=1}^{m}(h_{\theta}(x^{(i)})-y^{(i)})x^{(i)}_j
\]
Logistic Regression- hypothesis
\[h_{\theta}(x)=\frac{1}{1+e^{-\theta^Tx}}h_{\theta}(x)=\frac{1}{1+e^{-\theta^Tx}}
\]
Logistic Regression - Cost Function
\[\left\{
\begin{aligned}
J(\theta)&= \frac{1}{m}\sum_{i=1}^mCost(h_{\theta}(x^{(i)},y^{(i)})\Cost(h_{\theta}x^{(i)},y^{(i)})&=\frac{1}{2}(h_{\theta}(x^{(i)}-y^{(i)})^2\Cost(h_{\theta}(x^{(i)}),y^{(i)})&= \left\{
\begin{aligned}
-log(h_{\theta}(x)) & &y=1\-log(1-h_{{\theta}}(x)) & &y=0
\end{aligned}
\right.\end{aligned}
\right.
\]
\[Cost(h_{\theta}(x),y)=-ylog(h_{\theta}(x))-(1-y)log(1-h_{\theta}(x))
\]
Regularization （Logistic Regression Cost Function）
\[J(\theta)=\frac{1}{2m}\left[\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2+\lambda\sum_{j=1}^n\theta_j^2\right]
\]
\[J(\theta)=-\frac{1}{m}\left[\sum_{i=1}^my^{(i)}log(h_{\theta}(x^{(i)}))+(1-y^{(i)})log(1-h_{\theta}(x^{(i)}))\right]+\frac{\lambda}{2m}\sum_{j=1}^{m}\theta_j^2
\]
Regularized Logistic Regression
\[J(\theta)=-\left[\frac{1}{m}\sum_{i=1}^my^{(i)}logh_{\theta}(x^{(i)})+(1-y^{(i)})log(1-h_{\theta}(x^{(i)}))\right]
\]
Gradient descent
\[\theta_j:=\theta_j-\alpha \frac{\partial}{\partial \theta_j}J(\theta)
\]
\[\theta_j:=\theta_j-\alpha\frac{1}{m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})x_j^{(i)}
\]
After Regularizing Gradient descent
\[\theta_j:=\theta_j(1-\alpha\frac{\lambda}{m})-\alpha\frac{1}{m}\sum_{i=1}^{m}(h_{\theta}(x^{(i)})-y^{(i)})x_j^{(i)}
\]
Neural Network Cost Function
\[J(\Theta)=-\frac{1}{m}\left[\sum_{i=1}^m\sum _{k=1}^Ky_k^{(i)}log(h_{\Theta}(x^{(i)}))_k+(1-y^{(i)}_k)log(1-h_{\Theta}(x^{(i)}))_k\right]\+\frac{\lambda}{2m}\sum_{l=1}^{L-1}\sum_{i=1}^{s_l}\sum_{j=1}^{s_j+1}(\theta_{ji}^{(l)})^2
\]
机器学习常用公式
原文：https://www.cnblogs.com/eightos816/p/14200868.html
