原ppt下载：pan.baidu.com/s/1nv54p9R，密码：3mty
需深入实践并理解的重要概念：
Deep Learning：
- SoftMax Fuction（输出层归一化函数？）
DNN（Deep Neural Networks）：
- MSE（Means Square Error，均方误差） / CE（Cross Entropy，交叉熵）
- Use to minimum total loss for softmax layer. CE is better.
- Mini-batch & batch_size（decides how many examples in a mini-batch）
- Vanishing Gradient Problem（梯度消失问题）
- ReLU（Rectified Linear Unit，线性纠正单元）
- As an activative function, used when the number of layers is quite large.
- Special cases of MaxOut
- Learnable activation function
- Adaptive learning rate
- Use a large rate first, then change to a small one
- Momentum（动量原理）
- Use the optimizer Adam（Advanced Adagrad Momentum）
- Overfitting Problem（过拟合问题）
- Use early stopping
- Weight Decay（训练时用p%的dropout，测试时对权值做(1-p%)的调整后再获得输出）
- Dropout（训练的过程舍弃神经元）
- Will change structure of networks while training. better than MaxOut
CNN（Convolutional Neural Networks）：
- Image recognization suits to use CNN because of 3 important properties：
1) Patterns are much smaller than the whole image
2) The same patterns appear in different regions
3) Subsampling pixels does not change the object
- filter & channel
- stride(step)
- zero-padding
- max-pooling
- flattern
- less parameters because of sharing weights