1. Sigmoid Function: when z=0,g‘(z)=0.25
2. tanh Function: when x=0,tanh‘(x)=1
3. Relu
Activation Functions and Their Derivatives
原文：https://www.cnblogs.com/rhyswang/p/9188682.html
