考虑 \(x\), \(y\) 分别是 \(n\), \(m\) 维列向量, \(A\) 是 \(m\times n\) 矩阵, \(z\) 是标量.
想象分子不变, 分母转置.
Vector by vector 符合直观. Jacobian.
\[ \frac{\partial y}{\partial x} = \begin{pmatrix} \frac{\partial y_1}{\partial x_1} & \dots & \frac{\partial y_1}{\partial x_n}\\vdots & \ddots &\vdots\\frac{\partial y_m}{\partial x_1} & \dots & \frac{\partial y_m}{\partial x_n} \end{pmatrix} \]
Scalar by matrix 要做一次转置, 不舒服.
\[ \frac{\partial z}{\partial A}= \begin{pmatrix} \frac{\partial z}{\partial a_{11}} & \dots & \frac{\partial z}{\partial a_{m1}}\\vdots & \ddots &\vdots\\frac{\partial z}{\partial a_{1n}} & \dots & \frac{\partial z}{\partial a_{mn}} \end{pmatrix} \]
Chain rule 符合直观.
\[ \frac{\partial f\circ g}{\partial x} = \frac{\partial f}{\partial g}\frac{\partial g}{\partial x} \]
想象分母不变, 分子转置.
Vector by vector 不舒服. Hessian.
\[ \frac{\partial y}{\partial x} = \begin{pmatrix} \frac{\partial y_1}{\partial x_1} & \dots & \frac{\partial y_m}{\partial x_1}\\vdots & \ddots &\vdots\\frac{\partial y_1}{\partial x_n} & \dots & \frac{\partial y_m}{\partial x_n} \end{pmatrix} \]
Scalar by matrix 舒服.
\[ \frac{\partial z}{\partial A}= \begin{pmatrix} \frac{\partial z}{\partial a_{11}} & \dots & \frac{\partial z}{\partial a_{1n}}\\vdots & \ddots &\vdots\\frac{\partial z}{\partial a_{m1}} & \dots & \frac{\partial z}{\partial a_{mn}} \end{pmatrix} \]
Chain rule "倒过来" 了, 不舒服.
\[ \frac{\partial f\circ g}{\partial x} = \frac{\partial g}{\partial x}\frac{\partial f}{\partial g} \]
混用现象很常见. 比如 CS224n, 主体是采用 numerator layout, 但是 scalar by matrix 时是不转置的.
2 Main Layout Conventions of Matrix Calculus
原文：https://www.cnblogs.com/shiina922/p/11435371.html